$Id: NEWS,v 1.267 2004/10/09 20:43:25 flacoste Exp $

<-- be friendly for people pasting this text into email: 72 cols max -->

LogReport Lire NEWS - user visible changes (and some other changes too).

Refer to ChangeLog for detailed per-file info.

Copyright (C) 2000, 2001, 2002, 2003, 2004 Stichting LogReport Foundation
 LogReport@LogReport.org

version 2.0.1 - October 9 2004

- Lire now requires DBD::SQLite2. (The DBD::SQLite module switched to
  the SQLite 3.0 starting from version 1.00. DBD::SQLite2 must be used
  for the old database format.)

- Bug fixes

  * lr_cron will generate a report even if not all the DLF streams
    required by the report are available. (This prevented the default
    www report from working with CLF log file.)
  * lr_cron now set the period filter on the each report specs instead
    of the section. (The former behavior broke the default www report.)

version 2.0

- Bug fixes

  * Fixed double encoding problems when local codeset is UTF-8.
  * Updated User and Developer Manuals. Added categoriser example
    to doc/examples.
  * Do not try to include empty charts in HTML and PDF reports.
  * PDF reports which contained doubly nested groups were failing.

version 2.0rc2

- Bug fixes

  * Some www DlfAnalysers were left out of the package.
  * Non-ASCII characters are now displayed correctly in localized
    strings and in XML reports.
  * No more rotated pages in the PDF output.
  * postfix DLF converter wasn't working under lr_cron command.
  * ms_isa DLF converter wasn't working.

version 2.0rc1

- Changes to output formats

  * DocBook and all the extra tools are no longer required for any output
    format.
  * New HTML format which is a lot nicer than the previous one.
  * PDF output format is now generated through LaTeX. The only requirements
    are a recent TeX installation and GhostScript. Latex, dvi and ps
    were added as available output formats.
  * The dvi, latex, html, pdf and ps output formats contain the
    appropriate hyperlinked schemas reference.
  * The 'docbookx' and 'rtf' output formats were removed.
  * The 'html_onepage', 'xhtml' and 'xhtml_output' were removed.
    (The 'html' format can generate all of these through options.)

- Changes to the lire command

  * It is now possible to edit report configuration files through the
    lire interface. Report configurations are now stored in XML. The older
    report configuration files are obsolete and won't be supported in the
    next Lire releases. There is a lr_report_cfg2xml script to convert
    these files to XML templates. Old report configuration files used by
    configured jobs will be automatically converted when you use the
    lire tool.
  * It is now possible to have more than one superservice in a report.
    Each report section can use a different superservice.
  * NB: It is suggested to use a terminal with a height greater than 24
    lines for maximum usability.
  * If you are using a terminal with resizing capacity (like xterm), 
    the lire command will take advantage of the new size. (It is possible
    that you encounter a "screen too small" message while resizing, but
    trying different size can make the message disappear.)

- Changes to lr_log2report, lr_log2mail, lr_xml2mail and lr_xml2report

  * These commands were rewritten in Perl and now use the API directly.
    As a consequence most of the internal commands they used were removed:
    lr_check_prereq, lr_dlf2xml, lr_inflate, lr_log2xml, lr_store,
    lr_xml2ascii, lr_xml2chart, lr_xml2dbx, lr_xml2html, lr_xml2pdf,
    lr_xml2xls, lr_xml_merge and lr_xslt are no longer shipped as separate
    utilities.
  * The command line usage was improved. (Long options are now available and
    all commands can operate on other things then STDIN and STDOUT).
    Consult the man pages for details on the new features.
  * Old command line usage still works for now, with two exceptions:
     1. The second argument to lr_xml2report is the destination file. To merge
        additional reports use the --merge option.
     2. The first argument to lr_xml2mail is now the report to format. (It
        had to be specified using -x before.)
  * The commands now output legible messages on STDERR (they do not use
    the old lire logging format). As a consequence, they shouldn't be run
    under lr_run.

- Changes to charts
  * Three new types of charts are supported: scatterplot, stacked
    plots and distribution plot.
  * Most chart types now allow the plotting of more than one variable.
  * Charts can be configured through the new report configuration interface.
  * The ploticus tool is now required to generate charts. Support for GD::Chart
    was dropped.
  * ChartTypes are now Lire::Plugin, making it possible to add new chart types
    easily.

- Changes to XML report
  * Support for 1.0 DTD XML reports (as generated by Lire version < 1.2) was
    dropped.
  * The chartype attribute, image and file elements were removed from the
    LRML. Charts are now specified in a chart-configs element which contains
    the chart configuraiton in LRCML. The charttype attribute is still
    supported for older reports.

- Configuration API changes

  * It is now possible to set default values directly in the
    specification file. As a consequence, Lire doesn't install any files
    in the share/lire/config-defaults directory.
  * There is a report-config element that can be used to define a
    report configuration. The default report configuration for each
    superservice will be looked up under <superservice>_default_report in
    the configuration specification.
  * The new_from_config() method used to instantiate a custom object now takes
    the Lire::Config::Value as parameter and not its as_value()
    result.
  * All specification elements now have an 'obsolete' and 'required'
    attribute.
  * Compound objects now also support the is_valid() method which makes sure
    that all required contained parameters are valid.
  * String specifications now have an optional 'valid-re' attribute
    which should contain a regular expression that the is_valid() method
    will use for validation.

- New Lire::DlfAnalyser API

  * This API replaces the old undocumented DerivedFieldCreator and
    ExtendedFieldCreator. This API is similar to the Lire::DlfConverter one.
    Consult the Developer's manual for more information.
  * Analysers are now run after the log importation. The analysers to
    run can be configured by DlfStore through the lire command.

- New Lire::OutputFormat API

  * All output formats are now implemented as Lire::Plugin.

- API and other framework changes

  * Lire::DlfConverterManager was removed and replaced by Lire::PluginManager.
    As a result, the way a DlfConverter is initialised and registered changed:
    it is the responsability of the registration script to call
    register_plugin() to register its plugin. Furthermore, the plugin
    registration script should be located in the directories contained in
    the 'plugins_init_path' configuration variable.

    Registration scripts located in 'lr_converters_init_path' will still
    be loaded using the old mechanism as a backward compatibility measure.

- Bug fixes

  * Merging of first and last operator summary data was only working
    when the field used was the first in the schema.
  * Lots of other bugs were fixed by this major overhaul.  Comparing the
    BUGS file with the previous release will give you insight.

- Other minor changes

  * Lire::WWW::UserAgent was updated, your www reports will give more
    information on user agents.  Contributed by Egon Willighagen.

version 1.5 - April 12 2004

  Release 1.5 is the last interim release before the Lire 2.0 release. (See 
  http://www.logreport.org/lire/roadmap-2.0.php).

  It completes the Internationalisation and Store Configuration items on
  the 2.0 roadmap.

- Store Configuration

 The lr_config command which was used to configure Lire and schedule
 report generation was replaced by a new Curses-based command : lire. 

 Like the old lr_config command, it can be used to change Lire global
 settings as well as scheduling jobs that will be executed by
 crond(8). But unlike the old lr_config command, the way jobs are
 configured changed radically. Consult the README file for more information.

 Support for the old lr_config's jobs was dropped and that
 configuration cannot be migrated to the new paradigm.

- I18N

  * Lire now supports non-ascii log files. With perl 5.8.0 and later, 8bit
    characters will no longer be replaced with '?'.
  * The encoding of log files can be specified when they are in a different
    encoding than the system's default.
  * Any XML::Parser supported encoding can be used in XML files.
  * There is a lr_spec2pot command that can be used to extract
    translatable strings from XML specifications.
  * Many Lire commands were internationalized.

- Requirements changes:
  * Perl >= 5.6.1 is now required. To support non-ascii encoding perl >= 5.8.1
    is required.
  * The Curses::UI perl module is required (this module also required the 
    Curses and TermReadKey modules.)
  * The libintl-perl module.

- API and other framework changes

  * Three new configuration types are available: record, object and plugin.
    A record is a configuration type that contains many fields.
    An object is like a record but instead of being instantiated as a
    hash reference, it specifies a factory method which will be called
    in as_value(). A plugin is a type similar to a select, but which
    has additional properties based on the selected value.
  * Most XML specs have a new 'i18n_domain' attribute which can be
    used to specify the Gettext domain in which translations should be
    looked for.
  * DlfConverter's should specify their configuration variables using a
    record or objet specification. Users will be able to set the
    converter properties from the user interface.
  * Each stores has two DLF streams called "lire_import_stats" and
    "lire_import_log". The former contains stats about the execution of
    ImportJobs and the latter is used to store continuation lines, error
    messages and other such information. 

- Incompatible changes

  * lr_config was removed.
  * Many global configuration variables were moved to by-job parameters.
  * Support for DocBook's "userlevel" attribute was removed. The whole
    description will always be shown from now on.
  * The RTF output format was removed. 
  * Support for migration of old environment based configuration (pre-1.3) 
    was removed.
  * Files and categories were removed from lr_store.
  * Many other bug fixes.

version 1.4.1 - January 7 2004

  - Cisco syslog format understands the optinal TZ.
  - Requires DBD::SQLite 0.29 or later.

version 1.4 - November 15 2003

  Release 1.4 introduces a new report generation algorithm which
  improves performance a lot. DLF data is now stored in an SQLite
  database and the report is generated directly from the database. Tests
  show performance increases of ~18%-55% depending on the superservice
  used (superservices like www, which uses a lot of analyzers, will see
  less benefits than superservices like firewall which doesn't because
  the bulk of the time is spent in the analyzers).

  The new report generation algorithm has some impact on the requirements
  and introduces other incompatible changes:

- Requirement changes:
  
  * The DBI perl module is required.
  * The DBD-SQLite module is required. At this time, a patch
    (extras/DBD-SQLite-0.28.patch) needs to be applied. Versions 0.29
    and above of DBD::SQLite will not need this patch. A version with the 
    patch already applied is available from http://download.logreport.org/
  * The Digest::MD5 perl module is required (it is included with perl
    5.8.0).
  * There is no lire-full version available anymore. Expat and XML::Parser
    need to be installed separately before installing Lire.

- Changes to the reports:

  * Reports do not include 'empty groups' anymore. For example, in Lire
    1.3 and before, if you generated a 'Requests by hour' report and you
    had requests during hours 14:00 and 16:00 but not during 15:00,
    15:00 would appear with 0 in the report. In Lire 1.4, the 15:00 hour
    will not appear in the report at all. Merging such reports will
    reintroduce the empty groups though.
  * The ranges generated by rangegroup operator which uses a
    'size-scale' attribute != 1 are now different. For example, Lire 1.3
    would generate the following groups for a scale of 5:

    [0-1k>
    [1k-5k>
    [5k-25>

    whereas Lire 1.4 will generate

    [0-1k>
    [1k-6k>
    [6k-31k> 

    which was really the advertised behaviour. 
  * The column's name automatically generated for aggregator changed.
    For example, in Lire 1.3, the column of the timegroup operator
    took the name of the DLF schema's timestamp field. It's now called
    'timegroup:field'. Similar changes affect the column generated by
    the rangegroup, timeslot and records aggregators.

- Other incompatible changes:

  * Because of the changes related to reports' content, new reports
    can't be merged with old ones. For example, you can't merge
    XML reports generated by Lire 1.3 with reports generated by Lire 1.4.
  * The service_aliases.cf file is not used anymore. As a consequence,
    you will need to use the canonical names instead of the old ones.
    The aliases which were removed are 'bind8', 'bind9', 'cisco',
    'cups', 'lprng' and 'squid'. Use 'bind8_query', 'bind9_query',
    'cisco_ios', 'cups_pagelog', 'lprng_account' and 'squid_access'
    instead.
  * The experimental MySQL archiving support (lr_dlf2sql, lr_sql2dlf
    and lr_sql2report) was removed.
  * The lr_viconf configuration frontend was removed.
  * Some configuration variables are obsolete: lr_sql_archive, 
    lr_sql_max_size, lr_old_service_aliases_file, lr_dbi_uri,
    lr_dbi_user, lr_paswd_file.

- This release has undergone extensive testing on lots of platforms.
  We've been able to add FreeBSD, OpenBSD, NetBSD and Mac OS/X to our
  group of test platforms.

- Many bug fixes.

version 1.3 - May 17th 2003
- New improved lr_config(1) command which makes it easier to configure
  periodical reports.

- Documentation was improved. The internal APIs which should be 
  used by extension developers are documented in man pages. Tutorial on how 
  to to write new DLF converters and DLF schemas were added to the
  Developer's manual. There is also sample code in doc/examples.

- Several bug fixes:
    * Fixed merging problem with report using the timegroup operator.
    * Fixed merging problem with report containg missing subreport.
    * Fixed merging problem with report that were generated with 0 records.
    * Encode all outgoing mail using Base64 or quote-printable (no more
      problems related to long lines).
    * ISO week numbers are supported everywhere Lire is.

- Incompatible change:
    * Configuration is now held in XML files. Some of the old-shell
      based configuration can be migrated using the lr_env2conf(1)
      command. The periodical report configuration isn't migrated
      though. You will need to enter it anew using the improved lr_config(1).
    * Some internal commands were removed (lr_archive_log,
      lr_check_errlines, lr_db_fetch, lr_db_store, lr_db_flush, 
      lr_run_tests). If you used those, you will need to modify the
      relevant scripts.
    * The old archiving behavior is removed. Only the responder
      now saves the log files and reports. If you used LR_ARCHIVE
      while not running a responder, you will need to take care of
      archiving your reports and log files yourself for now.

- Many enhancements to Lire's core:
    * New configuration API making it easy to make extensions (DLF
      converter, analyzers, etc.) configurable.
    * New API for DLF converter with support for multiple DLF schemas
      and log continuation.
    * Extensive suite of unit tests and regression tests were added
      to Lire, making it easier to modify.

- New DLF converters contribued by Lire's users:
    * tinydns - DNS DLF converter for DJB TinyDNS. Contributed by
      Christopher Boumenot
    * s1ms - Email DLF converter for SunONE Messaging Server. Contributed 
      by Jean-Yves Monnier.
    * snort - Firewall DLF converter for Snort IDS. Contributed by 
      Torsten Fellhauer.
    * fw1_lea - Firewall DLF converter for CheckPoint Firewall-1 5.0. 
      This DLF converter isn't activated by default. See the notes
      in the fw1_lea2dlf(1) man page for activation instructions. 
      Contributed by Torsten Fellhauer.

- Other minor stuff:
    * added lr_vendor_cron script (mainly interesting for people making
      Lire packages for specific Unix and GNU/Linux distributions).

version 1.2.1 - 03 December 2002

    * Fixed broken variable substitution in defaults which prevents
      loading of "local" files created by lr_config in @sysconfdir@/lire.

version 1.2 - 29 November 2002

- Many enhancements to the report engine:
    * New ratio attribute to the sum and count operators to express
      their value as a percentage of the table's or group's total.
      All reports where upgraded to support this new XML Report
      format; they now include labels and ratio columns (whenever
      appropriate)
    * New weight attribute to the sum and avg operator to compute
      weighted average and sums.
    * New first and last operators that can be used to the first or
      last field's value in a group.  (See the top-user-leftover-bytes
      msgstore report for an example of lire:last.)
    * Report specifications can set columns label using the label
      attribute.
    * Reports now include summary information (group's and table's
      total for example).

- Major upgrade of the output layer, output formats
    * All typesetters can now format reports containing complex table
      containing multiple columns and group by entry.
    * All typesetters support column labels and group and table
      summary information.
    * Many improvements to the DTD and to the ReportParser API to make
      it easier to work with Lire XML Reports.
    * One can now typeset PDF reports using PassiveTeX instead of
      jade/DSSSL.
    * Support for output in LogML (
      http://www.cs.rpi.edu/projects/pb/WWWPal/logml.html ) format is
      removed. (LogML is an XML based language to compress log files
      in smaller files, enabling data mining techniques.) The Lire
      code for LogML has been badly maintained for some time; interest
      seems low. However, if you'd like to use this functionality,
      we'll gladly accept code contributions. Contact the LogReport
      team at the development@logreport.org list for hints on bringing
      LogML output back into Lire.
    * Support for generating graphs using ploticus added. See the Lire
      User Manual for details.

- New services and superservices
    * New syslog format supported: WTsyslog.
    * Support for DBMAIL ( http://www.dbmail.org/ ) POP3 log files,
      thanks to a code contribution by Edwin Groothuis.
    * New superservice: spamfilter, for email filters like those used
      for blocking UCE. Support for the spamassassin spam filter,
      thanks to a code contribution by Edwin Groothuis.
    * The nmsstore service will now parse log entries coming from the webmail
      interface, thanks to a fix provided by Jean-Yves Monnier.

- New reports
    * WWW report Client hosts by Size added
    * msgstore top-user-leftover reports added: diskspace used after
         session (thanks Edwin Groothuis)

- Documentation
  Lots of documentation updates and improvements: We're shipping more
  than 1MB of documentation in man page format now. Apart from this,
  we still offer documentation in our Lire User Manual, FAQ and
  Developers Manual, of course.
    * Some contributions to manpages by Edwin Groothuis.
    * A lot of Lire modules have been properly documented in manpages
      now, which is especially useful for Lire hackers.
    * Since we have been asked for this over and over again, a lire(7)
      manpage is added.
    * We now ship a ChangeLog with Lire.

- Small bugs fixed
    * Bug fixes in the Kiwi syslog parser.
    * Bug fix in the mysql parser (thanks Fatih Gey).
    * Bug fixes in the dbmail service and msgstore superservice (thanks
        Edwin Groothuis)
    * Some other small bugs fixed (see ChangeLog for details)
    * Fixed building the lire-full distribution with Solaris' make.

- Lire developers
    * Hook for typesetting documentation using pdfxmltex and xsltproc
      instead of pdfjadetex and (open)jade added.
    * New superservice `test' added, to debug changes to the Lire core.
    * All files in $sysconfdir/lire/ ending in defaults and defaults.local
      will be loaded. This makes it easier to add configuration hooks 
      for external modules.
    * New lr_env(1) command which can be used to find the location where
      various Lire extensions should be installed.

- Other business
    * Small helper script for setting up an online responder added:
      lr_setup_responder.
    * If you have written your own report definition files, or if you're
      using Lire's report merging capabilities, you might get hit by some
      backwards incompatible changes.  Refer to the `upgrading from Lire
      1.2 and earlier' section in the INSTALL file for details.


version 1.1 - 19 August 2002

- New superservices
    * dialup, including a isdnlog2dlf convertor for log files of
      isdn4linux.de's isdn4k-utils. The superservice has a few report types on
      total costs, connection time, and connections by period.
    * dnszone, for DNS log files about zone transfers.  Lire now supports Bind
      8's named log files.  Thanks to a code contribution from Edwin
      Groothuis.
    * msgstore, including support for Netscape Message Store (nmsstore) and
      Netscape Messaging Multiplexor (nmsmms). Thanks to a code contribution
      from Arnaud Taddei and Arnaud Gaillard.
    * syslog. It is now possible to have some general overview of your syslog
      log file.

- New services
    * Lire now supports logs from the WatchGuard Firebox firewall solution,
      thanks to a code contribution from Joost Bekkers.
    * Overhauled Syslog parser. Lire now supports log files coming from various
      syslog implementations. In addition to the BSD, Solaris 8 and Netscape
      Messaging Server logging daemon already supported, we now support 5 other
      formats: Kiwi Syslog Daemon (ISO, US and DD/MM date formats), Sendmail
      Switch Log daemon and an unknown daemon that includes the encoded
      priority and the year in its format.
    * The bind9 service supports the Bind 9.3 log format too.
    * The squid service now supports the Netcache network appliance 'squid 
      logging format'. Thanks to Arnaud Gaillard for sending us a sample log 
      file.
    * The cisco service now supports all the syslog log formats the syslog
      parser supports.
    * Both Cisco and PIX services now support a variety of timestamp settings.

- Other improvements
    * Much improved Netscape Messaging support. Thanks to Arnaud Taddei and
      Arnaud Gaillard!
    * Bugfix to xferlog parser which now support the "completion_status" field.
    * Generate catalog.xml at runtime so that it correctly reflects the
      configured location of the DocBook DTD.
    * Include an XML catalog for our DTDs. You shouldn't see the
      "Attempt to load network entity..." message from xsltproc anymore.
    * The image format of the charts is now configurable through the
      LR_IMAGE_FORMAT variable. We don't rely on ImageMagick's convert(1)
      anymore. All formats supported by the GD library are available (which
      means jpeg, png and/or gif depending on your version).
    * Superservice aren't statically defined anymore. The list of available
      superservices is now determined at runtime.
    * New email reports:
      Largest Email Exchange Per Relay Pair, Status Summary, Top Status
      Per Domain and Domains By Period.
      Enable by adapting your email.cfg file.
    * Work on using an SQL database for storing internal data has started.  If
      you're interested in Lire's development, you might like to take a look
      at the new experimental lr_sql2dlf(1), lr_dlf2sql(1) and
      lr_sql2report(1) scripts.  Thanks to Arnaud Gaillard for this spiffy code
      contribution!
    * New script lr_check_service (internally used).

- Documentation
    * Completely rewritten chapter on Lire's architecture in the Developer's
      Manual.
    * Documentation on all current DLF schemas is included in the Developer's
      Manual.
    * Updated development roadmap.

- Other business
    * Superservices and services are now found at runtime.  This makes Lire
      much more flexible: writing your own service plugin is now as easy as
      sticking it in libexec/lire/convertors.  Furthermore, the user interfaces
      to lots of scripts are now simpler: no more need to supply a redundant
      superservice argument.  Among these are lr_log2mail and lr_log2report.


      The subject of emails, sent by lr_log2mail (and by Online Responders)
      have changed: these no longer list the superservice name.  A jobfile,
      as used by lr_cron(1) no longer needs to set the superservice
      environment variable.  We have tried to remain backwards compatible.
      However, you're strongly advised to move to the new UI asap.  If you have
      made lots of local changes to your Lire installation, you're very likely
      gonna get hit by some changes.  Refer to the 'Upgrading from Lire 1.0'
      section in the INSTALL file for more details.


version 1.0 - 23 June 2002

* It is now possible to merge reports over time and over different servers:
  generating weekly and monthly reports, as well as site-wide reports can now
  be done in a much more easy and efficient way.  Refer to the "Merging
  Reports" section in Chapter 3 of the Lire User's Manual for details.
* lr_xml2mail and lr_xml2report supports multiple XML reports as input, by
  calling lr_xml_merge: this offers a user interface to the XML report merging
  functionality.  To support this, the interfaces of lr_archive_log,
  lr_xml2report and lr_xml2mail are slightly changed.  Since you probably don't
  call these scripts directly, this will not cause you any trouble.
* New firewall service: `pix', for Cisco PIX firewalls.  New database service
  `pgsql', for PostgreSQL query log files.  The list of supported services now
  is:

   superservice  services
   ============  ========
   database      mysql, pgsql
   dns           bind8, bind9
   email         argomail, exim, nms, postfix, qmail, sendmail
   firewall      cisco, ipchains, ipfilter, iptables, pix, welf
   ftp           iis_ftp, xferlog
   print         cups, lprng
   proxy         ms_isa, squid, welf_proxy
   www           combined, common, modgzip, referer, w3c_extended

* New output format: excel95. This outputs report in a Excel95 spreadsheet. 
  This can be useful if you want to generate sexy 3d charts from your
  Lire reports. You need the Spreadsheet::WriteExcel CPAN module for this.
* Renamed lr_anonimize and lr_deanonimize to lr_anonymize and lr_deanonymize:
  fixed spelling bug.
* Fixed bugs in sendmail2dlf: 1) which caused from=<foo@bar.com   > to get
  filed under the 'bar.com___' domain. 2) which caused messages from <> to a
  pipe (e.g. procmail) to get filed as being sent _from_ the controlling user
  (while it should be _to_ this user, in this case)  Tnx Edward Eldred for
  reporting these bugs.
* For Top-reports (e.g. `Top 10 Referring Pages' in the www report), specifying
  `0' in the superservice.cfg file as the parameter for this report (e.g.
  `referer_to_show=0' in www.cfg) will cause all available data to get
  displayed.
* Added keyword_to_show and engine_to_show variables to the www "Requests By
  Keywords" and "Requests by Search Engine with Keywords" reports: now these
  reports no longer show _all_ keywords.  Set the variables to 0 in www.cfg
  to get back the old behaviour.

version 20020415

* New email service: ArGoSoft Mail Server Pro log files are now supported 
  through the argomail service.
* Memory footprint of the report generation process is now limited. Lire 
  will not memory DoS your server because it takes too much memory. It can now 
  process very big log files without taking too much memory or disk space.
* lr_log2mail and lr_rawmail2mail now supports -i and -o options. The report
  will be sent using proper MIME encoding for the output format. 
  (X)HTML reports should be displayed inline by all mailers supporting 
  HTML mail.  Beware!  When specifying multiple destination addresses on the
  lr_log2mail command line, separate them by _spaces_.  Separating these by
  commas no longer does the trick.  Furthermore, lr_log2mail's last optional
  `flags' argument is no longer supported.
* To use the responder, the lr_log2mail and the lr_rawmail2mail command, 
  you will need to install the MIME::Tools CPAN package: We now use the more
  complete and reliable MIME::Tools package instead of the mpack(1) and
  munpack(1) command line tools.
* There is a new lr_xml2report command which acts as a frontend to all the
  supported output formats.
* lr_log2report doesn't need any error file argument anymore:
  Call it using lr_log2report superservice service < logfile > report.
* The report configuration file to use can be specified to lr_log2report and 
  lr_log2mail by using the -c option.
* New output formats: html_page and xhtml_page which generates a one page 
  (X)HTML report.
* There is now a records element that can be used to include fields from 
  DLF records matching certain criteria in a report specification.
* timegroup, rangegroup and timeslot element can now be nested like the group 
  element could.
* The uniq filter element was removed. You will want to use the fields
  attribute of the count element instead.
* Improvements to the ASCII formatter. Better formatting of DocBook itemized 
  lists. Long names are wrapped instead of being cropped.
* lr_log2report now handles compressed log file.
* Removed obsoleted *-by-day report specifications.
* The postfix2dlf convertor has been redesigned: it now no longer calls 
  sort(1), this makes it more flexible: it will be easier to support 
  non-syslog timestamped logs.  It now creates a time-sorted DLF.  It now 
  fully supports postfix logs timestamped by the Sun Solaris syslog.
* Improvements and bug fixes to the LPR-NG DLF converter.
* Many improvements to the www report. The report was reorganised in several
  sections. New reports were added: top-traversals, user_session-visit_number,
  top-referring_sites, keywords-by-search_engine, requests-by-search_engine.
* Important bug fixes in the www session analyzer. For one thing, all
  unifinished session were ignored. Images and stylesheets are now excluded
  from the page_count and other fields in sessions.
* Some report specifications were obsoleted by the use of global filters. 
  If you used them in custom report configuration, you will have to update
  your configuration. The User's Manual explain which report to use instead. 
  They will be removed in a future release.
    (www): top-pages-by-client_host top-tracked_pages-by-client_host
	   tracked_pages-by-period user_session-finished
    (dns): requests-by-period-by-method requesttype-distribution-by-method
	   top-requested-names-by-method top-requesting-hosts-by-method
* Using `apache' as a service name is deprecated.  Instead of `www apache',
  you will have to use `www common'.  Furthermore, using `boa' as a service name
  is deprecated: `www boa' is now called `www combined'.  Finally, `www
  apachemodgzip' is now called `www modgzip'.
* A lot of environment variables are renamed: an `LR_' prefix was added to the
  names, to prevent nameclashes.  If you have made some local configuration
  changes, you will have to adapt these: this change is NOT backwards
  compatible.  See INSTALL for upgrade details.

version 20020214

* Output when requesting HTML or XHTML was changed. The output is now
  always an uncompressed tar file (like when generating images
  previously). This makes HTML and XHTML output identical whether or
  not images are generated. Also, when report sections are used, the
  (X)HTML reports will be broken into several files.
* The HTML manuals are now splitted in several files.
* lr_config can now be run by any user (not only the Lire's
  administrative user).
* Reports can now be subdivided into sections. This can be achieved by
  using =section directives in the report's configuration file.
  Consult the User's Manual for the complete story.
* Sections can contain filters that will be applied to the input of
  all the section's subreports. Consult the User's Manual for the
  complete story.
* New proxy superservice, with support for MS ISA, WELF and squid logs
  and a lot of reports.
* New database superservice, with service MySQL and reports top-users,
  top-databases, actions-by-period, top-querytypes.
* New FTP service: IIS FTP Logs (iis_ftp).
* New firewall services: iptables, welf. 
  WELF is the WebTrends Enhanced Log Format supported by several
  firewall products (see
  http://www.webtrends.com/partners/firewall.htm for a list). Common
  firewalls using this format: SunScreen, SonicWall, Raptor, etc. This
  makes Lire support a lot of firewalls. We didn't test it with all of
  those products though. So we greatly appreciate all feedback
  regarding how Lire behaves with those firewalls' logs.
* The firewall superservice can also be used for network intrusion
  detection type of logs.
* The firewall superservice now includes a lot of new reports.
* The incompabitilities between the Cisco service and the other firewall 
  services were resolved.
* New ftp reports: bytes-by-user-by-period,
  bytes-by-dir-by-user-by-period, top-users-bytes.
* New www reports: requests-by-keywords, top-referers.
* New email service: Netscape Messaging Server (nms).
* New dns reports: requests-summary, requests-summary-by-method,
  requesttype-by-method, requests-by-timeslot,
  req-by-period-by-method, req-by-timeslot-by-method
* Support for Netscape Messaging Server log daemon.
* Improvements in postfix email service: more robust qmgr line
  handling (no longer creates bogus size fields in dlf), support for
  postfix/virtual lines. (Thanks Cedric Gross for helping us!)
* Online responder more robust against evil characters in subject of
  received email message. Directories in
  var/lib/lire/data/{email,log,report}, named after ID tags, which are
  contructed from these subjects, could contain \ in their names. This
  is ugly, and breaks non-robust scripts. It even might have had
  security implications. People running an online responder should
  upgrade. The change is in lr_processmail.
* Changes to the LRSML DTD: You may have to modify custom report
  specifications you wrote. 
    - The 'sort' attribute was removed from the filter-spec element.
* The WWW::* perl modules, like WWW::Useragent and WWW::Domain, are
  moved to Lire::WWW::*, in order to avoid future nameclashes.

version 20011205.1

* Fixed distclean target.
* Fixed typo in lr_xml2logml.
* Added missing doc/lire.ent in tarball.
* Fixed sendmail queue identifier parsing for very old and very new versions. 
* Fixed a timestamp bug in exim converter.

version 20011205

* Only RTF and PDF output now requires Jade. HTML and XHTML output can be 
  generated with xsltproc and Norman Walsh's DocBook XSL Stylesheets. 
  IMPORTANT: this means that you need to install Norman Walsh's XSL
  Stylesheets available from
  http://docbook.sourceforge.net/projects/xsl/index.html and
  libxslt version 1.0.4 or later to generate HTML output.
* New firewall service: Linux 2.2 ipchains log, as well as Darren Reed's
  IP Filter logs, as shipped with *BSD's.
  NOTE: there are some "known issues" with the IOS Cisco service in
  this release. The informations it gives isn't really equivalent to
  the other firewall services. You may consult
  http://www.nlnet.nl/projects/logreport/hypermail/logreport/development/0518.html
  for more informations.
* New superservice "print" with CUPS (page_log) and LPRng (lp-acct) plugins.
* Syslog parser now handles Solaris 8 syslog tag: [ID ...]
* lire:timegroup aggregator can now aggregate records by months or years.
* New aggregators available to build report specification: lire:summary, 
  lire:rangegroup and lire:timeslot
* New www service: W3C Extended Log Format (Used by Microsoft's Internet
  Information Server IIS 4.0 and IIS 5.0).
* New www reports: requests-summary, requests-by-size, requests-by-timeslot
* More www user session reports.
* Speed improvements to the www robot, country and OS analyzers.
* New robots and Nimda attack detected in www reports.
* New ftp reports: tracked-users, tracked-files.
* New email reports: deliveries-by-delay, deliveries-by-size, 
  tracked_senders, tracked_recipients.
* Sendmail converter is now more robust and interpret more lines. (Anti-spam
  messages will now get into the reports)
* The Lire client no longer is configured by running
  ./configure --disable-server.  If you want to run Lire as a client for an
  online responder, you just install Lire in the regular way.
* Various improvements to most DLF converters. (DNS converters were optimized, 
  all converters were updated to the new DLF API).
* All "by-day" reports have "by-period" equivalent: (www): bytes-by-period, 
  bytes-by-result-by-period, clienthost-by-period, requests-by-period, 
  requests-by-result-by-period; (ftp): bytes-by-period.
  The following report specification are now obsolete and will be removed in 
  a future Lire release: (www): bytes-by-day, bytes-by-result-by-day, 
  clienthost-by-day, requests-by-day, requests-by-result-by-day; 
  (ftp): bytes-by-day
  You should update your report configuration accordingly.
* Bytes, seconds and numbers can be scaled to more human readable format by 
  setting the variables LR_SCALE_BYTES, LR_SCALE_SEC and LR_SCALE_NUMBER to 
  'yes'. By default, bytes and seconds are scaled into '1k' and '2m' 
  in the reports.
* Fixed various small bugs (e.g. the one which caused .org to get skipped in
  the www 'Requests By Top Level Domain' report, as well as the one which
  caused requests from .net clients to get filed under .et, when using perl
  versions << 5.6.1 (5.005_03 is reported to behave in such a way))

version 20011017

* The www report is more exhaustive now: New www reports: requests-by-attack, 
  tracked_pages-by-period, top-last_page, user_sessions-by-period reports.
  Added detection of Code Red attack in WWW reports. Added detection of 
  country where client is located (WWW superservice). Added detection of 
  GNU/Hurd OS and Links browser in WWW reports.
* Added volume per hour email report.  Reordered email reports, as suggested
  by Thierry Montigneaux (thanks Thierry!)
* Added firewall superservice, added cisco service, for cisco acl logfiles
  (Thanks Joost Bekkers)
* Added FTP superservice. Xferlog file format is now supported.
* Overall performance improvement.
* This version introduces a complete overhaul of the reporting engine which
  will make customization and extension of Lire much easier. If you are 
  upgrading from a previous version, you probably want to read the "Upgrading
  From Lire's Version 20010903 or Earlier" section of the INSTALL
  file which has important informations about how to migrate your
  configurations.  Some changes in this release (especially the way 
  configuration files are handled) are _not_ backwards compatible.
* The xml report format has changed.  Therefore, if you install this version on
  a responder, submitters should install this version if they want to be able
  to typeset your reports, after deanonymizing them. 
* XML::Parser and expat (http://expat.sourceforge.net/) are now required
  to build and use Lire. You can install those components separately or you can
  get the lire-full tarball which includes and build them as part of the normal
  Lire installation process. The only supported XSLT processor with this 
  version is xsltproc which comes with the XSLT C library for Gnome 
  (http://xmlsoft.org/XSLT/)
* Reports are now written using the XML Lire's Report Specification Markup 
  Language (LRSML). DLF formats are specified using the XML Lire's DLF Schema 
  Markup Language.
* ASCII reports generated by the builtin formatter or the XSL tools are now 
  identical.
* Informations included with a report can now be tweaked according to the
  intended audience. You can set LR_TARGET_USER in your 
  $HOME/.lire/etc/defaults file to 'sysadmin' or 'manager'. There is also the 
  LR_USERLEVEL variable which can be set to either 'normal' or 'advanced'. 
  Those variables will modify the description accordingly. Note that not all 
  reports have different informations based on those variables.
* There is a LR_MAX_MEMORY variable which you can set to process big log 
  files without trashing. The default is 40Megs. For optimal
  performance, you should set this to half your available RAM (unless
  you are usually short on memory). 
* All Lire's programs now log performance information: time taken, 
  memory used (on Linux only).
* Reports which need missing information in the DLF file are now
  automatically skipped.
* Report dates are now in the local time zone.
* 8bits characters are escaped in the reports to ?.
* New supported output format: XHTML and RTF.
* Improvement to charts: Height increased to 300 pixels. X labels are drawn 
  vertically making it possible to read e.g. URLs.
* Charts for each report that support it are now generated when 
  LR_INCLUDEIMAGES is set to 1. (Not only the first one).
* New chart type: histogram (bars now have space between the bars)
* When generating an HTML, XHTML or DocBook report with images, the result 
  is a tar gzipped file that contains the report file and the images.
  When generating an RTF report with images, the result is a zip file 
  that contains the report file and the images.
* The user's and developer's manuals are reorganized. There are still a lot 
  of the new intended sections uncompleted, though.
* The apachemodgzip superservice is now a regular www service called 
  'modgzip'. The apache service was split into three different ones: common, 
  combined, referer.
* Fixed bugs (as always ;)

version 20010903

* Lot of bugs removed
* BEWARE!  The file ~/.lirerc is obsolete.  If you use it, please move it
  to ~/.lire/etc/defaults.
* Fixed "bogus message: mkdir: cannot create directory `/usr/local/var':
  Permission denied"-bug.  (This one occured when one did a systeminstall,
  and ran the lire scripts as a non-priviliged user afterwards.)
* Now setting print-category and print-severity in named.conf is optional:
  a wider range of dns / bind9 logs gets accepted.
* Added Apache referer parser: now one can see which links brought visitors
  to your site.
* For www user agents newly detected is: operating system, browser type, user
  browser UI language and web robots
* Added 6 new "www" reports: http version, user agents, bytes per directory,
  top referer-page connection and http method
* More reports are now customizable
* Added new output formats: LogML (an experimental XML application) and DocBook
* Responder use munpack for all MIME messages now, not just multipart one.
* Security check can be disabled by setting LR_SKIP_ID_TEST.
* Default report format (txt, xml, pdf or html) can be configured via
  LR_DEFAULT_OUTPUT_FORMAT in $sysconfdir/defaults.
* XML/SGML environment can be reconfigured at runtime from $sysconfir/defaults.
* Support more XSLT processor : Sablotron and XsltProc (from libxslt). 
  Xalan-C can be found by more names (testXSLT or xalan-c)

version 20010629.1

* developer.html and developer.txt now distributed again with the tarball.

version 20010629

* Cleaned up configure.in, to offer more hooks via the environment.  This
  is nice for package builders.

version 20010626

* Moved /usr/local/lib/xml to /usr/local/share/lire, to adhere to FHS.
* Added reports for www superservice: report_bytesperresultperday (this was
  report_bytesperday, but one does no longer split things up) and
  report_totalperday and report_iptotalperday.
* Reduce warnings generated by postfix2dlf.
* Responder should reply with a proper error message when it fails to
  generate a report.
* XML and SGML support should now be more portable between various
  environment.
* PDF reports now can include images. This can be turned on in the
  .../etc/lire/defaults file, by setting LR_INCLUDEIMAGES to 1.
* Fixed possible security problem: we now run with umask 037, so that copied
  logfiles no longer possibly get world readable.
* Fixed bug in apache common convertor: now processes lines with empty http
  request ok.
* Fixed bug which got sendmail2dlf spit evil perl error messages when
  fed an empty logfile.  Thanks to Claire Holleman for finding it.
* Fixed bug which caused postfix2dlf to produce bogus dlf from logs containing
  qmgr lines caused by failed deliveries, e.g. in case of 'invalid recipient
  syntax'.  Thanks to Mark 'Xaa' Huizer for reporting this.
* Keeps old reports and dlf's in a Lire archive, if variable LR_ARCHIVE is
  set.
* Stores metainfo on logs and dlf's in a Lire database.
* Added report_sizeperfromdomain and report_sizepertodomain email subreports
* Added lr_xml2html script.
* Fixed problem with Jade (now uses xml declaration)
* Description blurbs in email report resurrected.  Added description blurbs
  to email report.
* Lire's own logfile now easier machineparsable: added LR_ID job tags
* Default spooldir for responder setups is moved from
  /usr/local/var/spool/logreport to /usr/local/var/spool/lire.  THIS CAN
  BREAK UPGRADES!  When you're upgrading a responder, i.e. a Lire system which
  automatically processes logfiles which get received in email messages, move
  your spool manually to the new location before restarting lr_spoold.  One
  should also fix the .forward's (or any other mechanism your mta uses) to
  deliver mail to Maildirs in the new spool location.

version 20010509

* Fixed long line problems in ascii.xsl and docbook.xsl.
* Added lr_xml2pdf script.
* Fixed bug in "bytes-per-http-result" report.
* Info from README, doc/developer.txt converted to xml.  Now shipped as 
  {developer,manual}.{html,pdf,txt}.  Per default gets installed in 
  /usr/local/share/doc/lire/.

version 20010418

* Pointrelease for debian package.

version 20010407

* Fixed bugs and improved time handling.  We now take year and timezone in 
  account, if available in the logfile.  Some code cleanup.
* Internals: raw intermediate format replaced by xml format.
* Fixed bug which blew up Apache.pm while dealing with clients who's ip
  resolves to a hostname only.  Tnx Wytze!
* Fixed some bugs and added new elements to the logreport xml stylesheet, to 
  gain flexibility.
* Added Exim2DLF: convertor for logfiles from the exim Mail Transport Agent. 
  ( http://www.exim.org/ .)
* Added per-user ~/.lirerc file.

version 20010318

* Changed package name from lr to lire.
* Fixed anoying bug in lr_anonymize: now cleans up tmpfiles.
* Added option to generate XML output.
* Added lr_config script, for easier configuration.
* Responder now is capable of sending reports to more than one Reply-To 
  address.
* Added lr_getbody manpage.
* More sane defaults for things like e.g. From-address for emails sent by 
  lr_log2mail.
* Various bugfixes, especially in dealing with bogus logs.
* Responder now sends informative message in case the submitted log was 
  hardly parsable.
* Added responder for access logs from the boa ( http://www.boa.org/ ) 
  webserver.
* More configuration hooks for www reports: one can specify not to get top 30,
  but top n.
* Added ability to track specific webpages.

version 20010219

* Added local site config file defaults.local, kept between upgrades.
* New www reports, added apache mod_gzip engine, added responder for
  apache "combined" logfiles.
* Added lr_run script: running the scripts will no longer spam your terminal
  with debug output.
* Client software now includes tools (lr_rawmail2mail and friends) to 
  automatically de-anonymize received report. Added notes about this to 
  README.lr-client.
* Added notes about running lr-client software on Solaris to README.lr-client.
* Some cleanup of filesystem layout: got rid of some directories, merged config
  files.
* Some finetuning of email reports.

version 20010116

* No longer depends on GNU make.
* Finds perl and sendmail path during configure: now builds, installs and
  runs out of the box on Solaris.
* Scripts respect --prefix and other configure flags. We're really using
  autoconf and automake now, which makes the package act sanely in other
  directory hierarchies. No longer uses LR_HOME and other environment 
  variables: no need to set them any longer.
* Now package behaves sanely when configure is run without any arguments.
* Integrated a client package. During configure, one can choose to install 
  just the client, or the complete suite. The client enables one to send and 
  receive logs and reports in an anonymized format.
* Bugfixes in postfix and apache convertors.
* Documentation updated.

version 20001216

* Now runs on SunOS 5.7: fixed /bin/sh scripts and awk invocations.

version 20001213

* Various bugfixes, especially in sendmail2dlf
* Documentation updated
* New email dlf format, so that relay's ip number and fqdn can be used
  separately 
* postfix engine supports nqmgr
* Fixed awk scripts, to be able to run with the original awk, as shipped
  with OpenBSD
* lr_log2mail no longer relies on mutt(1), but uses the more generally 
  available /usr/sbin/sendmail

version 20001211

* Various bugfixes
* Documentation updated

version 20001205

* Bugfixes in qmail convertor
* An IPfilter convertor has been added
* A postfix convertor and report engine are added

version 20001130

* The responder uses a new mail receiving engine 
* The install now uses automake and autoconf
* Extra reports for www added
* Added manpages
* New setup of report scripts
* Scheduler no longer needed; more lean design
* Fixed various bugs

version 0.0.1 - september 2000

* First public release

