2010-11-10  -  SkyTools 2.1.12  -  "Portable Minefield"

To apply Londiste database-side fixes, run 'londiste.upgrade.sql'
where needed.

 = Features =

  * Support Postgres 9.0.  (Devrim Gündüz, Jason Buberel, Sébastien Lardière)

  * walmgr: Introduce a 'slave_pg_xlog' configuration variable.
    This allows master and slave pg_xlog files to be in different
    locations. During restore this directory is symlinked to
    slave pg_xlog.  (Steve Singer)

  * walmgr: Introduce a 'backup_datadir' configuration variable to
    control whether the slave data directory is kept or
    overwritten during restore.  (Steve Singer)

  * walmgr: consider wal_level during setup/stop (Martin Pihlak)

  * skytools.dbstruct: support version-dependant sql.

  * psycopgwrapper: always set db.server_version

 = Fixes =

  * londiste copy: restore index clustering info.  (André Malo)

  * londiste repair: use subprocess module instead os.popen4, where
    available.  (Shoaib Mir)

  * pgqadm: Remove unnecessary JOIN in refresh_queues().  (Artyom Nosov)

  * londiste add: dont send new table/seq list into queue, not useful,
    only can cause surprise removal in case of several 'provider add'+
    'subscriber add' pairs.

  * londiste launcher: don't interpret file 'londiste' as module.
    Needed for 3.0 compatibility.

  * walmgr: "sync" now omits unneeded WAL segments if the
    database has been cleanly shut down. This greatly reduces
    sync time during planned switchovers as usually there
    is only a single WAL file to be synched to slave.

  * londiste.link: Add missing quoting.  The functions are unused,
    thus missed the last round of fixes.

  * Building from CVS/GIT assumes --with-asciidoc automatically.

  * Newer asciidoc/docbook do not need fixman.py hack.  Remove it.

2010-02-03  -  SkyTools 2.1.11  -  "Replicates Like Randy Rabbit"

  = Fixes =

  * londiste, pgq smart triggers: crash in case the table info cache
    invalidation callback was called from signal handler.  Fix it by
    moving cache operations out of callback.

  * walmgr:
    - symlink handling for pg_log and pg_xlog
    - Set archive_command to "/bin/true" for restored postgresql.conf
    (Martin Pihlak, Mark Kirkwood)

  * londiste copy:
    - Make constraint dropping work for inherited tables
      (Hannu Krosing)

    - Do not restore fkeys to tables with unknown replication status
      or coming from different queue.  (Hannu Krosing)

    - Use TRUNCATE ONLY on 8.4+.  (Sergey Konoplev)

    - Replace last copy pidfile check with signal_pidfile,
      thus avoiding potential infinite stall with dead copy.

  * londiste repair: set LC_ALL=C when calling 'sort' to have
    byte based sorting.

  * skytools.DBScript:
    - Add --version switch to show skytools version.  (Hannu Krosing)

    - Safer pidfile handling - avoid creating zero-length files
      on disk full situation by deleting them on write error.

  * pgq.Event: ev.retry, ev.ev_retry fields for retry count.
    (Nico Mandery)

  * pgq.maint_retry_events(): lock table to allow only single mover.

  * pgq.logutriga() did not put custom pkey= value into events.

  * pgq.logutriga() and pgq.sqltriga() did allow UPDATE and DELETE
    on tables without pkey, running into SQL errors downstream.
    They should throw error in such case.

  * Fix DeprecationWarning on Python 2.6 vs. 'import sets'.

  * make deb: Work around Debian's --install-layout related braindamage.

2009-08-31  -  SkyTools 2.1.10  -  "As Stable As A Falling Anvil"

  = Fixes =

    * pgqadm: rename 'as' function argument as it's keyword in Python 2.6

    * londiste provider add-seq:
      - Detect existing sequences.
      - Make --all ignore pgq, londiste and temp schemas.
      - Proper quoting was missing in few places.
    
    * walmgr: Create pg_xlog/archive_status directory for slave restore.
      (Mark Kirkwood)

    * londiste provider add: detect and warn about user AFTER triggers
      that run before Londiste one.
    
    * docs:
      - Documentation for log_size, log_count and use_skylog options.
      - Refresh dependency list, add rsync
      - Mention --with-asciidoc in INSTALL.

    * --with-asciidoc: make it tolerate missing asciidoc and/or xmlto.
    
    * deb84 Makefile target.

2009-03-13  -  SkyTools 2.1.9  -  "Six Pack of Luck"

  = WalMgr improvements =

    * walmgr.py: WAL files purge procedure now pays attention to recovery
      restart points. (%r patch by Omar Kilani, Mark Kirkwood)

    * walmgr.py: archive_mode GUC is now set during setup. Followed by an
      optional restart if master_restart_cmd is specified (Mark Kirkwood)

    * walmgr.py: PostgreSQL configuration files are backed up to 
      "config_backup" and restored to "slave_config_dir". (Mark Kirkwood)

    * walmgr.py: Backups taken from slave now generate backup_label and
      history files.

    * walmgr.py Configuration files now default to PostgreSQL 8.3

  = Fixes =

    * londiste copy: Add missing identifier quoting for
      TRUNCATE and ADD CONSTRAINT.  (Luc Van Hoeylandt)

    * Quote identifiers starting with numeric.  (Andrew Dunstan)

    * Fix crash with pgq.sqltrigq/pgq.logutriga, which automatically
      detect table structure.  Problem was handling table invalidation
      event during internal query.
      (Götz Lange, André Malo)

    * Fix 'uninitialized variable' warning and potentially bad code
      in pgq.sqltriga().  (Götz Lange, André Malo)

    * skytools._cquoting: Fix crash with Decimal type.
      (May affects users who have compiled psycopg2 to use
       Decimal() instead of float for NUMERIC)

    * skytools.DBStruct: The DEFAULT keyword was missing when
      creating table columns with default value.  Table creation
      functionality is unused in 2.1.x thus no users should be affected.

    * skytools.magic_insert: be more flexible about whats dict.
      In particular, now accept DictRow.

    * configure.ac: don't detect xmlto and asciidoc version
      if --with-asciidoc is not used.  Otherwise unnecessary error
      message was printed.

    * Add few headers for Postgres 8.4 that are not included
      automatically anymore.  (Devrim Gündüz)

2008-10-12  -  SkyTools 2.1.8  -  "Perestroika For Your Data"

  = Fixes =

    * deb: Make debian package accept skytools-8.3 too.
    * add skylog.ini as installalble file (David Fetter)
    * Londiste:
      - Document the fkeys, triggers, restore-triggers commands.
      - Run ANALYZE after table is copied over.
      - Fix "provider add-seq --all"
    * pgq.SerialConsumer (used by Londiste) - fix position check,
      which got confused when sequence numbers did large jump.
    * PgQ database functions:
      - pgq.maint_rotate_step1() function removed tick #1
        even if consumer was registered on it.
      - pgq triggers: import cache invalidation from HEAD.
        Now pgq.sqltriga() and pgq.logutriga() should be preferable
        to pgq.logtriga().
      - uninstall_pgq.sql: Correct syntax for DROP SCHEMA CASCADE.
    * skytools.DBScript, used by all Python scripts:
      - Don't try to stay running if MemoryError was thrown.
      - Detect stale pidfile and ignore it.
      - Stop hooking SIGTERM anymore.  Python signal handling and
        libpq polling did not interact well.
    * scriptmgr:
      - Don't crash on missing pidfile.
      - Fix few typos on scriptmgr  (Martin Pihlak)
    * walmgr  (Martin Pihlak):
      - improved error messages on startup if no config file specified.
      - walmgr.py stop now also stops syncdaemon
      - pg_auth file is copied to slave as part of archiving. restored during boot.
      - cleanup in remote_walmgr() to always pass config file to slave walmgr.
      - master_backup() now reports exceptions.

  = Features =

    * londiste -s/-k now kill "copy" process with SIGTERM.
      Previously the process was left running.  On startup
      "replay" will check if "copy" process had died before
      finishing and launches one if needed.  (Dimitri Fontaine)
    * New skytools.signal_pidfile() function.
    * Londiste: New lock_timeout parameter to limit time
      locks are held on provider.  Applies to operations:
      provider add/remove, compare, repair.
    * Londiste: on psycopg2 v2.0.6+ use new .copy_expert() API,
      which does not crash on tables with large number of columns.

2008-06-03  -  SkyTools 2.1.7  -  "Attack of the Werebugs"

  = Fixes =

    * Fix pgq trigger compilation with Postgres 8.2 (Marcin Stępnicki)
    * Replace `make` calls with $(MAKE) in Makefiles (Pierre-Emmanuel André)
    * londiste: Fix incompatibility with Python 2.3 (Dimitri Fontaine)
    * walmgr: Fix typo in config symlinking code. (pychecker)
    * bulk_loader: Fix typo in temp table check. (pychecker)
    * Install upgrade .sql files.  Otherwise skytools_upgrade.py could
      be used only from source directory.
    * pgq.Consumer: Fix bug in retry/failed event handling. (pychecker)
    * pgq: Fix pgq.maint_retry_events() - it could create double events when
      amount of event to be moved back into main queue was more than 10.

  = Features =

    * Quoting of table and column names in Londiste and dispatcher scripts.

  = Upgrade procedure =

    * Database code under pgq and londiste schemas need to be upgraded.
      That can be done on running databases with following command:

        $ skytools_upgrade.py "connstr"

      Or by applying 'londiste.upgrade.sql' and 'pgq.upgrade.sql' by hand.
      The changes were only in functions, no table sctructure changed.

2008-04-05  -  SkyTools 2.1.6  -  "Quick Bugfix Release"

  Now we have upgrade script, see 'man skytools_upgrade'
  for info how to upgrade database code.

  = Fixes =

    * Include upgrade sql scripts in .tgz
    * Fix 'londiste provider seqs'
    * Fix 'londiste provider fkeys' in no tables added.
    * Fix "londiste copy" pidfile timing race
    * Fix Solaris build - avoid grep -q / define HAVE_UNSETENV
    * Fix "make debXX" when several Postgres version are installed.
    * New-style AC_OUTPUT usage.
    * Disable manpage creation by default, --with-asciidoc to enable.
      They are still included in .tgz so users should have less problems now.
    * Restore iter-on-values behaviour for rows from curs.fetch*.  The attempt
      to make them iter-on-keys seems now misguided, as iter-on-values is already
      used in existing code, and iter-on-keys returns keys in random order.
    * londiste subscriber add: Dont drop triggers on table if --expect-sync is used.
    * londiste copy: drop triggers and fkeys in case "replay" or "subscriber add" was skipped
    * walmgr restore: better detection if old postmaster is running (Charles Duffy)
    * walmgr xrestore: detect the death of parent process
    * walmgr restore: create pg_tblspc - its required for 8.3 (Zoltán Böszörményi)
    * walmgr restore: copy old config files over if exist (Zoltán Böszörményi)

  = Features =

    * Table name globbing for Londiste commands (Erik Jones)
    * New manpages for scripts (Asko Oja & me)
    * Upgrade script with manpage: scripts/skytools_upgrade.py
    * Add .version() function to pgq_ext & londiste schemas.
    * pgqadm: allow parameters without queue_ prefix in 'config' command.
    * skytools Python module:
      - intern() keys in db_urldecode() to decrease memory usage
      - udp-logger: more fields: hostaddr, service_name
      - udp-logger: dont cache udp socket, seem to hang in some cases
      - DBScript.get_database() allows explicit connect string
      - DBScript allows disabling config file loading
      - magic_insert on dicts allows missing columns, uses NULL
      - new parsing functions:
        - parse_pgarray(), parse_logtriga_sql(), parse_tabbed_table(),
        - parse_statements() - this one is used to split SQL install
          files to separate statements.
      - exists_function() checks both 'public' and 'pg_catalog'
        for unqualified functions names.
      - skytools.quoting: add C implementation for quote_literal, quote_copy,
        quote_bytea_raw, unescape, db_urlencode, db_urldecode.  This gives
        20x speedup for db_urlencode and 50x for db_urldecode on some
        real-life data.
      - unquote_ident(), unquote_literal()

2007-11-19  -  SkyTools 2.1.5  -  "Enterprise-Grade Duct Tape"

    = Big changes =

    * Lot of new docs [Dimitri Fontaine, Asko Oja, Marko Kreen]
    * Support for fkey and trigger handling in Londiste.  [Erik Jones]
    * Rewrite pgq.insert_event() and log triggers in C, thus SkyTools does
      not depend on PL/Python anymore.

    = Small changes =

    * pgq+txid: convert to new API appearing in 8.3 /contrib/txid/
    * Support psycopg2, preferring it to psycopg1.
    * Improved bulk_loader, using temp tables exclusively.
    * skytools.config: API change to allow usage without config file.
    * skytools module: quote_ident(), quote_fqident()
    * install .sql files under share/skytools in addition to contrib/
    * pgqadm: also vacuums londiste and pgq_ext tables, if they exist
    * londiste: provider add/remove --all [Hans-Juergen Schoenig]
    * backend modules support 8.3
    * pgq: switch pgq_lazy_fetch=NROWS for pgq.Consumer, which makes it use
      cursor for event fetching, thus allowing larger batches
    * txid: use bsearch() for larger snapshots

    = Fixes =

    * londiste fkeys: look also at dependers not only dependencies.
    * pgq.consumer: make queue seeking in case of failover more strict.
    * scriptmgr: dont die on user error.
    * pgq: there was still fallout from reorg - 2 missing indexes.
    * Due to historical reasons SerialConsumer (and thus Londiste)
      accessed completed tick table directly, not via functions.
      Make it use functions again.
    * londiste: set client_encoding on subscriber same as on provider
    * londiste: remove tbl should work also if table is already dropped [Dimitri Fontaine]
    * couple walmgr fixed [Martin Pihlak]

    = Upgrade procedure for database code =

    * PgQ (used on Londiste provider side), table structure, plpgsql functions:

        $ psql dbname -f upgrade/final/v2.1.5.pgq_core.sql

    * PgQ new insert_event(), written in C:

        $ psql dbname -f sql/pgq/lowlevel/pgq_lowlevel.sql

    * PgQ new triggers (sqltriga, logtriga, logutriga), written in C:

        $ psql dbname -f sql/pgq/triggers/pgq_triggers.sql

    * Londiste (both provider and subscriber side)

        $ psql dbname -f upgrade/final/v2.1.5.londiste.sql

    * pgq_ext:

        $ psql dbname -f upgrade/final/v2.1.5.pgq_ext.sql


2007-04-16  -  SkyTools 2.1.4  -  "Sweets from last Christmas"

    = Fixes =

    * logtriga.c was supposed to survive mismatched column string,
      but the logic were buggy.  Thanks go to Dmitriy V'jukov for
      good analysis.
    * Couple of scripts were not converted to new API.  Fix it.
    * Quiet a warning in textbuf.c
    * Make configure and Makefiles survive on BSD's where 'make'
      is not GNU make.  Thanks to Hans-Juergen Schoening.

    = Features =

    * Published WalMgr was an old version.  Sync with internal code,
      where Martin has done lot of enhancements.
    * Small incompat change in PGQ: add fields to pgq.get_consumer_info()
      return type.  Example upgrade procedure:

          DROP TYPE pgq.ret_consumer_info cascade;
	  \i structure/types.sql
	  \i functions/pgq.get_consumer_info.sql

      It will show some errors but thats ok.  Its annoying but needed
      for the tick_id cleanup in SerialConsumer/Londiste.

2007-04-10  -  SkyTools 2.1.3  -  "Brown Paper Bag"

    Still managed to sneak in a last-minute typo.

    * Fix copy-paste error in table_copy.py
    * Remember to bump version in pgq.version()

2007-04-09  -  SkyTools 2.1.2  -  "Help screen works"

    Most fallout from reorg is hopefully cleaned now.

    * Dumb bug in ticker wich made it almost non-working,
      except it managed to complete the regression test...
    * Move --skip-truncate switch from 'copy' to 'londiste add'.
    * 'londiste install' also installs plpgsql+plpythonu.
    * SQL installer logs full path to file it found.
    * Change pgq.get_queue_info() FOR loop variable to record
      instead text that was reported to fail, although it did work here.
    * Remember where the SQL files were installed.

2007-04-06  -  SkyTools 2.1.1  -  "Needs more thrust to fly"

    SkyTools got big reorg before release, but with the hoopla
    with the 3 projects at once, it did not get much testing...
    There are some untested areas still, but at least pgq/londiste
    are in better shape now.

    * pgqadm: finish conversion...
    * londiste.Syncer:
      - Convert to new API
      - Avoid ticking, parallel ticks are dangerous
      - Bad arg in repair
    * pgq:
      - too aggressive check in register_consumer
      - Algo desc for batch_event_sql
    * Add some knobs to make regtests for londiste pass
      more predictibly.

2007-03-13  -  SkyTools 2.1  -  "Radioactive Candy"

    * Final public release.

