Chaotic Mess
================

The most important are by far sendMessage and subscribe::

	pub.subscribe(listener, topicName)

	pub.sendMessage(topicName, **data)
	
Listener 
---------

In the subscribe example, the listener is any Python callable object: function, 
method, etc. Hence given the following definitions::
	
	def function0(): pass
	
	class Foo:
		def method(self): pass
		@staticmethod
		def static(): pass
		@classmethod
		def classMeth(cls): pass
		
	foo = Foo()

these are valid listeners::

	function0
	foo.method
	Foo.static
	Foo.classMeth

Topic 
-------

In the subscribe example, the topicName is a string representing the name 
of the topic of the message, such as "user_input", "new_card", 
"game_over". It can contain any character a-z, A-Z, 0-9 and _&%$#@ 
and the hyphen. The dot "." and "/" are allowed but have special meaning, 
as explained in BEYOND. 

Topic Message Data
-------------------

The **data** in pub.sendMessage() is a set of keyword arguments containing 
the data to send as part of the message. For instance, ::

	pub.sendMessage("game_over")
	pub.sendMessage("new_card", suit="hearts", rank=8)
	
will send a message of topic "game_over" without any data, 
followed by a message of topic "new_card" with two pieces of data, 
namely a string representing the name of the suit of the 
card that was drawn, and an integer representing its rank.

The sendMessage() can only accept message data arguments that are 
compatible with the topic. This is defined as the Message Data 
Specification (MDS).

Message Data Specification (MDS)
---------------------------------

Each topic has a Message Data Specification (MDS) that 
specifies what data is required vs optional, and what 
argument names are to be used. The **MDS is inferred for
you by the first listener to be subscribed to a topic, 
or, by explicitely defining the topic via a Topic Tree
Definition (TTD)**. Defining the MDS via the TTD is explained
in a later section. 

Given the following callables::

	def onCancelSave(): pass
	def onNewCard(arg1): pass
	def onGameOver(arg1, arg2=default): pass
	
then the following subscriptions::
	
	pub.subscribe( onCancelSave, "user_input")
	pub.subscribe( onNewCard,    "new_card")
	pub.subscribe( onGameOver,   "game_over")
	
will create, assuming they are the first one for 
each of those topics, 

* a topic "user_input" with an empty MDS, ie no 
  data can be sent with messages of this topic; 
* a topic "new_card" with an MDS of one required 
  argument, named "arg1"; no other data can be 
  sent in messages of this topic, and the data 
  must be present;
* a topic "game_over" with an MDS of one required
  message data named "arg1" and one optional named
  "arg2"; when not given, the listener will get 
  the default value
  
Note that the default value for an optional message
data parameter is not part of the MDS; each listener
can therefore decide what default value to use if the 
data is not provided. 

Any subsequent subscription of a listener to an already existing 
topic will succeed only if the listener matches the 
MDS for the topic. When this is not the case, pubsub raises a
InvalidListener exception. 

Sending vs Broadcasting
--------------------------

The pub.sendMessage() has a similar effect to broadcasting
data: the topic is akin to the radio frequency of the broadcast. 
Indeed 

* all listeners subscribed to the topic will receive the 
  message, as in broadcasting
* the sender does not know which (if any) listeners 
  have subscribed to a topic, as in broadcasting
* a listener does not know the "emitter" or source of 
  the broadcast 
* the order in which listeners receive the broadcast is 
  undefined
  
Unlike real broadcasting, a message is sent to each 
subscribed listener in sequence rather than in parallel (but 
the sequence order is undefined). Each listener must return 
(rather than raise an exception) in order for the broadcast to 
complete. Any return value of a listener is ignored. 
Your code must
not make any assumptions about the order. In fact, you 
should design your application to be independent of this 
order. 

Types of errors
----------------

The main type of error you will encounter is when you 
attempt to subscribe a listener that does not have a 
signature (call protocol) compatible with the topic's MDS: 
a ListenerInadequte exception. Given ::

	def listener0(arg1, arg2=default0): pass
	def listener1(arg1): pass
	def listener2(arg1, arg2): pass
	def listener3(arg1=val, arg2=default3): pass
	
	pub.subscribe(listener0, "topic")

then ::

	pub.subscribe(listener1, "topic")
	
will fail since arg2, which might get sent, could not be 
received by the listener. Similarly, ::

	pub.subscribe(listener2, "topic")

will fail because the second parameter is required, yet 
the MDS allows for that parameter to be missing from 
some messages, so listener2 is too "demanding". However, ::

	pub.subscribe(listener3, "topic")
	
will succeed because eventhough listener3 doesn't require 
arg1 (it has a default value), the MDS will force you to always
provide one. 

A second type of error is forgetting to 
include required data as part of the message. If the 
MDS for "new_card" is a required suit and an optional rank, 
then ::

	pub.sendMessage("new_card", rank=6) 

will raise an exception since suit, required, is missing. 

A third type of error is a silent one, unless you make 
use of a topic tree definition provider (TTDP, defined later): 
it occurs when you mistype the name of a topic. Without 
a topic tree definition provider, any new topic name causes
the creation of a new topic. Therefore "new-card" would create
a new topic. This can lead to hard-to-isolate bugs as the 
listeners subscribed to the topics having a typo in their 
name will never get a message! The strategy when a 
listener doesn't get a message should then be:

* confirm that the message gets generated
* confirm that the listener is subscribed to the correct 
  topic name (no typo); this is easiest with 
  `printTopicDocs(extra='L')` which will list the 
  listeners of each topic::
  
    from pubsub import pub
    from pubsub.utils.topictreeprinter import printTreeDocs
    printTreeDocs(pub, extra='L')

  
Architecture and Design Guidelines
-------------------------------------

* Look at examples. Typically GUI, controller, model
* A listener should not return any data: the return value of 
  a listener is not forwarded to the sender since senders and 
  receivers have no notion of each other. 
* A listener should not raise any exceptions or let any escape.
  When this happens, the exception will interrupt the 
  sendMessage. It is not a good idea to put an except clause 
  around the sendMessage() since the sender should remain 
  ignorant of what exceptions are possible. 
  See ListenerExcHandler to trap exceptions. 
* Topic names should represent user or system domain events, 
  not implementation details such as widgets or components. 
  For instance, "dialog1OK" is likely a bad idea: it says 
  nothing about the event (what is dialog 1 OK'ing?), and is 
  dependent on the widget's name (which could change). A 
  better name could be "fileSaveOK" if the dialog was asking 
  the user to confirm a file save. 
* Use pubsub.utils.exportTopicTree() to get a readout of 
  the topics and their MDS. Use pubsub.utils.printTopicTree() 
  to get a printout of the topics and optional characteristics
  such as MDS and list of currently registered listeners. 
* Start all listener functions and methods with *psOn*, for 
  instance ``def psOnCloseDocument()``.
  

Example Development Process
----------------------------

The following represents a typical development process when using pubsub:

- Design your application into independent modules or subpackages 
  that don't import one another
- Define basic event types exist in the application: 'user' (events from 
  user interface), 'filesystem' (events from local filesystem), etc. 
  These are your messaging topics. You may find it useful
  to use ``printTreeDocs`` from ``pubsub.utils.topictreeprinter``.
- Define some data for each event type, and which data are optional/required
- Implement your modules

    - Subscribe listeners with appropriate signature (according to 
      data for each topic/event type)
    - Send messages with appropriate data
    - Handle messages in listeners, without making any assumptions 
      about sender or order of receipt
    
- Testing: no need for API, all you need to do is send messages!

You can see a very informative view of an application before and after 
incorporatng pubsub, at `Steven Sproat's dev site`_ (click "expand all" 
and "show diffs side-by-side"). Steven says: 
  
  *You can see how I removed some GUI logic from the Canvas class (a 
  child of the GUI) and placed "controller" functions into my GUI that 
  subscribed to pubsub topics and delegates to the appropriate classes.*

.. _Steven Sproat's dev site: http://bazaar.launchpad.net/~sproaty/whyteboard/development/revision/286 


   
Topic Tree Specification
--------------------------------

*Topic Specification* can be used to have better control over your topic hierarchy. 
If you don't specify your application's topics, pubsub infers them from the first 
subscribed listener of each topic. E.g.::

    def listener1(arg1,      arg2=None): pass
    def listener2(arg1=None, arg2=None): pass
    
    pub.subscribe(listener1, 'topic.sub') 
    pub.subscribe(listener2, 'topic.sub')
    
Because listener1 is the first to be subscribed to 'topic.sub' topic, pubsub uses it to 
infer the specification of 'topic.sub': the specification is "messages of that topic 
*must* provide data for arg1, and *may* provide data for arg2". The second listener 
subscribed, listener2, is allowed to subscribe because it is compatible with the 
topic's specification created at the previous call. What if your intent was that arg1 is 
optional as well, i.e. the signature of listener1 is wrong (it should provide a default 
value for arg1)? Or what if per chance listener2 gets subscribed first (could happen if 
both are subscribed in different modules whose load order changes)?

There are two ways to go about this: 

- Call ``pub.newTopic()`` to explicitly create the topic; this must happen before
  any listener subscribes to it
- Use one or more *Topic definition providers* (TDP)  


Topic Definition Providers
---------------------------

The base class that defines the *Topic Definition Provider* API is 
``pubsubutils.ITopicTreeDefnProvider``. An implementation that uses a Python 
syntax to describe the topic tree, ie using Python classes and class variables, is 
available as ``pubsubutils.TopicTreeDefnSimple``.  Example::  

    from pubsubutils import TopicTreeDefnSimple
    
    class MainTopicTree(TopicTreeDefnSimple):
        class topic1:
            '''Explain what this topic is for'''
            optArg1 = 'explain what this optional arg is for'
            
            class subtopic2:
                '''Explain what this subtopic is for'''
                reqArg1 = 'explain what this required arg is for'
                _required = 'reqArg1'
                
                class subsubtopic3:
                    '''Explain what this subtopic is for'''
                    reqArg2 = 'explain what this second required arg is for'
                    reqArg3 = 'explain what this third required arg is for'
                    optArg2 = 'explain what this second optional arg is for'
                    _required = ('reqArg2', 'reqArg3')
        
    from pubsub import pub
    pub.addTopicDefnProvider( MainTopicTree() )

The use cases supported are: 

- Formalization of Topic tree: 

  - You have not used the above yet;
  - You have a preliminary topic tree which gets created for you by pubsub 
    based on listeners that you subscribe (default when ``pub.addTopicDefnProvider()``
    not called); 
  - You then want to formalize the tree such that can you specify clearly 
    what arguments are required and optional, verify that there are no 
    erroneous branches (due to a typo in a topic name), and document each 
    topic and argument;
  - You want to keep expanding the tree, without always having to update formal 
    topic tree specification 

  To handle this, you would use the above example code. Note that a preliminary tree 
  can be obtained from your application via ``pubsubutils.printTreeSpec()``.
  
- Constrain the Topic tree:

  - Your application reaches a milestone, such that you want to "freeze" the tree;
  - You don't want to allow any other topics to be created; 
  - If new topics are needed they will be few such that it's ok to require that
    they be first specified via a definition provider
  
  To handle this, you would add to the above example:: 

    pubsubconf.setTopicUnspecifiedFatal(True)

  Then any attempt to work with a topic that does not exist will raise an 
  ``pub.TopicUnspecifiedError``.

- Allow application components to extend the tree:

  - Your application defines a basic tree
  - Your applications uses a plugin mechanism to support extensions to itself, 
    or your application consists of several disjoint architectural components 
    that communicate exclusively via pubsub (presumably, one of those components
    is the "core" part of the application). 
  - The plugins or components should be able to add subtopics to the tree
    without the having to edit the main TDP of the tree. 
    
  The ``pub.addTopicDefnProvider()`` can be called as many times as you wish.
  When an unknown topic is required, each TDP will be queried for it. This 
  means that plugins need merely to call this function with their part of the 
  topic tree. For instance::
  
    class PluginATree(TopicDefnProviderSimple):
        class topic1:
            class subtopic2:
                class subsubtopic4:
                    '''Describe new subtopic created by plugin'''
                    optArg2 = 'explain what this second optional arg is for'
            class subtopic5:
                '''This is a new subtopic of topic1 so the plugin can...'''
                # no additional args
                
        class topic2: 
            '''New root topic used only by plugin'''
            optArg1 = '...'
           
  This works because any topic that does not have a docstring is not considered 
  to be a specification. 

Note that the representation of a Topic Tree Specification need only adhere to 
the ``ITopicTreeDefnProvider`` interface.  Other implementations are possible, 
e.g. one using XML from an XML file would be useful.  

 
Notification
-------------

Pubsub can call a specified handler everytime certain types of calls are made to it: 

- *subscribe*:    whenever a listener subscribes to a topic
- *unsubscribe*:  whenever a listener unsubscribes from a topic
- *deadListener*: whenever pubsub finds out that a listener has died
- *send*:         whenever the user calls sendMessage()
- *newTopic*:     whenever the user defines a new topic
- *delTopic*:     whenever the user undefines a topic

A handler class is available already: ``pubsubutils.NotifyByPubsubMessage``. 
This handler takes each notification received and generates a pubsub 
message in the "pubsub.*" branch. Your application gives the handler *class* 
(whether or not it is NotifyByPubsubMessage) to pubsubconf at initialization (first 
time pubsub is loaded into your application) by calling ``pubsubconf.setNotifierClass``.

Therefore, to use notification via this notifier, you must register one or more 
listeners for the various special topics. A default listener of
all those topics is available, ``pubsubutils.DefaultLogger``, which registers
its methods as listeners of the pubsub topics. These listeners merely print
a message to stdout. 

A shortcut is available from pubsubutils, to do all of the above. For example::

    #import pubsubconf              # optional
    #...use pubsubconf functions... # optional
    
    import pubsubutils
    logger = pubsubutils.useDefaultLoggingNotification()
    # probably no point in calling pubsubconf functions after this

You can make your own pubsub notification handler by deriving from 
``pubsubutils.INotificationHandler`` and giving it to ``setNotifierClass``. 


.. _exception-handling:

Exception Handling
-------------------

How to recover from exceptions raised in listeners


Multi-threading
----------------

The main challenge with using pubsub in multi-threaded application is
insuring that the listener is called only from within its own thread,
or provides some inter-thread synchronization mechanism. There are so
many ways of doing the latter that it is unlikely that anything pubsub
could provide would be of general use (that said, I'd be happy to be
proven wrong). See the discussion on pubsub_dev,
http://groups.google.com/group/pypubsub_dev/browse_thread/thread/7f414e82f62d64b7.


Multi-processing
-----------------

In progress: passing messages between python scripts spawned from a master 
application, to support transparent publish-subscribe of real multi-processing 
python applications (ie not limited by GIL). Consider:

- mmap module: mentioned on http://bytes.com/forum/thread25421.html
- pyprocessing package: http://pyprocessing.berlios.de/doc/index.html
- pp package: http://www.parallelpython.com/

and possibly other items on http://wiki.python.org/moin/ParallelProcessing. 


Utilities
---------

- Print:
  
  * current specification: ``pubsubutils.printTreeSpec()``
  * current tree: ``pubsubutils.printTreeDocs()``


Miscellaneous
--------------

other types of errors
other: topic creation, listener validation, Listener, TopicObj
arg1 messaging protocol
migrating from old pubsub
multithreaded apps
pubsub in wxPython (old vs new pubsub, autopubsubsetupv1/setupv1; wxPy v2.8.11)
multiple publisher domains