title: Sources

h3. Sources

Using liquidsoap is about writing a script describing how to build what you want. It is about building a stream using elementary streams and stream combinators, etc. Actually, it's a bit more than streams, we call them sources -- in liquidsoap's code there is a <code>Source.source</code> type, and in <code>*.liq</code> scripts one of the elementary datatypes is source.

A source is a stream with metadata and track annotations. It is discretized as a stream of fixed-length buffers of raw audio, the frames. Every frame may have metadata inserted at any point, independently of track boundaries. At every instant, a source can be asked to fill a frame of data. Track boundaries are denoted by a single denial of completely filling a frame. More than one denial is taken as a failure, and liquidsoap chooses to crash in that case.

To build sources in liquidsoap scripts, you need to call functions which return type is <code>source</code>. For convenience, we categorize these functions into three classes. The _sources_ (sorry for redundancy, poor historical reasons) are functions which don't need a source argument -- we might call them elementary sources. The _operators_ need at least one source argument -- they're more about stream combination or manipulation. Finally, some of these are called _outputs_, because they are active operators (or active sources in a few cases): at every instant they will fill their buffer and do something with it. Other sources just wait to be asked (indirectly or not) by an output to fill some frame.

All sources, operators and outputs are listed in the "scripting API reference":reference.html.

h4. How does it work?

To clarify the picture let's study in more details an example:
%%(lang_dummy_example.liq)
radio =
  output.icecast(
    %vorbis,mount="test.ogg",
    random(
      [ jingle ,
        fallback([ playlist1,playlist2,playlist3 ]) ]))
%%


At every cycle, the output asks the <code>random</code> node for data,
until it gets a full frame of raw audio.
Then, it encodes the frame and sends it to the Icecast server.
Suppose <code>random</code> has chosen the <code>fallback</code> node,
and that only <code>playlist2</code> is available, and thus played.
At every cycle, the buffer is passed from <code>random</code> to
<code>fallback</code> and then to "playlist2</code> which fills it,
returns it to <code>fallback</code> which returns it to <code>random</code>
which returns it to the output. Every step is local.

At some point, <code>playlist2</code> ends a track.
The fallback detects that on the returned buffer,
and selects a new child for the next filling,
depending on who's available.
But it doesn't change the buffer, and returns it to <code>random</code>,
which also (randomly) selects a new child at this point,
before returning the buffer to the output.
On next filling, the route of the frame can be different.

It is possible to have the route changed inside a track,
for example using the <code>track_sensitive</code> option of fallback,
which is typically done for instant switches to live shows when they start.

h4. Fallibility

By default, liquidsoap outputs are meant to emit a stream without
discontinuing. Since this stream is provided by the source passed to the
output operator, it is the source responsibility to never fail.
Liquidsoap has a mechanism to verify this, which helps you think of
all possible failures, and prevent them.
Elementary sources are either _fallible_ or _infallible_, and this
_liveness type_ is propagated through operators to finally
compute the type of any source.
For example,
a <code>fallback</code> or <code>random</code> source is infallible
if an only if at least one of its children is infallible,
and a <code>switch</code> is infallible if and only if it has one infallible
child guarded by the trivial predice <code>{ true }</code>.

On startup, each output checks the liveness type of its input source,
and issues an error if it is fallible. The typical fix for such problems
is to add one fallback to play a default file or a checked playlist
(<code>playlist.safe</code>) if the normal source fails.
One can also use the <code>mksafe</code> operator that will insert silence
during failures.

If you do not care about failures, you can pass the parameter 
<code>fallible=true</code> to most outputs. In that case, the output
will accept a fallible source, and stop whenever the source fails,
to restart when it is ready to emit a stream again.

h4. Caching mode

In some situations, a source must take care of the consistency of its 
output. If it is asked twice to fill buffers during the same cycle, it 
should fill them with the same data. Suppose for example that a playlist is 
used by two outputs, and that it gives the first frame to the first 
output, the second frame to the second output: it would give the third frame 
to the first output during the second cycle,
and the output will have missed one frame.

It is sometimes useful to keep this is mind to understand the behaviour
of some complex scripts. The high-level picture is enough for users,
more details follow for developers and curious readers.

The sources detect if they need to remember (cache) their previous output in 
order to replay it. To do that, clients of the source must register in 
advance. If two clients have registered, then caching should be enabled. 
Actually that's a bit more complicated, because of transitions. Obviously the 
sources which use a transition involving some other source must register to 
it, because they may eventually use it. But a jingle used in two transitions 
by the same switching operator doesn't need caching. The solution involves two 
kinds of registering: _dynamic_ and _static activations_. Activations are 
associated with a path in the graph of sources' nesting. The dynamic 
activation is a pre-registration allowing a single real _static activation_
to come later, possibly in the middle of a cycle.
Two static activations trigger caching. The other reason for enabling caching
is when there is one static activation and one dynamic activation which 
doesn't come from a prefix of the static activation's path. It means that the 
dynamic activation can yield at any moment to a static activation and that the 
source will be used by two sources at the same time.

h4. Execution model


In your script you define a bunch of sources interacting together. Each
source belongs to a "clock":clocks.html, but clocks only have direct access
to _active sources_, which are mostly outputs.
At every cycle of the clock, active sources are animated: a chunk of stream
(frame) is computed, and potentially outputted one way or another.

This streaming task is the most important and shouldn't be disturbed.
Thus, other tasks are done in auxiliary threads:
file download, audio validity checking, http polling, playlist reloading...
No blocking or expensive call should be done in streaming threads.
Remote files are completely downloaded to a local temporary file
before use by the root thread. It also means that you shouldn't access NFS
or any kind of falsely local files.
