.nr LL 7.5i 
.nr PD 2v
.nr PI 10n
.ce 1
.I SDS VERSION 2 C INTERFACE
.sp
.ce 1
Introduction
.sp
.PP
SDS is a library of calls, written in C but with interfaces to
Fortran and C++, whose primary job is to describe arbitrarily structured
data.
As well as describing 
existing collections of data, the SDS interface
provides ways of organising existing data and of requesting allocated memory
for data which only exists as description. 
.sp
.PP
At the moment [and perhaps forever] SDS can describe data structures that
it cannot help generate: in particular data with size and type information
embedded in them which are generated by special purpose hardware readout
systems. 
.sp
.ce 1
Terms
.sp
.PP
An Sds 
.I dataset
is a managed collection of objects and their descriptions. The objects 
may exist purely as descriptions or as arrays of one or more instantiations, and their storage locations may be
different. Datasets exist in two forms: a 'prototyping' stage when descriptions of objects are being added to the dataset: these objects may or may not have memory allocated for them; and an 'assembled' stage when all
memory that should be allocated has been. When assembled, the dataset is
"closed" and objects may not be added or removed, but full access is given to
manipulate the data themselves; in the proto state the data cannot
be modified but the 
.I structure
of the dataset may be.
.sp
.PP
Here, the term
.I object
refers to a 
.I data
object,  which may range from 'primitive' types such as
integer, float and  character to complex data types such as C structures
and structures with embedded size and type information. Examples of 
objects are:
.sp
.nf
      int i;

      int iarray[5];
      
      struct foo {
        int    a[20];
        char   adesc[32];
        double asize;
      } bar[20];

      struct Module {
        int   Type;
        int   DataSize;
        short Data[DataSize];
      } Module[NumberOfModules];
.fi
.sp
This last case shows an array of objects with embedded size information.
Using a simple naming convention described more fully below, SDS can apply
such descriptions to real data and present the user with the size
information and object addresses.
.sp
.PP
Data objects managed by a dataset may be stored with the SDS headers or may
be marked 'disjoint': that is, their storage is not with the headers. In
the latter case, SDS may be informed of the storage location by the
programmer (having retrieved this information from, for instance, a
database) or the knowledge may be explicitly stored with the header so that
loading or attaching to the object may be done automatically.
.sp
.PP
.ce 1
.I SDS Handles
.sp
Sds uses a number of 
.I handles
: that is, opaque values acting
as pointers into system tables which describe each of the components;
they are analagous to file descriptors.
.sp  
.PP
The
.I  sds_type_handle              
.B typeH
allows access to type information containing :
.sp
.IP
- A type code, indicating if the described data is 'primitive' (eg
SDS_INT, SDS_DOUBLE), 'primitive' with special interpretation (eg
SDS_UNIX_TIME, which is in fact SDS_LONG but can be interpreted in a
special fashion), or compound - referring to a structure. The typeH is
the type code for such a compound data type, and accesses a hierarchy of
type codes which finally resolve into primitives.
.sp
- For each type code, a multiplicity.
.br
.sp
- For each type code, a definition name.
.br
.sp
.IP
Each object is associated with a typeH, and subobjects used to describe
compuond objects may also need type handles.
Thus, when Sds describes the C structure:
.sp
.nf
    struct Fred {
      int aint;
      float bfloat[20];
    } Albert[3];
.fi
.sp
the typeH will access the information:
.sp
.IP
      -  one integer, name aint
.br
      -  twenty floats, name bfloat
.sp
.IP
Both of these names are referred to here as 
.I definition
names. The name 'Fred' is also a definition name, but in Sds it is replaced
by the typeH. By contrast, the name 'Albert' is an
.I instantiation
name, which is not part of the type description and is accordingly managed
through the 
.I sds_object_handle 
(see below).
.sp
.IP
To formalise data structures which cannot be easily described by C, in
particular when size and type information is embedded in the data
themselves, naming conventions are used. A structure described by Sds
(in pseudo-code) as :
.sp
.nf
    struct Fred {
      float scale_factor;
      short data_count;
      char data[MULTIPLICITY_ONE];
      int checksum;
    } Albert[MULTIPLICITY_TWO];
.fi
.sp
can be scanned by Sds to pick up the multiplicity of the 'data' arrays in
each element of the 'Albert' array from embedded information. Here, the
strategy is to look for an array called 'data' with undetermined length
after an [integer,short,byte] field called 'data_count' is encountered. Sds
can then scan the described data and, if required, store pointer and
multiplicity arrays with the header for subsequent fast access. Typically,
the total storage size will determine the value of Albert's
multiplicity.
.sp
.PP
The
.I storage_handle           
(storeH) allows access to information detailing where an object or subobject is stored:
.sp
.IP
- The storage name, for example a filename or the name of a shared
memory partition.
.sp
- the name of the host which controls the storage
.sp
- an indicator of the storage type - SDS_FILE, SDS_SHARED_MEM, SDS_DATABASE etc.
.sp
.PP
The
.I sds_dataset_handle
sdsH allows access to dataset information: number of objects, name and timestamp of dataset and object information such as
multiplicities, structures and addresses;
the 
.I sds_dataset_proto
sdsP refers to datasets whose contents - number and type of objects - may
be
changed. Many queries and manipulations may be asked of proto-sds datasets,
but they will not respond to queries about the placing of objects they
manage: this is to provide protection against acquiring
addresses that may change due to Sds internals or copying of objects. Such protection will be
given if decent programming practice is followed but inappropriate use of
pointers whose validity is not guaranteed will cause problems. In general,
the transformation from a proto dataset to an object-accessible dataset
should be followed by calls re-establishing the addresses of any objects
used.
.sp
.PP
The
.I sds_record_handle 
recordH and
.I sds_object_handle 
objectH allow access to object information (alignment, size, name,
multiplicity, instantiation names and indirectly their associated typeH and
storeH).

The
.I sds_object_index
objectI points to the position of an object within a dataset: user objects start
at number 1 [object 0 is the dataset directory, and should not be directly
accessed by the user].
.sp
.ce 1
.I Calls:
.PP
In general, sds calls will return a  long integer; a return of zero indicates an
error or warning which may be investigated with the error handling routines detailed below. All handles are invalid if 0.
.sp
.PP
.I
NOTE:
In what follows, 'Level 1' calls are those of immediate usefullness for
basic use of SDS, whereas Level 2 are of secondary interest but still
intended for the user. Internal calls are not described.
.br
.ce 1
.I General calls:
.sp
.I Level 1
.sp
.PP
int call_succeeded = sds_init();
.IP
Initialise the Sds system tables. Required. Has effect only on the first call.

.PP
.I Level 2
.sp
int call_succeeded = sds_reinit_enable();
.sp
.IP
Where a system may require a second initialisation, for instance a realtime system after partial failure, calling sds_reinit_enable() will allow a subsequent call to sds_init() to take effect. Note that only the first subsequent sds_init() will have effect; after that the normal behaviour returns.
.sp
.PP
int call_succeeded = sds_fprint_def(FILE *stream, objectH);
.sp
.IP
Print the structural definition of an object (ie names, types and structure levels) to the named output stream.
.sp
.PP
int call_succeeded = sds_fprint_object(FILE *stream, objectH);
.sp
.IP 
Print the data contents of an object to the named output stream. An attempt is made to give reasonable format.
.sp
.PP
int version = sds_version();
.sp
.IP
    Returns a floating point version number of the form 2.3

.PP
.ce 1
.I Calls to an sds_dataset:
.sp
.I Level 1
.sp
.PP
sdsH = sds_open(storeH) 
.sp
.IP
Gets access to an existing dataset. You may now query it about
what is there and where it is, but you do not yet have access
to the data....
.sp
.PP
sdsH = sds_attach(storeH, int access_mechanism);
.sp
.IP
Attach to a dataset; ie you are getting access to the actual
stuff, through mechanisms such as local shared memory, mapping
disk, or reflective memory links.
.sp
.PP
sdsP = sds_create(char *name) 
.sp
.IP
Gets a new dataset prototype ready for filling with records/objects. The
name given is the 
.I internal
dataset name, which must be unique for the datasets currently accessed by a
process.
.sp
.PP
objectI = sds_add_oh(sdsP, objectH);
.sp     
.IP
Add an object to a dataset prototype; returns the object_index of the object
within the dataset (Note that an object may be registered in more than
one dataset, in which case their indices are in general different.)
.sp
.PP
int call_succeeded = sds_delete_oh(sdsP, objectH);
.sp
.IP
Delete an object from a dataset prototype.
.sp
.PP
sdsH = sds_make_public(sdsP, storeH);
.sp
.IP
This call creates the Sds described by the input prototype; the dataset is
now in principio accessible by other processes (although system permissions may
regulate this). The call thus implies that some copying will be done: at
least the Sds headers and any objects declared in 
.I process
memory must be copied to the new storage area. Data objects marked 'disjoint' will not be copied. The sdsH returned now allows access to all
data pointers, but further manipulation of the dataset structure is
illegal.
.sp
.PP
sdsH = sds_make_private(sdsP);
.sp
.IP
This call is similar to sds_make_public() in that a transform is made
between a proto dataset whose structure may be changed to one where data
may be manipulated but structure remains constant. In this case however
objects in process memory remain there, and proto objects without
allocated memory will have process storage created.
.sp
.PP
sdsP = sds_protoize(sdsH);
.sp
.IP
Moves a dataset to the prototype state so that the dataset stucture may be
changed. Attempts to obtain object addresses from sdsP or sdsH are now
invalid. It cannot be assumed that objects whose addresses were found from
the generating sdsH before protisation (yuk) will remain in these locations
after further dataset manipulation. 
.sp
.PP
sdsH = sds_duplicate(sdsH, storeH);
.sp
.IP
Duplicates the dataset sdsH to new storage.
.sp
.PP
sdsH = sds_deep_duplicate(sdsH, storeH);
.sp
.IP
Duplicates the dataset sdsH to new storage, including all 'disjoint'
objects.
.sp
.PP
int call_succeeded = sds_discard_all(sdsH);
.sp
.IP
Discard all system descriptive information about a dataset.
.sp
.PP
int call_succeeded = sds_destroy_all(sdsH);
.sp
.IP
Discard all system descriptive information about a dataset and the
objects in it, having first free'd any Sds-system allocated memory
in the objects.
.sp
.PP
.ce 1
.I Creation of type_descriptor:
.sp
.PP
typeH = sds_make_th(struct typelist *tl);
.sp
.ce 1
.I Creation of storage_descriptor:
.sp
.PP
storeH = sds_make_sh(struct storeage *st);
.sp
.IP
.ce 1
.I Calls to an object:
.sp
.PP
objectH = sds_new_object(th,int number,char *names[]);
.br
int call_succeeded =
   sds_oh_instantiations(objectH, storeH[], char *instatiation_names[]);
.br
int call_succeeded =
   sds_oh_struct_storage(objectH, storeH[], char *name = NULL);
.sp
.PP
oh = sds_copy(storeH, objectI, pointer = NULL);
.sp
.IP
Having open'd the dataset, you may now *copy* some or all of
the objects (including architecture conversion if necessary).
Space will be allocated unless 'pointer' has a non-null value, in
which case data will be loaded starting at the pointer position:
caveat programmer.
.sp
.PP
objectH = sds_ohfromindex(sdsH, objectI);
objectH = sds_ohfromname(sdsH, name);
.sp    
.IP
Find the appropriate object handle from the index or name of an object
within a dataset.
.sp
.PP
objectH = sds_ohlikename(sdsH, part_name,int occurrence);
.sp
.IP
Find the occurence'th instance of an object within a dataset whose name
contains 'part_name' as a substring.
.sp
.PP
void *ohptr = sds_ohptr(objectH);
.sp
.IP
Find the pointer to the data acessed through objectH.
.br
Note: this call will give an
.I SDS_FATAL
error if the objectH was found by querying a proto-dataset: default
behaviour is to print out the error stack and stop.
.sp
.PP
char *ohname = sds_ohname(objectH);
.sp
.IP
Find the name of the data acessed through objectH.
.sp
.PP
int multiplicity = sds_ohmult(objectH);
.sp
.IP
Find the multiplicity of the data acessed through objectH - ie how many of
each element - the array size.
.sp
.PP
int sizeof = sds_ohsizeof(objectH);
.sp
.IP
Find the size in bytes of one element of an object.
The size could be undetermined; sds_error is then set to
SDS_SIZE_UNDETERMINED and the error level set to SDS_WARNING;
the size returned is 0.
.sp
.PP
int sizeof = sds_thsizeof(th);
.sp
.IP
Find the size in bytes of the data structure accessed by the type_handle. 
The size could be undetermined; sds_error is then set to
SDS_SIZE_UNDETERMINED and the error level set to SDS_WARNING;
the size returned is 0.
.sp
.PP
int call_succeeded = sds_ohtstamp(int tstamp);
.sp
.IP
Sets the time stamp of an object.
.sp
.PP
int tstamp = sds_get_ohtstamp(objectH);
.sp
.IP
Returns the time stamp of an object.
.sp
.PP
int call_succeeded = sds_discard(objectH);
.sp
.IP
Discard all system descriptive information about an object.
.sp
.PP
int call_succeeded = sds_destroy(objectH);
.sp
.IP
Free any Sds-system allocated memory in the object, then sds_discard()
all system information.
.sp
.PP
.ce 1
.I Object analysis:
.sp
.PP
int descriptor_handle = sds_desc_handle(objectH);
.sp
.IP
Get a handle to refer to subsequent analysis calls on an object.
.sp
.PP
There are two main types of object analysis: linear and hierarchical.
In each case repeated calls to the analysis routines return information
about the analysed objects. In linear analysis,  a sequence of statements
about the primitive elements that make up the object is made - eg 10
floats, 23 int, 120 char, 10 floats, 23 int, 120 char......
This is effected by sequential calls to
.sp
int level = sds_linear(descH, struct level_description **ld.sp);
.sp
In hierarchical elements, a series of index numbers refering to an array
of descriptor structures is returned from
.sp
int level = sds_hierarch(descH, struct level_description **ld);
.sp
each of which give:
.IP
The start address of the field
.br
Its name
.br
Its type_code
.br
The number of elements in the array (or 1).
.br
Its size in bytes.
.sp
.IP
When no more fields remain for analysis, 0 is returned and the warning SDS_DONE_ANALYSIS is set.
.sp
.PP
int call_succeeded = sds_close_dh(descH);
.sp
.IP
Clean up all system stuff created for an analysis. Use of descH before it
is reallocated with sds_desc_handle() will cause an error; however
sds_close_dh() is called automatically when an analysis sequence is run
completion. Re-closing an already closed descH has no effect.
.sp
.PP
int level = sds_find_level(descH, field_name, struct level_description **ld);
.sp
.IP
Makes calls to sds_hierarch() until a match is made to the field named
field_name.
.sp
.ce 1
.I Creation of record_handle:
.sp
recordH = sds_begin_record(char *name);
.sp
sds_record_entry(recordH, handle, int number, void *ptr, char *name);
.sp
sds_begin_sub_record(rh, name)
.sp
sds_end_sub_record(rh)
.sp
sds_end_and_declare(recordH, sdsH);
.sp
sds_destroy_record_def(recordH, int destroy_object);
.sp
sds_print_record_def(recordH);

.ce 1
.I Error handling.
.sp
.I Level 1
.sp
.PP
void sds_stop_if_error(char *comment);
.sp
.IP
Really crude. If there has been an Sds error, it prints out the error stack
and does exit(1); otherwise it is quiet.
.sp
.PP
int level = sds_push_error(int errcode, int errlevel, char *errstring);
.sp
.IP
Push a new error onto the stack. We assume that errstring is zero
terminated, and check to see if it is NULL. Returns the 'level' of
the error in the stack. Filename and line information is added by the
preprocessor from the defines __LINE__ and __FILE__.
.sp
.PP
  int error_code = sds_last_error();
.sp    
.IP
Returns the last sds_error, which is 0 if no problems found.
.sp
.PP
  int error_code = sds_last_warning();
.sp    
.IP
Returns the last sds_error at SDS_WARNING level, which is 0 if no problems found.
.sp
.PP
  void sds_perror(char *comment);
.sp  
.IP
Print out the top level comment, any filled parts of the error stack,
and the last operating system error if there was one.
.sp
.PP
.sp
.I Level 2:
.sp
.PP
void sds_output_proginfo(int truefalse);
.sp
.IP
Default 0 == Do not print out line and file information.
.sp
.PP
void sds_exit_on_fatal(int truefalse);
.sp
.IP
Default 1 == An SDS_FATAL error causes exit(truefalse & 0xff);
.sp
.PP
void sds_output_errors(int level);
.sp
.IP
Default 0 == Do not output errors as they are registered; this is when
you want to wait for a final catastrophe and then use sds_perror(), or
when you may be able to take corrective action; in most programs
sds_output_errors(SDS_FATAL) will be appropriate.
.sp
.PP
sds_clear_errors();
.sp
.IP
Start with a clean slate.
.sp
.PP
int error_code = sds_pop_error(int errlevel, char *errstring);
.sp
.PP
void printoutprog(int i);
.sp
.IP
Print out program info for the i'th entry on the stack 
.sp
.PP
void printout(int i);
.sp
.IP
Print out the i'th entry on the stack 
.sp
.IP
Currently, error levels are:
.sp
SDS_WARNING
.br
SDS_ERROR
.br
SDS_FATAL
.br
