<HEAD>
<TITLE>OINK Users Manual</TITLE>
<META NAME="docnumber" CONTENT="OINK and MapReduce-MPI Library">
<META NAME="author" CONTENT="http://www.sandia.gov/~sjplimp/mapreduce.html - Sandia National Laboratories">
<META NAME="copyright" CONTENT="Copyright (2009) Sandia Corporation.
This software and manual is distributed under the BSD License.">
</HEAD>

<BODY>

"MR-MPI WWW Site"_mws -"MR-MPI Documentation"_md - "OINK
Documentation"_od - "OINK Commands"_oc :c

:link(mws,http://www.sandia.gov/~sjplimp/mapreduce.html)
:link(md,../doc/Manual.html)
:link(od,Manual.html)
:link(oc,Section_script.html#comm)

:line

<H1></H1>

OINK Documentation :c,h3

Version info: :h4

The OINK "version" is the date when it was released, such as 1 Feb
2011.  OINK and MR-MPI library are updated continuously.  Whenever we
fix a bug or add a feature, we release it immediately, and post a
notice on "this page of the WWW site"_bug.  Each dated copy of OINK
contains all the features and bug-fixes up to and including that
version date. The version date is printed to the screen and log file
every time you run OINK. It is also in the file oink/version.h and in
the MR-MPI directory name created when you unpack a tarball.

If you browse the HTML or PDF doc pages for OINK on the MR-MPI WWW
site, they always describe the most current version of OINK. :ulb,l

If you browse the HTML or PDF doc pages for OINK included in your
tarball, they describe the version you have. :ule,l

OINK is a simple scripting wrapper around the "MapReduce-MPI"
library"_mws, and also provides a easy-to-use development framework
for writing new MapReduce algorithms and codes.  Like the MR-MPI
library, OINK will run on any platform (serial or parallel) that
supports "MPI"_mpi.  Note the MR-MPI library has its own "manual and
doc pages"_md.

The name OINK is meant to evoke the aroma of the "Apache Pig"_pig
platform which wraps the "Hadoop"_hadoop MapReduce capabilities with
its high-level Pig Latin language.  Since OINK has only a small
fraction of Pig's capability, it is more the sound of a pig, than the
pig itself.

Source code for OINK and the MR-MPI library were developed at Sandia
National Laboratories, a US Department of Energy facility.  They are
freely available for download from the "MR-MPI web site"_mrmpi and are
distributed under the terms of the modified "Berkeley Software
Distribution (BSD) License"_bsd.  This basically means they can be
used by anyone for any purpose.  See the LICENSE file provided with
the distribution for more details.

The authors of OINK and the MR-MPI library are "Steve Plimpton"_sjp at
and "Karen Devine"_kdd who can be contacted via email: sjplimp,kddevin
at sandia.gov.

:link(bsd,http://en.wikipedia.org/wiki/BSD_license)
:link(kdd,http://www.cs.sandia.gov/~kddevin)
:link(sjp,http://www.cs.sandia.gov/~sjplimp)
:link(mrmpi,http://www.cs.sandia.gov/~sjplimp/mapreduce.html)
:link(bug,http://www.sandia.gov/~sjplimp/mapreduce/bug.html)
:link(pig,http://pig.apache.org)
:link(hadoop,http://hadoop.apache.org)

:line
 
Goals of OINK :h4

(1) To allow MapReduce algorithms which call the MR-MPI library to be
written with a minimum of extraneous code, to work with input/output
in various forms, and to be chained together and driven via a simple,
yet versatile scripting language. :ulb,l

(2) To create an archive of map() and reduce() functions for re-use by
different algorithms. :l

(3) To provide a scripted interface to the lo-level MR-MPI library
calls that can speed development/debugging of new algortihms before
coding them up in C++ or another language. :l,ule

We think the first two goals are largely met.  See the section on
"Adding Commands to OINK"_Section_commands.html and the "named
command"_command.html, "input"_input.html, and "output"_output.html
doc pages for details of the first goal.  See the section on "Adding
Functions to OINK"_Section_functions.txt for details of the second
goal.

The third goal, however, is only partially met.  See the "MR-MPI
library commands"_mrmpi.html doc page for its current status.  The
sticking point here is that in a real programming language you can
pass a pointer to an arbitrary data structure to your map() or
reduce() functions, but it is hard to do that from a scripting
language using text input without re-inventing something like
"Python"_python.

:link(python,http://www.python.org)

:line

Contents of OINK Manual :c,h4

OINK aims to be a simple scripting interface and development
environment and the lightweight documentation reflects that.

Once you are familiar with OINK, you may want to bookmark "this
page"_Section_script.html#comm at Section_script.html#comm since it
gives quick access to documentation for all OINK commands.

"PDF file"_Manual.pdf of the entire manual, generated by
"htmldoc"_http://www.easysw.com/htmldoc

"Building OINK"_Section_build.html :l
  1.1 "Making OINK"_1_1 :ulb,b
  1.2 "Building OINK as a library"_1_2 :b
  1.3 "Running OINK"_1_3 :b
  1.4 "Command-line options"_1_4 :ule,b
"OINK Scripts"_Section_script.html :l
  2.1 "Input script operation"_2_1 :ulb,b
  2.2 "Parsing rules"_2_2 :b
  2.3 "Input script commands"_2_3 :ule,b
"Adding Functions to OINK"_Section_functions.html :l
  3.1 "Map() functions"_3_1 :ulb,b
  3.2 "Reduce() functions"_3_2 :b
  3.3 "Compare() functions"_3_3 :b
  3.4 "Hash() functions"_3_4 :b
  3.5 "Scan() functions"_3_5 :ule,b
"Adding Commands to OINK"_Section_commands.html :l
  4.1 "Source files for the new class"_4_1 :ulb,b
  4.2 "Methods in the new class"_4_2 :b
  4.3 "Calls to the OINK object manager"_4_3 :b
  4.4 "Calling back to map() and reduce() functions"_4_4 :ule,b
"Errors"_Section_errors.html :l
  5.1 "Error & warning messages"_5_1 :ulb,b,ule
:ole

:link(1_1,Section_build.html#1_1)
:link(1_2,Section_build.html#1_2)
:link(1_3,Section_build.html#1_3)
:link(1_4,Section_build.html#1_4)

:link(2_1,Section_script.html#2_1)
:link(2_2,Section_script.html#2_2)
:link(2_3,Section_script.html#2_3)

:link(3_1,Section_functions.html#3_1)
:link(3_2,Section_functions.html#3_2)
:link(3_3,Section_functions.html#3_3)
:link(3_4,Section_functions.html#3_4)
:link(3_5,Section_functions.html#3_5)

:link(4_1,Section_commands.html#4_1)
:link(4_2,Section_commands.html#4_2)
:link(4_3,Section_commands.html#4_3)
:link(4_4,Section_commands.html#4_4)

:link(5_1,Section_errors.html#5_1)

</BODY>
