This directory contains the code for the native client register based
(R15) solution for x86-64 validation. R15 is used to convert 32-bit
addresses to the corresponding 64-bit address for memory accesses.

Note: Although the current builds generate 32-bit versions, they do
not make any sense, since pointers are 32-bits in x86-32.


nc_cpu_checks.{c,h}

        Defines code that checks if the CPUID allows each
        instruction. If not, it replaces the instruction with HALTs
        during validation.

nc_illegal.{c,h}

        Defines code that checks each instruction to see if it meets
        basic requirements for native client. This includes the
        following checks:

        - More than one (non-REX) prefix byte is not specified (we do
          allow lock (f0) and DATA 16 (66) prefixes if the instruction
          has been specified to accept both).

        - The instruction has not been marked as illegal (see "Modeled
          Instructions" below).

        - The prefix/opcode sequence of the instruction has not been
          marked as invalid (see "Modeled Instructions" below).

        - The is not marked as a system instruction (see "Modeled
          Instructions" below).

        - Segment prefixes (except those used for jump prediction on
          conditional jumps) are not allowed.

        - The ADDR 16 (67) prefix has been used for the instruction,
          but native client doesn't allow it (see "Modeled
          Instructions" below).

        - No more than one REX prefix has been specified for the
          instruction.

        - The REP (f3) prefix is not allowed for the instruction (see
          "Modeled Instructions" below).

        - The REPNE (f2) prefix is not allowed for the instruction
          (see "Modeled Instructions" below).

        - The DATA 16 (66) prefix is not allowed for the instruction
          (see "Modeled Instructions" below).

        - A prefix byte was not duplicated (we allow 66 to be
          duplicated for special cases of the NOP instructions that
          are commonly generated by compilers, see "Modeled
          Instructions" for more information on these hard-coded
          cases).

        - Multiple, ambigious segment prefixes are not allowed for an
          instruction.

        - The REX prefix does not appear last in the sequence of
          prefixes.

nc_jumps.{c,h}

        Defines code to check jump instructions. In general, explicit
        jumps must jump to an instruction in the same segment if it is
        explicitly specified using a constant. All other jumps should
        be indirect and use a pre-approved pattern that adds R15 to a
        corresponding 32-bit address.

nc_memory_protect.{c,h}

        Checks that memory references use an appropriate pattern that
        converts a 32-bit data address (using R15) to a corresponding
        64-bit memory reference.

nc_opcode_histogram.{c,h}

        Simple histogram tracker of the frequency that each opcode is
        used.  Not needed to validate but provides useful test
        information.

nc_protect_base.{c,h}

        Checks that the instruction doesn't improperly modify one of
        the base registers RSP, RBP, and R15. Also checks LEA
        instructions to see if they calculate an address of the form

            [%reg+R15*1]

        and put the results into an appropriate 32-bit register.

ncvalidate_iter.c

        Defines an iterator that iterates over the instructions in a
        memory segment. Keeps track of the previous 2 instructions
        during this iteration so that one can check multiple
        instruction patterns. It also define

ncvalidate_iter.h

        Defines the API to the iterator defined in ncvalidate_iter.c.

ncvalidate_iter_internal.h

        Defines internal methods and data structures used to implement
        the iterator defined in ncvalidate_iter.c

ncvalidate_iter_verbose.c

        Defines a verbose error reporter that works for the
        instructions iterated over using the API methods in
        ncvalidate_iter.h

ncvalidate_utils.{c,h}

        Defines various common checking routines for instructions.

ncvalidate_registry.{c,h}

        Defines a registry of validator checking methods, that get
        applied to validate instruction.

Modeled Instructions
--------------------

Textual version(s) of the instructions understood by the validator can
be found in the following files.

   native_client/src/trusted/validator_x86/testdata/64/modeled_insts.txt

      Defines the set of instructions understood by the (full)
      decoder. This is the most human readable form of the generated
      decoder tables used by the validator.

   native_client/src/trusted/validator_x86/testdata/64/ncval_reg_sfi_modeled_insts.txt

      Defines the set of (partial) isntructions understood by the
      validator decoder. This file is the same as modeled_insts.txt
      above except that it shows how the modeled instructions have
      been simplified, based on the needs of the validator in this
      directory. In particular, most instruction mnemonics are
      simplified to a "don't care" name since the mnemonic name is not
      needed by the validator. In addition, most implicit arguments of
      the instruction are made explicit, and arguments that can't be
      used to compute memory addresses have been removed.

For more information on the contents of these files, see

   native_client/src/trusted/validator/x86/decoder/README

Pre/Post condition testing
--------------------------

The code in this directory is set up to allow instructions to be
validated individually.  When run in this mode, pre/post conditions
are generated rather than looking accross instructions to verify the
conditions. This is intentional, and is intended for testing multiple
validators. In such contexts, the pre/post conditions define what must
be checked by a compatible validator.

To generate these condition tests, one must compile the source using
the scons command:

    .> ./scons --mode=opt-linux platform=x86-64 ncval_testing=1

Note: --mode=opt-linux is not necessary. Any applicable mode will
work.

When native_client/src/trusted/validator_x86/ncval.c is compiled with
the above scons command, the corresponding executable will generate
pre/post conditions.

Code to generate pre/post conditions is conditionalized in the source
(under '#ifdef NCVAL_TESTING') so that it is only added when scons
option ncval_testing=1 is defined.

The default error reportor is used by the validator. This is done so
that only pre/post conditions will be printed.  Hence, if you are to
add debugging code within NCVAL_TESTING, be sure to not redirect the
output to the validator's error reporter. Rather, redirect using calls
to NaClLog or printf. To override this presumption and see other
debugging messages, change the line

        #define DEBUGGING 0

in native_client/src/trusted/validator_x86/ncval.c to be

        #define DEBUGGING 1

This will cause the validator to use the verbose error reporter within
ncval.

To run unit tests for the pre/post conditions, run the following scons
build command:

    .> ./scons --mode=opt-linux platform=x86-64 ncval_testing=1 \
         ncval_testing

Note: To see how to test scons builds with ncval_testing=1, see
native_client/src/trusted/validator_x86/build.scons and look for the
text 'test_env.Bit('ncval_testing')'.

Debugging
---------

Many of the source files contain #define DEBUGGING flags. When
DEBUGGING is set to 1, additional debugging print messages are
compiled into the code. Unfortunately, by default, these message
frequently call routines that are not compiled into corresponding
executables (such as ncval and ncdis). To add the additional routines,
edit file

   native_client/site_scons/site_tools/library_deps.py

For x86-32, edit lines

            # When turning on the DEBUGGING flag in the x86-32 validator
            # or decoder, add the following:
            #'nc_opcode_modeling_verbose_x86_32',

to

            # When turning on the DEBUGGING flag in the x86-32 validator
            # or decoder, add the following:
            'nc_opcode_modeling_verbose_x86_32',

For x86-64, edit lines

            # When turning on the DEBUGGING flag in the x86-64 validator
            # or decoder, add the following:
            # 'nc_opcode_modeling_verbose_x86_64',

to

            # When turning on the DEBUGGING flag in the x86-64 validator
            # or decoder, add the following:
            'nc_opcode_modeling_verbose_x86_64',

These changes will make sure that the corresponding print routines are
added to the executables during link time.
