Inherits from: object

A class to keep track of all the options. Options can be accessed with
the getopt() method, but standard options are also accessable directly
through the following instance variables: source, dct, pipeline, pipe,
loglevel, trap_errors, import_event_tags, perl, mallet, treetagger,
classifier, ee_model and et_model. There is no instance variable access for
user-defined options in the config.txt file.

Public Functions

__getitem__(self, key)

__init__(self, options)
Initialize options from the config file and the options handed in to
the tarsqi script. Put known options in instance variables.

__str__(self)

getopt(self, option_name, default=None)
Return the option, use None as default.

items(self)
Simplistic way to do dictionary emulation.

pp(self)

set_option(self, opt, value)
Sets the value of opt in self._options to value. If opt is also
expressed as an instance variable then change that one as well.

set_source_format(self, value)
Sets the source value, both in the dictionary and the instance
variable.

Private Functions

_initialize_options(self, command_line_options)
Reads options from the config file and the command line. Also loops
through the options dictionary and replaces some of the strings with
other objects: (1) replaces 'True', 'False' and 'None', with True, False
and None respectively, (2) replaces strings indicating an integer with
that integer (but not for the dct), (3) replaces the empty string with
True for the --pipe and --import-events options, and (4) replaces the
value of the --mallet and --treetagger options, which are known to be
paths, with the absolute path.

_initialize_properties(self)
Put options in instance variables for convenience. This is done for
those options that are defined for the command line and not for options
from config.txt that are user-specific. Note that due to naming rules
for attributes (no dashes allowed), options with a dash are spelled with
an underscore when they are instance variables.

Inherits from: object

Main Tarsqi class that drives all processing.

Instance variables:

   input                -  absolute path
   output               -  absolute path
   basename             -  basename of input file
   options              -  an instance of Options with processing options
   tarsqidoc            -  an instance of TarsqiDocument
   source_parser        -  a source-specific parser for the source
   metadata_parser      -  a source-specific metadata parser
   docstructure_parser  -  a document structure parser
   pipeline             -  list of name-wrapper pairs
   components           -  dictionary of Tarsqi components
   document             -  instance of TarsqiDocument
   tmp_data             -  path to directory for temporary files

The first nine instance variables are initialized using the arguments
provided by the user, the document variable is initialized and changed
during processing.

Public Functions

__init__(self, opts, infile, outfile)
Initialize Tarsqi object conform the data source identifier and the
processing options. Does not set the instance variables related to the
document model and the meta data. The opts argument has a list of
command line options and the infile and outfile arguments are typically
absolute paths, but they can be None when we are processing strings.

process_document(self)
Parse the source with the source parser, the metadata parser and the
document structure parser, apply all components and write the results to
a file. The actual processing itself is driven using the processing
options set at initialization. Components are given the TarsqiDocument
and update it.

process_string(self, input_string)
Similar to process(), except that it runs on an input string and not
on a file, it does not write the output to a file and it returns the
TarsqiDocument.

Private Functions

_apply_component(self, name, wrapper, tarsqidocument)
Apply a component by taking the TarsqDocument, which includes the
options from the Tarsqi instance, and passing it to the component
wrapper. Component-level errors are trapped here if --trap-errors is
True. If errors are trapped, it is still possible that partial results
were written to the TagRepositories in the TarsqiDocument.

_cleanup_directories(self)
Remove all fragments from the temporary data directory.

_create_pipeline(self)
Return the pipeline as a list of pairs with the component name and
wrapper.

_initialize_parsers(self)

_update_processing_history(self)

_write_output(self)
Write the TarsqiDocument to the output file.

Inherits from: object

Class that wraps the Tarsqi class, taking care of some of the IO aspects.

Public Functions

__init__(self, args)

pp(self)

run(self)
Main method that is called when the script is executed from the command
line. It creates a Tarsqi instance and lets it process the input. If the
input is a directory, this method will iterate over the contents, setting up
Tarsqi instances for all files in the directory. The arguments are the list
of arguments given by the user on the command line.

Private Functions

_run_tarsqi_on_directory(self)
Run Tarsqi on all files in a directory.

_run_tarsqi_on_file(self)

_run_tarsqi_on_pipe(self)
Read text from standard input and run tarsqi over it, then print the result
to standard out.

load_ttk_document(fname, loglevel=2, trap_errors=False)
Load a TTK document with all its Tarsqi tags and return the Tarsqi instance
and the TarsqiDocument instance. Do not run the pipeline, but run the source
parser, metadata parser and the document structure parser. Used by the
evaluation code.

process_string(text, pipeline='PREPROCESSOR', loglevel=2, trap_errors=False)
Run tarsqi on a bare string without any XML tags, handing in pipeline,
loglevel and error trapping options.

run_profiler(args)
Wrap running Tarsqi in the profiler.