Architecture

Tribuo is a library for creating Machine Learning (ML) models and for using those models to make predictions on previously unseen data.

A ML model is the result of applying some training algorithm to a dataset. Most commonly, such algorithms produce output in the form of a large number of floating point values; however, this output may take one of many different forms, such as a tree-structured if/else statement. In Tribuo, a model includes not only this output, but also the necessary feature and output statistics to map from the named feature space into Tribuo’s ids, and from Tribuo’s output ids into the named output space.

A Tribuo Model can also be thought of as a learned mapping from a sparse feature space of doubles to a dense output space (e.g., of class label probabilities, or regressed outputs etc). Every dimension of the input and output are named. This naming system makes it possible to check that the input and model agree on the feature space they are using.

Data flow overview

Tribuo loads data using a DataSource implementation, which might load from a location like a DB or a file on disk. This DataSource processes the input data, converting it into Tribuo’s storage format, an Example. An Example is a tuple of an Output (i.e., what you want to predict) and a list of Features, where each Feature is a tuple of a String feature name and a double feature value. The DataSource is then read into a Dataset, which accumulates statistics about the data for future use in model construction. Datasets can be split into chunks to separate out training and testing data, or to filter out examples according to some criterion. As Examples are fed into a Dataset, the Features are observed and have their statistics recorded in a FeatureMap. Similarly the Outputs are recorded in the appropriate OutputInfo subclass for the specified Output subclass. Once the Dataset has been processed, it’s passed to a Trainer, which contains the training algorithm along with any necessary parameter values (in ML these are called hyperparameters to differentiate them from the learned model parameters), and the Trainer performs some iterations of the training algorithm before producing the Model. A Model contains the necessary learned parameters to make predictions along with a Provenance object which records how the Model was constructed (e.g., data file name, data hash, trainer hyperparameters, time stamp, etc). Both Models and Datasets can be serialized out to disk using Java Serialization. Once a model has been trained, it can be fed previously unseen Examples to produce Predictions of their Outputs. If the new Examples have known Outputs, then the Predictions can be passed to an Evaluator, which calculates statistics like the accuracy (i.e., the number of times the predicted output was the same as the provided output).

Structure

Tribuo includes several top level modules:

  • Core provides Tribuo’s core classes and interfaces.
  • Data provides loaders for text, sql and csv data, along with the columnar package which provides infrastructure for working with columnar data.
  • Math provides Tribuo’s linear algebra library, along with kernels and gradient optimizers.
  • JSON provides a JSON data loader and a tool to strip provenance from trained models.

Tribuo has separate modules for each prediction task:

  • Classification contains an Output implementation called Label, which represents a multi-class classification.
  • Regression contains an Output implementation called Regressor, which represents multidimensional regression.
  • AnomalyDetection contains an Output implementation called Event, which represents the detection of an anomalous or expected event.
  • Clustering contains an Output implementation called ClusterID, which represents the cluster id number assigned.
  • MultiLabel contains an Output implementation called MultiLabel, which represents a multi-label classification.

Finally, there are cross-cutting module collections:

  • Common provides shared infrastructure for the prediction tasks.
  • Interop provides infrastructure for working with large external libraries like TensorFlow and ONNX Runtime.
  • Util provides independent libraries that Tribuo uses for specific tasks. For example, InformationTheory is a library of information theoretic functions, and Tokens provides the interface Tribuo uses for tokenization along with implementations of several tokenizers.

Configuration, Options and Provenance

Many of Tribuo’s trainers, datasources and other classes implement the Configurable interface. The configuration system is integrated into the command line arguments Options system build into OLCUT’s ConfigurationManager. Values in configuration files can be overridden on the command line. The configuration system provides the basis of Tribuo’s model tracking Provenance system, which records all hyperparameters, dataset parameters (e.g., file location, train/test split, etc.), and any user-supplied instance information.

The LinearSGDTrainer class above is configured by the xml snippet below:

<config>
   <component name="logistic" type="org.tribuo.classification.sgd.linear.LinearSGDTrainer">
        <property name="objective" value="log"/>
        <property name="optimiser" value="adam"/>
        <property name="epochs" value="10"/>
        <property name="loggingInterval" value="100"/>
        <property name="minibatchSize" value="1"/>
        <property name="seed" value="1"/>
    </component>
</config>

Data Loading

Built-in formats

Tribuo supports several common input formats for loading in data:

  • libsvm/svmlight - a sparse numerical format for classification and regression tasks.
  • IDX - a dense multidimensional numerical format for classification and regression.
  • CSV - a plain text delimited format (using an RFC4180 compliant parser).
  • JSON - JavaScript Object Notation.
  • SQL - Tribuo has a JDBC loader, which can query a database and convert the result set into Tribuo Examples.
  • text - a one document per line format.

The RowProcessor is a configurable mechanism for converting a ColumnarIterator.Row into an Example. The RowProcessor uses four interfaces to process the input map:

  • FieldExtractor
  • FieldProcessor
  • FeatureProcessor
  • ResponseProcessor

These interfaces are supplied to the RowProcessor on construction. If your columnar data is not in a format currently supported by Tribuo, you can subclass ColumnarDataSource.

Transforming datasets

Tribuo supports independent, feature-based transformations including the rescaling or binning of features. Transformed features can be chained to create pipelines that are applied in supplied sequence to specified feature(s). Local transformation pipelines can apply to features which match a regex.

Weights and Metadata

Examples can have metadata attached to them in the form of a Map<String,String>. In addition, each Example has a float-valued weight field, which can denote their importance.

Obfuscation

One of Tribuo’s benefits is its extensive tracking of model metadata; however, Tribuo provides transformation mechanisms to remove metadata from trained models. Provenance can be removed from Model objects using the StripProvenance program.