Adaptive Model Rules Regressor

Adaptive Model Rules Regressor

Adaptive Model Rules (AMRules) is an innovative algorithm for learning regression rules with streaming data. In AMRules, the rule model consists of a set of normal rules and a default rule (a rule with no features). Hoeffding bound is used to define a confidence interval to decide whether to expand a rule. If the ratio of the 2 largest standard deviation reduction (SDR) measure among all potential features of a rule is is within this interval, the feature with the largest SDR will be added to the rule to expand it. If the default rule is expanded, it will become a normal rule and will be added to the model's rule set. A new default rule is initialized to replace the expanded one. A rule in the set might also be removed if the Page-Hinckley test indicates that its cumulative error exceed a threshold.

Vertical Adaptive Model Rules Regressor

Vertical Adaptive Model Rules Regressor (VAMR) is the vertical parallel implementation of AMRules in SAMOA. The diagram below shows the components of the implementation.

The Source PI and Evaluator PI are components of the Prequential Evaluation task. The Source PI produces the incoming instances while Evaluator PI reads prediction results from VAMR and reports their accuracy and throughput.

The core of VAMR implementation consists of one Model Aggregator PI and multiple Learner PIs. Each Learner PI is responsible for training a subset of rules. The Model Aggregator PI manages the rule model (rule set and default rule) to compute the prediction results for incoming instances. It is also responsible for the training the default rule and creation of new rules.

For each incoming instance from Source PI, Model Aggregator PI appies the current rule set to compute the prediction. The instance is also forwarded from Model Aggregator PI to the Learner PI(s) to train those rules that cover this instance. If an instance is not covered by any rule in the set, the default rule will be used for prediction and will also be trained with this instance. When the default rule expands and create a new rule, the new rule will be sent from Model aggregator PI to one of the Learner PIs. When the Learner PIs expand or remove a rule, an update message is also sent back to the Model Aggregator PI.

The number of Learner PIs can be set with the -p option:

PrequentialEvaluationTask -l (org.apache.samoa.learners.classifiers.rules.VerticalAMRulesRegressor -p 4)

Horizontal Adaptive Model Rules Regressor

Horizontal Adaptive Model Rules Regressor (HAMR) is an extended implementation of VAMR. The components of a [[Prequential Evaluation task|Prequential Evaluation Task]] with HAMR are shown in the diagram below.

In HAMR, the Model Aggregator PI is replicated, each processes only a partition of the incoming stream from Source PI. The default rule is moved from the Model Aggregator PI to a special Learner PI, called Default Rule Learner PI. This new PI is reposible for both the training and predicting steps for default rule.

For each incoming instance from Source PI, Model Aggregator PIs apply the current rule set to compute the prediction. If the instance is covered by a rule in the set, its prediction is computed by the Model Aggregator PI and, then, it is forwarded to the Learner PI(s) for training. Otherwise, the instance is forwarded to Default Rule Learner PI for both prediction and training.

Newly created rules are sent from Default Rule Learner PI to all Model Aggregator PIs and one of the Learner PIs. Update messages are also sent from Learner PIs to all Model Aggregator PIs when a rule is expanded or removed.

The number of Learner PIs can be set with the -p option and the number of Model Aggregator PIs can be set with the -r option:

PrequentialEvaluationTask -l (org.apache.samoa.learners.classifiers.rules.HorizontalAMRulesRegressor -r 4 -p 2)