You can Observe a lot by just watching
The following Observers are available:
- Observer : abstract class;
- ErrorMinima : monitors an error variable to keep track of its minima.
- EarlyStopper : upon reaching minima, saves Experiment and notifies Subscribers;
- AdaptiveDecay : decays learning rate when new minima on validation error isn't found;
- Logger : abstract logging class (prints reports to cmdline);
- FileLogger : basic file-based report logging;
- CompositeObserver : a composite of observers;
Observer
An object that is called when events occur. Based on the Listen-Notify design pattern. Uses a mediator to publish/subscribe to channels. Observers cannot publish reports (for now). The reason for this is that a report is required of observers to do their job, thus making it impossible for them to participate to report generation. The only possibility would be to allow for Observers to modify the received report, but then the ordering of these modifications would be undefined, unless they make use of Mediator priorities.
dp.Observer(channels, callbacks)
Constructs an Observer. Arguments channels
and callbacks
can
be strings or tables thereof. The first argument specifies the name of
a Channel to which the corresponding callback
method of the Observer
will be registered via a Subscriber through the
Mediator.
If no callbacks
are provided, these default to taking on the same values
as channels
.
ErrorMinima
Monitors an error variable to keep track of its minima. Optionally notifies
Subscribers to the errorMinima
of the current state
of the error minima.
The monitored error variable can be obtained from an experiment
report or a Mediator Channel.
In the case of the report, the variable must be
located at a leaf of the report tree specified by a sequence of keys (branches) : error_report
.
In this case, the object subscribes to doneEpoch
channel in order to receive the reports
at the end of each epoch.
dp.ErrorMinima{...}
Constructs an ErrorMinima Observer. Arguments should be specified as key-value pairs. Other then the following arguments, those specified in Observer also apply.
start_epoch
is a number identifying the minimum epoch after which Experiments can begin to be persisted to disk. Useful to reduce I/O at beginning of training. Defaults to 5;error_report
is a table specifying a sequence of keys to access variable from epoch reports. Default is{'validator', 'loss', 'avgError'}
(for cross-validation), unless of course anerror_channel
is specified.error_channel
is a string or table specifying the Channel to subscribe to for early stopping. Should return an error value and the last experiment report. Over-rideserror_report
when specified.maximize
is a boolean having a default value of false. When true, the error channel or report is negated. This is useful when the channel returns a measure like accuracy that should be maximized, instead of an error that should be minimized.notify
is a boolean. When true (the default), notifies listeners (Subscribers) when a new minima is found.
EarlyStopper
An ErrorMinima instance that saves the version of
the subject
with the lowest error and terminates
the experiment when no new minima is found for max_epochs
.
Should only be called on Experiment, Propagator or Model subjects.
Error can be obtained from experiment report or mediator Channel.
If obtained from experiment report via error_report, subscribes to doneEpoch channel.
dp.EarlyStopper{...}
Constructs an EarlyStopper Observer. Arguments should be specified as key-value pairs. Other then the following arguments, those specified in ErrorMinima also apply.
save_strategy
is an object with asave(subject)
method for persisting the subject to disk. Defaults todp.SaveToFile()
max_epochs
specifies the maximum number of epochs to consider after a minima has been found. After that, a terminate signal is published to the mediator. Defaults to 30.max_error
is the maximum value for which the experiment should be stopped aftermin_epoch
epochs. Ifmaximize=true
this is minimum value. A value of 0 (the default) means that it is ignored.min_epoch
is a number having a default value of 1000000. Seemax_value
for a description.
AdaptiveDecay
An Observer that decays learning rate by decay_factor
when validation error doesn't reach a new
minima for max_wait
epochs. This object should observe in conjuction with an
ErrorMinima instance, such as EarlyStopper.
As example, suppose the object is initialized with decay_factor=0.5
and max_wait=1
, while the subject is initialized with learning_rate=10
.
If the sequence of errors is
10, 9, 8, 8, 8, 8, 9, 7, 7, 8
then the corresponding sequence of learning rates given these errors would be
10, 10, 10, 10, 1, 1, 0.1, 0.1, 0.1, 0.01
dp.AdaptiveDecay{...}
Constructs an AdaptiveDecay Observer. Arguments should be specified as key-value pairs. Other then the following arguments, those specified in Observer also apply.
max_wait
specifies the maximum number of epochs to wait for a new minima to be found. After that, the learning rate is decayed bydecay_factor
. Defaults to 2.decay_factor
specifies the factor by which learning ratelr
is decayed as per :lr = lr*decay_factor
.