public class AlgorithmSpecificDatasetExample
extends java.lang.Object

Example that shows how to implement your own algorithm (gradient descent trainer for linear regression) which uses dataset as an underlying infrastructure.

Code in this example launches Ignite grid and fills the cache with simple test data.

After that it creates an algorithm specific dataset to perform linear regression as described in more detail below.

Finally, this example trains linear regression model using gradient descent and outputs the result.

You can change the test data used in this example and re-run it to explore this functionality further.

The common idea behind using algorithm specific datasets is to write a simple local version algorithm at first, then find operations which involve data manipulations, and finally define algorithm specific version of the dataset extended by introducing these new operations. As a result your algorithm will work with extended dataset (based on DatasetWrapper) in a sequential manner.

In this example we need to implement gradient descent. This is iterative method that involves calculation of gradient on every step. In according with the common idea we define AlgorithmSpecificDatasetExample.AlgorithmSpecificDataset - extended version of Dataset with gradient method. As a result our gradient descent method looks like a simple loop where every iteration includes call of the gradient method. In the example we want to keep iteration number as well for logging. Iteration number cannot be recovered from the upstream data and we need to keep it in the custom partition context which is represented by AlgorithmSpecificDatasetExample.AlgorithmSpecificPartitionContext class.