[SINGA-126] Improve Python Binding for interactive training

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Implemented
• Labels: binding debugging interative python
• Assignee: Lee Chonho
• Reporter: wangwei
• Created: 09/Jan/16 03:24
• Updated: 28/Jun/16 03:55
• Resolved: 28/Jun/16 03:55

Description

Currently, python APIs only configure the layer and model. All objects are created after the the JobProto is passed to Driver. Hence, users cannot query the layer object returned by

conv1 = Convolution2D()

to get its internal data (e.g, feature and param values). These internal data is useful for debugging.

To support this feature, we need to create the SINGA::Layer object and store it in conv1.
Users can write their own BP algorithm like this,

data = numpy.loadtxt("csv.txt")
x, y = data[:, 1:], data[:, 0]
input = Dummy() // dummy layer to get input data
label = Dummy() // dummy layer to get label 
conv = Convolution2D(...)
pool = Pool2D()
inner = Dense()
loss = ...

for i in range(x.shape[0] / batchsize):
   xb, yb = ...
   input.SetData(x)
   label.SetData(y)
   conv.ComputeFeature(input)
   pool.ComputeFeature(conv)
   inner.ComputeFeature(pool)
   loss.ComputeGradient(inner, label)
   ....

In this way, users know exactly how the training is conducted, and can access the internal data of each layer directly, e.g., conv.data(), conv.GetParams().
We may also learn from chainer to call the ComputeGradient functions automatically for the backward pass.

This feature requires the python APIs for singa::Layer.
It is easy for training with a single worker. For multiple workers, we need to think more.