Updating MultiTask example to use new infer api and adding test for CI #11605
Conversation
|
|
||
| val batchSize = 100 | ||
| val numEpoch = 10 | ||
| val ctx = Context.cpu() |
There was a problem hiding this comment.
@nswamy is there an auto context in Scala?
There was a problem hiding this comment.
we don't have auto context, we planned to run these examples only on GPUs since it takes more time on CPU, We currently control this using a environment variable that is passed from a script. We will be moving away from this model with #11596
There was a problem hiding this comment.
Auto context is a method we use in our Python frontend where the context gets globally defined. The script then takes this configuration and uses it instead having the context explicitly defined.
Otherwise you run into the problem that you are not able to test the same code on CPU and GPU without having to make a Switch somewhere.
Thanks for elaborating, Naveen. I agree that GPU would make sense here. At the moment, it states CPU - is that a mistake or on purpose?
There was a problem hiding this comment.
I did CPU on purpose because this is training on MNIST data. It trains on my macbook in just a few minutes.
This is consistent with the other example that we have that uses mnist here.
|
Thanks a lot for adding the examples to our CI. In general, I really like the idea and would like to strive for having all examples in CI. I'm just a bit concerned that the training time in our examples could cause some problems, thus, in the long term I'd like to move them into a separate job. |
|
|
||
| val datasAndLabels = trainMultiIter.provideData ++ trainMultiIter.provideLabel | ||
|
|
||
| val (argShapes, outputShapes, auxShapes) = network.inferShape(trainMultiIter.provideData("data")) |
There was a problem hiding this comment.
bulk of the code in this file is training code, can you move it to generic runTraining method inside of the example, so customers can directly use it? you can call the runTraining code by passing training/test files, etc.,
|
Can we move test code to |
…cala suite convention
apache#11605) * Updating MultiTask example to use new infer api and adding test for CI * Moved common multitask code into methods. Changed multitask test to scala suite convention
Description
Updated the multitask example to use the new inference api. Also added a test case that will run the example.
Checklist
Essentials
Please feel free to remove inapplicable items for your PR.
Changes
Comments