TorchScript to Knossos test with preview PyTorch on GitHub Actions #669
Conversation
try running just one test
|
@toelli-msft any thoughts if this is a good direction to keep working on? |
|
As some extra context the virtual machines do have Visual Studio installed but not available on the path. The easiest way to do it would be to use: |
Only works on default branch
It's |
|
Seems like a fine idea in general, although I don't understand what functionality it gives us over Azure DevOps. Is it just so that we can have scheduled jobs, or something else? |
It's the ease of connection from the source to the build. The connection between GitHub and DevOps via the marketplace app always seems opaque. If we have the build on GitHub it's clearer to trigger manually or on a schedule (and I suspect easier to get artifacts back out on e.g. PRs). This was a specific build, useful by itself, but also demonstrates we could port the other builds over - I was going to get some experience and see how it goes before proposing more though. Alternatively I can try to see if we can get the DevOps schedules https://docs.microsoft.com/en-us/azure/devops/pipelines/process/scheduled-triggers?view=azure-devops&tabs=yaml working without it becoming a compulsory check for PRs. |
|
Discussed this more, think it makes sense to experiment with this initial nightly build then consider porting more over to GitHub Actions later. Something to investigate is how much build time and parallelism we have on Github Actions. |
avoid early exit
|
For information, the early |
cgravill
changed the title
|
It's a bit messy with the out heap on multiple tests #679. In spite of that I propose to merge this with a single test being run. That gives us a a working CI build with Windows against the preview of PyTorch. If something gets broken before the maintenance 1.8.1 release we want to know ASAP as we need a working stable release. I'll continue to investigate the memory issue. |
Agreed, I think that would be a good move as a separate change #686 Part of me would like to go further and it being a good real programming language configured for builds e.g. https://fake.build or Shake as @toelli-msft mentions here: As an anecdotal experience from other projects then moving from a mix of YAML and shell scripts to F# via fake.build made things much clearer and referenceable (reducing duplication). |
Prototype of porting DevOps pipeline to GitHub Action. This allows us to easily schedule the task and demos we could run more on GitHub Actions.
We're using the preview version of PyTorch due to #659
Unfortunately the PyTorch extension creation mechanism seems to leak memory - my guess is that it doesn't release modules so currently if you run all five tests the C++ build runs out of heap:
C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29910\include\memory(3337): fatal error C1060: compiler is out of heap space(running the same test by itself completes)
I've reordered the dependency install, it looks like Cabal is stopping early (before getting to Ninja). Need to investigate this.