Generalize .runBackground to work with all kinds of subprocesses
#657
Comments
|
Maybe could even make it a general-purpose wrapper that works with all kinds of Tasks as well |
|
We've hit the need for a non-JVM runBackground now that we're aiming to support Javascript https://github.com/com-lihaoyi/mill/pull/4022/files and Python #3928. |
|
@lihaoyi actually yesterday i was working on this same issue for web examples and i was mimicing the same code as we have for java/scala like having backgroundwrapper in .py extension and then run that file with all the necessary args. I will pull a request for this in some time and i think untill we don't get general support we can go with this one Your thoughts? |
|
@himanshumahajan138 I think it should be doable to generalize |
…4085) Fixes #657 We re-use `MillBackgroundWrapper` and extend it to optionally spawn subprocess instead of calling the in-process main method. There's a bit of overhead in the JVM wrapper, but we need some kind of process wrapper to manage the mutex and termination even when the Mill process terminates, and eventually when #4007 lands we can use that to provide lighter-weight wrappers. We add a shutdown hook and copy the termination grace-period logic from OS-Lib to try and gently kill the wrapped process when necessary Integrated this into `PythonModule#runBackground` and added a unit test to verify the asynchronous launch, background existence (by checking that a file lock is taken) and termination on deletion. We can integrate this into `TypescriptModule` as well but punting that for later The subprocess APIs are a mess but leaving that to fix in 0.13.0 #3772
…om-lihaoyi#4085) Fixes com-lihaoyi#657 We re-use `MillBackgroundWrapper` and extend it to optionally spawn subprocess instead of calling the in-process main method. There's a bit of overhead in the JVM wrapper, but we need some kind of process wrapper to manage the mutex and termination even when the Mill process terminates, and eventually when com-lihaoyi#4007 lands we can use that to provide lighter-weight wrappers. We add a shutdown hook and copy the termination grace-period logic from OS-Lib to try and gently kill the wrapped process when necessary Integrated this into `PythonModule#runBackground` and added a unit test to verify the asynchronous launch, background existence (by checking that a file lock is taken) and termination on deletion. We can integrate this into `TypescriptModule` as well but punting that for later The subprocess APIs are a mess but leaving that to fix in 0.13.0 com-lihaoyi#3772
Currently it's hardcoded to only work for the main method of Java/Scala modules, but it could be useful for running other sorts of long-running daemons too: webpack proxy,
docker run, databases, etc.We might need to replace the auto-shutdown thread with a auto-shutdown subprocess, since we'll be running non-JVM things
The text was updated successfully, but these errors were encountered: