Exclude persistent tasks from the concurrency limit

[radekn]

Problem

If the number of persistent tasks to run exceeds the default concurrency limit, turbo will refuse to run them unless you override the concurrency limit, which is likely to negatively affect the scheduling of non-persistent tasks.

Background

When you have a number of persistent tasks to start, you always expect that all of them are started eventually. If they're affected by the concurrency limit, this expectation is not met, and the only workaround is to increase the concurrency limit. This workaround leads to a problem, though, because if the limit has to be increased to accommodate for the number of persistent tasks, it means it can't be used effectively for its primary purpose, which is efficient scheduling of resource-bound tasks that are expected to complete.

Proposal

I propose to exclude persistent tasks from the concurrency limit. Ideally, they would be ignored for the purpose of concurrent scheduling of non-persistent tasks.

[radekn]

Some more thoughts on this.
If there is any risk that excluding persistent tasks from the limit completely would introduce problems for anyone, a less radical but more complex solution is possible. For example, you could keep counting persistent tasks towards the concurrency limit, but only as long as there are still any non-persistent tasks left to run. In this case, you'd need to priortise the non-persistent ones when scheduling, so that the persistent ones don't block them from being run by exhausting the limit first.