fix(async): set default concurrency to 10 tasks

[jhamman]

Unbounded concurrency is not a good idea. This sets the default to 10.

TODO:

• Add unit tests and/or doctests in docstrings
• Add docstrings and API docs for any new/modified user-facing classes and functions
• New/modified features documented in docs/tutorial.rst
• Changes documented in docs/release.rst
• GitHub Actions have all passed
• Test coverage is 100% (Codecov passes)

[dcherian]

I don't have an opinion but this is what the threading module does

Default value of max_workers is changed to min(32, os.cpu_count() + 4). This default value preserves at least 5 workers for I/O bound tasks. It utilizes at most 32 CPU cores for CPU bound tasks which release the GIL. And it avoids using very large resources implicitly on many-core machines.

[jhamman]

There are really two issues here.

1. how many threads we try to use at once, and
2. how many concurrent IO operations we attempt

Right now, we are using a single config for both of these but in the future, we may consider allowing users to configure the size of the thread pool and/or multi-threading behavior separately.

[dcherian]

But perhaps the default should scale with cpu_count like what threading does: min(32, os.cpu_count() + 4). If we are making arbitrary choices, perhaps we just go with what the standard library does.

[jhamman]

But perhaps the default should scale with cpu_count like what threading does: min(32, os.cpu_count() + 4). If we are making arbitrary choices, perhaps we just go with what the standard library does.

I hear what you are saying. But this applies to both IO and tasks that use to_thread. I would like us to revisit the way we distribute work to IO and threaded tasks another day.