Disable profiler in tests?

[fjetter]

We constantly have a profiler running in the background, often the threads associated to it are not properly cleaned up.

In #6033 we discovered that it can also be responsible for holding on to references longer than necessary which should not be a problem in production but it is for testing.

On top of these and likely other problems, the profiler surely slows down our tests, particularly with converage enabled.

Should we not run the profiler at all for non-profiling related tests?

[mrocklin]

I think that we should not run the profiler during non-profiling tests.

…

On Wed, Apr 6, 2022 at 3:54 AM Florian Jetter ***@***.***> wrote: We constantly have a profiler running in the background, often the threads associated to it are not properly cleaned up. In #6033 <#6033> we discovered that it can also be responsible for holding on to references longer than necessary which should not be a problem in production but it is for testing. On top of these and likely other problems, the profiler surely slows down our tests, particularly with converage enabled. Should we not run the profiler at all for non-profiling related tests? — Reply to this email directly, view it on GitHub <#6075>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACKZTFPKZSVXK6RY3JA2RLVDVGLDANCNFSM5SVJUQZQ> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[graingert]

see also #4091

[crusaderky]

See also #6421

[fjetter]

My proposed course of action is

• Introduce a config flag that toggles the profiler.
• This config flag is on by default but will be disabled in our test suite
• All tests that rely on profiling to be enabled have to enable it specifically
• We ensure the profiling dashboard does not crash if the toggle is off

Nice to have

• The profiling dashboard shows a nice message indicating that profiling is off instead of trying to collect information

[graingert]

we missed a few tests that start the profiler by accident see #6498