[FLINK-36838][state/forst] Fix the deadlock when quit forst state backend

[Zakelly]

What is the purpose of the change

In #25732 we join the async background threads of ForSt to avoid deadlock when JVM quit. That is an incomplete solution which is also problematic. The background threads are shared among multiple ForSt instances, meaning that simply join those threads may cause background job unfinished in other instances.

The ForSt project has investigated the root cause and gave a solution (ververica/ForSt#30). Thus we should bump the depended ForSt version and revert the thread joining in Flink.

Brief change log

• Bump ForSt version to 0.1.6
• Revert the change of background thread joining

Verifying this change

This change is already covered by IT case ForStStateBackendV2Test. If it can normally exit then the problem solved.

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): no
• The public API, i.e., is any changed class annotated with @Public(Evolving): no
• The serializers: no
• The runtime per-record code paths (performance sensitive): no
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
• The S3 file system connector: no

Documentation

• Does this pull request introduce a new feature? no
• If yes, how is the feature documented? not applicable

[fredia]

Thanks for the PR, LGTM. Let's wait for the CI...

[flinkbot]

CI report:

• baf5b7d Azure: SUCCESS

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build