[Bug] [Crash][Reproducible] EOFError: Ran out of input when import yapf with multiprocess

[whlook]

previous similar issue #1164

Problems

• When i use yapf (just import it) in multiprocess program, it will crash with 'EOFError: Ran out of inpu'，log info below:

Reproduction

1. yapf==0.40.2
2. rm -r ~/.cache/YAPF
3. run this python code:

 import multiprocessing
 
 def proc():
     import yapf
 
 if __name__ == "__main__":
     pp=[]
     for i in range(100):
         p = multiprocessing.Process(target=proc)
         pp.append(p)
 
     for p in pp:
         p.start()
 
     for p in pp:
         p.join()
                                                   

Reason

the problem code is here:

yapf/third_party/yapf_third_party/_ylib2to3/pgen2/driver.py

Line 247 in c0908d0

g = grammar.Grammar()

When in multi process enviroment,some process will create grammar cache(when ~/.cache/YAPF not exists), before the process write to the cache file, other process will see the file exist and load it, but the cache file is empty right now , so EOFError will raise and the program crashed.

How to fix it?

• I think we need to add try here(

  yapf/third_party/yapf_third_party/_ylib2to3/pgen2/driver.py

  Line 247 in c0908d0

  g = grammar.Grammar()

  ) like we had add try when we create the cache.
• Or we can add a multiprocess.Lock here
• Or we can do python -c 'import yapf' before we start our multiprocess program.(for users,trick)

[hartwork]

@whlook that's a great report, in particular the reproducer is very handy — thank you!

I have a pull request coming up for this — should be auto-linked below in a minute or two —, and I would like to share my extended version of your reproducer below:

import multiprocessing

def cleanup():
    import os
    import yapf
    from yapf_third_party._ylib2to3.pgen2.driver import _generate_pickle_name
    for i in ("Grammar.txt", "PatternGrammar.txt"):
        filename = _generate_pickle_name(i)
        print(f"  Removing {filename}...")
        os.remove(_generate_pickle_name(i))

def proc():
    import yapf

if __name__ == "__main__":
    max_parallelity = 30
    for parallelity in range(2, max_parallelity + 1):
        print("Cleaning up...")
        cleanup_p = multiprocessing.Process(target=cleanup)
        cleanup_p.start()
        cleanup_p.join()

        print(f"Testing for the race condition with {parallelity} processes...")
        pp = []
        for i in range(parallelity):
            p = multiprocessing.Process(target=proc)
            pp.append(p)

        for p in pp:
            p.start()

        for p in pp:
            p.join()

        if any(p.exitcode != 0 for p in pp):
            print(f"Done, race condition proven above, took {parallelity} processes.")
            break
    else:
        print("Done, no luck crashing this time.")

[TheUbuntuGuy]

This issue is not completely resolved as #1243 does not resolve the issue for me. When running highly parallel (128 processes on a server), I still get the same exception on the latest release 0.43.0, which has this fix. It is reproducible about half the time. It appears that always 2 processes crash with the same exception, so there is some kind of race still occurring.

Traceback (most recent call last):
  File "/usr/local/bin/yapf", line 5, in <module>
    from yapf import run_main
  File "/usr/local/lib/python3.11/dist-packages/yapf/__init__.py", line 40, in <module>
    from yapf.yapflib import yapf_api
  File "/usr/local/lib/python3.11/dist-packages/yapf/yapflib/yapf_api.py", line 38, in <module>
    from yapf.pyparser import pyparser
  File "/usr/local/lib/python3.11/dist-packages/yapf/pyparser/pyparser.py", line 44, in <module>
    from yapf.yapflib import format_token
  File "/usr/local/lib/python3.11/dist-packages/yapf/yapflib/format_token.py", line 23, in <module>
    from yapf.pytree import pytree_utils
  File "/usr/local/lib/python3.11/dist-packages/yapf/pytree/pytree_utils.py", line 30, in <module>
    from yapf_third_party._ylib2to3 import pygram
  File "/usr/local/lib/python3.11/dist-packages/yapf_third_party/_ylib2to3/pygram.py", line 29, in <module>
    python_grammar = driver.load_grammar(_GRAMMAR_FILE)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/yapf_third_party/_ylib2to3/pgen2/driver.py", line 248, in load_grammar
    g.load(gp)
  File "/usr/local/lib/python3.11/dist-packages/yapf_third_party/_ylib2to3/pgen2/grammar.py", line 128, in load
    d = pickle.load(f)
        ^^^^^^^^^^^^^^
EOFError: Ran out of input

[TheUbuntuGuy]

What is even more interesting is that the workaround of running python -c 'import yapf' prior to starting the multiprocess format operation helps improve stability to a failure rate of about 1 in 20, but does not resolve the issue completely.

[anmyachev]

This issue is not completely resolved as #1243 does not resolve the issue for me. When running highly parallel (128 processes on a server), I still get the same exception on the latest release 0.43.0, which has this fix. It is reproducible about half the time. It appears that always 2 processes crash with the same exception, so there is some kind of race still occurring.

Traceback (most recent call last):
  File "/usr/local/bin/yapf", line 5, in <module>
    from yapf import run_main
  File "/usr/local/lib/python3.11/dist-packages/yapf/__init__.py", line 40, in <module>
    from yapf.yapflib import yapf_api
  File "/usr/local/lib/python3.11/dist-packages/yapf/yapflib/yapf_api.py", line 38, in <module>
    from yapf.pyparser import pyparser
  File "/usr/local/lib/python3.11/dist-packages/yapf/pyparser/pyparser.py", line 44, in <module>
    from yapf.yapflib import format_token
  File "/usr/local/lib/python3.11/dist-packages/yapf/yapflib/format_token.py", line 23, in <module>
    from yapf.pytree import pytree_utils
  File "/usr/local/lib/python3.11/dist-packages/yapf/pytree/pytree_utils.py", line 30, in <module>
    from yapf_third_party._ylib2to3 import pygram
  File "/usr/local/lib/python3.11/dist-packages/yapf_third_party/_ylib2to3/pygram.py", line 29, in <module>
    python_grammar = driver.load_grammar(_GRAMMAR_FILE)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/yapf_third_party/_ylib2to3/pgen2/driver.py", line 248, in load_grammar
    g.load(gp)
  File "/usr/local/lib/python3.11/dist-packages/yapf_third_party/_ylib2to3/pgen2/grammar.py", line 128, in load
    d = pickle.load(f)
        ^^^^^^^^^^^^^^
EOFError: Ran out of input

Hi @TheUbuntuGuy,

I have a patch for this problem: #1263. BTW do you have a chance to try it?