'astunparse.unparse' error with Python 3.8.1

[matkuki]

Hi,

The following code throws an error when it parses a constant inside some_code:

astunparse.unparse(some_code)

The traceback is:

  File "test_astunparse.py", line 236, in process_code
    result = astunparse.unparse(some_code)
  File "C:\Python38\lib\site-packages\astunparse\__init__.py", line 13, in unparse
    Unparser(tree, file=v)
  File "C:\Python38\lib\site-packages\astunparse\unparser.py", line 38, in __init__
    self.dispatch(tree)
  File "C:\Python38\lib\site-packages\astunparse\unparser.py", line 66, in dispatch
    meth(tree)
  File "C:\Python38\lib\site-packages\astunparse\unparser.py", line 78, in _Module
    self.dispatch(stmt)
  File "C:\Python38\lib\site-packages\astunparse\unparser.py", line 66, in dispatch
    meth(tree)
  File "C:\Python38\lib\site-packages\astunparse\unparser.py", line 347, in _FunctionDef
    self.__FunctionDef_helper(t, "def")
  File "C:\Python38\lib\site-packages\astunparse\unparser.py", line 359, in __FunctionDef_helper
    self.dispatch(t.args)
  File "C:\Python38\lib\site-packages\astunparse\unparser.py", line 66, in dispatch
    meth(tree)
  File "C:\Python38\lib\site-packages\astunparse\unparser.py", line 791, in _arguments
    self.dispatch(d)
  File "C:\Python38\lib\site-packages\astunparse\unparser.py", line 66, in dispatch
    meth(tree)
  File "C:\Python38\lib\site-packages\astunparse\unparser.py", line 551, in _Constant
    if t.kind == "u":
AttributeError: 'Constant' object has no attribute 'kind'

A hack that fixes this is to comment out line 551 and 552 in C:\Python38\lib\site-packages\astunparse\unparser.py, like so:

        elif value is Ellipsis: # instead of `...` for Py2 compatibility
            self.write("...")
        else:
            # if t.kind == "u":
            #    self.write("u")
            self._write_constant(t.value)

Regards

[simonpercivall]

Hi! The code is directly based on the official 3.8 Tools/parser/unparse.py code.

If you give me a specific (runnable) example, I'll take a look at it.

[tbennun]

I just made a PR (#44) to address this, as well as the case of b"" strings and others. Turns out that Python will not fill in kind if you are creating ast.Num objects, which will be the case for many codes based on ast.

[isidentical]

I dont think this is a valid example, manually constructed nodes should have all fields specified by node. By the way Tools/parser/unparse.py is now exposed under ast.unparse, FYI.

[tbennun]

@isidentical in which version?

Python 3.8.1 (default, Jan  8 2020, 22:29:32)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ast
>>> ast.unparse
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'ast' has no attribute 'unparse'

[isidentical]

It will be in 3.9, you can download alpha versions or compile your self to test it.

[simonpercivall]

@matkuki @tbennun Hi! Sorry for the late reply.

Thanks for the report and the PR, however:

Since this package is meant to be a replica of what was provides in the Tools/parser/unparse.py module, and considering that Python 3.9 will expose this functionality directly (which this packages will simply proxy to), I'm going to close this issue (and PR) without merging.

I suggest either updating the code that creates Num instances, or subclassing the unparser and overriding the problematic method.

If you give me a use case where it's impossible to do any of the above, I may reconsider.

[isidentical]

Since this package is meant to be a replica of what was provides in the Tools/parser/unparse.py module, and considering that Python 3.9 will expose this functionality directly (which this packages will simply proxy to), I'm going to close this issue (and PR) without merging.

For future references, what do you think about adding a note to the readme this tool is now exposed under ast module#?

[strichter]

I think this should be re-opened as Python's ast module has different behavior in 3.8 and 3.9:

Python 3.8.6 (default, Sep 25 2020, 09:36:53) 
[GCC 10.2.0] on linux
>>> import ast
>>> c = ast.Constant(True)
>>> c.kind
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'Constant' object has no attribute 'kind'

and

Python 3.9.1 (default, Dec  8 2020, 02:26:20) 
[GCC 9.3.0] on linux
>>> import ast
>>> c = ast.Constant(True)
>>> c.kind

[ericwb]

I'm also running into this error on Python 3.8. I'm not sure I'm clear why this wasn't fixed. It works perfectly fine on Python 3.7 and fails on 3.8. 3.9 and above I use the base Python ast.unparse().
https://github.com/PyCQA/bandit/runs/5314237096?check_suite_focus=true

Bandit needs a way to suggest fixes and unparse modified ast nodes. I wanted to use this module, but will need Py3.8 support.

[ericwb]

Never mind, I think I can workaround this. Didn't realize the difference in how arguments in calls where the ast nodes differ in type between 3.7 and 3.8.