PyCharm: Unable to debug AWS lambda

[ro83rt]

Since updating AWS Toolkit to 1.2 and Pycharm Community to 2019.1.1 I'm no longer able to debug any python lambda due to the:

pydev debugger: warning: trying to add breakpoint to file that does not exist

Find some advices to setup path mappings, but did not find such option AWS Lambda Run/Debug configurations.

Thank you for any help.

• OS: Win 10 Pro
• JetBrains' Product: PyCharm
• JetBrains' Product Version: 2019.1.1 (Community Edition)
• Toolkit Version: 1.2
• SAM CLI Version: 0.14.2
• JVM/Python Version: python3.6

[tfx-alex]

I have the same problem - running locally in Docker works fine, also starting local debug mode of the lambda works just fine, but breakpoints can't be set because the file mapping is off.

"C:\Program Files\Amazon\AWSSAMCLI\bin\sam.cmd" local invoke --template C:\Users\sca\AppData\Local\Temp\lambdaBuild2\template.yaml --event "C:\Users\sca\AppData\Local\Temp\[Local] ManageIncomingEmails-event2.json" ManageIncomingEmails --debug-port 52760 --debugger-path "C:\Program Files\JetBrains\PyCharm 2019.1.1\helpers\pydev" --debug-args "-u /tmp/lambci_debug_files/pydevd.py --multiprocess --port 52760 --file"
2019-04-11 20:52:11 Found credentials in environment variables.
2019-04-11 20:52:12 Invoking app.lambda_handler (python3.7)

Fetching lambci/lambda:python3.7 Docker container image......
2019-04-11 20:52:13 Mounting C:\Users\sca\AppData\Local\Temp\lambdaBuild2\ManageIncomingEmails as /var/task:ro inside runtime container
Connected to pydev debugger (build 191.6605.12)
pydev debugger: warning: trying to add breakpoint to file that does not exist: /var/task/c:/data/qsync/projects/lambdatest010/email_receiver/app.py (will have no effect)

It looks like the full path of the source file is passed on to the debugger and not the relative path, so the debug process won't be able to map it to the correct file. The last line shows the strange path: mount point /var/task and then the full original path to the file on the host machine.

Other than that the code runs locally without problems.

• OS: Win 10 Pro
• JetBrains' Product: PyCharm
• JetBrains' Product Version: 2019.1.1 (Professional)
• Toolkit Version: v1.3-EAP.2019.04.11
• SAM CLI Version: 0.14.2
• JVM/Python Version: python3.7

[praghavan039]

I face the same issue. When can we expect the fix for this please.

[abrooksv]

Built a Windows machine, and was able to reproduce it

[abrooksv]

PyCharm lowercases all paths on Windows (https://github.com/JetBrains/intellij-community/blob/9d1d1a0b6e1661c14370e39e6c0d325879a615cc/python/src/com/jetbrains/python/debugger/PyLocalPositionConverter.java#L33) but our PathMapper is case sensitive.

[tfx-alex]

What I find odd though is that on a Windows machine Docker will mount the temp folder in %APPDATA%\Local\Temp\abc\def to /var/task, where a copy of the whole project folder has been placed.
But then it seems the debugger wants to access the original folder with an absolute path, hence /var/task/c:/.. - which is a strange mapping.

Or is that exactly what the PathMapper is supposed to resolve and can't because of lowercase / mixed case mismatch?

[abrooksv]

PathMapper maps the IDEs view of the world on to the Dockers view and back. This way it knows which file to put the break point in.

I have a fix I am testing

[8848mzy]

Same problem but on Mac OS since PyCharm change to sam build

[Rubyj]

@abrooksv After updating my aws toolkit today I now have the same issue with the new aws toolkit 1.3 (wasn't happening before) on linux. Did these changes break linux? ( I did not face this issue last week)

2019-04-30 16:08:28 Mounting /tmp/lambdaBuild/s3GetForTagetDBFunction as /var/task:ro,delegated inside runtime container
Connected to pydev debugger (build 191.6605.12)
pydev debugger: warning: trying to add breakpoint to file that does not exist: /home/rjacobs/git/s3-get-files-for-targetdb/s3-get-files-for-targetdb/build/app.py (will have no effect)

[abrooksv]

It should not have, since it was gated behind if your IDE is running on Windows.

We do test debugger does work in our integration tests on Linux, so not sure what is causing it to fail.

What is the actual path on your disk?

[Rubyj]

@abrooksv Actually, the message there is the actual path on my disk. I'm confused as to why it "can't find the file". I definitely wasn't having this issue last week. Maybe a docker issue?

The absolute path is indeed /home/rjacobs/git/s3-get-files-for-targetdb/s3-get-files-for-targetdb/build/app.py

However I am not sure if that is correct for build/run time inside an image?

template.yml

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
    s3-get-files-for-targetdb

    Get data and load into targetdb api


Resources:
    TargetBucket:
        Type: AWS::S3::Bucket
    s3GetForTagetDBFunction:
        Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
        Properties:
            CodeUri: .
            Handler: s3-get-files-for-targetdb/build.app.lambda_handler
            Runtime: python3.7
            Timeout: 30

[abrooksv]

It should map to /var/task/s3-get-files-for-targetdb/build/app.py.

Will attempt to reproduce it.

[Rubyj]

Is it weird then that I am seeing

pydev debugger: warning: trying to add breakpoint to file that does not exist: /home/rjacobs/git/s3-get-files-for-targetdb/s3-get-files-for-targetdb/build/app.py (will have no effect)

and not a mention about /var/task/s3-get-files-for-targetdb/build/app.py then?

Do I need to add the mapping manually to my interpreter? I am using venv not a remote interpreter. Ok thank you for your help.

[abrooksv]

No, we handle the path mappings.

The path is supposed to be translated to the location inside the container, but it fails to map. When it fails to map, it is passed as is in, this is why you see the real path in the message.

[Rubyj]

That makes sense. FYI I even tried reverting to a 1.3EAP version from last week (Thursday/Friday) when I know the issue was not happening and my breakpoints were being hit and yet I still am getting the same issue.

[abrooksv]

We have logs, but they are at the debug level. Let me try to remember how to enable debug logs in IntelliJ.

[Rubyj]

Sounds good. If it helps at all I am using PyCharm professional 2019.1.1 and the only things I have done was update my linux kernel (5.0), some packages on my system as part of a rolling system update, and updated the aws toolkit to 1.3 and then I started receiving this behavior.

[abrooksv]

Go to Help->Debug Log Settings and add software.aws.toolkits. You may need to restart. You should then see a log line containing "Failed to map" in the idea.log file. It can be found through Help->Show logs in Explorer

[Rubyj]

2019-04-30 16:55:18,886 [  27993]  DEBUG - es.lambda.execution.PathMapper - Failed to map /home/rjacobs/git/s3-get-files-for-targetdb/s3-get-files-for-targetdb/build/app.py to remote file system: [PathMapping(localRoot='/home/rjacobs/git/s3-get-files-for-targetdb/./', remoteRoot='/var/task/'), PathMapping(localRoot='/tmp/lambdaBuild/s3GetForTagetDBFunction/', remoteRoot='/var/task/'), PathMapping(localRoot='/home/rjacobs/.local/share/JetBrains/Toolbox/apps/PyCharm-P/ch-0/191.6605.12/helpers/pydev/', remoteRoot='/tmp/lambci_debug_files/')] 

Not sure if this maters, but when I look in /tmp/lambdaBuild/s3GetForTagetDBFunction I see everything except for my app.py file

[abrooksv]

If I had to guess, it is the /./ at the end of the first path. We should have cleaned the path, will try to repro with a unit test

[Rubyj]

Ok thanks! If there is anything I can do to help please let me know.

[Rubyj]

@abrooksv can confirm my issues are sorted. Built from source and tested. Great job!