Instrumented Python3.6 Lambda Functions return Missing Module: Queue error in Console

[jonrau1]

Previously, I had a working function in Python 3.6 below that I built by installing Boto3 and Botocore and packaging it up. No errors running in Python 3.6

import boto3
import json

def handler(event,context):
    client = boto3.client('ce')
    s3 = boto3.resource('s3')
--- Python Code Shortened ---
    object.put(Body=json.dumps(response).encode())
    return str (response)

Decided to add instrumentation to the Function, enabled X-Ray in the Console, attached needed IAM permissions and packaged the new function with aws-xray-sdk as well as multiprocessing due to having Missing Module: Queue error the first time.

import boto3
import json
from aws_xray_sdk.core import xray_recorder
from aws_xray_sdk.core import patch

patch(['boto3'])
@xray_recorder.capture("handler")

def handler(event,context):
    client = boto3.client('ce')
    s3 = boto3.resource('s3')
--- Python Code Shortened ---

Continued to receive Missing Module: Queue error -- which does not make a whole lot of sense since Python 3.x+ has renamed it queue and it is included in the multiprocessing library. I now changed the beginning of the code to this

import boto3
import json
try:
    import queue
except ImportError:
    import Queue as queue

from aws_xray_sdk.core import xray_recorder
from aws_xray_sdk.core import patch

I have also tried the above as

try:
    import Queue
except ImportError:
    import Queue as queue

However, I am still facing the issue of Missing Module: Queue -- is this a known issue for working with the automated decorator in X-Ray with Lambda? Is there another library I can try adding?

[chanchiem]

Hey,

Thanks for reporting the issue you are having.

As a way to help us find the root cause to this issue, I have a few questions to ask you.

1. Can you provide the full stack trace of the error that is occurring?
2. Does this issue only happen when you call patch() at the beginning of the application start?

[jonrau1]

Here is the full stack trace from the console

START RequestId: f246ecb3-281b-41c1-8f83-02dd5c8e24a9 Version: $LATEST
Unable to import module 'lambda_function': No module named 'Queue'

END RequestId: f246ecb3-281b-41c1-8f83-02dd5c8e24a9
REPORT RequestId: f246ecb3-281b-41c1-8f83-02dd5c8e24a9	Duration: 0.33 ms	Billed Duration: 100 ms 	Memory Size: 128 MB	Max Memory Used: 48 MB

And this issue happens when I called Patch() as well as when I did not have it included at all

My full code is below

import boto3
import json
try:
    import Queue
except ImportError:
    import Queue as queue

from aws_xray_sdk.core import xray_recorder
from aws_xray_sdk.core import patch

@xray_recorder.capture("handler")

def handler(event,context):
    client = boto3.client('ce')
    s3 = boto3.resource('s3')
    object = s3.Object('bucket', 'lambda/forecast.json')
    response = client.get_cost_forecast(
    TimePeriod={
        'Start': '2019-07-08', ## Start must be one day after creation date of function
        'End': '2020-07-01'
    },
    Metric='UNBLENDED_COST',
    Granularity='MONTHLY',
    PredictionIntervalLevel=90 ## 51 - 99 Range
    )
    object.put(Body=json.dumps(response).encode())
    return str (response)

[chanchiem]

Hi,

I can't seem to reproduce this on my end. Does the issue still occur when you remove X-Ray from your function? I have a feeling that this issue might be related to AWS Lambda and Botocore as the AWS X-Ray SDK does not use queues. I will investigate this further and see if I can reproduce this

[heitorlessa]

How are you packaging it? I had this issue before and it was related to system pip was installing Python 2.x dependencies as opposed to Python 3 (pip3).

If you're not using SAM CLI to build your dependencies for Lambda, try using pip3 and that should fix it.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs in next 7 days. Thank you for your contributions.