Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sqlkvcache errors when using value_from #7693

Closed
ajkerrigan opened this issue Aug 25, 2022 · 0 comments · Fixed by #7711
Closed

sqlkvcache errors when using value_from #7693

ajkerrigan opened this issue Aug 25, 2022 · 0 comments · Fixed by #7711
Labels

Comments

@ajkerrigan
Copy link
Member

ajkerrigan commented Aug 25, 2022

Describe the bug

Discussion salvaged from closed PR #7692

When using the sqlite3-based cache, I'm hitting errors running policies that contain a value_from lookup. This may point to additional issues with cache connections.

What did you expect to happen?

Cache lookups to work if value_from is in play.

Cloud Provider

Amazon Web Services (AWS)

Cloud Custodian version and dependency information

develop build using commit a88b467

Policy

policies:
  - name: sqs-metrics
    resource: aws.sqs
    filters:
      - type: value
        key: tag:Owner
        op: in
        value_from:
            url: s3://my_bucket/valid_tags.json
            format: json
            expr: 'Owner[]'

Relevant log/traceback output

Traceback (most recent call last):
  File "/home/aj/code/cloud-custodian/c7n/commands.py", line 302, in run
    policy()
  File "/home/aj/code/cloud-custodian/c7n/policy.py", line 1281, in __call__
    resources = PullMode(self).run()
  File "/home/aj/code/cloud-custodian/c7n/policy.py", line 294, in run
    resources = self.policy.resource_manager.resources()
  File "/home/aj/code/cloud-custodian/c7n/query.py", line 535, in resources
    resources = self.filter_resources(resources)
  File "/home/aj/code/cloud-custodian/c7n/manager.py", line 111, in filter_resources
    resources = f.process(resources, event)
  File "/home/aj/code/cloud-custodian/c7n/filters/core.py", line 553, in process
    return super(ValueFilter, self).process(resources, event)
  File "/home/aj/code/cloud-custodian/c7n/filters/core.py", line 190, in process
    return list(filter(self, resources))
  File "/home/aj/code/cloud-custodian/c7n/filters/core.py", line 540, in __call__
    matched = self.match(i)
  File "/home/aj/code/cloud-custodian/c7n/filters/core.py", line 566, in match
    self.v = values.get_values()
  File "/home/aj/code/cloud-custodian/c7n/resolver.py", line 157, in get_values
    self.cache.save(("value-from", key), contents)
  File "/home/aj/code/cloud-custodian/c7n/cache.py", line 158, in save
    with self.conn as cursor:
AttributeError: __enter__

Extra information or context

Prior to the fix in a88b467 this policy was dying on the get and afterward it started dying on the save. Looked like it was because in between the cache get and cache save, we call contents = self._get_values() which goes get_contents() --> resolve() which closes the connection and sets it to None.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant