Snippet bot is exhausting memory in Cloud Functions

[JustinBeckwith]

Seeing this in the logs today:

Error: memory limit exceeded. Function invocation was interrupted.

Search query for logs:

severity=ERROR
resource.labels.function_name="snippet_bot"

[tmatsuo]

Not 100% sure, but it seems like it is happening on few repos:

googleapis/google-api-python-client
and
googleapis/go-genproto

I'll try to debug the bot with my clone.

[tmatsuo]

Hm, I'm buffled.

The bot works perfectly fine with a test PR:
googleapis/go-genproto#494

So it's not repo specific. Maybe the payloads are somehow corrupted and it causes some kind of infinite loop?

[tmatsuo]

I think I spotted a PR which likely caused the memory death:
googleapis/go-genproto#493

3443 files were deleted in a single PR.

[tmatsuo]

Downgrading to P2 since it's not a widespread issue.

[tmatsuo]

The bot is using parse-diff which accepts a string for the diff. We pass around the 75MB of string several times. I guess that the string is copied multiple times.

[JustinBeckwith]

I bumped the function from 512MB to 1GB - let's do that, and keep an eye on the loggies

[tmatsuo]

The other one is: googleapis/google-api-python-client#1059

2704 files affected. The diff is ~180MB.

[tmatsuo]

The request for the above PR went through, amusingly it took 539 seconds (almost 9 minutes).

snippet_boturs0tuecwwy6 Function execution took 538793 ms, finished with status code: 200

[tmatsuo]

Interestingly the go-genproto pr took about 10 seconds.

l5rfsoage2jn Function execution took 10902 ms, finished with status code: 200

This is because this PR only deletes the files whereas the other one modifies the files (and the bot is fetching all the files).

[tmatsuo]

It's still possible that we'll see similar memory errors, but I feel good now because the bot can handle such significant PRs without dying now.