Is it possible that connection pool diminish the ability of other servers connecting to my db server?

[Nithur-M]

I have deployed the MySQL server to an EC2 instance. And a lambda function update the data to it using a connection pool. I am using the same database server to fetch data to a Nextjs app and also have a workbench to quickly query data.
Most of the time, I am unable to connect to the DB. Nextjs app says ETIMEDOUT. Mysql workbench throws system error 104.
I have changed my EC2 instance to something bigger with memory and CPU also, but still, the same issue persists.

Did anyone face this issue before? I'll appreciate your help. Thanks.