Memory leak due to caching container properties for SDK hosts which access unbounded number of containers

[johngallardo]

Describe the bug
An SDK host that accesses a potentially unbounded set of containers experiences a memory leak due to caching ContainerName->ContainerProperty metadata within the SDK.

To Reproduce

            var client = new CosmosClient("<redacted>",
                new CosmosClientOptions
                {
                    ConnectionMode = ConnectionMode.Direct,
                });

            var tasks = Enumerable.Range(0, 16).Select(async _ =>
            {
                while (true)
                {
                    var containerId = $"container_{Guid.NewGuid()}";
                    var container = client.GetContainer("database", containerId);

                    try
                    {
                        using (var feedIterator = container.GetItemLinqQueryable<object>().ToFeedIterator())
                        {
                            while (feedIterator.HasMoreResults) { await feedIterator.ReadNextAsync(); }
                        }
                    }
                    catch (Exception) { }
                }
            }).ToArray();
            await Task.WhenAll(tasks);

Expected behavior
Steady memory consumption within the host process.

Actual behavior
Memory consumption grows unbounded.

Environment summary
SDK Version: 3.13 (although likely repros on newer versions)
OS Version : Windows (dotnetcore 3.1)

Additional context
I have observed this specifically within an SDK host process that handles change feed processing for a multitude of Containers.

[j82w]

What is the use case for having a single SDK instance to do requests against an unbounded number of containers? I'm not aware of any production use cases for this. Even in the scenario where there are hundreds of containers the overhead of caching the container properties is worth the cost to avoid the additional network latency in the future requests.

[johngallardo]

The scenario I am aware of in a service I work on is we have host processes for Change Feed Processors, where the set of monitored containers dynamically changes over time. This cached metadata continues to grow over time. The impact seems disproportional in terms of memory utilization when the container does not exist as well. Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10 From: ***@***.***> Sent: Monday, May 17, 2021 4:24 AM To: ***@***.***> Cc: John ***@***.***>; ***@***.***> Subject: Re: [Azure/azure-cosmos-dotnet-v3] Memory leak due to caching container properties for SDK hosts which access unbounded number of containers (#2471) What is the use case for having a single SDK instance to do requests against an unbounded number of containers? I'm not aware of any production use cases for this. Even in the scenario where there are hundreds of containers the overhead of caching the container properties is worth the cost to avoid the additional network latency in the future requests. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.jparrowsec.cn%2FAzure%2Fazure-cosmos-dotnet-v3%2Fissues%2F2471%23issuecomment-842244638&data=04%7C01%7C%7Cb43b1204caf54d2d4dee08d919264c74%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637568474545274860%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Nhi5PsPavC5ufLidvjWl7xH4ZtTydrv4D5L7tLBwsWI%3D&reserved=0>, or unsubscribe<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.jparrowsec.cn%2Fnotifications%2Funsubscribe-auth%2FAAJ7ETX6OW7NPHQYWBQBWYLTOD4FZANCNFSM445JMODQ&data=04%7C01%7C%7Cb43b1204caf54d2d4dee08d919264c74%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637568474545284852%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=441D7VhJtpA%2F8wv8D2IXAlfO%2FT3LN5I46KtiYfIkl60%3D&reserved=0>.

[johngallardo]

I'm not sure that this is a "Feature-Request."

What I am observing in my service is that collection cache continues to grow unbounded, and additionally the cache entries become polluted with exceptions. In these cases, (where the callback to public async Task<TValue> GetAsync ) throws an exception, it seems that the cache entry goes unused anyways (the code will simply attempt to invoke the provided callback again). In such a case, I wonder if it would be possible to eliminate the entry from the cache all together rather than retain the exception.