MemoryConsumer::try_grow Underflow

[tustvold]

Describe the bug

MemoryConsumer::try_grow calls through to MemoryManager::can_grow_directly which determines the remaining space by performing self.pool_size - trk_total.

This can potentially underflow if the tracked memory exceeds the pool size, something for which there is no protection.

To Reproduce

#[test]
fn test_underflow() {
    let config = MemoryManagerConfig::try_new_limit(100, 0.5).unwrap();
    let manager = MemoryManager::new(config);
    manager.grow_tracker_usage(100);

    manager.register_requester(&MemoryConsumerId::new(1));
    assert!(!manager.can_grow_directly(20, 0));
}

Expected behavior

Additional context

[askoa]

I can pick this up. I am planning to return zero if trk_total is more than self.pool_size in below function. Let me know what you think.

https://github.com/apache/arrow-datafusion/blob/df8aa7a2e2a6f54acfbfed336b84144256fb7ff8/datafusion/core/src/execution/memory_manager/mod.rs#L331-L335

[alamb]

I think this is now complete with #4351 and #4522