Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing ZooKeeper data log retention configuration and Pinot controller segment deletion retention configuration #2

Closed
iamdorsa opened this issue Aug 2, 2023 · 1 comment · Fixed by #3

Comments

@iamdorsa
Copy link
Collaborator

iamdorsa commented Aug 2, 2023

I have encountered two issues while deploying the Pinot cluster using the ansible playbook available in this repository.

Firstly, I noticed that the ZooKeeper configuration file does not have the data log retention configuration set. This is important because without a retention policy, the ZooKeeper data logs can consume a large amount of disk space over time. It is recommended to configure the data log retention to limit the disk usage of ZooKeeper data logs.

Secondly, I observed that the Pinot controller does not have a configuration for segment deletion retention. This means that the deleted segments are not removed from the disk, and they can accumulate over time, leading to disk space issues. It is recommended to configure a retention policy for deleted segments to ensure that they are removed from the disk after a certain period.

@iamdorsa
Copy link
Collaborator Author

iamdorsa commented Aug 2, 2023

This PR fixes it. #3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants