Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix system config using double memory #496

Merged
merged 1 commit into from
Dec 3, 2014
Merged

Fix system config using double memory #496

merged 1 commit into from
Dec 3, 2014

Conversation

sonots
Copy link
Member

@sonots sonots commented Dec 1, 2014

The current code calls read_config twice on the parent (supervisor) process and the child (main) process. Because the member variable @config_data is overwritten in the child process, the memory usage for @config_data becomes double. This causes a problem on an environment whose conf file is large like 10,000 lines.

Commenting out this line

apply_system_config(opt)
suppressed this issue.

This patch fixes the problem without removing the system config feature.

end
end

# TODO: this method should be moved to SystemConfig class method
def apply_system_config(opt)
# Create NULL file to avoid $log uninitialized problem before call @log.init
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed this trick. Now, read_config and apply_system_config uses the default (or commad line option arguments) log_level on starting up. After the system_config is applied, the log_level is changed.

@sonots sonots changed the title Fix system config using double memory [WIP] Fix system config using double memory Dec 1, 2014
@sonots sonots force-pushed the fix_system_config branch 2 times, most recently from 96bfef4 to a3b5797 Compare December 1, 2014 12:13
@sonots sonots force-pushed the fix_system_config branch from 18b4c20 to bb6c0bd Compare December 1, 2014 18:45
@sonots sonots changed the title [WIP] Fix system config using double memory Fix system config using double memory Dec 1, 2014
@sonots
Copy link
Member Author

sonots commented Dec 2, 2014

Verified this code works well by running on my production environment.

opt.merge!(SystemConfig.new(systems.first).to_opt)
ensure
file.close
SystemConfig.new(systems.first).apply(self)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this line is in ensure?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Look. Not in ensure.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ouch!

@repeatedly
Copy link
Member

#497 provides new plugin instantiation approach.
New way doesn't need to assign format parameter to conf.
This avoids CoW of Element object.

@repeatedly repeatedly added the enhancement Feature request or improve operations label Dec 2, 2014
@repeatedly
Copy link
Member

It seems ok.

@tagomoris Do you see any concern?

@tagomoris
Copy link
Member

No problem.

sonots added a commit that referenced this pull request Dec 3, 2014
Fix system config using double memory
@sonots sonots merged commit d65f5e4 into master Dec 3, 2014
@sonots sonots deleted the fix_system_config branch December 3, 2014 09:05
sonots added a commit that referenced this pull request Dec 3, 2014
Fix system config using double memory
@sonots sonots added the v0.10 label Dec 3, 2014
@coveralls
Copy link

Coverage Status

Changes Unknown when pulling bb6c0bd on fix_system_config into * on master*.

ganmacs added a commit to ganmacs/fluentd that referenced this pull request Oct 15, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Feature request or improve operations
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants