Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

three bugs fixed #239

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

three bugs fixed #239

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
1f82b44
bug fix
cliffordsun May 10, 2019
109c369
Fixed issue where logs that are at the end of file were being lost wh…
cliffordsun May 14, 2019
39da3b6
change the condition of stat the opened file
cliffordsun May 20, 2019
2d8676c
Merge branch 'master' into master
kares Nov 11, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion lib/filewatch/sincedb_collection.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -193,7 +193,14 @@ def flush_at_interval

def handle_association(sincedb_value, watched_file)
watched_file.update_bytes_read(sincedb_value.position)
sincedb_value.set_watched_file(watched_file)
if watched_file.all_read?
# avoid the last_changed_at of sincedb_value which record the watched file has bean read over being changed when reload sincedb
sincedb_value.set_watched_file_without_touch(watched_file)
logger.trace("handle_association call set_watched_file_without_touch")
else
sincedb_value.set_watched_file(watched_file)
logger.trace("handle_association call set_watched_file")
end
watched_file.initial_completed
if watched_file.all_read?
watched_file.ignore
Expand Down
4 changes: 4 additions & 0 deletions lib/filewatch/sincedb_value.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,10 @@ def set_watched_file(watched_file)
@watched_file = watched_file
end

def set_watched_file_without_touch(watched_file)
@watched_file = watched_file
end

def touch
@last_changed_at = Time.now.to_f
end
Expand Down
5 changes: 4 additions & 1 deletion lib/filewatch/tail_mode/processor.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -187,7 +187,10 @@ def process_rotation_in_progress(watched_files)
potential_sdb_value.set_watched_file(watched_file)
end
end
logger.trace("---------- >>>> Rotation In Progress: after handling rotation", :this_watched_file => watched_file.details, :sincedb_value => (potential_sdb_value || sdb_value))
# Clean the path_in_sincedb because a file with the same name appears, this file must have been renamed
# Fix the duplicating collection after reloading sincedb(logstash restart or change logstash.conf)
sdb_value.add_path_in_sincedb(nil) unless sdb_value.nil?
logger.trace("---------- >>>> Rotation In Progress: after handling rotation", "this watched_file details" => watched_file.details, "sincedb_value" => (potential_sdb_value || sdb_value))
end
end

Expand Down
6 changes: 6 additions & 0 deletions lib/filewatch/watched_file.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,12 @@ def restat!
if rotation_detected?
# switch to new state now
rotation_in_progress
# update the original file's size, there may be new data written after last read
if file_open?
original_stat = @file.stat
@size = original_stat.size
update_bytes_unread
end
return true
else
@size = @stat.size
Expand Down