Skip to content

Commit

Permalink
more GC tuning
Browse files Browse the repository at this point in the history
on ruby2.1 and later, do an incremental GC on every write
on older ruby, full GC every 100 writes

see #67
  • Loading branch information
jcupitt committed Jan 21, 2016
1 parent 99dbfa3 commit 1b75ac8
Show file tree
Hide file tree
Showing 4 changed files with 48 additions and 13 deletions.
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
# master

# Version 0.3.14

* more GC tuning [felixbuenemann]
* add `write.rb` example program [felixbuenemann]

# Version 0.3.13

* don't use generational GC options on old Rubys [John Cupitt]
Expand Down
28 changes: 28 additions & 0 deletions examples/write.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
#!/usr/bin/env ruby

require 'rubygems'
require 'vips'

repeat = 2000

def rss
`ps -o rss= -p #{$$}`.chomp.to_i/1024
end

GC.start
puts "RSS at startup with gems loaded: %d MB" % [rss]

repeat.times do |i|
ARGV.each do |filename|
img = VIPS::Image.jpeg filename, :sequential => true

img = VIPS::JPEGWriter.new(img, quality: 50)
img.write('test.jpg')

print "\rIteration: %-8d RSS: %6d MB File: %-32s".freeze % [i+1, rss, filename]
end
end
puts

GC.start
puts "RSS at exit: %d MB" % [rss]
2 changes: 1 addition & 1 deletion lib/vips/version.rb
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
module VIPS
VERSION = "0.3.13"
VERSION = "0.3.14"
end
26 changes: 14 additions & 12 deletions lib/vips/writer.rb
Original file line number Diff line number Diff line change
Expand Up @@ -6,29 +6,31 @@ def write(path)

private

# write can fail due to no file descriptors
# write can fail due to no file descriptors, memory can fill if large
# objects are not collected fairly soon
#
# we can't try a write and GC and retry on fail, since the write may take a
# long time
#
# GCing before every write would have a horrible effect on performance, so
# as a compromise we GC every @@gc_interval writes

# ruby2.1 introduced a generational GC which can do a low-impact GC ... we
# behave differently for this
#
# ruby2.1 introduced a generational GC which
# is fast enough to be able to GC on every write

@@generational_gc = RUBY_ENGINE == "ruby" && RUBY_VERSION.to_f >= 2.1

@@gc_interval = if @@generational_gc then 10 else 100 end
@@gc_interval = 100
@@gc_countdown = @@gc_interval

def write_gc(path)
@@gc_countdown -= 1
if @@gc_countdown < 0
@@gc_countdown = @@gc_interval
if @@generational_gc
GC.start full_mark: false, immediate_sweep: false
else
GC.start
if @@generational_gc
GC.start full_mark: false
else
@@gc_countdown -= 1
if @@gc_countdown < 0
@@gc_countdown = @@gc_interval
GC.start
end
end

Expand Down

0 comments on commit 1b75ac8

Please sign in to comment.