supervisor spawn does not carry forward any ruby options passed

[larsf]

In fluentd 0.12 it simply forked itself, and so it was a clone of the process starting up:

@main_pid = fork do
main_process(&block)
end

In fluentd 0.14 it now builds its own exec command and spawns a new process using server engine:
fluentd_spawn_cmd = ServerEngine.ruby_bin_path + " -Eascii-8bit:ascii-8bit "
fluentd_spawn_cmd << $0.shellescape + ' '
$fluentdargv.each{|a|
fluentd_spawn_cmd << (a.shellescape + " ")
}
end

  fluentd_spawn_cmd << ("--no-supervisor")
  $log.info "spawn command to main: " + fluentd_spawn_cmd

  params = {}
  params['main_cmd'] = fluentd_spawn_cmd

Any ruby parameters are lost this way:

If you used bundler with fluentd, the first process that started would get started using:
ruby -r bundler/setup

the -r bundler/setup does not get passed for the command line for the server engine spawn, and the second fluentd process no longer finds the gems it is bundled with, and you will see a lot of this during startup:

016-06-07 15:41:19 -0700 [info]: reading config file path="../etc/fluentd/fluentd.conf"
2016-06-07 15:41:19 -0700 [info]: starting fluentd-0.14.0
2016-06-07 15:41:19 -0700 [info]: spawn command to main: /opt/mapr/fluentd/fluentd-0.14.00/lib/fluentd-0.14.00-linux-x86_64/lib/ruby/bin/ruby -Eascii-8bit:ascii-8bit /opt/mapr/fluentd/fluentd-0.14.00/bin/../lib/fluentd-0.14.00-linux-x86_64/lib/app/bin/fluentd -c ../etc/fluentd/fluentd.conf --no-supervisor
/opt/mapr/fluentd/fluentd-0.14.00/lib/fluentd-0.14.00-linux-x86_64/lib/ruby/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in require': cannot load such file -- yajl (LoadError) from /opt/mapr/fluentd/fluentd-0.14.00/lib/fluentd-0.14.00-linux-x86_64/lib/ruby/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:inrequire'
from /opt/mapr/fluentd/fluentd-0.14.00/lib/fluentd-0.14.00-linux-x86_64/lib/app/lib/fluent/config/literal_parser.rb:20:in <top (required)>' from /opt/mapr/fluentd/fluentd-0.14.00/lib/fluentd-0.14.00-linux-x86_64/lib/ruby/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:inrequire'
from /opt/mapr/fluentd/fluentd-0.14.00/lib/fluentd-0.14.00-linux-x86_64/lib/ruby/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in require' from /opt/mapr/fluentd/fluentd-0.14.00/lib/fluentd-0.14.00-linux-x86_64/lib/app/lib/fluent/config/element.rb:18:in<top (required)>'
from /opt/mapr/fluentd/fluentd-0.14.00/lib/fluentd-0.14.00-linux-x86_64/lib/ruby/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in require' from /opt/mapr/fluentd/fluentd-0.14.00/lib/fluentd-0.14.00-linux-x86_64/lib/ruby/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:inrequire'
from /opt/mapr/fluentd/fluentd-0.14.00/lib/fluentd-0.14.00-linux-x86_64/lib/app/lib/fluent/config.rb:18:in <top (required)>' from /opt/mapr/fluentd/fluentd-0.14.00/lib/fluentd-0.14.00-linux-x86_64/lib/ruby/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:inrequire'
from /opt/mapr/fluentd/fluentd-0.14.00/lib/fluentd-0.14.00-linux-x86_64/lib/ruby/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in require' from /opt/mapr/fluentd/fluentd-0.14.00/lib/fluentd-0.14.00-linux-x86_64/lib/app/lib/fluent/supervisor.rb:20:in<top (required)>'
from /opt/mapr/fluentd/fluentd-0.14.00/lib/fluentd-0.14.00-linux-x86_64/lib/ruby/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in require' from /opt/mapr/fluentd/fluentd-0.14.00/lib/fluentd-0.14.00-linux-x86_64/lib/ruby/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:inrequire'
from /opt/mapr/fluentd/fluentd-0.14.00/lib/fluentd-0.14.00-linux-x86_64/lib/app/lib/fluent/command/fluentd.rb:19:in <top (required)>' from /opt/mapr/fluentd/fluentd-0.14.00/lib/fluentd-0.14.00-linux-x86_64/lib/ruby/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:inrequire'
from /opt/mapr/fluentd/fluentd-0.14.00/lib/fluentd-0.14.00-linux-x86_64/lib/ruby/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in require' from /opt/mapr/fluentd/fluentd-0.14.00/bin/../lib/fluentd-0.14.00-linux-x86_64/lib/app/bin/fluentd:5:in

'
2016-06-07 15:41:19 -0700 [info]: Worker 0 finished unexpectedly with status 1

Adding "-rbundler/setup" to the fluentd_spawn_cmd in app/lib/fluent/supervisor.rb like this:
fluentd_spawn_cmd = ServerEngine.ruby_bin_path + " -Eascii-8bit:ascii-8bit -rbundler/setup "

fixes it, but a more generic solution is required to preserve the ruby options give at start of the original process

[repeatedly]

@naritta Could you check this?

[naritta]

@tagomoris @repeatedly
Is it assumed to use RUBYOPT from command line in fluentd and do we need to pass it to worker process?
If so, I'll fix soon.

[tagomoris]

Yes, it looks work fine > RUBYOPT
I think it's the best to announce to use RUBYOPT for options of ruby, and to fix supervisor.rb to propagate RUBYOPT to worker processes.

[naritta]

Thank you for the comment.
I'll fix in #1066

[repeatedly]

We just merged the patch.
We will release v0.14.1 soon so please wait...