UUID from Chef fails on RHEL 7.3 #292

clintoncwolfe · 2018-04-30T16:44:19Z

Description

Reported on inspec/inspec#2985 by @djdees

When trying to read a Chef-generated UUID, the file content detection logic fails and passes a nil to the JSON parser, throwing an exception.

Looks like it's this line:
(https://github.com/chef/train/blame/master/lib/train/platforms/detect/helpers/os_common.rb#L102)

Offhand that seems fine; perhaps there are SELinux constraints preventing us from reading the file?

Train and Platform Version

1.4.1 targeting RHEL 7.3

Introduced on #270

Unknown SELinux status

Replication Case

See inspec/inspec#2985

Possible Solutions

Could check nil? on the file content

Check to verify that the file is really a file (not a dir)

Stacktrace

Traceback (most recent call last):
        30: from /ebiz/tools/ruby/bin/inspec:23:in `<main>'
        29: from /ebiz/tools/ruby/bin/inspec:23:in `load'
        28: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/inspec-2.1.54/bin/inspec:12:in `<top (required)>'
        27: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/thor-0.20.0/lib/thor/base.rb:466:in `start'
        26: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/thor-0.20.0/lib/thor.rb:387:in `dispatch'
        25: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/thor-0.20.0/lib/thor/invocation.rb:126:in `invoke_command'
        24: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/thor-0.20.0/lib/thor/command.rb:27:in `run'
        23: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/inspec-2.1.54/lib/inspec/cli.rb:168:in `exec'
        22: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/inspec-2.1.54/lib/inspec/runner.rb:104:in `run'
        21: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/inspec-2.1.54/lib/inspec/runner.rb:132:in `run_tests'
        20: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/inspec-2.1.54/lib/inspec/runner_rspec.rb:77:in `run'
        19: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/runner.rb:112:in `run_specs'
        18: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/reporter.rb:81:in `report'
        17: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/reporter.rb:171:in `finish'
        16: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/reporter.rb:191:in `close_after'
        15: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/reporter.rb:172:in `block in finish'
        14: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/reporter.rb:199:in `stop'
        13: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/reporter.rb:205:in `notify'
        12: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/reporter.rb:205:in `each'
        11: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/reporter.rb:206:in `block in notify'
        10: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/inspec-2.1.54/lib/inspec/formatters/base.rb:72:in `stop'
         9: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/inspec-2.1.54/lib/inspec/formatters/base.rb:190:in `platform'
         8: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/inspec-2.1.54/lib/resources/platform.rb:41:in `[]'
         7: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/train-1.4.1/lib/train/platforms/platform.rb:52:in `[]'
         6: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/train-1.4.1/lib/train/platforms/platform.rb:45:in `uuid'
         5: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/train-1.4.1/lib/train/platforms/detect/uuid.rb:21:in `find_or_create_uuid'
         4: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/train-1.4.1/lib/train/platforms/detect/helpers/os_common.rb:92:in `unix_uuid'
         3: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/train-1.4.1/lib/train/platforms/detect/helpers/os_common.rb:102:in `unix_uuid_from_chef'
         2: from /ebiz/tools/ruby-2.5.1/lib/ruby/2.5.0/json/common.rb:156:in `parse'
         1: from /ebiz/tools/ruby-2.5.1/lib/ruby/2.5.0/json/common.rb:156:in `new'
/ebiz/tools/ruby-2.5.1/lib/ruby/2.5.0/json/common.rb:156:in `initialize': no implicit conversion of nil into String (TypeError)

The text was updated successfully, but these errors were encountered:

clintoncwolfe · 2018-04-30T16:49:42Z

@djdees , could you tell us a bit more:

Are you using Chef to manage the machine? If so, anything unusual about the path /var/chef/cache/data_collector_metadata.json?
Are you using SElinux, or any other secondary RBAC system which might result in being able to detect but not read the above path?
Are you connecting to the machine with sudo enabled?

Thanks!

djdees · 2018-04-30T19:12:49Z

C. - - Chef is not actively managing the machine. It's used as part of the initial build, but not beyond that. Nothing unusual with that path, the file is there and accessible by my management account. - Nope. - Sudo is enabled and the service account I use has passwordless sudo.

…

On Mon, Apr 30, 2018 at 11:49 AM, Clinton Wolfe ***@***.***> wrote: @djdees <https://github.com/djdees> , could you tell us a bit more: - Are you using Chef to manage the machine? If so, anything unusual about the path /var/chef/cache/data_collector_metadata.json? - Are you using SElinux, or any other secondary RBAC system which might result in being able to detect but not read the above path? - Are you connecting to the machine with sudo enabled? Thanks! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#292 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AIWF7WJySpl2j-d1SYWX-NXSe9_dtu9lks5tt0CogaJpZM4Ts4FT> .

-- Derek ========= [email protected] There is no frigate like a book to take us lands away. Nor any Coursers like a page of prancing poetry. This traverse may the poorest take without oppress of toll. How frugal is the chariot that bears the Human Soul? - Emily Dickinson

rojomisin · 2018-06-29T04:39:22Z

I am seeing this as well consistently now on centos-69 and latest chefdk 3.0.36. I am using test-kitchen ec2 driver on a hardened centos image. I have verified uuidgen is installed on centos, but I am unsure why it's failing as there's no debugging info other than:

[2018-06-28T20:46:44-07:00] DEBUG: Loading controls/main.rb into #<Inspec::ProfileContext:0x00007fab1877e370>
[2018-06-28T20:46:44-07:00] DEBUG: Loading <anonymous content> into #<Inspec::ProfileContext:0x00007fab17b1b678>
[2018-06-28T20:46:54-07:00] ERROR: Cannot find a UUID for your node.

I was thinking of adding a debugging print statement in train, but not sure how that would work with my local chefdk install and gems.

What would my nodes be missing to not be able to generate a uuid?

chris-rock · 2018-06-29T08:11:28Z

@rojomisin The implementation of that features is located here:

train/lib/train/platforms/detect/helpers/os_common.rb

Lines 91 to 121 in 16f70fc

    
           def unix_uuid 
        
             uuid = unix_uuid_from_chef 
        
             uuid = unix_uuid_from_machine_file if uuid.nil? 
        
             uuid = uuid_from_command if uuid.nil? 
        
             raise Train::TransportError, 'Cannot find a UUID for your node.' if uuid.nil? 
        
             uuid 
        
           end 
        
           def unix_uuid_from_chef 
        
             file = @backend.file('/var/chef/cache/data_collector_metadata.json') 
        
             if file.exist? && !file.size.zero? 
        
               json = ::JSON.parse(file.content) 
        
               return json['node_uuid'] if json['node_uuid'] 
        
             end 
        
           end 
        
           def unix_uuid_from_machine_file 
        
             %W( 
        
               /etc/chef/chef_guid 
        
               #{ENV['HOME']}/.chef/chef_guid 
        
               /etc/machine-id 
        
               /var/lib/dbus/machine-id 
        
               /var/db/dbus/machine-id 
        
             ).each do |path| 
        
               file = @backend.file(path) 
        
               next unless file.exist? && !file.size.zero? 
        
               return file.content.chomp if path =~ /guid/ 
        
               return uuid_from_string(file.content.chomp) 
        
             end 
        
             nil 
        
           end

Essentially we try to read the following files:

/etc/chef/chef_guid
#{ENV['HOME']}/.chef/chef_guid
/etc/machine-id
/var/lib/dbus/machine-id
/var/db/dbus/machine-id

@rojomisin could you help us understand if your system has none of those files?
@jquick Can we document how we document how to override the uuid?

This in addition to the issue that the machine-id is not always unique (cloned vms), we may add an easy mechanism to override that?

jquick · 2018-06-29T12:32:52Z

This is a odd one. This error should not be hurting your setup unless your trying to use the A2 report with a non-chef node. We try to attach a UUID here:

https://github.com/inspec/inspec/blob/master/lib/inspec/formatters/base.rb#L192

but if we cannot find one we set to nil and continue. The only time its mandatory is when we are reporting up to A2. You can override it for that case using the documentation here: https://www.inspec.io/docs/reference/reporters/

jquick · 2018-06-29T13:41:03Z

The error message [2018-06-28T20:46:54-07:00] ERROR: Cannot find a UUID for your node. should not be stopping your run at all. @rojomisin can you share your kitchen stack trace? I am assuming the failure is from something else.

jquick · 2018-06-29T13:47:17Z

@djdees @clintoncwolfe What I assume is happening is we have a empty "/var/chef/cache/data_collector_metadata.json" file from the chef startup. We should add a check in train to make sure the file has data before we try to parse it.

rojomisin · 2018-06-29T18:10:47Z

hi @chris-rock @jquick thanks for info.

I'm using test-kitchen + chef-zero so those files are not there. That being said if I do a straight chef exec inspec exec ../profile/ -t ssh://user@<kitchen ip> --sudo --attrs ../attrs.yml it works. But in test kitchen it is reporting the error.

Could it be that the test-kitchen method of generating the uuid on my workstation (os x el cap 10.11.6 ... kinda old)?

train/lib/train/platforms/detect/specifications/os.rb

Lines 439 to 496 in de3ec80

    
           # bsd family 
        
           plat.family('bsd').in_family('unix') 
        
               .detect { 
        
                 # we need a better way to determin this family 
        
                 # for now we are going to just try each platform 
        
                 true 
        
               } 
        
           plat.family('darwin').in_family('bsd') 
        
               .detect { 
        
                 if unix_uname_s =~ /darwin/i 
        
                   cmd = unix_file_contents('/usr/bin/sw_vers') 
        
                   unless cmd.nil? 
        
                     m = cmd.match(/^ProductVersion:\s+(.+)$/) 
        
                     @platform[:release] = m.nil? ? nil : m[1] 
        
                     m = cmd.match(/^BuildVersion:\s+(.+)$/) 
        
                     @platform[:build] = m.nil? ? nil : m[1] 
        
                   end 
        
                   @platform[:release] = unix_uname_r.lines[0].chomp if @platform[:release].nil? 
        
                   @platform[:arch] = unix_uname_m 
        
                   true 
        
                 end 
        
               } 
        
           plat.name('mac_os_x').title('macOS X').in_family('darwin') 
        
               .detect { 
        
                 cmd = unix_file_contents('/System/Library/CoreServices/SystemVersion.plist') 
        
                 @platform[:uuid_command] = "system_profiler SPHardwareDataType | awk '/UUID/ { print $3; }'" 
        
                 true if cmd =~ /Mac OS X/i 
        
               } 
        
           plat.name('darwin').title('Darwin').in_family('darwin') 
        
               .detect { 
        
                 # must be some other type of darwin 
        
                 @platform[:name] = unix_uname_s.lines[0].chomp 
        
                 true 
        
               } 
        
           plat.name('freebsd').title('Freebsd').in_family('bsd') 
        
               .detect { 
        
                 if unix_uname_s =~ /freebsd/i 
        
                   @platform[:name] = unix_uname_s.lines[0].chomp 
        
                   @platform[:release] = unix_uname_r.lines[0].chomp 
        
                   true 
        
                 end 
        
               } 
        
           plat.name('openbsd').title('Openbsd').in_family('bsd') 
        
               .detect { 
        
                 if unix_uname_s =~ /openbsd/i 
        
                   @platform[:name] = unix_uname_s.lines[0].chomp 
        
                   @platform[:release] = unix_uname_r.lines[0].chomp 
        
                   true 
        
                 end 
        
               } 
        
           plat.name('netbsd').title('Netbsd').in_family('bsd') 
        
               .detect { 
        
                 if unix_uname_s =~ /netbsd/i 
        
                   @platform[:name] = unix_uname_s.lines[0].chomp 
        
                   @platform[:release] = unix_uname_r.lines[0].chomp 
        
                   true 
        
                 end 
        
               }

I've been querying this issue in the test-kitchen slack channel a bit too, because it doesn't seem to be inspec.

will post debug output in a bit
the test-kitchen -D does not show any info really

-----> Verifying <default-centos-69>...
       Detected alternative framework tests for `inspec`
       Loaded tests from {:path=>"*"} 
       Loaded config_oss 
[2018-06-29T11:06:46-07:00] ERROR: Cannot find a UUID for your node.

Profile: tests from {:path=>"*"} (tests from {:path=>"*"})
Version: (not specified)
Target:  ssh://[email protected]:22

     No tests executed.

Profile: InSpec Profile (config_oss)
Version: 0.1.9
Target:  ssh://[email protected]:22

     No tests executed.

Profile: InSpec Java in system (java)
Version: 0.0.1
Target:  ssh://[email protected]:22

     No tests executed.

Test Summary: 0 successful, 0 failures, 0 skipped
       Finished verifying <default-centos-69> (0m24.16s).
-----> Kitchen is finished. (8m5.29s)

clintoncwolfe mentioned this issue Apr 30, 2018

JSON Parsing Error when running tests with Inspec 2.1.54. inspec/inspec#2985

Closed

chris-rock added the bug label Jun 29, 2018

clintoncwolfe added Platform: Unix-like labels Jul 13, 2018

clintoncwolfe added Component/UUID Generation Type: Bug Feature not working as expected and removed Component: UUID Generation labels Mar 19, 2019

zenspider added the Triage/Needs Verify label Oct 10, 2019

zenspider added Platform: UNIX-Like Status: Needs Verify and removed Component/UUID Generation labels Nov 15, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

UUID from Chef fails on RHEL 7.3 #292

UUID from Chef fails on RHEL 7.3 #292

clintoncwolfe commented Apr 30, 2018

clintoncwolfe commented Apr 30, 2018

djdees commented Apr 30, 2018 via email

rojomisin commented Jun 29, 2018

chris-rock commented Jun 29, 2018

jquick commented Jun 29, 2018

jquick commented Jun 29, 2018 •

edited

Loading

jquick commented Jun 29, 2018

rojomisin commented Jun 29, 2018 •

edited

Loading

UUID from Chef fails on RHEL 7.3 #292

UUID from Chef fails on RHEL 7.3 #292

Comments

clintoncwolfe commented Apr 30, 2018

Description

Train and Platform Version

Replication Case

Possible Solutions

Stacktrace

clintoncwolfe commented Apr 30, 2018

djdees commented Apr 30, 2018 via email

rojomisin commented Jun 29, 2018

chris-rock commented Jun 29, 2018

jquick commented Jun 29, 2018

jquick commented Jun 29, 2018 • edited Loading

jquick commented Jun 29, 2018

rojomisin commented Jun 29, 2018 • edited Loading

jquick commented Jun 29, 2018 •

edited

Loading

rojomisin commented Jun 29, 2018 •

edited

Loading