Cannot get new Hiera module_data to work correctly

[frenkel]

Hi,

I followed you instructions regarding Hiera, but I cannot get it to work on an OpenBSD server. I know that OpenBSD is not supported yet because there is pending pull request, but I want to convert that pull request to the new module_data Hiera structure.

These are the steps I followed and the errors I get.

puppet module install jfryman-nginx --modulepath ./modules

I modified /etc/hiera.yaml to look like this:

# $OpenBSD: hiera.yaml,v 1.3 2013/06/11 14:51:30 jasper Exp $

---
:backends:
  - yaml
  - module_data
:hierarchy:
  - defaults
  - '%{clientcert}'
  - '%{environment}'
  - global

:yaml:
# datadir is empty here, so hiera uses its defaults:
# - /var/db/hiera on *nix
# - %CommonAppData%\PuppetLabs\hiera\var on Windows
# When specifying a datadir, make sure the directory exists.
  :datadir: /etc/puppet/hieradata`

Created a symlink from /etc/puppet/hiera.yaml to /etc/hiera.yaml:

ln -s /etc/hiera.yaml /etc/puppet/hiera.yaml

I modified my site.pp to look like this:

node default {
  nginx::resource::vhost { $hostname:
    ensure => present,
  }
}

Now when I run puppet apply --modulepath modules manifests/site.pp --verbose --debug --noop I receive the following errors:

Info: Loading facts in /etc/puppet/modules/stdlib/lib/facter/facter_dot_d.rb
Info: Loading facts in /etc/puppet/modules/stdlib/lib/facter/pe_version.rb
Info: Loading facts in /etc/puppet/modules/stdlib/lib/facter/puppet_vardir.rb
Info: Loading facts in /etc/puppet/modules/stdlib/lib/facter/root_home.rb
Info: Loading facts in /etc/puppet/modules/apt/lib/facter/apt_package_updates.rb
Info: Loading facts in /etc/puppet/modules/apt/lib/facter/apt_security_updates.rb
Info: Loading facts in /etc/puppet/modules/apt/lib/facter/apt_updates.rb
Info: Loading facts in /etc/puppet/modules/concat/lib/facter/concat_basedir.rb
Debug: importing '/etc/puppet/modules/nginx/manifests/init.pp' in environment production
Debug: importing '/etc/puppet/modules/nginx/manifests/resource/vhost.pp' in environment production
Debug: Automatically imported nginx::resource::vhost from nginx/resource/vhost into production
Debug: importing '/etc/puppet/modules/nginx/manifests/config.pp' in environment production
Debug: Automatically imported nginx::config from nginx/config into production
Warning: Scope(Nginx::Resource::Vhost[puffy]): Could not look up qualified variable 'nginx::config::spdy'; class nginx::config has not been evaluated
Warning: Scope(Nginx::Resource::Vhost[puffy]): Could not look up qualified variable 'nginx::config::proxy_read_timeout'; class nginx::config has not been evaluated
Warning: Scope(Nginx::Resource::Vhost[puffy]): Could not look up qualified variable 'nginx::config::proxy_connect_timeout'; class nginx::config has not been evaluated
Warning: Scope(Nginx::Resource::Vhost[puffy]): Could not look up qualified variable 'nginx::config::conf_dir'; class nginx::config has not been evaluated
Warning: Scope(Nginx::Resource::Vhost[puffy]): Could not look up qualified variable 'nginx::config::global_owner'; class nginx::config has not been evaluated
Warning: Scope(Nginx::Resource::Vhost[puffy]): Could not look up qualified variable 'nginx::config::global_group'; class nginx::config has not been evaluated
Warning: Scope(Nginx::Resource::Vhost[puffy]): Could not look up qualified variable 'nginx::config::global_mode'; class nginx::config has not been evaluated
Error:  is not valid. It should be 4 digits (0644 by default). at /etc/puppet/modules/nginx/manifests/resource/vhost.pp:418 on node example.com
Error:  is not valid. It should be 4 digits (0644 by default). at /etc/puppet/modules/nginx/manifests/resource/vhost.pp:418 on node example.com

I'm using puppet version 3.4.2 and 1.3.1. Any ideas what is wrong? Am I doing something wrong?

[jfryman]

Hmm... that is very odd. Is pluginsync enabled?

[frenkel]

Ah, my email was not added as a reply, sorry.
This is using a masterless setup, with all needed files in /etc/puppet/.The command I run is in the original report.

[jfryman]

@frenkel Please try by adding --pluginsync to the command.

[frenkel]

Same problem and same output unfortunately.

[drdaeman]

Just a wild suggestion: see whenever there's a complaint hiera(): Cannot load backend module_data: no such file to load -- hiera/backend/module_data_backend somewhere in your logs.

For me (Puppet 3.4.3 and 3.7.1 on Ubuntu, installed from official PuppetLabs APT repo) the issue was that Hiera's unaware about modules/module_data/lib. This seems to be not puppet-nginx or puppet-module-data but overall Puppet/Hiera issue.

Hiera loads its modules with plain and simple require, and while I haven't found any places where Puppet sets up $LOAD_PATH to include modules. Facter works, because Puppet properly sets paths for it into indirector/facts/facter.rb, but I haven't encountered anything like this for Hiera. And dumping $LOAD_PATH from Hiera's fault handler shows there are indeed no module dirs in there.

Setting libdir=$confdir/modules/module_data/lib in puppet.conf (a terribly ugly kludge, for sure) solved the issue for me.

This is just a wild guess, but maybe this will be helpful. Sorry if that was misleading or anything.

[frenkel]

Thanks for the hint. Unfortunately, puppet doesn't seem to produce a log when run in master-less mode. Nevertheless, I tried your workaround but it did not solve my problem.

[taylorleese]

I'm also having issues with the hiera defaults. This is the error I receive:

Error: Could not retrieve catalog from remote server: Error 400 on SERVER: $worker_connections must be an integer. at /etc/puppet/modules/nginx/manifests/config.pp:87 on node

[taylorleese]

Ah, I missed the new docs about the changes for hiera.

[taylorleese]

Actually, even with setting - module_data and --pluginsync in Vagrant I'm still getting the same error.

[BcTpe4HbIu]

Solved same issue by creating symlinks in /var/lib/puppet/lib
Like: ln -s /etc/puppet/modules/module_data/lib/* /var/lib/puppet/lib/

[3flex]

I think the issue stems from the way Puppet loads any lib dirs from the modulepath during Puppet bootstrapping. There's a comment and code at https://github.com/puppetlabs/puppet/blob/3.6.2/lib/puppet/util/autoload.rb#L104-L153 which suggests this behavior.

I'm not great at debugging so I haven't traced the full program execution to determine the reason why in some cases it loads and in others it doesn't. But it does explain why @BcTpe4HbIu can symlink and get it working, because it's skipping the limitations of the Puppet autoloader, and @drdaeman got those results.

If someone could confirm and/or take this investigation further that'd be very helpful! But this code in autoloader's been there since 3.0.0 and it hasn't changed in master, so it looks like it will continue to work that way for the foreseeable future.

[frenkel]

Well, for me the problem must be something else, because the symlink trick doesn't fix it. Not even when I start a local puppet master and client.

[BcTpe4HbIu]

Check if libdir is actualy /var/lib/puppet/lib with command sudo puppet config print libdir

[frenkel]

Thanks, but on my system it's /var/puppet/lib, so that was what I tested.

[mkrakowitzer]

I too followed the documentation and could not get this to work.
I can confirm that creating the soft links did work for me however.

Not sure if this is useful info, I am running CentOS 7, puppetserver-0.4.0, r10k with dynamic environments.

[jfryman]

I have actually removed the support for puppet-module-data in the most recent release (0.2.0). Apologies for the inconvenience. Take a look and see if this fixes your issue and let me know.

[jrosental]

Hi,

I'm using latest version (0.2.1) and it still doesn't work.

When I call the 'nginx::resource::vhost' defined type like this:

nginx::resource::vhost { $name:
server_name => $server_name,
www_root => $www_root,
access_log => $access_log,
error_log => $error_log,
proxy => $proxy,
}

I get this error in my puppet client

Error: Could not retrieve catalog from remote server: Error 400 on SERVER: is not valid. It should be 4 digits (0644 by default). at /etc/puppet/environments/testing/modules/nginx/manifests/resource/vhost.pp:418 on node
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run

I thought initially it was a bug with hiera, but even if I don't pass parameters to the defined type and use hard-coded values I get the same output error.

I'm using Debian 7 Wheezy 64 bits with puppet version 3.7.2

[frenkel]

What happens if you use master?

[scottames]

In case anyone runs across this issue, #554 has the fix:

Ok, after looking at #566 it was just some issue loading the proper nginx module.

when i changed 'include nginx' to 'include ::nginx' has resolved all the loading issues