Require base folder for resources #624

Tombar · 2015-05-06T15:15:14Z

Hello James

Not sure if this issue is related to the changes in the concat module, but since it was updated we start having errors like:

==> single: Error: Could not set 'file' on ensure: No such file or directory - /etc/nginx/conf.d/drupal-upstream.conf20150506-4095-ep3q6b.lock
==> single: Error: Could not set 'file' on ensure: No such file or directory - /etc/nginx/conf.d/drupal-upstream.conf20150506-4095-ep3q6b.lock
==> single: Wrapped exception:
==> single: No such file or directory - /etc/nginx/conf.d/drupal-upstream.conf20150506-4095-ep3q6b.lock
==> single: Error: /Stage[main]/Dotdrupal/Dotdrupal::Site[drupal]/Dotdrupal::Nginx[nginx for drupal]/Nginx::Resource::Upstream[drupal]/Concat[/etc/nginx/conf.d/drupal-upstream.conf]/File[/etc/nginx/conf.d/drupal-upstream.conf]/ensure: change from absent to file failed: Could not set 'file' on ensure: No such file or directory - /etc/nginx/conf.d/drupal-upstream.conf20150506-4095-ep3q6b.lock

We weren't having this before :( i've dig into NGINX manifest and find a way to solve them by just requiring the destination folder in the concat resources.

Hope it can be merged

Regards

M

3flex · 2015-05-06T17:46:41Z

manifests/resource/upstream.pp

@@ -70,6 +70,7 @@
  concat { "${::nginx::config::conf_dir}/conf.d/${name}-upstream.conf":
    ensure => $ensure_real,
    notify => Class['::nginx::service'],
+    require => File["${::nginx::config::conf_dir}/conf.d/"]


You're requiring a file here that doesn't exist. You should require File["${::nginx::config::conf_dir}/conf.d"]

Also please ensure the => arrows are aligned.

3flex · 2015-05-06T18:22:24Z

Hi @Tombar did you test that this change in fact solved your issue? The proposed change to upstream.pp would have resulted in an error when you tried in your environment.

Also can I ask which version of concat you were using that failed, and the last version that worked? And your Puppet version?

From what I've read this may be a permissions problem in your environment. Looking at the manifests different users and groups are the owners of the upstream config vs. the rest of the config, which may be causing issues for you.

Tombar · 2015-05-07T13:47:12Z

Hello @3flex

Sorry for the poor issue, being in a hurry yesterday and wanted to submit the PR

I'm having issues both with vhosts and upstreams resources trying to be created before NGINX is even installed.


==> single: Error: Could not set 'file' on ensure: No such file or directory - /etc/nginx/sites-available/zbx_vhost_nginx.conf20150507-3507-rta6nd.lock
==> single: Error: Could not set 'file' on ensure: No such file or directory - /etc/nginx/sites-available/zbx_vhost_nginx.conf20150507-3507-rta6nd.lock
==> single: Wrapped exception:
==> single: No such file or directory - /etc/nginx/sites-available/zbx_vhost_nginx.conf20150507-3507-rta6nd.lock
==> single: Error: /Stage[main]/Dotnginx/Nginx::Resource::Vhost[zbx_vhost_nginx]/Concat[/etc/nginx/sites-available/zbx_vhost_nginx.conf]/File[/etc/nginx/sites-available/zbx_vhost_nginx.conf]/ensure: change from absent to file failed: Could not set 'file' on ensure: No such file or directory - /etc/nginx/sites-available/zbx_vhost_nginx.conf20150507-3507-rta6nd.lock

==> single: Error: Could not set 'file' on ensure: No such file or directory - /etc/nginx/conf.d/drupal-upstream.conf20150507-3507-1ls6qwp.lock
==> single: Error: Could not set 'file' on ensure: No such file or directory - /etc/nginx/conf.d/drupal-upstream.conf20150507-3507-1ls6qwp.lock
==> single: Wrapped exception:
==> single: No such file or directory - /etc/nginx/conf.d/drupal-upstream.conf20150507-3507-1ls6qwp.lock
==> single: Error: /Stage[main]/Dotdrupal/Dotdrupal::Site[drupal]/Dotdrupal::Nginx[nginx for drupal]/Nginx::Resource::Upstream[drupal]/Concat[/etc/nginx/conf.d/drupal-upstream.conf]/File[/etc/nginx/conf.d/drupal-upstream.conf]/ensure: change from absent to file failed: Could not set 'file' on ensure: No such file or directory - /etc/nginx/conf.d/drupal-upstream.conf20150507-3507-1ls6qwp.lock
==> single: Error: Could not set 'file' on ensure: No such file or directory - /etc/nginx/sites-available/drupal.conf20150507-3507-1jz0r7w.lock
==> single: Error: Could not set 'file' on ensure: No such file or directory - /etc/nginx/sites-available/drupal.conf20150507-3507-1jz0r7w.lock
==> single: Wrapped exception:
==> single: No such file or directory - /etc/nginx/sites-available/drupal.conf20150507-3507-1jz0r7w.lock
==> single: Error: /Stage[main]/Dotdrupal/Dotdrupal::Site[drupal]/Dotdrupal::Nginx[nginx for drupal]/Nginx::Resource::Vhost[drupal]/Concat[/etc/nginx/sites-available/drupal.conf]/File[/etc/nginx/sites-available/drupal.conf]/ensure: change from absent to file failed: Could not set 'file' on ensure: No such file or directory - /etc/nginx/sites-available/drupal.conf20150507-3507-1jz0r7w.lock

...
... ...
... ... ... 
==> single: Notice: /Stage[main]/Nginx::Package::Debian/Package[nginx]/ensure: ensure changed 'purged' to 'present'

I can confirm that by adding the requires above the errors are solved, just check it again in a local env.

It maybe related to how we use NGINX module in our manifests, but seems logical for me to require the base folder for both vhost and upstreams or to ensure NGINX is installed before creating resources.

Digging in our manifests, we don't require NGINX to be installed before creating them, but this use to work as expected before.

 include dotnginx

  $nginx = parseyaml(template('dotdrupal/nginx_vhost.yaml.erb'))

  $vhost = $nginx['nginx::nginx_vhosts']
  create_resources('nginx::resource::vhost', $vhost, {})

  $locations = $nginx['nginx::nginx_locations']
  create_resources('nginx::resource::location', merge($locations, hiera('dotdrupal::nginx::extra_locations', {})), {})

  $upstream = $nginx['nginx::nginx_upstreams']
  create_resources('nginx::resource::upstream', $upstream, {})

Regarding versions, i'm testing against latest commit from both https://github.com/jfryman/puppet-nginx and https://github.com/puppetlabs/puppetlabs-concat with puppet version 3.7.4 in a debian7 box

Tombar · 2015-05-07T14:03:17Z

Also puppet-nginx/#610 is having the same issue.

3flex · 2015-05-07T17:18:28Z

Can you try puppetlabs-concat 1.2.1? In that version the standard file resource is used to create the upstream.conf file, and should autorequire the parent directory provided nginx::config is included in the catalogue.

From what I can see this has been changed in concat master branch, now the custom type concat_file is used which doesn't autorequire the parent directory (see https://github.com/puppetlabs/puppetlabs-concat/blob/master/lib/puppet/type/concat_file.rb)

I just want to make sure this gets fixed in the right place.

Tombar · 2015-05-07T17:45:31Z

@3flex just tested it against latest nginx and puppetlabs-concat 1.2.1 and the issues reported don't happen.

Let me know if you want me to perform any other test.

Regards

M

3flex · 2015-05-07T18:22:15Z

It sounds like an issue with puppetlabs-concat. I've raised https://tickets.puppetlabs.com/browse/MODULES-2023

I'll keep this open until there's a resolution of that ticket. For now I'd suggest switching to the Forge version of the puppetlabs-concat module.

3flex · 2015-05-13T16:14:02Z

Unfortunate that my ticket didn't result in any changes before 2.0.0 was released.

@Tombar if you'd be so kind could you please check with concat 2.0.0 both with and without your patch, and let me know results? Thanks!

Tombar · 2015-05-14T20:10:54Z

@3flex give me a couple days more and I will report back my findings.

Regards and thank for your help!

M

SamStonehouse · 2015-05-21T08:32:34Z

Using this repo gives me the following warnings:

==> default: Warning: Scope(Concat::Fragment[localhost-header]): The $ensure parameter to concat::fragment is deprecated and has no effect.
==> default: Warning: Scope(Concat::Fragment[localhost-footer]): The $ensure parameter to concat::fragment is deprecated and has no effect.
==> default: Warning: Scope(Concat::Fragment[851f2f1ae466fdec0f580031f05917d8]): The $ensure parameter to concat::fragment is deprecated and has no effect.
==> default: Warning: Scope(Concat::Fragment[fd0b234173aed6ab4786b7032214c741]): The $ensure parameter to concat::fragment is deprecated and has no effect.

But it does seem to work

Tombar · 2015-05-25T14:10:57Z

Hello @3flex, reporting back

concat module at 2.0.x branch + nginx module at last commit = same errors as reported here.

concat module at 2.0.x branch + nginx module with my PR = works + concat ensure parameter warning since my PR doesn't have this previous PR merged

Sorry for my late response.
Hope this can get merged.

Regards

M

Require base folder for resources

3flex · 2015-06-01T17:15:45Z

Just FYI, will revert this once puppetlabs/puppetlabs-concat#319 is included in a release to Forge.

This commit updates the nginx submodule to the voxpupuli repo (https://github.com/jfryman/puppet-nginx.git is just a reference to the voxpupuli repo, so nothing should change here) and checks out to a commit before the one that reverted the hostfix from voxpupuli/puppet-nginx#624 This fixes the problem with nginx trying to create a vhost configuration before installing the nginx package. This is caused by concat not requiring the underlying nginx file hierarchy. This solution needs to be reverted when concat devs come up with a working implementation of the autorequire function in `lib/puppet/type/concat_file.rb`. The current solution from ba643617a2a03f0588a0ec6625824a707528d9f0 (L112-L115) is not working and the original approach: puppetlabs/puppetlabs-concat#319 has been abandoned at some point. For more details see the ticket: https://datacentred.atlassian.net/browse/PD-2567.

Require base folder for resources

Tombar added 2 commits May 6, 2015 12:03

require conf.d folder

23fe040

require vhost folder

72bb663

3flex reviewed May 6, 2015
View reviewed changes

fix traling slash and aligment

952805a

3flex added needs-feedback Further information is requested bug Something isn't working labels May 7, 2015

3flex removed the needs-feedback Further information is requested label May 7, 2015

3flex mentioned this pull request May 19, 2015

puppet tries to create vhost before nginx is installed? #610

Closed

3flex added a commit that referenced this pull request May 26, 2015

Merge pull request #624 from Tombar/master

796bf63

Require base folder for resources

3flex merged commit 796bf63 into voxpupuli:master May 26, 2015

3flex mentioned this pull request Jun 1, 2015

Invalid parameter: 'key_source' Apt::Source[nginx] at /etc/puppetlabs/code/modules/nginx/manifests/package/debian.pp:37 #629

Closed

3flex mentioned this pull request Jun 13, 2015

Revert "Require base folder for resources" #643

Merged

mkarpiarz mentioned this pull request Sep 20, 2016

[docs] Switch to nginx with working vhosts: datacentred/puppet#2949

Merged

cegeka-jenkins pushed a commit to cegeka/puppet-nginx that referenced this pull request Oct 23, 2017

Merge pull request voxpupuli#624 from Tombar/master

4e9f91b

Require base folder for resources

slm0n87 pushed a commit to slm0n87/puppet-nginx that referenced this pull request Mar 7, 2019

Merge pull request voxpupuli#624 from Tombar/master

8d0da0d

Require base folder for resources

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Require base folder for resources #624

Require base folder for resources #624

Tombar commented May 6, 2015

3flex May 6, 2015

3flex May 6, 2015

3flex commented May 6, 2015

Tombar commented May 7, 2015

Tombar commented May 7, 2015

3flex commented May 7, 2015

Tombar commented May 7, 2015

3flex commented May 7, 2015

3flex commented May 13, 2015

Tombar commented May 14, 2015

SamStonehouse commented May 21, 2015

Tombar commented May 25, 2015

3flex commented Jun 1, 2015

Require base folder for resources #624

Require base folder for resources #624

Conversation

Tombar commented May 6, 2015

3flex May 6, 2015

Choose a reason for hiding this comment

3flex May 6, 2015

Choose a reason for hiding this comment

3flex commented May 6, 2015

Tombar commented May 7, 2015

Tombar commented May 7, 2015

3flex commented May 7, 2015

Tombar commented May 7, 2015

3flex commented May 7, 2015

3flex commented May 13, 2015

Tombar commented May 14, 2015

SamStonehouse commented May 21, 2015

Tombar commented May 25, 2015

3flex commented Jun 1, 2015