Repeated agent runs should generate identical config files (concat ordering)

[hammondr]

Each time my agents run, the icinga2 module will reorder lines in various config files. This triggers a refresh of the agent (not a big deal) but also indicates the puppet run made changes (incorrect).

Examples from an agent run

Notice: /Stage[main]/Icinga2::Feature::Api/Icinga2::Object::Endpoint[NodeName]/Icinga2::Object[icinga2::object::Endpoint::NodeName]/Concat[/etc/icinga2/zones.conf]/File[/etc/icinga2/zones.conf]/content:
--- /etc/icinga2/zones.conf	2017-06-30 11:30:34.377003794 +0000
+++ /tmp/puppet-file20170630-13169-cy7hxu-0	2017-06-30 11:57:50.556778800 +0000
@@ -12,8 +12,8 @@
 }

 object Zone ZoneName {
-  parent = "master"
   endpoints = [ NodeName, ]
+  parent = "master"
 }

 object Zone "global-templates"  {

Notice: /Stage[main]/Icinga2::Feature::Mainlog/Icinga2::Object[icinga2::object::FileLogger::mainlog]/Concat[/etc/icinga2/features-available/mainlog.conf]/File[/etc/icinga2/features-available/mainlog.conf]/content:
--- /etc/icinga2/features-available/mainlog.conf	2017-06-30 10:27:16.540731299 +0000
+++ /tmp/puppet-file20170630-13169-12g9e5l-0	2017-06-30 11:57:33.068056880 +0000
@@ -1,6 +1,6 @@
 # This file is managed by Puppet. DO NOT EDIT.

 object FileLogger "main-log"  {
-  severity = "information"
   path = "/var/log/icinga2/icinga2.log"
+  severity = "information"
 }

Expected Behavior

icinga2 config files only change when necessary.

Current Behavior

icinga2 config files are changed (by reordering of directives / stanzas), causing a service restart and causing the puppet agent run to indicate a change was made.

Possible Solution

I think these files are generated using the concat module. If so, specify ordering of the contents so files are only changed when necessary.

Steps to Reproduce (for bugs)

1. configure icinga2 module
2. run on agent
3. run on agent again
4. config files are changed (content is reordered) and service is restarted

Context

The restart of the icinga2 agent is not a huge deal, but flagging the puppet run as a change means every run is a change. This hurts our ability to find 'real' changes that should be investigated further.

Your Environment

• Module version (puppet module list):
  /etc/puppet/modules
  ├── Aethylred-postfix (v0.1.1)
  ├── AlexCline-fstab (v0.3.0)
  ├── _deprecated (???)
  ├── abstractit-monitoring (v1.2.2)
  ├── abstractit-nrpe (v1.1.3)
  ├── argo (???)
  ├── bjoern-ossec (v0.0.1)
  ├── bootstrap (???)
  ├── certdeploy (???)
  ├── ddf (???)
  ├── default_firewall (???)
  ├── dgutierrez1287-centrify (v0.1.0)
  ├── dib40 (???)
  ├── duritong-sysctl (v0.0.11)
  ├── epel (???)
  ├── erwbgy-limits (v0.3.1)
  ├── example42-puppi (v2.2.1)
  ├── example42-yum (v2.1.28)
  ├── hardening (???)
  ├── herculesteam-augeasproviders_core (v2.1.2)
  ├── herculesteam-augeasproviders_pam (v2.1.0)
  ├── hieradata (???)
  ├── icinga-icinga2 (v1.1.0)
  ├── jc2cui (???)
  ├── jdowning-awscli (v1.3.0)
  ├── joshcooper-powershell (v0.0.6)
  ├── kemra102-auditd (v2.2.0)
  ├── mapwidget-loader (???)
  ├── mcafee_av (???)
  ├── nanliu-staging (v1.0.4)
  ├── netrc (???)
  ├── nzin-ossec (???)
  ├── openam (???)
  ├── opendj (???)
  ├── openjdk7 (???)
  ├── opentable-download_file (v0.0.2)
  ├── opentable-nsclient (v0.0.2)
  ├── ozone (???)
  ├── ozone-mapwidget (???)
  ├── petems-swap_file (v3.1.4)
  ├── profile (???)
  ├── puppet-nexus-master (???)
  ├── puppet-selinux (???)
  ├── puppetlabs-apache (v1.11.0)
  ├── puppetlabs-apt (v4.1.0) invalid
  ├── puppetlabs-concat (v2.1.0) invalid
  ├── puppetlabs-firewall (v1.8.2)
  ├── puppetlabs-firewall (v1.8.1)
  ├── puppetlabs-inifile (v1.6.0)
  ├── puppetlabs-java_ks (v1.4.1)
  ├── puppetlabs-mongodb (v0.11.0)
  ├── puppetlabs-mysql (v3.4.0)
  ├── puppetlabs-ntp (v4.0.0)
  ├── puppetlabs-postgresql (v4.9.0)
  ├── puppetlabs-puppetdb (v5.2.0)
  ├── puppetlabs-rsync (v0.4.0)
  ├── puppetlabs-stdlib (v4.17.0) invalid
  ├── puppetlabs-vcsrepo (v1.3.0)
  ├── puppetlabs-xinetd (v1.5.0)
  ├── qpid (???)
  ├── role (???)
  ├── saz-dnsmasq (v1.0.1)
  ├── saz-memcached (v2.8.1)
  ├── saz-resolv_conf (v1.0.4)
  ├── saz-ssh (v2.8.1)
  ├── saz-sudo (v3.0.1)
  ├── saz-timezone (v2.0.0)
  ├── schrepfler-jdk_oracle (v1.0.8)
  ├── selinux (???)
  ├── seteam-splunk (v3.0.1)
  ├── sharumpe-tcpwrappers (v1.0.2)
  ├── stackforge-ceph (v1.0.0)
  ├── stephenrjohnson-puppet (v1.4.0)
  ├── tomcat (???)
  ├── tomcat7_rhel (???)
  ├── torrancew-cron (v0.2.1)
  ├── widget (???)
  └── yum_groupinstall (???)
• Puppet version (puppet -V): 3.8.7
• Operating System and version: CentOS 6.9 (master and agent)

[lazyfrosch]

This is a problem on Ruby 1.8, which comes with CentOS 6.

Please use a more modern Puppet, or at least Ruby, this is not fixable on our side!

See:

• current README.md
• [dev.icinga.com #13179] Ruby 1.8 testing #146 (comment)
• https://tickets.puppetlabs.com/browse/PUP-1755

[hammondr]

Sorry for the very late reply. Which puppet components need to run ruby >= 1.9 for the ordering to work? The puppetmaster process (e.g. via rvm'd passenger) or the agent, or both?

[SimonHoenscheid]

If you update to puppet >=4, you get a bundled ruby version. This might be less painful.

[hammondr]

Thanks. We do understand that and we desperately want to move to puppet 5 but we are looking for a way to get this functionality before then. For us -- if it solved the problem -- using RVM with puppet 3 is much easier than migrating to puppet 4.

[lbetz]

The requirement of Ruby >=1.9 is for the master servers only. To parse the icinga2 configuration we use a function and function are just executed on a master. But note if you use puppet apply on an 'agent' it's simulate a master and then you have to install ruby >=1.9 on this host too.