Category: Puppet

Bootstrap your home folder. Puppet!

Need to log in to a lot of different systems but hate setting your environment up each time? Keeeeeeeep adding your ssh key whenever you log in the first time? Or worse, regret adding it the previous time you logged in over and over?
Feel like its dirty to put your personal setup in the company wide puppetmaster?

I use this.

echo $( wget -q -O - http://YOUR_URL_HERE/homedir.pp; echo "class {'homedir::jan': gid => '10001',}" ) | puppet apply

Aahhhhhh, one copy-paste-able to rule them all.

Puppet: notes on using defined() and class scope.

I was debugging a little problem just today and figured out that defined(Class[‘something’]) would return true if in the current scope, there is a class something.
Example:

class foo {
  notify{'I am class foo': }
}
class bar::foo {
  notify {'I am class bar::foo': }
  if ! defined(Class['foo']) {
    notify {'foo was not declared yet. do it!': }
    include foo
  }
}
include bar::foo

This results in 

Notice: I am class bar::foo
Notice: /Stage[main]/Bar::Foo/Notify[I am class bar::foo]/message: defined 'message' as 'I am class bar::foo'

Not quite what I expected. I added some debug statements in the defined function and figured out that he resolved Class[‘foo’] to Class[‘bar::foo’].
After this, It was pretty easy to fix. Also note that you need to add the ‘::’ when including foo too!

class foo {
  notify{'I am class foo': }
}
class bar::foo {
  if ! defined(Class['::foo']) {
    notify {'foo was not declared yet. do it!': }
    include ::foo
  }
}
Notice: foo was not declared yet. do it!
Notice: I am class foo
Notice: I am class bar::foo

HURRAY! So, as a general rule, always ::scope everything where you can ;)

Vagrant: Using the shell provider for running puppet.

The case for…

Why would you use the shell provider instead of the native puppet support?

Sometimes you want to tweak your base-box before running puppet, in that case, using a shell script might be a good idea. I started using the shell provider for deploying a puppetmaster. This way, I can initially bootstrap the puppetmaster using puppet apply and then have further configuration done by letting further configuration be done by just running puppet agent like I would in an actual environment.

The second advantage is that I sync my complete puppet tree to /etc/puppet vagrant box, making the differences with an actual deploy even smaller. If you need custom configuration files, you can use the proper file paths while developing and/or put them in your puppet folder.
(more…)

Why not to use Puppet::Parser::Functions.autoloader.loadall

Recently (about 5 minutes ago), I was writing a custom puppet-function to offload some puppet magic. In short: I’m writing a wrapper around create_resources so I can keep syntax for the end-users of my module crispy clean. This means I need the create_resources function to be available in my custom function. This can be done by using Puppet::Parser::Functions.autoloader.loadall as suggested on the puppetlabs custom modules guide. Unfortunately, when using #loadall, all functions will be loaded.

Why unfortunately? In my case: A function defined in puppet-foreman depends on the rest-client gem and I do not have this installed. Some people might say: Just install the gem and be done with it! This is hardly a proper solution. The way to go would to be only include the function I really  need, being create_resources.

And here is how:

Puppet::Parser::Functions.autoloader.load(:create_resources) unless Puppet::Parser::Functions.autoloader.loaded?(:create_resources)

This will basically load the create_resources function after checking that it has not been loaded before. This (the function already being loaded) could be the case if you properly depend on puppetlabs-create_resources in your manifests. Side note: I added a small dummy class so my modules can depend on this function being available.

This has resolved my issues with #loadall, but if I ever needed to include another function that DOES use #loadall, I’ll be screwed all over again. So (pretty) pls, don’t use #loadall.

Puppet Module Patterns

INTRODUCTION

I’ve used puppet quite intensively since a couple of months (about 4 I would guess). Before that, I’ve played with it, change something here and there. But quite not as much as now. I’ve used several puppet modules from wherever google leads me, roamed github, inherited a few from colleagues and created several from scratch. While doing so, I saw a lot of stuff I disliked and learned a lot on how we I can (ab)use puppet to do what I want it to do. Over those last months, I have grown my set of ideas on how a puppet module should look. So, before every statement I make, you should probably add ‘IMHO’.

(more…)

Puppet modules in Jenkins.

Code style checking

Prerequisites:

  • You will need a recent enough version of puppet-lint that supports the --log-format flag. Install the gem so that the Jenkins can use it.
  • On Jenkins, you will need the Warnings Plugin and the HTML Publisher Plugin.
  • Make sure that when checking the module from your VCS, it ends up in WORKSPACE/modules/module_name.

Configuration:

Jenkins

Go to the Configure System page and find the Compiler Warnings settings. Add a new console log parser and call it puppet-lint. I use following configuration for parsing puppet-lint warnings and errors.

The warnings plugin has been updated and now has puppet-lint support out of the box! So configuring puppet-lint manually is kind of useless now.

Name:

puppet-lint

Regular Expression:

^\s*([^:]+):([0-9]+):([^:]+):([^:]+):\s*(.*)$

Mapping Script:

import hudson.plugins.warnings.parser.Warning
// map regular expression to strings
String fileName = matcher.group(1);
String lineNumber = matcher.group(2);
String kind = matcher.group(3);
String check = matcher.group(4);
String message = matcher.group(5);
// return a Warning.
return new Warning(fileName, Integer.parseInt(lineNumber), check, kind, message);

Example Log Message:

./manifests/params.pp:25:autoloader_layout:error:apache::params not in autoload module layout

Jenkins job configuration

We will add several build steps that will run certain actions on our puppet modules.

  1. Check syntax
  2. Check style
  3. Generate documentation

1. For the syntax check, I use following shell script (add a build step):

for file in $(find . -iname '*.pp'); do
  puppet parser validate --color false --render-as s --modulepath=modules $file || exit 1;
done;

2. For the style check, we use puppet-lint (add another build step):

find . -iname *.pp -exec puppet-lint --log-format "%{path}:%{linenumber}:%{check}:%{KIND}:%{message}" {} \;

3. And for generating documentation:

## Cleanup old docs.
[ -d doc/ ] && rm -rf doc/
## Dummy manifests folder.
! [ -d manifests/ ] && mkdir manifests/
## Generate docs
puppet doc --mode rdoc --manifestdir manifests/ --modulepath ./modules/ --outputdir doc
 
## Fix docs to how I want them, I don't like that the complete workspace is included in all file paths.
if [ -d ${WORKSPACE}/doc/files/${WORKSPACE}/modules ]; then
  mv -v "${WORKSPACE}/doc/files/${WORKSPACE}/modules" "${WORKSPACE}/doc/files/modules"
fi;
grep -l -R ${WORKSPACE} * | while read fname; do sed -i "s@${WORKSPACE}/@/@g" $fname; done;

In your post build section:

  • Enable Scan for compiler warnings and select puppet-lint.
  • Enable publish HTML reports (use ‘doc‘, ‘index.html‘ and ‘Puppet Docs‘ as values). This will add a link to the Job page linking your generated puppet docs.

That’s about it! Any suggestions / improvements on this are always welcome!

Notes:

  • I have some examples/tests setup on my Jenkins instance for testing at http://jenkins.vstone.eu. Since I use this for testing, it might be offline / broken / buggy at times.
  • The scripts I use may also require some changes if you are using an older version of puppet. I’m currently using 2.7.x for testing my modules.

Vagrant quickstart for Puppet dev(op)s

A quick introduction on how I use vagrant for developing my puppet manifests/modules/. You can almost certain also use this for other purposes. In general, this will get you up to speed fast!
We will quickly go over installation and/or updating and maybe even removing an old version using ruby gems.
Furthermore: adding a vagrant box and preparing a project to develop a puppet module (or something of the likes).
(more…)