GitHub pull requests as seen by GIT

First off, some links to github documentation:

For just working with pull requests and even manually fetching / merging pull requests locally this information is more than enough. I found that by adding a new refspec, my branches are getting polluted with a looooong list of pull requests for certain projects. I would rather have a simple quick execute-once way of doing the same thing.

An extremely simple way is by adding a new alias:

git config alias.fpr '!sh -c "git fetch origin pull/${1}/head:pr/${1}" -'

This allows you to use git fpr 696 to pull in #696. Mind you, the remote is hardcoded to ‘origin‘ here.

In some cases, I’m really inpatient towards pull requests on upstream repositories and I want to merge them in my local repository more easily.
I created the following script for just that: git-merge-pr.sh.

A quick example:

$ ./git-merge-pr.sh 696
$ git log -n1
commit c0f62a0b1ed09ce928d35a90854a1eb2c2ac6d10
Merge: 1d837ed 54988e1
Author: Jan Vansteenkiste 
Date:   Tue May 6 06:48:43 2014 +0200

    Merged pull request: #696: Bugfix/debian control file permissions
    
    See jordansissel/fpm#696 for more information
    
    * pr/696:
      Ignore lsb init scripts warning since we are not providing a proper init script for testing
      Feeling brave, enable lintian for travis
      Fix file permissions on deb control files
    
    Conflicts:
    	.travis.yml

Diffing gems in GIT.

Recently, I’ve been doing some patching on fpm and today I also started writing some tests for the changes I made. One change required that the gem in the repository used for testing had a bin in it so I modified the .gemspec and rebuild it.

After building and about to push it, I thought on how it must seem to the maintainer when somebody sends a pull request for a binary blob. I would like to know what really changed in there without having to go checkout the internals myself. How wonderful would life be if we could just use git diff to show the differences.

Here is a way to do just that using .gitattributes and a little shell script:

misc/gemdiff.sh:

#!/bin/bash
echo "============== metadata =============="
tar -xOf $1 metadata.gz 2>/dev/null | gunzip -c 2>/dev/null
echo "============== checksums ============="
tar -xOf $1 checksums.yaml.gz 2>/dev/null | gunzip -c 2>/dev/null
echo "=============== files ================"
tar -xOf $1 data.tar.gz 2>/dev/null | tar -xvOzf - 2>&1

.gitattributes:

*.gem diff=gemdiff

.git/config:

[diff "gemdiff"]
    textconv = misc/gemdiff.sh

The result looks like this:

diff --git a/spec/fixtures/gem/example/example-1.0.gem b/spec/fixtures/gem/example/example-1.0.gem
index 0241779..f762e52 100644
--- a/spec/fixtures/gem/example/example-1.0.gem
+++ b/spec/fixtures/gem/example/example-1.0.gem
@@ -3,19 +3,17 @@
 name: example
 version: !ruby/object:Gem::Version
   version: '1.0'
-  prerelease: 
 platform: ruby
 authors:
 - sample author
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2012-03-15 00:00:00.000000000 Z
+date: 2014-05-01 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: dependency1
   requirement: !ruby/object:Gem::Requirement
-    none: false
     requirements:
     - - ! '>='
       - !ruby/object:Gem::Version
@@ -23,7 +21,6 @@ dependencies:
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
-    none: false
     requirements:
     - - ! '>='
       - !ruby/object:Gem::Version
@@ -31,7 +28,6 @@ dependencies:
 - !ruby/object:Gem::Dependency
   name: dependency2
   requirement: !ruby/object:Gem::Requirement
-    none: false
     requirements:
     - - ! '>='
       - !ruby/object:Gem::Version
@@ -39,42 +35,59 @@ dependencies:
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
-    none: false
     requirements:
     - - ! '>='
       - !ruby/object:Gem::Version
         version: '0'
 description: sample description
 email: sample email
-executables: []
+executables:
+- example
 extensions: []
 extra_rdoc_files: []
-files: []
+files:
+- bin/example
 homepage: http://sample-url/
 licenses: []
+metadata: {}
 post_install_message: 
 rdoc_options: []
 require_paths:
 - lib
 required_ruby_version: !ruby/object:Gem::Requirement
-  none: false
   requirements:
   - - ! '>='
     - !ruby/object:Gem::Version
       version: '0'
 required_rubygems_version: !ruby/object:Gem::Requirement
-  none: false
   requirements:
   - - ! '>='
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 1.8.18
+rubygems_version: 2.0.14
 signing_key: 
-specification_version: 3
+specification_version: 4
 summary: sample summary
 test_files: []
 has_rdoc: 
 ============== checksums =============
+---
+!binary "U0hBMQ==":
+  metadata.gz: !binary |-
+    MjYwNGQ5MDZjYTE0MjY5MWQyZTA5Yzk0MjgyYjk2ZGM0ZTk3YzE3Mw==
+  data.tar.gz: !binary |-
+    YmZmYzJlNDU0ZGNmMDEzZjJmZGExYWZiYWE2ZjBjYTQ2MTMzYzFkNA==
+!binary "U0hBNTEy":
+  metadata.gz: !binary |-
+    MzYyYTVlNDk0ODRiNzdhNDBjOWE0ZmEyOGM0NjAzNzg3M2VlZTA4MzgzZDAw
+    ZjVhMjc5ZjFjYTIzOTk2MWFhYjZmMjNiNmVkNzRlMzMzMTdiOTMxMzlmM2Nl
+    ZGE3ZDc5ZDRkNjUwNDE1ODkzNDkxNDhmNmI5YmUyYjg2NjEwMjk=
+  data.tar.gz: !binary |-
+    MGY3NzllNTgwOGQ2YzhmOGUwNjlkMTk5NTlhOTIzZjJkZTkyMjdiNzQxZDQ3
+    NDc4ZjU5NGU1YTA4ODc1NzQwNTc3OWNlOWZkZjMwNWFmMDE3MWE4ZDUzNGY3
+    NDE0NGVmMjA4MDM1MDA4ZjdiMzliNWIwYTNiNDVkMWExZmIyYTY=
 =============== files ================
+bin/example
+#!/usr/bin/env ruby

Life.wonderfulness++;

Working with git submodules: tips ‘n tricks

Some people hate it, nobody loves it, but it’s a good way to split codebase in different components/repositories.

I have been using submodules a LOT for puppet development (all those puppet modules…). Some people might propose alternatives (puppet-tree, librarian), but I rather stick with what I already know.

Dealing with submodules in git is mainly painful because the parent repository doesn’t really know/care what is inside the submodule. He only keeps track of the hash that links the commit. Another downside is that your submodules mostly always end up in a detached state and after checking out a branch, you kinda forget on what commit the parent repository has.

You can put them in your ~/.gitconfig file in the alias section:

git tags

Little different from the default git tag: Uses sort to do natural sort with version numbers. Note, your sort version must be new enough.

tags = !sh -c 'git tag | sort -V'

git update

Run in the root of the ‘parent’ repository

update = !sh -c 'git pull && git fetch --tags && git submodule update --recursive && git submodule foreach git tag -f parent-$(git describe --contains --all HEAD)'
  1. Pull from the remote
  2. Fetch remote tags
  3. Update submodules (recursive)
  4. Create a tag on each submodule called parent-BRANCH with BRANCH being the branch the current parent repository is on

git noparent

Removes the parent-* tags from all repositories (recursive).

noparent = !sh -c 'git tag -d $(git tag | grep ^parent ) &&  git submodule foreach git noparent'
  1. Remove all tags matching ^parent
  2. Do the same for each submodule (recursive)

git safepush

Remove parent tags, make sure we don’t create a merge commit and push.

safepush = !sh -c 'git noparent && git pull --rebase && git push && git push --tags'
  1. Remove parent tags, we don’t want to push them by accident
  2. Fetch remote changes and rebase
  3. Push push push!

git pushtags

Remove parent tags and push all the tags.

pushtags = !sh -c 'git noparent && git push --tags'
  1. Remove the parent tags we have set
  2. Push tags

Bootstrap your home folder. Puppet!

Need to log in to a lot of different systems but hate setting your environment up each time? Keeeeeeeep adding your ssh key whenever you log in the first time? Or worse, regret adding it the previous time you logged in over and over?
Feel like its dirty to put your personal setup in the company wide puppetmaster?

I use this.

echo $( wget -q -O - http://YOUR_URL_HERE/homedir.pp; echo "class {'homedir::jan': gid => '10001',}" ) | puppet apply

Aahhhhhh, one copy-paste-able to rule them all.

Puppet: notes on using defined() and class scope.

I was debugging a little problem just today and figured out that defined(Class[‘something’]) would return true if in the current scope, there is a class something.
Example:

class foo {
  notify{'I am class foo': }
}
class bar::foo {
  notify {'I am class bar::foo': }
  if ! defined(Class['foo']) {
    notify {'foo was not declared yet. do it!': }
    include foo
  }
}
include bar::foo

This results in 

Notice: I am class bar::foo
Notice: /Stage[main]/Bar::Foo/Notify[I am class bar::foo]/message: defined 'message' as 'I am class bar::foo'

Not quite what I expected. I added some debug statements in the defined function and figured out that he resolved Class[‘foo’] to Class[‘bar::foo’].
After this, It was pretty easy to fix. Also note that you need to add the ‘::’ when including foo too!

class foo {
  notify{'I am class foo': }
}
class bar::foo {
  if ! defined(Class['::foo']) {
    notify {'foo was not declared yet. do it!': }
    include ::foo
  }
}
Notice: foo was not declared yet. do it!
Notice: I am class foo
Notice: I am class bar::foo

HURRAY! So, as a general rule, always ::scope everything where you can ;)

Instant Messaging @ Work. Do / don’t

If you, like me, use instant messaging a lot to contact colleagues, you will probably recognize these kind of conversations. The great thing about using something text based is that you can still keep working on other stuff. I tend to have plenty terminals open and using a IM allows me to ‘not sit around waiting’ until the other party replies. You can argue that giving somebody a call is a lot faster, but I’ll be losing focus on the conversation anyhow as long as I have a screen in front of me. Thats bad for both of us.

This is a common conversation that sometimes annoys me. Especially if you are the third person or so who initiates it.

Somebody: Hi!
Me: Hi
Somebody: Good morning
Me: ‘Mornin
Somebody: How are you doing?
Me: Fine, you?
Somebody: Ok.
Somebody: Can I ask you something?
Somebody: Or is this a bad time?
Me: Shoot.
Somebody: …. <Question follows>

If you are like me, this is not the kind of conversation I want to have with everybody that needs something.
The following example is how most conversations go with the colleagues in the sysadmins team:

Somebody: ping
Me: pong
Somebody: …. <Question follows>

As you can see, the amount of interrupts is reduced dramatically. “But that’s not very friendly” you say? No need for, I know you are a nice dude and even if you are not, we are working together, right?

So, for all you “friendly” people out there on the work-floor: One rule to live by: Don’t ask to ask, just ask! Keep the friendly talk for the coffee machine which we will both be using frequently anyhow.

Vagrant: Using the shell provider for running puppet.

The case for…

Why would you use the shell provider instead of the native puppet support?

Sometimes you want to tweak your base-box before running puppet, in that case, using a shell script might be a good idea. I started using the shell provider for deploying a puppetmaster. This way, I can initially bootstrap the puppetmaster using puppet apply and then have further configuration done by letting further configuration be done by just running puppet agent like I would in an actual environment.

The second advantage is that I sync my complete puppet tree to /etc/puppet vagrant box, making the differences with an actual deploy even smaller. If you need custom configuration files, you can use the proper file paths while developing and/or put them in your puppet folder.
(more…)

Vagrant and Virtualbox: a debugging story

Recently, I ran into VirtualBox bug #10077:APIC Bug. I wanted to help out so I had to enable console logging to my machine to give useful output. For that same reason, I started setting console=ttyS0 console=tty0 ignore_loglevel to kernel options in grub.cfg on my vagrant-baseboxes.

The apic bug did not occur on every startup so I had to do a lot of them before I got it right. This tempted me to get the console redirect feature of VirtualBox working from within a Vagrantfile. Well, in the end, it’s not that hard… 

config.vm.define :base6 do |base_config|
    base_config.vm.customize [
      "modifyvm", :id, "--name", "CentOS 6 x86_64 Base",
      "--uart1", "0x3F8", "4", 
      "--uartmode1", "file", "/tmp/base6-console.log"
    ]
end

Note that this file gets overwritten every time you vagrant up your box. So if you want time stamped logs, you’ll have to introduce some magic (do let me know if you do ;))