Building Vagrant Machines with Packer

Most Popular

UPDATE: We are delighted that Mitchell Hashimoto, the creator of Vagrant and Packer, shared our blog post and recommended reading it. Retweet to share the love!


Sharing a common development environment with everyone on your team is important. It is really hard though to keep the same dependencies, database versions and other systems in sync between different machines.

Vagrant is a great tool that helps with this and manage the lifecycle of a virtual machine. As nice as Vagrant is, provisioning machines with it has always been a pain. A couple of months ago Mitchell Hashimoto, the creator of Vagrant, launched Packer.

Packer lets you build Virtual Machine Images for different providers from one json file. You can use the same file and commands to build an image on AWS, Digital Ocean or for virtualbox and vagrant. This makes it possible to use exactly the same system for development which you then create in production.

In this blog post we will show you how you can use Packer to build your vagrant machines. In a follow up post we will focus on how we use Packer for building all of our Continuous Deployment Infrastructure.

Prerequisites for building Vagrant Machines

You need Virtualbox and Packer installed. Virtualbox provides packages for different Operating systems. Packer is even easier, just download the right zip for your system and unzip it into your PATH

Building your Virtual Machine with Packer

We’ve collected all the files necessary to build a Vagrant Machine with Packer in our Packer Example repository.

Packer uses builders, provisioners and post-processors as the main configuraition attributes. A builder can for example be virtualbox or AWS. A provisioner can be used to run different scripts. Post-processors can be run after the machine image is done. For example converting a Virtualbox image into a suitable image for vagrant is done in a post-processor.

Here is the main packer.json file. You can see the builder, provisioner and post-processor defined:

{
  "provisioners": [
    {
      "type": "shell",
      "scripts": [
        "scripts/root_setup.sh"
      ],
      "override": {
        "virtualbox": {
          "execute_command": "echo 'vagrant' | sudo -S sh '{{ .Path }}'"
        }
      }
    },
    {
      "type": "shell",
      "scripts": [
        "scripts/setup.sh"
      ]
    }
  ],
  "builders": [
    {
      "type": "virtualbox",
      "boot_command": [
        "<esc><esc><enter><wait>",
        "/install/vmlinuz noapic preseed/url=http://{{ .HTTPIP }}:{{ .HTTPPort }}/preseed.cfg <wait>",
        "debian-installer=en_US auto locale=en_US kbd-chooser/method=us <wait>",
        "hostname={{ .Name }} <wait>",
        "fb=false debconf/frontend=noninteractive <wait>",
        "keyboard-configuration/modelcode=SKIP keyboard-configuration/layout=USA keyboard-configuration/variant=USA console-setup/ask_detect=false <wait>",
        "initrd=/install/initrd.gz -- <enter><wait>"
      ],
      "boot_wait": "4s",
      "guest_os_type": "Ubuntu_64",
      "http_directory": "http",
      "iso_checksum": "4d1a8b720cdd14b76ed9410c63a00d0e",
      "iso_checksum_type": "md5",
      "iso_url": "http://releases.ubuntu.com/13.10/ubuntu-13.10-server-amd64.iso",
      "ssh_username": "vagrant",
      "ssh_password": "vagrant",
      "ssh_port": 22,
      "ssh_wait_timeout": "10000s",
      "shutdown_command": "echo 'shutdown -P now' > shutdown.sh; echo 'vagrant'|sudo -S sh 'shutdown.sh'",
      "guest_additions_path": "VBoxGuestAdditions_{{.Version}}.iso",
      "headless": false,
      "virtualbox_version_file": ".vbox_version",
      "vboxmanage": [
        [
          "modifyvm",
          "{{.Name}}",
          "--memory",
          "2048"
        ],
        [
          "modifyvm",
          "{{.Name}}",
          "--cpus",
          "4"
        ]
      ]
    }
  ],
  "post-processors": ["vagrant"]
}

It builds for virtualbox and then exports it into vagrant. The http folder contains a preseed.cfg file that is necessary to set up Ubuntu.

In the scripts folder you can find a root_setup.sh and setup.sh scripts.

The root_setup.sh script sets up necessary packages and parameters for Vagrant:

#!/bin/bash

set -e

# Updating and Upgrading dependencies
sudo apt-get update -y -qq > /dev/null
sudo apt-get upgrade -y -qq > /dev/null

# Install necessary libraries for guest additions and Vagrant NFS Share
sudo apt-get -y -q install linux-headers-$(uname -r) build-essential dkms nfs-common

# Install necessary dependencies
sudo apt-get -y -q install curl wget git tmux firefox xvfb vim

# Setup sudo to allow no-password sudo for "admin"
groupadd -r admin
usermod -a -G admin vagrant
cp /etc/sudoers /etc/sudoers.orig
sed -i -e '/Defaults\s\+env_reset/a Defaults\texempt_group=admin' /etc/sudoers
sed -i -e 's/%admin ALL=(ALL) ALL/%admin ALL=NOPASSWD:ALL/g' /etc/sudoers

#Install Redis
sudo apt-get -y -q install libjemalloc1
wget -q http://d7jrzzvab3wte.cloudfront.net/checkbot/deb/redis-server_2.6.13-1_amd64.deb
sha1sum redis-server_2.6.13-1_amd64.deb | grep 'ab50cf037fd63e160946f8946b6d318cdf11800d'
dpkg -i redis-server_2.6.13-1_amd64.deb
rm redis-server_2.6.13-1_amd64.deb

# Install required libraries for RVM and Ruby
sudo apt-get -y -q install gawk libreadline6-dev zlib1g-dev libssl-dev libyaml-dev libsqlite3-dev sqlite3 autoconf libgdbm-dev libncurses5-dev automake libtool bison pkg-config libffi-dev libxml2-dev libxslt-dev libxml2


# Install Postgresql
sudo apt-get -y -q install postgresql libpq-dev postgresql-contrib

# Set Password to test for user postgres
sudo -u postgres psql -c "ALTER USER postgres WITH PASSWORD 'test';"

The setup.sh script install different dependencies like ruby or redis to set up the virtual machine exactly how you need it:

#!/bin/bash

set -e

echo "Instaling for rof"

# Installing vagrant keys
mkdir ~/.ssh
chmod 700 ~/.ssh
cd ~/.ssh
wget --no-check-certificate 'https://raw.github.com/mitchellh/vagrant/master/keys/vagrant.pub' -O authorized_keys
chmod 600 ~/.ssh/authorized_keys
chown -R vagrant ~/.ssh

# Node.js Setup
wget --retry-connrefused -q -O - https://raw.github.com/creationix/nvm/master/install.sh | sh
source ~/.nvm/nvm.sh

nvm install 0.10.18
nvm alias default 0.10.18

echo "source ~/.nvm/nvm.sh" >> ~/.bash_profile

# RVM Install
wget --retry-connrefused -q -O - https://get.rvm.io | bash -s stable
source /home/vagrant/.rvm/scripts/rvm

rvm autolibs read-fail

rvm install 2.0.0-p195

gem install bundler zeus

You don’t have any limits what you can run in your virtual machine through these scripts.

Building the Machine

We’ve added a create_box script that makes it easy for you to get started

#!/bin/bash

set -e

#export PACKER_LOG=1
rm packer_virtualbox_virtualbox.box || true
packer build -only=virtualbox packer.json
vagrant box remove vagrant_machine || true
vagrant box add vagrant_machine packer/packer_virtualbox_virtualbox.box

You will then see Virtualbox start up and build the machine

Packer Virtualbox

Run the script and it will create the packer machine and import it into vagrant. Then all you have to do is run

vagrant destroy
vagrant up

And you have your development environment set up.

You can now get into the machine with vagrant ssh and start coding.

Codeship – A hosted Continuous Deployment platform for web applications

Conclusions

Vagrant is an incredibly powerful tool and together with Packer it is easy to build development environments for your whole team.

But this is only the beginning. Packer can go much further than just providing your development environment. We are currently implementing Packer as the tool to build all of our test infrastructure servers. This new set of tools is great for Immutable Infrastructure and Continuous Deployment so you can build more stable, secure and easy to change infrastructure than ever before.

Let us know in the comments how you use Packer and Vagrant. We are excited to hear your thoughts!

Additional Links

Subscribe via Email

Be sure to join 13,643 subscribers of our newsletter to receive updates on software development best practices, Continuous Delivery and tips and tricks to start shipping your product faster.

Join the Discussion

Leave us some comments on what you think about this topic or if you like to add something.

  • Pingback: Talking Immutable Infrastructure with CodeShip | The Virtualization Practice

  • israelbarba

    I can’t run create_box.sh is there other way to do that? thanks you.

    • https://www.codeship.io/ Florian Motlik

      Did you clone the Github repo? I’ve had a typo in the post, the script is called create_box without the .sh part

  • Pingback: Diacode weekly #8 | Blog de Diacode

  • Stuck

    One of the most common situations young developers face is the following:

    — New boss on first morning: “Congratulations on your new job! Here’s some really expensive equipment and lots of coffee!”

    — New boss on second afternoon: “Also, we need you to upgrade our heavily customized WordPress instance. This app is vital, so we can’t have any downtime. It’s only a dozen updates behind. That’s not a problem, right?”

    — New boss on third day (when asked about a testing environment): “We just went with the default EC2 AMI, then tweaked some of the PHP settings — though we don’t remember exactly what we changed. That shouldn’t be difficult to clone, right?”

    — New boss on fourth day: “Where’d my developer go?”

    (Disclaimer: This is just a general picture of a classic corporate problem, not a dig at any specific employer.)

    After more than a year of watching admins wax poetic about how Vagrant/Veewee/etc can help us in this fictional world where everyone sets up reproducible environments *before* deployment, I have a much more practical question for the maintenance programmers among us: Has anyone ever found a reliable script or compiled app that can convert the basics of my company’s *live* EC2 setup into a roughly equivalent local VM? That is to say that my phpInfo() output should look almost identical.

    By the time a good answer pops up, I’ll have just taken my chances. But hopefully this question can help somebody else.

    • https://www.codeship.io/ Florian Motlik

      The problem you have with getting from a production system to a local vm is that as soon as you want to change the infrastructure you need to run through that process again which is totally not reproducible.

      The only real way out of it that I see has is

      1) Write tests that make sure your main workflow is fine. It should at least hit the main pages in your blog and go throught the most important workflows your customers follow. We’ve written a blog post about this a while ago: http://blog.codeship.io/2013/03/15/testing-top-to-bottom.html

      2) Automate the complete setup with a tool like packer and then flip the switch at some point after thoroughly testing it.

      Every other step is simply prolonging the pain. Of course you need to do a calculation if it’s worth it, but if you regularly change your system in my opinion it’s definitely worth it.

      Don’t think that there is any other way to do this. Not starting from a completely new and clean instance means it’s hard for you to really know your infrastructure well.

    • Chris O’Neil

      How would it even be possible to write a tool that would automatically do that kind of reverse engineering for an arbitrary application? It’s that company’s fault for not automating the setup in the first place, and as a maintenance programmer, it’s your job to pick apart the setup and figure out how to automate it, so that you aren’t in that position again. To be honest, I would take the reverse engineering and automation as an interesting challenge. One of the most fun things about doing automation work is that it really is one of the best ways for you to learn every detail about how something works.

      • https://www.codeship.io/ Florian Motlik

        I agree and packer or some other automation tools are perfect for this, as you can slowly build exactly what you really need and everything else you might have needed in the past will just be dropped.

        Although if I could choose I’d prefer to just have that automation in the first place :)

  • pekhee

    Thank you Florian.
    I looked into Packer before and couldn’t see its aim. Your article just helped me understand where can I use it.

    • https://www.codeship.io/ Florian Motlik

      Great. Happy to help. Let me know if other questions come up as we will definitely have a follow up article on this as well.

  • amowu

    Thank you :D

  • Pingback: Setting Up Vagrant for Octopress | Andrew G. Allen

  • Pingback: November Pocket link collection with 15 links

  • Mihai Rusan

    Hi, packer build command give me this error:

    ==> virtualbox: Waiting for SSH to become available…

    ==> virtualbox: Timeout waiting for SSH.

    ==> virtualbox: Error unregistering ISO: VBoxManage error: VBoxManage: error: Failed to get a console object from the direct session (Unknown Status 0x80BB0007)

    ==> virtualbox: VBoxManage: error: Details: code VBOX_E_VM_ERROR (0x80bb0003), component Machine, interface IMachine, callee nsISupports

    ==> virtualbox: Context: “LockMachine(a->session, LockType_Shared)” at line 295 of file VBoxManageStorageController.cpp

    ==> virtualbox: Unregistering and deleting virtual machine…

    ==> virtualbox: Error deleting virtual machine: VBoxManage error: VBoxManage: error: Cannot unregister the machine ‘packer-virtualbox’ while it is locked

    ==> virtualbox: VBoxManage: error: Details: code VBOX_E_INVALID_OBJECT_STATE (0x80bb0007), component Machine, interface IMachine, callee nsISupports

    ==> virtualbox: Context: “Unregister(fDelete ? (CleanupMode_T)CleanupMode_DetachAllReturnHardDisksOnly : (CleanupMode_T)CleanupMode_DetachAllReturnNone, ComSafeArrayAsOutParam(aMedia))” at line 160 of file VBoxManageMisc.cpp

    ==> virtualbox: Deleting output directory…

    Build ‘virtualbox’ errored: Timeout waiting for SSH.

    ==> Some builds didn’t complete successfully and had errors:

    –> virtualbox: Timeout waiting for SSH.

    ==> Builds finished but no artifacts were created.

    • https://www.codeship.io/ Florian Motlik

      Try to open the Virtualbox manage Application and remove all existing virtualbox machines. Then try again. We’ve seen something like this before as well.

      • Mihai Rusan

        Same error. I modify “ssh_wait_timeout”: “20s”

        ==> virtualbox: Waiting for SSH to become available…

        ==> virtualbox: Timeout waiting for SSH.

        ==> virtualbox: Error unregistering ISO: VBoxManage error: VBoxManage: error: Failed to get a console object from the direct session (Unknown Status 0x80BB0007)

        ==> virtualbox: VBoxManage: error: Details: code VBOX_E_VM_ERROR (0x80bb0003), component Machine, interface IMachine, callee nsISupports

        ==> virtualbox: Context: “LockMachine(a->session, LockType_Shared)” at line 295 of file VBoxManageStorageController.cpp

        ==> virtualbox: Unregistering and deleting virtual machine…

        ==> virtualbox: Error deleting virtual machine: VBoxManage error: VBoxManage: error: Cannot unregister the machine ‘packer-virtualbox’ while it is locked

        ==> virtualbox: VBoxManage: error: Details: code VBOX_E_INVALID_OBJECT_STATE (0x80bb0007), component Machine, interface IMachine, callee nsISupports

        ==> virtualbox: Context: “Unregister(fDelete ? (CleanupMode_T)CleanupMode_DetachAllReturnHardDisksOnly : (CleanupMode_T)CleanupMode_DetachAllReturnNone, ComSafeArrayAsOutParam(aMedia))” at line 160 of file VBoxManageMisc.cpp

        ==> virtualbox: Deleting output directory…

        Build ‘virtualbox’ errored: Timeout waiting for SSH.

        ==> Some builds didn’t complete successfully and had errors:

        –> virtualbox: Timeout waiting for SSH.

        ==> Builds finished but no artifacts were created.

        • https://www.codeship.io/ Florian Motlik

          So you did remove all Virtualbox machines first? This is not an ssh problem, but Virtualbox can’t start correctly

          • Mihai Rusan

            all virtualbox machines removed.

          • jhvaras

            I suffer the same issue, launching packer without any VM does not seem to solve, neither increasing ssh_wait_timeout.

          • https://www.codeship.io/ Florian Motlik

            Which OS are you using?

          • jhvaras

            Ubuntu 12.04

  • Pingback: Быстрая сборка образов ОС с помощью Packer » CreativLabs

  • GregPK

    To whom it may concern:

    I’ve had no success running through this process. I’ve got stuck on:

    ==> virtualbox-iso: Error sending boot command: VBoxManage error: VBoxManage: error: Guest not running

    I’ve tried updating virtualbox (4.2.1 => 4.3.6) and still no dice. I’m running mint 14 kernel 3.5.0-17-generic.

    I’ve also tried changing the template to run in headless mode – still no luck – giving up.

  • Michael Aguiar

    I download the repo from github, run “sh create_box” and it goes through everything, then says: Build ‘virtualbox-iso’ errored: Script exited with non-zero exit status: 1

    • https://www.codeship.io/ Florian Motlik @codeship

      Do you have the latest packer version installed?

    • c0smic

      Also getting this error. Most recent packer installed.

      Not sure what the exact error was but I pulled in parts from https://github.com/mitchellh/packer-ubuntu-12.04-docker to get this to work.

    • Yaroslav Ravlinko

      I have the same error. Are you find a problem?

  • Jonathan

    How can we debug the boot_option step? I think there is an error at that point but I don’t know what it is. The result is that instead of running all the boot_option script and then runing the other script (root_setup.sh and setup.sh), it boot directly in the live CD givent the choice to install from there. Also, how does the boot_option param change from Ubuntu? 13.10 will not be the same then 12.04.4? I try adding the -debug flag to packer build but the only thing I see is all the opcode sent at the boot_option step. It’s very hard to debug!

    • https://www.codeship.io/ Florian Motlik @codeship

      Hi Jonathan,

      the boot_option step with preseed is very hard to debug indeed. I’ve used an existing preseed file there.

      You can also use the relatively new virtualbox ovf builder which takes an existing Virtualbox image and starts from there. You can create that base image by hand and take it from there. Not ideal, especially if you want to start from scratch but it is a pretty good option to get started quickly:

      http://www.packer.io/docs/builders/virtualbox-ovf.html

      best,
      Flo

      • Jonathan

        Ok, thanks for ovf-builder suggestion.

        Too bad testing the boot_option is soo hard.
        It’s weird, just playing around with this example here: https://github.com/rokhmanov/packer-teiid and I am able to make a 12.04.4 base box. But I need a 12.04.3 and just changing the ISO and the md5 doesn’t work. My first guest a a boost_option problem but I’m not sure. There is not a lot of information to debug…

        Thanks anyway,

        Jonathan

  • Seph

    So if you want to develop on a particular os, you’ll have cook up some way of installing/partitioning/configuring that os via a command line option on bootup and write that into boot_command. That is such a hassle. Any idea on how to do that with Centos6.5?

    • Seph

      So you have to use the kickstart configurator to make the kickstart file and then host that file somewhere. Which adds another point of failure for your build process. Minimal Centos has to be installed first and then configured with dhcp or static addressing for networking to work, defeating the whole purpose. Just not seeing the advantage of using this as a workflow.

    • http://www.asimihsan.com/ Asim Ihsan

      veewee is another tool for building Vagrant boxes. It has an extraordinary array of kickstarter files available. Here’s one for Centos 6.5:

      https://github.com/jedi4ever/veewee/blob/master/templates/CentOS-6.5-x86_64-minimal/ks.cfg

      And here’s what sequence of keys to type in during bootup:

      https://github.com/jedi4ever/veewee/blob/master/templates/CentOS-6.5-x86_64-minimal/definition.rb

      Although I really like veewee it seems like packer is sexier nowadays, so I’d suggest copy/pasting knowledge from veewee into packer. Enjoy!

  • http://www.monkeytainment.com Ethan Stillman

    Is there anyway to automate the initial Ubuntu credential ask following preseed.cfg running? My goal is to have this thing start-to-finish automated and taking time to put in credentials is a major downer. Made a Stackoverflow post with a bit more info – http://stackoverflow.com/questions/22929564/packer-log-in-automation-for-new-machine-image-post-boot . Thanks for any thoughts.

    • https://www.codeship.io/ Florian Motlik @codeship

      You don’t have to enter any credentials there at all. This is just the default box that is shown while packer connects via SSH. Did the build not finish?

      • http://www.monkeytainment.com Ethan Stillman

        Build didn’t finish, but as you pointed out on Stackoverflow, it was b/c I didn’t include ssh in the `preseed.cfg`. Thanks for your help.

  • Pingback: Building #Vagrant Machines with #Packer http://t.c… | Stéphane Thibault

  • Pingback: Packer unloaded | Vasilkoff

  • tan

    So if I want to use codeship as a Packer service with following workflow:
    push packer.json to github -> codeship build the virtualbox image -> upload the image to amazon, is this workable?

    • https://www.codeship.io/ Florian Motlik @codeship

      You can push to Github then the build starts on codeship and creates the AWS machine from Codeship. We can’t build virtualbox images on codeship, but you can build amazon machines directly

  • kelonye

    vagrant + ansible vs vagrant + packer :)

    • https://www.codeship.io/ Florian Motlik @codeship

      You can also use packer + ansible + vagrant. Build machines with packer, provision them with Ansibe, export them to vagrant format and use them to run your development environment with Vagrant.

  • ULHPC

    Is there a way, upon provisioner error, to authorize the connection within the vagrant box being built ? I have to debug provisioner scripts that fail and having to wait 5 minutes for every test attempts is quite annoying… Accessing the logs is useful but it would be more efficient to be able to freeze the packer process (before the cleaning) and somehow connect to the vm before its deletion. Any hint on that?

  • Geoff

    Thanks for this. Ideally however I want to use packer to build both a vagrant box and a bunch of VMs suitable for things like EC2, vmware etc all from the same source (that is its purpose after all). However I cant work out a way of only installing vagrant stuff for the vagrant box, it doesnt belong anywhere else. I dont want a vagrant user and vagrant packages existing on my production boxes. Do you know if thats possible, without tying the vagrant-only stuff to “virtualbox-iso” or similar, which technically it doesnt belong in (what if I want to build a virtualbox ovf without vagrant as well as a .box file ?)

    • https://www.codeship.io/ Florian Motlik @codeship

      Hi Geoff,

      don’t think that’s possible as vagrant is only a postprocessor for Virtualbox. You might be able Best thing is probably to ask the packer guys directly at either their github or Google Group.

      best,
      Flo