Building Docker from source on the Raspberry Pi

Posted: December 30, 2013 in Docker, Raspberry Pi
Tags: , ,

I’ve been exploring the world of linux containers a bit lately and so I thought I’d write a quick tutorial on how to set up the Raspberry Pi (my go to remote embedded build machine) to build Docker from source. If you don’t know what Docker is then you should do yourself a favour and read up about it from their site. It’s a truly amazing piece of software and I’m hoping to use it for instances of node applications as well as embedded build instances in the future. For now lets just build it. As a forewarning I have built my own 3.12 kernel for this particular Raspberry Pi so if something doesn’t work on the default Raspbian image that could be why.

First up install the userspace tools for linux containers. You could of course install the ‘lxc’ package from the repositories but at this time it is on version 0.8.0-rc1 and as I’m building everything bleeding edge I’m sure you’ll need the latest version.

sudo apt-get install libcap-dev
cd /opt
sudo git clone https://github.com/lxc/lxc.git
cd lxc
sudo ./autogen.sh && sudo ./configure && sudo make && sudo make install
lxc-version
lxc version: 1.0.0.beta1

Next we need to get go. Again we could install the ‘golang’ package but this is the 1.0.2 version and 1.1 version contains many improvements for ARM. So I’ve chosen to get a precompiled version.

cd /opt
sudo wget -c http://dave.cheney.net/paste/go1.2rc1.linux-arm~multiarch-armv6-1.tar.gz
sudo tar -C /usr/local -xzf go1.2rc1.linux-arm~multiarch-armv6-1.tar.gz
Add export PATH=$PATH:/usr/local/go/bin to your ~/.profile or /etc/profile
go version
go version devel +5b367120d592 Thu Sep 19 17:12:00 2013 +1000 linux/arm

Building from source is also pretty easy

curl https://go.googlecode.com/files/go1.2.src.tar.gz | tar xz -C /usr/local
cd /usr/local/go/src
./make.bash
go version go1.2 linux/arm

I have included the AUFS patches from here into my kernel but from docker version 0.7 you do not explicitly need them. Patching is done by compying in the new sources and then running the patches from the root of your kernel source tree.

AUFS-Utils is also useful if you plan on using this feature.

cd /opt
sudo git clone git://git.code.sf.net/p/aufs/aufs-util
cd aufs-util
sudo git checkout aufs3.x-rcN
sudo make && sudo make install

The most difficult part of this whole process is the fact that the docker build system uses docker containers to build itself so you need to have docker to build docker. The guys at resin.io have made things a whole lot easier by publishing a raspberry pi built docker binary. This can be found here, extracted and used.

docker -v
Docker version 0.7.2, build 28b162e-dirty

Building from source can then be completed using the standard installation instructions from the docker website.

Advertisements
Comments
  1. Steve says:

    Thank you for this post. I have been looking for a way to run docker on an existing RPI which I am already using for existing services.

    • rosterloh says:

      Glad to help but I’d have been really lost if it weren’t for the resin.io guys. It’ll be really interesting to see what comes of their project.

      • Steve says:

        Yes, I have a large learning curve here too.

        I have been unable to install the AUFS-utils as I get compilation errors. This is probably because I don’t have the kernel source & include files on the RPI I am using for this R&D. I am using this RPI to run openvpn to act as my tunnel with squid as a proxy to the internet. I probably should buy another one for this, my 4th πŸ™‚

        Docker still runs and I thought all was ok, but when I try to run an ubuntu shell via docker I get the following errors:

        lxc-start: failed to create veth87KHNI-vethWV65JA : Operation not supported
        lxc-start: failed to create netdev
        lxc-start: failed to create the network
        lxc-start: failed to spawn ‘6b5fbcdebc2a067d866f4df874e163df5a7cef9cdc0baf04e8cf62f9d57ebc61’

        2014/01/28 22:58:56 Error: start: Cannot start container 6b5fbcdebc2a067d866f4df874e163df5a7cef9cdc0baf04e8cf62f9d57ebc61: exit status 1

        I thought I may have a problem with the lxc install, but when I followed the instructions from this link: http://raspberrypicloud.wordpress.com/2013/03/12/creating-an-lxc-container-on-the-raspberry-pi/
        I was able to run the new container successfully.

        Any thoughts? I have also tried this with openvpn stopped with the same error message.

  2. rosterloh says:

    So what base docker image are you using for this? I’ve found this one works for me https://index.docker.io/u/resin/rpi-raspbian/. I can remember if this was relevant but there was something to do with port forwarding for the network. http://docs.docker.io/en/latest/installation/ubuntulinux/#ufw. If I recall it was `sysctl -w net.ipv4.ip_forward=1`. Let me know how you get on

    • Steve says:

      I am using the docker version you provided the link to, from Resin:
      Docker version 0.7.5, build c348c04-dirty

      I remember running sysctl -w net.ipv4.ip_forward=1 when setting up the openvpn, but I have just run this command again before running docker with the same result. I have tried using a docker image I pushed to the public repository when using the vanilla docker Resin full image before Christmas , but I still have the same issue.

      I don’t think I have the aufs module installed though (using lsmd) and I really don’t want to have to compile a kernel to get it as last time I did that I was almost successful, but it took the best part of a day and then my external USB device was no longer recognized 😦

    • Steve says:

      A quick +’ve update.

      I have managed to rebuild a kernel (now on 3.10.28+) with aufs patched in. A bit of ‘C’ editing required to manually apply some of the patches. I have pulled the resin/rpi-rasbian image and docker now is working πŸ™‚

      The only issue I have now is that I can’t compile the aufs-utils (branch aufs3.9) I get the following error, but I am guessing it isn’t important:

      perror.c:30:3: error: β€˜EAU_MVDOWN_NOUPPER’ undeclared here (not in a function)

      I switched to this branch as using aufs3.x-rcN as tying this gave this error:
      Wrong version!
      aufs-util for aufs3.x-rcN, but aufs is 3.10-20130826.

      • rosterloh says:

        Thanks for the update Steve. As soon as I get some time I’m going to move my Pi to the 3.13 kernel and then I’ll have to repatch and try AUFS and utils again so I’ll let you know how it goes.

  3. Steve says:

    One thing I have just noticed is that docker run command takes an age to run – I mean 10 minutes+. Top shows cp is active. I have moved the docker directory back onto the ssd card from my external usb device without any speedup.

    • Steve says:

      I wish you luck with the 3.13 work. I finally worked out that my kernel does not have the aufs enabled. I had trouble getting the AUFS option to appear in menumaker as I couldn’t find the EXPERIMENTAL option which it depends upon. In the end I hacked aufs/Kconfig to remove that dependency and then ran into loads of aufs compilation issues. This isn’t because I didn’t make the patches properly. It seems to me that the aufs code has not been compiled with the 3.10 kernel! I am slowing working through the compilation issues, but there is a big risk of introducing bugs 😦

      • rosterloh says:

        Wow, I guess I must have just been lucky that I decided to go with the 3.12 kernel then. I definitely didn’t have any compilation errors. I’ve seen that the AUFS project has pushed some updates so I’ve integrated them into my kernel and am busy compiling. I’ll try AUFS utils again after that.

  4. Steve says:

    Well, after a lot of work I finally managed to build a kernel with aufs in it. I was able to run the mount example for aufs, but I hit a kernel panic when I tried to ‘ls’ the mount point. Using docker with this kernel also caused a panic so my aufs installation is totally broken.

    From looking at the aufs patch files it is obvious that the linux source code I downloaded is not even vaguely similar to what the patch files are expecting. The aufs utils code basically only built when I ‘wholesale’ started to delete code that used constants that are not defined anywhere. Clearly, I have been flogging a dead horse here 😦

    If your set-up is now working would you mind giving me the appropriate git commands to pull down the same kernel source code and aufs installation as you so I can try to reproduce starting from scratch?

    • rosterloh says:

      Eek, sorry Steve. I haven’t had a chance to fully test my updates yet so I can’t say if everything is good. Life has just been a bit crazy lately. I’ll get a chance tomorrow night to give it a run through and check everything looks ok. If you want to have a look before the source is at https://github.com/rosterloh/linux-rpi/tree/rpi-3.12.y. The only changes I’ve made since my current running kernel are updating the AUFS sources and some upstream patches that seem to mostly relate to GPIO and audio.

  5. Steve says:

    No problem. I am very used to being overly busy myself, although right now I have made some space for me and am having a break from work over the winter πŸ™‚
    I have now managed to build the kernel on the 3.10.y branch with aufs 3.10. As you said no issues. I still can’t get aufs-util to build though. I first tried the 3.12 kernel with aufs and did have a compilation problem with d_count (from memory) in struct dentry not being defined. Adding the unsigned int to the struct was easy, but then I looked a little harder and realised that it wouldn’t work as a couple of inline functions in the 3.10 version of the include file actually set d_count and there were not in the later version, so I decided to regresss to the 3.10 kernel for safety. I am hoping that docker will now run a a reasonable speed now copy on write is enabled through aufs (I hope!)

  6. rosterloh says:

    Success! I finally managed to get aufs-utils to compile correctly. The headers install script was not correctly moving the aufs_type.h file to /usr/include. I fixed this by running make install_headers from the aufs3-standalone directory. Then aufs-utils compiles without error. I used the aufs3.x.rcN branch by the way.

  7. Steve says:

    I have put up the results of my labours on Github and have just (finally) managed to build the docker-11.1 release for the RPi. I am sorry the README is so brief, but I am now time poor having returned to work a few weeks ago: https://github.com/stevef1uk/docker_for_rpi.git

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s