Hey everyone, let's talk about Docker!

There's a good wiki page about how to run OpenWrt images on Docker.  This post is about the converse situation--how to run a Docker container on an OpenWrt device.

First, what is a Docker container?  A Docker container is a lightweight virtual machine.  It's possible to run many Docker containers simultaneously on a low-end computer with excellent performance and low overhead.  The promise (mostly true) is also that a Docker container can run on any Linux computer without modification.

It's the portability property that excites me about running Docker on OpenWrt because there's already a growing library of free Docker containers.  If we can make it possible to run a Docker container on an OpenWrt box, then we can run these containers without modification.

OpenWrt already does a fantastic job of lowering the learning curve for me to create an application on a low-end device.  I get to use Linux and a package manager that resembles the Debian one I'm used to.  With Docker support this learning curve would go even further down to zero--to create a new application on an OpenWrt device I would just create a container the way I'm already creating containers that run in VirtualBox and on Amazon EC2.  I would have the confidence of knowing that my applications would work without having to factor the idiosyncrasies of OpenWrt into my test cases.

Of course this will probably work only on beefier OpenWrt devices, which is fine with me.  I don't think there should be any significant technical hurdles as someone has already figured out how to run Docker containers on Android.

What do you think?

(Last edited by NodeGuy on 28 Apr 2014, 03:29)

Cool. Do you know where I can find some documentation on how to build/compile docker?

Yes, it's here:

Setting Up a Dev Environment

Yes and I saw that, but it doesn't build docker from source checkout from git

[debian@Debian:/opt/openwrt-svn-trunk/build_dir/target-arm_xscale_uClibc-0.9.33.2_eabi 1182%] ~ tar -Jxf /opt/downloads/docker_git-e83fc70.tar.xz 
2.096u+0.272s=0:02.28e(103.5%) TDSavg=0k+0k+0k max=9076k 50880+81600io 2pf+0sw
[debian@Debian:/opt/openwrt-svn-trunk/build_dir/target-arm_xscale_uClibc-0.9.33.2_eabi 1183%] ~ cd docker_git/
[debian@Debian:/opt/openwrt-svn-trunk/build_dir/target-arm_xscale_uClibc-0.9.33.2_eabi/docker_git 1184%] ~ make build
mkdir bundles
docker build -t "docker:master" .
/bin/sh: 1: docker: not found
make: *** [build] Error 127
[debian@Debian:/opt/openwrt-svn-trunk/build_dir/target-arm_xscale_uClibc-0.9.33.2_eabi/docker_git 1185%] ~

Unless I can build it from source using GCC, there is no way I will install a pre-compiled environment tools and use it to build docker.

Thank you for trying to build Docker.

I'm not understanding your concern.  Are you objecting to building Docker using a prebuilt binary of Docker on some kind of philosophical grounds?  You want to build it from source using GCC, but you can't build GCC itself without GCC so wouldn't you have the same problem there?

What is the problem with docker developers releasing the information on how to compile docker using GCC from ground up instead of installing a docker binary distribution?

I don't know the answer to your question and I'm not a Docker developer.

If I were to make a guess, it's that using Docker to create a build environment is genuinely the best tool for the job.  Look at the following enthusiasm around using Docker to make Debian packages:

-- Debian/Ubuntu Package Developing with Docker

Perhaps, you don't understand my point. In your OP, you mentioned that docker container is a lightweight virtual machine and it's possible to run many docker containers simultaneously on a low-end computer with excellent performance and low overhead. If that is true, then docker will probably be an excellent program to port to OpenWRT. That's what I tried to do. However, unless I can (cross) compile the docker source using OpenWRT toolchain, there is no way I can port it to OpenWRT. I lurked around the docker site and found no mentions on how to compile the docker source. This bring me to question whether the real docker source code is an open-source. What I meant is I suspect the current open and downloadable source isn't really the docker main source, but rather some sort of docker files that can only be compiled by the docker tools into containers.

They host their source code on github: https://github.com/dotcloud/docker
Notice the Makefile in their source root. Compiling it should be as simple as running "make"... but obviously, you'll have to do some more work to build an openwrt package.

I wish it would be as simple as you just said above. However, if you peruse my post above, you will see that is not the case. As pointed out by NodeGuy, one needs to install a prebuilt binary of docker to compile the docker source which I am not willing to do. However, I am willing to (cross) compile and build docker for OpenWRT from its source code using the toolchain from OpenWRT.

I don't think that will be any issue for me if I can (cross) compile docker with an OpenWRT toolchain.

(Last edited by mazilo on 29 Apr 2014, 04:14)

The situation is a little confusing but I'm sure we can clear it up.

The Docker build instructions include two steps:

1.  Make a virtual machine, using the Docker binary, that automatically includes all the dependencies for building Docker from source.
2.  Build Docker from source on that virtual machine.

I imagine that after you complete step 1 you can add the OpenWrt toolchain to the virtual machine and use it to cross-compile Docker as you're hoping.

From the above two steps, I gathered you can use docker binary to create a virtual OpenWRT environment. In this OpenWRT environment, you can run an OpenWRT firmware. If that is the case, then can you at least create a virtual OpenWRT environment for an ARMv5 platform. Then, go to OpenWRT snapshot trunk for kirkwood to download and extract both the openwrt-kirkwood-generic-rootfs.tar.gz and openwrt-kirkwood-uImage files to the virtual OpenWRT environment. Once you have done that, try to see if you can let the virtual OpenWRT environment to run the firmware. BTW, the openwrt-kirkwood-uImage file must reside inside the /boot directory.

If the above test works, then at least there is something we can do with docker.

That will take me into new territory but I'm willing to give a go and see how far I get.  Here's my plan:

1.  Start with Mac OS. (done)
2.  Install Homebrew. (done)
3.  Use Homebrew to install QEMU.
4.  Use QEMU to run the ARM port of Arch Linux.
5.  Run Docker on Arch Linux.
6.  Follow your directions.

Thank you. But, your approach is different from what I proposed.

In my proposal, just run docker on a host computer to create an OpenWRT environment to run the OpenWRT Marvell Kirkwood firmware.

(Last edited by mazilo on 30 Apr 2014, 16:35)

You wrote "If that is the case, then can you at least create a virtual OpenWRT environment for an ARMv5 platform."  Doesn't my host computer need to be an ARMv5 platform for that to work?  (Hence the need for QEMU.)

You will need to read my post from the 1st sentence down to understand what I clearly asked.

It's already been documented how to run OpenWrt in a Docker container here (as I referenced in my first post):

Docker OpenWRT Image

Yes and I am aware of that. But, that is for the openwrt-x86 platform and I am looking for a possibility to run ANY OpenWRT in a docker! From what I saw, that is not possible unless docker GPLs its source code or at least provide the environment to support all platforms supported by OpenWRT. Eventually, I would like to run docker on any OpenWRT since you mentioned in your OP that It's possible to run many Docker containers simultaneously on a low-end computer with excellent performance and low overhead. To achieve this, docker source must be ported to OpenWRT. In this case, unless one can (cross) compile the docker source using OpenWRT toolchain and/or any toolchain to support OpenWRT, there is no way it can be ported to OpenWRT.

(Last edited by mazilo on 2 May 2014, 13:39)

Wow, having this conversation with you feels like a lot of work.

Docker is a virtualization technology, not an emulation technology.  It can't make ARM code run on a x86 chip.  That's why I proposed running it on QEMU to satisfy your request "can you at least create a virtual OpenWRT environment for an ARMv5 platform".

Docker's license is easily found in their GitHub repository and is the Apache License which is just as open as the GPL.

I still haven't seen you respond to my suggestion that you run the OpenWrt toolchain in Docker's build environment.

I was initially excited about this but I'm losing interest.

When you 1st posted the above statement, you clearly indicated docker is a virtualization tech and I was very excited about it. However, when I downloaded the source code (from the link you provided) and could not even compile it using an ordinary GCC on my AMD64 PhenomII X3 running a Debian 7.0 Linux, my excitement ended there. The reason is simply I just don't know how to port any open-source package to OpenWRT if I can't even compile it using any default Linux compiler, i.e. GCC, on a Linux computer. As such, I don't see an ordinary approach to cross compile and/or port any docker container to any OpenWRT platform will prevail. However, if you can run any docker container on any OpenWRT device, i.e. ARMv5, etc., I certainly would like to know.

As with your suggestion, I have no interest at all. My interest is the same as yours, i.e. to run a Docker container on an OpenWrt device.

I've asked for help from the Docker developers.

Thank you and hope you will get the answers.

BTW, I started to read the hack/PACKAGERS.md file. It says go is needed to compile docker. I tried to gccgo to noavail, unfortunately.

(Last edited by mazilo on 5 May 2014, 04:15)

Any update? Does Docker work on OpenWRT now?

I got distracted by other things before making any real progress on this and probably won't.

Any progress ? I am interested in doing a docker port as well. Earlier I did a node.js port for openwrt for some of my work. If someone is still pursuing this, I would be happy to contribute.