Docker - The Linux container runtime

Docker complements LXC with a high-level API which operates at the process level. It runs unix processes with strong guarantees of isolation and repeatability across servers.

Docker is a great building block for automating distributed systems: large-scale web deployments, database clusters, continuous deployment systems, private PaaS, service-oriented architectures, etc.

• Heterogeneous payloads Any combination of binaries, libraries, configuration files, scripts, virtualenvs, jars, gems, tarballs, you name it. No more juggling between domain-specific tools. Docker can deploy and run them all.
• Any server Docker can run on any x64 machine with a modern linux kernel - whether it’s a laptop, a bare metal server or a VM. This makes it perfect for multi-cloud deployments.
• Isolation docker isolates processes from each other and from the underlying host, using lightweight containers.
• Repeatability Because containers are isolated in their own filesystem, they behave the same regardless of where, when, and alongside what they run.

What is a Standard Container?

Docker defines a unit of software delivery called a Standard Container. The goal of a Standard Container is to encapsulate a software component and all its dependencies in a format that is self-describing and portable, so that any compliant runtime can run it without extra dependency, regardless of the underlying machine and the contents of the container.

The spec for Standard Containers is currently work in progress, but it is very straightforward. It mostly defines 1) an image format, 2) a set of standard operations, and 3) an execution environment.

A great analogy for this is the shipping container. Just like Standard Containers are a fundamental unit of software delivery, shipping containers (http://bricks.argz.com/ins/7823-1/12) are a fundamental unit of physical delivery.

Images

An original container image. These are stored on disk and are comparable with what you normally expect from a stopped virtual machine image. Images are stored (and retrieved from) repository

Images are stored on your local file system under /var/lib/docker/images

Containers

A container is a local version of an image. It can be running or stopped, The equivalent would be a virtual machine instance.

Containers are stored on your local file system under /var/lib/docker/containers

Dependencies

The linux-image-extra package is only needed on standard Ubuntu EC2 AMIs in order to install the aufs kernel module.

sudo apt-get install linux-image-extra-`uname -r`

Installation

Docker is available as a Ubuntu PPA (Personal Package Archive), hosted on launchpad which makes installing Docker on Ubuntu very easy.

Add the custom package sources to your apt sources list. Copy and paste the following lines at once.

sudo sh -c "echo 'deb http://ppa.launchpad.net/dotcloud/lxc-docker/ubuntu precise main' >> /etc/apt/sources.list"

Update your sources. You will see a warning that GPG signatures cannot be verified.

sudo apt-get update

Now install it, you will see another warning that the package cannot be authenticated. Confirm install.

sudo apt-get install lxc-docker

Verify it worked

docker

Done!, now continue with the Hello World example.

Install dependencies:

sudo apt-get install lxc bsdtar
sudo apt-get install linux-image-extra-`uname -r`

The linux-image-extra package is needed on standard Ubuntu EC2 AMIs in order to install the aufs kernel module.

Install the docker binary:

wget http://get.docker.io/builds/Linux/x86_64/docker-master.tgz
tar -xf docker-master.tgz
sudo cp ./docker-master /usr/local/bin

Note: docker currently only supports 64-bit Linux hosts.

Run the docker daemon

sudo docker -d &

Run your first container!

docker run -i -t ubuntu /bin/bash

Continue with the Hello World example.

Dependencies

Docker depends on several packages which are specified as dependencies in either AUR package.

• aufs3
• bridge-utils
• go
• iproute2
• linux-aufs_friendly
• lxc

Installation

The instructions here assume yaourt is installed. See Arch User Repository for information on building and installing packages from the AUR if you have not done so before.

Keep in mind that if linux-aufs_friendly is not already installed that a new kernel will be compiled and this can take quite a while.

yaourt -S lxc-docker-git

Starting Docker

Prior to starting docker modify your bootloader to use the linux-aufs_friendly kernel and reboot your system.

There is a systemd service unit created for docker. To start the docker service:

sudo systemctl start docker

To start on system boot:

sudo systemctl enable docker

Install Vagrant and Virtualbox

1. Install virtualbox from https://www.virtualbox.org/ (or use your package manager)
2. Install vagrant from http://www.vagrantup.com/ (or use your package manager)
3. Install git if you had not installed it before, check if it is installed by running git in a terminal window

Spin it up

1. Fetch the docker sources (this includes the Vagrantfile for machine setup).

   git clone https://github.com/dotcloud/docker.git
   

2. Run vagrant from the sources directory

   vagrant up
   

   Vagrant will:

   • Download the ‘official’ Precise64 base ubuntu virtual machine image from vagrantup.com
   • Boot this image in virtualbox
   • Add the Docker PPA sources to /etc/apt/sources.lst
   • Update your sources
   • Install lxc-docker

   You now have a Ubuntu Virtual Machine running with docker pre-installed.

Connect

To access the VM and use Docker, Run vagrant ssh from the same directory as where you ran vagrant up. Vagrant will connect you to the correct VM.

vagrant ssh

Run

Now you are in the VM, run docker

docker

Continue with the Hello World example.

Requirements

1. Install virtualbox from https://www.virtualbox.org - or follow this tutorial

2. Install vagrant from http://www.vagrantup.com - or follow this tutorial

3. Install git with ssh from http://git-scm.com/downloads - or follow this tutorial

We recommend having at least 2Gb of free disk space and 2Gb of RAM (or more).

Opening a command prompt

First open a cmd prompt. Press Windows key and then press “R” key. This will open the RUN dialog box for you. Type “cmd” and press Enter. Or you can click on Start, type “cmd” in the “Search programs and files” field, and click on cmd.exe.

This should open a cmd prompt window.

Alternatively, you can also use a Cygwin terminal, or Git Bash (or any other command line program you are usually using). The next steps would be the same.

Launch an Ubuntu virtual server

Let’s download and run an Ubuntu image with docker binaries already installed.

git clone https://github.com/dotcloud/docker.git
cd docker
vagrant up

Congratulations! You are running an Ubuntu server with docker installed on it. You do not see it though, because it is running in the background.

Log onto your Ubuntu server

Let’s log into your Ubuntu server now. To do so you have two choices:

• Use Vagrant on Windows command prompt OR
• Use SSH

Using Vagrant on Windows Command Prompt

Run the following command

vagrant ssh

You may see an error message starting with “ssh executable not found”. In this case it means that you do not have SSH in your PATH. If you do not have SSH in your PATH you can set it up with the “set” command. For instance, if your ssh.exe is in the folder named “C:Program Files (x86)Gitbin”, then you can run the following command:

set PATH=%PATH%;C:\Program Files (x86)\Git\bin

Using SSH

First step is to get the IP and port of your Ubuntu server. Simply run:

vagrant ssh-config

You should see an output with HostName and Port information. In this example, HostName is 127.0.0.1 and port is 2222. And the User is “vagrant”. The password is not shown, but it is also “vagrant”.

You can now use this information for connecting via SSH to your server. To do so you can:

• Use putty.exe OR
• Use SSH from a terminal

Use putty.exe

You can download putty.exe from this page http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html Launch putty.exe and simply enter the information you got from last step.

Open, and enter user = vagrant and password = vagrant.

SSH from a terminal

You can also run this command on your favorite terminal (windows prompt, cygwin, git-bash, …). Make sure to adapt the IP and port from what you got from the vagrant ssh-config command.

ssh vagrant@127.0.0.1 –p 2222

Enter user = vagrant and password = vagrant.

Congratulations, you are now logged onto your Ubuntu Server, running on top of your Windows machine !

Running Docker

First you have to be root in order to run docker. Simply run the following command:

sudo su

You are now ready for the docker’s “hello world” example. Run

docker run busybox echo hello world

All done!

Now you can continue with the Hello World example.

Installation

Docker can now be installed on Amazon EC2 with a single vagrant command. Vagrant 1.1 or higher is required.

1. Install vagrant from http://www.vagrantup.com/ (or use your package manager)

2. Install the vagrant aws plugin

   vagrant plugin install vagrant-aws
   

3. Get the docker sources, this will give you the latest Vagrantfile.

   git clone https://github.com/dotcloud/docker.git
   

4. Check your AWS environment.

   Create a keypair specifically for EC2, give it a name and save it to your disk. I usually store these in my ~/.ssh/ folder.

   Check that your default security group has an inbound rule to accept SSH (port 22) connections.

5. Inform Vagrant of your settings

   Vagrant will read your access credentials from your environment, so we need to set them there first. Make sure you have everything on amazon aws setup so you can (manually) deploy a new image to EC2.

   export AWS_ACCESS_KEY_ID=xxx
   export AWS_SECRET_ACCESS_KEY=xxx
   export AWS_KEYPAIR_NAME=xxx
   export AWS_SSH_PRIVKEY=xxx
   

   The environment variables are:

   • AWS_ACCESS_KEY_ID - The API key used to make requests to AWS
   • AWS_SECRET_ACCESS_KEY - The secret key to make AWS API requests
   • AWS_KEYPAIR_NAME - The name of the keypair used for this EC2 instance
   • AWS_SSH_PRIVKEY - The path to the private key for the named keypair, for example ~/.ssh/docker.pem

   You can check if they are set correctly by doing something like

   echo $AWS_ACCESS_KEY_ID
   

6. Do the magic!

   vagrant up --provider=aws
   

   If it stalls indefinitely on [default] Waiting for SSH to become available..., Double check your default security zone on AWS includes rights to SSH (port 22) to your container.

   If you have an advanced AWS setup, you might want to have a look at the https://github.com/mitchellh/vagrant-aws

7. Connect to your machine

   vagrant ssh
   

8. Your first command

   Now you are in the VM, run docker

   docker
   

Continue with the Hello World example.

Open a docker container

docker run -i -t base /bin/bash

Building your image

Update your docker container, install the redis server. Once installed, exit out of docker.

apt-get update
apt-get install redis-server
exit

Snapshot the installation

docker ps -a  # grab the container id (this will be the last one in the list)
docker commit <container_id> <your username>/redis

Run the service

Running the service with -d runs the container in detached mode, leaving the container running in the background. Use your snapshot.

docker run -d -p 6379 <your username>/redis /usr/bin/redis-server

docker ps  # grab the new container id
docker inspect <container_id>    # grab the ipaddress of the container
redis-cli -h <ipaddress> -p 6379
redis 10.0.3.32:6379> set docker awesome
OK
redis 10.0.3.32:6379> get docker
"awesome"
redis 10.0.3.32:6379> exit

docker ps  # grab the new container id
docker port <container_id> 6379  # grab the external port
ifconfig   # grab the host ip address
redis-cli -h <host ipaddress> -p <external port>
redis 192.168.0.1:49153> set docker awesome
OK
redis 192.168.0.1:49153> get docker
"awesome"
redis 192.168.0.1:49153> exit

Contribution guidelines

curl -o .git/hooks/pre-commit https://raw.github.com/edsrzf/gofmt-git-hook/master/fmt-check && chmod +x .git/hooks/pre-commit

Starting Docker

If you have used one of the quick install paths’, Docker may have been installed with upstart, Ubuntu’s system for starting processes at boot time. You should be able to run docker help and get output.

If you get docker: command not found or something like /var/lib/docker/repositories: permission denied you will need to specify the path to it and manually start it.

# Run docker in daemon mode
sudo <path to>/docker -d &

Running an interactive shell

# Download a base image
docker pull base

# Run an interactive shell in the base image,
# allocate a tty, attach stdin and stdout
docker run -i -t base /bin/bash

Starting a long-running worker process

# Start a very useful long-running process
JOB=$(docker run -d base /bin/sh -c "while true; do echo Hello world; sleep 1; done")

# Collect the output of the job so far
docker logs $JOB

# Kill the job
docker kill $JOB

Listing all running containers

docker ps

Expose a service on a TCP port

# Expose port 4444 of this container, and tell netcat to listen on it
JOB=$(docker run -d -p 4444 base /bin/nc -l -p 4444)

# Which public port is NATed to my container?
PORT=$(docker port $JOB 4444)

# Connect to the public port via the host's public address
# Please note that because of how routing works connecting to localhost or 127.0.0.1 $PORT will not work.
IP=$(ifconfig eth0 | perl -n -e 'if (m/inet addr:([\d\.]+)/g) { print $1 }')
echo hello world | nc $IP $PORT

# Verify that the network connection worked
echo "Daemon received: $(docker logs $JOB)"

Committing (saving) an image

Save your containers state to a container image, so the state can be re-used.

When you commit your container only the differences between the image the container was created from and the current state of the container will be stored (as a diff). See which images you already have using docker images

# Commit your container to a new named image
docker commit <container_id> <some_name>

# List your containers
docker images

You now have a image state from which you can create new instances.

Read more about Working with the repository or continue to the complete Command Line Interface

Connecting to the repository

You create a user on the central docker repository by running

docker login

If your username does not exist it will prompt you to also enter a password and your e-mail address. It will then automatically log you in.

Committing a container to a named image

In order to commit to the repository it is required to have committed your container to an image with your namespace.

# for example docker commit $CONTAINER_ID dhrp/kickassapp
docker commit <container_id> <your username>/<some_name>

Pushing a container to the repository

In order to push an image to the repository you need to have committed your container to a named image (see above)

Now you can commit this image to the repository

# for example docker push dhrp/kickassapp
docker push <image-name>

Docker Usage

To list available commands, either run docker with no parameters or execute docker help:

$ docker
  Usage: docker COMMAND [arg...]

  A self-sufficient runtime for linux containers.

  ...

Available Commands

Usage: docker attach CONTAINER

Attach to a running container

Usage: docker build -
Example: cat Dockerfile | docker build -
Build a new image from the Dockerfile passed via stdin

Usage: docker commit [OPTIONS] CONTAINER [REPOSITORY [TAG]]

Create a new image from a container's changes

  -m="": Commit message

Usage: docker diff CONTAINER [OPTIONS]

Inspect changes on a container's filesystem

Usage: docker export CONTAINER

Export the contents of a filesystem as a tar archive

Usage: docker history [OPTIONS] IMAGE

Show the history of an image

Usage: docker images [OPTIONS] [NAME]

List images

  -a=false: show all images
  -q=false: only show numeric IDs

Usage: docker import [OPTIONS] URL|- [REPOSITORY [TAG]]

Create a new filesystem image from the contents of a tarball

Usage: docker info

Display system-wide information.

Usage: docker inspect [OPTIONS] CONTAINER

Return low-level information on a container

Usage: docker kill [OPTIONS] CONTAINER [CONTAINER...]

Kill a running container

Usage: docker login

Register or Login to the docker registry server

Usage: docker logs [OPTIONS] CONTAINER

Fetch the logs of a container

Usage: docker port [OPTIONS] CONTAINER PRIVATE_PORT

Lookup the public-facing port which is NAT-ed to PRIVATE_PORT

Usage: docker ps [OPTIONS]

List containers

  -a=false: Show all containers. Only running containers are shown by default.
  -notrunc=false: Don't truncate output
  -q=false: Only display numeric IDs

Usage: docker pull NAME

Pull an image or a repository from the registry

Usage: docker push NAME

Push an image or a repository to the registry

Usage: docker restart [OPTIONS] NAME

Restart a running container

Usage: docker rm [OPTIONS] CONTAINER

Remove a container

Usage: docker rmimage [OPTIONS] IMAGE

Remove an image

Usage: docker run [OPTIONS] IMAGE COMMAND [ARG...]

Run a command in a new container

  -a=map[]: Attach to stdin, stdout or stderr.
  -d=false: Detached mode: leave the container running in the background
  -e=[]: Set environment variables
  -h="": Container host name
  -i=false: Keep stdin open even if not attached
  -m=0: Memory limit (in bytes)
  -p=[]: Map a network port to the container
  -t=false: Allocate a pseudo-tty
  -u="": Username or UID

Usage: docker start [OPTIONS] NAME

Start a stopped container

Usage: docker stop [OPTIONS] NAME

Stop a running container

Usage: docker tag [OPTIONS] IMAGE REPOSITORY [TAG]

Tag an image into a repository

  -f=false: Force

Usage: docker wait [OPTIONS] NAME

Block until a container stops, then print its exit code.

1. Format

The Docker builder format is quite simple:

instruction arguments

The first instruction must be FROM

All instruction are to be placed in a file named Dockerfile

In order to place comments within a Dockerfile, simply prefix the line with “#“

2. Instructions

Docker builder comes with a set of instructions:

1. FROM: Set from what image to build
2. RUN: Execute a command
3. INSERT: Insert a remote file (http) into the image

3. Dockerfile Examples

# Nginx
#
# VERSION               0.0.1
# DOCKER-VERSION        0.2

from ubuntu

# make sure the package repository is up to date
run echo "deb http://archive.ubuntu.com/ubuntu precise main universe" > /etc/apt/sources.list
run apt-get update

run apt-get install -y inotify-tools nginx apache openssh-server
insert https://raw.github.com/creack/docker-vps/master/nginx-wrapper.sh /usr/sbin/nginx-wrapper

# Firefox over VNC
#
# VERSION               0.3
# DOCKER-VERSION        0.2

from ubuntu
# make sure the package repository is up to date
run echo "deb http://archive.ubuntu.com/ubuntu precise main universe" > /etc/apt/sources.list
run apt-get update

# Install vnc, xvfb in order to create a 'fake' display and firefox
run apt-get install -y x11vnc xvfb firefox
run mkdir /.vnc
# Setup a password
run x11vnc -storepasswd 1234 ~/.vnc/passwd
# Autostart firefox (might not be the best way to do it, but it does the trick)
run bash -c 'echo "firefox" >> /.bashrc'