Docker (in progress!)

Table of Contents

Chapter 1 - Vagrantfile, Virtual Box, and Docker for Mac Installation

Check Docker Client

docker info
Containers: 0
 Running: 0
 Paused: 0
 Stopped: 0
Images: 0
Server Version: 1.12.1
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 0
 Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Security Options: seccomp
Kernel Version: 4.4.20-moby
Operating System: Alpine Linux v3.4
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 1.953 GiB
Name: moby
ID: KK2I:MODV:URR2:C2OY:GMP7:NT4C:3TS7:5FP2:GDVM:SMQN:XXDG:B46B
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
 File Descriptors: 16
 Goroutines: 27
 System Time: 2016-09-29T11:30:25.905005719Z
 EventsListeners: 1
No Proxy: *.local, 169.254/16
Registry: https://index.docker.io/v1/
Insecure Registries:
 127.0.0.0/8

Create CoreOS Docker host using Vagrant VM by:

  • Note: Use of Vagrant seed for docker development is an alternative approach to using docker-machine.

  • Cloning coreos-vagrant Git Repo into host machine directory docker-host

  • Making the following changes to the Vagrantfile

+VAGRANTFILE_API_VERSION = "2"

+Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
...
+        # http://serverfault.com/questions/495914/vagrant-slow-internet-connection-in-guest
+        vb.customize ["modifyvm", :id, "--natdnshostresolver1", "on"]
+        # vb.customize ["modifyvm", :id, "--natdnsproxy1", "on"]
       end
 
       ip = "172.17.8.#{i+100}"
       config.vm.network :private_network, ip: ip
+      config.vm.network "public_network", :bridge => 'en1: Wi-Fi (AirPort)', ip: "192.168.1.201"
 
       # Uncomment below to enable NFS for sharing the host machine into the coreos-vagrant VM.
+      config.vm.synced_folder "~/code/docker-host-coreos-share", "/home/core/share", id: "core", :nfs => true, :mount_options => ['nolock,vers=3,udp']
  • Expose Docker TCP connection port in config.rb of coreos-vagrant directory
    • Note that Vagrantfile creates a CoreOS VM using VirtualBox software hypervisor
    • Note that Vagrantfile parses config.rb containing a set of options configuring CoreOS cluster
echo "\$expose_docker_tcp=2375" > config.rb
  • Generate user-data file containing #cloud-config to simplify provisioning of CoreOS VM using built-in coreos-cloudinit
#cloud-config
coreos:
  units:
    - name: docker-tcp.socket
      command: start
      enable: yes
      content: |
        [Unit]
        Description=Docker Socket for the API
        [Socket]
        ListenStream=2375
        BindIPv6Only=both
        Service=docker.service
        [Install]
        WantedBy=sockets.target
    - name: enable-docker-tcp.service
      command: start
      content: |
        [Unit]
        Description=Enable the Docker Socket for the API
        [Service]
        Type=oneshot
        ExecStart=/usr/bin/systemctl enable docker-tcp.socket
  • Startup and SSH using VirtualBox Provider (default Vagrant provider)
    • Note vagrant up triggers vagrant to download CoreOS image (if necessary) and relaunch instance
    • Note vagrant ssh connects to VM from directory with Vagrantfile

Use Vagrant VM:

vagrant up
Bringing machine 'core-01' up with 'virtualbox' provider...
==> core-01: Box 'coreos-alpha' could not be found. Attempting to find and install...
    core-01: Box Provider: virtualbox
    core-01: Box Version: >= 0
==> core-01: Loading metadata for box 'https://storage.googleapis.com/alpha.release.core-os.net/amd64-usr/current/coreos_production_vagrant.json'
    core-01: URL: https://storage.googleapis.com/alpha.release.core-os.net/amd64-usr/current/coreos_production_vagrant.json
==> core-01: Adding box 'coreos-alpha' (v1185.0.0) for provider: virtualbox
    core-01: Downloading: https://alpha.release.core-os.net/amd64-usr/1185.0.0/coreos_production_vagrant.box
    core-01: Calculating and comparing box checksum...
==> core-01: Successfully added box 'coreos-alpha' (v1185.0.0) for 'virtualbox'!
==> core-01: Importing base box 'coreos-alpha'...
==> core-01: Matching MAC address for NAT networking...
==> core-01: Checking if box 'coreos-alpha' is up to date...
==> core-01: Setting the name of the VM: coreos-vagrant_core-01_1475139949586_82673
==> core-01: Clearing any previously set network interfaces...
==> core-01: Preparing network interfaces based on configuration...
    core-01: Adapter 1: nat
    core-01: Adapter 2: hostonly
==> core-01: Forwarding ports...
    core-01: 2375 (guest) => 2375 (host) (adapter 1)
    core-01: 22 (guest) => 2222 (host) (adapter 1)
==> core-01: Running 'pre-boot' VM customizations...
==> core-01: Booting VM...
==> core-01: Waiting for machine to boot. This may take a few minutes...
    core-01: SSH address: 127.0.0.1:2222
    core-01: SSH username: core
    core-01: SSH auth method: private key
    core-01: Warning: Remote connection disconnect. Retrying...
==> core-01: Machine booted and ready!
==> core-01: Setting hostname...
==> core-01: Configuring and enabling network interfaces...
  • Setup shell environment so local Docker client may communicate with Docker daemon on VM
unset DOCKER_TLS_VERIFY
unset DOCKER_CERT_PATH
export DOCKER_HOST=tcp://127.0.0.1:2375
  • Show Vagrant environments on host machine
vagrant global-status
id       name    provider   state   directory                                                               
------------------------------------------------------------------------------------------------------------
824ce36  core-01 virtualbox running /Users/Ls/code/docker-host/coreos-vagrant                               

The above shows information about all known Vagrant environments
on this machine. This data is cached and may not be completely
up-to-date. To interact with any of the machines, you can go to
that directory and run Vagrant, or you can use the ID directly
with Vagrant commands from any directory. For example:
"vagrant destroy 1a2b3c4d"
  • Check Docker environment variables
env | grep -i DOCKER
  • Important: Restart Bash terminal to avoid error connecting to Docker Daemon 
    $ docker info
An error occurred trying to connect: Get http://127.0.0.1:2375/v1.24/info: EOF

  • Connect to Docker Daemon (running on port 2375). Note use unencrypted only in Development.

  • Important Note: Docker Daemon of Docker for Mac or Docker Toolbox already runs in a Linux VM, so you do not need to (and cannot) run it manually with dockerd. It is already running in the whale icon is in the top bar. https://github.com/docker/docker/issues/27102
  • Connect to Shell on Vagrant VM
vagrant ssh
Last login: Thu Sep 29 09:06:08 UTC 2016 from 10.0.2.2 on ssh
CoreOS alpha (1185.0.0)
core@core-01 ~ $ pwd
/home/core
core@core-01 ~ $ help
GNU bash, version 4.3.46(1)-release (x86_64-cros-linux-gnu)

Share Folders between Vagrant VM (CoreOS) and Host (OSX). IDE may now be used to edit files on Vagrant VM.

  • Enable synced folders by editing Vagrantfile as follows (uncommenting the config.vm.synced_folder line of code)
config.vm.synced_folder "~/code/docker-host-coreos-share", "/home/core/share", id: "core", :nfs => true, :mount_options => ['nolock,vers=3,udp']
  • Restart Vagrant VM
vagrant halt
vagrant up
...
==> core-01: Exporting NFS shared folders...
==> core-01: Preparing to edit /etc/exports. Administrator privileges will be required...
Password:
==> core-01: Mounting NFS shared folders...
==> core-01: Machine already provisioned. Run `vagrant provision` or use the `--provision`
==> core-01: flag to force provisioning. Provisioners marked to run always will still run.
vagrant ssh

  • Note: If an exports invalid error occurs then open and validate the contents of /etc/exports on the Host (OSX)

  • Check that files synchronise between these folders

Interactive Docker Containers within Vagrant VM (Docker host):

  • From the shell of Vagrant VM, test download a Docker Image from Docker Hub for the base distribution specified using Docker Daemon (i.e. Ubuntu, Fedora, and CentOS). A Docker Container inside Vagrant VM filesystem is created based on the downloaded Docker Image. The new Docker Container has a network, IP addresss, and bridge to communicate with localhost. The command and flags run by Docker on the new container launch an interactive Bash shell instance of the host (i.e. Ubuntu) in it that we are logged into as root user.
docker run --name luke_web_container --rm -ti alpine:latest /bin/bash
docker run --name luke_db_container --rm -ti ubuntu:latest /bin/bash
docker run --rm -ti fedora:latest /bin/bash
docker run --rm -ti centos:latest /bin/bash
Unable to find image 'ubuntu:latest' locally
latest: Pulling from library/ubuntu
cad964aed91d: Pull complete 
3a80a22fea63: Pull complete 
50de990d7957: Pull complete 
61e032b8f2cb: Pull complete 
9f03ce1741bf: Pull complete 
Digest: sha256:28d4c5234db8d5a634d5e621c363d900f8f241240ee0a6a978784c978fe9c738
Status: Downloaded newer image for ubuntu:latest
  • List all available Linux commands, aliases, built-ins, keywords, and functions
  • Show Docker Container Host Entries
  • Show Docker Container IP address
  • Show Docker Container Running Processes
  • Install software package on Docker Container
compgen -A function -abck
cat /etc/hosts
hostname -I
ps -aux
apt-get update && apt-get install <software-name>
  • Exit Docker Container to return to Vagrant VM
exit
  • List Docker Containers (both stopped and running) from within Vagrant VM. Note: Those that were run with --rm flag are not shown.
docker ps -a
CONTAINER ID  IMAGE  COMMAND  CREATED  STATUS  PORTS  NAMES
  • List Docker Containers (only last 2 that were stopped or running)
docker ps -n 2
  • Create Container (but not Run), Start, Restart, Re-Attach (to process using original Options, i.e. command, flags), and Delete Docker Container
docker create --name luke_web_container_alpine --rm -ti alpine:latest /bin/bash
docker start luke_web_container
docker restart luke_web_container
docker attach luke_web_container
docker rm luke_web_container

Important Note: Press Enter after Re-Attach to Container (otherwise it hangs)

Daemonized (non-Interactive) Docker Containers within Vagrant VM (Docker host) for running apps and services:

  • Create and run a Daemonized Docker Container within Vagrant. Flags detach the container to the background and performs a command until process stopped.
docker run --name daemon_luke -d --restart=on-failure:5 ubuntu /bin/sh -c "while true; do echo hello world; sleep 100; done"
  • View under the hood of the Docker Container by debugging the last few log entries
docker logs daemon_luke
  • Monitor (follow) the Docker Container log entries using the -f flag (similar to the tail -f binary) with timestamps and without reading previous entries to the log file
docker logs -ft --details --tail 0 daemon_luke

  • Note: Transport Layer Security (TLS) is a cryptographic protocol that provides communications security over a network (predecessor SSL)

  • Inspect processes (and PIDs) of Daemonized Docker Container

docker start daemon_luke
docker top daemon_luke
UID   PID   PPID  C  STIME  TTY  TIME      CMD
root  2603  2591  0  13:24  ?    00:00:00  /bin/sh -c while true; do echo hello world; sleep 100; done
root  2630  2603  0  13:24  ?    00:00:00  sleep 100
  • Statistics (CPU, memory, network, storage I/O, metrics) of Daemonized Docker Container
docker stats daemon_luke daemon_other
CONTAINER     CPU %   MEM USAGE / LIMIT   MEM %   NET I/O         BLOCK I/O   PIDS
c3b9e4ca6990  0.00%   364 KiB / 997 MiB   0.04%   648 B / 738 B   0 B / 0 B   2
...
  • Background Processes (using detach -d flag) started on already running containers are executed (management, monitoring, or maintenance) with Process Owner in Daemonized Docker Container
docker exec -u "luke" -d daemon_luke touch /etc/new_config_file
  • Note: docker exec works on already running containers, otherwise error returned:
Error response from daemon: Container 6ca0ccd2 is not running
  • Inspect Docker Container (config info, names, commands)
docker inspect daemon_luke
[
    {
        "Id": "c3b9",
        "Created": "2016-10-03T10:21:04.773981628Z",
        "Path": "/bin/sh",
        "Args": [
            "-c",
            "while true; do echo hello world; sleep 100; done"
        ],
        "State": {
            "Status": "exited",
            "Running": false,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 0,
            "ExitCode": 137,
            "Error": "",
            "StartedAt": "2016-10-03T13:28:24.784919827Z",
            "FinishedAt": "2016-10-03T13:29:34.533835206Z"
        },
        "Image": "sha256:c73a085dc3782",
        "ResolvConfPath": "/var/lib/docker/containers/c3b9/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/c3b9/hostname",
        "HostsPath": "/var/lib/docker/containers/c3b9/hosts",
        "LogPath": "/var/lib/docker/containers/c3b9/c3b9-json.log",
        "Name": "/daemon_luke",
        "RestartCount": 0,
        "Driver": "overlay",
        "MountLabel": "system_u:object_r:svirt_lxc_file_t:s0:c115,c543",
        "ProcessLabel": "system_u:system_r:svirt_lxc_net_t:s0:c115,c543",
        "AppArmorProfile": "",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": null,
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "default",
            "PortBindings": {},
            "RestartPolicy": {
                "Name": "no",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "CapAdd": null,
            "CapDrop": null,
            "Dns": [],
            "DnsOptions": [],
            "DnsSearch": [],
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": null,
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "ConsoleSize": [
                0,
                0
            ],
            "Isolation": "",
            "CpuShares": 0,
            "Memory": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": null,
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": [],
            "DiskQuota": 0,
            "KernelMemory": 0,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": -1,
            "OomKillDisable": false,
            "PidsLimit": 0,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0
        },
        "GraphDriver": {
            "Name": "overlay",
            "Data": {
                "LowerDir": "/var/lib/docker/overlay/dd47/root",
                "MergedDir": "/var/lib/docker/overlay/d0f6/merged",
                "UpperDir": "/var/lib/docker/overlay/d0f6/upper",
                "WorkDir": "/var/lib/docker/overlay/d0f6/work"
            }
        },
        "Mounts": [],
        "Config": {
            "Hostname": "c3b9",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
            ],
            "Cmd": [
                "/bin/sh",
                "-c",
                "while true; do echo hello world; sleep 100; done"
            ],
            "Image": "ubuntu",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": {}
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "535d520572a2113",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": null,
            "SandboxKey": "/var/run/docker/netns/535d520572a2",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "bridge": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": null,
                    "NetworkID": "8c0a",
                    "EndpointID": "",
                    "Gateway": "",
                    "IPAddress": "",
                    "IPPrefixLen": 0,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": ""
                }
            }
        }
    }
]
docker inspect --format='' \
daemon_luke daemon_luke2
docker inspect --format='' daemon_luke
true false
172.18.0.2
  • View Docker Container storage location on Vagrant VM (i.e. CoreOS). Start shell as superuser first
sudo -s
cd /var/lib/docker/containers
ls

Log Collection

Chapter 2 - Dockerfile, Docker Images

  • Docker Container based on a Docker Image (based on a public image or custom) stored in Repository of a Registry
  • Docker Image has filesystem layers with mapping to each build step
  • Docker Image Management storage backend communicates with underlying Linux filesystem to build layers into usable image
  • Docker Image Management storage backends include fast Copy-on-Write (CoW)
  • Docker Image Management storage backends supported include AUFS, BTRS, Device-mapper, and overlayfs
  • Docker Hub is the default Registry
  • Top-Level Repositories are managed by Docker and vendors (i.e. ubuntu)
  • Top-Level Docker Images offer bare runtime to run the distribution
  • User Repositories are by individual developers (i.e. ltfschoen/ruby)
  • Tag comprises a series of image layers in single repository (representative of version control)

  • Docker Image is built upon the Union Mount of its parent image layers above the base image (each layer is a filesystem)

  • Show Docker Images downloaded from repositories that contain layers and metadata
docker images
REPOSITORY  TAG      IMAGE ID        CREATED      SIZE
ubuntu      latest   c73a085dc378    2 days ago   127.1 MB
alpine      latest   ee4603260daa    10 days ago  4.799 MB
  • Go to local storage location of Docker Images
vagrant ssh
sudo -s
cd /var/lib/docker
  • Pull specific Tag (i.e. 3.4) for Top-Level Docker Image
docker pull alpine:3.4
docker images
  • Search Docker Hub for publicly available Docker Images that may already be pre-installed with applications
docker search ub*
NAME      DESCRIPTION                                     STARS   OFFICIAL  AUTOMATED
ubuntu    Ubuntu is a Debian-based Linux operating s...   4817    [OK]      
ruby      Ruby is a dynamic, reflective, object-orie...   739     [OK] 
...
docker login
docker logout

Docker Build Command (using Dockerfile)

Docker File

  • Dockerfile and docker build is repeatable, transparent, and idempotent, and so is preferred over docker commit
  • Dockerfile instructs how Docker Image is to be assembled using application code.
  • Dockerfile uses DSL with instructions paired with arguments on each line of code
  • Dockerfile lines of code are processed from top to bottom
  • Each line of code creates a new Docker Image Layer.
  • Only layers deviating from previous Docker Image Layer must be built
  • Dockerfile docker build process is: 1) Docker runs an initial new Docker Container from the base Docker Image (FROM) 2) Docker executes instruction (first line of code) inside initial Docker Container 3) Docker runs a docker commit to commit initial Docker Image Layer 4) Docker runs a second new Docker Container based on the initial Docker Image Layer 5) Docker executes instruction (next line of code) inside second Docker Container 6) Docker runs a docker commit to commit second Docker Image Layer 7) Docker runs a third new Docker Container based on the second Docker Image Layer 8) Docker repeats steps 5) to 7) for subsequent lines of code to build up a Docker Container
  • Docker Container that is built houses the application (i.e. Rails-based app) with all dependencies

  • Debug errors when building from Dockerfile by running Docker Container from the last successfully created Docker Image

  • Dockerfile FROM
    • Default instance is base Linux image
    • Custom Docker Registry offers official framework Docker Images and Tags (i.e. Node.js). Docker Image base inherits from Docker Container defined by FROM (i.e. node:0.10.33 specifically locks Docker Image base to specific Ubuntu Linux image running Node 0.10.33)
  • Dockerfile MAINTAINER - provides author field metadata in Docker Image
  • Dockerfile LABEL - provides KV pairs of metadata in Docker Image for search purposes. Find with docker inspect
  • Dockerfile USER
    • Important: Docker Containers run processes on the host machine kernel. Do not be run as root user in Production for security
  • Dockerfile ENV - set shell environment variables used in build process on the image
  • Dockerfile RUN
    • execute instruction/command to append new Docker Image Layer on the current Docker Image
    • start and create file structure, and install dependencies
    • commit the new Docker Image Layer if command successful
  • Important: Reduce build time by abstracting common RUN processes (i.e. updates) into code lines (Docker Image layers) of Base Image
  • Important: Reduce rebuild time by ordering commands with code imports at the end (as all layers after introduced change are rebuilt)
  • Important: Combine logically grouped commands into single Dockerfile code line since each instruction creates a new Docker Image layer (i.e. use ADD to import scripts and required supporting files from local filesystem into image)
  • Dockerfile WORKDIR - changes working directory in image for remaining build instructions
  • Dockerfile CMD - final instruction launches process to run in Docker Container
  • Dockerfile EXPOSE - ports Docker Container listens to. docker run flag (--expose) opens these ports at runtime
  • Important: Single function in CMD to easy horizontal scaling of individual functions in architecture

Build Custom Container

  • Build new Custom Container based on Top-Level Container that we build from a Top-Level Image

Example - Static Web Server

  • Create build environment (build context) for Docker to upload to Docker Daemon when build is run.

  • Note: Previously we configred the Vagantfile to synchronise the following folders between Vagrant VM (CoreOS) and Host (OSX):

    • ~/code/docker-host-coreos-share (Host)
    • /home/core/share (Vagrant VM)
vagrant up
vagrant ssh

  • Create Dockerfile that builds a Docker Image where the Docker Container is a static web server

  • Note: Perform these on Host machine since folders are synchronised

cd ~/code/docker-host-coreos-share
mkdir ltfschoen && mkdir ltfschoen/static_web && cd ltfschoen/static_web
touch Dockerfile
# Version: 0.0.1
FROM ubuntu:16.04
MAINTAINER Luke Schoen "ltfschoen@gmail.com"

# changing this date causes cache to bust from this line of code and a re-build of subsequent image layers 
ENV REFRESHED_AT 2016-10-07

# refresh with latest content
RUN apt-get update && apt-get install -y nginx
RUN echo 'Hi, I am from planet Docker Container' \
   >/var/www/html/index.html
EXPOSE 80

  • Build Dockerfile into Docker Image. Tagging with repository/image_name:version. Use -f flag to specify directory within build context containing Dockerfile.

  • Note: docker build must be run on the Vagrant VM (CoreOS), otherwise the following error message appears Cannot connect to the Docker daemon. Is the docker daemon running on this host?

core@core-01 ~/share $ pwd
/home/core/share
core@core-01 ~/share $ sudo docker build -t="ltfschoen/static_web:v1" --file="/home/core/share/ltfschoen/static_web/Dockerfile" .
Sending build context to Docker daemon 10.24 kB
Step 1 : FROM ubuntu:16.04
16.04: Pulling from library/ubuntu
Digest: sha256:28d4c5234db8d5a634d5e621c363d900f8f241240ee0a6a978784c978fe9c737
Status: Downloaded newer image for ubuntu:16.04
 ---> c73a085dc378
Step 2 : MAINTAINER Luke Schoen "ltfschoen@gmail.com"
 ---> Running in 1a252c810904
 ---> 8dc999c3da79
Removing intermediate container 1a252c810904
Step 3 : RUN apt-get update && apt-get install -y nginx
 ---> Running in fd86d184e875
Get:1 http://archive.ubuntu.com/ubuntu xenial InRelease [247 kB]
Get:2 http://archive.ubuntu.com/ubuntu xenial-updates InRelease [95.7 kB]
...
Fetched 24.3 MB in 53s (457 kB/s)
Reading package lists...
Building dependency tree...
Reading state information...
The following additional packages will be installed:
  fontconfig-config fonts-dejavu-core geoip-database libexpat1 libfontconfig1
  libfreetype6 libgd3 libgeoip1 libicu55 libjbig0 libjpeg-turbo8 libjpeg8
  libpng12-0 libssl1.0.0 libtiff5 libvpx3 libx11-6 libx11-data libxau6 libxcb1
  libxdmcp6 libxml2 libxpm4 libxslt1.1 nginx-common nginx-core sgml-base ucf
  xml-core
Suggested packages:
  libgd-tools geoip-bin fcgiwrap nginx-doc ssl-cert sgml-base-doc debhelper
The following NEW packages will be installed:
  fontconfig-config fonts-dejavu-core geoip-database libexpat1 libfontconfig1
  libfreetype6 libgd3 libgeoip1 libicu55 libjbig0 libjpeg-turbo8 libjpeg8
  libpng12-0 libssl1.0.0 libtiff5 libvpx3 libx11-6 libx11-data libxau6 libxcb1
  libxdmcp6 libxml2 libxpm4 libxslt1.1 nginx nginx-common nginx-core sgml-base
  ucf xml-core
0 upgraded, 30 newly installed, 0 to remove and 4 not upgraded.
Need to get 15.5 MB of archives.
After this operation, 57.4 MB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu xenial/main amd64 libxau6 amd64 1:1.0.8-1 [8376 B]
Get:2 http://archive.ubuntu.com/ubuntu xenial/main amd64 sgml-base all 1.26+nmu4ubuntu1 [12.5 kB]
...
debconf: delaying package configuration, since apt-utils is not installed
Fetched 15.5 MB in 34s (451 kB/s)
Selecting previously unselected package libxau6:amd64.
(Reading database ... 7256 files and directories currently installed.)
Preparing to unpack .../libxau6_1%3a1.0.8-1_amd64.deb ...
Unpacking libxau6:amd64 (1:1.0.8-1) ...
...
Preparing to unpack .../nginx-common_1.10.0-0ubuntu0.16.04.2_all.deb ...
Unpacking nginx-common (1.10.0-0ubuntu0.16.04.2) ...
Selecting previously unselected package nginx-core.
Preparing to unpack .../nginx-core_1.10.0-0ubuntu0.16.04.2_amd64.deb ...
Unpacking nginx-core (1.10.0-0ubuntu0.16.04.2) ...
Selecting previously unselected package nginx.
Preparing to unpack .../nginx_1.10.0-0ubuntu0.16.04.2_all.deb ...
Unpacking nginx (1.10.0-0ubuntu0.16.04.2) ...
Processing triggers for libc-bin (2.23-0ubuntu3) ...
Processing triggers for systemd (229-4ubuntu8) ...
Setting up libxau6:amd64 (1:1.0.8-1) ...
...
debconf: unable to initialize frontend: Dialog
debconf: (TERM is not set, so the dialog frontend is not usable.)
debconf: falling back to frontend: Readline
debconf: unable to initialize frontend: Readline
debconf: (Can't locate Term/ReadLine.pm in @INC (you may need to install the Term::ReadLine module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at /usr/share/perl5/Debconf/FrontEnd/Readline.pm line 7.)
debconf: falling back to frontend: Teletype
...
Setting up geoip-database (20160408-1) ...
...
Setting up nginx-common (1.10.0-0ubuntu0.16.04.2) ...
Setting up nginx-core (1.10.0-0ubuntu0.16.04.2) ...
invoke-rc.d: could not determine current runlevel
invoke-rc.d: policy-rc.d denied execution of start.
Setting up nginx (1.10.0-0ubuntu0.16.04.2) ...
Processing triggers for libc-bin (2.23-0ubuntu3) ...
...
 ---> 5fc4e6d013ea
Removing intermediate container fd86d184e875
Step 4 : RUN echo 'Hi, I am from planet Docker Container'    >/var/www/html/index.html
 ---> Running in 7d5debfb7bad
 ---> 6c588ca828ca
Removing intermediate container 7d5debfb7bad
Step 5 : EXPOSE 80
 ---> Running in 5e09ac8e5ce4
 ---> 477e991bce63
Removing intermediate container 5e09ac8e5ce4
Successfully built 477e991bce63
  • Check Docker Images updated
core@core-01 ~/share $ docker images
REPOSITORY             TAG                 IMAGE ID            CREATED             SIZE
ltfschoen/static_web   v1                  477e991bce63        19 minutes ago      222.8 MB
ubuntu                 16.04               c73a085dc378        10 days ago         127 MB
  • Check Docker Image history (Docker Image Layers are shown)
core@core-01 ~/share $ docker history 477e991bce63
IMAGE               CREATED             CREATED BY                                      SIZE                COMMENT
477e991bce63        About an hour ago   /bin/sh -c #(nop)  EXPOSE 80/tcp                0 B                 
6c588ca828ca        About an hour ago   /bin/sh -c echo 'Hi, I am from planet Docker    38 B                
5fc4e6d013ea        About an hour ago   /bin/sh -c apt-get update && apt-get install    95.81 MB            
8dc999c3da79        About an hour ago   /bin/sh -c #(nop)  MAINTAINER Luke Schoen "lt   0 B                 
c73a085dc378        10 days ago         /bin/sh -c #(nop)  CMD ["/bin/bash"]            0 B                 
<missing>           10 days ago         /bin/sh -c mkdir -p /run/systemd && echo 'doc   7 B                 
<missing>           10 days ago         /bin/sh -c sed -i 's/^#\s*\(deb.*universe\)$/   1.895 kB            
<missing>           10 days ago         /bin/sh -c rm -rf /var/lib/apt/lists/*          0 B                 
<missing>           10 days ago         /bin/sh -c set -xe   && echo '#!/bin/sh' > /u   745 B               
<missing>           10 days ago         /bin/sh -c #(nop) ADD file:cd937b840fff16e04e   127 MB

Docker Image Templates using Build Cache

  • Docker uses cache and re-builds from earliest image layer change. Bypass using --no-cache flag.

  • Dockerfile should have template at top of file with package repositories and updates so cache is hit.

Launch Container using Docker Image

  • -d flag detaches in background for long processes (i.e. Nginx daemon)
  • -p flag manages network ports published by Docker at container runtime to the host machine, with options including:
  • Mapping of randomly assigned Docker Host port(i.e. 32768 to 61000) to Port 80 on Docker Container
  • Mapping of specific Docker Host port (i.e. 8080) direct binding to Port 80 on Docker Container (i.e. sudo docker run -d -p 8080:80 --name static_web ltfschoen/static_web_8080 nginx -g "daemon off;")
  • Mapping of specific Interface (i.e. 127.0.0.1) and specific Docker Host port (i.e. 8080) direct binding to Port 80 on Docker Container (i.e. sudo docker run -d -p 127.0.0.1:8080:80 --name static_web ltfschoen/static_web_8080_i127_0_0_1:v1 nginx -g "daemon off;")
  • Mapping of specific Interface (i.e. 127.0.0.1) and random Docker Host port to Port 80 on Docker Container (i.e. sudo docker run -d -p 127.0.0.1::80 --name static_web ltfschoen/static_web_random_i127_0_0_1 nginx -g "daemon off;")
  • Mapping of random Docker Host port to Port 80 on Docker Container and publish additional ports specified with EXPOSE instructions (i.e. sudo docker run -d -P --name static_web_all ltfschoen/static_web_all nginx -g "daemon off;")
  • Note: Random port is assigned when not specified
  • Note: Only one Docker Container may be bound to a specific port on host machine

  • Note: Bind UDP ports by appending to port binding /udp

  • Command to run (i.e. nginx -g "daemon off;") launches Nginx in foreground to run web server

  • Port Mapping Refresher
sudo docker run -d -p 127.0.0.1:8080:80 --name static_web ltfschoen/static_web:v1 \
nginx -g "daemon off;"
  • Check Port mapping on host machine assigned to a given port 80 of a Docker Container ID/name
core@core-01 ~/share $ docker ps -l  
CONTAINER ID   IMAGE                     COMMAND                  CREATED          STATUS          PORTS                    NAMES
f169101630a7   ltfschoen/static_web:v1   "nginx -g 'daemon off"   31 seconds ago   Up 30 seconds   127.0.0.1:8080->80/tcp   static_web
core@core-01 ~/share $ docker port static_web 80
127.0.0.1:8080
  • Selectively Inspect Docker Container
core@core-01 ~/share $ docker inspect --format='' static_web
172.18.0.2
core@core-01 ~/share $ docker inspect --format='' static_web
true
  • Connect to the Docker Container housing the Nginx web server using cURL
core@core-01 ~/share $ curl localhost:8080
Hi, I am from planet Docker Container
docker stop static_web
docker restart static_web
docker ping 172.18.0.2
  • Attempt with the following:
ip route show
uname -a
sudo iptables -t nat -L -n
route -n
ifconfig
  • Destroy Docker Containers and re-build Docker Image with different Dockerfile with change to EXPOSE 80 (not necessary to delete the Docker Container with docker rmi ltfschoen/static_web:v1 though due to caching)
docker rm docker rm static_web
sudo docker build -t="ltfschoen/static_web:v1" --file="/home/core/share/ltfschoen/static_web/Dockerfile" .
  • Check port 80 is correctly binded by Nginx
core@core-01 ~ $ netstat -nap | grep :80
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
tcp        0      0 127.0.0.1:8080          0.0.0.0:*               LISTEN      -
  • Check IP Table rules that are in place
sudo iptables -L

  • Attempt Bridged Adapter option

  • Error below when running vagrant up after changing from NAT to Bridge Adapter in VirtualBox GUI > Settings > Network > Adapter 1 because Adapter 3 is already set to Bridge Adapter, and making following changes to Vagrantfile:

# config.vm.network :private_network, ip: ip
config.vm.network "public_network", :bridge => 'en1: Wi-Fi (AirPort)', ip: ip
There was an error while executing `VBoxManage`, a CLI used by Vagrant
for controlling VirtualBox. The command and stderr is shown below.
Command: ["modifyvm", "8f638618-c3dd-489f-8691-22a769273c46", "--natpf1", "tcp2375,tcp,127.0.0.1,2375,,2375", "--natpf1", "ssh,tcp,127.0.0.1,2222,,22"]
Stderr: VBoxManage: error: A NAT rule of this name already exists
VBoxManage: error: Details: code NS_ERROR_INVALID_ARG (0x80070057), component NATEngineWrap, interface INATEngine, callee nsISupports
VBoxManage: error: Context: "AddRedirect(Bstr(strName).raw(), proto, Bstr(strHostIp).raw(), RTStrToUInt16(strHostPort), Bstr(strGuestIp).raw(), RTStrToUInt16(strGuestPort))" at line 1758 of file VBoxManageModifyVM.cpp

  • Solved with vagrant halt, started VM from Virtual Box GUI, then goto bar menu and chose quit VM. https://github.com/mitchellh/vagrant/issues/1809 Note: This solution causes another cascading problem:

  • Error encountered when run vagrant up again

NFS requires a host-only network to be created.
Please add a host-only network to the machine (with either DHCP or a
static IP) for NFS to work.
ip = "172.17.8.#{i+100}"
config.vm.network :private_network, ip: ip
config.vm.network "public_network", :bridge => 'en1: Wi-Fi (AirPort)'
  • Note: Still unable to get response from VM when curl from Host to Docker Container IP/port (i.e. curl 172.18.0.2:8080)

Misc Links: http://stackoverflow.com/questions/33814696/how-to-connect-to-a-docker-container-from-outside-the-host-same-network-windo https://www.virtualbox.org/manual/ch06.html http://stackoverflow.com/questions/25433488/vagrant-can-not-ping-guest-machine-from-the-host http://serverfault.com/questions/495914/vagrant-slow-internet-connection-in-guest https://friendsofvagrant.github.io/v1/docs/bridged_networking.html

TODO Default Router approach: http://superuser.com/questions/752954/need-to-do-bridged-adapter-only-in-vagrant-no-nat https://www.vagrantup.com/docs/networking/public_network.html

TODO Key Link: Alternative using Docker Compose instead of Vagrant https://hharnisc.github.io/

In progress up to CMD section on page 99

Other

  • Note: Docker Toolbox is for older Macs.
  • Uninstall Docker Toolbox. Download shell script and run with sudo bash uninstall.sh https://github.com/docker/toolbox/blob/master/osx/uninstall.sh

TODO

Written on September 28, 2016