{"id":52,"date":"2018-06-30T22:48:33","date_gmt":"2018-06-30T22:48:33","guid":{"rendered":"http:\/\/edspace.com\/blog\/?p=52"},"modified":"2018-07-06T14:52:04","modified_gmt":"2018-07-06T14:52:04","slug":"local-development-with-docker","status":"publish","type":"post","link":"http:\/\/edspace.com\/blog\/2018\/06\/30\/local-development-with-docker\/","title":{"rendered":"Local Development with Docker"},"content":{"rendered":"

Alright, I wish I could take back my previous Docker entry<\/a>. It was pretty useless, so I’m going to take another shot at this at do it right. I’ve given this talk about Docker about a dozen times in person, done a recorded (sadly, proprietary) teaching session on it, but I still find myself giving it over and over again, so I thought it might be best to just start writing it down. The target audience for this is people who have only really heard of Docker without knowing what it is. At the end of this guide, you should be able to write your own Dockerfile for your project and deploy it locally for testing purposes.<\/p>\n

What is Docker?<\/h2>\n

You can think of Docker as yet another layer of virtualization, one that’s not as heavyweight as full hardware virtualization or paravirtualization. It’s a level known as “operating-system-level virtualization,” where the guest machine shares the same kernel as the host, but gets its own file system to itself and network stack. This allows you to run your application as a process on the host operating system while fooling the guest application into thinking that it has all of its own resources to use.<\/p>\n

\"\"<\/p>\n

What should I use it for?<\/h2>\n

Docker makes it easy to spin up multiple stateless application services onto a cluster. If anything requires storage, e.g. a database, it is much better to use a standard virtual machine with dedicated mounted storage. Docker is not designed to manipulate stored data very efficiently.<\/p>\n

Installation and Example<\/h2>\n

The first step, obviously, is to install Docker. Follow the directions here<\/a> and find your platform.<\/p>\n

After you have it installed, we’ll get a quick “Hello, World!” going. We’ll execute two lines, docker pull hello-world<\/em> and docker run hello-world<\/em>.<\/p>\n

$ docker pull hello-world\r\nUsing default tag: latest\r\nlatest: Pulling from library\/hello-world\r\n9bb5a5d4561a: Pull complete \r\nDigest: sha256:f5233545e43561214ca4891fd1157e1c3c563316ed8e237750d59bde73361e77\r\nStatus: Downloaded newer image for hello-world:latest\r\n$ docker run hello-world\r\n\r\nHello from Docker!\r\nThis message shows that your installation appears to be working correctly.\r\n\r\nTo generate this message, Docker took the following steps:\r\n 1. The Docker client contacted the Docker daemon.\r\n 2. The Docker daemon pulled the \"hello-world\" image from the Docker Hub.\r\n    (amd64)\r\n 3. The Docker daemon created a new container from that image which runs the\r\n    executable that produces the output you are currently reading.\r\n 4. The Docker daemon streamed that output to the Docker client, which sent it\r\n    to your terminal.\r\n\r\nTo try something more ambitious, you can run an Ubuntu container with:\r\n $ docker run -it ubuntu bash\r\n\r\nShare images, automate workflows, and more with a free Docker ID:\r\n https:\/\/hub.docker.com\/\r\n\r\nFor more examples and ideas, visit:\r\n https:\/\/docs.docker.com\/engine\/userguide\/\r\n\r\n<\/pre>\n
The first line pulls down an image from hub.docker.com <\/a>and the second instantiates a container from that image and runs it. Now, this could all be done with the run<\/em> command, but I broke it out into different steps to show two separate steps. The first is to obtain the image, while the second is to create a container from that image.<\/p>\n
We’ll take a look at the two separately with docker images<\/em> and docker container<\/em>.<\/p>\n
$ docker images \r\nREPOSITORY          TAG                 IMAGE ID            CREATED             SIZE\r\nhello-world         latest              e38bc07ac18e        2 months ago        1.85kB\r\n$ docker container ls -a\r\nCONTAINER ID        IMAGE               COMMAND             CREATED             STATUS                      PORTS               NAMES\r\n79530d8a293c        hello-world         \"\/hello\"            38 minutes ago      Exited (0) 38 minutes ago                       nervous_joliot\r\n<\/pre>\n
We see that we have an image that we downloaded from the hub. We also have a container created using said image. It’s assigned a randomly generated name nervous_joliot<\/em> because we didn’t bother naming it. You can name your containers when you run them with the –name<\/em> directive, e.g docker run –name my_hello_world hello-world<\/em>.<\/p>\n
Images vs. Containers<\/h2>\n
Let’s go into more detail on what images and containers are as they pertain to Docker.<\/p>\n
Images<\/h3>\n
Images are immutable data layers that contain portions of the filesystem needed to run your application. Typically, you would start with the base operating system image, add language\/library support, then top it off with your application and ancillary files.<\/p>\n
\"\"<\/p>\n
Each image layer starts by declaring a base image to inherit from. Notice that earlier, when you were pulling the hello-world<\/em> image, it was downloading four images. It was downloading not only the hello-world <\/em>image layer, but also all the layers that it depends on. We’ll cover this more in depth later on.<\/p>\n
Containers<\/h3>\n
Containers are instantiated instances of images that can support execution of the installed application. You can think of images as a class definitions in Object Oriented Programming, an containers are analogous as objects. You can create multiple containers from the same image, allowing you to spin up a cluster of processes with a few simple commands.<\/p>\n
When a container is created, a new read\/write layer is introduced on top of the the existing image layers. If a change is made to a file existing in an image layer, that file is copied into the container read\/write layer while the image is untouched.<\/p>\n
Creating Dockerfiles<\/h2>\n
A Dockerfile<\/em> is a build descriptor for a Docker image, much like a Makefile<\/em> is used to build your application (if you still write C-code). You would typically include you Dockerfile<\/em> inside your project, run your regular project artifact build, and then run, either manually or via a build target (make<\/em> docker <\/em>or mvn -Pdocker<\/em>, etc) to produce your Docker image.<\/p>\n
For this example, we’ll take a look at Pastr<\/em>, a quick and dirty PasteBin clone I wrote with Python and a Redis storage backend. You can clone the project from here: https:\/\/gitlab.com\/ed11\/pastr<\/a>.<\/p>\n
The project uses Flask and Flask-Restful to serve up data stored from a connected Redis database presented with a VueJS UI front-end. (At the time of this writing, it’s still… very much lacking in quality; this was just the quickest thing I could slap together for a demo). The application just spins up a Flask WSGI development server for simplicity’s sake.<\/p>\n
Let’s take a look at the Dockerfile<\/em> to see what we’re building:<\/p>\n
FROM python:3.6\r\n\r\nADD pastr \/opt\/pastr\r\nCOPY requirements.txt \/opt\/\r\nRUN pip install -r \/opt\/requirements.txt\r\n\r\nCMD DB_SERVER=$DB_SERVER python \/opt\/pastr\/__init__.py\r\n<\/pre>\n
We’ll break this down line by line, remembering that each line creates its own image layer, as visualized earlier in the Images<\/em> section.<\/p>\n
FROM<\/h5>\n
This line tells the Docker<\/em> engine to start our image off by pulling the python<\/em> base image from the official repository (hub.docker.com<\/a>). The 3.6<\/em> after the colon tells it that we want specifically version 3.6 of Python. This is a tag<\/em> for the image. You can specify tags as a point release for your application or combine it with other text to mean variants (e.g.<\/em> myapp:<\/em>1.0-debug<\/em> to indicate that the image runs your application in debug<\/em> mode).<\/p>\n
ADD<\/h5>\n
This command copies the contents of the pastr<\/em> directory (in the current project working directory) into the image at \/opt\/<\/em>. Note there are special rules on what ADD<\/em> does. I recommend reading the documentation on the official Docker website:<\/p>\n
https:\/\/docs.docker.com\/engine\/reference\/builder\/<\/a><\/p>\n
COPY<\/h5>\n
This command copies a single file (the requirements.txt<\/em> file) into the \/opt<\/em> directory. If you’re still in doubt on what to use, USE COPY<\/em><\/strong> instead of ADD<\/em>.<\/p>\n
RUN<\/h5>\n
This command starts up a temporary container from the previous image layers, pops open a shell inside the virtual file system, and then begins executing commands. In this case, it simply runs the pip install<\/em> command, which, in a Python project, downloads all the required libraries needed to execute the application. You would normally use this to download third party dependencies, extract tarballs, or change permissions of files to grant execute privileges. After the command is done, it takes the mutable file system layer created by the container and saves it off as an immutable image layer.<\/p>\n
Be very mindful of the layer saving when using the RUN command when dealing with large files. For example, if you use this to download a large executable from a third party resource and then change the permissions, you will end up with two layers of the same size. Example:<\/p>\n
RUN wget http:\/\/my-file-server\/large-binary-executable\r\nRUN chmod +x large-binary-executable<\/pre>\n
Say our large-binary-executable<\/em> is 500MB. The first command will save off an image where the file is not executable, taking up 500MB. The second command will take the 500MB file, change the permissions, and save another image where the 500MB file is executable, essentially taking up 1GB of disk space for anyone who has to download it. Instead, you should run them in one command, like so:<\/p>\n
RUN wget http:\/\/my-file-server\/large-binary-executable && chmod +x large-binary-executable<\/pre>\n
CMD<\/h5>\n
The CMD <\/em>directive specifies the command that is to be executed when the container starts up. In our example, we run the python<\/em> command and point it to our application. The DB_SERVER=$DB_SERVER<\/em> is an environment variable that we pass to our application as a rudimentary form of configuration management.<\/p>\n
There are actually two ways to specify the container startup command: the CMD<\/strong> and the ENTRYPOINT <\/strong>directives. In most cases, these might be interchangeable, but there are nuanced differences on which to use, which are more suitable for a more advanced topic. For now, I will say that semantically, ENTRYPOINT<\/strong> is generally used to specify the executable and CMD <\/strong>is used to pass in parameters. The latter can be overridden on the command line prior to starting up.<\/p>\n
Building the Image<\/h3>\n
Using the Dockerfile<\/em>, we can build the image manually with the following command:<\/p>\n
docker build -t pastr .<\/pre>\n
What this command does is build the image using the current working directory (specified by the trailing dot) and naming it pastr<\/em> as indicated by the -t<\/em> directive. We can validate that the image is created by checking the image list.<\/p>\n
$ docker images\r\nREPOSITORY         TAG                 IMAGE ID            CREATED             SIZE\r\npastr              latest              6635be8bc083        4 seconds ago       941MB\r\n<\/pre>\n
Typically, this would be handled by your build script using a build target plugin, as mentioned earlier.<\/p>\n
Running the Container<\/h3>\n
We run the container much like we did with our hello-world<\/em> example above.<\/p>\n
docker run --detach --rm --name pastr1 --publish 5000:5000 pastr<\/pre>\n
A breakdown of the flags:<\/p>\n