# Docker | | Docker is a set of platform as a service (PaaS) products that use OS-level virtualization to deliver software in packages called containers. The service has both free and premium tiers. The software that hosts the containers is called Docker Engine. It was first released in 2013 and is developed by Docker, Inc. | |-|-| | | wikipedia:: [Docker (software)](https://en.wikipedia.org/wiki/Docker_(software)) | > [!summary]- Wikipedia Synopsis > Docker is a set of platform as a service (PaaS) products that use OS-level virtualization to deliver software in packages called containers. > > The service has both free and premium tiers. The software that hosts the containers is called Docker Engine. It was first released in 2013 and is developed by Docker, Inc.Docker is a tool that is used to automate the deployment of applications in lightweight containers so that applications can work efficiently in different environments in isolation. https://training.play-with-docker.com/ https://docs.docker.com/v17.12/get-started/ - Docker is a client program, named Docker, it's a command you type at the terminal. It's also a server program that listens for messages from that command, and manages a running Linux system (runs a Linux VM). - Building on top of facilities provided by the Linux kernel (primarily cgroups and namespaces), a Docker container, unlike a virtual machine, does not require or include a separate operating system.\[33\] Instead, it relies on the kernel's functionality and uses resource isolation for CPU and memory,\[31\] and separate namespaces to isolate the application's view of the operating system. - Docker Image - A Docker image is a read-only template used to build containers. Images are used to store and ship applications. - Has an id which is different than its container id - docker run - Takes an image and creates and starts a container (in contrast to docker commit which takes a modified container and turns it into an image) - docker run -p 4000:80 friendlyhello - -d - —detach - detached mode - leaves it running in background (can see it with docker ps) - —name - name\_of\_container - docker container run --detach --name mydb -e MYSQL\_ROOT\_PASSWORD=my-secret-pw mysql:latest - docker attach name\_or\_id - attach to a detached container - ctrl-p, ctrl-q - exits you from the attached container and leaves it running - docker run -p 4000:80 username/repository:tag - run directly from remote repository - -ti - terminal interactive - gives you a full terminal which gives you a shell, tab completion etc. - docker run --rm - Just want to run a container and delete it after you're done - Saves the step of afterwards doing docker rm container\_name - docker run -ti ubuntu bash -c "sleep 3; echo all done" - starts container, starts bash process and gives that process a command - docker system prune - clean up any resources — images, containers, volumes, and networks — that are dangling (not associated with a container) - docker system prune -a - To additionally remove any stopped containers and all unused images (not just dangling images), add the -a flag to the command - docker images - docker image ls - docker image rm nginx - Dockerfile - https://docs.docker.com/get-started/part2/\#dockerfile - Creates a custom image with the build command - Each lines takes the image from the previous line and makes another image (previous image is unchanged) - Try to put the parts of your dockerfile that changes most frequently toward the end so you don't have to rebuild as many parts of your dockerfile - Processes started on one line will ot be running on the next line, but environment variables will persist - You can build a new container from any previous image you've built so you can chain them together so to speak - docker build - docker build -t friendlyhello . - Docker File Syntax - https://docs.docker.com/engine/reference/builder/ - FROM - which image to download and start from - Should always be first - MAINTAINER - MAINTAINER Evan Harmon - RUN - Runs from the command line, waits for it to finish, saves the result - CMD - CMD "nano" "/notex.txt" - Exec form - \["/bin/nano", "/tmp/notes"\] - ENTRYPOINT - specifies the start of the command to run so you can just specify the last part of each command - ADD - add local files to container - can auto uncompress archives - can work with URLs - ENV - sets environment variables - WORKDIR - Sets working directory for the dockerfile and for the resulting container when you run it. Like typing cd basically. - USER - USER evan - COPY - EXPOSE - EXPOSE 8080 - VOLUME - Defines shared or ephemeral volumes - VOLUME \["/host/path/" "/container/path/"\] - VOLUME \["/shared-data"\] - \# Use an official Python runtime as a parent image FROM python:2.7-slim \# Set the working directory to /app WORKDIR /app \# Copy the current directory contents into the container at /app COPY . /app \# Install any needed packages specified in requirements.txt RUN pip install --trusted-host pypi.python.org -r requirements.txt \# Make port 80 available to the world outside this container EXPOSE 80 \# Define environment variable ENV NAME World \# Run app.py when the container launches CMD \["python", "app.py"\] - Docker Container - A Docker container is a standardized, encapsulated environment that runs applications. - Has an id which is different than its image id - Containers have one main process. Container stops when the main process stops. All other processes are child processes - Processes use cgroups - docker container --help - List Docker containers (running, all, all in quiet mode) - docker container ls - docker container ls --all - docker container ls -aq - docker container stop webserver - docker container rm webserver - docker container logs - docker container top - docker rm - deletes container - docker ps - -a - all, not just running containers, even stopped containers - -l - last container to exit - docker exec - add a process to a running container - e.g. docker exec -it idofcontainer bash - e.g. for debugging, etc. - BASH Access to Running Container - docker exec -it idofcontainer bash - or docker run -it ubuntu bash ? - docker exec -it idofcontainer /bin/ash for the common lightweight Alpine image - docker kill - stops a container - Docker for Mac o - Docker Swarm, Docker Machine, Clusters, & Nodes - docker swarm init - docker swarm join - docker swarm leave --force - Docker Swarm - Multi-container, multi-machine applications are made possible by joining multiple machines into a “Dockerized” cluster called a swarm. - A swarm is a group of machines that are running Docker and joined into a cluster. After that has happened, you continue to run the Docker commands you’re used to, but now they are executed on a cluster by a swarm manager. The machines in a swarm can be physical or virtual. After joining a swarm, they are referred to as nodes. Swarm managers can use several strategies to run containers, such as “emptiest node” -- which fills the least utilized machines with containers. Or “global”, which ensures that each machine gets exactly one instance of the specified container. You instruct the swarm manager to use these strategies in the Compose file, just like the one you have already been using. Swarm managers are the only machines in a swarm that can execute your commands, or authorize other machines to join the swarm as workers. Workers are just there to provide capacity and do not have the authority to tell any other machine what it can and cannot do. Up until now, you have been using Docker in a single-host mode on your local machine. But Docker also can be switched into swarm mode, and that’s what enables the use of swarms. Enabling swarm mode instantly makes the current machine a swarm manager. From then on, Docker runs the commands you execute on the swarm you’re managing, rather than just on the current machine. - Docker Swarm provides native clustering functionality for Docker containers, which turns a group of Docker engines into a single virtual Docker engine.\[46\] In Docker 1.12 and higher, Swarm mode is integrated with Docker Engine.\[47\] The swarm CLI utility allows users to run Swarm containers, create discovery tokens, list nodes in the cluster, and more.\[48\] The docker node CLI utility allows users to run various commands to manage nodes in a swarm, for example, listing the nodes in a swarm, updating nodes, and removing nodes from the swarm.\[49\] Docker manages swarms using the Raft Consensus Algorithm. According to Raft, for an update to be performed, the majority of Swarm nodes need to agree on the update. - docker-machine ssh myvm1 "docker swarm init --advertise-addr " - docker-machine ssh myvm2 "docker swarm join --token :2377" - Be sure to use port 2377, you have to add it manually for some weird technical reason - Configure a docker-machine shell to the swarm manager - So far, you’ve been wrapping Docker commands in docker-machine ssh to talk to the VMs. Another option is to run docker-machine env  to get and run a command that configures your current shell to talk to the Docker daemon on the VM. This method works better for the next step because it allows you to use your local docker-compose.yml file to deploy the app “remotely” without having to copy it anywhere. Type docker-machine env myvm1, then copy-paste and run the command provided as the last line of the output to configure your shell to talk to myvm1, the swarm manager. - Run docker-machine env myvm1 to get the command to configure your shell to talk to myvm1. $ docker-machine env myvm1 export DOCKER\_TLS\_VERIFY="1" export DOCKER\_HOST="tcp://192.168.99.100:2376" export DOCKER\_CERT\_PATH="/Users/sam/.docker/machine/machines/myvm1" export DOCKER\_MACHINE\_NAME="myvm1" \# Run this command to configure your shell: \# eval $(docker-machine env myvm1) Run the given command to configure your shell to talk to myvm1. eval $(docker-machine env myvm1) Run docker-machine ls to verify that myvm1 is now the active machine, as indicated by the asterisk next to it. - • To set your shell to talk to a different machine like myvm2, simply re-rundocker-machine env in the same or a different shell, then run the given command to point to myvm2. This is always specific to the current shell. If you change to an unconfigured shell or open a new one, you need to re-run the commands. Use docker-machine ls to list machines, see what state they are in, get IP addresses, and find out which one, if any, you are connected to. To learn more, see the Docker Machine getting started topics. • Alternatively, you can wrap Docker commands in the form ofdocker-machine ssh "", which logs directly into the VM but doesn’t give you immediate access to files on your local host. • On Mac and Linux, you can use docker-machine scp :\~to copy files across machines, but Windows users need a Linux terminal emulator like Git Bash for this to work. - Docker Machine - https://docs.docker.com/machine/overview/ - Basically, management of physical and vm hosts for Docker? - Works on local machine, or cloud, digital ocean VMs, etc. - docker-machine create --driver virtualbox myvm1 - docker-machine ls - docker-machine ssh "your ssh command" - docker-machine rm baz - Nodes - docker node ls - Registries, Repos, Docker Hub, & Docker Cloud - Registries & Repositories - A registry is a collection of repositories, and a repository is a collection of images—sort of like a GitHub repository, except the code is already built. An account on a registry can create many repositories. The docker CLI uses Docker’s public registry by default. - ?Docker Hub? - You can setup your own private registries - Docker clients connect to registries to download ("pull") images for use or upload ("push") images that they have built - You also have a local registry. Or you can use Docker Hub - docker commit id\_or\_name - Takes a modified container and turns it into an image (in contrast to docker run) - docker commit name\_of\_container new\_image\_name\_you\_want - both creates a new image from a container and lets you name it a friendly name - docker tag shaoutput\_of\_image new\_name\_of\_image - if you want to rename it - e.g., for pushing to Docker Hub - Apparently, you need to first tag the commited image with the username/reponame if you want the push to work. - E.g. I needed to do docker tag devops telosmachina/devops before I could do docker push telosmachina/devops - docker push - pushed to Docker Hub by default - docker push username/repository:tag - docker pull - Docker Hub o - Default registry where Docker looks for images - Docker Hub Webhooks - https://docs.docker.com/docker-hub/webhooks/ - Docker Cloud o - A place to actually host your docker apps I think as opposed to AWS, etc. - Services, Stacks, & Docker Compose - Docker Compose o - Docker Compose is a tool for defining and running multi-container Docker applications.\[40\] It uses YAML files to configure the application's services and performs the creation and start-up process of all the containers with a single command. The docker-compose CLI utility allows users to run commands on multiple containers at once, for example, building images, scaling containers, running containers that were stopped, and more.\[41\] Commands related to image manipulation, or user-interactive options, are not relevant in Docker Compose because they address one container.\[42\] The docker-compose.yml file is used to define an application's services and includes various configuration options. For example, the build option defines configuration options such as the Dockerfile path, the command option allows one to override default Docker commands, and more - For running multiple containers on a single machine - Best for testing and development. You'd probably move to Docker Swarm or something for deployment? - docker-compose CLI - docker-compose.yml File - version - services - build - ports - expose - volumes - links - depends\_on - environment - image - Docker Service - A Docker service allows containers to be run and also optionally scaled across multiple Docker daemons. The result is known as a swarm, a set of cooperating daemons that communicate through the Docker API. - Can create services - About services In a distributed application, different pieces of the app are called “services.” For example, if you imagine a video sharing site, it probably includes a service for storing application data in a database, a service for video transcoding in the background after a user uploads something, a service for the front-end, and so on. Services are really just “containers in production.” A service only runs one image, but it codifies the way that image runs—what ports it should use, how many replicas of the container should run so the service has the capacity it needs, and so on. Scaling a service changes the number of container instances running that piece of software, assigning more computing resources to the service in the process. Luckily it’s very easy to define, run, and scale services with the Docker platform -- just write a docker-compose.yml file. - To recap, while typing docker run is simple enough, the true implementation of a container in production is running it as a service. Services codify a container’s behavior in a Compose file, and this file can be used to scale, limit, and redeploy our app. Changes to the service can be applied in place, as it runs, using the same command that launched the service: docker stack deploy. - docker-compose.yml File - version: "3" services: web: \# replace username/repo:tag with your name and image details image: username/repo:tag deploy: replicas: 5 resources: limits: cpus: "0.1" memory: 50M restart\_policy: condition: on-failure ports: - "4000:80" networks: - webnet networks: webnet: - docker stack deploy -c docker-compose.yml getstartedlab - docker stack ls - docker stack ps - docker service ls - docker service ps getstartedlab\_web - docker container ls -q - docker inspect - docker stack rm - TLS Certificates - Commands - Shell Completion - Install shell completion Docker for Mac comes with scripts to enable completion for the docker,docker-machine, and docker-compose commands. The completion scripts may be found inside Docker.app, in the Contents/Resources/etc/ directory and can be installed both in Bash and Zsh. Bash Bash has built-in support for completion To activate completion for Docker commands, these files need to be copied or symlinked to your bash\_completion.d/ directory. For example, if you installed bash via Homebrew: etc=/Applications/Docker.app/Contents/Resources/etc ln -s $etc/docker.bash-completion $(brew --prefix)/etc/bash\_completion.d/docker ln -s $etc/docker-machine.bash-completion $(brew --prefix)/etc/bash\_completion.d/docker-machine ln -s $etc/docker-compose.bash-completion $(brew --prefix)/etc/bash\_completion.d/docker-compose Zsh In Zsh, the completion system takes care of things. To activate completion for Docker commands, these files need to be copied or symlinked to your Zsh site-functions/directory. For example, if you installed Zsh via Homebrew: etc=/Applications/Docker.app/Contents/Resources/etc ln -s $etc/docker.zsh-completion /usr/local/share/zsh/site-functions/\_docker ln -s $etc/docker-machine.zsh-completion /usr/local/share/zsh/site-functions/\_docker-machine ln -s $etc/docker-compose.zsh-completion /usr/local/share/zsh/site-functions/\_docker-compose - docker --version - docker info - docker login - docker tag image username/repository:tag - docker tag friendlyhello gordon/get-started:part2 - Examples - docker run hello-world - docker run -d -p 80:80 --name webserver nginx - From part 2 - docker build -t friendlyhello . \# Create image using this directory's Dockerfile docker run -p 4000:80 friendlyhello \# Run "friendlyname" mapping port 4000 to 80 docker run -d -p 4000:80 friendlyhello \# Same thing, but in detached mode docker container ls \# List all running containers docker container ls -a \# List all containers, even those not running docker container stop \# Gracefully stop the specified container docker container kill \# Force shutdown of the specified container docker container rm \# Remove specified container from this machine docker container rm $(docker container ls -a -q) \# Remove all containers docker image ls -a \# List all images on this machine docker image rm \# Remove specified image from this machine docker image rm $(docker image ls -a -q) \# Remove all images from this machine docker login \# Log in this CLI session using your Docker credentials docker tag username/repository:tag \# Tag for upload to registry docker push username/repository:tag \# Upload tagged image to registry docker run username/repository:tag \# Run image from a registry - Networking - programs in containers are isolated from the Internet by default - You can group containers in private virtual networks via bridges - Done via exposing ports and linking ports - docker run -p 5000:5001 - internal ip/external ip - Expose ports dynamically - specify only inside port and the outside port is chosed dynamically - -p ipaddress - Storage - Volumes - Bind Mounts - Mount a directory from the host OS in a container - docker run -d -it --mount source=devVolume,target=/devVolume ubuntu - Sensitive Data - Secrets - https://docs.docker.com/v17.12/engine/swarm/secrets/ - Config files - Shared - Ephemeral - Uses Copy on Write - File System Sharing on Mac - https://docs.docker.com/docker-for-mac/osxfs/ - Developer Environment with Docker - E.g., what I setup at Lending Standard - Remember, VS Code can attach to running containers via its remote shell plugin - Orchestration for Docker - Runs containers and restarts them if they die - Service Discovery - allows them to find each other - Resource Allocation - match containers to computers, storage, etc. - Docker Swarm - Kubernetes in AWS - AWS EC2 - Kubernetes in Google Cloud - Kubernetes in Azure - Testing - https://docs.docker.com/docker-hub/builds/ - General Tips - Don't fetch dependencies on container start. Store them in your container in case a library gets removed - Don't rely on golden images where you make a change to it you can't reproduce from the dockerfile. - Sources - Official Get Started Tutorial o - LinkedIn Learning/Lynda - Learning Docker o - [Dockerize your React app - DEV Community](https://dev.to/karanpratapsingh/dockerize-your-react-app-4j2e) - Used this - Add alias ll='ls -lah' to Docker alpine image - Dockerfile - RUN echo 'alias ll="'ls -lah'"' >> /etc/profile - Normal way to shell in doesn't load the profile file so you need to append -l to the shell command: - docker container exec -it meep-backend\_api\_1 /bin/ash -l ## Sources - [How to fix M1 Mac Puppeteer chromium arm64 bug](https://linguinecode.com/post/how-to-fix-m1-mac-puppeteer-chromium-arm64-bug) - [Top 8 Docker Best Practices for using Docker in Production - YouTube](https://www.youtube.com/watch?v=8vXoMqWgbQQ) - [The chromium binary is not available for arm64 · Issue #7740 · puppeteer/puppeteer · GitHub](https://github.com/puppeteer/puppeteer/issues/7740) - [Installation fails on Apple Silicon / M1 · Issue #6622 · puppeteer/puppeteer · GitHub](https://github.com/puppeteer/puppeteer/issues/6622) - [Running non-native platform images with Docker, what could go wrong?](https://world.hey.com/cpuguy/running-non-native-platform-images-with-docker-what-could-go-wrong-ca847f30) - [Useful Docker Aliases · GitHub](https://gist.github.com/jgrodziski/9ed4a17709baad10dbcd4530b60dfcbb) - [Useful Docker Aliases · GitHub](https://gist.github.com/satnami/9f9031cded7274e0dcbd13ab702acf89) - [Useful Docker Commands and Aliases | by Mayank Patel | FAUN — Developer Community 🐾](https://faun.pub/useful-docker-commands-and-aliases-9ea79191832f) - [Developing inside a Container using Visual Studio Code Remote Development](https://code.visualstudio.com/docs/devcontainers/containers) - [Container environment variables](https://code.visualstudio.com/remote/advancedcontainers/environment-variables) - [Dev Containers: Getting Started - Engineering Fundamentals Playbook](https://microsoft.github.io/code-with-engineering-playbook/developer-experience/devcontainers/)