When building a Docker image, you may want to make sure to keep it light. You can save any local image as a tar archive and then inspect its contents. Each Dockerfile instruction creates a layer at build time: To see the intermediate layers of an image, type the following command: You can see belowthat eachlayer have a size and a command associated to create it. But still, the large base image means reduced productivity for developers. In typical software development, each service will have multiple versions/releases, and each version requires more dependencies, commands, and configs. The best way to see that is to audit your layers with the techniques mentioned above. We also used to write bash/shell scripts and apply hacks to remove the unnecessary artifacts. LogRocket is a frontend application monitoring solution that lets you replay problems as if they happened in your own browser. Also, If you follow all the standard container best practices, you can reduce the docker image size to have lightweight image deployments. Each layer in your image might have a leaner version that is sufficient for your needs. At the end of the run, the statement performs apt-get remove curl or wget, once you no longer need them. Note: Using this method, you will need to rebuild the entire image each time you add a new package to install. DevOps Engineer at RazorPay | GSoC18 student at Drupal | #GitHubCampusExpert | GCI mentor at Drupal and JBoss | GSoC19 mentor at Drupal and Jenkins | Speaker and writer, to optimize your application's performance, How to use subjects to multicast observers in RxJS, Interesting use cases for JavaScript bitwise operators, What you need to know about the React useEvent, Creating a multi-step form in Flutter using the Stepper, Installing and keeping unnecessary dependencies in your image increases complexity and chances of vulnerability in your application, It will take a lot of time to download and spawn the containers, It will also take a lot of time to create and push the image to the registry and ends up blocking our CI/CD pipelines, Sometimes, we end up leaving keys and secrets in the Dockerfile due to build context, To make the container immutable (yeah you read that right) we cant even edit a file in the final container. The multi-stage build divides Dockerfile into multiple stages to pass the required artifact from one stage to another and eventually deliver the final artifact in the last stage. Here at nClouds, DevOps is a major focus, and we build CI/CD pipelines for many customers. This is all a result of how images are structured as a series of read-only layers. Also, by installing unwanted libraries, we increase the chance of a potential security risk by increasing the attack surface. EdTech Startup Case Study Migrating Their Application To Kubernetes On Cloud. Free Self-Service Migration Readiness Assessment by nClouds - Learn more. I hope this article helped you understand Docker and its multi-stage builds feature. florianlopes I love Open Source, Java, Spring and Docker. Please comment and let us know other how-tos youd like us to address in future blog posts. So the new reduced image size is 171MBs as compared to the image with all dependencies. If not, you should consider starting with a smaller base image like Alpine (https://hub.docker.com/_/alpine/) which will likely become the base image for all official Docker images (Jenkins, Maven). s/^#\s*\(deb. 140d9fb3c81c 9 days ago /bin/sh -c #(nop) ADD file:ed7184ebed5263e677 187.8 MB, https://www.brianchristner.io/docker-image-base-os-size-comparison/, -- There are no messages in this forum --, Do not install packages recommendations (, Remove no longer needed packages or files, in the. If you are too busy to focus on reducing your image size, here is a tool you could consider:https://github.com/jwilder/docker-squash. florian.lopes You can further reduce the base image size using distroless images. Docker is one of the most important technologies in enterprises nowadays. The naive way on the left results in a 961 MB image. Large container images have increased potential for security holes. Because of this strategy, the mysql-client package is still part of the final image (in the third layer actually) although being removed further. 74a2c71e6050 9 days ago /bin/sh -c set -xe && echo ', > /u 194.5 kB A container packages the application services or functions with all of the libraries, configuration files, dependencies, and other necessary parts to operate. Your email address will not be published. This introduces a challenge in Docker image build, as now the same code requires more time & resources to be built before it can be shipped as a container. Shishir is a passionate DevOps engineer with a zeal to master the field. After that, only the necessary app files required to run the application are copied over to another image with only the required libraries, i.e., lighter to run the application. Since it is not needed anymore, it can be removed (in the SAME instruction). Now we have the power of multi-stage builds with which we can reduce the size of the image without compromising the readability of the Dockerfile. Feel free to comment and ask me anything. Now, lets use this method to create a multistage build. Again, the difference is not very significanthere because we only remove one package. 5 tips to reduce Docker image size Docker images can quickly weight 1 or more GB. Despite removing the file in the last layer, the image still contains the file in other layers which contributes to the overall size of the image. Again, the difference is not very significanthere because we only remove one package. You can follow me on Twitter and Medium. Although this can be somehow difficult to understand, this structureis very important as it allows Docker to cache layers to make the builds much faster. Discover posts about Java, Spring, and Docker. Here are some basic steps recommended to follow, which will help create smaller and more efficient Docker images. Previously, when we didnt have the multi-stage builds feature, it was very difficult to minimize the image size. In addition to cost, often customers run into performance and timeout-related issues when the image size is large. Docker images are created by writing Dockerfiles lists of instructions automatically executed for creating a specific Docker image. Image sizes vary between them: Its not just a matter of image size though, each of these images comes with its own philosophy or tools you might prefer. Selectively copying artifacts from one stage to another. Why do we need to reduce the size of the Docker image in this modern era of tech, where memory and storage are relatively cheap? Having a large image can make every step of the process slow and tedious. Do not perform multiple installs in multiple RUN instructions. The following are the methods by which we can achieve docker image optimization. Lets see another interesting examplein whichwe remove a temporary package in a separate instruction: You can see here that the curl package is immediately removed after being installed, in a separateinstruction. Whats driving the excitement is the ability of containers to make application deployments more manageable, versionable, and faster. There are many other things that are responsible for increasing the size of the image, like the context, base image, unnecessary dependencies, packages, and a number of instructions. Do you need every Ubuntu (or other base images) packages? To solve this issue Docker introduced multi-stage builds starting from Docker Engine v17.05. But first, lets understand a bit about Docker. The multi-stage build is the dividing of Dockerfile into multiple stages to pass the required artifact from one stage to another and eventually deliver the final artifact in the last stage. Execute the following command to enable the feature. Docker images can quickly weight 1 or more GB. For example, running apt-get update will update internal files that you dont need to persist because youve already installed all the packages you need. Any time you run a new command in Docker, it creates a new layer. There are a lot of ways to do this, including: Multi-stage builds in Docker are a new feature introduced in Docker 17.05. Ohh wait..wait..wait..!!! It is also necessary because: Reducing Docker Images is something we should know how to do to keep our application secure and stick with the proper industry standards and guidelines. This is especially common for compiled languages like Go. Including navigating a treemap of your image: With these methods, you're set up to assess improvements to your image size. Also, in a single Dockerfile, you can have multiple stages with different base images. In addition to logging Redux actions and state, LogRocket records console logs, JavaScript errors, stacktraces, network requests/responses with headers + bodies, browser metadata, and custom logs. I have seen cases where the initial application image started with 350MB, and over time it grew to more than 1.5 GB. So it all boils down to how efficiently we can manage these resources inside the container image. The process consists of: We can transform the last Dockerfile snippet into the next one: The main difference is that, with Docker Multi-Stage, we build two different images in the same Dockerfile. (For yum, use, to download some package, remember to combine them all in one, statement. Our contains.dev offers many tools to analyze layers, their contents, and their size. The less specific or specialized your parent image is, the more control you have over the image size: A closer look at node:16-bullseye shows that it has buildpack-deps as its parent image, which comes with lots of dependencies you might not need. This post will give you 5tips to help reduce your Docker images size and why focusing on it is important. Here is the same Dockerfile with a multistage build step that uses the google nodeJS distroless image instead of alpine. You can see a great comparison of Docker base images here:https://www.brianchristner.io/docker-image-base-os-size-comparison/. To illustrate this statement, lets build an image with two separate RUN instructions which install curland mysql-clientpackages: Now, lets gather the two instructionsin only one: Although the size difference is not so significant, you can expect better results when installing multiple packages. that arent needed anymore. To report unethical behavior confidentially, email hr@nclouds.com, Copyright 2012-2022 nClouds, Inc. All rights reserved | Privacy Policy. The post 5 tips to reduce Docker image size appeared first on Florian Lopes's blog. Lets combine the RUN commands into a single layer & save it as Dockerfile4. Realize that you probably dont need many of those libraries that you are installing. Getting familiar with how caching works can save you time and avoid surprises when working with Docker locally, or building images on a CI/CD service. This way, our final image wont have any unnecessary content except our required artifact. This feature should be kept in mind while optimizing the docker image. Try to install all the packages on a single RUN command to reduce the number of steps in the build process and reduce the size of the image. DONT INSTALL DEBUG TOOLS LIKE curl/vim/nano. When building an image, the Docker daemon will check if the intermediate image (layer created by the instruction) already exists in its cache to reuse it. If the intermediate layer is not found or has changed, the Docker daemon will pull or rebuild it. Clean-up artifacts no longer needed before moving on to the next layer. This blog talks about different optimizations techniques that you can quickly implement to make the smallest and minimal docker image. Most tech companies are using Docker to improve the deployment strategy of its products and services, making them robust and scalable. As we just saw, the layers play an important role in the final image size. Even though this pattern helps reduce the image size, it puts little overhead when it comes to building pipelines. Lets see this in action, with the help of a practical example: lets create a ubuntu image with updated & upgraded libraries, along with some necessary packages installed such as vim, net-tools, dnsutils. After the build is complete the execution time comes to be 117.1 seconds. Thislayers stackallows Docker to reuse images when a similarinstruction is found. Container-based infrastructures are among the hottest cloud computing solutions today. This way, our final image wont have any unnecessary content except the required artifact. However, if we would have used the same base image we used in the build stage, we wouldnt see much difference. For example, if you have a Node.js app you can use the node image, or python if youre using Python, etc. In the above RUN command, we have used --no-install-recommends flag to disable recommended packages. Docker images are an essential component for building Docker containers. This is just the one instruction of the Dockerfile in which we need to download the abc.tar.gz file from some http://xyz.com website and extract the content and run make install. @LopesFlorian Do you need to maintain two Dockerfiles? Often you need to weigh the convenience of having a base image with all dependencies pre-installed against the size of the resulting image. Thanks for reading! He is a certified AWS & Kubernetes engineer. To easily inspect a DockerHub image, you can use the MicroBadger service:https://microbadger.com/. It isnt always obvious that this is happening, for example: Were just chmod'ing an existing file, but Docker cant change the file in its original layer, so that results in a new layer where the file is copied in its entirety with the new permissions. The above methods should help you build optimized Docker images and write better Dockerfiles. This post will give you 5tips to help reduce your Docker images size and why focusing on it is important. Lets see what the new Dockerfile might look like. Even though the Docker build process is easy, many organizations make the mistake of buildingbloated Docker imageswithout optimizing the container images. WHY IS IT IMPORTANT FOR ORGANIZATIONS TO HAVE A HYBRID, MULTI CLOUD STRATEGY? Usually, the first choice you need to make is which distribution you want. Now we can imagine the complexity of the Dockerfile for n number of instructions. After the build is complete. Docker and the Docker logo are trademarks or registered trademarks of Docker, Inc. 2022 Contains.dev by Nutmeg Studios LLC, # File moved but layer now contains an entire copy of the file, automatically for their official Debian and Ubuntu images. In this example,the package curl is only needed to retrieve an install file. Lets check its size using. Is each instruction in the Dockerfile adding a layer to the image? After the build is complete the execution time comes to be 91.7 seconds. Docker images are a core component in our development and production lifecycles. Although some of them cannot be reduced (especially the one you start from), we can use a few tips to help reduce the finalimage size. It isn't easy to see exactly which files are added and where. base image, the size of the image will get reduced to 5MB from 128MB. In this article, we looked at what Docker is, why we need to reduce the size of images, and how can we do this using multi-stage builds effectively. Docker images work in the following way each RUN, COPY, FROM Dockerfile instructions add a new layer & each layer adds to the build execution time & increases the storage requirements of the image. Docker is like a version control system. Read more about the multi-stage build from Docker official docs. (For yum, use yum clean all). Each container shares the services of one underlying operating system. Lets build it and see the storage & build time. Lets look at different established methods of optimizing Docker images. Docker Images are the set of instructions written in a file called Dockerfile. If you are to install wget or curl to download some package, remember to combine them all in one RUN statement. Thats why its so important to have the most optimal Docker images. This allows us to perform all preparation steps as before, but then copy only the essential files or output from these steps. Ubuntu has long-term enterprise support, comes bundled with many utilities and supports a vast amount of packages, and so on. Since it is not needed anymore, it can be removed (in the SAME instruction). Lets see the final imagesize: This time, lets combinethese instructions into one line: You can see that the size of the image has slighltly reduced. line you add in your Dockerfile, you will be increasing the image size based on the size of the library is added. Smaller containers usually have a smaller attack surface as compared to containers that use large base images. If you are learning container orchestration using Kubernetes, checkout comprehensive Kubernetes tutorials for beginners. In some organizations, the security team itself publishes base images every month after testing & security scanning. In this article, we will review several techniques to reduce Docker image size without sacrificing developers and ops convenience. Bottom line, try to reduce Docker image size today. Here is a pictorial representation of a Docker multistage build. 2. Smaller the image size better the resource utilization and faster the operations. If you are getting started with your Docker journey, you can check out my article on 3 methods to run docker in docker. There are several important considerations that go into picking a base image. Identify the ones you really need and install only those. Next, you can decide if you want your parent image to come bundled with additional dependencies. Therefore, DevOps engineers must optimize the docker images to ensure that the docker image is not getting bloated after application builds or future releases. By reducing the Docker image size, we keep only the required artifacts in the final image and remove all the unnecessary data. The Dockerfile might include several steps that take care of setting up an environment for compiling the program that will run at runtime. You can clone it and follow along the tutorial. Well show you the old way and a better way. For example, lets take a look at the following two Dockerfiles. As we saw earlier, the Docker daemon creates an image for each instruction to execute the associatedcommand. Docker makes it especially easy to add files you didnt mean to add to an image. This website uses cookies and third party services. Not just for production environments, at every stage in the CI/CD process, you should optimize your docker images. How can you fix this issue? Docker recognize this is an issue and went as far as adding apt-get clean automatically for their official Debian and Ubuntu images. It is recommended whenever you use install in your Dockerfiles. This base image weights around 5MB whereas Ubuntu one is about 188MB. The final size of this image is 513MB. Dockeras a container engine makes it easy to take a piece of code & run it inside a container. document.getElementById( "ak_js_1" ).setAttribute( "value", ( new Date() ).getTime() ); In this tutorial, you will learn how to set up a four node docker swarm cluster. Either you use the examples given in the article or try the optimization techniques on existing Dockerfiles. Although the gigabyte price is decreasing, keeping your Docker images light will bring some benefits. By default, it comes with the sh shell that helps debug the container by attaching it. which means that we reduced it by 45% by including only the important artifacts of our first container, leaving behind all the dependencies that we used to build it. Required fields are marked *. 2. Lets see the storage space that it requires by building it. Are you currently using the builder pattern? A simple Dockerfile for this application would like this Save it as Dockerfile1. As a rule, only the necessary files need to be copied over the docker image. Let's compare multiple and single instructions by installing/removing packages : To illustrate this statement, lets build an image with two separate RUN instructions which install curl and mysql-clientpackages: Now, lets gather the two instructionsin only one: Although the size difference is not so significant, you can expect better results when installing multiple packages. 4. It is recommended that install these tools only in the development Dockerfile and remove it once the development is completed and is ready for deployment to staging or production environments. 5. This file has a similar syntax to .gitignore : Then when you run COPY . Illustration of pulling a large Docker image layer. These instructions act as a multi-layered filesystem in Docker. Are you currently faced with the challenge of Docker images sized from 600MB to 1G? Because of this strategy, the mysql-clientpackage is still part of the final image (in the third layer actually) although being removed further. Using multiple FROM statements in your Dockerfile. So the first step is to be able to quickly inspect which files are added to each layer. If so, you are faced with some common issues: When developers are developing locally, they are normally fetching the Docker image. I'm a Java developer living in Bordeaux, Published on Docker is a tool to easily create, deploy, and run applications by using containers that are independent of the OS. The container itself looks pretty good, but the size of that image (934MB) is large. It is not always useful as we lost the previous stage intermediate containers so we wont be able to leverage build cache in Docker. The multistage build pattern is evolved from the concept of builder pattern where we use different Dockerfiles for building and packaging the application code. Group commands in ONE instruction when possible (do not perform multiple installs in multiple RUN instructions). USE no-install-recommends ON apt-getinstall. A Dockerfile to achieve this would be as follows Save this as Dockerfile3. Reducing the size of the image can have benefits both for developers and your users. But before we talk about how to reduce the image size, I think its worth understanding why the Docker image size grows. Thefinal image built from this Dockerfile contains 3 layers plus all Ubuntu image layers. For each apt-get installer or yum install line you add in your Dockerfile, you will be increasing the image size based on the size of the library is added. Otherwise, this may cause each step in the build process to increase the size of the image. Multi-stage builds introduce a lot of flexibility with support for advanced cases like multiple FROM statements, copying a single file from an external image, and more. LowTouch Cloud ME DMCC, Unit No: 826, DMCC Business Centre Level No 1, Dubai, United Arab Emirates. As we just saw, the layers play an important role in the final image size. Note:You cannot directly use the publicly available base images in project environments. Each ADD or COPY and even the RUN commands in your Dockerfile can include files you werent expecting. The Docker daemon has an in-built capability to display the total execution time that a Dockerfile is taking. And, you cant do rm -rf $source_files_directory because it will simply create a new layer. MISTAKES TO AVOID IN DOCKER IMAGES WITH REASON AND SOLUTION, DevSecOps, The Key To Successful Digital Transformations, CloudControl Solutions AppZ for Automobile Company. In this article, we will look at one of the most promising features of writing Dockerfiles efficiently to reduce the final image size. Do not install packages recommendations (-no-install-recommends) when installing packages, Remove no longer needed packages or files, in the SAME instruction if possible. Within those images, usually you can specify the distribution youd like using the appropriate tag, for example, node:alpine for Alpine Linux and python:3-bullseye for Debian Bullseye. How to Setup and Configure Docker Swarm Cluster, Cloud Based Docker Container Monitoring Using Datadog, How To Install and configure Docker on Ubuntu, How to Setup AWS ECS Cluster as Build Slave for Jenkins, Getting Started With Docker : Working With Containers, How to Provision Docker Hosts on Azure using Docker Machine. It also instruments the DOM to record the HTML and CSS on the page, recreating pixel-perfect videos of even the most complex single-page web and mobile apps. We can make n number of containers from a single Docker image similar to OOPs concept of creating n objects instances (which share common characteristics and behavior) from a single Class. When a Docker user runs the images, it produces one or multiple containers. Then we need to install `openjdk` and `grande` to compile it and build the `ROOT.war`. The whole concept of run anywhere images starts from a simple configuration file called Dockerfile. SR Cloud Dev-Ops Engineer | Cloud Control, Senior Cloud DevOps Engineer with more than five years of experience in supporting, automating, and optimizing deployments to hybrid cloud platforms using DevOps processes, CI/CD, containers and Kuberneties in both Production and Development environments, India Office: ACE-12, 4th Floor, C-DAC Building, Technopark, Trivandrum, Kerala, India. Reducing Docker final image size will eventually lead to: To reduce animage size, its important to understand what a layer is. Docker helps in such cases by storing the cache of each layer of a build, hoping that it might be useful in the future. Realize that you probably dont need many of those libraries that you are installing. Depending on the environment you are using to deploy the Docker containers, many times you have constraints on disk size, so its important to reduce the image size. One slimmed-down image to ship the results of the first build without the penalty of the build-chain and tooling in the first image.