Increase IT project ROI using containers
In the last few years containers have changed the way we approach development, deployment and maintenance of applications. While they are not a new concept (containers have existed in Linux for a while), they did not really reach fame until the arrival of the Docker company. Docker made using containers simple and available to everyone. Beyond their flexibility and ease of use, they make economic sense since they :
Reduce development environment setup time
Increased developer productivity
Simplify software deployment
Improve app maintainability
Enable new ways to develop applications
To understand how containers provide these benefits, let's first talk about images and containers
Understanding image and containers
We will use the simple analogy of creating a book to put these two terms in context. The process of writing a book is very similar to the process of creating images and containers:
An image is a self contained, isolated compilation of everything needed to run an app, including the app itself. Creating an image is very much like creating the final version of a book. You gather all the code written by developers, gather all the dependencies (think of dependencies as font, formatting, artwork, etc...) and compile them into a self contained image.
Containers are in essence copies from an image that are deployed and run the app.
Finally, just like a user making notes in a book, containers can be modified as needed. This allows developers to further enhance the code in the container and eventually apply the changes to the image. This feedback loop leads to a completed application living inside a final image which is pushed to the production servers to allow the general public to access our application.
The original problem that containers solved
In the olden days of application development (which in this fast changing digital world means only a few years ago), each developer would have to waste hours and even days installing all the project tools and libraries in their machines. As more machines needed to be setup, more time (and money) was wasted away in unproductive setup time.
Eventually virtual machines (VMs) came around, and allowed us to create perfect copies of a machine. The copies would include everything needed, from the operating system to the wall paper. This was a significant improvement as only one person would need to create the VM that would be distributed to the team. All was great in the world...for a bit. Soon enough, problems starting appearing. Since VMs are full copies of a machine, the following issues became evident:
Hard drive space: VMs are so large that businesses had to start buying larger, faster and more expensive hard drives to accommodate them. Even then, machines could not fit more than a couple of VMs. Thus, supporting multiple environments or multiple projects became a real issue
Start up time: Every time a VM needs to start, it is like starting a whole new machine and we all have gone to get coffee and chat with our colleagues while the OS starts up.
Resource intensive: Each VM, even if idle, eats a minimum, rather large, set of resources. Most people that have had to juggle multiple VMs will tell you of the time they started a VM by mistake only to find out their machine frozen due to being out of resources
The solution to these issues came out in the form of container technology that the Docker
company made widely popular. Docker made it easy to create images and containers, made them available in most operating systems. More importantly, Docker made images easy to share.
Unlike VMs, containers only incorporate the basic libraries necessary to run apps. They are significantly smaller, they use few resources and they start up in seconds. For example, in terms of space usage, a typical MySQL base image is about 400 MB while a VM running just Windows 10 requires a minimum of 32GB
Container Benefits
Environment Setup
Pre built images: Many software companies have made their solutions available as images from public repositories like docker hub. These images are ready for use and can be used as springboards to create new images. From Python to Tensorflow, these images make it easy and straight forward to get started with a technology.
Machine setup: One developer can create the image with all the required software and libraries. The image is then easily sharable through a private repository docker hub
Environment consistency: Since images are self contained apps, using the wrong version of a library is no longer an issue. Additionally, the image is agnostic of any software installed in the host machine.
On boarding new members: Since the project image is always up to date, providing new members the latest image ensures their environment is consistent with the rest of the team.
Application development
Multi-container environments:
It's easy to start up multiple containers at once. Typical example would be to bring up containers for web application, app server and database with just one command.
Containers can easily share a common network where they can address each other by name. No need to worry about IP addresses or network protocols.
Developer can have all pieces of an application running in their machine, reducing the need to mock up data or mimic calls to a server
Multiple application versions: It is usual that developers will need to manage multiple versions of an app and related libraries in their machines. This leads to applications not working properly due to the wrong dependencies or the wrong library. Containers solve this issue since they are self contained environments.
Testing new software: We can experiment with new/trial software within a container and just delete the container once we are done. No need to install the software directly in machine or affect the machine performance by installing too much gunk in it.
Micro services: While containers are not technically necessary for micro services, they make it easier to develop, deploy and maintain them. With the help of orchestration tools like Kubernetes, managing multiple containerized apps is just a few commands away.
Serverless: Companies like Amazon and Google pioneered (and marketed) serverless functions to simplify app development. In the backend, these functions run in containers launched on demand
Application deployment
Software deployment: Containers are self-contained entities. Pushing the same container from test to production guarantees that the code tested is the same as the code in production. Additionally, pushing code to an environment is as easy as pushing a new container.
Application rollback: If a deployment goes wrong, rolling back to the previous app version means simply restoring the previous container. No more messy clean ups or leaving behind the wrong library.
Automation and orchestration: Containers are front and center when it comes to Infrastructure as software (IaS). They enable developers to codify the software and dependencies required by an app. When combined with infrastructure tools as Kubernetes, they allow us to codify both software and hardware needs. This code can then be versioned and stored just like any other software.
Container Challenges
Using an image: Each image is a bit different depending on how it is built. Thus, they may have some idiosyncrasies on how they are expected to run. Also, caution should be used when using an image from an unknown source
Management: Containers need to be managed properly like any other code, specially if there are multiple versions of the image deployed or if you are running Microservices
Change in development flow: Developer workflow needs to be adjusted to the new paradigm and to use new tools more adept to at debugging container based apps.
Containers overhead: Containers run inside Docker so there is an associated overhead.
Graphical applications: By design, containers do not include the libraries associated with displaying a GUI. While there are work arounds to this, they are clunky at best. Thus, container based applications expose the GUI through a web browser or a thin client that is installed in the host machine.
Bottom line
Containers change and improve the software development process the same way the cellphone changed how we approach communications, but they are not a silver bullet. Just as we currently live tethered to our cellphone charging cables, containers do come with some constraints. However, the benefits gained in terms agility and portability by far outweigh the costs. Furthermore, containers enable us to move the industry forward and implement/simplify new strategies like micro services and serverless functions. While it may be over kill to use containers for a quick proof of concept (unless you already have an environment setup), they are the best path forward when building enterprise and big data applications.