Understanding Linux Containers

Being pragmatic, the purpose of using machines is just to run applications, and have an operating system is just the way to run them. We really want machines to run applications and not to run operating systems. OS are only a side effect of our purpose.

Virtual Environments, THE TWO WAYS.

Sometimes, you cannot run some applications in the same user space.
These are just some of the most common causes:

  • Troubles about co-existing multiple incompatible environments, like php 5.3 and php 5.6 because them need different and incompatible libraries, etc.
  • You need to confine the scope of users, like web hosting providers which offer virtual private servers; They need to limit the cpu utilization per user, and the available memory, and the networking usage, and the filesystem limit (space, or I/O, etc), to guarantee a liable service and guard all users from each other (including their processes, memory, files, etc).
  • You need to reduce the provisioning and maintenance complexity of applications and environments, and provide high availability of them and have a simple solution for disaster recovery.

Virtual Machines

A virtual machine (VM) is a software implementation of a machine (for example, a computer) that executes programs like a physical machine — Wikipedia

Virtual Machine

This virtualization solution allow us to emulate multiples machines in a single physical machine, but each of them needs to run a dedicated operating system with its own kernel, etc, so the performance of the running applications decreases.

I don’t hate Virtual Machines nor run dedicated guest Operating Systems in my host Operating System.
I love them so much when I need to run Windows or Linux applications in my Mac Os X.

Containers

Containers are the correct way to virtualize environments when you don’t need the overhead of having another dedicated OS running on your host OS.

Linux ContainersContainers are like cages (aka namespaces), which allow us to limit and isolate things like processes, cpu/memory/hard-disk/network usage, etc.

And all of this without having another dedicated operating system per each virtual environment.

Understanding Linux Containers

TL;DR: Use docker.

Otherwise, to understand how Linux Containers really work, we are going to create our own Container step by step with low-level APIs.

The following examples were tested on an Ubuntu 14.10 Utopic Unicorn.
If you want, you can use the following Vagrantfile to play with containers inside a virtual machine:

Vagrant.configure("2") do |config|
    config.vm.box = "utopic64"
    config.vm.box_url = "https://cloud-images.ubuntu.com/vagrant/utopic/current/utopic-server-cloudimg-amd64-vagrant-disk1.box"
end

It’s simple, only download and install VirtualBox and Vagrant and then run: 

vagrant up
 to create and start the virtual machine, and finally run: 
vagrant ssh
 to enter using ssh into the started virtual machine.

Creating the virtual file system

To begin, we need to assign the root directory of our virtual environment.
In fact we want to confine our container to this file system directory, and disallow its applications know any file outside its designated directory tree.

C’mon! what are you waiting for? Create the directory where you want:

mkdir frankenstein

This path will be the root path (/) of our Container, so we need some tools inside it, at least a shell binary. So, you can download a pack with essential binary linux tools (like ls, cat, echo, cd, ps, vi, etc):

wget http://www.busybox.net/downloads/binaries/latest/busybox-x86_64
chmod +x busybox-x86_64
mkdir -p {frankenstein,frankenstein/usr}/{bin,sbin,etc}
for x in $(./busybox-x86_64 --list-full); do ln -s /bin/sh frankenstein/$x; done
unlink frankenstein/bin/sh
cp busybox-x86_64 frankenstein/bin/sh
touch frankenstein/etc/os-release

(Note: other busybox architectures are available in: http://www.busybox.net/downloads/binaries/latest/)

Now our virtual environment would have all of the essential linux binary tools. It’s the moment to confine a shell in the created directory (aka create an init process):

chroot frankenstein /bin/sh

Ok, seems nice, but we don’t have isolated processes nor network, etc.
So, exit from the shell and let me show you how to do it better.

In the following examples we will use systemd.
Systemd is a system and service manager for Linux, which allow us among other things to isolate processes, network, etc.
In Ubuntu 14.10 Utopic Unicorn, systemd is installed by default, but it’s not enabled.
To enable it, edit your /etc/default/grub file and change the line:

GRUB_CMDLINE_LINUX=""

to:

GRUB_CMDLINE_LINUX="init=/bin/systemd"

then run the grub updater and restart your system:

update-grub && reboot

Note: if you are using an older version of ubuntu follow these steps or google about how to install and enable systemd in any linux distribution)

Now spawn the container!

systemd-nspawn --private-network -D frankenstein /bin/sh

You will get a shell with the previous chroot features but isolating processes and network.

Limiting the resources usage (CPU, RAM, disk I/O, etc)

cgroups is a Linux kernel feature which allow us to limit hardware resources usage of containers.

With cgroups you can for example limit which cpus are your container allowed to use, and the usage cuota, for example:

cpu usage per process

Or you can limit which physical devices can your container use, or limit the available memory, or the network usage, etc. (See https://www.kernel.org/doc/Documentation/cgroups/ to view all available features)

In the following example we are going to limit the memory usage of our container:

First, create a folder in /sys/fs/cgroup/memory/ with the name of your container, and add the PID of your container’s init process inside the tasks file:

echo 1234 > /sys/fs/cgroup/memory/frankenstein/tasks

and then limit the memory usage (1 byte in the following example will crash your container):
echo 1 > /sys/fs/cgroup/memory/frankenstein/limit_in_bytes

Conclusion

dockerDon’t use systemd-nspawn directly, seriously.
Use a REAL high-level solution, like LXC, which is a great userspace interface for the Linux kernel containment features.
Through a powerful API and simple tools, it lets Linux users easily create and manage system or application containers.

And of course, docker, which implements a high-level API to use Linux Containers and provides other really great features.

And one more thing…

Are Containers supported in Mac OS X?

Nope, OS X doesn’t support operating system-level virtualization like the last mentioned Linux Containers, so you cannot run Containers in OS X, even though there are some projects like boot2docker to enjoy Docker on your OS X, them both really create a Linux virtual machine behind the containers.

Understanding Linux Containers

Dealing business transactions with Unit of Work and Repositories

The Data Access Layer (as part of the Infrastructure Layer) provides the ability to persist data (which may reside in a database, or in an external web service, etc) and access them.

A repository represents all objects of a certain type as a conceptual set. It acts like a collection. — Eric Evans

We use Repositories to centralize and segregate the data access functionality from the Domain Layer, and we should create one Repository for each Domain Entity (also known as Aggregate).

A Repository is not a Data Access Object, the latter is an abstraction of data persistence (but can be implicitly used in a Repository), because directly performs the persistence operations into the data source, while a Repository is an abstraction of a collection of objects, which keeps the operations which want to be performed, but won’t be done until the Application Layer takes charge of it, then all at once will be performed at the same time. This Pattern is named Unit of Work (which also can help to prevent race conditions).

Very simple end-to-end overview from the Application Layer:

# Instantiate a Unit of Work with the required database connectors
# and a strategy mapping if it is needed
unit_of_work = UnitOfWork(mysql_driver)

# Instantiate some Repositories
user_repository = UnitOfWorkUserRepository(unit_of_work)
order_repository = UnitOfWorkOrderRepository(unit_of_work)

# Instantiate some Domain Services passing the required repositories
user_registration = UserRegistration(user_repository)
order_handler = OrderHandler(order_repository, user_repository)

# Perform some domain actions
user_registration.register_new_user(username, password)
order_handler.premium_feature(username)

# Persist Aggregates' tracked changes in the database
unit_of_work.commit()

As we have seen, our Repositories are attached to a particular implementation of a technology (like a Unit of Work or an ORM), and to define how to use them they should follow an interface or contract which should be placed in the Domain Layer.
With this abstraction you will hide the business transaction technology so you can focus on the domain model.
This concept is also known as Persistence Ignorance, which means the domain model completely ignores how data is persisted or queried from any data source.

Dealing business transactions with Unit of Work and Repositories

Handling annotations in PHP

Annotations are code in comments, and a lot of projects (i.e: Doctrine 2, PHPUnit, Symfony 2 Routing) use them to inject behaviour without extending, or to contextualize configuration without external configuration files, etc.

There are the two available parser tokens which reference comments:

T_COMMENT

// This is a comment
# This is a comment too
/* This is a multiline
comment */

T_DOC_COMMENT

/**
 * This is a PHPDoc
 */

But only T_DOC_COMMENT can affect your code, this means you must use PHPDoc to access to the available annotations in runtime.

Note: All mentioned comments might be discarded from your opcode cache.
For more info check the following runtime configuration parameters:
opcache.save_comments and opcache.load_comments.

This is a simple snippet code which dumps the annotations of a property of a class:

<?php

class SomeClass
{
    /**
     * @someKey someValue
     */
    public $someProperty;
}

$someObject = new SomeClass();

$reflectedObject = new ReflectionClass($someObject);
$properties = $reflectedObject->getProperties();

function parseDocComment($docComment) {
    $annotations = array();
    $docComment = substr($docComment, 3, -2);

    $pattern = '/@(?P<name>[A-Za-z_-]+)(?:[ \t]+(?P<value>.*?))?[ \t]*\r?$/m';
    if (preg_match_all($pattern, $docComment, $matches)) {
        $numMatches = count($matches[0]);

        for ($i = 0; $i < $numMatches; ++$i) {
            $annotations[$matches['name'][$i]][] = $matches['value'][$i];
        }
    }

    return $annotations;
}

foreach ($properties as $property) {
    $docComment = $property->getDocComment();
    $annotations = parseDocComment($docComment);
    var_dump($annotations);
}

/*
    array(1) {
        ["someKey"]=>
        array(1) {
            [0]=>
            string(9) "someValue"
        }
    }
 */

There are several libraries which work as engine to parse these annotations, for example:

Handling annotations in PHP

DI and IoC aren’t about Testing

DI and IoC are about decoupling, flexibility and centralization of the point which provides us the maintainability of our applications.

The Testing is important but is not the first nor the most important reason about why to use DI and IoC.

Quote