LXC 1.0: Scripting with the API [8/10]

This is post 8 out of 10 in the LXC 1.0 blog post series.

The API

The first version of liblxc was introduced in LXC 0.9 but it was very much at an experimental state. LXC 1.0 however will ship with a much more complete API, covering all of LXC’s features. We’ve actually been rebasing all of our tools (lxc-*) to using that API rather than doing direct calls to the internal functions.

The API also comes with a whole set of tests which we run as part of our continuous integration setup and before distro uploads.

There are also quite a few bindings for those who don’t feel like writing C, we have lua and python3 bindings in-tree upstream and there are official out-of-tree bindings for Go and ruby.

The API documentation can be found at:
https://linuxcontainers.org/lxc/documentation/

It’s not necessarily the most readable API documentation ever and certainly could do with some examples, especially for the bindings, but it does cover all functions that are exported over the API. Any help improving our API documentation is very much welcome!

The basics

So let’s start with a very simple example of the LXC API using C, the following example will create a new container struct called “apicontainer”, create a root filesystem using the new download template, start the container, print its state and PID number, then attempt a clean shutdown before killing it.

#include <stdio.h>

#include <lxc/lxccontainer.h>

int main() {
    struct lxc_container *c;
    int ret = 1;

    /* Setup container struct */
    c = lxc_container_new("apicontainer", NULL);
    if (!c) {
        fprintf(stderr, "Failed to setup lxc_container struct
");
        goto out;
    }

    if (c->is_defined(c)) {
        fprintf(stderr, "Container already exists
");
        goto out;
    }

    /* Create the container */
    if (!c->createl(c, "download", NULL, NULL, LXC_CREATE_QUIET,
                    "-d", "ubuntu", "-r", "trusty", "-a", "i386", NULL)) {
        fprintf(stderr, "Failed to create container rootfs
");
        goto out;
    }

    /* Start the container */
    if (!c->start(c, 0, NULL)) {
        fprintf(stderr, "Failed to start the container
");
        goto out;
    }

    /* Query some information */
    printf("Container state: %s
", c->state(c));
    printf("Container PID: %d
", c->init_pid(c));

    /* Stop the container */
    if (!c->shutdown(c, 30)) {
        printf("Failed to cleanly shutdown the container, forcing.
");
        if (!c->stop(c)) {
            fprintf(stderr, "Failed to kill the container.
");
            goto out;
        }
    }

    /* Destroy the container */
    if (!c->destroy(c)) {
        fprintf(stderr, "Failed to destroy the container.
");
        goto out;
    }

    ret = 0;
out:
    lxc_container_put(c);
    return ret;
}

So as you can see, it’s not very difficult to use, most functions are fairly straightforward and error checking is pretty simple (most calls are boolean and errors are printed to stderr by LXC depending on the loglevel).

Python3 scripting

As much fun as C may be, I usually like to script my containers and C isn’t really the best language for that. That’s why I wrote and maintain the official python3 binding.

The equivalent to the example above in python3 would be:

import lxc
import sys

# Setup the container object
c = lxc.Container("apicontainer")
if c.defined:
    print("Container already exists", file=sys.stderr)
    sys.exit(1)

# Create the container rootfs
if not c.create("download", lxc.LXC_CREATE_QUIET, {"dist": "ubuntu",
                                                   "release": "trusty",
                                                   "arch": "i386"}):
    print("Failed to create the container rootfs", file=sys.stderr)
    sys.exit(1)

# Start the container
if not c.start():
    print("Failed to start the container", file=sys.stderr)
    sys.exit(1)

# Query some information
print("Container state: %s" % c.state)
print("Container PID: %s" % c.init_pid)

# Stop the container
if not c.shutdown(30):
    print("Failed to cleanly shutdown the container, forcing.")
    if not c.stop():
        print("Failed to kill the container", file=sys.stderr)
        sys.exit(1)

# Destroy the container
if not c.destroy():
    print("Failed to destroy the container.", file=sys.stderr)
    sys.exit(1)

Now for that specific example, python3 isn’t that much simpler than the C equivalent.

But what if we wanted to do something slightly more tricky, like iterating through all existing containers, start them (if they’re not already started), wait for them to have network connectivity, then run updates and shut them down?

import lxc
import sys

for container in lxc.list_containers(as_object=True):
    # Start the container (if not started)
    started=False
    if not container.running:
        if not container.start():
            continue
        started=True

    if not container.state == "RUNNING":
        continue

    # Wait for connectivity
    if not container.get_ips(timeout=30):
        continue

    # Run the updates
    container.attach_wait(lxc.attach_run_command,
                          ["apt-get", "update"])
    container.attach_wait(lxc.attach_run_command,
                          ["apt-get", "dist-upgrade", "-y"])

    # Shutdown the container
    if started:
        if not container.shutdown(30):
            container.stop()

The most interesting bit in the example above is the attach_wait command, which basically lets your run a standard python function in the container’s namespaces, here’s a more obvious example:

import lxc

c = lxc.Container("p1")
if not c.running:
    c.start()

def print_hostname():
    with open("/etc/hostname", "r") as fd:
        print("Hostname: %s" % fd.read().strip())

# First run on the host
print_hostname()

# Then on the container
c.attach_wait(print_hostname)

if not c.shutdown(30):
    c.stop()

And the output of running the above:

stgraber@castiana:~$ python3 lxc-api.py
/home/stgraber/<frozen>:313: Warning: The python-lxc API isn't yet stable and may change at any point in the future.
Hostname: castiana
Hostname: p1

It may take you a little while to wrap your head around the possibilities offered by that function, especially as it also takes quite a few flags (look for LXC_ATTACH_* in the C API) which lets you control which namespaces to attach to, whether to have the function contained by apparmor, whether to bypass cgroup restrictions, …

That kind of flexibility is something you’ll never get with a virtual machine and the way it’s supported through our bindings makes it easier than ever to use by anyone who wants to automate custom workloads.

You can also use the API to script cloning containers and using snapshots (though for that example to work, you need current upstream master due to a small bug I found while writing this…):

import lxc
import os
import sys

if not os.geteuid() == 0:
    print("The use of overlayfs requires privileged containers.")
    sys.exit(1)

# Create a base container (if missing) using an Ubuntu 14.04 image
base = lxc.Container("base")
if not base.defined:
    base.create("download", lxc.LXC_CREATE_QUIET, {"dist": "ubuntu",
                                                   "release": "precise",
                                                   "arch": "i386"})

    # Customize it a bit
    base.start()
    base.get_ips(timeout=30)
    base.attach_wait(lxc.attach_run_command, ["apt-get", "update"])
    base.attach_wait(lxc.attach_run_command, ["apt-get", "dist-upgrade", "-y"])

    if not base.shutdown(30):
        base.stop()

# Clone it as web (if not already existing)
web = lxc.Container("web")
if not web.defined:
    # Clone base using an overlayfs overlay
    web = base.clone("web", bdevtype="overlayfs",
                     flags=lxc.LXC_CLONE_SNAPSHOT)

    # Install apache
    web.start()
    web.get_ips(timeout=30)
    web.attach_wait(lxc.attach_run_command, ["apt-get", "update"])
    web.attach_wait(lxc.attach_run_command, ["apt-get", "install",
                                             "apache2", "-y"])

    if not web.shutdown(30):
        web.stop()

# Create a website container based on the web container
mysite = web.clone("mysite", bdevtype="overlayfs",
                   flags=lxc.LXC_CLONE_SNAPSHOT)
mysite.start()
ips = mysite.get_ips(family="inet", timeout=30)
if ips:
    print("Website running at: http://%s" % ips[0])
else:
    if not mysite.shutdown(30):
        mysite.stop()

The above will create a base container using a downloaded image, then clone it using an overlayfs based overlay, add apache2 to it, then clone that resulting container into yet another one called “mysite”. So “mysite” is effectively an overlay clone of “web” which is itself an overlay clone of “base”.

 

So there you go, I tried to cover most of the interesting bits of our API with the examples above, though there’s much more available, for example, I didn’t cover the snapshot API (currently restricted to system containers) outside of the specific overlayfs case above and only scratched the surface of what’s possible to do with the attach function.

LXC 1.0 will release with a stable version of the API, we’ll be doing additions in the next few 1.x versions (while doing bugfix only updates to 1.0.x) and hope not to have to break the whole API for quite a while (though we’ll certainly be adding more stuff to it).

About Stéphane Graber

Project leader of Linux Containers, Linux hacker, Ubuntu core developer, conference organizer and speaker.
This entry was posted in Canonical voices, LXC, Planet Ubuntu and tagged . Bookmark the permalink.

21 Responses to LXC 1.0: Scripting with the API [8/10]

  1. vasilisc says:

    >clone it again as a website instance based itself on the intermediary container.
    website OR mysite?
    #mysite = web.clone(“mysite”,

    1. I guess ‘as an instance of “web” called “mysite”‘ would have been clearer 🙂

      I’ll update the post.

  2. bmullan says:

    Stephane… I just want to say thank you for this great series of technical writeups for lxc. These have really been informative and useful.
    brian

  3. bmullan says:

    Stephane…

    Has anyone discussed the idea of creating a repo for lxc python scripts?
    This would make a lot of sense to do sooner than later so contributors could collect python snippets together in one place much like pypi

    Just a thought to help spur use/re-use and grow the collection of apps related to this lxc.

    Brian

    1. I don’t think anyone has but that’d probably be quite useful to some people.

  4. Paul Thomson says:

    hey, is there an easy way to extend the lxc-python3 binding? For example to add lxc-autostart with its options to the lxc-python3 api? I want to start for example a group of containers without looping through existing containers to use the lxc.start.order flag within my configuration files. Thanks.

  5. Paul Thomson says:

    hey there,
    is it possible to set ip address and bridge device from python via the lxc api? With a stopped container calling: container.set_config_item(“lxc.network.0.ipv4, “10.0.3.200/24”) and then container.save_config() ?

    Would be very nice, thanks for your answer.

  6. Phi-Ho says:

    Hi Stephan,

    Is it possible to have a Python version of the /usr/bin/lxc-* commands.

    Best regards,

    PhiHo

  7. Satish says:

    I am trying to create containers which can be accessed from outside.So i’ve chosen the macvlan mode of networking which can assign an ip to the containers in the same network as the host. I have added the following in my /etc/network/interfaces file

    auto macvlan0

    iface macvlan0 inet static

    pre-up route del default

    pre-up route del -net aa.bb.cc.0 netmask 255.255.255.0

    pre-up ip link add link eth0 name macvlan0 type macvlan mode bridge

    address 192.168.171.141
    this creates a macvlan bridge by the name macvlan0 but i am not able to ping it from inside the container

    this is the conf file of my container (i’ve mentioned only the nerworking part)

    lxc.network.type = macvlan

    lxc.network.macvlan.mode = bridge

    lxc.network.flags = up

    lxc.network.link = eth0

    lxc.network.ipv4 = aa.bb.cc.155/24
    can you tell me what i am doing wrong?

  8. Raph says:

    Hi, how do you can add more parameters when you create new container using python3-lxc, like that in cli : lxc-create -t ubuntu -n u1 -B lvm –vgname ubuntu-vg –fssize 500M –fstype ext4 — -u bob –password bob

    I try to do something like that (without user and password), but it is not working..
    lxc.create(“download”, lxc.LXC_CREATE_QUIET,
    (‘–dist’,dist, ‘–release’, release, ‘–arch’, arch, ‘-B’, ‘lvm’, ‘–vgname’, ‘ubuntu-vg’, ‘–fssize’, ‘500M’, ‘–fstype’, ‘ext4’)

    Thank you for your help.

  9. Malina says:

    It seems in trusty 14.04 , lxc macvlans don’t work?

    It works on the host, and it gets an ip from upstream dhcp server., but defining one inside the config file (eg. bridge/vepa) = no go. Looking at the log output when starting guest, only when I add a veth does it recognise it and set it up.

    No errors in log about any macvlan, aka it seems as it doesn’t even know about parsing such an option.?

    lxc version = 1.0.6
    3.18 kernel

    1. Malina says:

      Sorry, Didn’t see the lxc-usernet man. It is not supported (yet) in unprivileged space.

  10. brian mullan says:

    Stephane.. just fyi but the link referenced above is broken now:

    The API documentation can be found at:
    https://qa.linuxcontainers.org/master/current/doc/api/

  11. John Crunk says:

    I really love the Python bindings. I’m only having a couple of problems. How do you pass arguments to the Python function when calling attach.wait?

    And I can’t seem to get the lxc.attach_run_command to execute my files in the container. It works OK for the normal commands, but when I give it MY file, it says “not runnable”

    1. John Crunk says:

      Just an addendum:

      This is very confusing. The attach command appears to be running the program in the host, not in the container. Example:
      container = lxc.Container(‘mycontainer’)
      container.attach_wait(lxc.attach_run_command([‘/root/testecho’])).

      This runs “/root/testecho” in my host, not the container.

  12. Tonio says:

    The API documentation link is still broken. Any other link ?

    1. Updated to a new working link.

  13. Rohit Sehgal says:

    lxc-attach executes the command in pseudo terminal,is there any way to execute the same command in specific console.
    I am struggling a lot to find the help regarding the same.

  14. Rohit says:

    lxc-attach -n — tty :

    outputs : Not a tty

    How can i attach a process to a specific tty device. I mean is there any possible for that.

  15. Jackie says:

    I am getting the error like below, I tried to start a new api container with the api ” c = lxc_container_new(“apicontainer”, NULL);”
    ERROR lxc_execute – execute.c:execute_start:93 – Bad address – failed to exec /usr/sbin/init.lxc
    lxc 20181114075947.204 ERROR lxc_sync – sync.c:__sync_wait:57 – An error occurred in another process (expected sequence number 7)

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.