Category Archives: LXCFS

LXC/LXCFS/Incus 6.0.1 LTS release

Posted on 2024/07/01 by Stéphane Graber

Introduction

The Linux Containers project maintains Long Term Support (LTS) releases for its core projects.
Those come with 5 years of support from upstream with the first two years including bugfixes, minor improvements and security fixes and the remaining 3 years getting only security fixes.

Our current LTS release, 6.0, is as the name implies the 6th time we’ve released an LTS release of our projects, starting over 10 years ago, in February 2014.

At the time of writing, we have three currently supported LTS releases:

  • 4.0 (supported until June 2025, security-only)
  • 5.0 (supported until June 2027, security-only)
  • 6.0 (supported until June 2029).

The 6.0 LTS release begun in April 2024 and was the first to include Incus.

LXC

LXC is the oldest Linux Containers project and the basis for almost every other one of our projects.
This low-level container runtime and library was first released in August 2008, led to the creation of projects like Docker and today is still actively used directly or indirectly on millions of systems.

Announcement: https://discuss.linuxcontainers.org/t/lxc-6-0-1-lts-has-been-released/20283

Highlights of this point release:

  • Fixed some build tooling issues
  • Fixed startup failures on system without IPv6 support
  • Updated AppArmor rules to avoid potential warnings

LXCFS

LXCFS is a FUSE filesystem used to workaround some shortcomings of the Linux kernel when it comes to reporting available system resources to processes running in containers.
The project started in late 2014 and is still actively used by Incus today as well as by some Docker and Kubernetes users.

Unfortunately the LXCFS approach is starting to run into issues due to tools relying more and more on system call interfaces or other methods to obtain resource information these days requiring more complex solution such as Incus’ system call interception support (using the Seccomp Notifier).

Because of that development, we’ve been slowly discussing better ways to provide reliable resource information to userspace without having to rely on filesystem tricks or costly system call interception, but as with anything that requires widespread userspace adoption, it will take a while until such a solution is in place and so LXCFS isn’t going anywhere any time soon!

Announcement: https://discuss.linuxcontainers.org/t/lxcfs-6-0-1-lts-has-been-released/20277

Highlights of this point release:

  • Support for running multiple instances of LXCFS (--runtime-dir)
  • Detect systems that has a Yama policy preventing reading process personalities

Incus

Incus is our most actively developed project. This virtualization platform is less than a year old but has already seen over 3000 commits by over 100 individual contributors. Its first LTS release made it usable in production environments and significantly boosted its user base.

Announcement: https://discuss.linuxcontainers.org/t/incus-6-0-1-lts-has-been-released/20297

Highlights of this point release:

  • Extended source syntax for ZFS pools (allows mirror & raidz1/raidz2)
  • Cross-project listing on all objects (instances, profiles, images, storage volumes/buckets, networks, …)
  • Additional functions exposed to instance placement scriptlet
  • All create sub-commands in the CLI now accept YAML input
  • All list sub-commands in the CLI now accept customizable columns
  • The migration.stateful config key was expanded to containers too
  • Stateless network ACLs are now supported on OVN
  • New timestamp exposed for instance uptime
  • New incus top command (uses existing metric API)
  • System load information in incus info --resources
  • PCI devices information in incus info --resources
  • Ability to query who has access to a given project or instance
  • Forceful deletion of projects
  • Improved alias handling in incus-simplestreams

What’s next?

We’re going to keep backporting all relevant fixes and minor improvements to our LTS branches and will likely be releasing another LTS point release of those 3 projects later this year.

There is no set schedule on LTS point releases as we instead prefer to wait until we feel there are significant enough fixes to warrant one, then make sure that all three projects are properly tested and ready for a release.

This year we’ve also decided to start releasing non-LTS releases of both LXC and LXCFS.
It’s something we used to do some years ago but then stopped, mostly due to lack of time.
So you can look forward to LXC and LXCFS 6.1 in Q4 of 2024!

Posted in Incus, LXC, LXCFS, Planet Ubuntu | Leave a comment

A month later

Posted on 2023/08/10 by Stéphane Graber

It’s now been a whole month since I left Canonical and started working as an independent!

This has been quite the month, both professionally and personally!
In no particular order, this included, setting up a new business, dealing with a somewhat last minute datacenter move (thankfully just one floor down), doing some initial sponsored work, helping out with a LXD fork, selling a house and caring for a sick cat (now all back to normal).

Given everything that’s been happening, I thought I’d use the opportunity to write down some details on the most relevant things I’ve been doing and what to expect moving forward.

Zabbly

Zabbly is the name of the business I’ve registered here in Canada.

I didn’t really like the idea of doing all business moving forward just under my own name as I may want to sub-contract some aspects of it or even have employees down the line.
Having the business part of my life have its own name will make that a fair bit cleaner.

For now, the main things that have been moved over to Zabbly are my organization and IP allocations with ARIN, membership on the Montreal Internet Exchange (QIX) and a number of associated contracts related to AS399760 (my BGP ASN). As part of that, Zabbly is also now listed as the sponsor for all the Linux Containers infrastructure.

Allowing to more clearly separate personal and work-related expenses is going to be another benefit of this move even if legally and from a tax point of view, it’s still all me.

ZFS delegation

An initial bit of sponsored work I got to do this month has been adding support for ZFS delegation to LXD. This makes use of a ZFS 2.2 feature which allows for a dataset to be delegated to a particular user namespace. The ZFS tools can then be used from within that container to create nested datasets or manage snapshots.

This is very exciting as it was the one feature that btrfs had which ZFS offered no equivalent for. It should allow for things like running Docker with the ZFS backend inside of LXD containers, having VPS users be able to create their own datasets, handled their own snapshots and be able to send and receive datasets.

The pull request can be found here: https://github.com/canonical/lxd/pull/12056

Incus

Some of you may have seen the announcement of a new LXD fork called Incus and its subsequent inclusion into the Linux Containers project.

This was quite an exciting development and the LXC team spent quite a bit of time over the past couple weeks chatting with Aleksa and seeing where things were headed.

On my end, I initially helped out trying to make the thing actually pass the testsuite, quite a bit harder than it may sound when dealing with a pretty big codebase and everything having been renamed! I also contributed some ideas of what such a fork may want to change compared to stock LXD.

It’s not often that you get a second chance at designing something like LXD/Incus.
While having a working upgrade path and good backward compatibility is obviously still very important, the fact that anyone migrating will need to deal with some amount of manual work also makes it possible to do away with past mistakes and remove some bits that are seldom used.

I expect I’ll be spending a bunch of my time over the next couple of months helping get Incus into a releasable state. Continuing with the current cleanups, getting the documentation back into shape, putting CI and publishing infrastructure back online (basically re-using what I was once providing to LXD).

The biggest task yet to come is to write tooling and processes to monitor changes happening in Canonical’s LXD and then cherry-pick those into Incus. Again, the hard fork, name and path changes and variety of other changes is going to make that a bit of a challenge but once done, it should make it quite easy to do weekly syncs and reviews of changes.

What’s next

As mentioned, I expect to spend a fair bit of my time over the next few weeks/months helping out with Incus, getting it into shape for an initial release.

For those who enjoyed the LXD YouTube channel, I’m also setting up a new channel that will primarily cover Incus but also some other of my projects: https://www.youtube.com/@TheZabbly.

I’m all set up for contract work and sponsorship now, so if there’s anything you think I can do for you, feel free to reach out at info@zabbly.com.

I’ve also been added to the Github Sponsors program, so if you’d just like to help out with my work on those various projects, that’s available too: https://github.com/sponsors/stgraber

Posted in Incus, LXC, LXCFS, Planet Ubuntu, Zabbly | 4 Comments

LXCFS 2.0 has been released!

Posted on 2016/03/31 by Stéphane Graber

LXD logo

What’s LXCFS?

LXCFS is a side project of LXC and LXD. It’s basically a tiny FUSE filesystem which gets mounted in your containers and mask a number of proc files.

At present, it supports the following files:

  • /proc/cpuinfo
    Only returns the CPUs listed in your cpuset
  • /proc/diskstats
    Returns I/O usage from the container
  • /proc/meminfo
    Only shows the amount of memory and SWAP the container can use
  • /proc/stat
    Related to cpuinfo, only lists the right CPUs
  • /proc/swaps
    Related to meminfo, only shows your container’s swap consumption
  • /proc/uptime
    Shows the container uptime instead of the host’s

It’s basically a userspace workaround to changes which were deemed unreasonable to do in the kernel. It makes containers feel much more like separate systems than they would without it.

On top of the proc virtualization feature, lxcfs also supports rendering a partial cgroupfs view which can then be mounted into a container on top of /sys/fs/cgroup, allowing processes in the container to interact with the cgroups in a safe way.

This part is only enabled on kernels that do not support the cgroup namespace, as newer kernels (4.6 upstream, 4.4 Ubuntu) no longer need this.

Why do I need it?

lxcfs isn’t absolutely needed to run LXC or LXD containers.

That being said, you will want it if:

  • You want proper resource consumption reporting inside your container
  • You need to start a systemd based container on a system running a kernel older than 4.6 upstream (or 4.4 Ubuntu)

LXD in Ubuntu actually depends on LXCFS as we think it’s a critical part of offering a good container experience on Ubuntu.

How to get it?

LXCFS is available in quite a few distributions, so chances are you can just grab it with your package manager. It may take a few days/weeks for 2.0 to be available though.

Ubuntu users have had lxcfs available for a few years now and the 2.0 release is now in the Ubuntu development release. Up to date packages for all Ubuntu releases can also be found in our PPAs.

What kind of support will this get?

LXCFS 2.0 is a long term support release. That means that upstream LXCFS will be pushing out bugfix and security releases for the next 5 years.

A separate stable branch will be setup upstream and bugfixes will be cherry-picked into it, when enough fixes have accumulated a bugfix release (like 2.0.1) will be released.

Project information

Upstream website: https://linuxcontainers.org/lxcfs/
Release announcement: https://linuxcontainers.org/lxcfs/news/
Code: https://github.com/lxc/lxcfs
IRC channel: #lxcontainers on irc.freenode.net
Mailing-lists: https://lists.linuxcontainers.org

Try it online

Want to see what a container with LXCFS installed feels like?
You can get one online to play with here.

Posted in Canonical voices, LXCFS, Planet Ubuntu | Tagged | 4 Comments