Category Archives: Incus

LXC/LXCFS/Incus 6.0.2 LTS release

Introduction

The Linux Containers project maintains Long Term Support (LTS) releases for its core projects.
Those come with 5 years of support from upstream with the first two years including bugfixes, minor improvements and security fixes and the remaining 3 years getting only security fixes.

This is now the second round of bugfix releases for LXC, LXCFS and Incus 6.0 LTS.

LXC

LXC is the oldest Linux Containers project and the basis for almost every other one of our projects.
This low-level container runtime and library was first released in August 2008, led to the creation of projects like Docker and today is still actively used directly or indirectly on millions of systems.

Announcement: https://discuss.linuxcontainers.org/t/lxc-6-0-2-lts-has-been-released/21632

Highlights of this point release:

  • Reduced log level on some common messages
  • Fix compilation error on aarch64

LXCFS

LXCFS is a FUSE filesystem used to workaround some shortcomings of the Linux kernel when it comes to reporting available system resources to processes running in containers.
The project started in late 2014 and is still actively used by Incus today as well as by some Docker and Kubernetes users.

Announcement: https://discuss.linuxcontainers.org/t/lxcfs-6-0-2-lts-has-been-released/21631

Highlights of this point release:

  • Fix building of LXCFS on musl systems (missing include)

Incus

Incus is our most actively developed project. This virtualization platform is just over a year old but has already seen over 3500 commits by over 120 individual contributors. Its first LTS release made it usable in production environments and significantly boosted its user base.

Announcement: https://discuss.linuxcontainers.org/t/incus-6-0-2-lts-has-been-released/21633

Highlights of this point release:

  • Completion of transition to native OVSDB for OVS/OVN
  • Baseline CPU defintiion for clustered users
  • Filesystem support for io.bus and io.cache
  • CPU flags in server resources
  • Unified image support in incus-simplestreams
  • Completion of libovsdb transition
  • Using a sub-path of a volume as a disk
  • Per storage pool projects limits
  • Isolated OVN networks (no uplink)
  • Per-instance LXCFS
  • Support for environment file at create/launch time
  • Instance auto-restart
  • Column selection in all list commands
  • QMP command hooks and scriptlet
  • Live disk resize support in virtual machines
  • PCI devices hotplug
  • OVN load-balancer health checks
  • Promiscuous mode for OVN NICs
  • Ability to run off IP allocation on OVN NICs
  • Customizable OIDC scope request
  • Configurable LVM PV metadata size
  • Configurable OVS socket path

What’s next?

We’re expecting another LTS bugfix release for the 6.0 branches later this year.

On top of that, Q4 of 2024 will also feature non-LTS feature releases of both LXC and LXCFS as we’re trying to push out new releases of those two projects every 6 months now.

Incus will keep going with its usual monthly feature release cadence.

Posted in Incus, LXC, LXCFS, Planet Ubuntu | 1 Comment

Announcing Incus 6.5

This release contains a very good mix of bug fixes and performances improvements as well as exciting new features across the board!

The highlights for this release are:

  • Instance auto-restart
  • Column selection in all list commands
  • QMP command hooks and scriptlet
  • Live disk resize for VMs
  • PCI devices hotplug for VMs
  • OVN load-balancer health checks
  • OVN Interconnect ECMP support
  • OVN NICs promiscuous mode
  • OVN NICs disabling of IP allocation
  • Configurable LVM PV metadata size
  • Configurable OVS socket path

The full announcement and changelog can be found here.
And for those who prefer videos, here’s the release overview video:

You can take the latest release of Incus up for a spin through our online demo service at: https://linuxcontainers.org/incus/try-it/

And as always, my company is offering commercial support on Incus, ranging from by-the-hour support contracts to one-off services on things like initial migration from LXD, review of your deployment to squeeze the most out of Incus or even feature sponsorship. You’ll find all details of that here: https://zabbly.com/incus

Donations towards my work on this and other open source projects is also always appreciated, you can find me on Github Sponsors, Patreon and Ko-fi.

Enjoy!

Posted in Incus, LXD, Planet Ubuntu, Zabbly | 1 Comment

Announcing Incus 6.4

This release builds upon the recently added OCI support from Incus 6.3, making it even easier to run application containers. It also adds a number of useful new features for clustered and larger environments with more control on the virtual CPU used when live migrating VMs and finer grained resource constraints within projects.

The highlights for this release are:

  • Cluster group configuration
  • Per-cluster group CPU baseline
  • Attaching sub-directories of custom storage volumes
  • Per storage pool project limits
  • Isolated OVN networks (no uplink)
  • Per-instance LXCFS
  • Environment files at create/launch time

The full announcement and changelog can be found here.
And for those who prefer videos, here’s the release overview video:

You can take the latest release of Incus up for a spin through our online demo service at: https://linuxcontainers.org/incus/try-it/

And as always, my company is offering commercial support on Incus, ranging from by-the-hour support contracts to one-off services on things like initial migration from LXD, review of your deployment to squeeze the most out of Incus or even feature sponsorship. You’ll find all details of that here: https://zabbly.com/incus

Donations towards my work on this and other open source projects is also always appreciated, you can find me on Github Sponsors, Patreon and Ko-fi.

Enjoy!

Posted in Incus, LXD, Planet Ubuntu, Zabbly | Leave a comment

Announcing Incus 6.3

This release includes the long awaited OCI/Docker image support!
With this, users who previously were either running Docker alongside Incus or Docker inside of an Incus container just to run some pretty simple software that’s only distributed as OCI images can now just do it directly in Incus.

In addition to the OCI container support, this release also comes with:

  • Baseline CPU definition within clusters
  • Filesystem support for io.bus and io.cache
  • Improvements to incus top
  • CPU flags in server resources
  • Unified image support in incus-simplestreams
  • Completion of libovsdb transition

The full announcement and changelog can be found here.
And for those who prefer videos, here’s the release overview video:

You can take the latest release of Incus up for a spin through our online demo service at: https://linuxcontainers.org/incus/try-it/

And as always, my company is offering commercial support on Incus, ranging from by-the-hour support contracts to one-off services on things like initial migration from LXD, review of your deployment to squeeze the most out of Incus or even feature sponsorship. You’ll find all details of that here: https://zabbly.com/incus

Donations towards my work on this and other open source projects is also always appreciated, you can find me on Github Sponsors, Patreon and Ko-fi.

Enjoy!

Posted in Incus, LXD, Planet Ubuntu, Zabbly | 1 Comment

One year of freelancing

Introduction

It was exactly one year ago today that I left my day job as Engineering Manager of LXD at Canonical and went freelance. It’s been quite a busy year but things turned out better than I had hoped and I’m excited about year two!

Zabbly

Zabbly is the company I created for my freelance work. Over the year, a number of my personal projects were transferred over to being part of Zabbly, including the operation of my ASN (as399760.net), my datacenter co-location infrastructure and more.

Through Zabbly I offer a mix of by-the-hour consultation with varying prices depending on the urgency of the work (basic consultation, support, emergency support) as well as fixed-cost services, mostly related to Incus (infrastructure review, migration from LXD, remote or on-site trainings, …).

Other than Incus, Zabbly also provides up to date mainline kernel packages for Debian and Ubuntu and associated up to date ZFS packages. This is something that came out as needed for a number of projects I work on, from being able to test Incus on recent Linux kernels to avoiding Ubuntu kernel bugs on my own and NorthSec’s servers.

Zabbly is also the legal entity for donations related to my open source work, currently supporting:

And lastly, Zabbly also runs a Youtube channel covering the various projects I’m involved with.
A lot of it is currently about Incus, but there is also the occasional content on NorthSec or other side projects. The channel grew to a bit over 800 subscribers in the past 10 months or so.

Now, how well is all of that doing? Well enough that I could stop relying on my savings just a few months in and turn a profit by the end of 2023. Zabbly currently has around a dozen active customers from 7 countries and across 3 continents with size ranging from individuals to large governmental agencies.

2024 has also been very good so far and while I’m not back to the level of income I had while at Canonical, I also don’t have to go through 4-5 hours of meetings a day and get to actually contribute to open source again, so I’ll gladly take the (likely temporary) pay cut!

Incus

A lot of my time in the past year has been dedicated to Incus.

This wasn’t exactly what I had planned when leaving Canonical.
I was expecting LXD to keep on going as a proper Open Source project as part of the Linux Containers community. But Canonical had other plans and so things changed a fair bit over the few months following my departure.

For those not aware, the rough timeline of what happened is:

So rather than contributing to LXD while working on some other new projects, a lot of my time has instead gone into setting up the Incus project for success.

And I think I’ve been pretty successful at that as we’re seeing a monthly user base growth (based on image server interactions) of around 25%. Incus is now natively available in most Linux distributions (Alpine, Arch Linux, Debian, Gentoo, Nix, Ubuntu and Void) with more coming soon (Fedora and EPEL).

Incus has 6 maintainers, most of whom were the original LXD maintainers.
We’ve seen over 100 individual contributors since Incus was forked from LXD including around 20 students from the University of Texas in Austin who contributed to Incus as part of their virtualization class.

I’ve been acting as the release manager for Incus, also running all the infrastructure behind the project, mentoring new contributors and reviewing a number of changes while also contributing a number of new features myself, some sponsored by my customers, some just based on my personal interests.

A big milestone for Incus was its 6.0 LTS release as that made it suitable for production users.
Today we’re seeing around 40% of our users running the LTS release while the rest run the monthly releases.

On top of Incus itself, I’ve also gotten to contribute to both create the Incus Deploy project, which is a collection of Ansible playbooks and Terraform modules to make it easy to deploy Incus clusters and contribute to both the Ansible Incus connection plugin and our Incus Terraform/OpenTofu provider.

The other Linux Containers projects

As mentioned in my recent post about the 6.0.1 LTS releases, the Linux Containers project tries to do coordinated LTS releases on our core projects. This currently includes LXC, LXCFS and Incus.

I didn’t have to do too much work myself on LXC and LXCFS, thanks to Aleksandr Mikhalitsyn from the Canonical LXD team who’s been dealing with most of the review and issues in both LXC and LXCFS alongside other long time maintainers, Serge Hallyn and Christian Brauner.

NorthSec

NorthSec is a yearly cybersecurity conference, CTF and training provider, usually happening in late May in Montreal, Canada. It’s been operating since 2013 and is now one of the largest on-site CTF events in the world along with having a pretty sizable conference too.

I’m the current VP of Infrastructure for the event and have been involved with it from the beginning, designing and running its infrastructure, first on a bunch of old donated hardware and then slowly modernizing that to the environment we have now with proper production hardware both at our datacenter and on-site during the event.

This year, other than transitioning everything from LXD to Incus, the main focus has been on upgrading the OS on our 6 physical servers and dozens of infrastructure containers and VMs from Ubuntu 20.04 LTS to Ubuntu 24.04 LTS.

At the same time, also significantly reducing the complexity of our infrastructure by operating a single unified Incus cluster, switching to OpenID Connect and OpenFGA for access control and automating even more of our yearly infrastructure with Ansible and Terraform.

Automation is really key with NorthSec as it’s a non-profit organization with a lot of staffing changes every year, around 100 year long contributors and then an additional 50 or so on-site volunteers!

I went over the NorthSec infrastructure in a couple of YouTube videos:

Conferences

I’ve cut down and focused my conference attendance a fair bit over this past year.
Part of it for budgetary reasons, part of it because of having so many things going on that fitting another couple of weeks of cross-country travel was difficult.

I decided to keep attending two main events. The Linux Plumbers Conference where I co-organizer the Containers and Checkpoint-Restore Micro-Conference and FOSDEM where I co-organize both the Containers and the Kernel devrooms.

With one event usually in September/October and the other in February, this provides two good opportunities to catch up with other developers and users, get to chat a bunch and make plans for the year.

I’m looking forward to catching up with folks at the upcoming Linux Plumbers Conference in Vienna, Austria!

What’s next

I’ve got quite a lot going on, so the remaining half of 2024 and first half of 2025 are going to be quite busy and exciting!

On the Incus front, we’ve got some exciting new features coming in, like the native OCI container support, more storage options, more virtual networking features, improved deployment tooling, full coverage of Incus features in Terraform/OpenTofu and even a small immutable OS image!

NorthSec is currently wrapping up a few last items related to its 2024 edition and then it will be time to set up the development infrastructure and get started on organizing 2025!

For conferences, as mentioned above, I’ll be in Vienna, Austria in September for Linux Plumbers and expect to be in Brussels again for FOSDEM in February.

There’s also more that I’m not quite ready to talk about, but expect some great Incus related news to come out in the next few months!

Posted in Conferences, Incus, LXC, LXCFS, Planet Ubuntu, Zabbly | 2 Comments