As we are getting closer and closer to tagging the final releases of LXC, LXD and LXCFS 2.0, I figured it would be a good idea to talk a bit about everything that went into LXD since we first started that project a year and a half ago.
This is going to be a blog post series similar to what I’ve done for LXC 1.0 a couple years back.
The topics that will be covered are:
- LXD 2.0: Introduction to LXD [1/12]
- LXD 2.0: Installing and configuring LXD [2/12]
- LXD 2.0: Your first LXD container [3/12]
- LXD 2.0: Resource control [4/12]
- LXD 2.0: Image management [5/12]
- LXD 2.0: Remote hosts and container migration [6/12]
- LXD 2.0: Docker in LXD [7/12]
- LXD 2.0: LXD in LXD [8/12]
- LXD 2.0: Live migration [9/12]
- LXD 2.0: LXD and Juju [10/12]
- LXD 2.0: LXD and OpenStack [11/12]
- LXD 2.0: Debugging and contributing to LXD [12/12]
Related posts:
- LXD 2.0 has been released!
- Directly interacting with the LXD API
- LXD is now available in the Ubuntu Snap Store
- Network management with LXD (2.3+)
- Running snaps in LXD containers
- Running Kubernetes inside LXD
- LXD on Debian (using snapd)
- Ubuntu Core in LXD containers
- LXD client on Windows and MacOS
- Run your own LXD demo server
- NVidia CUDA inside a LXD container
- USB hotplug with LXD containers
- Custom user mappings in LXD containers
I’m hoping to post a couple of those every week for the coming month and a half leading to the Ubuntu 16.04 release.
If you can’t wait for all of those to come out to play with LXD, you can also take the guided tour and play with LXD, online through our online demo page.
Github
Twitter
LinkedIn
Mastodon
RSS feed
Thank you. I was hoping you would write another series, this will be super helpful.
Where will I find the canonical documentation for LXD once it has been released? Will there be a complete set of man pages? To give an example, for the documentation on LXD config keys/values, I currently visit
https://github.com/lxc/lxd/blob/master/doc/configuration.md
I couldn’t find this information in the man pages yet.
We may look at dynamically converting those documents into man pages, but so far the most likely outcome for 2.0 is that we’ll have those re-published on our website instead (easier to convert markdown to html than to a man page).
Stéphane, will the API be discussed in any depth in this series? It seems to be one of the lesser documented areas and some practical examples would be unmeasurably useful!
Probably not as part of this series since it’s mostly focused on regular users, but I’m hoping to do a follow up post on it afterwards.
Look forward to it, thank you!
Hi Stéphane,
thanks for your fantastic work on LXD. Are you planning to release a more extensive documentation (beside this great blog posts)?
cheers
edo
We are working on a few website updates to surface more of the upstream documentation on our website, that should help a bit.
Hi Stéphane, and thank you for the ongoing work. I’ve been experimenting with LXD as a replacement for QEMU/KVM based VMs for a while, and it’s great.
Regarding this blog post serie, what about adding an extra (13/12) article: running graphical applications within a LXD container… like the GUI in containers you did for LXC 1.0.
+1
This is awesome. I have been toying with this for two weeks now and am super impressed.
This blog series has also been extremely helpful for setting up and tetsing LXD containers.
Looking forward to more goodness!!!
Please add a post about running a GUI app that shares audio with the host (for example, running Skype in an LXD container).
Many years ago I was able to run an older version of Debian in a chroot and get XDM on VT8, in addition to the host’s XDM on VT7. It would be nice if you could show something similar, too (for example: on an Ubuntu 16.04 host, running GDM on VT7, create a Fedora LXD guest, running GDM on VT8).
I also came here looking for gui in containers, as a way to sandbox and make different contexts for web browsing (work, play, dev etc) was thinking of xephyr or xpra.
Any chance of resurrecting your sandbox project from 2011?
How about a blog post about best practice for backup of containers?
Tthanks for LXD/LXC!
Hi Stéphane, I think LXD is really useful, we use it in development and some production environment.
I wrote a blog post about multi-host LXD clouds because I needed to understand the networking connectivity better. What about a section on multi-host environments?
My post is here:
http://peter-lenderyou.blogspot.com/2016/05/a-lightweight-multi-host-cloud-using-lxd.html
For the network aspect specifically, if you ever want to do live migration or even easy normal migration, you’ll want a shared layer 2. Either using whatever your cloud provider may get you for that or by running some kind of small SDN to give you a virtual network spanning your cloud instances.
You’d then have one of your instances act as the gateway for that network with all traffic going through it as would be the case on a physical network.
Getting traffic into the containers in the cloud, usually involves you doing some NAT as it’s rare for them to route you public subnets directly.
I use PeerVPN for providing L2 (or even L3) network connections between multiple hosts or even clouds between my LXD containers.
I use this to interconnect my AWS and Digital Ocean server LXD containers.
Features
Ethernet tunneling support using TAP devices.
IPv6 support.
Full mesh network topology (w/o requiring any sort of “super-node”)
— the mesh network is “auto-learning” also… big plus..!!
Automatically builds tunnels through firewalls and NATs without any further setup (for example, port forwarding).
Shared key encryption and authentication support.
PeerVPN is super easy to configure/deploy (my config files only have 7-8 statements), written in C++ by a phd graduate who also implemented alot of really nice built-in’s such as auto-fragmentation/reassembly of large frames etc. that are often overlooked in other VPN implementations.
PeerVPN site: https://peervpn.net/
Flockport also did a nice writeup on how to use PeerVPN for L2 or L3 LXC connections (note their writeup uses LXC but it should be easily modified to LXD)
https://www.flockport.com/build-layer2-and-layer-3-overlay-networks-with-peervpn/
Hi Stéphane, Thanks for these blogs they certainly helped with the configuration of our set up.
I was wondering if you had any examples for multi-host set-up, LXD by its very nature can be managed remotely.
We use it to increase the density of services on EC2 containers in AWS. I’ve written an article about our setup here http://peter-lenderyou.blogspot.com/2016/05/a-lightweight-multi-host-cloud-using-lxd.html
Peter
Don’t really have any example handy. It usually just involves having one control machine that’s got its certificate trusted by all the daemons and that’s then used to drive everything.
For larger scale, you may want to look into nova-lxd, using OpenStack to drive all the LXD daemons.
I’m rather interested in seeing a practical example of nova-lxd being driven from the OpenStack webui.
I’ve done a fair share of goofing around with OpenStack, but under almost all circumstances it’s required several VMs and I’d really quite like to run everything from one hefty one.
So what happened to the last three episodes?
They should appear soon. I’ve been traveling quite a bit lately and am also busy preparing for the security contest this weekend.
Hoping to have the next one out next week.
Cannot wait to see an episode on openstack integration.
Thank you,
Hi Stephane,
I have a question about LXD in Ubuntu 16.04. I recently upgraded my server from 14.04 to 16.04 and the new lts is now using the bios device names for the Ethernet naming scheme. I also switched to using LXD vs LXC, which I loved btw. My question or problem I am having is setting up a bridge in the new system. In one of your previous articles it was so easy with LXC and worked like a charm but this new setup has me bouncing back and forth and nothing seems to be doing the trick to get my bridge working with containers. Any ideas or help would greatly be appreciated. I also want thank you for your work and help with these great articles. Keep it up and thanks again.
Try this: https://bayton.org/2016/05/lxd-zfs-and-bridged-networking-on-ubuntu-16-04-lts/
In which cases it make sense to use only LXC and in which LXD? For example I want to host few own web-projects (each project on one container) on one host. But only I have access to this containers. So, it is not so important to provide high isolation security to each container. This containers can be “privileged”. Also I not plan to use special features like live-migration. I want to keep it stupid simple. Also I looks for solution which can work stable in production. Then make it sense to use LXC, because it is lightweight, simple and match my requirements?
https://github.com/lxc/lxd/issues/1857#issuecomment-229962287
dnsmasq part is not explicitly mentioned on introductory articles on lxd, I mean the fact that lxd include and automatically setup a dns server with dnsmasq, neither in a feature list, say “lxd solve the name resolution via dnsmasq”
It would be very valuable to add this on these series, I think it is the most complete introductory posts on lxd
This is great.Got devstack to use lxd as hypervisor for mitaka on 16.04 but got a bit lost on how to get OpenSTack to use a 500GB ZFS pool. zfs list seems to say its being used but Openstack reports only the host disk size.
So +1 for the OpenStack integration article:-)
Thank you
Hi Stéphane,
I have successfully set up a working container following your documentation.
The only problem now is that I would like to add a pass-through nic into the container.
I was able to get this working using the older lxc by editing the lxc config files, but I cannot find the required config files under lxd/lxc?
I have tried the following where enp17s4 is the physical nic in the host
lxc config device add snifferlxc1 eth1 nic nictype=physical parent=enp17s4
error: wtf? hwaddr= name=eth1
But it fails with the above error message.
Can you please provide an example of passing a nic from the host into a container using lxd?
Thanks.
So what happened to the last two episodes?
I’ve done Jason Bayton’s excellent, beautifully written, and articulate tutorial on how to setup LXD containers using zfs as the backing store, and that get an IP address automatically using DHCP. That tutorial shows how to use a block device for your lxd zpool, and also a loopback method- both of which work perfectly.
My only problem is that when I launch a container from an Ubuntu image, say 15.10 or 16.04, I can ssh from the container out to my home network, but cannot ssh into the container from outside. The sshd daemon is up and running in the container, but I always get a publickey error when trying to ssh into the container. I’ve tried all of the ssh tricks to generate the key and copy it to the container server; nothing worked. I can ping the container OK and even create an nginx webserver in it, and reach the webserver from my network, but no ssh into it!
Is there some safeguard or privilege that I’m not seeing inside those container-generating images?
I’m glad you found my guide useful 🙂
3 options:
1. Add your local public key to the container:
> ssh-keygen -t rsa
[run through the setup]
copy the contents of “/home/user/.ssh/id_rsa.pub” to a server user’s /home/user/.ssh/authorized_keys file
2. Enable password authentication
> vim /etc/ssh/sshd_config
Set “PasswordAuthentication” to “yes” and restart the ssh service
3. Go in through LXC
> ssh user@lxdhost
> lxc exec container1 bash
My first few activities on a new LXD container are to disable the ubuntu account, add a new user and set passwordauthentication to yes. They’re all inside my network and hidden behind other servers so I choose convenience over security in opting for password over key authentication.
Thanks for replying to this so fast, and in such a detailed manner! I’ll try these 3 things, as I tried to say in my original post, I think I did all of the possible variations of #1. that could be done. I am far from an ssh guru, generally on my servers and with the vanilla installs I do, it just works as server and client out of the box with no changes- I’m spoiled.
If you would permit me, the only glitch I found in your tutorial was that lxd-client was not installed by the command you gave to install lxd. That may be because I am using Mint 18 on my servers. So I had to install lxd-client in Part 3 : Test, right after doing the zpool check and before doing lxc info. With sudo apt-get install lxd-client
Thanks very sincerely again,
Bob Koretsky
I believe a combination of items 1. and 2. did the trick, I can now ssh into the container. So on my ssh client, I wiped the .ssh directory clean, then generated the rsa key as you suggested. Then wiped the .ssh directory in the container, and created an authorized_keys sub-directory in that .ssh directory of the user in question in the container.
I used the following command to copy the id_rsa.pub from client to the container server:
sudo lxc file push /home/bob/.ssh/id_rsa.pub xen1/home/bob/.ssh/authorized_keys/
where xen1 is the name of the container.
Finally edited both /etc/ssh/sshd_config files on client and in the container, to change PasswordAuthentication to yes.
I must add that I did not disable the ubuntu user, but did create the user bob in the container.
Thanks again for your considerate help!
Great work.
where can I get some help. I have installed ubuntu on virtualbox and from there installed lxd and created a container. All this goes well but I cannot access the container via my host pc(win 7) as I do not know how to set up lxd that the containers are allocated an ip adress on my network (192.168.2.0/254). i have tried editing the lxd defaults file but that does not work. I have also tried creating a bridge in my interfaces file without success. What am I doing wrong?
See Jason Bayton’s post on LXD, ZFS, and setting container IP addresses using bridged networks. Google the post, I don’t have the URL handy. This will solve your problem.
Thank you.
I have looked at this post and tried to set up the interfaces file as he has but no success. The “interfaces” file on my installation does not have any “iface eth0 inet manual” line in it. When I add any line with iface eth0 to this file the linux does not boot with networking. Is there another place that ubuntu 14.04 configures the network?
Typically I’ve only seen interface files at:
/etc/network/interfaces
/etc/network/interfaces.d/[some name.cfg]
If you’ve created a bridge already that should be in the same interfaces file, what’s in my article is using a) 16.04 and b) bare metal.
As you’re doing LXD on a VM via VBOX you should ensure promiscuous mode is enabled for the NIC in question too. (https://www.virtualbox.org/manual/ch06.html)
First, Jason’s post works on Ubuntu 16.04, if you’re doing it on 14.04, I don’t know if everything there is the same! I would recommend upgrading your system to 16.04. It uses systemd, which rocks! If you’re working in a VirtualBox guest, destroy that machine after you get the data off it, and build a new guest with Ubuntu 16.04.
Second, if his post is indeed workable on Ubuntu 14.04, then you need to use the command ip addr show to see what the actual name of your nic is. My nic shows up in /etc/network/interfaces as enp2s0, not eth0. So in editing the interfaces file, I just put that in instead of eth0.
Also, in Virtualbox, you need to set the Network interface to bridged, rather than NAT. Once you do that, the DHCP server on your LAN automatically give the new containers you create a “public facing” IP address.
Re-edit you /etc/network/interfaces file and then continue with Jason’s instructions. Tell me what happens.
Works!
Turns out that it was the “promiscuous mode” in vbox. Once I set the adapter that way things worked a lot better. Used Ubuntu 16 but will also try 14 again and let you know. The problem I have now is that when I try to Putty into the container it wont let me login and returns “Disconnected: No supported authentication methods avaiable (server sent: publickey). Any ideas?
Thanks for your help and a great tutorial
Yeah, by default SSH only allows key-based authentication.
If you don’t want to SSH into the host, then “lxc shell c1” every time you can either generate a key in putty (google the guide) and add it to the server or edit:
/etc/ssh/ssh_config
Locate and amend this line so that it’s enabled (yes):
PasswordAuthentication yes
Obviously less secure, but that’ll give you password authentication which you can use with your user/password. If you haven’t created a new user, it’s a good time to do so rather than trying to authenticate with the “default” user added to all LXD containers:
adduser username
work through the prompts as required, lots of generic info (name, room number.. ) can be skipped.
Hope that helps.
(to add, the SSH config edit will need to be done in every container, no need to touch the LXD host ssh config)
Here’s the link:
https://bayton.org/2016/05/lxd-zfs-and-bridged-networking-on-ubuntu-16-04-lts/
Any tutorials about running GUI in LXD containers? I want to exec Firefox in containers, how to? Thanks
I’ve found that https://github.com/aarnaud/lxd-webui
Thanks in advance for your job, it’s very useful to know well LXD. I saw that this tutorial ihas been made using an Ubuntu 14.04Lts for the 16.04 have you any suggests, because some thinks are changed…..
Thanks for finally writing about >LXD 2.0: Blog post series [0/12] | Stéphane Graber’s website <Loved it!