With the DNS changes in Ubuntu 12.04, most development machines running with libvirt and lxc end up running quite a few DNS servers.
These DNS servers work fine when queried from a system on their network, but aren’t integrated with the main dnsmasq instance and so won’t let you resolve your VM and containers from outside of their respective networks.
One way to solve that is to install yet another DNS resolver and use it to redirect between the various dnsmasq instances. That can quickly become tricky to setup and doesn’t integrate too well with resolvconf and NetworkManager.
Seeing a lot of people wondering how to solve that problem, I took a few minutes yesterday to come up with an ssh configuration that’d allow one to access their containers and VM using their name.
The result is the following, to add to your ~/.ssh/config file:
Host *.lxc
StrictHostKeyChecking no
UserKnownHostsFile /dev/null
ProxyCommand nc $(host $(echo %h | sed "s/\.lxc//g") 10.0.3.1 | tail -1 | awk '{print $NF}') %p
Host *.libvirt
StrictHostKeyChecking no
UserKnownHostsFile /dev/null
ProxyCommand nc $(host $(echo %h | sed "s/\.libvirt//g") 192.168.122.1 | tail -1 | awk '{print $NF}') %p
After that, things like:
- ssh user@myvm.libvirtu
- ssh ubuntu@mycontainer.lxc
Will just work.
For LXC, you may also want to add a “User ubuntu” line to that config as it’s the default user for LXC containers on Ubuntu.
If you configured your bridges with a non-default subnet, you’ll also need to update the IPs or add more sections to the config.
These also turn off StrictHostKeyChecking and UserKnownHostsFile as my VMs and containers are local to my machine (reducing risk of MITM attacks) and tend to exist only for a few hours, to then be replaced by a completely different one with a different SSH host key. Depending on your setup, you may want to remove these lines.
Github
Twitter
LinkedIn
Mastodon
RSS feed
Stephane,
I would need one simple clarification : how do you get your specific lxc/libvirt domains defined to start with ? Any specific paramater for dnsmasq ?
TIA,
…Louis
Nope, these fake .lxc and .libvirt domains are there just to tell ssh which config to apply, it’s then stripped in the ProxyCommand when querying the right dnsmasq server with host.
Any tricks to make these fake domains work with curl/chrome/telnet/…
If you are using the UEC images in KVM, you have to provide a couple userdata options to get it dnsmasq to have the correct hostname (and the ssh tricks to work):
hostname: myvm
runcmd:
- [ "/etc/init.d/networking", restart ]
The reason for restarting networking with userdata: the real hostname will be “ubuntu” until cloud-init runs – and cloud-init runs after networking. This means dnsmasq has the hostname of ubuntu. Restarting networking goes through enough of the dhcp dance to update the hostname in dnsmasq.
Doesn’t work for me (on 12.10).
$ ssh lxc_test3.lxc -vvv
OpenSSH_6.0p1 Debian-3ubuntu1, OpenSSL 1.0.1c 10 May 2012
debug1: Reading configuration data /home/peter/.ssh/config
debug1: /home/peter/.ssh/config line 173: Applying options for *.lxc
debug1: /home/peter/.ssh/config line 187: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug2: ssh_connect: needpriv 0
debug1: Executing proxy command: exec nc $(host $(echo lxc_test3.lxc | sed "s/\\\\\\\\.lxc//g") 10.0.3.1 | tail -1 | awk '{print $NF}') 22
debug1: permanently_drop_suid: 1000
debug3: Incorrect RSA1 identifier
debug3: Could not load "/home/peter/.ssh/id_rsa" as a RSA1 public key
debug1: identity file /home/peter/.ssh/id_rsa type 1
debug1: Checking blacklist file /usr/share/ssh/blacklist.RSA-2048
debug1: Checking blacklist file /etc/ssh/blacklist.RSA-2048
debug1: identity file /home/peter/.ssh/id_rsa-cert type -1
nc: getaddrinfo: Name or service not known
ssh_exchange_identification: Connection closed by remote host
That’s because wordpress is stupid… it should only be ‘sed “s/\.lxc//g”‘ with a single backslash… wordpress seems to be adding an extra one everytime I update the post or something (with a minimum of two…)
With that in mind you might want to correct the .libvert line too…
Fixed for good, I ended up using the html code to workaround WordPress’ silly escaping…
Well I see that it is just trying to resolve the container name. It doesn’t work:
$ sudo lxc-info -n lxc_test3
state: RUNNING
pid: 27129
$ sudo lsof -i4udp:53 -n -P
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
dnsmasq 1676 lxc-dnsmasq 6u IPv4 14559 0t0 UDP 10.0.3.1:53
dnsmasq 1934 libvirt-dnsmasq 6u IPv4 1703 0t0 UDP 192.168.122.1:53
$ host lxc_test3 10.0.3.1
Using domain server:
Name: 10.0.3.1
Address: 10.0.3.1#53
Aliases:
Host lxc_test3 not found: 3(NXDOMAIN)
It’s resolving based on what dnsmasq received from the container, so the container needs to be using dhcp and you need to look for the container’s hostname, not for the container’s name (the two are usually the same but not necessarily).
You’ll want to add the -N0 option to the host command so that the number of dots that have to be in a name for it to be considered absolute is zero, otherwise it may be interpreted as a relative name and will be search for in the domains listed in the search or domain directive in /etc/resolv.conf
I second this. I needed to add -N0 (dash N zero) to my host command to make this all work properly.
Hi,
Do you know a way to do this on the host side? I have multiple web server in lxc containers and I want the users to access their container with ssh. Something like a apache virtual host but for ssh. My starting point is that post from linux.org:
http://www.linux.org/threads/linux-containers-part-7-http-and-https-routing.4544/
ssh p1.lxc
works great but it asks for the password of ubuntu.
I know I could manually copy my public ssh-key to the containters, but I wonder how to avoid/do that programatically.
Intel said it will continue to keep its Chinese mainland headquarters and a research and development facility in Shanghai while it increases investment in its Dalian facility in northeast China, its first 12-inch wafer plant and biggest investment project in China.