Intelligente Lösungen
in neuer Dimension

LXD: Container mit Hostonly-Netzwerk

Üblicherweise werden LXD/LXC-Container so aufgesetzt, dass sie per NAT einen Zugriff auf’s Internet haben. Problem: Wenn jemand den Container knackt, kann er relativ problemlos aus dem Container heraus weitere Aktivitäten unternehmen und sich im Netz verbreiten. Wir wollen stattdessen ein Hostonly-Netz für die Container, dann kann sich der Knacker nicht ganz so leicht verbreiten.

Ausgangslage

Rechner mit LXD-Basisinstallation und einigen LXD-Containern, die per NAT-Brücke eingerichtet sind.

LXD-Host:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

lxdhost# lxc network list
+-------------+----------+---------+-------------+---------+
|    NAME     |   TYPE   | MANAGED | DESCRIPTION | USED BY |
+-------------+----------+---------+-------------+---------+
| enp0s31f6   | physical | NO      |             | 0       |
+-------------+----------+---------+-------------+---------+
| lxdnat      | bridge   | YES     |             | 2       |
+-------------+----------+---------+-------------+---------+

lxdhost# lxc network info lxdnat
Name: lxdnat
MAC address: 9e:1e:af:00:00:e8
MTU: 1500
State: up

Ips:
  inet    10.2.131.1
  inet6   fe80::0000:d5ff:0000:52eb

Network usage:
  Bytes received: 1.40MB
  Bytes sent: 28.10MB
  Packets received: 9611
  Packets sent: 13169

lxdhost# lxc list
+-------------+---------+---------------------+------+------------+-----------+
|    NAME     |  STATE  |        IPV4         | IPV6 |    TYPE    | SNAPSHOTS |
+-------------+---------+---------------------+------+------------+-----------+
| tine20      | RUNNING | 10.2.131.20 (eth0)  |      | PERSISTENT | 0         |
+-------------+---------+---------------------+------+------------+-----------+
| ubuntu-1804 | STOPPED |                     |      | PERSISTENT | 0         |
+-------------+---------+---------------------+------+------------+-----------+

LXC-Container – google.com ist erreichbar:

1
2
3
4
5
6
7
8
9
10
11

lxctine20$ ping -c5 google.com
PING google.com (172.217.22.174) 56(84) bytes of data.
64 bytes from arn09s11-in-f14.1e100.net (172.217.22.174): icmp_seq=1 ttl=56 time=7.74 ms
64 bytes from arn09s11-in-f14.1e100.net (172.217.22.174): icmp_seq=2 ttl=56 time=7.74 ms
64 bytes from arn09s11-in-f14.1e100.net (172.217.22.174): icmp_seq=3 ttl=56 time=7.76 ms
64 bytes from arn09s11-in-f14.1e100.net (172.217.22.174): icmp_seq=4 ttl=56 time=7.73 ms
64 bytes from arn09s11-in-f14.1e100.net (172.217.22.174): icmp_seq=5 ttl=56 time=7.76 ms

--- google.com ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4007ms
rtt min/avg/max/mdev = 7.739/7.750/7.769/0.112 ms

Hostonly-Netz anlegen

1

lxdhost# lxc network create lxdhostonly ipv4.address=10.2.130.1/24 ipv4.nat=false ipv6.address=none

Container anlegen

1
2
3
4
5
6

lxdhost# CONTAINER=hostonly
lxdhost# IPADDRESS=10.2.130.20
lxdhost# lxc copy ubuntu-1804 "${CONTAINER}"
lxdhost# lxc network attach lxdhostonly "${CONTAINER}" eth0 
lxdhost# lxc config device set "${CONTAINER}" eth0 ipv4.address "${IPADDRESS}"
lxdhost# echo "${IPADDRESS} ${CONTAINER}" >>/etc/hosts

Kurztest

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

lxdhost# CONTAINER=hostonly
lxdhost# lxc start "${CONTAINER}"
lxdhost# lxc list
+-------------+---------+---------------------+------+------------+-----------+
|    NAME     |  STATE  |        IPV4         | IPV6 |    TYPE    | SNAPSHOTS |
+-------------+---------+---------------------+------+------------+-----------+
| tine20      | RUNNING | 10.2.131.20 (eth0)  |      | PERSISTENT | 0         |
+-------------+---------+---------------------+------+------------+-----------+
| hostonly    | RUNNING | 10.2.130.20 (eth0)  |      | PERSISTENT | 0         |
+-------------+---------+---------------------+------+------------+-----------+
| ubuntu-1804 | STOPPED |                     |      | PERSISTENT | 0         |
+-------------+---------+---------------------+------+------------+-----------+
lxdhost# ssh hostonly
The authenticity of host 'hostonly (10.2.130.20)' can't be established.
ECDSA key fingerprint is SHA256:Gq43Yt65PzK15Cxxxxxxxxxxo6aUxkAa8OczS9OYapY.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'hostonly,10.2.130.20' (ECDSA) to the list of known hosts.

hostonly# ping -c5 google.com
PING google.com (172.217.22.174) 56(84) bytes of data.

--- google.com ping statistics ---
5 packets transmitted, 0 received, 100% packet loss, time 4103ms

hostonly# apt-get update
Err:1 http://archive.ubuntu.com/ubuntu bionic InRelease                                                  
  Cannot initiate the connection to archive.ubuntu.com:80 (2001:67c:1560:8001::14). - connect (101: Network is unreachable)...
...

Erkenntnisse:

  • DNS scheint zu funktionieren
  • Zugriffe in’s Internet klappen nicht (=gut)
  • Aktualisierung des Betriebssystems klappt nicht (=schlecht)

Aktualisierung des Betriebssystems – Apt-Cacher-NG

Apt-Cacher-NG installieren

1

lxdhost# apt-get install apt-cacher-ng

Plattenbereich anlegen

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

lxdhost# vgdisplay
  --- Volume group ---
  VG Name               vg0
...
  Alloc PE / Size       18944 / 74.00 GiB
  Free  PE / Size       102987 / 402.29 GiB

lxdhost# lvcreate -L 10G -n apt-cacher-ng vg0
  Logical volume "apt-cacher-ng" created.

lxdhost# mkfs.ext4 /dev/mapper/vg0-apt--cacher--ng 
mke2fs 1.44.1 (24-Mar-2018)
Creating filesystem with 2621440 4k blocks and 655360 inodes
Filesystem UUID: ba3ee563-b887-4068-afe0-8c13356f95da
Superblock backups stored on blocks: 
  32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done 

lxdhost# mkdir /data
lxdhost# mkdir /data/apt-cacher-ng
lxdhost# echo >>/etc/fstab "/dev/vg0/apt-cacher-ng /data/apt-cacher-ng ext4  defaults 0 2"
lxdhost# mount /data/apt-cacher-ng

Neuen Plattenbereich nutzen

1
2
3
4
5
6
7
8

lxdhost# cd /var/cache
lxdhost# ls -ld apt-cacher-ng
drwxr-sr-x  3 apt-cacher-ng apt-cacher-ng 4096 Dec  7 09:33 apt-cacher-ng
lxdhost# getfacl apt-cacher-ng|(cd data; setfacl --restore=-)
lxdhost# systemctl stop apt-cacher-ng 
lxdhost# rm -rf apt-cacher-ng
lxdhost# ln -sf /data/apt-cacher-ng apt-cacher-ng
lxdhost# systemctl start apt-cacher-ng

Apt-Cacher-NG nutzen

1
2
3
4

hostonly# LXDHOST="$(ip addr|grep -o "inet [0-9.]*/"|grep -v 127.0.0.1|grep -o "[0-9.]*"|sed -e "s/[.][0-9]*$/.1/")"
hostonly# echo 'Acquire::http::Proxy "http://'${LXDHOST}':3142";' >>/etc/apt/apt.conf.d/01proxy
hostonly# cat /etc/apt/apt.conf.d/01proxy
Acquire::http::Proxy "http://10.2.130.1:3142";

Test

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

hostonly# apt-get update
Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 
Get:3 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [593 kB]
Get:4 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]       
...

hostonly# apt-get upgrade
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
The following packages will be upgraded:
  bind9-host dnsutils libbind9-160 libdns-export1100 libdns1100 libirs160 libisc-export169 libisc169 libisccc160 libisccfg160 liblwres160
...

Die Aktualisierung klappt!

Automatische Nutzung

/root/bin/apt-proxy.sh

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

#!/bin/sh

# port of apt-cacher-ng on the lxd host
PORT=3142
APT_PROXY_FILE="/etc/apt/apt.conf.d/01proxy"
LXDHOST=

for h in $(ip addr|grep -o "inet [0-9.]*/"|grep -v 127.0.0.1|grep -o "[0-9.]*"|sed -e "s/[.][0-9]*$/.1/"); do
  nc -z "${h}" "${PORT}" && { LXDHOST="${h}"; break; }
done

if [ -n "${LXDHOST}" ]; then
  echo >"${APT_PROXY_FILE}" "Acquire::http::Proxy \"http://${LXDHOST}:${PORT}\";"
else
  rm -f "${APT_PROXY_FILE}"
fi

/root/systemd/apt-proxy.service

1
2
3
4
5
6
7
8
9
10
11
12

[Unit]
Description=Apt proxy to apt-cacher-ng on lxd host
After=network.target

[Service]
ExecStart=/root/bin/apt-proxy.sh
TimeoutSec=0
StandardOutput=tty
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

Aktivieren

1
2
3

lxdhost# systemctl enable /root/systemd/apt-proxy.service
Created symlink /etc/systemd/system/multi-user.target.wants/apt-proxy.service → /root/systemd/apt-proxy.service.
Created symlink /etc/systemd/system/apt-proxy.service → /root/systemd/apt-proxy.service.

Test

1
2
3
4
5
6
7
8
9
10
11

lxdhost# reboot
...
lxdhost# systemctl status apt-proxy
● apt-proxy.service - Apt proxy to apt-cacher-ng on lxd host
   Loaded: loaded (/root/systemd/apt-proxy.service; enabled; vendor preset: enabled)
   Active: active (exited) since Sun 2019-12-08 07:10:09 CET; 12min ago
  Process: 217 ExecStart=/root/bin/apt-proxy.sh (code=exited, status=0/SUCCESS)
 Main PID: 217 (code=exited, status=0/SUCCESS)

Dec 08 07:10:09 hostonly systemd[1]: apt-proxy.service: Failed to reset devices.list: Operation not permitted
Dec 08 07:10:09 hostonly systemd[1]: Started Apt proxy to apt-cacher-ng on lxd host.

Weiterleitung

Klappt die Weiterleitung einer Netzwerkverbindung vom LXD-Host an einen Hostonly-Container?

Apache HTTPD installieren

1
2
3
4
5
6
7
8
9
10

hostonly# apt-get install apache2
hostonly# netstat -tan
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State      
tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN     
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN     
tcp        0      0 10.2.110.10:43552       10.2.110.1:3142         TIME_WAIT  
tcp        0      0 10.2.110.10:22          10.2.110.1:32976        ESTABLISHED
tcp6       0      0 :::22                   :::*                    LISTEN     
tcp6       0      0 :::80                   :::*                    LISTEN

Weiterleitung aktivieren

1
2

lxdhost# CONTAINER=hostonly
lxdhost# lxc config device add ${CONTAINER} http proxy listen=tcp:0.0.0.0:8080 connect=tcp:127.0.0.1:80

Testen

1

lxdhost# curl http://localhost:8080 # klappt

Test im Browser: http://lxdhost:8080/ – klappt!

Umschalten: Hostonly->NAT

Ausgangspunkt: Container mit Hostonly-Netz

1
2
3
4
5

hostonly# ping -c5 google.com
PING google.com (172.217.22.174) 56(84) bytes of data.

--- google.com ping statistics ---
5 packets transmitted, 0 received, 100% packet loss, time 4097ms

Umkonfigurieren

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

lxdhost# CONTAINER=hostonly
lxdhost# IPADDRESS_NAT=10.2.131.20
lxdhost# lxc network detach lxdhostonly "${CONTAINER}" eth0
lxdhost# lxc network attach lxdnat "${CONTAINER}" eth0
lxdhost# lxc config device set "${CONTAINER}" eth0 ipv4.address "${IPADDRESS_NAT}"
lxdhost# # /etc/hosts editieren
lxdhost# lxc stop hostonly
lxdhost# lxc start hostonly
lcdhost# lxc list
+-------------+---------+---------------------+------+------------+-----------+
|    NAME     |  STATE  |        IPV4         | IPV6 |    TYPE    | SNAPSHOTS |
+-------------+---------+---------------------+------+------------+-----------+
| tine20      | RUNNING | 10.2.131.20 (eth0)  |      | PERSISTENT | 0         |
+-------------+---------+---------------------+------+------------+-----------+
| hostonly    | RUNNING | 10.2.131.20 (eth0)  |      | PERSISTENT | 0         |
+-------------+---------+---------------------+------+------------+-----------+
| ubuntu-1804 | STOPPED |                     |      | PERSISTENT | 0         |
+-------------+---------+---------------------+------+------------+-----------+

Umschalten: NAT->Hostonly

Ausgangspunkt: Container mit NAT-Netz

1
2
3
4
5
6
7
8
9
10
11

hostonly# ping -c5 google.com
PING google.com (172.217.22.174) 56(84) bytes of data.
64 bytes from arn09s11-in-f14.1e100.net (172.217.22.174): icmp_seq=1 ttl=56 time=7.53 ms
64 bytes from arn09s11-in-f14.1e100.net (172.217.22.174): icmp_seq=2 ttl=56 time=7.73 ms
64 bytes from arn09s11-in-f14.1e100.net (172.217.22.174): icmp_seq=3 ttl=56 time=7.75 ms
64 bytes from arn09s11-in-f14.1e100.net (172.217.22.174): icmp_seq=4 ttl=56 time=7.76 ms
64 bytes from arn09s11-in-f14.1e100.net (172.217.22.174): icmp_seq=5 ttl=56 time=7.77 ms

--- google.com ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4007ms
rtt min/avg/max/mdev = 7.533/7.711/7.770/0.090 ms

Umkonfigurieren

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

lxdhost# CONTAINER=hostonly
lxdhost# IPADDRESS_HOSTONLY=10.2.130.20
lxdhost# lxc network detach lxdnat "${CONTAINER}" eth0
lxdhost# lxc network attach lxdhostonly "${CONTAINER}" eth0
lxdhost# lxc config device set "${CONTAINER}" eth0 ipv4.address "${IPADDRESS_HOSTONLY}"
lxdhost# # /etc/hosts editieren
lxdhost# lxc stop hostonly
lxdhost# lxc start hostonly
lcdhost# lxc list
+-------------+---------+---------------------+------+------------+-----------+
|    NAME     |  STATE  |        IPV4         | IPV6 |    TYPE    | SNAPSHOTS |
+-------------+---------+---------------------+------+------------+-----------+
| tine20      | RUNNING | 10.2.131.20 (eth0)  |      | PERSISTENT | 0         |
+-------------+---------+---------------------+------+------------+-----------+
| hostonly    | RUNNING | 10.2.130.20 (eth0)  |      | PERSISTENT | 0         |
+-------------+---------+---------------------+------+------------+-----------+
| ubuntu-1804 | STOPPED |                     |      | PERSISTENT | 0         |
+-------------+---------+---------------------+------+------------+-----------+