Announcement

Collapse
No announcement yet.

Hetzner Root-Server with Debian/KVM IPv4 and IPv6 Networking

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Hetzner Root-Server with Debian/KVM IPv4 and IPv6 Networking

    Introduction


    Since the IPv4 address space is almost completely used up, the future protocol is IPv6 which has been around for lots of years already and it is highly recommended to at least enable it for your machines and use it in dual-stack mode - so your service or website is available in the old IPv4 internet as well in its IPv6 counterpart.

    I've been working with IPv6 for 6 years already - started with an IPv6 Tunnel from Hurricane Electric (They offer free IPv6 certification by the way!) over a SixXS Tunnel to native IPv6 with the latest KabelDeutschland modem firmware update. Also some years ago i started to use dual-stack IP for the first customer servers (mostly single purpose Root-servers or vServers from Hetzner or JiffyBox/Domainfactory). Later i'm going to migrate most smaller virtual customer machines to my larger Root-server machines which i can maintain better than any single vServer.

    The largest problem i encountered was not setting up dual-stack for the Server itself - that was the easy part - but to enable it also for every single virtual machine, so that the customers servers had their own IPv4 and IPv6 address - globally routed of course.

    This was the largest problem because i did not find any complete tutorials - so this is why i wrote this one. I think there are a lot more people around who are searching for the same answers. So here they are.



    Limited to Debian and KVM?


    Of course, not! You can use whatever distribution or virtualization technology you want. I chose Debian because i've been working with it since Woody and Etch and because i'm using KVM for my Root-Servers due to their damn good virtualization performance and good direct kernel integration.

    Besides that you may need to adapt certain commands or configurations to fit your distribution or virtualization technology but i try to explain it as generic as possible.




    What about using libvirt itself?


    This is also possible and there is a very neat solution. Check out this german tutorial available here.

    What do i normally get from Hetzner?


    Normally you get a single IPv4 address and an /64 IPv6 Subnet with more than enough addresses. This is 18.446.744.073.709.551.616 host addresses - have fun using them all!

    For this setup i chose another /29 IPv4 Subnet (with 6 host adresses when using it as a /29 subnet or with 8 host addresses when using all single addresses) to give every customer it's own IPv4 address.

    For explanation here is a small list of what addresses we have now:
    140.194.250.23 - this is the single IPv4 for the Root-server. 56.23.34.56/29 - this is the additional /29 Subnet or if you need to see all addresses it would read like this 56.23.34.56 - 56.23.34.63.

    Besides that we have our single IPv6 Subnet. I explicitly made single bold because most tutorials on how to set up Hetzner IPv6 Networking is about ordering a second IPv6 Subnet - which is not necessary (anymore). The Subnet is this one 2a01:4f8:212:2122::/64. I won't explain how to read IPv6 addresses here - there are much better tutorials for this out there. Setup all the networks!



    Setup the KVM Host (or physical machine)


    At first let us setup the Host. I will post the complete configuration here with inline comments and some explanation afterwards, so experienced sysadmins can just use my configuration without reading all the stuff afterwards.

    Be sure to install the bridge-utils package beforehand, otherwise we cannot setup the VM-Bridge.

    Code:
     # Loopback device:
    auto lo
    iface lo inet loopback
    
    # device: eth0
    auto eth0
    iface eth0 inet static
      address   140.194.250.23 # Single Host IPv4 address
      netmask   255.255.255.255
      gateway   140.194.250.4 # The Hetzner Gateway IP
      pointopoint 140.194.250.4
    
    iface eth0 inet6 static
      address 2a01:4f8:212:2122::2 # We gave our Host a single IPv6
      netmask 128 # /128 because our Bridge has the whole /64 Subnet!
      gateway fe80::1 # The Hetzner IPv6 Gateway
    
    # VM-Bridge
    auto br0
    iface br0 inet static
      address 140.194.250.23 # Our Host will be the Gateway for all VM's
      netmask 255.255.255.255
      bridge_stp off # Some bridge configuration
      bridge_ports none
      bridge_fd 0
      bridge_maxwait 0
    
      # Add all single IP's from your /29 subnet
      up route add -host 56.23.34.56 dev br0
      up route add -host 56.23.34.57 dev br0
      up route add -host 56.23.34.58 dev br0
      up route add -host 56.23.34.59 dev br0
      up route add -host 56.23.34.60 dev br0
      up route add -host 56.23.34.61 dev br0
      up route add -host 56.23.34.62 dev br0
      up route add -host 56.23.34.63 dev br0
    
    iface br0 inet6 static
      address 2a01:4f8:201:2122::2 # Our Host again as the Gateway
      netmask 64 # Here we use the /64 subnet!

    What we did here besides setting up the loopback device is to configure the hosts eth0 device first because we want to be able to SSH into it later too. This configuration is straight forward and normally already been done by the installimage command from the rescue system. The only thing to mention here is the /32 netmask. Normally we are in a subnet with our neighbors from our rack - so mostly some kind of 255.255.255.192 or something. We don't want to talk to our neighbors so we set the /32 subnet. Also the pointopoint directive is important and tells our host to send all requests to the Hetzner IPv4 Gateway - even if we want to talk to our neighbors later.

    The next configuration part is our static IPv6 config. This is quite easy. We just need to choose a single IPv6 address for our host. Important here is to choose a /128 subnet (it's like a 255.255.255.255). I'll explain it a bit later why. And also we add the Hetzner IPv6 Gateway here which is just a link-local address.

    Now the interesting part. The VM-Bridge where we will add all our VM's later.

    At first set up the IPv4 part by adding the usual bridge config. The bridge address will be our Host-IP so our VM's can send their stuff trough the Host to the internet. After some basic bridge config directives we are adding our ordered /29 subnet. We could just add it as a subnet in a single line but we would loose 2 IP addresses because a subnet always has a network address (56.23.34.56) and a broadcast address (56.23.34.63) and we would have only 6 usable host addresses. This is why we add every single IP as a single address and we can use all 8 IP's for customer machines. So just add all single routes to the bridge, so the bridge knows where to route the things to.

    In the last step we add our globally routed(!) IPv6 Subnet - now as the desired /64 subnet. We did'nt have used /64 for our Host because our Host will be part of the IPv6 Network and it will not consume the whole subnet for itself. Both configurations are crucial for a working Guest-IPv6! Again our Host's IPv6 addresses is the address for the bridge and at the same time the gateway address for our virtual machines later.

    Now we also need to change some kernel parameters for IP forwarding.

    Code:
    net.ipv4.ip_forward=1
    net.ipv4.conf.eth0.send_redirects=0
    net.ipv6.conf.all.forwarding=1
    You can set them on-the-fly like this sysctl -w net.ipv6.conf.all.forwarding=1 or add a config file in /etc/sysctl.d/99-networking.conf and add all three parameters in there by just copy and pasting them.

    What we configure here is enabling IPv4 and IPv6 forwarding which is necessary for our host to act as the gateway for our guests. The .send_redirects=0 parameter tells our host not to send icmp redirect messages to our guests to tell them to directly use the Hetzner gateway. This is not possible because the Hetzner gateway doesn't know the MAC-adddresses of our guests. More about this later in Why do i need to choose my Host as the IPv4/v6 Gateway?

    Now save this file and restart or save this file and set the parameters on-the-fly without rebooting.

    Now let's continue with the guests - which is much easier.
    Setting up the guests



    Here is the complete guest configuration and the explanation afterwards.

    Code:
    # Loopback device:
    auto lo
    iface lo inet loopback
    
    # device: eth0
    auto eth0
    iface eth0 inet static
        address 56.23.34.56 # Any IP of our /29 subnet
        netmask 255.255.255.255
        gateway 140.194.250.23 # Our Host machine will do the job!
        pointopoint 140.194.250.23
    
    iface eth0 inet6 static
        address 2a01:4f8:212:2122:56:23:34:56 # Any IPv6 of our /64 subnet
        netmask 64 # The same subnet as the bridge!
        gateway 2a01:4f8:212:2122::2 # Our Host is our gateway

    You can see that the guest config is much easier but you still have to take care of some things. At first, make sure that your guest's NIC is connected to the bridge br0 we configured on the host earlier.

    Now we just configure IPv4 and IPv6 again. For IPv4 you can choose any IP of your /29 subnet and add the restrictive /32 subnet again. Remember that we don't like our neighbors that much!
    For the gateway we will add our Host machine as well as for the route which is configured by the pointopoint directive (Take care of the single t in the middle!).

    Now the IPv6 guest configuration is straight forward as well. Choose an IP out of those smallnumber of possible host addresses. I prefer using the IPv4 as the second part of the IPv6 so i can remember it easier later but you can literally choose any IP you want as long as the prefix remains the same!

    Then add the same /64 netmask and our Host's IPv6 address as the gateway.



    Why do i need to choose my Host as the IPv4/v6 Gateway?


    This is because Hetzner uses a fully routed IP network and assigns IP's to the known MAC-addresses of the physical machines. So while Hetzner will never know about our guests MAC-Addresses, we need to go this way.




    Final words and references



    I hope this post helps you setting up your machine. If there are any questions, leave them in the comments and i'll try to cover them up in an update.

    For initial research i used those sites provided by Hetzner:

    Original Article: https://www.sysorchestra.com/hetzner...v6-networking/
    Last edited by DaMysterious; 08-03-2019, 12:31 PM.

  • #2
    Debian network configuration for Proxmox VE server running on a Hetzner host

    Code:
    # /etc/network/interfaces
    #
    auto lo
    iface lo inet loopback
    
    # device: eth0
    iface eth0 inet manual
    
    # IPv4 bridge
    # (connect ONLY your firewall/router KVM instance here, this is the WAN device!)
    auto vmbr0
    iface vmbr0 inet static
            # Hetzner primary WAN IP
            address   176.9.xxx.xxx
    
            # Hetzner primary WAN IP broadcast address
            broadcast 176.9.xxx.xyz
    
            # This netmask needs to have all bits set
            netmask   255.255.255.255
    
            # Our gateway is reachable via Point-to-Point tunneling
            # put the Hetzner gateway IP address here twice
            pointopoint 176.9.xxx.yyy
            gateway   176.9.xxx.yyy
    
            # Virtual bridge settings
            # this one is bridging physical eth0 interface
            bridge_ports eth0
            bridge_stp off
            bridge_fd 0
    
            # add a static route through the Hetzner gateway IP
            # for the subnet our primary IP belongs to
            up route add -net 176.9.xxx.zyx netmask 255.255.255.224 gw 176.9.xxx.yyy vmbr0
    
            # Add routing for up to 4 dedicated IP's we get from Hetzner
            # You need to 
            up ip route add 176.9.xxx.xx1/32 dev vmbr0
            up ip route add 176.9.xxx.xx2/32 dev vmbr0
            up ip route add 176.9.xxx.xx3/32 dev vmbr0
            up ip route add 176.9.xxx.xx4/32 dev vmbr0
    
            # Assure local routing of private IPv4 IP's from our
            # Proxmox host via our firewall's WAN port
            up ip route add 192.168.0.0/16 via 176.9.xxx.xx1 dev vmbr0
            up ip route add 172.16.0.0/12 via 176.9.xxx.xx1 dev vmbr0
            up ip route add 10.0.0.0/8 via 176.9.xxx.xx1 dev vmbr0
    
    # IPv6 bridge
    # (connect ONLY your firewall/router KVM instance here, this is the WAN device!)
    iface vmbr0 inet6 static
            address 2a01:4f8:151:XXX::3
            netmask 64
            up ip -6 route add 2a01:4f8:151:XXX::1 dev vmbr0
            up ip -6 route add default via 2a01:4f8:151:XXX::1
    
    # Virtual switch for DMZ
    # (connect your firewall/router KVM instance and private DMZ hosts here)
    auto vmbr1
    iface vmbr1 inet manual
            bridge_ports none
            bridge_stp off
            bridge_fd 0
    
    # Virtual switch for Private LAN
    # (connect your firewall/router KVM instance and private hosts here)
    auto vmbr2
    iface vmbr2 inet manual
            bridge_ports none
            bridge_stp off
            bridge_fd 0
    
    # Virtual switch for Test Data Center
    # (connect your firewall/router KVM instance and private hosts here)
    auto vmbr3
    iface vmbr3 inet manual
            bridge_ports none
            bridge_stp off
            bridge_fd 0

    Comment

    Working...
    X