Hetzner reported not allowed MAC addresses in subnet

[Andy Haack]

Hello

We had email from Hetzner network team asking us not to use MAC addresses 
belonging to the Virtual Machines belonging to the subnet.

We configured Xen server host as a router using this guide:

https://wiki.hetzner.de/index.php/Citrix_XenServer/en#Network_Configuration

 

After asking for additional details Hetzner support answered that 
normally, the network configuration of your hypervisor should only let packets exit the system with the MAC address of the real NIC.
But if you do not find the issue, you might try to block those outgoing packets using IPtables.

 

So our questions:

1) If anyone had this kind of problem with Hetzner or other dedicated server provider.
2) How you solved it? Is blocking using iptables the only way?

 

Thank you

[Andy Haack]

olamber768, thank you for solution! I didn't test it though as I already applied  solution from superuser.com. It worked for me

All credits to Alexey Degtyarev

Thanks to Hetzner network team also as they pointed to the solution, but I went to wrong direction trying to create second xenbr in the same network and assign VIFs to it. But It's not working because after virtual machine restart VIF number is regenerated.

In Hetzner you have two types of additional IP-addresses which you can use with a dedicated server: a single IPv4 & an IPv4 subnet. For each single IP you get a MAC-address and you must use that MAC on the network interface of a new VM instance. For each additional subnet you must setup a new network and setup routing between that network and the default network (associated with eth0) of the server.

In XenServer this can be done using Linux console:
 

xe network-create name-label="Additional network" name-description="46.xx.yy.zz/28"

Tip1: xapi0 bridge didn't show up on host until I assigned one of VMs to new network in Xen Center. Just tip that may save time if it happens to anyone else.


This will create a new network attached to a new bridge (xapi0 by default) in XenServer. Then assign the first usable IP-address of the network (according to it's netmask) to that bridge:
 

ip addr add 46.xx.yy.1/28 dev xapi0

Now you can add new VMs with autogenerated MACs attached to the newly created network instead of the default network. The traffic will be switched & routed inside XenServer.

Having that setup I got confirmation from Hetzner network team that only allowed MACs are visible on the switchport.

Tip2: You can verify there is no offending traffic on eth0 anymore from your side at host:
 

tcpdump -i eth0 -en|egrep -i 'mac1|mac2'

It all should be seen on xapi0 instead
 

tcpdump -i xapi0 -en|egrep -i 'mac1|mac2'

Where mac1,mac2 - MAC addresses reported by Hetzner as prohibited.

[Olivier Lambert]

There is multiple possibilities to stay in bridged more.

 

Solution 1: additional IP

I made this guide for XCP-ng, but it's the same for XenServer: https://xcp-ng.org/blog/2018/08/07/xcp-ng-on-hetzner-dedicated-hosts/

 

Main drawback: those IPs are linked to a host, so you can't live migrate the VM to another host while keeping this IP address.

 

Solution 2: using vSwitch

This is the best solution:

 

1. Create a vSwitch in your console and use the same VLAN on each host of your pool

2. Create a network in XenServer or XCP-ng, using this VLAN.

3. Purchase an IP or a range of IP for this VLAN

4. Now your VMs can use a VIF inside this VLAN and you can migrate them in all host within this network!

 

If you want more info, feel free to ask me or ask on XCP-ng forum, there is multiple users hosted at Hetzner there!

[Olivier Lambert]

Routing works, but sadly, as you may noticed, it's less straightforward (configuration needed). Also you aren't able to live migrate VMs to another host (well, you can but you'll need to change the IP address, which isn't really easy and will disrupt the service…).

 

Since their vSwitch feature, this is all a problem from the past :)