VMware, oh VMware.

Due to certain circumstances I've been attempting to forward ports from a NATed machine to a guest VM on that machine. What a hellacious experience.

The setup is as follows:

Router    <->  VM host    <->   VM guest
142.x.x.x     10.5.1.10       192.168.87.5
             192.168.87.1

Normally I would use bridged mode with VMware but the DHCP server on our subnet will only hand out one IP per port, as far as I can tell. So NAT it is! This means that the VM guest OS is behind 2 levels of NAT, how ugly.

The goal here is to give ssh access to the VM guest. The VM host has a few ports forwarded to it, 5900 for example. This is what I'd like to do:

     Router     <->    VM host     <->    VM guest
142.x.x.x:10900  -> 10.5.1.10:5900  -> 192.168.87.5:22

Very ugly, but that in and of itself should work. Should work, but it didn't. I was at a loss at this point. Luckily, I have a friend who works debugging network issues. We obtained network traces (dumps of all packets going through an interface) from the VM guest and VM host, and took a look at them. The first thing he noticed was a fishy MAC address. I checked the MAC of the VM host's eth0 and vmnet8 AND the MAC of the VM guest's eth0. Neither of them matched.

We tried again, but this time checked the ARP cache. AHA! There's the mysterious MAC address. It belongs to....192.168.87.2? What the hell is that? The VM host is 192.168.87.1 and the VM guest is 192.168.87.5. It's pingable, so it has to be _somewhere_. My buddy suggested checking the default gateway of the VM guest. Voila! 192.168.87.2.

The best we can determine VMware has some sort of internal router when using NAT mode. It's definently not accessible or configurable.

Nonetheless, a workaround was proposed. Simply changing the gateway from 192.168.87.2 (the internal VMware thingy) to 192.168.87.1 (the VM host) will get rid of that weird MAC. I did that.....and I could ssh to the VM guest! Hooray! But....I can't intiate any connections from the guest.

I originally thought that the NAT was done by the VM host. Now it seems more likely that it is done by the internal, non-configurable VMware NIC. If I actually setup a NAT between 10.5.1.0/24 and 192.168.87.0/24 I *think* it would work...but that's overkill for a temporary solution.

I decided to switch back to bridged mode and set the VM guest IP statically. This works fine for getting on the network and the Internet. I should be able to get the Powers in Charge to forward ports to them.

So what did I learn from all of this?

• Network traces are very useful.
• VMware NAT mode should only be used as a LAST resort.
• Politics suck.