These methods are doing fundamentally different things. To understand why, you need to understand the layered model of networking. For our purposes here, layers 1, 2 and 3 are important:
- Layer 1 is the physical layer - this specifies things like what cables you can use, what voltage / current patterns represent 1s and 0s on that cable, how the devices at each end of a cable negotiate what bit rate they operate at and so on.
- Layer 2 is the link layer - this specifies what language things at each end of a cable talk to each other. Ethernet devices at this layer have things like frames and MAC addresses.
- Layer 3 is the network layer - this specifies how devices use a direct layer 2 link to another device to reach a third device that they can't reach directly at layer 2. Devices at this layer have IP addresses and routing tables.
MACVLAN / MACVTAP
MACVLAN creates a virtual layer 2 or link layer device, with its own MAC address, which shares the layer 1 or physical layer with an existing device. The most obviously understandable case is where you have an Ethernet device plugged into a network and you create a MACVLAN device based on that Ethernet device; now you have two Ethernet "devices" with different MAC addresses but which both transmit their frames on the same cable. I'll talk about MACVTAP a bit further down.
MACVLAN interfaces can interact in several different ways with the existing Ethernet interface, in particular when a frame appears on one of the interfaces which is addresses to the other one:
- In private mode, the frame is thrown away; it is not possible for the two interfaces to communicate with each other, only with external devices.
- In vepa mode, the frame is sent over the physical layer like any other frame. If you have the device plugged into a switch that is clever enough to spot that the frame then needs to be send back down the same port it arrived on, then it will be received by the same physical layer that sent it and then layer 2 will use the MAC to send it to the intended network interface.
- In bridge mode, when a frame appears on one device, it is checked to see if it is intended for the other and if so, it is sent there without going through layer 1.
- There are also a couple of more obscure modes.
Note that MACVLAN interfaces have an important restriction: They're not capable of address learning. So you can't bridge a MACVLAN interface to a second physical device and expect to be able to reach that second physical device over the first one. This works with the original Ethernet interface but not with a MACVLAN interface attached to it.
A TAP interface is also a new virtual layer 2 device but with no layer 1 attached to it. Instead, a program can get a file descriptor representing the physical layer. It can then write raw Ethernet frame data into that file descriptor and the kernel will treat it like any other Ethernet packet it receives on a real physical interface.
The big thing about TAP interfaces is that the physical layer is in user mode; any bit of software with the appropriate permissions can generate Ethernet frames in any way it likes and shove them into something the kernel treats the same as a real physical interface. This makes them very useful for things like VPNs and tunnelling; you can write whatever sort of tunnelling software you like in user space and there is no need to meddle in kernel space to get the frames into the networking stack, you just create a TAP device and write the frames into its file descriptor.
TUN devices are just like TAP devices except they operate at layer 3 instead of layer 2 and the user mode software has to write raw IP packets into the file descriptor instead of raw Ethernet frames.
Going back to MACVTAP devices, these are sort of mix-up between MACVLAN and TAP interfaces. Like TAP interfaces, a user-mode program can get a file descriptor and write raw Ethernet frames into it. Like a MACVLAN interface, those frames are then sent over the physical layer of a real Ethernet device. This allows you to easily adapt software that's written to use TAP devices to use a MACVLAN device instead.
This is conceptually similar to TUN/TAP networking but has a more developed control plane (so the user-mode software using it can configure the interface more flexibly) and a more optimised data plane (so you can move data through the virtual network device more efficiently).
All of these do similar things but have slightly different capabilities. All of them can be used to connect a VM to an Ethernet network:
- A virtualisation product can take Ethernet frames from the guest and write them into the file descriptor for a TAP device. That TAP device can be assigned its own IP address by the host, or it can slaved to a bridge along with an Ethernet interface to share the host's IP address, or iptables can be configured to forward traffic on it using NAT.
- A virtualisation product can that Ethernet frames from the guest and write them into the file descriptor for a MACVTAP device; these then get transmitted directly on an Ethernet device's physical layer, effectively giving the VM a "real" Ethernet device (though note that it is possible to create MACVLAN / MACVTAP devices for other types of network interfaces such as bridges).
- A virtualisation product can connect a virtio driver in the guest to the virtio driver in the host for very efficient networking.