virtio-net and iPXE interoperability #1251
Comments
|
My virtio code has always had this bug with two directional transport queues. The OS sends empty buffers into the receive queue, and the device is expected to pop them from the queue as it writes data into it. The problem is, because they are ring buffers, the buffer looking full and the buffer looking empty both results in the start and end pointers being at the same location. The code works around this by returning the last buffer that its sent, so the rest of the virtio code thinks theres entries in the queue. This works on linux as linux give a bunch of buffers all in one notification. ipxe notifies once per buffer that it sends, and so the hack just drops each buffer as soon as it gets it :( |
I'm only half way through understanding the virtio code, but I'm getting there. So maybe I'm wrong for not knowing better yet, but wouldn't a naive buffer counter in I'm asking because this is one of the traditional solutions to this recurring problem with ring buffers. |
|
That is the traditional solution. I thought that wouldn't work because the value of read pointer is set by the OS and we just read it out of system memory. But as it turns out the virtio spec says that value is the unwrapped value. I have a patch for this almost ready to go. Im still messing with it though because its not enough to get iPXE to netboot for some reason. With the fix it gets an IP address now, but then just stops sending packets shortly after. |
|
Okay, looks like whats actually happening here, is when ipxe reads the MTU information from the device, it reads it as two bytes instead of as a single word. There's a few bugs in v86 that stop this from working. The first is that addr isn't actually passed to shim that adapts an 8bit read to a 16bit value on capability struct only registers the handler to the first byte. The second is that the shim function expects to be passed the actual address of the read, but the io function doesn't forward it. |
|
With #1278 netboot.xyz seems to work with virito-net. |
Yes, I can confirm that your PR fixes it, awesome! To test it I installed Debian 12 into a blank 2G So far I could not make my iSCSI-experiment boot with iPXE, I'll look into that tomorrow, I will then also try to NFS-boot with iPXE. I don't think my iSCSI-stuff has anything to do with virtio-net though, netboot.xyz being based on iPXE is proof enough. So, thank you very much! |
|
The problem with iPXE and iSCSI was that I had never initialized the MBR of the iSCSI disk. So installing |
Networking in iPXE (and its derivates like netboot.xyz) works with v86's
ne2knetwork device, but unfortunately fails withvirtio. To reproduce, simply download CD-ROMipxe.isoand boot it in v86 (I recommend to also look at netboot.xyz.iso, it's a pure network-based installer for a bunch of 32-bit OSes useable for v86).While the iPXE guest correctly detects v86's virtio-net device (and also prints a MAC address with the expected v86 prefix of
00:22:15), traffic does not work. Using tcpdump, I can see a DHCP DISCOVER request coming from the iPXE guest through the virtio-net device, and I can also see a DHCP OFFER reply directed back to the iPXE guest (which means the request was OK). For some reason, the iPXE guest does not accept the DHCP OFFER and sends the next DHCP DISCOVER after a short timeout, and then fails. Inbound guest traffic does not work correctly.The DHCP exchange works fine when using the iPXE guest with v86's
ne2knetwork device instead ofvirtio. Unfortunately,ne2kshows stability problems under heavy load and is also not reliably usable here.I'd like to ask for any ideas, starting points or debugging methods to understand and fix this interoperability issue in
virtio. The network stack in iPXE has a different origin than, for example, the Linux kernel, so it cannot be excluded that this is actually a bug in iPXE (which would also be a valuable finding).Details
latest-102-g2b162b1ewsnicandbellenottelling/websockproxy)iPXE console output using "ne2k" network device
tcpdump of iPXE using "ne2k" network device
iPXE console output using "virtio" network device
tcpdump of iPXE using "virtio" network device
The text was updated successfully, but these errors were encountered: