<herbert> rusty: has the tun gso patch been merged yet?
<rusty> herbert: no :(
<herbert> herbert: ok, was there any negative feedback, or no feedback at all :)
<herbert> rutsy: hmm s/herbert/rusty/ :)
<rusty> herbert: Positive, just too late. We also discussed a ringbuffer interface.
<herbert> rusty: too late for 2.6.25 or 2.6.26?
<rusty> 25
<herbert> rusty: ok perhaps it's time to do another push for 26 then
<rusty> herbert: yeah, thanks for the push... I'll also try to get a virtio ring implementation for tap... tomorrow is good for that.
<rusty> (Grr cut my finger making dinner, hard to type...)
<herbert> rusty: cool, i'll let you to enjoy your dinner then :)
<rusty> Thanks :)
<herbert> rusty: when you get around to it, the negotiation between tun/virtio-back/virtio-net needs to include the actual protocols supported rather than just GSO
<herbert> e.g., we only support TSO right now and not UFO or anything else in software
<rusty> herbert: ah... I hadn't realised. I might have to fix that somehow.
<herbert> otherwise the frontend may generate packets that the backend can only drop if the hardware also doesn't support the required GSO
<herbert> alternatively you can write the code to do UFO in software :)
<rusty> We currently have one feature option we tell guest: that we support GSO (ie. everything)
<herbert> that's fine if all backends always support all protocols
<herbert> which isn't currently the case
<rusty> Yes, I didn't know that... ok, so do we do TCPv6?
<herbert> hmm, even if we did implement UFO we may one day have a new protocol that we don't support
<herbert> yes we do
<herbert> TCPv4/TCPv6 are the only two protocols supported in software right now
<rusty> herbert: ECN ok? Well I can implement UFO, but I'll still need a feature bit to say TSO only for older kernels, or just not use GSO at all for them.
<herbert> rusty: see my comment above re new protocols, i think even if we had UFO it would be best if the protocols were included in the negotiation
<herbert> otherwise we'd be in a sticky situation should some new protocol crop up
<herbert> e.g., DCCP or SCTP
<rusty> herbert: yeah... the current GSO means UFO, TSO, ECN and TSO6. We can't change that, but I'll introduce them as individual features as well.
<rusty> (We don't actually use this in the kernel yet, since tap stuff not in)
<herbert> rusty: TSO/ECN/TSO6 is all OK, UFO isn't actually supported by the soft GSO stack
<herbert> e.g., if you hardware doesn't support it the packets go to the bit bucket
<rusty> herbert: yeah, that sucks. Think it'd be a pain to implement?
<herbert> rusty: should be pretty straightforward
<rusty> herbert: it'll be fun to learn more about that code anyway, so I might try to patch it.
<herbert> rusty: sure, all you need to do is copy the code in net/ipv4/tcp.c to udp.c and modify accordingly
<herbert> should be self-explanatory
<herbert> rusty: that'll handle both IPv4/IPv6
<herbert> rusty: by code i mean tcp_tso_segment etc.
<rusty> herbert: OK, I'll do that tomorrow. To test, I'll want to sendfile onto a UDP socket, right?
<herbert> rusty: yeah, you can confirm with tcpdump to see whether it's actually generating large packets
<herbert> netperf should do the trick too i think, with the right parameters
<herbert> rusty: oh and of course a tcpdump on the remote end to make sure that the packets did make it out :)
<rusty> herbert: I'll have debug comments in my code, but I have a sendfile test already for the previous stuff so easy to change it to UDP
<rusty> herbert: yeah :)
<herbert> rusty: great
<rusty> herbert: my plan for tap is to make it understand the virtio rings... they're designed to be an efficient inter-guest ABI for ringbuffers, makes sense for tap too.
<herbert> rusty: cool
<herbert> rusty: btw did tell you that my benchmark shows kvm with virtio is already faster than xen?
<rusty> herbert: wow, no! Cool, did you tell aliguori?
<herbert> that's with your tun patches and a user-space frontend
<herbert> i told cdub, i'm not sure whether aliguori knows
<rusty> herbert: hey, are you going to be at the virt minisummit BTW?
<herbert> rusty: i was surprised that the user-space backend is able to match xen with its zero-copy on send
<herbert> rusty: yeah i'll be there
<rusty> herbert: awesome, think you can throw together some results to present? Or are you already ahead of me?
<herbert> rusty: i will once i do the backend properly, my earlier results were done with a hacked frontend that pretended the backend did tso :)
<herbert> you know it actually doesn't matter whether the backend does tso
<herbert> for a guest-to-host test
<rusty> Yeah,
* Jes_wk spots volunteers for speaking at the mini summit
<Jes_wk> herbert: tag! we'll need a small presentation on that! :)
* jsgf has quit (Ping timeout: 480 seconds)
<aliguori> herbert, wow! does it beat xen for both tx and rx?
<aliguori> or just for tx?
<herbert> aliguori: kvm is just slightly ahead on guest => host but significantly better on host => guest
<herbert> fwiw the two directions are about the same under kvm
<herbert> while with xen guest => host is significantly faster compared to host => guest
<herbert> in either case tso with a 1500 mtu going from guest to host gives between 50%-60% of the native lo performance
<herbert> s/performance/throughput/
<herbert> this is with one single cpu maxed out with netperf
<aliguori> wow, that's impressive
<aliguori> I'm honestly quite surprised
<herbert> the most surprising fact for me is that copying with kvm doesn't seem that bad
<herbert> but one thing to keep in mind is that with xen's model the backend is really in user-space anyway
<aliguori> herbert, why's that?
<herbert> well it's running in dom0 and it has to perform mmu ops to actually map/unmap guest memory
<herbert> as opposed to direct access which would be available if we had a kvm kernel backend
<aliguori> ah, I see what you mean :-)
<herbert> so what i'm saying is that given the current model kvm and xen are really operating on a level playing field
<herbert> except for one extra copy in kvm's case
<aliguori> on the rx path
<herbert> but in theory kvm could be even better
<aliguori> so the ring queue should eliminate that copy nicely
<herbert> in the guest => host direction, because xen uses zero-copy there
<herbert> aliguori: that's what i thought too, but given the current numbers i'm no longer sure :)
<herbert> because xen already does zero-copy yet it isn't faster than kvm i'm beginning to think that the mmu overhead associated with user-space is the real killer
<herbert> i suppose we'll have to wait for a kernel backend to prove/disprove this
<aliguori> hrm, i thought rusty's tun/tap patches gave zero-copy on guest->host
<aliguori> i thought when gso was in use, it did get_user_pages() to setup the skb
<herbert> aliguori: i haven't turned that on yet :)
<aliguori> ah :-)
<herbert> so my tests aren't with that
<herbert> i used a hacked gso version
<herbert> which basically is equivalent to just setting a large mtu
<herbert> but there is definitely copying
<rusty> aliguori, herbert: can't you just accept that my sheer brilliance bleeds into benchmarks?