Saturday, August 04, 2007

About Network I/O, CPU cycles, and Multicore architectures

With the advent of 10 Gig Ethernet, network I/O has become unbelievably fast. This poses substantial challenges for kernel designers, because the erstwhile thinking that I/O is not CPU intensive does not apply any more. With 10 GigE, you can easily exhaust all the CPU in your box just transmitting or receiving raw packets without doing anything substantial.

This poses an interesting dillema, (i) What is the best network I/O you can perform with your box having 10 GigE, and (ii) Given that an application is running in the background and is the producer/ consumer of network I/O, how do you allocate your limited CPU cycles between network I/O and your application. In my PhD dissertation, we tried to solve the above on a macro scale. We developed a stochastic model to predict the best rate at which network I/O may be performed given the application workload on our system. Our work presented at PfldNet 2007 is availeble here.

With the advent of multicore chipsets such as the Intel quad-core Xeon and AMD quad-core Barcelona besides the Sun Microsystems Niagara chip which has existed for almost two years, there is an added dimension to this problem, how to cooperate the application and network I/O across cores. And with applications becoming increasingly multithreaded, this problem is definitely trickier. Since the interplay between CPU cycles, cache misses, memory latencies etc. becomes even more interesting.

I read a recent paper by authors from Virginia Tech and Argonne National Lab illustrating how a single threaded application can be bound to a core to get higher network I/O throughput. It is a very interesting read, would strongly recommend it.

2 comments:

Anonymous said...

Hi Amitabha,

You might also want to check out our paper at HSN Infocom 2007 on this where we have presented results at 10 and 20Gbps rates for multi-core, Multi-ic/ i.e. Multi-rail networks.

Here is a link:
http://www.evl.uic.edu/files/pdf/HPN2007_MuliRail.pdf


Regards,
Venkat

Unknown said...

Thanks for letting me know, I will read it today. Nice to see you reading this blog :)