The power of video clips

Slightly off-topic here... owing to my spare time in the holiday season. I am impressed at how predominant video clips are becoming over the internet... the success of You Tube being just one example. Advertising is increasingly shifting to using video clips over the internet. Look through YouTube and you will find ad clips from well recognized companies including Sony Playstation. As an example, here is Sun Microsystems' video clip advertising its 10 Gig Ethernet adapters... something on topic.



In case you are a video producer or amateur and would like to explore ways of making money over the Internet using your video/ movie clips, here is an excellent article by a well-known independent cinema veteran.

Btw, VMware released their ESX 3.5 with support for Neterion's cards with a claim of delivering 10 Gbps from a virtualized domain with near native OS CPU utilization. Anybody able to verify this?

Happy Holidays!!!

Google's 10- GigE switch

Just read an article from Nyquist Capital about Google designing its own 10 GigE switches. Its interesting how the authors traced the source of thousand of SFP+ components to Google to determine this. The article discusses how the strategy is very similar to Google designing its own servers and compute farms from basic components as opposed to buying servers from companies like IBM, HP SUN etc.. The article also mentions Google adding 5K+ 10GigE ports a month to manage its 500,000+ compute nodes.

Thats a ton of money being saved, given the cost of 10GigE switches in the market today. Which brings us to the question, what is so proprietary about a switch that vendors like Cisco, Juniper, Woven etc., can sell them at a huge price and a huge margins. Does IOS-X from Cisco have some patented stuff that open source software cannot implement or don't have. I would love to see the above Google technique revolutionize the switch market and motivate some startup to design a switch fabric based on commodity hardware.

About the optical world

A bit of shift to the optical technologies in my blog today, with LightReading's Optical Expo going on in Dallas. One interesting thing that strikes me is that today optical stocks are soaring, both Ciena (CIEN) and Infinera (INFN) have > 5% gains with no extraordinary news from either company. May be Optical Expo is delivering a strong message which is going unnoticed. Any coments?

From news reports, 100 Gig technologies are getting quite some attention, with AT&T, Verizon both complaining that the standards may not be ready fast enough. Interesting to note that they have been observing / are anticipating tremendous bandwidth flow in their pipes. There has been much talk about how much bandwidth demand video can hog. Certainly there is extraordinary promise. Netflix is already hosting video and provides download services.

I think more than anything else, it depends on how the access world picks up. Give people optical fiber to their home, give them a 1Gbps bandwidth pipe, and see how much video and bandwidth they will use. The potential is extraordinary.

So my appeal to AT&T, Verizon and the likes. Focus on access. Get all PON technologies rolling fast. Invest your money in digging those trenches to the apartments and homes and you will be extraordinarily rewarded.

By the way, this blog has had double views over the last two days. Is it just me dreaming about the success of Optical Expo?

Teknovus gets $28 million funding

Teknovus, one of the leading companies making EPON chipsets, yesterday announced $28 million funding from a group of equity firms. New investors in Teknovus include Lightspeed Venture Partners and the Galleon Crossover Fund, Vision Venture Capital from Taiwan and Technology Alliance Investment and Itochu Technology Ventures from Japan. Teknovus’ existing investors also participated in this round, including USVP, Partech, Focus, Samsung, and Mitsubishi.

Our blog turns two years

We turn two years today. For those interested, our first post is here.


It has been a great experience following trends in this very hot and growing field. Just looking back at the progress, Ethernet PON is now widely deployed with 10 Gig standardization coming up, 40/100 Gig standardization is going on, 10 Gig Ethernet adapters are close to entering the volume market, and high speed I/O is now being considered more and more important by chip and server vendors.

Thanks to all readers, all those to supported the blog sending comments and emails. Hope you continue to find this blog useful in the future.

Also we will soon come up with a new layout and a poll. Please help us understand your needs better by participating in the poll. Thanks again.

EPON to be deployed around Paris, big win for the open access model

Sipperec , which provides power and telecom services on behalf of 86 municipalities around Paris today announced a fiber-to-the-premises (FTTP) network called Sequantic, which will build on its fiber backbone. This will provide connectivity to buildings targetting about 6,500 enterprises in 147 business areas located in the municipalities. Sipperec will then sell capacity on the network to service providers, so adopting the "open access" model. This is a major boost to this model promoting neutrality to the network provider also adopted by UTOPIA in Utah and the City of Amsterdam.

At UC Davis Networks Research group, we proposed advanced network protocols that may be useful to promote the "open access" model. This protocol aims to guarantee established Service Level Agreements (SLAs) to both users and service providers simultaneously. Our work was published in the IEEE Journal of Selected Areas in Communication, a copy available here.

Matisse Networks announces $45 million Series C VC funding

Matisse Networks which is building Ethernet over Optical Burst Switch technology today announced that it had closed $45 million Series C funding from Merrill Lynch PCG and other VCs. Matisse's EtherBurst switch is one of the few companies in the Optical Burst Switching (OBS) Domain.

OBS attempts to bridge the gap between optical circuit switching technology (which is widely deployed now) and packet switching which is non-existent in the optical domain because optical components cannot match electronic speeds. OBS establishes dedicated circuits for short durations of time by sending a signal in advance in the control domain which reconfigures the network to form the circuit. Data (which is called an optical burst) is then transmitted over this circuit. The network can be then reconfigured to form a different circuit along which a different optical burst may be sent. While OBS is a neat concept and has existed in academic research since the late nineties, it has not been commercially exploited other than some prototype demonstrations.

I am curious as to how the EtherBurst switches Ethernet 1500 MTU size packets over an OBS network since the granularity of a burst will be much much (1000x) larger than an Ethernet packet. May be they aggregate packets to a destination together and then transfer them. But then latency suffers. It would be great if Matisse came up with some performance characteristics such as throughput delivered, latency, etc.

Cisco launches 802.11n Wireless Access Point

Yesterday Cisco launched its first 802.11n-based Ethernet access point (AP), the Aironet 1250. This is based on the 802.11n draft 2 which is currently being standardized and will offer 300 Mbps of wireless access connectivity. Moreover, the access point can be powered by wired Ethernet which avoids having power installations at the deployment point. This product is priced at $1299.

I also read a recent research group report which projected that by 2010 most of wired ethernet in LANs will be replaced by 802.11n technology. I am not so sure. While wireless has the advantage of ubiquity, it still lacks consistant steady performance and security. For example, I see a marked difference between viewing video over my home 802.11g network and with a direct ethernet connection. And the issue is not of bandwidth, 54 Mbps is far more than what my DSL provides.

I am also sure that wired ethernet will be dominant in corporate LANs. Dont think a business can imagine it crucial data floating in the air :)

I have also been having some connectivity issues in my home wireless network, and would be glad if any of you could leave a comment on how to solve it. I live in a densely populated area. There are at least 20 wireless networks around my area. When I switch on my laptop it often has problems connecting to my wireless network. It takes quite a bit of time, sometimes 15-20 minutes before I get connectivity. I suspect it is because of running out of wireless channels in the 801.11g frquency domain. Any thoughts?

From Hot Interconnects

I could find some time to attend the second day today at Hot Interconnects at Stanford. Here are some notes. Although some papers may not be directly relate dto the discussions in this blog, I decided to include them anyways.

Paper 1:

A Memory-Balanced Linear Pipeline Architecture for Trie-based IP Lookup on FPGA, USC

The trie data structure is used for IP lookup. Efficient pipelining is possible for IP lookup. In current approaches there exists unbalanced memory distribution across stages. Their solution is to improve pipelining in trie lookups.

Paper 2:
Building a RCP (Rate Control Protocol) Test Network, Stanford
They build an experimental network to support RCP using NetFPGA implementation and modifying Linux by adding a shim layer inbetween to support RCP.

Paper 3:
ElephantTrap: A low cost device for identifying large flows, Stanford
ElephantTrap identifies and measures big flows in a network using random sampling of packets and algorithms to have select the large flows in the network.

Intel invited talk titled On-Die Interconnect and Other Challenges for Chip-Level Multi-Processing

Chips will become increasingly multicore because performance tapes of with increase in transistors in a single core. However, gain from parallelism is lost in communication overhead. Speaker advocated on-chip high speed interconnect which ties cache, memory, network I/O etc to processors. Also advocated ring based topology for interconnect. He talked briefly about Intel terascale processor with 80 cores.

Paper 4:
An Analysis of 10-Gigabit Ethernet Protocol Stacks in Multicore
Environments, Virginia Tech

They consider the interaction of protocols with multicore architectures. They consider TCP/IP and iWarp protocols and the interaction between application, TCP/IP stack and network I/O. They show that if application is scheduled on different core than where interrupt processing is (but in same socket), it leads to best performance as CPU load is distributed across cores and it leads to lesser cache misses too. Did not consider MSI/MSI-X or multithreaded applications.

Paper 5:
Assessing the Ability of Computation/Communication Overlap and Communication Progress in Modern Interconnects, Queen's University

They consider interplay between processing and I/O so as to have minimum stalling cycles. They consider various MPI based applications over different interconnects: Infiniband, Myricom and 10 Gig Ethernet

Paper 6:
Performance Analysis and Evaluation of Mellanox ConnectX InfiniBand
Architecture with Multi-Core Platforms, Ohio State Univ

They demonstrate how ConnectX (the latest generation of Infiniband technology) performs on multicore architectures.

Paper 7:
Memory Management Strategies for Data Serving with RDMA, Ohio Supercomputer Center

They demonstrate that virtual to physical memory translation is a considerable overhead in high-speed networks. They demonstrate how to optimize memory registeration to achieve better performance in RDMA

Paper 8:
Reducing the Impact of the Memory Wall for I/O Using Cache Injection, Universiy of New Mexico

Cache Injection is a technique by which data is transferred directly fron device to L2/L3 cache. They show the tradeoff between cache injection and prefetching and propose an algorithm to perform efficient cache injection for network I/O,

UltraSparc T2 with integrated Network I/O

Although I do not follow hot chips, and have never blogged about microprocessors, I would love to talk about Sun's UltraSparc T2 launched last week, the first microprocessor ever to support an integrated network I/O support. Yes, UltraSparc T2 which I would prefer to call by its moniker Niagara 2, supports multithreaded dual 10 Gig Ethernet. From product brochures, this chip seems to be a true microsystem, just connect this to RAM and you get it running.

Although Sun has only mentioned some CPU intensive benchmarks such as SPEC int and SPEC float, may be they come up with some network I/O oriented benchmark numbers as well.

Also Sun is releasing all design documents under the GPL license. Wonder what that means? Can someone take up all the technical documentation and fab a chip independently? How does open sourcing work in the hardware market, does it help in added growth, or is this simply a propganda by Sun to get more interest in this chip, and new buyers thereby.

Sun's CEO Jonathan Schwatrz writes a very interesting blog, one of the best by public company CEO. JOnathan stresses that he thinks about Niagara 2 as a commodity, and Sun MicroElectronics division is encouraged to see the chip independent of Sun systems, and to even competitors of Sun. Interesting and encouraging indeed :)

About Network I/O, CPU cycles, and Multicore architectures

With the advent of 10 Gig Ethernet, network I/O has become unbelievably fast. This poses substantial challenges for kernel designers, because the erstwhile thinking that I/O is not CPU intensive does not apply any more. With 10 GigE, you can easily exhaust all the CPU in your box just transmitting or receiving raw packets without doing anything substantial.

This poses an interesting dillema, (i) What is the best network I/O you can perform with your box having 10 GigE, and (ii) Given that an application is running in the background and is the producer/ consumer of network I/O, how do you allocate your limited CPU cycles between network I/O and your application. In my PhD dissertation, we tried to solve the above on a macro scale. We developed a stochastic model to predict the best rate at which network I/O may be performed given the application workload on our system. Our work presented at PfldNet 2007 is availeble here.

With the advent of multicore chipsets such as the Intel quad-core Xeon and AMD quad-core Barcelona besides the Sun Microsystems Niagara chip which has existed for almost two years, there is an added dimension to this problem, how to cooperate the application and network I/O across cores. And with applications becoming increasingly multithreaded, this problem is definitely trickier. Since the interplay between CPU cycles, cache misses, memory latencies etc. becomes even more interesting.

I read a recent paper by authors from Virginia Tech and Argonne National Lab illustrating how a single threaded application can be bound to a core to get higher network I/O throughput. It is a very interesting read, would strongly recommend it.

Slides on evolution of 1 Gig to 10 Gig EPON

An excellent presentation made by Dr. Glen Kramer, Chief Scientist at Teknovus at the ITU-T/IEEE Workshop on Carrier-Class Ethernet in Geneva Switzerland is available here.

The presentation motivates the need to upgrade to a 10 Gig EPON standard showing the high bandwidth requirement from a Multiple Dwelling Unit (MDU) with large number of EPON subscribers. Challenges in the standardization activity, particularly meeting the power budget are documented.

Although Dr. Kramer puts some emphasis on the bandwidth requirements of HDTV broadcast TV and 4th generation mobile, I believe that the greatest demand is going to depend on the success of Video on Demand (VoD). I believer that I have aired my views on the potential of VoD before, please read my thoughts here.