EPON to be deployed around Paris, big win for the open access model

Sipperec , which provides power and telecom services on behalf of 86 municipalities around Paris today announced a fiber-to-the-premises (FTTP) network called Sequantic, which will build on its fiber backbone. This will provide connectivity to buildings targetting about 6,500 enterprises in 147 business areas located in the municipalities. Sipperec will then sell capacity on the network to service providers, so adopting the "open access" model. This is a major boost to this model promoting neutrality to the network provider also adopted by UTOPIA in Utah and the City of Amsterdam.

At UC Davis Networks Research group, we proposed advanced network protocols that may be useful to promote the "open access" model. This protocol aims to guarantee established Service Level Agreements (SLAs) to both users and service providers simultaneously. Our work was published in the IEEE Journal of Selected Areas in Communication, a copy available here.

Matisse Networks announces $45 million Series C VC funding

Matisse Networks which is building Ethernet over Optical Burst Switch technology today announced that it had closed $45 million Series C funding from Merrill Lynch PCG and other VCs. Matisse's EtherBurst switch is one of the few companies in the Optical Burst Switching (OBS) Domain.

OBS attempts to bridge the gap between optical circuit switching technology (which is widely deployed now) and packet switching which is non-existent in the optical domain because optical components cannot match electronic speeds. OBS establishes dedicated circuits for short durations of time by sending a signal in advance in the control domain which reconfigures the network to form the circuit. Data (which is called an optical burst) is then transmitted over this circuit. The network can be then reconfigured to form a different circuit along which a different optical burst may be sent. While OBS is a neat concept and has existed in academic research since the late nineties, it has not been commercially exploited other than some prototype demonstrations.

I am curious as to how the EtherBurst switches Ethernet 1500 MTU size packets over an OBS network since the granularity of a burst will be much much (1000x) larger than an Ethernet packet. May be they aggregate packets to a destination together and then transfer them. But then latency suffers. It would be great if Matisse came up with some performance characteristics such as throughput delivered, latency, etc.

Cisco launches 802.11n Wireless Access Point

Yesterday Cisco launched its first 802.11n-based Ethernet access point (AP), the Aironet 1250. This is based on the 802.11n draft 2 which is currently being standardized and will offer 300 Mbps of wireless access connectivity. Moreover, the access point can be powered by wired Ethernet which avoids having power installations at the deployment point. This product is priced at $1299.

I also read a recent research group report which projected that by 2010 most of wired ethernet in LANs will be replaced by 802.11n technology. I am not so sure. While wireless has the advantage of ubiquity, it still lacks consistant steady performance and security. For example, I see a marked difference between viewing video over my home 802.11g network and with a direct ethernet connection. And the issue is not of bandwidth, 54 Mbps is far more than what my DSL provides.

I am also sure that wired ethernet will be dominant in corporate LANs. Dont think a business can imagine it crucial data floating in the air :)

I have also been having some connectivity issues in my home wireless network, and would be glad if any of you could leave a comment on how to solve it. I live in a densely populated area. There are at least 20 wireless networks around my area. When I switch on my laptop it often has problems connecting to my wireless network. It takes quite a bit of time, sometimes 15-20 minutes before I get connectivity. I suspect it is because of running out of wireless channels in the 801.11g frquency domain. Any thoughts?

From Hot Interconnects

I could find some time to attend the second day today at Hot Interconnects at Stanford. Here are some notes. Although some papers may not be directly relate dto the discussions in this blog, I decided to include them anyways.

Paper 1:

A Memory-Balanced Linear Pipeline Architecture for Trie-based IP Lookup on FPGA, USC

The trie data structure is used for IP lookup. Efficient pipelining is possible for IP lookup. In current approaches there exists unbalanced memory distribution across stages. Their solution is to improve pipelining in trie lookups.

Paper 2:
Building a RCP (Rate Control Protocol) Test Network, Stanford
They build an experimental network to support RCP using NetFPGA implementation and modifying Linux by adding a shim layer inbetween to support RCP.

Paper 3:
ElephantTrap: A low cost device for identifying large flows, Stanford
ElephantTrap identifies and measures big flows in a network using random sampling of packets and algorithms to have select the large flows in the network.

Intel invited talk titled On-Die Interconnect and Other Challenges for Chip-Level Multi-Processing

Chips will become increasingly multicore because performance tapes of with increase in transistors in a single core. However, gain from parallelism is lost in communication overhead. Speaker advocated on-chip high speed interconnect which ties cache, memory, network I/O etc to processors. Also advocated ring based topology for interconnect. He talked briefly about Intel terascale processor with 80 cores.

Paper 4:
An Analysis of 10-Gigabit Ethernet Protocol Stacks in Multicore
Environments, Virginia Tech

They consider the interaction of protocols with multicore architectures. They consider TCP/IP and iWarp protocols and the interaction between application, TCP/IP stack and network I/O. They show that if application is scheduled on different core than where interrupt processing is (but in same socket), it leads to best performance as CPU load is distributed across cores and it leads to lesser cache misses too. Did not consider MSI/MSI-X or multithreaded applications.

Paper 5:
Assessing the Ability of Computation/Communication Overlap and Communication Progress in Modern Interconnects, Queen's University

They consider interplay between processing and I/O so as to have minimum stalling cycles. They consider various MPI based applications over different interconnects: Infiniband, Myricom and 10 Gig Ethernet

Paper 6:
Performance Analysis and Evaluation of Mellanox ConnectX InfiniBand
Architecture with Multi-Core Platforms, Ohio State Univ

They demonstrate how ConnectX (the latest generation of Infiniband technology) performs on multicore architectures.

Paper 7:
Memory Management Strategies for Data Serving with RDMA, Ohio Supercomputer Center

They demonstrate that virtual to physical memory translation is a considerable overhead in high-speed networks. They demonstrate how to optimize memory registeration to achieve better performance in RDMA

Paper 8:
Reducing the Impact of the Memory Wall for I/O Using Cache Injection, Universiy of New Mexico

Cache Injection is a technique by which data is transferred directly fron device to L2/L3 cache. They show the tradeoff between cache injection and prefetching and propose an algorithm to perform efficient cache injection for network I/O,

UltraSparc T2 with integrated Network I/O

Although I do not follow hot chips, and have never blogged about microprocessors, I would love to talk about Sun's UltraSparc T2 launched last week, the first microprocessor ever to support an integrated network I/O support. Yes, UltraSparc T2 which I would prefer to call by its moniker Niagara 2, supports multithreaded dual 10 Gig Ethernet. From product brochures, this chip seems to be a true microsystem, just connect this to RAM and you get it running.

Although Sun has only mentioned some CPU intensive benchmarks such as SPEC int and SPEC float, may be they come up with some network I/O oriented benchmark numbers as well.

Also Sun is releasing all design documents under the GPL license. Wonder what that means? Can someone take up all the technical documentation and fab a chip independently? How does open sourcing work in the hardware market, does it help in added growth, or is this simply a propganda by Sun to get more interest in this chip, and new buyers thereby.

Sun's CEO Jonathan Schwatrz writes a very interesting blog, one of the best by public company CEO. JOnathan stresses that he thinks about Niagara 2 as a commodity, and Sun MicroElectronics division is encouraged to see the chip independent of Sun systems, and to even competitors of Sun. Interesting and encouraging indeed :)

About Network I/O, CPU cycles, and Multicore architectures

With the advent of 10 Gig Ethernet, network I/O has become unbelievably fast. This poses substantial challenges for kernel designers, because the erstwhile thinking that I/O is not CPU intensive does not apply any more. With 10 GigE, you can easily exhaust all the CPU in your box just transmitting or receiving raw packets without doing anything substantial.

This poses an interesting dillema, (i) What is the best network I/O you can perform with your box having 10 GigE, and (ii) Given that an application is running in the background and is the producer/ consumer of network I/O, how do you allocate your limited CPU cycles between network I/O and your application. In my PhD dissertation, we tried to solve the above on a macro scale. We developed a stochastic model to predict the best rate at which network I/O may be performed given the application workload on our system. Our work presented at PfldNet 2007 is availeble here.

With the advent of multicore chipsets such as the Intel quad-core Xeon and AMD quad-core Barcelona besides the Sun Microsystems Niagara chip which has existed for almost two years, there is an added dimension to this problem, how to cooperate the application and network I/O across cores. And with applications becoming increasingly multithreaded, this problem is definitely trickier. Since the interplay between CPU cycles, cache misses, memory latencies etc. becomes even more interesting.

I read a recent paper by authors from Virginia Tech and Argonne National Lab illustrating how a single threaded application can be bound to a core to get higher network I/O throughput. It is a very interesting read, would strongly recommend it.

Slides on evolution of 1 Gig to 10 Gig EPON

An excellent presentation made by Dr. Glen Kramer, Chief Scientist at Teknovus at the ITU-T/IEEE Workshop on Carrier-Class Ethernet in Geneva Switzerland is available here.

The presentation motivates the need to upgrade to a 10 Gig EPON standard showing the high bandwidth requirement from a Multiple Dwelling Unit (MDU) with large number of EPON subscribers. Challenges in the standardization activity, particularly meeting the power budget are documented.

Although Dr. Kramer puts some emphasis on the bandwidth requirements of HDTV broadcast TV and 4th generation mobile, I believe that the greatest demand is going to depend on the success of Video on Demand (VoD). I believer that I have aired my views on the potential of VoD before, please read my thoughts here.

Intel introduces 10Gig cards with I/O acceralaration technology

Quick update, two days ago Intel announced its 10 Gig Ethernet card based on its I/O accelaration technology. Intel is promoting an alternative to TCP and RDMA offload wuth a card that can work better with the processor's I/O capabilities. Basically Intel's strategy is to promote its processors while discounting ethernet cards. This product will a[pparently be launched in September with Inte's Caneland platform serving 4-socket 4-core Xeon processors. I will dig up some more research literature about this product in the coming days.

IEEE HSSG approves PAR and 5 criteria

Quick update, from the archives of the IEEE HSSG public reflector, the PAR and 5 Criteria has been approved by the HSSG. The objectives now include supporting 40 Gig Ethernet over a distance of of 100m, and support 100 Gig Ethernet over a distance of 40km on single-mode fiber. Support will also be provided for OTN.

This is a big win for 40 Gig E proponents, whose support was considerably weak until January this year. It will be interesting to see how the working group handles the tricky issue of dealing with 40 Gig and 100 Gig physical layer challenges simultaneously.

Presentation for IEEE HSSG meet

Presentations for the IEEE HSSG meet next week have been uploaded at the IEEE HSSG website. Some of the presentations I found interesting are from Dell and Deutsche Telekom about the use of 40/100 Gig Ethernet from their points of view.

Standardization of 40 Gig vs 100 Gig Ethernet

Next week at the IEEE 802.3 plenary in SanFrancisco, the High Speed Study Group (HSSG) will meet to gather consensus on a 100 Gbps Ethernet standardization activity. There has been considerable support within the HSSG committee for an interim 40 Gig Ethernet standard, or a standard which is interoperable between 40 Gig and 100 Gig. There have been several news reports covering this battle since the last plenary meet in Geneva in May 2007, unfortinately I have not found time to research this and write about it in the past month in this blog. Here are some interesting articles that I found on a Google search:

Controversy threatens 100G standards work

Group pushes 100 Gigabit Ethernet

To those unfamiliar with the process of standardization, the Study Group has to develop sufficient consensus to have a Project Authorization Request (PAR) and resposes to 5 Criteria which cover (i) Broad Market Potential (ii) Compatibility with existing standards (iii) Distinct Identity (iv) Technical Feasibility and (v) Economic Feasibility. I believe the 5 Criteria of the HSSG has already been decided. Once the Study Group develops consensus, the IEEE forms a standardization committee which develops the standard (also known as the task force). If the HSSG is unable to gather a consensus, then the IEEE may scrap the creation of the standard or create a new Study group with new deadlines. The HSSG has already taken 2 extensions to submitting the PAR.

While I will keep you posted about developments in next week's plenary, please do post your comments on this issue. I am of the opinion that wif a technology is sufficiently mature, it ought to be standardized irrespective of what volumes and market it may drive. Also the standard must taken into account driving a low price point for the technology so that volumes grow up. As an example, although the 10 Gig Ethernet standard is about a couple of years old, 10 Gig NIC volumes are still verry low. One of the reasons I see here is that the standard mandates support for a distance over 300 mts which drives the cost of the PHY devices very high, and thus prevents lowering the entry price point of this technology. Hope the same doesn't happen to the HSSG standard.

Infonetics Research Group PON and FTTH report

Apologies for not having any blog updates for about a month, it has been a pretty dry June, and I was vacationing for a bit. Am now recharged :), ready to blog for another year. By the way, this September will mark the two year anniversary of this blog.

The Infonetics Research PON and FTTH Equipment and Subscribers" report released last week was very encouraging. Worldwide PON sales are up to $336 million in the first quarter of 2007, while Ethernet FTTH (including EPON) sales are up to $81 million.

Some other good news are:
- Korea is on track to add 700,000 to 800,000 EPON subscribers by the end of 2007.
- EPON subscribers make up 64% of worldwide subscribers, BPON 33%, GPON 3% in 2006. However, this market share will change very soon as Verizon deploys GPON in 2007. The report also quotes Verizon already having 500,000 FiOS customers.

TThe report also predicts high growth for FTTH technologies as service providers aim to deliver more and more bandwidth to the end-user to provide enhanced quality video-on-demand, peer-to-peer networking, online gaming and other high bandiwdth services.