SAN JOSE, Calif. – The network cable plugs into the port, clicking as it snaps into place. That’s always been the way that IT devices have created physical connections.
That may need to change in coming years, as the growth of hyperscale computing tests the limits of switching hardware. That’s the message from networking pioneer Andy Bechtolsheim, who says the industry should prepare for hardware designs in which optical connectivity is integrated directly into chips.
“Today, almost all optics plug into the front of the switch,” Bechtolsheim said during a keynote at the recent Open Compute Summit. “What would happen if you could put these optics right on the board, or preferably on the switch chip itself?”
Bechtolsheim argued that this approach, known as co-packaged optics, will be critical to advancing network speeds beyond 400 GB Ethernet, the next major networking transition, which is expected to be widely adopted in 2019-21.
“Co-packaged optics have significant promise of power reduction and cost reduction,” said Bechtolsheim, the founder and Chairman of Arista Networks. “What’s absolutely key is to do this in a standardized fashion. This is clearly a multi-year project, so let’s get started now.”
Key Benefits: Cost, Power Savings
Co-packaged optics – also known as on-board optics or embedded optics – have been available for some time as proprietary or targeted solutions. This approach has operational benefits and improves the use of silicon, Bechtolsheim said, but the biggest opportunity is to reduce the power use and cost of components.
“If the optics were moved directly onto that module, it could save up to 30 percent of the total system power, because it eliminates the power for the high-speed I/O that drives the signal from the switch chip to the optics,” said Bechtolsheim.
A look at the nebefits of co-packaged or on-board optics. (Slide: Arista Networks)
Bechtolsheim’s keynote at the OCP Summit seems likely to boost interest in co-packaged optics, which has the potential to create cost and efficiency savings that align with the interests of Open Compute members.
Leading network equipment vendors are already collaborating through The Consortium for On-Board Optics (COBO), which is working to develop specifications for board-mounted optical modules. Founding members include Cisco, Arista Networks, Juniper, BroadCom and Mellanox Technologies, along with Microsoft, which is the sole hyperscale operator involved in the project.
At an optical fiber conference last month, COBO published its first spec for standardized on-board optical modules for high-performance data center networks.
“The COBO specification is an industry breakthrough for the power dissipation, placement and routing benefits it offers for 400Gbps and greater networks,” said Brad Booth, COBO President and Principal Engineer at Microsoft. “These features solve a number of operation challenges associated with today’s pluggable standards while offering much more performance headroom for scaling next-generation networks to 1.6 Tbps as organizations like OIF and IEEE 802.3 work to standardize 100G serial interfaces.”
Support from Key Hyperscalers
What might a switch with co-packaged optics look like?
A co-packaged optics concept from Luxtera, presented during the Open Compute Summit 2018.
During his OCP Summit presentation, Bechtolsheim shared a concept developed by Luxtera Systems, a network systems vendor focused on silicon photonics and on-board optics. The unit, described as “packaging study” and not an actual product, supports up to 128 400G connections and potential throughput of 51.2 terabit per seconds. The design could double the density of pluggable switches, but would also use about 1 kilowatt of power in a 1U rack form factor.
Much work remains, according to Bechtolsheim.
“The challenges are primarily technical,” he said. “This clearly needs some multi-vendor standardization. We need the optics vendors, the chip vendors and the packaging vendors to work together here. Maybe the most befuddling challenge is the supply chain. We have to make this manufacturable, so it can be done by contract manufacturers at a low cost.”
The Open Compute Project has been notable for its role in creating an ecosystem – and thus a market – for cutting-edge technology for hyperscale customers. The rise of open hardware has raised the profile of contract manufacturers and ODMs (original design manufacturers), who have adapted OCP designs into server and storage product lines.
Hyperscale data center operators are focused on improving their network capabilities, and figure to be logical early adopters of on-board optics, as suggested by Microsoft’s role in COBO. It’s hardly the only one interested, as confirmed in another keynote from the OCP Summit.
“We strongly believe that co-packaging the optics with the ASIC may be the next way to go,” said Sree Sankar, Technical Product Manager at Facebook.
