Optics to the Chip: The Key to Super-Fast Networks of the Future

April 3, 2018
Integrating optical network connections directly onto chips could be critical to the future of hyperscale data centers as network speeds advance. We look at efforts to develop technology and standards for on-board optics.

SAN JOSE, Calif. – The network cable plugs into the port, clicking as it snaps into place. That’s always been the way that IT devices have created physical connections.

That may need to change in coming years, as the growth of hyperscale computing tests the limits of switching hardware. That’s the message from networking pioneer Andy Bechtolsheim, who says the industry should prepare for hardware designs in which optical connectivity is integrated directly into chips.

“Today, almost all optics plug into the front of the switch,” Bechtolsheim said during a keynote at the recent Open Compute Summit. “What would happen if you could put these optics right on the board, or preferably on the switch chip itself?”

Bechtolsheim argued that this approach, known as co-packaged optics, will be critical to advancing network speeds beyond 400 GB Ethernet, the next major networking transition, which is expected to be widely adopted in 2019-21.

“Co-packaged optics have significant promise of power reduction and cost reduction,” said Bechtolsheim, the founder and Chairman of Arista Networks. “What’s absolutely key is to do this in a standardized fashion. This is clearly a multi-year project, so let’s get started now.”

Key Benefits: Cost, Power Savings

Co-packaged optics – also known as on-board optics or embedded optics – have been available for some time as proprietary or targeted solutions. This approach has operational benefits and improves the use of silicon, Bechtolsheim said, but the biggest opportunity is to reduce the power use and cost of components.

“If the optics were moved directly onto that module, it could save up to 30 percent of the total system power, because it eliminates the power for the high-speed I/O that drives the signal from the switch chip to the optics,” said Bechtolsheim.

A look at the nebefits of co-packaged or on-board optics. (Slide: Arista Networks)

Bechtolsheim’s keynote at the OCP Summit seems likely to boost interest in co-packaged optics, which has the potential to create cost and efficiency savings that align with the interests of Open Compute members.

Leading network equipment vendors are already collaborating through The Consortium for On-Board Optics (COBO), which is working to develop specifications for board-mounted optical modules. Founding members include Cisco, Arista Networks, Juniper, BroadCom and Mellanox Technologies, along with Microsoft, which is the sole hyperscale operator involved in the project.

At an optical fiber conference last month, COBO published its first spec for standardized on-board optical modules for high-performance data center networks.

“The COBO specification is an industry breakthrough for the power dissipation, placement and routing benefits it offers for 400Gbps and greater networks,” said Brad Booth, COBO President and Principal Engineer at Microsoft. “These features solve a number of operation challenges associated with today’s pluggable standards while offering much more performance headroom for scaling next-generation networks to 1.6 Tbps as organizations like OIF and IEEE 802.3 work to standardize 100G serial interfaces.”

Support from Key Hyperscalers

What might a switch with co-packaged optics look like?

A co-packaged optics concept from Luxtera, presented during the Open Compute Summit 2018.

During his OCP Summit presentation, Bechtolsheim shared a concept developed by Luxtera Systems, a network systems vendor focused on silicon photonics and on-board optics. The unit, described as “packaging study” and not an actual product, supports up to 128 400G connections and potential throughput of 51.2 terabit per seconds. The design could double the density of pluggable switches, but would also use about 1 kilowatt of power in a 1U rack form factor.

Much work remains, according to Bechtolsheim.

“The challenges are primarily technical,” he said. “This clearly needs some multi-vendor standardization. We need the optics vendors, the chip vendors and the packaging vendors to work together here. Maybe the most befuddling challenge is the supply chain. We have to make this manufacturable, so it can be done by contract manufacturers at a low cost.”

The Open Compute Project has been notable for its role in creating an ecosystem – and thus a market – for cutting-edge technology for hyperscale customers. The rise of open hardware has raised the profile of contract manufacturers and  ODMs (original design manufacturers), who have adapted OCP designs into server and storage product lines.

Hyperscale data center operators are focused on improving their network capabilities, and figure to be logical early adopters of on-board optics, as suggested by Microsoft’s role in COBO. It’s hardly the only one interested, as confirmed in another keynote from the OCP Summit.

“We strongly believe that co-packaging the optics with the ASIC may be the next way to go,” said Sree Sankar, Technical Product Manager at Facebook.

About the Author

Rich Miller

I write about the places where the Internet lives, telling the story of data centers and the people who build them. I founded Data Center Knowledge, the data center industry's leading news site. Now I'm exploring the future of cloud computing at Data Center Frontier.

Sponsored Recommendations

Get Utility Project Solutions

Lightweight, durable fiberglass conduit provides engineering benefits, performance and drives savings for successful utility project outcomes.

Guide to Environmental Sustainability Metrics for Data Centers

Unlock the power of Environmental, Social, and Governance (ESG) reporting in the data center industry with our comprehensive guide, proposing 28 key metrics across five categories...

The AI Disruption: Challenges and Guidance for Data Center Design

From large training clusters to small edge inference servers, AI is becoming a larger percentage of data center workloads. Learn more.

A better approach to boost data center capacity – Supply capacity agreements

Explore a transformative approach to data center capacity planning with insights on supply capacity agreements, addressing the impact of COVID-19, the AI race, and the evolving...

Courtesy of CoolIT Systems

Considerations for Selecting a Row-Based CDU for your Data Center

Ian Reynolds, Senior Project Engineer with CoolIT Systems, outlines the factors you need to consider when selecting the best CDU for your data center's needs.

Image was generated by DALL E 3 GPT-4, courtesy of EdgeConneX

White Papers

Get the full report.

From Cloud-Native Applications to Composable Infrastructure: 5 New Realities at the Edge

July 8, 2022
DartPoints outlines five new realities at the edge from decentralization to cloud-native applications and composable infrastructure.