Skip to content

WaytoICT

通往ICT之路 · 光通信与AI网络技术解读

中文English
登录注册
Menu
Technology Optical comms · silicon photonics · CPO → Market Insights Financials · trends · industry → In-Depth Analysis Optical chips, devices & modules → Video Understand a topic in minutes → Documents Latest docs · summit materials →
Menu

Fundamentals of High-Performance Networking

Posted on 2025-02-11 by
Fundamentals of high-performance networking

Among the traditional pillars of data center infrastructure (namely compute, storage, and networking), networking is unique because it has no inherent value on its own. While a standalone server can still compute and storage devices can still hold data, the value of the network depends solely on the information it transports.

The same logic applies to High-Performance Networking (HPN). However, the negative impact of an HPN failure can be more significant. If data cannot arrive at its destination perfectly, reliably, and in order, it usually causes upstream and downstream disruptions beyond the basic transmission scope. Why? Because “high performance” is far more complex than mere bandwidth and Application-Specific Integrated Circuit (ASIC) latency.

Network engineers know well that, in traditional environments, the dividing line lies at the moment when a data packet enters the cable on one side and is handed to the Network Interface Card (NIC) on the other side. This division is simple, clear, and convenient for troubleshooting (or at least allows shifting responsibility elsewhere by claiming “it is not a network problem”), and it covers only a tiny part of the whole process.

This article will explore the components of high-performance networking, the available transmission technologies, and the “secret” that truly makes it run at high speed.

What Are the Components of a High-Performance Network?

  • Latency

When a packet passes through a single switch ASIC, the NIC-to-NIC latency is usually between 100-300 nanoseconds (specific values vary by chip and platform). All else being equal, the core-to-core latency is about 50 microseconds. What does this mean? About 95% of the latency occurs inside the server.

To understand this, one needs to look deep into the internal structure of the server. Note that applications, sockets, and protocol drivers all reside in main memory.

Fundamentals of high-performance networking

A highly simplified packet transmission process (“a day in the life of a packet”) begins with two 8-byte registers entering the CPU, outputting one 8-byte register, which is regarded as a 64-bit floating-point operation. From here, our packet needs to pass through the L1 (extremely fast), L2 (very fast), and L3 (fast) caches, main memory (slower), and the PCI bus in sequence, finally reaching the NIC for transmission to another device.

  • Bandwidth

Bandwidth demand keeps growing. Currently, the minimum requirement for high-performance networks is 400Gbps, expected to jump to 800Gbps to 1600Gbps within the next two years. Bandwidth is a constantly moving target, and investment should be understood as having a limited “top-tier” service life, because bandwidth currently considered ultra-high-speed will soon become commonplace.

Nonetheless, although the engineering and physical principles to achieve such bandwidth are quite complex and the size of the pipe does matter, bandwidth itself is not what makes a network “high-performance”.

If it is not latency and bandwidth, then what makes applications run faster?

RDMA

RDMA: Remote Direct Memory Access, an I/O bypass technology implemented on the NICs of participating computing devices. RDMA uses zero-copy networking, that is, directly reading data from the main memory of one device to the main memory of another, while completely bypassing the NIC, sockets, and transport buffers running in that memory.

Fundamentals of high-performance networking

RDMA is a multidisciplinary field involving a multi-layered understanding of application, network, compute, and storage technologies. Once RDMA is enabled, the network can achieve a 90% to 96% reduction in end-to-end latency (rather than NIC-to-NIC latency).

Fundamentals of high-performance networking

However, RDMA also has some drawbacks. The most notable is that it has extremely strict quality and reliability requirements: data must be transmitted reliably and in order, without packet loss or jitter, and must always be so. The “best-effort” service level agreement (SLA) of standard data center networks is insufficient. This is where the “high-performance” part of high-performance networking comes in.

So, what is a high-performance network?

The problem of defining a high-performance network is not merely a matter of latency or bandwidth, although both are factors. High performance actually depends on how you transport RDMA (or the next truly important application). Just as networks must adapt to voice-over-IP, Fiber Channel-over-Ethernet, and dozens of other “sensitive applications”, modern networks must also adapt to the needs of today’s high-end compute and storage capabilities.

Currently, there are two main industry-standard approaches for RDMA transport:

InfiniBand

InfiniBand is a communication standard supporting RDMA. The latency and bandwidth of InfiniBand ASICs are roughly equivalent to Ethernet performance. In fact, the latency of IP-over-InfiniBand is the same as unaccelerated IP-over-Ethernet. Its physical hardware includes a Host Channel Adapter (HCA, essentially a NIC) and InfiniBand switches. It is a controller-based architecture (through a subnet manager), is non-routable, meaning the failure domain size is the same as the entire network fabric. The subnet manager runs as software on one of the switches.

RoCEv2

RoCE (pronounced “rocky”) is a similar standard using Ethernet, but with some adjustments. It also consists of HCAs and switches, but relies on HCA point-to-point negotiation for reliable transport, rather than an InfiniBand-based controller. Priority Flow Control (PFC) and Explicit Congestion Notification (ECN) parameters must be configured on the switches to identify and prioritize RDMA-related traffic.

Which one to choose depends on engineering preference and deep-rooted beliefs.

Considerations of InfiniBand vs. RoCE

The following is a quick comparison:

Fundamentals of high-performance networking

Flexibility

InfiniBand networks are tailored for specific workloads (such as AI) and can support only one type of workload. If a customer’s data center only contains a limited number of continuously running AI workloads with similar performance characteristics, InfiniBand may be a good choice. High-performance Ethernet is more flexible. If a customer’s data center focuses on cloud computing (hybrid cloud or cloud-ready) or contains an application ecosystem with multiple different needs, Ethernet/RoCE may be a better choice. Migrating from InfiniBand to Ethernet may face challenges.

Scalability

As AI/ML workloads grow, the scale of data centers will also expand. If current growth trends continue, today’s NVIDIA SuperPOD DGX cluster (127 nodes / 1016 GPUs) will need to expand 20-fold within two years. Although InfiniBand’s specification allows building very large network fabrics, some complications may arise when facing large failure domains. InfiniBand has no Layer 3 option.

When to use each technology?

Best practices recommend using InfiniBand for the most challenging and performance-demanding applications (that is, not just any AI model, but truly large AI models). The other 95% of use cases can run comfortably on the more flexible Ethernet option.

In 2022, Meta conducted a study comparing the pre-training time of its LLAMA2 AI model. Two identical clusters were built in the study: one based on InfiniBand and the other on RoCEv2. The study found that below 2000 GPUs, the performance of the two is very close. At current rack density and pricing, 2000 GPUs require about 63 cabinets, cost about 100 million dollars, and take more than a year to build.

The general industry consensus is that, although InfiniBand always has its place, Ethernet-based solutions will continue to evolve to meet most market demand.

Reference documents:

White Paper on RDMA-Based Long-Distance Lossless Data Movement Technology

InfiniBand and Ethernet RDMA Technologies

Enjoy this article?

Subscribe to get new optical communication insights delivered to your inbox. Free, unsubscribe anytime.

Category: Technology

Leave a Reply

You must be logged in to post a comment.

订阅更新

订阅后,新文章发布时第一时间收到邮件通知

李东霏
光互连技术研究者

香农信息技术研究院,专注光模块、CPO、硅光与 AI 光互连的技术解读。

行业报告

光芯片·光器件·光模块 2026 报告汇编(含 OFC 2026)

光芯片 · 光器件 · 光模块
2026 报告汇编(含 OFC 2026)

¥1,900

上/下两册,收录 OFC 2026 一线报告
添加微信 WaytoICT 订购

广告

📢 广告位招商:面向全球光通信客户 侧边栏广告位 Email:Dongfei.li@shannoninstitute.cn

最新文档

  • 互连拐点:11 页解析机架内到多机架 Scale-Up 的 XPO/CPX/OCI 蓝图(OCP 2026 全球峰会)
  • AI 时代的相干光技术:17 页解析 Scale-Across、液冷 XPO 与全谱转发器(OCP 2026 全球峰会)
  • XPO 高密度光互连的系统级评估:20 页解析 212G SerDes 平台与 FRO/LRO/LPO 实测(OCP 2026 全球峰会)
  • CPO 与 OCI 标准及未来技术:17 页 Broadcom 精简版演讲提纲(OCP 2026 全球峰会)
  • VCSEL 高密度互连:7 页解析 50G NRZ「慢而宽」阵列与光 PCIe 演示(OCP 2026 全球峰会)

最新评论

暂无评论,欢迎登录后发表第一条评论。

最近访客

👥 今日访客 -
加载中…
© 2026 WaytoICT | 蜀ICP备2020035321号-2 | 隐私政策 | 免责声明
Wechat Pay
Alipay
请使用 支付宝 或 微信 扫码支付