Since the commercialization of CMOS chips and indium phosphide optical device technology in 2008, companies such as Marvell have promoted the transformation of coherent optical modules from traditional fixed equipment to pluggable form through six generations of DSP chip iterations.
Taking the first-generation DSP in the 40 nm process as an example, its power consumption reached 40 W and it supported a transmission distance of 4000 kilometers; whereas the latest 5 nm-process Orion™ chip achieves 2000-kilometer transmission through 800G QAM16-PS modulation technology at 10-20 W power consumption, and integrates LDPC error correction and OTN/Ethernet protocol conversion functions. This technological progress has reduced the volume of optical modules by 60%, making them more suitable for high-density data center deployment.

The industry is currently focused on the research and development of 1.6T single-wavelength transmission technology, which places higher requirements on signal processing. To achieve stable transmission at a 240G baud rate, the Marvell team developed the dual-tone test method.
As shown in the figure below, this method precisely captures high-frequency-band signal distortion by generating a 45 GHz amplitude-envelope signal. The wideband model established on this basis optimized the analog-to-digital converter design, increasing the effective number of bits (ENoB) from 4.8 bit to 5.2 bit while controlling the voltage range within ±0.75 V, improving signal quality by 18%.

Figure: TIA output signal comparison (45 GHz high-gain and low-gain states) (a) clipping phenomenon under high gain (b) comparison of optimal voltage and maximum voltage
The wideband model established on this basis optimized the analog-to-digital converter design, increasing the effective number of bits from 4.8 bit to 5.2 bit while controlling the voltage range within ±0.75 V, improving signal quality by 18%.
In laser co-optimization, the research team found that relying solely on the linewidth indicator is insufficient to cope with sudden frequency fluctuations. By improving the measurement method and adopting 64QAM coding with interleaving depth adjustment (from 32 to 96), the timing-error tolerance was increased by 3 times. This reduced the update frequency of the clock-recovery circuit from once per symbol to once per 8 symbols, cutting power consumption by 22%, while also lowering the requirements for laser performance.
Analog front-end design faces new challenges brought by CMOS process scaling. Each generation of process-node progress (such as from 28 nm to 16 nm) brings about a 2-fold increase in transmission rate. However, although 7 nm FinFET transistors raise the cutoff frequency to 300 GHz, the increase in parasitic capacitance restricts bandwidth expansion. To this end, Marvell integrated a T-coil peaking structure at the input of the transimpedance amplifier, extending the -3 dB bandwidth from 45 GHz to 58 GHz, and suppressed power-supply noise to below 0.8 mVrms through a nested voltage-regulator design. These improvements stabilized the bit error rate of 112G Baud signals within 1E-15 after 30 km fiber transmission.

Figure 2: CMOS process node and transmission rate evolution trend (trend-line annotation: rate doubles every 4.7 years, 1.6T expected to be achieved in 2025)
Power optimization of the receiving-end DSP is crucial. For the 1.6T data stream, the team adopted a folded FFT architecture based on the Radix-2³ algorithm, reducing the number of complex multipliers required for the 128-point transform by 64% (from 1,024 to 372), with latency controlled within 12 ns. At the same time, through algorithmic downsampling, the computational density of carrier phase estimation was reduced by 75%, achieving a 4-fold throughput improvement at the same power consumption.
Cross-domain collaboration has become the key to technological breakthroughs. Marvell worked with optical-component suppliers to develop a photonic integrated chip model, incorporating package impedance fluctuation (±5 Ω) into DSP equalizer training, improving channel-estimation accuracy by 15%. This end-to-end optimization enabled the 1.6T module to achieve the 2000-kilometer transmission target at 20 W power consumption, a 2.3-fold improvement in energy efficiency over the previous generation.
With the 3 nm process expected to enter mass production in 2025, 3.2T transmission technology has entered the pre-research stage.
Reference paper: Toward_1.6T_Low-Power_Coherent_DSP_Challenges_and_Lessons_Learned_from_Preceding_Generations
Reference documents:
Marvell 1.6T AEC DSP Chips and Copper-Cable Interconnect
Leading Optical-Module Companies: 800G and 1.6T Usher in a New Growth Era
Jointly built by several bloggers, the ICT-domain knowledge-base ima Q&A tool is currently free to join. Welcome to experience it:

Related Reading
- How to Understand the Performance of Coherent Network Solutions
- For Data Center 800G, PAM4 or Coherent Technology?
- A Brief History and Popular Science of Digital Coherent Systems
Enjoy this article?
Subscribe to get new optical communication insights delivered to your inbox. Free, unsubscribe anytime.
