Cloud data centers: the demand for 400Gb/s (and beyond) optical links
Every day, we produce massive amounts of data. Think of e-mails and text messages, pictures and videos posted on social media, and the huge amount of data processed by companies such as Google, Facebook, and Amazon. All of this data is stored and processed in cloud data centers. These data centers typically reveal building-wide rich optical fiber networks. Hundreds of thousands of optical links interconnect the server racks through a complex network of fiber optical cable. Today, 100Gb/s optical links – made up of 4 x 25Gb/s single channels or lanes – are sufficient to sustain the data traffic within the data center. They transmit data over several hundreds of meters, up to 2km.
Due to the continuing growth in social networking, cloud computing and big data applications, the demand for data communication in these datacenters is projected to grow exponentially in the next few years. According to Cisco Global Cloud Index, the annual global data center IP traffic will have surpassed 20 Zettabytes (or 20x1021 bytes) by 2021. To meet this demand, data center operators will have to upgrade their network to 400Gb/s links by 2019, and this will be further upscaled to 1.6Tb/s by 2022. These next-generation 400Gb/s optical links will need to cover 500m, 1km and 2km distances within the data center. Currently, the industry is considering to implement such links either with 8 x 50Gb/s lanes, or with 4 x 100Gb/s lanes to reduce the component count. Because of the huge amounts of optical links within a data center, they need to be low cost and consume as little power as possible.
Imec is currently investigating several implementations for optical links at 400Gb/s and beyond in Si photonics technology. In this article, we will first discuss imec’s Si photonics platform. This platform co-integrates compact high-speed modulators and photodetectors with electro-optical bandwidths beyond 50GHz. These devices can be operated comfortably at 56Gb/s NRZ (non-return-to-zero) single-lane data rates. By adopting higher-order modulation formats, single-channel data rates beyond 100Gb/s are realized. Two such complex modulation formats have been explored: the electrical duobinary (EDB) modulation format, as well as the pulse-amplitude modulation format PAM-4. Finally, we will discuss the electrical and optical packaging challenges to realize fully integrated 50GHz Si photonics interposers, enabling tight integration of future multi-Tb/s optical modules with high-bandwidth logic and memory chips.
Imec’s Si photonics platform
Single-mode fiber is the optical medium of choice for high-speed optical transceivers spanning 500m, 1km or 2km within the data center. Si photonics is ideally suited to integrate the essential building blocks of the single-mode optical fiber links. With this scalable technology, compact and low-power transceivers can be implemented at low cost and high volume, by leveraging existing CMOS fabrication infrastructure.
Imec’s Si photonics prototyping platform uses 200mm SOI wafers as a starting substrate. The fabrication process uses a modified 130nm CMOS flow augmented with 193nm lithography for all the waveguide patterning layers and germanium (Ge) for the photodetectors. An additional oxide/poly-Si stack is deposited to allow for more degrees of freedom in the design of the optical components. This stack is used for integrating the advanced Si passive components, including fiber couplers, waveguides and wavelength multiplexing filters. Subsequently, ion implantation, selective-area growth of Ge and standard CMOS metallization modules are used to co-integrate active components including electro-optic modulators, thermo-optic devices and Ge photodetectors. Electronics circuits such as drivers and trans-impedance amplifiers (TIA) can be fabricated on a separate wafer and assembled at the die level with the Si photonics circuits using flip-chip assembly methods. In these circuits, the driver converts a standard CMOS bit state into an electrical voltage or current that is compatible with the optical device, while the TIA translates the photocurrent back into a standard CMOS bit state.
Demonstrating 56Gb/s building blocks
Among the main building blocks of optical transceivers are optical modulators, which imprint the data signals onto the optical carrier wavelength. Modulators can come in several flavors, such as Si ring modulators, Si Mach-Zehnder modulators and GeSi based electro-absorption modulators – each having their own distinct properties and (dis)advantages. Imec has developed solutions for each of these modulators, enabling them to operate at 56Gb/s NRZ. NRZ or non-return to zero is a simple two-level, one-bit modulation format, turning ‘on’ or ‘off’ the light depending on the bit state (1 or 0).
Carrier-based Si Mach-Zehnder and ring modulators – originally developed for operation in the C-band (1530 – 1565nm wavelength) – have recently been optimized to operate in the O-band (1260 – 1360nm). The O-band is the preferred band for applications requiring single-mode-fiber transmission distances beyond 1km. Optimizations were based on design modifications and process modifications. This has enabled the realization of a compact traveling-wave Mach-Zehnder modulator with 37GHz bandwidth at -1V bias, which is sufficient for 56Gb/s NRZ operation.
Alternatively, ring modulators can be used at the transmitter side. By exploiting the optical resonance effect, ring modulators can offer a reduced dynamic power consumption and smaller footprint. However, they are more sensitive to temperature fluctuations and fabrication variability, and as such require a feedback-loop control circuit and integrated heater to stabilize the modulation performance. The imec team realized O-band ring modulators with modulation bandwidths in the range of 35GHz up to 45GHz. With a 1Vpp driver, the 35GHz modulator operates with ~3dB extinction ratio and ~5dB insertion loss. The integrated heater enables thermo-optic tuning and stabilization of the ring operation wavelength across more than 100 degrees C.
Finally, a GeSi-based electro-absorption modulator was developed with operation wavelength in the C-band. Electro-absorption modulators in general offer a wider optical bandwidth than e.g. ring modulators. On top of that, when designed in a lateral p-i-n Ge/Si diode configuration, the same device can be optimized to work as a photodiode. This way, a single-chip Si photonics transceiver has been developed – with both electro-absorption modulators and photodiodes – operating with a bandwidth well beyond 50GHz. A driver with a 2Vpp swing driver realizes an extinction ratio of ~4dB with similar optical insertion loss.
C-band GeSi electro-absorption modulator: (left) static insertion loss and extinction ratio and (right) 56Gb/s NRZ eye diagram.
Towards 400Gb/s optical links
A team of IDLab and the Photonics Research Group, both imec research groups at Ghent University, have used these modulators and photodiodes as building blocks to demonstrate advanced 100Gb/s single-lane transmitters. This has been realized by cleverly applying the building blocks and by combining them with advanced, dedicated high-speed electronics. The team explored and assessed three different modulation formats: NRZ, EDB and PAM-4. With these 100Gb/s transmitters, 400Gb/s optical links can be built, either through the use of four parallel fibers (through which light of one specific wavelength is sent in parallel), or through wavelength multiplexing. With wavelength multiplexing, the signals are encoded onto (four) different carrier wavelengths which are transmitted through the same optical fiber.
100Gb/s NRZ and EDB transmission
The most ‘obvious’ approach towards single-lane 112Gb/s transmission is to double the speed of the original 56Gb/s NRZ modulator. However, making the electronics fast enough to enable this high data rate, is not straightforward.
For this experiment, the team used an in-house developed electrical transmitter and receiver capable of generating and processing 100Gb/s signals, together with a GeSi-based electro-absorption modulator. Transmitter and receiver chips were developed in a 0.13µm SiGe BiCMOS technology.
A key innovation was required to realize the first 100Gb/s NRZ error-free transmission over 500m in real time. At the transmitter side, an analog equalizer on-chip was applied to compensate for any non-idealities in the subsequent electrical or optical components within the link. This compensation was achieved by pre-distorting the electrical signal that comes out of the transmitter. At 100Gb/s NRZ, a bit-error ratio down to 6E-9 was achieved for transmission over 500m single-mode fibers.
Key components of the 100Gb/s NRZ transceiver
However, at this high speed, chromatic distortion in the fiber channel becomes the limiting factor for transmission over more than 500m. As a result, bandwidth is decreasing at larger distances, especially at 2km. So, if a transceiver is required that covers all relevant distances in a data center, i.e. 500m, 1km and 2km, a different approach is needed. Therefore, in a second experiment, the team focused on EDB modulation, as it is more resilient to this type of fiber distortion. The EDB modulation format is more complex than NRZ, but provides improved performance for longer distances. In their experiment, the researchers used the bandwidth limitation of the channel, as observed in the first experiment, to shape the signal into a 3-level signal. These 3 levels, denoted as -1, 0 and 1, are translated into three optical intensities in contrast to optical duobinary modulation. The EDB 3-level signal also translates into a different eye diagram – which is a key ‘metric’ for assessing the transmitter’s performance. In an eye diagram, an open eye typically corresponds to a minimal distortion of the signal. While NRZ transmission results in eye diagrams with one series of eyes, the EDB format results in two series of eyes (optimally, each with open eyes). Using this approach, the team was able to demonstrate the first real-time 100Gb/s EDB transmission over more than 2km of single mode fiber.
Bit-error-rate (BER) curves for 100Gb/s NRZ and EDB transmissions (RT = real-time measurement results).
Optical eye diagram at 100Gb/s EDB.
100Gb/s single-lane PAM-4 transmission: meeting the industry standard
Only recently, the industry has adopted PAM-4 as the modulation format of choice for 100Gb/s single-lane transmission over 500m, 1km and 2km distances. Other than NRZ and EDB, which are 2-level and 3-level modulation formats, respectively, PAM-4 is a 4-level modulation format, with levels denoted as 00, 01, 10 and 11 (combining two bits in each level). While NRZ ideally switches between ‘all the light’ (1) or ‘no light’ (0), PAM-4 levels correspond to ‘no light’, ‘a third of the light’, ‘two thirds of the light’ or ‘all the light’. Hence, with this 4-level format, the data rate can be doubled while keeping the same bandwidth.
Generating and receiving PAM-4 at line rates of 112Gb/s has proven very challenging. In general, using a multi-level modulation format leads to reduced eye openings. On top of that, in a single-modulator implementation, the modulator needs to have a linear transfer function. Most types of modulators, however, are non-linear devices. Attempts to solve this issue are mainly limited to the electrical domain, and rely on power-hungry tools as digital signal processing (DSP) and digital-to-analog converters (DACs). This way, they consume significantly more power than their NRZ counter parts at the same data rate.
The team of IDLab and the Photonics Research Group realized a unique solution to enable low-power 112Gb/s PAM-4 transmission. First, instead of going from bits to an analog signal in the electrical domain, they go from bits in the electrical domain to an analog signal in the optical domain. By using these digital-to-optical converters, the digital-to-analog conversion is postponed to the optical domain. And this significantly reduces the complexity and the power consumption at the transmitter side. Second, instead of using a single modulator, they propose a novel transmitter topology based on the addition of 2 parallel 56Gb/s NRZ electro-absorption modulators. By introducing a 33%-66% power split and a 90° phase difference between the two modulators, an equidistant PAM-4 eye can be obtained. Using this innovative solution, the linearity requirement is completely removed from the transmitter electronics and modulators. With this DAC-less and DSP-less transmitter, 112Gb/s PAM-4 transmissions could be obtained over 2km of single mode fiber, with clear open eyes at 112Gb/s.
(Left) proposed topology for the PAM-4 transmitter and (right) optical eye diagram.
Optical and electrical interfaces to inter-connect the optical module
Si photonics provides a highly integrated platform for co-integrating Si waveguides, and active as well as passive components. Electronics circuits can be fabricated on a separate die and assembled with the Si photonics circuits using flip-chip assembly methods. However, the optical module needs to connect to single mode fibers, with low optical loss and acceptable packaging cost. The optical module also needs to interconnect to an application-specific integrated circuit (ASIC) or field-programmable gate array (FPGA) elsewhere in the package through high-density and high-speed electrical interconnects. Therefore, optical and electrical interfaces need to be co-integrated with the high-speed optical components discussed previously, essentially realizing a Si photonics interposer.
Key metrics for the optical interface are the optical coupling efficiency, spectral bandwidth and sensitivity to polarization. For coupling to the single mode fiber, imec has designed, fabricated and tested optically broadband edge couplers, leveraging a hybrid platform consisting of a Si photonic layer combined with a Si nitride photonic layer. Innovations to the layer stack have enabled an optimized coupling efficiency, of better than -2.5dB for O-band operation, and better than -2dB for operation in the C- and L-bands with less than 0.5dB polarization sensitivity.
To enable dense high-speed electrical interconnection over short distances, 10µm x 100µm through-Si vias (TSVs) have been co-integrated in imec’s Si photonics platform on 300mm SOI wafers. They connect the ASIC or FPGA to the CMOS driver and TIA circuits through the Si photonics interposer. Performance measurements reveal high-speed TSVs, with electrical 3dB bandwidth exceeding 50GHz.
Si photonics interposer: system overview.
Conclusion
An overview is given of recent progress in imec’s Si photonics interposer technology, targeting compact, low-power and low-cost transceivers for 400Gb/s and beyond optical interconnects. First, by using imec’s Si photonics platform, high-speed modulators and photodetectors have been developed, capable of operation at 56Gb/s NRZ. Second, with these devices as key building blocks, single-channel data rates up to 112Gb/s have been demonstrated by using PAM-4, an advanced modulation format proposed by industry for 400Gb/s optical links. Besides, we show that both NRZ and EDB provide an elegant and realistic alternative towards 100Gb/s single-lane optical channels. And finally, to enable a true 50GHz Si photonics interposer platform, broadband optical and electrical interfaces are proposed to enable efficient coupling with the single mode fiber and the electrical chips, respectively.
This article was originally published in Photonics Spectra, July 2018.
Want to know more?
- These results have been presented in three invited papers at the Optical Fiber Communications Conference (OFC 2018), held in March 2018 in San Diego. The papers can be requested via this link.
