Key Takeaways:
CPO and NPO require new approaches for connecting optical fibers to chip packages.
Detachable connectors can simplify assembly, test, and replacement of optical components.
Numerous bespoke designs exist today, but standardization hasn’t begun yet.
As photonics shifts from pluggable optical transceivers toward near- and co-packaged optics (NPO, CPO), engineers are devising new ways to attach fibers to the photonics chips on a board. Several different designs exist today, but so far there is no common interface.
With pluggable photonics, lasers and connections are attached at the server faceplate, with the board itself housing only electronic components. Yet long electrical connections to the faceplate mean higher power and longer latency. Fibers would need to move onto the interior of the board to solve that. That eliminates the need for pluggables, but it requires new solutions.
“With the rapid growth of generative AI, AI data centers are required to process enormous volumes of data faster and more efficiently than ever before,” said Yoshimasa Bando, principal scientist at Mitsubishi Chemical. “To meet this demand, further improvements in communication speed and energy efficiency have become critical challenges.”
The goal of CPO and NPO is to reduce reliance on long electrical connections.
Long lines
One reason for maintaining faceplate pluggables is that lasers are not always reliable. Using a pluggable module makes it easy to replace the laser.
“What we are focused on has the laser on the faceplate, because the laser is the highest reliability risk,” said Vishal Chandrasekar, director of product management at Ayar Labs.
All connections, whether they have a laser or not, happen in pluggable modules today. That means the signals those fibers carry must travel electrically from their origin to the edge of the board. Bandwidth necessitates serial lines driven by serializer/deserializer (SerDes) circuits. Driving those signals over that distance requires high power.
CPO aims to reduce system power by eliminating those long electrical lines and bringing fiber into the processor or switch packages. “CPO unlocks the token economics that everyone cares about,” said Chandrasekar. “The major bottlenecks are compute, memory, and moving data back and forth. If you’re trying to make step changes in hardware every one or two years, the way you make 5× or 10× kinds of improvements is by having new hardware. How do you use that hardware intelligently? How do I ensure that whatever hardware I have, I’m using it for the most important problem so my utilization is good?”
Fig. 1: With co-packaged optics, an optical engine is placed inside a package along with other chiplets. The laser remains outside the package for thermal reasons, and in the near term it will remain pluggable. Source: Bryon Moyer/Semiconductor Engineering
Photonics plays a key role in this effort. “You’re trying to reduce copper, so you’re trying to bring the PIC [photonic integrated circuit] closer to the processor or the switch,” said David Harame, COO at AIM Photonics, which operates with NY Creates (New York Center for Research, Economic Advancement, Technology, Engineering, and Science). “The industry is still very much using pluggables; you plug in the front panel. So there’s a lot of copper going between this [and the processor]. The next step is to move the optical transceiver closer to the switch by putting it on the PCB and reducing the copper line length to the switch. This is called near-packaged optics. Essentially, it moves the optical pluggable connector further inside the box on the PC board, so the optical-to-electrical connection takes place closer to the switch.
CPO is yet another variant of advanced packaging. “And all the challenges in advanced packaging are the same for co-packaged optics, except for fiber coupling and getting the PIC closer to the target chip,” noted Harame.
CPO receives the bulk of the near-term optical development efforts.
Fig. 2: In near-packaged optics, the optical components lie on the board near where their signals originate. Lasers will remain pluggable for now. Source: Bryon Moyer/Semiconductor Engineering
Unknown fiber length
Both CPO and NPO share the need to connect fiber directly to components on the board. One straightforward way is to attach a nominal length of fiber to the chip package and ship it that way, with the system integrator connecting the other end of the fiber directly to the destination.
The challenge there is that chip producers have no way of knowing how long that fiber will need to be. If it’s long enough for designs that require long runs, it will be too long for the many designs that don’t. That said, there are ways of dealing with extra fiber.
“The fiber goes into a fiber management tray, and there are spool points on the board,” said Nick Harris, CEO of Lightmatter. “They’re neatly managed for airflow.”
Others agree. “Usually, you’ll have some kind of excess fiber management,” explained Obie Roy, vice president, strategic commercial development at Senko. “But depending on the system that you’re building, you could cut it to length and terminate it once you know the exact application.”
Some engineers see this as a non-issue, which may be true if one is fusing the fibers or having the integrator attach a connector. “The fiber length is irrelevant,” said Professor Amr Helmy at the University of Toronto. “You connect a strand of fiber to the chip. With the other end, you can connectorize it, cut it, or fuse it with another fiber to make it longer. Fusing is one approach, but each one of these fibers can come with a connector depending on the amount of loss appetite you have.”
For ease of assembly, some means of attaching and detaching a fiber would also be helpful, such as a fixture or clip on the end to which another fiber can attach. But several considerations make this decision about more than simply adding wire connections as shown in the figures above.
Moving beyond the die edge
Today, fibers typically are placed using well-established V-groove technology, where the fibers exit out the side of a photonic chip. This attachment method works at low volumes and low total fiber counts, but it lacks the modularity and convenience needed for high-volume automation.
“The V-groove is a great concept, but the fibers are attached,” explained Gerard John, senior director, chiplets/FCBGA business unit at Amkor. “When being assembled onto a board, if one of them breaks, there’s no rework. People are generally thinking in terms of detachables and moving away from V-grooves.”
In addition, attachment space is bound by shoreline, which is essentially the real estate on the edge of the die. With metal connections, it might include the first few rows of bumps on the package, but fiber footprint is bigger and can chew up a lot of shoreline.
“The fiber diameter is 10 µm, but you have all the cladding around it, making it 125 µm,” John said. “So if you’re putting, let’s say, 16 cables like that, even if it’s staggered you will use a lot of real estate.”
Nevertheless, this is the predominant approach today. For example, Senko has put together a LEGO-like connector, with one part glued to the chip and the other attached to a fiber. This is one of many approaches underway at different companies.
There are three configurations for bringing fiber out (see figure 3, below). One takes the light from a waveguide, reflects it upward 90°, and sends it horizontally to a connector. These connectors would be situated around the periphery of the die.
Another is simply to run the waveguide to the edge of the die and have a connector butt up to the die to capture the light and route it to a fiber. The number of such connections is also determined by the perimeter.
A third approach moves the light vertically but then connects to the top surface rather than going out horizontally. This makes the entire area of the chip available for connections, providing more “shoreline” than the other two schemes.
“You have the waveguide shooting out laterally, and then it hits a mirror at a 45° angle,” said John. “It shoots upward, and once you get it upward, you put a connector with all the fibers on it.”
Fig. 3: The three ways of getting a waveguide signal into a fiber — horizontal-to-vertical-to-horizontal, fully horizontal, and horizontal-to-vertical. Source: Bryon Moyer/Semiconductor Engineering
But vertical approaches must avoid any heat sinks or other elements attached to the top. “When you send it vertically, you may get in the way of your thermal solution,” said Ravi Mahajan, fellow at Intel Foundry. “So if you have a solution that does both vertical and lateral, then you have more flexibility. In cases where thermal is a potential problem, go horizontal. In cases where you don’t worry about it, you can go vertical.”
The problem with pigtails
As depicted above, these connectors could be short pieces of fiber — pigtails — that are affixed firmly to the die and with some sort of clip connector at the end. A matching connector clip could then be plugged in to make the connection. Such a clip would likely cause some insertion loss, but if managed well, it can work. And it can be made to be sturdy.
“We have taken a part with a pigtail and banged it on the desk to convince a VP that it doesn’t come off,” said Mahajan. “We expose it to temperature cycles, to a little bit of moisture, and bake it for a long time, and it doesn’t come off.”
But the greater concerns are the high-volume manufacturing implications. Even without considering how the fiber attaches to the die, a more immediate question is how these chips would be packaged in bulk for test and delivery. The same issue holds for shipping a longer fiber and cutting it to order during integration.
Today, chips go into trays, and automated handling equipment can pick and place them onto boards or into test heads or anywhere else they’re needed. What happens if it’s no longer just a chip in the tray pocket, but potentially a chip wrapped with pigtails?
“If you have a pigtail, it’s very difficult to wrap that pigtail around and put it in a tray, and you probably won’t have many parts on the tray,” said John. “So our choice is to have a detachable connector, and it also makes sense from a manufacturing perspective.”
In addition, it is inconvenient to try to place a chip with pigtails into such a tray in an orderly, repeatable way. And then, when pulling the chip out of the tray, it will have dangling pigtails floating about, potentially interfering with whatever operation the equipment is attempting. Then, presumably, those pigtails would be manually connected.
This arrangement, while suitable at low volumes, may not work in high-volume production.
“I understand people being worried about a pigtail dangling off the edge, so we decided, ‘Why not just bring it to the edge of the die,’” said Mahajan. “Then an integrator just attaches it [to the fiber].”
Fig. 4: A fiber plugs into the side of a package. Source: Intel Foundry
Old habits change slowly, however. “Pigtails are probably still more common because people are trying to make incremental changes,” noted Sanjay Gangadhara, senior architect at Synopsys.
Scaling volumes also must address the assembly of any connector on the package. “Terminating a fiber is complex,” said Badri Gomatam, CTO of STL. “You’ve got to make it low cost, high yield, and repeatable. You’ve got hundreds of connections. No speck of dust can be allowed, and the assembly has to survive high-temperature reflow and all the other growth steps.”
Alignment concerns
One of the unique and challenging characteristics of photonics is that where two conductors meet, as much light as possible must get from one to the other. That applies…
AI Brief
Your highlights
Key Takeaways:
CPO and NPO require new approaches for connecting optical fibers to chip packages.
Detachable connectors can simplify assembly, test, and replacement of optical components.
Numerous bespoke designs exist today, but standardization hasn’t begun yet.
As photonics shifts from pluggable optical transceivers toward near- and co-packaged optics (NPO, CPO), engineers are devising new ways to attach fibers to the photonics chips on a board. Several different designs exist today, but so far there is no common interface.
With pluggable photonics, lasers and connections are attached at the server faceplate, with the board itself housing only electronic components. Yet long electrical connections to the faceplate mean higher power and longer latency. Fibers would need to move onto the interior of the board to solve that. That eliminates the need for pluggables, but it requires new solutions.
“With the rapid growth of generative AI, AI data centers are required to process enormous volumes of data faster and more efficiently than ever before,” said Yoshimasa Bando, principal scientist at Mitsubishi Chemical. “To meet this demand, further improvements in communication speed and energy efficiency have become critical challenges.”
The goal of CPO and NPO is to reduce reliance on long electrical connections.
Long lines
One reason for maintaining faceplate pluggables is that lasers are not always reliable. Using a pluggable module makes it easy to replace the laser.
“What we are focused on has the laser on the faceplate, because the laser is the highest reliability risk,” said Vishal Chandrasekar, director of product management at Ayar Labs.
All connections, whether they have a laser or not, happen in pluggable modules today. That means the signals those fibers carry must travel electrically from their origin to the edge of the board. Bandwidth necessitates serial lines driven by serializer/deserializer (SerDes) circuits. Driving those signals over that distance requires high power.
CPO aims to reduce system power by eliminating those long electrical lines and bringing fiber into the processor or switch packages. “CPO unlocks the token economics that everyone cares about,” said Chandrasekar. “The major bottlenecks are compute, memory, and moving data back and forth. If you’re trying to make step changes in hardware every one or two years, the way you make 5× or 10× kinds of improvements is by having new hardware. How do you use that hardware intelligently? How do I ensure that whatever hardware I have, I’m using it for the most important problem so my utilization is good?”
Fig. 1: With co-packaged optics, an optical engine is placed inside a package along with other chiplets. The laser remains outside the package for thermal reasons, and in the near term it will remain pluggable. Source: Bryon Moyer/Semiconductor Engineering
Photonics plays a key role in this effort. “You’re trying to reduce copper, so you’re trying to bring the PIC [photonic integrated circuit] closer to the processor or the switch,” said David Harame, COO at AIM Photonics, which operates with NY Creates (New York Center for Research, Economic Advancement, Technology, Engineering, and Science). “The industry is still very much using pluggables; you plug in the front panel. So there’s a lot of copper going between this [and the processor]. The next step is to move the optical transceiver closer to the switch by putting it on the PCB and reducing the copper line length to the switch. This is called near-packaged optics. Essentially, it moves the optical pluggable connector further inside the box on the PC board, so the optical-to-electrical connection takes place closer to the switch.
CPO is yet another variant of advanced packaging. “And all the challenges in advanced packaging are the same for co-packaged optics, except for fiber coupling and getting the PIC closer to the target chip,” noted Harame.
CPO receives the bulk of the near-term optical development efforts.
Fig. 2: In near-packaged optics, the optical components lie on the board near where their signals originate. Lasers will remain pluggable for now. Source: Bryon Moyer/Semiconductor Engineering
Unknown fiber length
Both CPO and NPO share the need to connect fiber directly to components on the board. One straightforward way is to attach a nominal length of fiber to the chip package and ship it that way, with the system integrator connecting the other end of the fiber directly to the destination.
The challenge there is that chip producers have no way of knowing how long that fiber will need to be. If it’s long enough for designs that require long runs, it will be too long for the many designs that don’t. That said, there are ways of dealing with extra fiber.
“The fiber goes into a fiber management tray, and there are spool points on the board,” said Nick Harris, CEO of Lightmatter. “They’re neatly managed for airflow.”
Others agree. “Usually, you’ll have some kind of excess fiber management,” explained Obie Roy, vice president, strategic commercial development at Senko. “But depending on the system that you’re building, you could cut it to length and terminate it once you know the exact application.”
Some engineers see this as a non-issue, which may be true if one is fusing the fibers or having the integrator attach a connector. “The fiber length is irrelevant,” said Professor Amr Helmy at the University of Toronto. “You connect a strand of fiber to the chip. With the other end, you can connectorize it, cut it, or fuse it with another fiber to make it longer. Fusing is one approach, but each one of these fibers can come with a connector depending on the amount of loss appetite you have.”
For ease of assembly, some means of attaching and detaching a fiber would also be helpful, such as a fixture or clip on the end to which another fiber can attach. But several considerations make this decision about more than simply adding wire connections as shown in the figures above.
Moving beyond the die edge
Today, fibers typically are placed using well-established V-groove technology, where the fibers exit out the side of a photonic chip. This attachment method works at low volumes and low total fiber counts, but it lacks the modularity and convenience needed for high-volume automation.
“The V-groove is a great concept, but the fibers are attached,” explained Gerard John, senior director, chiplets/FCBGA business unit at Amkor. “When being assembled onto a board, if one of them breaks, there’s no rework. People are generally thinking in terms of detachables and moving away from V-grooves.”
In addition, attachment space is bound by shoreline, which is essentially the real estate on the edge of the die. With metal connections, it might include the first few rows of bumps on the package, but fiber footprint is bigger and can chew up a lot of shoreline.
“The fiber diameter is 10 µm, but you have all the cladding around it, making it 125 µm,” John said. “So if you’re putting, let’s say, 16 cables like that, even if it’s staggered you will use a lot of real estate.”
Nevertheless, this is the predominant approach today. For example, Senko has put together a LEGO-like connector, with one part glued to the chip and the other attached to a fiber. This is one of many approaches underway at different companies.
There are three configurations for bringing fiber out (see figure 3, below). One takes the light from a waveguide, reflects it upward 90°, and sends it horizontally to a connector. These connectors would be situated around the periphery of the die.
Another is simply to run the waveguide to the edge of the die and have a connector butt up to the die to capture the light and route it to a fiber. The number of such connections is also determined by the perimeter.
A third approach moves the light vertically but then connects to the top surface rather than going out horizontally. This makes the entire area of the chip available for connections, providing more “shoreline” than the other two schemes.
“You have the waveguide shooting out laterally, and then it hits a mirror at a 45° angle,” said John. “It shoots upward, and once you get it upward, you put a connector with all the fibers on it.”
Fig. 3: The three ways of getting a waveguide signal into a fiber — horizontal-to-vertical-to-horizontal, fully horizontal, and horizontal-to-vertical. Source: Bryon Moyer/Semiconductor Engineering
But vertical approaches must avoid any heat sinks or other elements attached to the top. “When you send it vertically, you may get in the way of your thermal solution,” said Ravi Mahajan, fellow at Intel Foundry. “So if you have a solution that does both vertical and lateral, then you have more flexibility. In cases where thermal is a potential problem, go horizontal. In cases where you don’t worry about it, you can go vertical.”
The problem with pigtails
As depicted above, these connectors could be short pieces of fiber — pigtails — that are affixed firmly to the die and with some sort of clip connector at the end. A matching connector clip could then be plugged in to make the connection. Such a clip would likely cause some insertion loss, but if managed well, it can work. And it can be made to be sturdy.
“We have taken a part with a pigtail and banged it on the desk to convince a VP that it doesn’t come off,” said Mahajan. “We expose it to temperature cycles, to a little bit of moisture, and bake it for a long time, and it doesn’t come off.”
But the greater concerns are the high-volume manufacturing implications. Even without considering how the fiber attaches to the die, a more immediate question is how these chips would be packaged in bulk for test and delivery. The same issue holds for shipping a longer fiber and cutting it to order during integration.
Today, chips go into trays, and automated handling equipment can pick and place them onto boards or into test heads or anywhere else they’re needed. What happens if it’s no longer just a chip in the tray pocket, but potentially a chip wrapped with pigtails?
“If you have a pigtail, it’s very difficult to wrap that pigtail around and put it in a tray, and you probably won’t have many parts on the tray,” said John. “So our choice is to have a detachable connector, and it also makes sense from a manufacturing perspective.”
In addition, it is inconvenient to try to place a chip with pigtails into such a tray in an orderly, repeatable way. And then, when pulling the chip out of the tray, it will have dangling pigtails floating about, potentially interfering with whatever operation the equipment is attempting. Then, presumably, those pigtails would be manually connected.
This arrangement, while suitable at low volumes, may not work in high-volume production.
“I understand people being worried about a pigtail dangling off the edge, so we decided, ‘Why not just bring it to the edge of the die,’” said Mahajan. “Then an integrator just attaches it [to the fiber].”
Fig. 4: A fiber plugs into the side of a package. Source: Intel Foundry
Old habits change slowly, however. “Pigtails are probably still more common because people are trying to make incremental changes,” noted Sanjay Gangadhara, senior architect at Synopsys.
Scaling volumes also must address the assembly of any connector on the package. “Terminating a fiber is complex,” said Badri Gomatam, CTO of STL. “You’ve got to make it low cost, high yield, and repeatable. You’ve got hundreds of connections. No speck of dust can be allowed, and the assembly has to survive high-temperature reflow and all the other growth steps.”
Alignment concerns
One of the unique and challenging characteristics of photonics is that where two conductors meet, as much light as possible must get from one to the other. That applies to assembly and test.
Today, this is done through active alignment, where a tool measures the light exiting a waveguide and positions a fiber to maximize the captured light. This is a six-degree-of-freedom problem, where the aligner must be able to move in three linear dimensions as well as three angles of approach. It can take time.
With the pigtail configuration, presumably the clips would be designed to provide the alignment, but you still need to affix the stationary end of the pigtail during assembly. Given the downsides of pigtails, an alternative is to build the connector into the package itself. That way, nothing is hanging off the package, and it can be handled like other chips.
But alignment is still necessary. One method places a lens over the optical TSV to focus the light into an attached fiber while the other connection hardware aligns.
“One of the advantages when you have a lens is that, even if you do have some misalignment, the lens can correct for some of that because it’s focusing the light into a common region,” said Synopsys’ Gangadhara. “So this idea of using micro-optics, whether in a vertical-coupling or an edge-coupling architecture, is becoming more and more common.”
The Senko technique, for instance, provides passive mechanical fine alignment and restraint when snapped together. Roy explained that the snap-on structure provides the fine alignment, but you need something to hold it down. “We have screws and a structure that pushes it down into the fine alignment. This is the mechanical latching.”
It’s one thing to perform alignment during assembly, but it’s another to require it when attaching system fibers to the chip. Magnetic connectors, such as those used by Apple on iPhones, are being considered. Once the receiver end is built into the package, someone (or an automated handler) attaching the fiber must first align it roughly in place. From there, the magnet pulls it into the necessary precise location.
Unfortunately, some magnetic connectors are easily damaged. Any acceptable CPO fiber connection must be firm.
Stronger magnets could help, and this is an active area of development, possibly along with a mechanical restraint (such as Senko’s screws). What’s unclear, however, is how strong that magnetic field would be and whether it may adversely affect surrounding components such as MRAMs.
“When you design your system board, are you going to put sensitive memory close to these devices?” mused John.
As a result, any mechanical alignment would likely also be supplemented with a latch.
Serviceability and testability are critical
One way to connect fibers, common in long-haul optical applications, is to literally fuse them together. This permanent attachment method does not allow detachment, so it provides poor serviceability if a photonics chip fails.
“People have this concern about any kind of failure,” said Gangadhara. “How do you replace it? That serviceability part of the RAS requirement becomes a challenge.”
Serviceability, reliability, and availability make up the RAS feature that’s in high demand these days. It means components at greater risk of failure must be easy to replace, so you don’t have to throw out an entire expensive module or server.
“Once you attach a pigtail, if it gets damaged during shipment, the whole thing needs to be thrown out,” said John.
Roy agreed. “You need to make it detachable because if you permanently attach it and the fiber breaks, you have to fail the entire module out, which is very expensive,” he said.
Testing is heavily affected by these connector types, especially without standards. “An important aspect is making sure your solution is testable in real life,” noted Roy.
Addressing testability, especially for future high-volume manufacturing, means collaborating with test-equipment manufacturers. “We worked with Advantest and developed a test connector made out of tungsten carbide,” explained Roy. “It has the same bumps [as the LEGO-like connector] on the other side, and so it can come down, align precisely, and test very quickly. And you can scale it to test one at a time or a row of these things.”
Critically, the connection doesn’t require active alignment, saving test time. “We demonstrated 10,000 matings with very little variation in the loss [through the connection],” said Roy.
The connectors themselves must be reliable enough, but only up to a point. “Ideally, you’re not detaching and reattaching frequently,” noted Gangadhara. “If these things are up and running and working, then they’re running for thousands or tens of thousands of hours. Meta published a study last year where they had done serviceability/reliability analysis of some CPO devices, and they showed 10,000 hours of constant usage without any degradation of signal integrity or performance.”
Settling on a common design
Connector attachment is a textbook case of a development that needs standards so that any user can plug into anyone else’s photonics chip. At present, there are numerous pigtail-based designs and no standards. Likewise, no magnetic (or other) pigtail-less approach has been standardized or proven in high volumes. Customer requirements also appear to vary in ways that may make standardization difficult.
Consider the mechanical latching of Senko’s solution, for example. “Each of these parts is custom, depending on the ganging requirements and the mechanical sizes that are important to avoid — cooling plates and thermal elements and such,” Roy said. “Our vision is to take the receptacle part [that gets attached to the package], make that a common interface of the chip, and then sell the cover plate to any FAU manufacturer.”
For that to become an actual standard, however, typical licensing will presumably need to be reasonable and non-discriminatory (RAND). That’s a situation where different connector companies will work closely with photonics companies to find an approach that works functionally and is suited to high-volume manufacturing. That means compatibility with handling equipment.
These efforts will likely continue until standards (or at least multi-source agreements, or MSAs) are put in place, removing one major photonics volume barrier once the solution is established.
Related Articles
CPO Is Extending The Limits Of What’s Possible In AI Data Centers
Co-packaged optics technology will have a big impact on system power and the cost of data movement.
Making On-Chip Photonics Manufacturable
AI systems are pulling optics closer to logic, but scalable manufacturing will require front-end fabrication, packaging, thermal management, materials, and test to evolve together.
Designing Electro-Optical Chips
Photonics is forcing EDA to verify not just chips, but the physics of light inside full electro-optical systems