PCIe Gen5
PCIe Gen5 is the slot generation that sets how fast an adapter card can talk to an IBM Power processor, at 32 GT/s per lane. For an IBM i shop sizing AI work it matters for one narrow reason: every Gen5 slot IBM documents on Power 11 is eight lanes wide, so it carries the same 64 GB/s as a Gen4 sixteen lane slot.
What is PCIe Gen5?
PCIe is short for Peripheral Component Interconnect Express. It is the standard that lets a plug-in card talk to the processor. Picture a small highway between the card and the chip. Each new generation raises the speed limit. Each lane adds another traffic lane. Gen5 doubled the speed limit of Gen4.
PCI-SIG, the group that owns the standard, announced the finished PCIe 5.0 specification on 29 May 2019. It set a raw bit rate of 32 GT/s per lane, which means 32 billion transfers every second, and up to 128 GB/s through a sixteen lane link. It stayed backwards compatible with PCIe 4.0, 3.x, 2.x and 1.x, so an older card still works in a newer slot at the older speed.
Peak bandwidth of a sixteen lane PCIe link, by generation
These are the PCI-SIG headline figures. Each generation doubles the one before it. No Power 11 slot IBM documents is a Gen5 sixteen lane slot, so 128 GB/s is the ceiling of the standard, not what a Power 11 slot hands you.
Which slots on a Power 11 server are Gen5?
Only some of them, and never at full width. IBM prints a description for every physical slot, and that description, not the shape of the connector, is what decides the speed. A slot can have a sixteen lane connector and still be wired for eight lanes. The card drops in and latches; it just gets half the lanes.
Adapter slots IBM lists for the Power 11 9824-22A and 9856-22H
| Slot description IBM prints | How many | Peak bandwidth |
|---|---|---|
| PCIe4 x16 or PCIe5 x8 | 4 | 64 GB/s either way you run it |
| PCIe5 x8 in a sixteen lane connector | 4 | 64 GB/s |
| PCIe4 x8 in a sixteen lane connector | 2 | 32 GB/s |
| eBMC service slot | 1 | Not available for adapters |
Eleven slots in total. The Power E1180 node, machine type 9080-HEU, repeats the pattern across eight slots: four read PCIe4 x16 or PCIe5 x8, two read PCIe5 x8, two read PCIe4 x16. Bandwidth here is the PCI-SIG rate for each generation scaled to the lane count of the slot.
A Gen5 slot does not always run at Gen5 speed. IBM puts this caution on the same slot table: the adapter slot has a maximum speed of Generation4 with system processor CCINs 5CD1 or 5CD3 installed. CCIN is the Custom Card Identification Number, the code that identifies the exact processor module in your server. So the processor you ordered decides whether the Gen5 slots actually run at Gen5.
Does a Gen5 slot make AI inference faster?
Not on its own, and probably not at all for the accelerator you are pricing. The Spyre Accelerator cards IBM sells for Power do not sit in the server's own Gen5 slots. They sit in an expansion drawer that is a Gen4 device end to end.
- IBM documents Spyre Accelerator cards for Power in the ENZ0 PCIe Gen4 I/O expansion drawer, a 4U drawer with up to 12 PCIe Gen4 adapter slots.
- That drawer reaches the server over a single sixteen lane PCIe Gen4 link. One 64 GB/s link is shared by every card in the drawer.
- Trade coverage of the Power announcement describes the Spyre device itself as a PCI-Express 4.0 card.
- Every Gen5 slot inside the server is eight lanes, which peaks at the same 64 GB/s as a Gen4 sixteen lane slot. Gen5 buys the same bandwidth from half the lanes, not more bandwidth.
Descriptions of the Spyre card's PCIe generation are not consistent across sources, and IBM has not published a public card datasheet that spells out the slot requirement. IBM's own ENZ0 documentation is clear that the drawer and its twelve slots are Gen4. Confirm the feature code, the drawer and the slot requirement with IBM or your reseller before you order.
What should I check before ordering an accelerator card?
Four checks before an accelerator card order
- 1. Read the slot description, not the connectorIBM prints slots such as PCIe5 x8 in a sixteen lane connector. The connector is the shape. The lane count is the speed. Go by the printed description in IBM's slot table for your machine type.
- 2. Check the processor featureIBM notes that the slots top out at Gen4 with system processor CCINs 5CD1 or 5CD3 installed. Ask which CCIN is in the server before you count on Gen5 speed.
- 3. Count everything else that needs a slotStorage and network adapters want slots too, and you run out of slots before you run out of anything else. A Virtual I/O Server partition can hand out virtual disk and virtual network so those cards do not each need their own physical slot.
- 4. Ask whether the card lives in the server or in a drawerIBM puts Spyre Accelerator cards for Power in the ENZ0 Gen4 expansion drawer, so the quote is a card order plus a drawer order, and the server's own slot count is not the limit that applies.
Slot counts differ by machine type. The figures on this page come from IBM's slot tables for the 9824-22A, the 9856-22H and the 9080-HEU node.
Related Terms
Spyre Accelerator
IBM's PCIe AI inference card. On IBM Power 11 it carries 32 cores, 128 GB of memory, and more than 300 TOPS inside a 75 watt envelope.
ENZ0 Expansion Drawer
The PCIe4 expansion drawer IBM documents for supported Spyre Accelerator for Power configurations on Power 11 systems.
IBM Power S1122
The Power S1122 is the two-socket 2U IBM Power 11 server, machine type 9824-22A, and IBM lists it for 8 or 12 Spyre AI accelerator cards ... half the ceiling of the larger S1124. For an IBM i shop it is the smaller Power 11 route into on-site model inference, with one trap: only the ERGR processors let an IBM i partition grow past 4 cores.
IBM Power E1180
The Power E1180 is the high-end IBM Power 11 server, machine type 9080-HEU, and the only Power model IBM lists for more than 24 Spyre AI accelerator cards ... up to 48 per system. An IBM i shop can keep production on IBM i 7.4 TR12 or later and run large AI inference in a Red Hat Enterprise Linux partition on the same machine.
Virtual I/O Server
VIOS is a dedicated IBM Power partition that shares one set of physical disk, Fibre Channel and Ethernet adapters out to every other partition on the server. For an IBM i shop that means a Linux partition running an AI model can get its storage and its network connection from VIOS instead of buying its own physical cards.