The Spyre Card Will Not Start Without Red Hat. Here Is the Whole Stack.

There is a line in IBM's Spyre documentation that changes the shape of a project, and most people reading a card datasheet never reach it. The Spyre Accelerator for Power requires Red Hat AI Inference Server or Red Hat OpenShift AI, running on Red Hat Enterprise Linux 9.6, 9.8, or 10.2.

Not recommends. Requires. If your AI budget has a card in it and no Red Hat line underneath, the budget is not finished.

Five layers between the card and an answer

It helps to see the whole thing at once. Getting from a piece of silicon to a useful answer on IBM Power takes five separate pieces, and you have to own all five.

1. The card

   An IBM Spyre Accelerator. 32 cores, 128 GB of memory, more than 300 TOPS, inside a 75 watt envelope.

2. The drawer

   An ENZ0 PCIe4 expansion drawer. The card does not go in the server body, so this is its own purchase and its own rack space.

3. The operating system

   Red Hat Enterprise Linux 9.6, 9.8, or 10.2, in a Linux partition alongside your IBM i partitions on the same machine.

4. The serving layer

   Red Hat AI Inference Server or Red Hat OpenShift AI. This is the piece IBM names by product, and it is a subscription.

5. The model

   Something for the card to actually run. IBM's own Granite family is open and free, which makes it the usual starting point.

Miss any one of the five and nothing happens. The card is the only part most proposals mention.

What the serving layer is actually for

Picture a very fast kitchen with one chef. Ten orders come in at once. A naive system cooks them strictly in turn, so nine people sit and wait while the kitchen mostly idles between plates. A good system notices that six of those orders need the same preparation and runs them together.

That is inference serving in one image. Red Hat AI Inference Server accepts requests, packs them onto the accelerator sensibly, manages the memory each conversation needs while it is in progress, and returns the answers. Without it, a 128 GB card is a very expensive way to serve one person at a time.

The open engine underneath, and why it matters to your card count

Red Hat did not build this from nothing. Red Hat AI Inference Server is built on vLLM, an open source serving engine, with optimisation work from the Neural Magic team folded in after Red Hat acquired them. Two of vLLM's ideas have a direct effect on how much hardware you need to buy.

The first is called PagedAttention. A model keeps a working memory of the conversation so far. Reserving the worst-case amount of memory for every user, just in case, wastes most of the card. PagedAttention hands out small blocks as they are actually needed. Anyone who has worked with paged memory on a real operating system will recognise this immediately, because it is the same idea.

The second is continuous batching. Instead of waiting for a neat group of requests to form before starting, new requests join a batch that is already running. The card stops idling between tidy queues.

These are not academic points. Two shops can buy identical hardware and get very different capacity from it, purely because of how the serving layer is configured. That is why the software decision belongs in the hardware conversation and not after it.

Inference Server or OpenShift AI

What this does to a first budget

The uncomfortable summary is that the card is often the cheap part. A realistic first Spyre project carries a card, an expansion drawer, a Red Hat Enterprise Linux subscription, a Red Hat AI subscription, and the people who will keep all of it patched and supported. The model, if it is Granite, is the only line that costs nothing.

This is also the strongest argument for starting on the Matrix Math Acceleration already built into your Power 10 or Power 11 processor. It needs no card, no drawer, and no new power circuit. It will not carry a heavy production load, but it will tell you whether the idea is worth the rest of the stack before you buy any of it.