vLLM
The open source engine inside the product IBM requires for a Spyre card. Worth knowing by name, because it is the reason the same card can answer many people at once instead of queuing them.
vLLM is the open source project that does the actual work of serving a model. Red Hat wraps it, supports it, and sells it as Red Hat AI Inference Server, which is the product name IBM prints in the Spyre requirements. Knowing the engine's real name is useful when you are reading documentation or hiring.
The two ideas that make it matter
The first is PagedAttention. A model holds a running memory of the conversation so far, and that memory is expensive. Reserving the worst-case amount for every user wastes most of the card. PagedAttention hands out small blocks as they are needed, which is exactly what an operating system does with paged memory. Any AS/400 person will recognise the idea immediately.
The second is continuous batching. Rather than waiting for a full group of requests before starting work, new requests join the batch while it is already running. The card never sits idle waiting for a tidy queue to form.
Why an infrastructure buyer should care
These are not academic details. They decide how many concurrent users one 128 GB Spyre card supports, and therefore whether the project needs one card or four. Two shops can buy identical hardware and get very different capacity out of it purely from how the serving layer is configured.
You can run vLLM yourself without a subscription. For a supported production configuration on IBM Power with a Spyre card, IBM names Red Hat AI Inference Server or Red Hat OpenShift AI, not a self-assembled vLLM install.
Related
Red Hat AI Inference Server
One of the two products IBM names as required before a Spyre card will do anything. It is the driver, the traffic controller, and the reason a 128 GB card can serve more than one person at a time.
Red Hat OpenShift AI
The other product IBM accepts for driving a Spyre card. It is the heavier option: a full container platform for teams running several models, several projects, and more than one server.
IBM Granite Models
IBM's own family of open language models, and the cargo an accelerator card is built to carry. You can download them free, run them on your own hardware, and nobody meters the tokens.
Red Hat Enterprise Linux AI (RHEL AI)
A bootable Linux image with IBM Granite models and the InstructLab tuning tools already inside. It is the shortest route from bare metal to a working model, and it is where Granite and Red Hat stop being two separate decisions.