IBM Granite Models
IBM's own family of open language models, and the cargo an accelerator card is built to carry. You can download them free, run them on your own hardware, and nobody meters the tokens.
An accelerator card is an empty lorry. Granite is what you put in it.
Granite is IBM's own family of language models, and the important detail is the licence. They are released under Apache 2.0, which is the plain permissive open licence. Download them, run them on your own server, change them, put them in production. No per-token bill, no seat count, no renewal conversation.
The two releases that matter right now
Granite 4.0 and Granite 4.2
| Granite 4.0 | Granite 4.2 | |
|---|---|---|
| Released | October 2025 | August 25, 2026 |
| Headline change | Hybrid Mamba-2 and transformer design | Reasoning you can switch on or off |
| Sizes | 3B dense, 7B with 1B active, 32B with 9B active | 3B, 8B, 30B |
| Licence | Apache 2.0 | Apache 2.0 |
| Why you would care | More than 70 percent less memory, about twice the inference speed | Better tool calling for agent work, and step-by-step reasoning on demand |
Both remain available. Newer does not automatically mean better for a given job.
The memory point, in plain terms
A Spyre card has 128 GB of memory on it. That is a fixed ceiling. Granite 4.0's hybrid design cut memory use by more than 70 percent for long conversations and for serving many users at once. Less memory per request means more requests fit on the same card, which means the card you bought does more work. That is the whole argument in one sentence.
Granite is a model family, not an IBM i feature. It does not run inside IBM i and it does not read Db2 for i on its own. It runs in a Linux partition, under a serving engine, and something has to connect it to your data. Check the current model card for context length and hardware guidance before sizing.
Related
Red Hat AI Inference Server
One of the two products IBM names as required before a Spyre card will do anything. It is the driver, the traffic controller, and the reason a 128 GB card can serve more than one person at a time.
InstructLab
The open source project that lets the person who knows your business teach a model, without that person being a data scientist. For a shop whose knowledge lives in three people's heads, this is the interesting one.
IBM Spyre Accelerator for Power
A 75 watt PCIe card that adds more than 300 TOPS of AI inference to a Power 11 server. It is the only IBM AI chip an AS/400 shop can actually order as an add-on, and it comes with a short list of things it refuses to work without.
Red Hat Enterprise Linux AI (RHEL AI)
A bootable Linux image with IBM Granite models and the InstructLab tuning tools already inside. It is the shortest route from bare metal to a working model, and it is where Granite and Red Hat stop being two separate decisions.