NorthPole, the AIU, and Where Your Spyre Card Came From

There is a reason the Spyre Accelerator behaves the way it does, and it is not marketing. The card is the fourth step in a line of work that started with a very specific problem in banking, and every step in that line has made the same argument in a slightly better way.

Understanding the family tree is useful for a practical reason: it tells you what the card is actually good at, and therefore whether it fits the job you have in mind.

The deadline that started it

Banks wanted to know whether a card payment was fraudulent while the payment was still happening. Not in a batch report that evening. During the transaction, in the time it takes a card machine to say approved.

That is a brutal deadline, and it broke the normal way of doing things. The normal way is to send the data off to a separate machine that runs AI models and wait for a reply. The journey there and back was the problem. So IBM stopped making the journey and put the AI engine directly on the mainframe processor. That chip is Telum, and its successor Telum II runs eight cores at 5.5 GHz with 360 MB of cache, an on-chip AI accelerator, and a data processing unit to handle input and output.

Scaling the engine up

An AI engine sitting inside a processor core is small by definition. It shares its home with everything else. IBM Research asked the obvious next question: what if the whole chip was that engine?

The answer was the Artificial Intelligence Unit, usually just called the AIU. Thirty-two cores, closely related to the AI engine already proven inside Telum, each with its own small block of fast scratchpad memory rather than all of them queuing for one shared pool. It was a research chip and was never sold.

It did not need to be sold, because it became the product. The Spyre Accelerator is that design turned into something you can order: the same 32 cores, the same per-core scratchpad arrangement, built on a 5 nm process with 25.6 billion transistors and 128 GB of memory on the card.

NorthPole, and removing the walk entirely

While the AIU was heading towards production, a different group at IBM Research pushed the same idea considerably further. Almost every computer ever built keeps memory in one place and compute in another, and burns most of its electricity moving data between them.

NorthPole's answer is blunt. It has no off-chip memory at all. The memory is woven through the compute, so the data never travels. Picture a kitchen where the ingredients sit at each station rather than in a walk-in fridge down the corridor. The cooking is not faster because the chefs got quicker. It is faster because the walking disappeared.

The chip carries 22 billion transistors on a 12 nm process, which is a deliberately unfashionable process node, and that is part of the point. The efficiency comes from the architecture, not from the smallest available transistors. In one demonstration it handled dense high-resolution video and identified multiple classes of object on roughly 5 watts, which is less than a phone charger draws.

In September 2024 IBM presented results at the IEEE High Performance Extreme Computing conference showing NorthPole running a 3 billion parameter Granite language model. That detail closes the circle: the research silicon and the open model family IBM ships with Red Hat are part of the same programme.

Four chips, one argument

What the family tree tells a buyer

Two useful things, and one warning.

The first useful thing is what the card is for. This entire family was bred for answering quickly, next to the data, inside a tight power budget. It was not bred for training large models from scratch. If your project is training rather than answering, this hardware family is the wrong shape and no future generation will change that.

The second is where IBM is betting. More than 300 TOPS inside 75 watts is a low-power design decision, not a brute-force one. NorthPole says the same thing much louder. If your concern about AI hardware is the electricity bill and the rack space, IBM's research programme is pointing at your concern rather than away from it.