TOPS
Trillions of operations per second. A rough headline rating for AI hardware, useful for comparison and misleading on its own.
TOPS counts trillions of operations per second and is the figure vendors put on the front of an accelerator datasheet. IBM publishes more than 300 TOPS for the Spyre Accelerator inside a 75 watt envelope. The number is genuinely useful for comparing cards from the same family, and genuinely misleading when used alone, because real throughput depends on the model, the number precision it uses, the memory available, and how the serving layer batches work. A card with a high TOPS rating and a badly configured serving layer will lose to a modest card that is kept busy.
Related Terms
Spyre Accelerator
IBM's PCIe AI inference card. On IBM Power 11 it carries 32 cores, 128 GB of memory, and more than 300 TOPS inside a 75 watt envelope.
Quantization
Storing a model's numbers with less precision so it fits in less memory and runs faster, at some cost to accuracy.
Inference serving
The software layer that takes requests for a model, packs them onto the hardware efficiently, and returns answers. It decides how much work one accelerator card can actually do.