tokens per second

A system using 768GB of Intel Optane DIMM memory successfully ran a 1-trillion-parameter LLM with a single GPU, achieving 4 tokens per second performance. This demonstrates how alternative memory architectures can enable large AI model deployment without massive GPU clusters, potentially reducing AI infrastructure costs.