Resources

Tokens Per Second is Not All You Need

Written by Mingran Wang | May 1, 2024

In this post, we explore why tokens per second doesn't paint the full picture of enterprise LLM inference performance.