Tokasaurus: An LLM Inference Engine for High-Throughput Workloads

[rsehrlich]

Article URL: https://scalingintelligence.stanford.edu/blogs/tokasaurus/

Comments URL: https://news.ycombinator.com/item?id=44195961

Points: 77

# Comments: 6