Tag

tpu

Oldest Newest URL A-Z URL Z-A Title A-Z Title Z-A Random
How To Scale Your Model
https://jax-ml.github.io/scaling-book/

A Systems View of LLMs on TPUs.

This book aims to demystify the art of scaling LLMs on TPUs. We try to explain how TPUs work, how LLMs actually run at scale, and how to pick parallelism schemes during training and inference that avoid communication bottlenecks.

 
ebook e-learning foss llm machine-learning open-source tpu
Added 7 months ago