Signed in
We value your privacy
This site uses cookies to improve your browsing experience, analyze site traffic, and show personalized content.
| No holders yet. | ||||||||||
AirLLM optimizes inference memory usage, allowing 70B large language models to run inference on a single 4GB GPU card without quantization, distillation and pruning. And you can run 405B Llama3.1 on 8GB vram now.