Run large language models at home, BitTorrent‑style

Petals is a platform that allows users to run large language models like Llama 3.1 and Mixtral on consumer-grade GPUs by distributing model parts across a peer-to-peer network, similar to BitTorrent. Users can generate text and fine-tune models while maintaining the flexibility of PyTorch, achieving inference speeds of 4-6 tokens per second suitable for chatbots and interactive applications.