About Petals
The Petal tool is a decentralized platform that runs a large language model like Bloom-176b. It is capable of loading small parts of the model to run inference and fine-tuning. Single-batch inference takes approximately 1 second per step (token) and can run parallel inference up to hundreds of tokens/sec. This tool offers more than just a classic language model API, using fine-tune sampling methods and allowing users to execute custom paths and see hidden states. Petal also offers flexible PyTorch API. It is part of the BigScience research workshop project.
No review yet!
More Development AI Tools
Explore other development tools with user ratings, pricing details, and in-depth descriptions. Updated regularly by our editorial team.
