BentoML
Open source framework and cloud platform for serving AI models
About BentoML
BentoML maintains an open source framework for packaging and serving ML models and AI applications, and BentoCloud, a managed platform for deploying them with GPU inference, autoscaling and monitoring. It supports popular ML frameworks and open source LLMs.
Common use cases
- Serving ML models
- LLM inference
- Batch inference jobs
- Multi-model pipelines
Products and services
BentoML
Open source framework for building and serving AI applications.
BentoCloud
Managed platform for deploying and operating BentoML services.
Key features
- Model packaging
- REST and gRPC serving
- Adaptive batching
- Autoscaling to zero
- GPU inference
- Model monitoring
Integrations
- PyTorch
- TensorFlow
- Hugging Face
- vLLM
- Kubernetes
- AWS
BentoML alternatives
SeldonMLOps platform for deploying, monitoring and explaining ML models on Kubernetes
BasetenInference platform to train, serve and scale open source and custom models
ModalServerless cloud for running AI inference, training and batch workloads
AnyscaleManaged platform for running Ray-based AI workloads at scale
FAQs
Is BentoML open source?
Yes, the framework is open source; BentoCloud is the managed commercial service.
How is it deployed?
As REST or gRPC services, batch jobs, or streaming, on BentoCloud or your infrastructure.
Does it scale to zero?
BentoCloud advertises serverless autoscaling including scale-to-zero.
Work at BentoML? Request updates to this profile. Update this profile
Compare with similar providers
All ML Operations providersBlend360Data science, analytics and AI services firm
CerebriumServerless GPU infrastructure for AI workloads
ChalkReal-time feature platform and inference runtime for production AI
ClearMLAI infrastructure control plane with experiment tracking and GPU management
Edge ImpulsePlatform for building and deploying machine learning on edge devices
falGenerative media inference platform with model APIs and serverless GPUs