Welcome to Truss

The simplest way to serve AI/ML models in production

Why Truss?

• Write once, run anywhere: Package and test model code, weights, and dependencies with a model server that behaves the same in development and production.
• Fast developer loop: Implement your model with fast feedback from a live reload server, and skip Docker and Kubernetes configuration with a batteries-included model serving environment.
• Support for all Python frameworks: From transformers and diffusers to PyTorch and TensorFlow to TensorRT and Triton, Truss supports models created and served with any framework.

See Trusses for popular models including:

• 🦙 Llama 2 7B (13B) (70B)
• 🎨 Stable Diffusion XL
• 🗣 Whisper

and dozens more examples on GitHub.

Deploy your first model

Quickstart

Package, deploy, and invoke an ML model in production all in less than five minutes.

Truss tutorial

Learn model deployment step-by-step from “Hello, World!” to streaming output from an open-source LLM.

Truss contributors

Truss is backed by Baseten and built in collaboration with ML engineers worldwide. Special thanks to Stephan Auerhahn @ stability.ai and Daniel Sarfati @ Salad Technologies for their contributions.

We enthusiastically welcome contributions in accordance with our contributors’ guide and code of conduct.