How to use secrets

A guide to using secrets securely in your ML models

Your model server may need to use access tokens, API keys, passwords, or other secret values. Truss gives you everything you need to use secrets securely.

Setting secrets in config.yaml

If your model needs a secret, first add its name in config.yaml with a placeholder value:

config.yaml

secrets:
  hf_access_token: null

Never set the actual value of a secret in the config.yaml file. Only put secret values in secure places, like the Baseten workspace secret manager.

Using secrets in model.py

Secrets are passed to your Model class as a keyword argument in init. They can be accessed with:

model/model.py

def __init__(self, **kwargs):
    self._secrets = kwargs["secrets"]

You can then use the self._secrets dictionary in the load and predict functions:

model/model.py

def load(self):
    self._model = pipeline(
        "fill-mask",
        model="baseten/docs-example-gated-model",
        use_auth_token=self._secrets["hf_access_token"]
    )

Storing secrets on your remote

On your remote host, such as your Baseten account, store both the secret name and value before deploying your model. On Baseten, you can add secrets to your workspace on the secrets workspace settings page.

Make sure to use the same name (case sensitive) as used in the Truss on the remote.

Deploying with secrets

For additional security, models don’t have access to secrets by default. To deploy a model and give it access to secrets, pass the --trusted flag during truss push as follows:

truss push --trusted

Your model will be deployed with access to secrets stored on your remote.