Skip to main content

Endpoints: endpoint_name.yaml

Each endpoint is defined in a separate YAML file, stored by name in the endpoints directory. For example, the endpoint named prod_classifier would be defined in endpoints/prod_classifier.yaml.

File format​

The file is YAML-formatted. Here are the type definitions and field descriptions:

endpoints/my_endpoint.yaml
backends: Dict[str, Dict[str, ...]]
  [backend_name]: Dict[str, ...]
    deployment: Dict[str, str | int]
      branch: str
      name: str
      version: "latest" | int
    weight: Optional[int] = 1
mirrors: Optional[Dict[str, ...]]
  [mirror_name]: Dict[str, ...]
    deployment: Dict[str, str | int]
      branch: str
      name: str
      version: "latest" | int
schemaVersion: 1

The fields:

  • backends: A dict of backend names to their properties. These names appear in the logs Modelbit sends to your logging endpoints so you can distinguish which traffic went to each backend. Aliased deployments have only one backend, while A/B splitting deployments have two or more.

    • [backend_name]: The name of the backend. Use a unique strings like experiment and control, or my_alias.

      • deployment: The deployment that should receive traffic.

        • branch: The branch name for the target deployment.
        • name: The deployment name of the target deployment.
        • version: The version of the deployment. It can be latest or a version number.

      • weight: For A/B splitting deployments, the relative weights for traffic splitting between the backends.

  • mirrors: A dict of backends names to their properties for use in shadow deployments. All mirrors receive a copy of the endpoint's traffic. The properties within are the same for [backend_name].

  • schemaVersion: The file format version of this file.

Examples​

Here are example endpoint.yaml files showing various popular configurations.

Aliased deployment​

An endpoint that aliases a deployment's API URL has one backend:

endpoints/example_alias_endpoint.yaml
backends:
  split_1:
    deployment:
      branch: main
      name: example_deployment
      version: 2
    weight: 1
schemaVersion: 1

Split traffic between two deployments:​

To split traffic and send 90% of traffic to deployment_control and 10% to deployment_experiment, use two backends:

endpoints/example_ab_endpoint.yaml
backends:
  split_1:
    deployment:
      branch: main
      name: deployment_control
      version: latest
    weight: 9
  split_2:
    deployment:
      branch: main
      name: deployment_experiment
      version: latest
    weight: 1
schemaVersion: 1

Shadow traffic to a new deployment​

To mirror of all requests to candidate_deployment, use a mirror:

endpoints/example_mirror_endpoint.yaml
backends:
  split_1:
    deployment:
      branch: main
      name: example_deployment
      version: 2
    weight: 1
mirrors:
  mirror_1:
    deployment:
      branch: main
      name: candidate_deployment
      version: latest
schemaVersion: 1