Skip to main content

Monitoring deployments

Once you have deployments in production, you'll want to monitor them to be aware if any issues arise.

Safely calling Modelbit models​

When calling Modelbit via the REST API, if the call is successful, the inference results will be returned via JSON, with the data field containing the results.

If there was an error, the error key will be present and its value will contain the error.

Additionally, it's always wise to wrap the entire REST call in an exception-handling block in case there are exceptions.

Putting it all together, a Modelbit call from Python code might look like this:

import requests
import logging

def get_lead_score(hdyhau: str, utm_source: str, industry: str) -> float:
  try:
    json_response = requests.post("https://harrys-house.app.modelbit.com/v1/lead_scorer/latest", json = {
      "data": [hdyhau, utm_source, industry]
    }).json()
    if 'error' in json_response:
      raise json_response['error']
    return float(json_response['data'])
  except Exception as e:
    logging.error(f"Error calling Modelbit: {e}")
    return 0.0 # Placeholder logic when inferences are unavailable goes here

Setting up Modelbit for monitoring​

Modelbit's Slack Alerting feature is recommended for quick and easy alerting on model errors.

For more robust alerting, Modelbit's DataDog integration can be paired with pager systems like PagerDuty or OpsGenie.

Diagnosing an alert​

To diagnose an alert, first determine if the problem is in Modelbit or in your running model.

Diagnosing Modelbit problems​

To determine if a Modelbit production system is having an error, visit the Modelbit Status Page at https://status.modelbit.com:

First, determine your region. In your Modelbit account, the first component of the URL shows your region. app corresponds to Modelbit Ohio, ap-south-1 corresponds to Modelbit Mumbai, and us-east-1 corresponds to Modelbit Virginia.

Click the plus icon next to your region on the status page. "Running Models" shows any current issues related to your running models, which is most pertinent when responding to an alert. "Web Application and API" refer to workflows whose loss may be inconvenient but should not impact running models in production:

Diagnosing problems in your models​

To diagnose problems in your models inside of Modelbit, head to Modelbit's logs viewer for your model. Select your deployment from the list of Deployments and then click "Logs" in the left-hand navigation.

Any errors will be shown in bright red with their full stack trace present:

Escalating to the Modelbit team​

Never hesitate to reach out to the Modelbit team for further assistance! Use your shared Slack channel to talk to the team directly.

Enterprise customers can reach the Modelbit team via cell phone. Reach out to your contacts to get the numbers.