Adding inference timeouts

Modelbit's REST API has a default timeout of 5 minutes. You can control that timeout with the timeout_seconds parameter.

For example, to apply a 10-second timeout to your online inference requests:

curl -s -XPOST "https://<your-workspace-url>/v1/my_deployment/latest" -d '{"data": ..., "timeout_seconds": 10}'

Inference requests that take longer than the timeout will be terminated and an error will be returned.

Longer timeouts for async requests

To use a timeout longer than 5 minutes you must also use the async REST API. The maximum timeout when using the async API is 15 minutes. For example, a 10-minute timeout:

curl -s -XPOST "https:..." -d '{"data": ..., "response_webhook": "https://...", "timeout_seconds": 600}'