Using private cloud
Modelbit can deploy your ML models onto servers running in your cloud account. To get started, log into Modelbit, go to Settings and click Private Cloud. The wizard will walk you though connecting Modelbit to your cloud.
What is private cloud?
Modelbit's private cloud gives you the best of both worlds: control over the infrastructure without the devops overhead of managing and updating on-prem software.
With private cloud you give Modelbit access to operate on resources within your cloud account. This allows Modelbit to provide ongoing improvements and monitoring while all of your deployments run on servers and within networks you control.
Setting up private cloud for AWS
In order to connect Modelbit to AWS, you'll need IAM permissions to run a CloudFormation template that create a role and S3 bucket.
We recommend creating a new AWS account within your organization for use with Modelbit.
In Modelbit, head to Settings, click Private Cloud, then click Set up Modelbit Private Cloud. A wizard will guide you through running a CloudFormation template to create the role that Modelbit will use to manage resources in this AWS account.
Enabling private cloud will not disrupt existing deployments. They'll continue to run on Modelbit's infrastructure until you transition them to running in your private cloud.
Completing the the setup process will take about 15 minutes. During that time Modelbit will create EC2s, VPCs, Global Accelerators, S3 buckets, IAM roles, and more. All of this infrastructure is used to serve deployments running in your AWS account.
Testing setup
Once the wizard completes you'll be in a "testing" mode. The testing mode lets you test that existing deployments have been replicated into your AWS account, and that new deployments run in your AWS account.
Deployments running in your private cloud have a different REST URL than those running in Modelbit. Within each deployment's API Endpoints screen is a toggle that lets you choose between the Modelbit and Private Cloud versions of the URL. Use these two different URLs to make sure your deployments work as expected.
Finishing setup
Once your tests are complete, head back to the Private Cloud section in settings and click Switch to Private Cloud.... This button will turn off the replication features used during testing and make your AWS account the primary infrastructure for running deployments.
Managing infrastructure
Modelbit automates most of the management of infrastructure within your AWS account. One thing that you'll need to choose is how many servers, and what size, will be used to run your deployments.
Once set up, the Private Cloud section in settings will show CPU, RAM, and VRAM utilization charts for each EC2 instance managed by Modelbit. To change the number or size of the servers running your deployments, click Edit Settings in the top-right and change the server type or count.
Under each instance's usage chart is a table of Deployment Containers. Each row represents an environment that can run one or more deployments.
These instances and containers should be self-managing. However, if you ever need to terminate them you can by clicking the three-dot settings icon and clicking Shut down instance... or Shut down container.... Modelbit will boot new instances and containers to replace what was shut down.
Shutting down private cloud
You can turn off private cloud in two steps. First, scale the number of instances serving your deployments to 0. Once there are no running instances a button will appear that'll let you disconnect Modelbit from your AWS account.