BYOC to BentoCloud: Privacy, Flexibility, and Cost Efficiency in One Package

July 24, 2024 • Written By Sean Sheng, Tim Liu and Sherlock Xu

Engineering leaders are often confronted with the question of whether to buy a fully managed service or to self-host their own solution. While a fully managed service provides benefits like a fully featured platform out of the box, self-hosting allows you to have full control of your data. However each solution comes at a cost:

• Self-hosting requires in-house expertise and can greatly increase the cost and time to develop and maintain a solution.
• Fully managed services require you to trust a third party with your proprietary data and can cost more and be less reliable at scale.

Bring your own cloud (BYOC) is an architecture trend which has grown in popularity with engineering teams, because it provides the best of both worlds to organizations where data ownership and security is a first class concern. It also has added benefits like simple support for hybrid cloud scenarios in any given region of the world.

BentoCloud’s BYOC offers a full featured AI inference platform which can be deployed into any organization’s own cloud account for maximum control and customization.

How Does BYOC Work?

The BYOC architecture works by granting the BentoCloud control plane secure access to your cloud account. This secure connection is used to instantiate the necessary components to streamline AI/ML workflows in production.

Developers can then easily deploy AI/ML services (called Bentos) via the BentoCloud Console or CLI. Behind the scenes, the BentoCloud Control Plane interacts with remote operators in your clusters. These operators are responsible for creating and managing Bento Deployments according to the configurations defined. The BentoCloud Console syncs the states with your clusters through the operators and updates status in real-time.

BentoCloud’s BYOC architecture allows developers to seamlessly deploy AI applications on clusters in any of their cloud accounts, whether it's AWS, Google Cloud, or Microsoft Azure, from the BentoCloud Console or API.

Why BYOC?

BentoCloud's BYOC approach streamlines workflows which ship AI/ML apps to production while also providing a secure and cost-effective solution. Below are the top 5 reasons to use BentoCloud’s BYOC.

1. Data Privacy

In an age where generative AI, such as large language models (LLMs), is revolutionizing industries, leveraging your proprietary data has never been more important.  BentoCloud’s BYOC ensures that your sensitive data and models will never leave your cloud’s secure environment but also can be used with the latest foundational models and AI inference technologies.

2. Cost Efficiency

BYOC instantiates computing resources in the same environment as your data, effectively reducing the hefty egress charges that arise when transferring data out of your cloud account. For businesses that need to process vast volumes of data on a regular basis, executing model inference within your cloud environment can lead to substantial cost savings.

3. Credits and Discounts

If you're involved with a startup or an incubator program, you likely qualify for cloud credits and discounts from leading cloud service providers. To maximize the benefit of these credits and discounts, you must build an infrastructure that integrates with all these providers. BentoCloud BYOC manages compute resources in your own cloud account, which allows developers to easily make use of cloud credits while they are early in their AI journey.

4. GPU Availability

For AI applications, GPUs are both a necessity and a rarity. With BentoCloud’s BYOC, you can select the cloud provider with the best GPU availability for your use case. In addition, you also have the opportunity to choose a provider that offers the most attractive pricing, which ensures you get the most for your money.

5. No Vendor Lock-In

BentoCloud's architecture is built with standard cloud components, which ensures it’s compatible with all general-purpose cloud providers. This design allows your infrastructure to stay independent from any specific cloud vendor, granting you the ability to adapt, switch, or expand your cloud strategy as your needs evolve.

Conclusion

At BentoML, we work to empower every organization to compete and succeed with AI applications. In doing so, we don’t just offer streamlined workflows and an extensive ecosystem of tools, we also champion data privacy and the ability to choose your own cloud providers. This is why we build the BentoCloud BYOC offering to give development teams the best inference infrastructure, but in their own secure environments.

Getting Started

As a part of the Enterprise plan, BentoCloud’s BYOC deployment option is available to both new and existing customers. You can choose to deploy it on AWS, Google Cloud, Microsoft Azure, Oracle Cloud Infrastructure and more cloud platforms. If you are interested in the plan, visit the BentoML website or contact us.