BentoML 2023 Year in Review
December 15, 2023 • Written By Sherlock Xu
As we close the chapter on 2023, it's time to look back at a year that was as challenging as it was rewarding. It witnessed significant advancements in the realm of artificial intelligence (AI), particularly in large language models (LLMs), diffusion models, and generative AI technologies. These developments have not just pushed the boundaries of what's possible, but have also opened up new opportunities for AI-driven innovation and application.
At BentoML, we've been more than mere spectators; we've been active participants and contributors to these exciting trends, marked by the release of various practical projects and an increase in global open source contributions. They represent the core of our commitment to staying at the forefront of AI and machine learning (ML) and empowering every organization to compete and succeed with AI.
In this post, join us as we take a deep dive into the highlights of 2023, celebrating the milestones that have defined BentoML's journey, which has been made possible only through the support and collaboration of the open-source community.
Team and community growth
2023 is important in transforming our aspirations into tangible achievements, setting new benchmarks for what we can accomplish as a team and a community. Here's a glimpse into some of the key milestones and developments that have marked this year for BentoML:
- $9M seed funding: In a landmark moment this June, BentoML secured $9 million in seed financing, a significant milestone in our ongoing mission. As we develop from a single open-source project to a growing ecosystem, this funding is set to expand our team and improve our offerings.
- Doubling down on team expansion: 2023 has been a year of growth, not just in our aspirations but also in our team size, which has doubled. Each new team member brings a unique set of skills and perspectives, further enriching the BentoML experience.
- A fresh face with our new website and mascot: Our website has undergone a transformation, offering an interactive experience for easy deployment of ML models. Alongside this change, meet Ben, our new bento-box mascot, symbolizing our fresh and innovative approach. Discover more on our updated website.
A thriving open-source project is deeply rooted in its community, and for us at BentoML, the community is the cornerstone of everything we do. Here's BentoML by the numbers in 2023:
- 15K+ total GitHub stars
- Used by 1300+ projects on GitHub
- 4000+ community members (Slack and Discord)
We recognize that these numbers represent real people — developers, innovators, and enthusiasts — who believe in the power of open source and the possibilities it holds. We're grateful for every contribution that helps us build BentoML and its ecosystem.
Expanding the ecosystem with AI-driven solutions
In 2023, we released a variety of projects that have significantly contributed to our expanding ecosystem. Going forward, we will continue to focus on providing practical solutions to real-world problems just like these projects. Here's a look at some of the notable ones in 2023:
- BentoCloud: BentoCloud is a fully-managed platform designed for building and operating AI applications. BentoCloud accelerates AI application development by providing powerful workflows to deploy and scale everything from LLMs to your custom ML models. It provides solutions for addressing deployment, scalability, and collaboration challenges in the AI application delivery lifecycle.
- OpenLLM: OpenLLM is an open-source platform designed to facilitate the deployment and operation of large language models (LLMs) in real-world applications. With OpenLLM, you can run inference on any open-source LLM, deploy them on the cloud or on-premises, and build powerful AI applications.
- OneDiffusion: Tailored for diffusion models, OneDiffusion is an open-source platform for deploying these models in production. It supports both pre-trained and fine-tuned diffusion models with LoRA adapters, offering tools for fine-tuning, serving, deploying, and monitoring.
- CLIP-API-Service: Integrating OpenAI CLIP with streamlined APIs, this tool offers functionalities like image and sentence embedding, object recognition, and visual reasoning.
- Transformers NLP Service: This project provides a modular, composable, and scalable solution for building NLP services with Transformers. It incorporates BentoML's best practices from setup to production deployment.
- Fraud Detection: Demonstrating online model serving with a custom XGBoost model trained on the IEEE-CIS dataset, this project highlights the practical application of BentoML in fraud detection.
- OCR as a Service: This project makes serving OCR models effortless, accepting PDF inputs and returning extracted text, employing Microsoft's DiT and Meta's detectron2 for image segmentation, and EasyOCR for OCR.
- Replace Anything: A unique combination of Segment Anything and Stable Diffusion, this web application allows you to replace any part of an image with a generated image based on descriptions.
- DeepFloyd IF Multi-GPU serving: This project facilitates easy serving of DeepFloyd IF models across multiple GPUs, simplifying the deployment of state-of-the-art Stable Diffusion models.
- Sentence Embedding as a Service: Providing a high-performance REST API server with a single command for text embeddings, this project comes with all-MiniLM-L6-v2 as the default model, but is customizable for other embedding models.
- BentoSVD: BentoSVD is designed to streamline the process of serving and deploying Stable Video Diffusion models in production, eliminating the setup and configuration complexity with such models.
Content: Enlightening and engaging our community
2023 has been a prolific year for BentoML in terms of creating and sharing insightful content. Our blog has served as a platform for discussing innovations, sharing knowledge, and guiding users in leveraging the full potential of BentoML. Here are the top 5 blog posts released in 2023:
Our content has also been enriched by collaborations with other communities:
- In partnership with Hugging Face, we published Deploying Hugging Face Models with BentoML: DeepFloyd IF in Action
- OpenLLM’s integration with LangChain: OpenLLM LangChain integration
- OpenLLM’s integration with LlamaIndex: OpenLLM LlamaIndex integration
We are happy to see an increase in community-generated video content this year, helping to spread knowledge and use cases of BentoML. Some notable contributions include:
A big thank you goes out to all our contributors, collaborators, and community members. Your efforts, insights, and enthusiasm are the driving force behind BentoML's continued growth and success!
Looking forward
As we move forward, BentoML is ready for a series of exciting developments and enhancements, especially in the following two aspects:
- BentoML 1.2: Currently under active development, BentoML 1.2 will see a significant leap forward. This upcoming version is focused on delivering important improvements in performance, ease of use, and flexibility. Developers will be able to use more intuitive and user-friendly APIs to create their ML services, with a wide range of example projects providing practical guides and insights for real-world AI scenarios.
- BentoCloud General Availability (GA): A major highlight in our upcoming roadmap is the GA of BentoCloud. We would like to thank all our partners and customers who have played an important role in helping us build this platform. BentoCloud has come a long way since its inception and is becoming the best infrastructure for running BentoML and any AI inference workload. As BentoCloud approaches its GA, we're excited about the possibilities it opens up for our users. If you're interested in being a part of this journey and want to contribute to the beta testing phase, don’t forget to sign up here. Your feedback and insights are important for us to refine and improve BentoCloud.
More on BentoML
To learn more about BentoML, check out the following resources: