Note: This blog post is also published on TomTom's Engineering Blog site.
TomTom stands as a pioneering mapmaker, providing location data and technology to a diverse clientele including drivers, carmakers, businesses, and developers. Its application-ready maps, routing solutions, real-time traffic updates, as well as APIs and SDKs enable dreamers and doers to shape the future of mobility. With its headquarters in Amsterdam, TomTom's global presence is marked by a dedicated team of over 3,800 employees. For more than 30 years, TomTom has been a trusted companion, guiding people through the world with precision and reliability.
Over the past year, the AI industry has witnessed significant advancements in Generative AI (GenAI) technologies. Large Language Models (LLMs) like GPT-4 and open-source alternatives such as Llama2 have made building AI applications more accessible and user-friendly, reducing the need to invest years into developing deep ML skills. This transformation has caught the attention of not only AI experts but also business professionals, emphasizing the unique aspects and potential of using LLMs in contrast to traditional ML projects.
Amid these trends, TomTom has actively engaged with AI advancements. As a company deeply rooted in maps and navigation technologies, its data-centric nature positions it well for AI and ML innovations. This aligns with its mission to provide global real-time maps and navigation services. Specifically, TomTom's strategic response to these developments has included democratizing innovation across teams, prioritizing impactful projects, and establishing a GenAI center of excellence. In addition, TomTom is working with academia and startups for early R&D and cloud providers on foundational models and infrastructure. These have helped TomTom quickly get started and experiment with the latest AI technologies, ensuring it stays at the forefront of the evolving tech landscape.
One of TomTom’s early partners was BentoML, which provides a unified AI application framework that’s helped the mapmaker get experiments off the ground, especially in model serving and deployment. With its end-to-end solution for streamlining the deployment process, BentoML simplifies the transition from a machine learning model to a fully operational AI service, making it a comprehensive tool for modern AI-driven solutions.
Working on AI experiments with BentoML has helped TomTom maintain focus on its core competency in maps and navigation services, while also trying out the latest AI technologies speedily. TomTom selected BentoML as a partner for rapid experimentation and innovation due to the following principal reasons:
For BentoML, the partnership with TomTom represents a significant opportunity. Working with a global leader in navigation technologies validates BentoML’s strong capabilities in LLM serving and deployment in production. It allows the BentoML team to refine the project further for real-world LLM use cases.
TomTom's exploration into LLM-powered services involves addressing various challenges. While starting with Azure OpenAI APIs is fantastic for quickly getting a prototype out the door, you’ll probably want more control to improve upon your prototype.
For example, while LLMs can effectively perform tasks like data classification, scaling these services requires more cost-effective and efficient strategies. Therefore, experimenting with different, possibly open-source models, replacing large models with smaller ones, and applying optimizations such as mini-batching techniques become essential.
In some cases, TomTom's approach to improving LLM-powered applications emphasizes using different models for specific tasks. By analyzing the problematic areas of the system — whether in quality, latency or cost — TomTom can replace parts with more suitable alternatives, streamlining its application. However, this requires a glue solution for coordinating the pipeline.
This is where BentoML comes in. It provides a straightforward way to integrate various models, simplifying the process of model composition and inference. BentoML, particularly with its serverless platform BentoCloud, acts as a cohesive agent in AI apps, enabling developers to focus on core functionalities without getting bogged down by extensive microservice architecture and complex infrastructure.
The collaboration between TomTom and BentoML has yielded fruitful results in the following aspects: