Aug 9, 2022 • Written By Bujar Bakiu
Originally posted on Medium.
*Photo by Tran Mau Tri Tam on *Unsplash
BentoML is currently one of the hottest frameworks for serving, managing and deploying machine learning models. If you are reading this blog, I guess you have heard of it and have some interest in checking it out. So was I, and that’s why I did this small experiment.
One of the simplest exercises in ML is the classification of pets: is it a dog or a cat, based on an image provided. Personally, I never did that exercise but I was intrigued by it. Serving it with BentoML would make it even more challenging. Here is the Github link to my repository.
A few reasons about the technologies I have chosen to serve these models:
This post is about describing how to serve a model in AWS EKS with BentoML.
For the data set used in this post is from Kaggle dataset found in the official Microsoft Download Center. Link: https://www.microsoft.com/en-us/download/details.aspx?id=54765.
The model is a Convolutional Neural Network (CNN, or ConvNet) with many layers (Convolution, Pooling, Flatten, Dropout and Dense layers). For this post, the model was borrowed from another source. The accuracy of the model is reasonably good compared to the dataset size used for training. The training is done on a dataset of 1000 pictures of dogs and cats each. Testing and validation is done on a dataset of 500 pictures per set per category. The dataset provided contains almost 12.5k pictures per each category so the training set size can be increased. So would the time required for training the model, therefore keeping training dataset size at 1000 seems reasonable.
While playing around in the beginning with the dataset and the model two issues came up:
1. The dataset has some corrupted images.
2. The script for creating the test, train and validation datasets was not very flexible and would always generate the same datasets.
Manually fixing them proved to be not very efficient. An automated solution was required as it would make the solution replicable and hopefully help other people who might run in similar issues. Therefore an automated solution would be worth it.
A contributor in Github had already solved the verification of the images issue in a very accurate way. Unfortunately trying to find the contributor again proved to be not so easy. Different versions of the same solution would come up while trying to find it, so it is safe to assume by now, it is a public property. To everyone who contributed to that script: Thank you!
Second issue, as the script for creating the images in different datasets was based on the index of the image (used as the file name also), after clean up, the script was broken. To improve on that, I refactored the script to only use file names to generate the datasets. This way every time a new dataset is generated, it won’t be the same as the previous one, and the model might produce better, or in the unlucky case, or worse predictions.
The BentoML Service in current case looks something like this:
https://github.com/bbakiu/bentoml-cat-dog-classifier/blob/main/pet_classifier_bentoml_service.py
It takes an image input in the request. The image is then transformed into a format the model can accept and serve a prediction on. After the prediction is made, the service returns if it is a cat or a dog and hopefully (at least most of the times) it is right.
Here are the steps needed to serve the model in AWS EKS and the container image stored in ECR:
BentoML has a very good documentation on how to containerize the service and make it available in ECR here. AWS Management Console has also very nice documentation and provides a simple set of commands that help in pushing the image to ECR.
The easiest way to up create the cluster in EKS is via eksctl
. Something like this would do well:
eksctl create cluster \ --name \ --version <1.18> \ --nodegroup-name standard-workers \ --node-type t3.micro \ --nodes \ --nodes-min 1 \ --nodes-max 4 \ --node-ami auto
To start running the service in EKS a kubernetes load balancer service and a deployment are required. You can check here or here for samples on how to do it.
Pro Tip: kubernetes is awesome and easy to start learning, play around with it.
Check the endpoint of the service kubectl get endpoints
to find the url on how the service can be accessed. Also AWS Management Console can be helpful in this case.
Here is my Github Repo.
Happy Coding ;)