It’ll be a discussion about three topics. In the first one, which is this article, we’ll create a model for image recognition. In the others, we’ll try to use that model in Xamarin applications.
- Creating the model
- Binding for Xamarin.Android
- Binding for Xamarin.iOS
It was a usual evening. I was reading Medium and in parallel thinking how I should begin learning neural networks/DL/ML/etc. So, my girlfriend has a cat, very annoying cat. Another useless fact is that I’m a big fan of Silicon Valley show. Well, suddenly I got an idea. What if I’d create an app like in the show, you know, the “hotdog, not hotdog app”, but instead of hotdog an app would be searching and telling me where the cat is. Like “Melvy, not Melvy” cat (Yes, Melvy is the cat name). So, that’s an idea.
The article is going to be step by step instruction how you could create your own model and you don’t have to be a neural network specialist! I’m serious, I’m not a NN specialist at all, but even I could make a specific DL model without any help from someone.
But, apparently, for understanding what’s going on, you have to know some basics stuff. I’ll briefly touch some certain terms to make it more clear:
Convolutional neural network - this type of models really good fit for solving image recognition problems.
Softmax regression - a type of regression which allows you really good split something into categories.
Transfer Learning - this technique allows you remove the top layer in your model and retrain it for something else. The reason is why it’s very useful it’s because usually (at least in my case) we don’t have enough CPU/GPU power to train the whole model. So, we could take some existing, pretty well-working model which was trained for some issue and retrain only one top layer from model for our particular issue which also has to be similar with “main problem”. In our case, we are going to take latest Inception-v3 model and retrain it.
So, first of all, we’ll try to train the model for certain categories of pictures - flowers. Wy not cat pictures? You could do that. Flower pictures already exist and we don’t have to collect of 100 pics at least for each category. But if you have ones, especially if they’re cat pictures, go ahead. Let’s make a folder for our project:
Then let’s download our flowers into the folder and unpack them:
cd flowersproject curl -O http://download.tensorflow.org/example_images/flower_photos.tgz tar xzf flower_photos.tgz
So, the next step is cloning tensorflow from github into our folder
git clone https://github.com/tensorflow/tensorflow cd tensorflow
Make sure that you’re on master branch
git checkout master
In the simplest cases the retrainer can then be run like this:
python tensorflow/examples/image_retraining/retrain.py --image_dir ../flower_photos
During the time you’ll see similar lines in terminal:
It means the model began learning.
You need only one thing. The thing is data, which in our case it’s categorized pictures. Each name of flower folders represents a label for this type of data. So, you could find another flower pictures such as carnations, collect more than 100 pics of this type of flowers, put them into ‘carnation’ folder and the model would be able to recognize it.
After a few minutes (or 20 - 30 mins, depends on your CPU) you’ll get a brand new retrained model! That’s all! This is super easy, isn’t? By default, your model will be created as
/tmp/output_graph.pb and labels information as
/tmp/output_labels.txt. We need these files for creating mobile applications. If you don’t know, the
.pb format is protobuf which is binary format of representation of structured data. You could imagine it as JSON but in binary representation. Much smaller, but not readable for humans.
Well, we have the model, but how can we test it? So, for that, we have other python script which already has written for us. It’s
Here’s an example of how to run the label_image example with your retrained graphs:
python tensorflow/examples/label_image/label_image.py \ --graph=/tmp/output_graph.pb --labels=/tmp/output_labels.txt \ --input_layer=Mul \ --output_layer=final_result \ --input_mean=128 --input_std=128 \ --image=$HOME/flower_photos/daisy/21652746_cc379e0eea_m.jpg
Since you’ve replaced the top layer, you will need to specify the new name in the script, for example with the flag
--output_layer=final_result if you’re using label_image.
You should see a list of flower labels, in most cases with daisy on top (though each retrained model may be slightly different). You can replace the –image parameter with your own images to try those out.
Improvements for mobile applications
So, we got our model. Let’s see the size of this model. I don’t know about your’s, but mine was something around 95 Mb and, of course, it’s unacceptable. Not only because it’s sort of a big size for a mobile application but also using this model in a mobile app could be with lags and freezes.
Let’s find out how we can reduce the size of this model with acceptable losses in quality of recognition. There are two basics ways to reduce the size of the model in this particular situation. It’s juggling of input picture resolution or amount of parameters of model architecture. Fortunately for us, we already have pretrained 32 models. To use them we have to pass
--architecture parameter in
retrain.py script. For example this command:
python tensorflow/examples/image_retraining/retrain.py \ --image_dir ../flower_photos --architecture mobilenet_0.25_128
will create a 941KB model file in
/tmp/output_graph.pb, with 25% of the parameters of the full Mobilenet, taking 128x128 sized input images. You can choose ‘1.0’, ‘0.75’, ‘0.50’, or ‘0.25’ to control the number of weight parameters, and so the file size (and to some extent the speed), ‘224’, ‘192’, ‘160’, or ‘128’ for the input image size, with smaller sizes giving faster speeds
This article briefly explains how we can train a simple neural network model for image recognition. For making all steps which explaided here you should have installed python and git. I recommend visiting official TensorFlow tutorial pages for much more information about the topic. Next topic is going to show how we could use the trained model in Xamarin.Android application!