Simple Surveillance System with the Tensorflow Object Detection API
--
Surveillance is an integral part of security and is used all over the world to ensure the safety of valuables as well as people.
Even though surveillance is controversial in some situations. Especially when connected to surveillance of human beings it is necessary for other tasks like watching over valuables, monitoring operations, ensuring employee safety as well as loss prevention and public safety.
Surveillance, mostly, is a repetitive mostly trivial tasks which raises the question:
Can machines possibly automate surveillance?
This is a question a lot of people not familiar with deep learning might have on their mind.
Anyone with a little knowledge of deep learning and computer vision can probably tell them that deep learning has surpassed human performance on most computer vision task.
The same progress was also achieved in the domain of object detection so yes, a deep learning models is more that capable to detect the right objects in images most of the time.
How can we implement a simple surveillance system ourselves?
So now that we know that surveillance, for the most part, can be automated using machine learning we might wonder how we can build our own simple surveillance system.
Building a ‘small’ surveillance system is quite simple. We only need to train a object detection model on the classes we want to detect and then react when an object is detected.
In this article, we will simply save an image of the detected object and record the time it was detected into a csv file.
For creating the object detection model we will be using the Tensorflow Object Detection API.
The Tensorflow Object Detection API is an open source framework that allows you to use pretrained object detection models or create and train new models by making use of transfer learning. This is extremely useful because building an object detection model from scratch can be difficult and can take a very long time to train.
If you haven’t installed the Tensorflow Object Detection API yet you can check out my article covering how to do so.
Recap on how to use the Tensorflow Object Detection API for Live Object Detection
In my article called “Live Object Detection” I covered how to rewrite the Tensorflow Object Detection demo script to work with a live video stream. For this we rewrote the run_inference_for_single_image method to only include the code that needs to be executed once and moved all the other code outside the method.
This code is a useful starting point but we aren’t yet accessing the individual objects. Furthermore because I will just use a model trained on the coco data-set I will also create a variable that specifies what objects we are looking for so we can check if an object has that class.
Get the coordinates and labels for each box
We can access the label and coordinates of the objects using the detection_classes and detection_boxes keys for the output_dict dictionary.
In the code above we loop through all the scores for the detected objects. If the score of the object is higher than the specified threshold we will get the label and coordinates and append them to our output array. If the score is lower than the threshold we will just skip the object.
Check if the box contains a human
To now react on some classes we can simply loop through the output array and check if the label corresponds to the class we are looking for.
If the label matches the class we will cut out the part of the image that contains the class and save it as a jpg. We will also save the datetime and filepath into a csv so we know the time when the image was taken.
Putting things together
Now the only thing left to do is to but all the pieces together. We will also create a method for loading in the data-set and use the argparse library so we can pass the path of the model, labelmap, threshold, output directory and the class we want to search for.
We can test this script by typing
python simple_surveillance_system.py -m <path_to_model> -l <path_to_labelmap>You can also specify any of the other parameters but you don’t need to because all of them have preset values.
Use ImUtils to allow for support of the Raspberry Camera
Another thing we can to is to add support for the Raspberry Camera. We can to so using the imutils library.
Imutils is build on top of OpenCV and offers a lot of convenience functions to make basic image processing easier. It includes functions for translating, rotating, resizing images as well as many more.
To incorporate ImUtils in our script we only need to change the initialization of the camera, the reading of a frame as well as the closing of the camera when exiting the script.
Can we trust our model
Being able to trust a model is really important especially when the model is used to make important decisions like whether to let someone lend money.
In order to be sure that our model only detects humans when their really is a human in the picture we need to test. We can do so by simply running our model and watching if it does the right thing but we can also use model interpretation tools to help us understand our model.
If you want to learn more about model interpretation and why it is important you can check out my article series on model interpretation.
Summary
Today machine learning models are capable of automating repetitive trivial tasks like image classification or object detection. The only thing they need to do so is lots of data.
A Surveillance System can be build using a object detection model that monitors the desired classes.
What’s next?
In the upcoming months I will blog about different applications of machine learning and data science as well as some new technologies and libraries. Furthermore, I will focus more on learning the theory behind machine learning in depth.
That’s all from this article. If you have any questions or just want to chat with me feel free to leave a comment below or contact me on social media. If you want to get continous updates about my blog make sure to follow me on Medium and join my newsletter.
