Custom Object Detection using TensorFlow — (From Scratch)

Harshil Patel

Oct 30, 2020·7 min read

In this tutorial, we’re going to create and train our own face mask detector using a pre-trained SSD MobileNet V2 model.

First we will create our own image dataset and later we will see how to train a Custom Model for Object Detection (Local and Google Colab!)

1. Installation

1.1 Tensorflow

Install Tensorflow using the following command:

$ pip install tensorflow or pip install tensorflow==1.15

If you have a GPU that you can use with Tensorflow:

$ pip install tensorflow-gpu

1.2 Other dependencies

$ pip install pillow Cython lxml jupyter matplotlib

Install protobuf using Homebrew (you can learn more about Homebrew here)

$ brew install protobuf

For protobuf installation on other OS, follow the instructions here.

1.3 Clone the Tensorflow models repository

In this tutorial, we’re going to use resources in the Tensorflow models repository. Since it does not come with the Tensorflow installation, we need to clone it from their Github repo:

First change into the Tensorflow directory:

# For example: ~/anaconda/envs/<your_env_name>/lib/python3.6/site-packages/tensorflow$ cd <path_to_your_tensorflow_installation>

Clone the Tensorflow models repository:

$ git clone https://github.com/tensorflow/models.git

From this point on, this directory will be referred to as the modelsdirectory

1.4 Setting up the environment

Every time you start a new terminal window to work with the pre-trained models, it is important to compile Protobuf and change your PYTHONPATH.

Run the following from your terminal:

$ cd <path_to_your_tensorflow_installation>/models/research/$ protoc object_detection/protos/*.proto --python_out=.$ export PYTHONPATH=$PYTHONPATH:`pwd`:`pwd`/slim

Run a quick test to confirm that the Object Detection API is working properly:

$ python object_detection/builders/model_builder_test.py

2. Images

For Dataset images I am using face masks Images, You can use your own dataset images or download it from google. For that enable fatkun batch download extension in google. Download bulk images and remove unwanted images from downloaded folder.

Now go to tensorflow object_Detection directory and delete the data folder. Create a new empty data folder, ‘training’ folder, ‘images’ folder.

Now in images folder create 2 new empty folder ‘test, ‘train’. It will look something like this. Copy the images you downloaded and paste into models>research>object_detection>images

object_detection
    ├── data
    |   
    ├── training
    |
    ├── images
    |   ├── test
    |   ├── train
    |   ├── ...

2.1 Image Labeling

Now to label images clone — https://github.com/tzutalin/labelImg repo, go to the anaconda cmd prompt and cd to labelImg path

C:\Users\path\labelImg> python labelImg.py

if you are getting any error in running the cmd, pip install lxml. Once the labelimg window is opened, select open dir and save dir to images folder in object_detection.

Now use W to crop image and label your image, and save it with the same name of img into images folder. Same way crop and label all images. If you have more than 1 label make sure you can name it differently. Make sure you get img and xml file with same name(no space).

Now that you have label all the images, copy 10% of images and xml files into test and rest into train.

object_detection
  ├── images
    ├── test
    |   ├── cmimage1.jpg
    |   ├── cmimage1.xml
        ├── cmimage2.jpg
    |   ├── cmimage2.xml 
    ├── train
    |   ├── ...

now make xml_to_csv.py file and paste above code into it. Make file path in models>research>object_detection. Now make a new file generatetfrecord and paste below code into it.

Now replace label and num on obj for your model. Now open conda cmd and cd to models/research/object_detection and run-

path/object_detection> python xml_to_csv.py

Now 2 files will be created into your data folder. Open both the files and check the labels and number of images into it.

2.2 Generate_tfrecord

Now that you have generated csv files, now cd/ to dir.

If you got any error check the file names and dir for the images. Once you successfully run both cmds you will get 2 .record file into data folder. That's it for part one, if you have generated .record file you are doing great. Make sure you use proper version, file name, and dir. path for the project.

I have added some imp files in this repo — https://github.com/its-harshil/tensorflow-custom-objectdetection

Now We will Train a Custom Model for Object Detection (Local and Google Colab!)

1. TensorFlow Detection Model

Open the above repo and download any model you want and place the folder into models>research>object_detection. I am using ssd_mobilenet_v1_coco

Now add https://github.com/Bengemon825/TF_Object_Detection2020/blob/master/masks/training/ssd_mobilenet_v1_pets.config pet file into ‘training’ folder.

Make changes to the PATHs and num of objects you have

Now copy pipline.conifg file in ssd_mobilenet_v1_coco and paste it into training folder. It should look something like this

object_detection
  ├── data
    ├── object_detection.pbtxt
    ├── train.record
    ├── test.record
    ├── train_lables.csv
    ├── test_labels.csv
 ├── training
    ├── object_detection.pbtxt
    ├── pipline.config
    ├── ssd_mobilenet_v1_pets.config

2. PBTX File

In training folder create a .txt file and paste the below code

item { id: 1 name: 'cloth_mask'}

Now click on Save As, Name it ‘object-detection.pbtxt’ and select all files and Save it. Make n item if you have n label for object detection, make sure you use same label name used in csv files. Now copy this pbtxt file and paste it into data folder.

3. Training

Now that you have added pbtxt file in data and training folder. Open anaconda cmd cd to models\research\object_Detection and run -

set PYTHONPATH=$PYTHONPATH:"pwd":"pwd"/slim'

then

python train.py --train_dir=training/ --pipeline_config_path=training/ssd_mobilenet_v1_pets.config --logtostderr

if you get error saying no module found, then just copy that file and paste in into object_detection. Make sure you just copy it, don’t delete it from main dir.

If any other error is showing make sure you check your .record file, generate_tfrecord file. Make sure they are not empty or have any error in them.

Once you successfully run the cmd you will be able to see that model is getting trained with losses and time. It will look something like this-

I1030 12:52:00.296686  3708 learning.py:768] Starting Queues.
INFO:tensorflow:global_step/sec: 0
I1030 12:52:09.401392 30180 supervisor.py:1099] global_step/sec: 0
INFO:tensorflow:Recording summary at step 3217.
I1030 12:52:21.126251 16976 supervisor.py:1050] Recording summary at step 3217.
INFO:tensorflow:global step 3218: loss = 1.7246 (21.576 sec/step)
I1030 12:52:22.335144  3708 learning.py:507] global step 3218: loss = 1.7246 (21.576 sec/step)
INFO:tensorflow:global step 3219: loss = 1.6730 (3.673 sec/step)
I1030 12:52:26.409953  3708 learning.py:507] global step 3219: loss = 1.6730 (3.673 sec/step)
INFO:tensorflow:global step 3220: loss = 1.8269 (4.190 sec/step)
I1030 12:52:30.603331  3708 learning.py:507] global step 3220: loss = 1.8269 (4.190 sec/step)
INFO:tensorflow:global step 3221: loss = 1.9480 (4.277 sec/step)
I1030 12:52:34.883605  3708 learning.py:507] global step 3221: loss = 1.9480 (4.277 sec/step)
INFO:tensorflow:global step 3222: loss = 1.5400 (5.243 sec/step)
I1030 12:52:40.129421  3708 learning.py:507] global step 3222: loss = 1.5400 (5.243 sec/step)
INFO:tensorflow:global step 3223: loss = 1.3421 (5.265 sec/step)
I1030 12:52:45.397172  3708 learning.py:507] global step 3223: loss = 1.3421 (5.265 sec/step)

make sure you are generating losses, once your loss rate is equal or less than 1, press CTRL+C to stop training.

4. EXPORT GRAPH

Now that your training is over head to object_Detection folder and open training folder. You will see data,index,meta file generated with same name. Check the latest files like — model.ckpt-3217

now run, and edit the ckpt file name in below cmd with your latest file.

python export_inference_graph.py --input_type image_tensor --pipeline_config_path training/ssd_mobilenet_v1_pets.config --trained_checkpoint_prefix training/model.ckpt-3217 --output_directory new_graph

Now go to object_detection folder, new_graph folder will be generated.

5. Web Cam Detection

Create a custom web cam detection python file in object_detection dir. , paste below code —

Now open pycharm and run the above file. Your web cam will pop-up and you will be detect face mask with label and accuracy. You can even run this on your cmd prompt.

If you are not able to run any cmd —

• check with the version code
• check the path
• check .record, .csv, .config etc files are there
• lxml, slim and other are installed
• check label name it should be same in csv, generate tf record file and pbtxt file.
• use about 100+ images for each object

6. Google Colab

ipynb file — https://github.com/Bengemon825/TF_Object_Detection2020/blob/master/ModelTrainingOnColab.ipynb

Now run all the code given in the ipynb file and download the generated graph to object detection folder, with name on new graph. Now run the web cam detection python file and test your model.

7. TENSORBORAD (Optional )

TensorBoard provides the visualization and tooling needed for machine learning experimentation:

• Tracking and visualizing metrics such as loss and accuracy
• Visualizing the model graph (ops and layers)
• Viewing histograms of weights, biases, or other tensors as they change over time
• Projecting embeddings to a lower dimensional space
• Displaying images, text, and audio data
• Profiling TensorFlow programs

Now open another anaconda cmd and run in object_detection

tensorboard — logdir=training/

It will provide link to your local host, like this http://localhost:6006/#

Now practice with more objects and more images, Happy Coding !!!

Imp — Links

• https://github.com/tzutalin/labelImg
• https://github.com/datitran/raccoon_dataset
• https://chrome.google.com/webstore/detail/fatkun-batch-download-ima/nnjjahlikiabnchcpehcpkdeckfgnohf?hl=en