问题
I am just wondering what is the difference between the "classification" and "detection" fine tuning types for the available checkpoints in the object detection API. Are they both eligible to train novel classes? The checkpoints obtained from such training, can be further trained with the very same pipeline.config, or does it need to have a different fine tuning type?
EDIT
to make the question clearer, one can take as reference the ckpts mentioned in https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf2_training_and_evaluation.md#model-parameter-initialization .
Digging in the code one could find for example that some classes of CenterNet models use the CenterNetResnetV1FpnFeatureExtractor class to restore the model from a checkpoint, which seems to accept "classification" types only (https://github.com/tensorflow/models/blob/1b5a4c9ed33242783eaf29e664618331dbb59e1b/research/object_detection/models/center_net_resnet_v1_fpn_feature_extractor.py#L168), while one can find some CenterNet models addressed as "detection" types (one can compare the fine_tune_checkpoint_type that comes with the pipeline.config file in the download of centernet_hg104_512x512_coco17_tpu-8 ckpt from the "detection" cktp list mentioned above).
I find it in general rather confusing, and I am not sure that this does not generate undesired effects like retraining the whole model from scratch ignoring the desired ckpt or similar. Obviously it would be nice not to have to inspect the whole code to find how every each model is restored from a ckpt.
回答1:
Machine Learning on Images basically have three types.
Object Detection which is a combination of
- Object Localisation
- Image Classification
Apart from the above two, There is also Image Segmentation which I won't devlve into as far as object detection API from tensorflow is considered.
Object Detection algorithms act as a combination of image classification and object localization. It takes an image as input and produces one or more bounding boxes with the class label attached to each bounding box. These algorithms are capable enough to deal with multi-class classification and localization as well as to deal with the objects with multiple occurrences.ref.
To answer your questions.
Are they both eligible to train novel classes
The fine tuned checkpoint option is basically for transfer learning. Consider you have a pre-trained model from a previous dataset, those same weights and checkpoint can be used to further train your novel dataset via transfer learning.
Ofcourse, this would mean a pre-trained model trained on classes can you help you futher transfer learning to new set of images with their corresponding classes. For a list of models trained have a look here Model Zoo TF2 OD API.
Eg. You could use a coco dataset model (any from the above list infact) trained on around 100+ classes to train on new set of classes like iphone 7/8 or iphone x/xs/11/12 with as few as 100-1000 images for each category in different surroundings and shapes and sizes.
The checkpoints obtained from such training, can be further trained with the very same pipeline.config, or does it need to have a different fine tuning type?
No, you would have to modify the pipeline.config with a bare minimum of options that fit your use cases. In order to start with, have a look at the official documentation for Object Detection at github pages. If you still have doubts or is stuck. Raise a new question with your issue.
UPDATE: The fine tune checkpoint will be set to detection if you are looking for localisation + classification of multiple "objects" in the image. classification only refers to simply labelling an image (refer above image).
The ModelZoo models I referred can be used for detection type checkpoints. Object Detection supposedly mentions models with detection as a type only, and this based on my own trials and ultimate failure. If you are just looking at classification, have a look at this useful resource : teachablemachine.withgoogle.com/train for image classification
来源：https://stackoverflow.com/questions/64908388/tensorflow-object-detection-api-fine-tune-ckpt-classification