Training darknet finishes immediately

Question

I would like to use the yolo architecture for object detection. Before training the network with my custom data, I followed these steps to train it on the Pascal VOC data: h

Answer 1

Also if using AlexeyAB/darknet they might have a problem with -clear option, in detector.c:

if (clear) *nets[k].seen = 0 

should really be:

if (clear) {*nets[k].seen = 0;*nets[k].cur_iteration = 0;}

otherwise the training loop will exit immediately.

Answer 2

Adding -clear 1 at the end of your training command will clear the stats of how many images this model has seen in previous training. Then you can fine-tune your model on new data(set).

You can find more info about the usage in the function signature void train_detector(char *datacfg, char *cfgfile, char *weightfile, int *gpus, int ngpus, int clear) at https://github.com/pjreddie/darknet/blob/b13f67bfdd87434e141af532cdb5dc1b8369aa3b/examples/detector.c

I doubt it that increasing the max number of iterations is a good idea, as the learning rates are usually associated with current # of iteration. We usually increase the max # of iterations, when we want to resume a previous training task that ended because of reaching the max # of iterations, but we believe that with more iterations, it will give better results.

FYI, when you have a small dataset, training on it from scratch or from a classification network may not be a great idea. You may still want to re-use the weights from a detection network trained on large dataset like Coco or ImageNet.

Answer 3

This is an old question so I hope you have your answer by now, but here is mine just in case it helps.

After working with darknet for about a month, I've run into most of the roadblocks that people have asked/posted about on forums. In your case, I'm pretty certain it's because the weights have been trained for the max number of batches already, and when the pre-trained weights were read in darknet assumed training was done.

Relevant personal experience: when I used one of the pretrained weights files, it started from iteration 40101 and ran until 40200 before cutting off.

I would stick to training from scratch if you have custom data, but if you want to try the pre-trained weights again, you might find that changing max batches in the cfg file helps.

Answer 4

Modify OpenCV number in your darknet/Makefile to 0

OpenCV=0