Saving ORB feature vectors using OpenCV4Android (java API)

Question

I have a training set of images, for each of which I\'ve detected and computed their feature vectors (using ORB feature descriptors and extractors. The questions is<

Answer 1

I see that you have considered using the Android SQLite database:

My current choice is to opt for a sqlLite database, having a table with two cols; id and feature (as frequently suggested online); to tabulate all the 16k features in sqlLite. That seems rather phone-storage intensive, but it's the most reasonable solution I can find.

There is a way to save and retrieve MatOfKeyPoint to SQLite database with reasonable efficiency.

Using the database has the advantage of not needing to request write external storage permission from the user (although that permission might be needed for some other of your apps functions).

There is a complete Android solution, with Java code which can be found in this StackOverflow Answer.

The following is a description of what's going on in the code from that answer...

MatOfKeyPoint to byte[] and some attributes

To save to the database, you need to save to a byte[] object. Using the MatOfKeyPoint.get() method, you can get a populated float[]. Then using ByteBuffer.putFloat(), you can loop through all of your floats, finally getting a populated byte[] by using ByteBuffer.array().

You also need to save some attirbutes MatOfKeyPoint.rows(), MatOfKeyPoint.cols(), and MatOfKeyPoint.type() to your database, along with the blob of byte[].

Database blob (byte[]) to MatOfKeyPoint

To reconstitute your MatOfKeyPoint object from the database, first you make a float[] out of your blob. Use ByteBuffer.wrap(blob), then run ByteBuffer.asFloatBuffer(), and finally FloatBuffer.get() with a properly sized new float[].

Now that you have yourFloatArray you run MatOfKeyPoint.create(rows, cols, type), those three things coming from the database record. Finally you run MatOfKeyPoint.put(0, 0, yourFloatArray).

Answer 2

In my opinion the most universal way to store the keypoints is to first convert them to a data-interchange format like JSON.

After you are able to do that conversion you have a lot of flexibility to store it. JSON is easily converted to a String and/or sent through a network connection.

With OpenCV C++ you are able to store data as YAML, but that is not available for Android yet.

To parse JSON in Java you can use this easy to use library Google GSON.

And here is my first attempt to do exactly that:

 public static String keypointsToJson(MatOfKeyPoint mat){
    if(mat!=null && !mat.empty()){          
        Gson gson = new Gson();

        JsonArray jsonArr = new JsonArray();            

        KeyPoint[] array = mat.toArray();
        for(int i=0; i<array.length; i++){
            KeyPoint kp = array[i];

            JsonObject obj = new JsonObject();

            obj.addProperty("class_id", kp.class_id); 
            obj.addProperty("x",        kp.pt.x);
            obj.addProperty("y",        kp.pt.y);
            obj.addProperty("size",     kp.size);
            obj.addProperty("angle",    kp.angle);                          
            obj.addProperty("octave",   kp.octave);
            obj.addProperty("response", kp.response);

            jsonArr.add(obj);               
        }

        String json = gson.toJson(jsonArr);         

        return json;
    }
    return "{}";
}

public static MatOfKeyPoint keypointsFromJson(String json){
    MatOfKeyPoint result = new MatOfKeyPoint();

    JsonParser parser = new JsonParser();
    JsonArray jsonArr = parser.parse(json).getAsJsonArray();        

    int size = jsonArr.size();

    KeyPoint[] kpArray = new KeyPoint[size];

    for(int i=0; i<size; i++){
        KeyPoint kp = new KeyPoint(); 

        JsonObject obj = (JsonObject) jsonArr.get(i);

        Point point = new Point( 
                obj.get("x").getAsDouble(), 
                obj.get("y").getAsDouble() 
        );          

        kp.pt       = point;
        kp.class_id = obj.get("class_id").getAsInt();
        kp.size     =     obj.get("size").getAsFloat();
        kp.angle    =    obj.get("angle").getAsFloat();
        kp.octave   =   obj.get("octave").getAsInt();
        kp.response = obj.get("response").getAsFloat();

        kpArray[i] = kp;
    }

    result.fromArray(kpArray);

    return result;
}

Answer 3

I would suggest storing the feature vectors as images to have a simple and compact representation. You could even use non-destructive compression such as png to minimize file size.