Android SpeechRecognizer “confidence” values are confusing

Question

I\'m using the SpeechRecognizer via Intent:

Intent i = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH);
i.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL         


        

Answer 1

According to my interpretation of the documentation:

recognizerIntent.Extra_Results returns an ordered arrayList of strings, each of which is one suggestion as to what was said, with the string at index 0 being the suggestion the Recognizer is most confident of.

recognizerIntent.Extra_Confidence_Scores returns an array of floats corresponding to each of these suggestions.

So, if the results you are getting are correct(otherwise this might be a bug), then the recognizer has 1, and only 1, suggestion that it has confidence in and several others that it has only negligible or no confidence in.

I've been getting similar results. I've never had a set of results in which more than one suggestion had non-negligible confidence, just like you. e.g. 0.7435, 0.0, 0.0, 0.0, ......

I have however sometimes gotten a set of results in which ALL results have negligible confidence. e.g. 0.0, 0.0, 0.0, 0.0, 0.0, ......

So yes the first element in Results will always be the one the Recognizer is most confident of.

Answer 2

The conventional speech recognition algorithm allows to return confidence of just 1-best result because it is the result compared with other results to calculate the confidence. It is also possible to return N best results instead of just 1-best, however, it is much harder to calculate confidence for them.

It seems that Google implemented the conventional approach only and reserved place in the API for more detailed results with n-best confidence.

You just have to wait for Google to implement everything properly.

Answer 3

I haven't work with speech reorganization. But still, as you said you are getting float array value as 0.0, this implies float array is null . can you please check is the float[] is returning null or else.

ArrayList<String> results = data
            .getStringArrayListExtra(RecognizerIntent.EXTRA_RESULTS);

float[] confidence = data.getFloatArrayExtra(
            RecognizerIntent.EXTRA_CONFIDENCE_SCORES);
if (confidence == null)
{
 for (int i = 0; i < results.size(); i++)
  {
   Log.d(TAG, i + ": " + results.get(i));
  }
}
else
{
   for (int i = 0; i < results.size(); i++)

   {
     Log.d(TAG, i + ": " + heard.get(i) + " confidence : "  + confidence[i]);
  }
}

Can you please check the book Professional Android Sensor Programming  By Greg Milette, Adam Stroud this will surely help you. You will get some details on page 394 on this book.