Searching for compound indexes in IndexedDB

Question

After reading here for ages, I've finally registered to ask a question. I've been messing around with IndexedDB lately and stumbled over a problem with compound indexes (I use them somilar to the example here).

I have an object in the objectstore with a string value, and a couple of integer values. E. g.:

[description:text, value1:int, value2:int, value3:int]

I created a compound index on this object like this:

("compoundIndex", ["value1" , "value2" , "value3"] , { unique: false });

In the html I got a couple of select boxes and a textfield, that allows the user to search for specific entries. The integers are passed as a keyrange to the opencursor-function on the index. Then I use indexOf(textfield) on the resulting set (like done here)

If the selectbox has a value, that value is used as upper and lower bound. If the select box is untouched, the lower range is 1 and the upper is a MAX_INT variable I declared (like described here).

sample code:

transaction = db.transaction(["schaden"] , "readonly").objectStore("schaden");
index = transaction.index("compoundIndex");

// keyrange-arrays from another function    
lowerBound = [valueOneLower, valueTwoLower, valueThreeLower];
upperBound = [valueOneUpper, valueTwoUpper, valueThreeUpper];
range = IDBKeyRange.bound( lowerBound, upperBound );

index.openCursor(range).onsuccess = function(e){
  var cursor = e.target.result;
  if (cursor){
    if (getTextfield.length == 0){
      console.log("Entry found: " + cursor.value.description + ". Object: " + JSON.stringify(cursor.value));
    }else if (cursor.value.bezeichnung.indexOf(getTextfield) !== -1){
      console.log("Entry found: " + cursor.value.description + ". Object: " + JSON.stringify(cursor.value));
    };
    cursor['continue']();                           
    };
  };    

I can search for entries perfectly well, when I have all values set in all the select-boxes. However, if I leave a field open, it messes up the search. Let's say I have not touched the value1-select box, and set the other boxes to 2, I'll get the lowerBound = [1,2,2] and the upperBound = [4294967295,2,2]. This will give me back all entries in my IDB, it doesn't take the 2nd and 3rd value into account.

Is this intended? Or is there a way around this? I have been searching for information about this over and over but seem to be in a dead end. My naive understanding of this API led me to believe it would take all array fields into account on the search. Since the object and therefor also the index I use are much more complex than the example above, performing searches on multiple indexes would be quite messy.

Thanks for your insights!

Edit: To make it a little more clear after the first comments. Let's say if have the following object in the object store:

obj1 { val1 = 1 , val2 = 3 , val3 = 1 }
obj2 { val1 = 1 , val2 = 2 , val3 = 2 }
obj3 { val1 = 2 , val2 = 1 , val3 = 3 }
obj4 { val1 = 1 , val2 = 1 , val3 = 1 }
obj5 { val1 = 1 , val2 = 2 , val3 = 3 }

The index sorts it the way expected:

#1 [1,1,1] obj4
#2 [1,2,2] obj2
#3 [1,2,3] obj5
#4 [1,3,1] obj1
#5 [2,1,3] obj3

Let's assume now I search for the range (lower[1,1,1] , upper[1,1,1]) I'll get obj4. This is the behaviour when all select boxes have selected option 1. Now if I search for an entry with val1 = 1, val2 = unknown and val3 = 1, I get the following range: lower[1,1,1] , upper[1,4294967295,1]. Expected results are obj4 [1,1,1] and obj1 [1,3,1]. Instead of these, the result is giving me 4 hits, namely obj4, obj2, obj5 and obj1 although val3 of obj2 and obj5 doesn't match the key range.

Answer 1

1. When you create an index on an array, the entries of your store only appear in the index if each element in the array that corresponds to a property in the underlying object has a defined value.

   To get around this obstacle, always store defined values in your underlying object store. For example, to represent a boolean property, use an integer, where 0 is false, and 1 is true. This way, each object in the store can appear in the index. indexedDB's behavior here is quite different than truthy/falsy handling in plain old javascript (where 0 == undefined ).

2. The key range you specify when opening a cursor on a array-based index must use defined parameters for each element of the array.

   To get around this obstacle, you must specify all boundaries, even if those boundaries are not real values (e.g. like in my example you linked to, 200 as max age works because we can safely assume no one is 200 yrs old).

So to address your question, it might be a problem in your code in that one of the parameters to your boundaries variables (either [valueOneLower, valueTwoLower, valueThreeLower] or [valueOneUpper, valueTwoUpper, valueThreeUpper]) is not defined.

Based on your comments, I suggest that you test your expectations with indexedDB.cmp. It is pretty simple to write these tests. It does not require any database connection. Here is a pretty basic example to get you started:

// Build our test values

var lower1 = 1, lower2 = 1, lower3 = 1;
var upper1 = 3, upper3 = 3, upper3 = 3;
var middle1 = 2, middle2 = 2, middle3 = 2;

var lowerBound = [lower1,lower2,lower3];
var upperBound = [upper1,upper2,upper3];
var middleValue = [middle1,middle2,middle3];

// As the linked page provides, cmp returns -1 if first is less than second, 0 if equal, 1 if first is greater than second.

var lowerVsMiddle = indexedDB.cmp(lowerBound, middleValue);
console.log('Is %s < %s ? %s', lowerBound, middleValue, lowerVsMiddle == -1);
console.log('Is %s > %s ? %s', lowerBound, middleValue, lowerVsMiddle == 1);

var upperVsMiddle = indexedDB.cmp(upperBound, middleValue);
console.log('Is %s < %s ? %s', upperBound, middleValue, upperVsMiddle == -1);
console.log('Is %s > %s ? %s', upperBound, middleValue, upperVsMiddle == 1);

You should be able to answer your questions accurately by running tests like this.

I retrieved the relevant part of the indexedDB spec for you. First note that "An Array is only a valid key if every item in the array is defined and is a valid key...". This ties into whether the object will appear in the index, and also ties into whether your key parameters to either cmp or IDBKeyRange.bound/lowerBound/upperBound will work. Second, farther down, note the following:

Values of type Array are compared to other values of type Array as follows:

1. Let A be the first Array value and B be the second Array value.
2. Let length be the lesser of A's length and B's length.
3. Let i be 0.
4. If the ith value of A is less than the ith value of B, then A is less than B. Skip the remaining steps.
5. If the ith value of A is greater than the ith value of B, then A is greater than B. Skip the remaining steps.
6. Increase i by 1.
7. If i is not equal to length, go back to step 4. Otherwise continue to next step.
8. If A's length is less than B's length, then A is less than B. If A's length is greater than B's length, then A is greater than B. Otherwise A and B are equal.

From the KeyRange section: A key is in a key range if both the following conditions are fulfilled:

• The key range lower value is undefined or less than key. It may also be equal to key if lowerOpen is false.
• The key range upper value is undefined or greater than key. It may also be equal to key if upperOpen is false.

One more clarification now that I understand your question based on the comments and your further edits: Essentially indexedDB is providing a union behavior of the criteria but you want is an intersection. One way of solving this is to not think about the data in normal form at all, but to think about how to setup the data so it can be queried in the manner you want. It is interesting food for thought and I do not have an immediate answer for you.

Answer 2

You need another compound index for that case.

Alternatively you can use key joining as describe here YDN-DB - Incorrect results using mixed data types with SortedMerge