Storing a query in Mongo

Question

This is the case: A webshop in which I want to configure which items should be listed in the sjop based on a set of parameters. I want this to be configurable, because that

Answer 1

I think the error message contains the important info you need to consider:

QUERY Error: field names cannot start with $

Since you are trying to store a query (or part of one) in a document, you'll end up with attribute names that contain mongo operator keywords (such as $or, $ne, $gt). The mongo documentation actually references this exact scenario - emphasis added

Field names cannot contain dots (i.e. .) or null characters, and they must not start with a dollar sign (i.e. $)...

---

I wouldn't trust 3rd party applications such as Robomongo in these instances. I suggest debugging/testing this issue directly in the mongo shell.

My suggestion would be to store an escaped version of the query in your document as to not interfere with reserved operator keywords. You can use the available JSON.stringify(my_obj); to encode your partial query into a string and then parse/decode it when you choose to retrieve it later on: JSON.parse(escaped_query_string_from_db)

Answer 2

Your approach of storing the query as a JSON object in MongoDB is not viable.

You could potentially store your query logic and fields in MongoDB, but you have to have an external app build the query with the proper MongoDB syntax.

MongoDB queries contain operators, and some of those have special characters in them.

There are rules for mongoDB filed names. These rules do not allow for special characters. Look here: https://docs.mongodb.org/manual/reference/limits/#Restrictions-on-Field-Names

The probable reason you can sometimes successfully create the doc using Robomongo is because Robomongo is transforming your query into a string and properly escaping the special characters as it sends it to MongoDB.

This also explains why your attempt to update them never works. You tried to create a document, but instead created something that is a string object, so your update conditions are probably not retrieving any docs.

Answer 3

Obviously my attempt to store a query in mongo the way I did was foolish as became clear from the answers from both @bigdatakid and @lix. So what I finally did was this: I altered the naming of the fields to comply to the mongo requirements.

E.g. instead of $or I used _$or etc. and instead of using a . inside the name I used a #. Both of which I am replacing in my Java code.

This way I can still easily try and test the queries outside of my program. In my Java program I just change the names and use the query. Using just 2 lines of code. It simply works now. Thanks guys for the suggestions you made.

String documentAsString = query.toJson().replaceAll("_\\$", "\\$").replaceAll("#", ".");
    Object q = JSON.parse(documentAsString);

Answer 4

I see two problems with your approach.

In following query

db.queries.insert({
   "name" : "query1",
   "query": { the thing printed above starting with "$or"... }
})

a valid JSON expects key, value pair. here in "query" you are storing an object without a key. You have two options. either store query as text or create another key inside curly braces.

Second problem is, you are storing query values without wrapping in quotes. All string values must be wrapped in quotes.

so your final document should appear as

db.queries.insert({
    "name" : "query1",
    "query": 'the thing printed above starting with "$or"... '
})

Now try, it should work.