How create a new array field with the aggregate framework

Question

I\'m starting to use mongoDb and I\'m stuck with a simple use case.

Let\'s say I\'ve got a collection \'aCollection\' with entries such as this:

Answer 1

In Modern MongoDB releases the most efficient way is to simply notate the array using the existing document properties. Direct notation of arrays was introduced in MongoDB 3.2:

db.collection.aggregate([
  { "$project": {
    "lat": 1,
    "long": 1,
    "geometry": {
      "type": { "$literal": "Point" },
      "coordinates": [ "$lat", "$long" ]
    }
  }},
  { "$out": "newcollection" }
])

Or even using $addFields to simply "append" the new property to the documents:

db.collection.aggregate([
  { "$addFields": {
    "geometry": {
      "type": { "$literal": "Point" },
      "coordinates": [ "$lat", "$long" ]
    }
  }},
  { "$out": "newcollection" }
])

---

If you are using MongoDB 2.6 and above you can do this with the aggregation framework and avoid looping results in your client program in order to create a new collection.

The main feature here that help you are the $out operator for sending the output to a new collection. But also being a little clever in order to create the array that you need.

db.collection.aggregate([
    { "$project": {
        "lat": 1,
        "long": 1,
        "type": { "$literal": ["lat","long"] }
    }},
    { "$unwind": "$type" },
    { "$group": {
        "_id": "$_id",
        "lat": { "$first": "$lat" },
        "long": { "$first": "$long" },
        "coordinates": {
            "$push": {
                "$cond": [
                    { "$eq": [ "$type", "lat" ] },
                    "$lat",
                    "$long"
                ]
            }
        }
    }},
    { "$project": {
        "lat": 1,
        "long": 1,
        "geometry": { 
            "type": { "$literal": "Point" },
            "coordinates": "$coordinates"
        }
    }},
    { "$out": "newcollection" }
])

So this makes use of the $literal operator in order to specify a new array at the head of the pipeline. This operator will put content in the document property exactly how it is supplied. So no variable substitutions are allowed, hence "literal".

In order to create the "coordintes" array, we simply unwind that first array which essentially creates two of every document with a different value in "type". This is then used in the $group stage to conditionally $push either the "$lat" or "$long" value onto that array.

Finally use $project again to finalize the document structure and then $out sends all output to the new collection.

---

Note that this only makes sense if your intention is to create a new collection and avoid sending traffic "over the wire". This could not be used purely within the aggregation framework to re-shape your document with the intent to then do a "geo-spatial" query in that same aggregation pipeline as "geo-spatial" queries will only work when actually indexed on a collection.

So this may help you create a new collection as you want to, but at least it serves as example ( or two examples actually ) of how to create an array's out of different values with the aggregation framework.

Answer 2

Ok, so first of all let us understand what aggregation framework does. It only queries the available data and returns the result. It does NOT modify the original documents! That is why the second piece of code that you have written performs an aggregation, does a projection and displays some result of a successful run of an aggregate query.

To make the changes that you are trying to do, you can either try the 'quick and dirty' approach of Martin Konecny's answer or you can modify the code for uploading the data and perform a fresh upload.

Since I see that instead of using decimal point you are using comma in the lat/long values I would say that if later on in time you will try to query this geo-spatial data, you better upload it again properly.

Answer 3

You don't need the aggregation feature for this. A find, forEach, and insert is one potential approach:

db.aCollection.find().forEach( function(myDoc) {
  myDoc.geometry = {type: "Point", coordinates: [myDoc.lat, myDoc.long]};
  db.newCollection.insert(myDoc);
});

It calls an individual insert for each document, but is quick and dirty if you have a small collection.

Answer 4

It looks like MongoDB 3.2 provides fairly simple and elegant way to create GeoJSON points using aggregation framework.

We had to perform the transformation on approximately 2 Million records two times a day, so aggregation framework was the fastest and probably most reliable approach.

Below is a Mongoose example of how to transform geolocation data from collection with longitude/latitude into a collection with GeoJSON points.

Locations
    .aggregate([
        {
            $project : {
                _id: 0,
                "location": {
                    "type": { $literal: "Point" }, 
                    "coordinates": ["$longitude", "$latitude"]
                }
            }
        },
        {
            $out : 'test_1'
        }])
    .exec(function(err,data) {
        if (err) {
            console.error(err);
        } else {
            console.log("Done transforming.");
        }
    });