Aggregation Project Group By extracted day, month and year with Spring Data

Question

To be direct, how do i do this:

group._id = { 
    year: { $year : [{ $subtract: [ "$timestamp", 25200000 ]}] }, 
    month: { $month : [{ $subtract: [ "$timestamp", 25200000 ]}] }, 
    day: { $dayOfMonth : [{ $subtract: [ "$timestamp", 25200000 ]}] }
};

with spring Data

I tried this already and some other forms and were not successfull

Aggregation aggregation = Aggregation.newAggregation(
                Aggregation.match(c),
                Aggregation.project("programa", "custo", "duracao", "dataHora")
                    .andExpression("dataHora").minus(25200000).extractDayOfMonth().as("dia")
                    .andExpression("dataHora").minus(25200000).extractMonth().as("mes")
                    .andExpression("dataHora").minus(25200000).extractYear().as("ano"),
                Aggregation.group("programa", "ano", "mes", "dia")
                    .count().as("count")
                    .sum("custo").as("valorTotal")
                    .sum("duracao").as("duracaoTotal")
                    .first("dataHora").as("dataHora"),
                Aggregation.sort(Direction.ASC, "dataHora")
        );

I need to group by day, month and year in mongodb, or else i will need to convert all this grouped data in code.

Thanks in advance

Answer 1

You are running into a limitation of spring mongo in what you can do for field calculations in a single $project stage, and indeed you are likely already writing as a separate $project since you discover that you cannot project custom named fields directly in a $group _id at present either.

So you would be better off keeping this all in the $group, as well as using a different method for rounding your adjusted dates to local time.

The better way to write your $group would therefore be:

{ "$group": {
  "_id": {
    "programa": "$programa",
    "dataHora": {
      "$add": [
        { "$subtract": [
          { "$subtract": [{ "$subtract": ["$dataHora", new Date(0)] }, 25200000 ] },
          { "$mod": [
            { "$subtract": [{ "$subtract": ["$dataHora", new Date(0)] }, 25200000 ] },
            1000 * 60 * 60 * 24
          ]}
        ]},
        new Date(0)
      ]
    }
  },
  "count": { "$sum": 1 },
  "valorTotal": { "$sum": "$custo" },
  "duracaoTotal": { "$sum": "$duracao" },
  "dataHora": { "$first": "$dataHora" }
}}

Of course to use this sort of structure with spring-mongo you need a custom implementation of the aggregation stage operation that can take a defined DBObject:

public class CustomGroupOperation implements AggregationOperation {
    private DBObject operation;

    public CustomGroupOperation (DBObject operation) {
        this.operation = operation;
    }

    @Override
    public DBObject toDBObject(AggregationOperationContext context) {
        return context.getMappedObject(operation);
    }
}

Which you then use in context like this:

    Aggregation aggregation = Aggregation.newAggregation(
            Aggregation.match(c),
            new CustomGroupOperation(
                new BasicDBObject("$group",
                    new BasicDBObject("_id",
                        new BasicDBObject("programa","$programa")
                            .append("dataHora",
                                new BasicDBObject("$add",Arrays.asList(
                                    new BasicDBObject("$subtract",Arrays.asList(
                                        new BasicDBObject("$subtract",Arrays.asList(
                                            new BasicDBObject("$subtract",Arrays.asList(
                                                "$dataHora", new Date(0)
                                            )),
                                            25200000
                                        )),
                                        new BasicDBObject("$mod",Arrays.asList(
                                            new BasicDBObject("$subtract",Arrays.asList(
                                                new BasicDBObject("$subtract",Arrays.asList(
                                                    "$dataHora", new Date(0)
                                                )),
                                                25200000
                                            )),
                                            1000 * 60 * 60 * 24
                                        ))
                                    )),
                                    new Date(0)
                                ))
                            )
                    )
                    .append("count",new BasicDBObject("$sum",1))
                    .append("valorTotal",new BasicDBObject("$sum","$custo"))
                    .append("duracaoTotal",new BasicDBObject("$sum","$duracao"))
                    .append("dataHora",new BasicDBObject("$first","$dataHora"))
                )
            ),
            Aggregation.sort(Direction.ASC,"_id.dataHora")
    );

Since the custom class abstracts from the same basic class used by the built in helper methods, it can be used alongside them as shown.

How the basic process with the date math works here is that when you $subtract one BSON Date object from another then the result is the milliseconds of difference, and in this case from the epoch date ( Date(0) ) which just extracts the milliseconds value. This allows you to do the math to round to the current date value by the modulo ( $mod ) from the number of milliseconds in one day.

Much as you originally tried, when you then $add that millisecond value to a BSON Date object the returned value is again a BSON Date. So adding to an object representing epoch returns a new Date Object, but rounded to the current date.

This is usually a lot more useful than extracting parts via date aggregation operators, and also works out to be a bit shorter to code, especially when adjusting the time from UTC as you are doing here.

Though the contruction of the $group here is a bit more terse than the helper functions of spring mongo are trying to avoid, it is a lot more efficient in the end than running a separate $project stage to transform the field values that you really only want in the $group stage anyway.