Why Mongo Spark connector returns different and incorrect counts for a query?

Question

I\'m evaluating Mongo Spark connector for a project and I\'m getting the inconsistent results. I use MongoDB server version 3.4.5, Spark (via PySpark) version 2.2.0, Mongo S

Answer 1

This issue was mostly due to SPARK-151 bug in 2.2.0 Mongo Connector. It is resolved in 2.2.1 version, which I have confirmed. You can continue to use default partitioner with 2.2.1.