Cloud Firestore collection count

Question

Is it possible to count how many items a collection has using the new Firebase database, Cloud Firestore?

If so, how do I do that?

Answer 1

I created a universal function using all these ideas to handle all counter situations (except queries).

The only exception would be when doing so many writes a second, it slows you down. An example would be likes on a trending post. It is overkill on a blog post, for example, and will cost you more. I suggest creating a separate function in that case using shards: https://firebase.google.com/docs/firestore/solutions/counters

// trigger collections
exports.myFunction = functions.firestore
    .document('{colId}/{docId}')
    .onWrite(async (change: any, context: any) => {
        return runCounter(change, context);
    });

// trigger sub-collections
exports.mySubFunction = functions.firestore
    .document('{colId}/{docId}/{subColId}/{subDocId}')
    .onWrite(async (change: any, context: any) => {
        return runCounter(change, context);
    });

// add change the count
const runCounter = async function (change: any, context: any) {

    const col = context.params.colId;

    const eventsDoc = '_events';
    const countersDoc = '_counters';

    // ignore helper collections
    if (col.startsWith('_')) {
        return null;
    }
    // simplify event types
    const createDoc = change.after.exists && !change.before.exists;
    const updateDoc = change.before.exists && change.after.exists;

    if (updateDoc) {
        return null;
    }
    // check for sub collection
    const isSubCol = context.params.subDocId;

    const parentDoc = `${countersDoc}/${context.params.colId}`;
    const countDoc = isSubCol
        ? `${parentDoc}/${context.params.docId}/${context.params.subColId}`
        : `${parentDoc}`;

    // collection references
    const countRef = db.doc(countDoc);
    const countSnap = await countRef.get();

    // increment size if doc exists
    if (countSnap.exists) {
        // createDoc or deleteDoc
        const n = createDoc ? 1 : -1;
        const i = admin.firestore.FieldValue.increment(n);

        // create event for accurate increment
        const eventRef = db.doc(`${eventsDoc}/${context.eventId}`);

        return db.runTransaction(async (t: any): Promise<any> => {
            const eventSnap = await t.get(eventRef);
            // do nothing if event exists
            if (eventSnap.exists) {
                return null;
            }
            // add event and update size
            await t.update(countRef, { count: i });
            return t.set(eventRef, {
                completed: admin.firestore.FieldValue.serverTimestamp()
            });
        }).catch((e: any) => {
            console.log(e);
        });
        // otherwise count all docs in the collection and add size
    } else {
        const colRef = db.collection(change.after.ref.parent.path);
        return db.runTransaction(async (t: any): Promise<any> => {
            // update size
            const colSnap = await t.get(colRef);
            return t.set(countRef, { count: colSnap.size });
        }).catch((e: any) => {
            console.log(e);
        });;
    }
}

This handles events, increments, and transactions. The beauty in this, is that if you are not sure about the accuracy of a document (probably while still in beta), you can delete the counter to have it automatically add them up on the next trigger. Yes, this costs, so don't delete it otherwise.

Same kind of thing to get the count:

const collectionPath = 'buildings/138faicnjasjoa89/buildingContacts';
const colSnap = await db.doc('_counters/' + collectionPath).get();
const count = colSnap.get('count');

Also, you may want to create a cron job (scheduled function) to remove old events to save money on database storage. You need at least a blaze plan, and there may be some more configuration. You could run it every sunday at 11pm, for example. https://firebase.google.com/docs/functions/schedule-functions

This is untested, but should work with a few tweaks:

exports.scheduledFunctionCrontab = functions.pubsub.schedule('5 11 * * *')
    .timeZone('America/New_York')
    .onRun(async (context) => {

        // get yesterday
        const yesterday = new Date();
        yesterday.setDate(yesterday.getDate() - 1);

        const eventFilter = db.collection('_events').where('completed', '<=', yesterday);
        const eventFilterSnap = await eventFilter.get();
        eventFilterSnap.forEach(async (doc: any) => {
            await doc.ref.delete();
        });
        return null;
    });

And last, don't forget to protect the collections in firestore.rules:

match /_counters/{document} {
  allow read;
  allow write: if false;
}
match /_events/{document} {
  allow read, write: if false;
}

Update: Queries

Adding to my other answer if you want to automate query counts as well, you can use this modified code in your cloud function:

    if (col === 'posts') {

        // counter reference - user doc ref
        const userRef = after ? after.userDoc : before.userDoc;
        // query reference
        const postsQuery = db.collection('posts').where('userDoc', "==", userRef);
        // add the count - postsCount on userDoc
        await addCount(change, context, postsQuery, userRef, 'postsCount');

    }
    return delEvents();

Which will automatically update the postsCount in the userDocument. You could easily add other one to many counts this way. This just gives you ideas of how you can automate things. I also gave you another way to delete the events. You have to read each date to delete it, so it won't really save you to delete them later, just makes the function slower.

/**
 * Adds a counter to a doc
 * @param change - change ref
 * @param context - context ref
 * @param queryRef - the query ref to count
 * @param countRef - the counter document ref
 * @param countName - the name of the counter on the counter document
 */
const addCount = async function (change: any, context: any, 
  queryRef: any, countRef: any, countName: string) {

    // events collection
    const eventsDoc = '_events';

    // simplify event type
    const createDoc = change.after.exists && !change.before.exists;

    // doc references
    const countSnap = await countRef.get();

    // increment size if field exists
    if (countSnap.get(countName)) {
        // createDoc or deleteDoc
        const n = createDoc ? 1 : -1;
        const i = admin.firestore.FieldValue.increment(n);

        // create event for accurate increment
        const eventRef = db.doc(`${eventsDoc}/${context.eventId}`);

        return db.runTransaction(async (t: any): Promise<any> => {
            const eventSnap = await t.get(eventRef);
            // do nothing if event exists
            if (eventSnap.exists) {
                return null;
            }
            // add event and update size
            await t.set(countRef, { [countName]: i }, { merge: true });
            return t.set(eventRef, {
                completed: admin.firestore.FieldValue.serverTimestamp()
            });
        }).catch((e: any) => {
            console.log(e);
        });
        // otherwise count all docs in the collection and add size
    } else {
        return db.runTransaction(async (t: any): Promise<any> => {
            // update size
            const colSnap = await t.get(queryRef);
            return t.set(countRef, { [countName]: colSnap.size }, { merge: true });
        }).catch((e: any) => {
            console.log(e);
        });;
    }
}
/**
 * Deletes events over a day old
 */
const delEvents = async function () {

    // get yesterday
    const yesterday = new Date();
    yesterday.setDate(yesterday.getDate() - 1);

    const eventFilter = db.collection('_events').where('completed', '<=', yesterday);
    const eventFilterSnap = await eventFilter.get();
    eventFilterSnap.forEach(async (doc: any) => {
        await doc.ref.delete();
    });
    return null;
}

I should also warn you that universal functions will run on every onWrite call period. It may be cheaper to only run the function on onCreate and on onDelete instances of your specific collections. Like the noSQL database we are using, repeated code and data can save you money.

UPDATE 11/20

I created an npm package for easy access: https://fireblog.io/blog/post/firestore-counters

Answer 2

I have try a lot with different approaches. And finally, I improve one of the methods. First you need to create a separate collection and save there all events. Second you need to create a new lambda to be triggered by time. This lambda will Count events in event collection and clear event documents. Code details in article. https://medium.com/@ihor.malaniuk/how-to-count-documents-in-google-cloud-firestore-b0e65863aeca

Answer 3

Solution using pagination with offset & limit:

public int collectionCount(String collection) {
        Integer page = 0;
        List<QueryDocumentSnapshot> snaps = new ArrayList<>();
        findDocsByPage(collection, page, snaps);
        return snaps.size();
    }

public void findDocsByPage(String collection, Integer page, 
                           List<QueryDocumentSnapshot> snaps) {
    try {
        Integer limit = 26000;
        FieldPath[] selectedFields = new FieldPath[] { FieldPath.of("id") };
        List<QueryDocumentSnapshot> snapshotPage;
        snapshotPage = fireStore()
                        .collection(collection)
                        .select(selectedFields)
                        .offset(page * limit)
                        .limit(limit)
                        .get().get().getDocuments();    
        if (snapshotPage.size() > 0) {
            snaps.addAll(snapshotPage);
            page++;
            findDocsByPage(collection, page, snaps);
        }
    } catch (InterruptedException | ExecutionException e) {
        e.printStackTrace();
    }
}

• findDocsPage it's a recursive method to find all pages of collection

• selectedFields for otimize query and get only id field instead full body of document

• limit max size of each query page

• page define inicial page for pagination

From the tests I did it worked well for collections with up to approximately 120k records!

Answer 4

var variable=0
variable=variable+querySnapshot.count

then if you are to use it on a String variable then

let stringVariable= String(variable)

Answer 5

Increment a counter using admin.firestore.FieldValue.increment:

exports.onInstanceCreate = functions.firestore.document('projects/{projectId}/instances/{instanceId}')
  .onCreate((snap, context) =>
    db.collection('projects').doc(context.params.projectId).update({
      instanceCount: admin.firestore.FieldValue.increment(1),
    })
  );

exports.onInstanceDelete = functions.firestore.document('projects/{projectId}/instances/{instanceId}')
  .onDelete((snap, context) =>
    db.collection('projects').doc(context.params.projectId).update({
      instanceCount: admin.firestore.FieldValue.increment(-1),
    })
  );

In this example we increment an instanceCount field in the project each time a document is added to the instances sub collection. If the field doesn't exist yet it will be created and incremented to 1.

The incrementation is transactional internally but you should use a distributed counter if you need to increment more frequently than every 1 second.

It's often preferable to implement onCreate and onDelete rather than onWrite as you will call onWrite for updates which means you are spending more money on unnecessary function invocations (if you update the docs in your collection).

Answer 6

There is no direct option available. You cant't do db.collection("CollectionName").count(). Below are the two ways by which you can find the count of number of documents within a collection.

1 :- Get all the documents in the collection and then get it's size.(Not the best Solution)

db.collection("CollectionName").get().subscribe(doc=>{
console.log(doc.size)
})

By using above code your document reads will be equal to the size of documents within a collection and that is the reason why one must avoid using above solution.

2:- Create a separate document with in your collection which will store the count of number of documents in the collection.(Best Solution)

db.collection("CollectionName").doc("counts")get().subscribe(doc=>{
console.log(doc.count)
})

Above we created a document with name counts to store all the count information.You can update the count document in the following way:-

• Create a firestore triggers on the document counts
• Increment the count property of counts document when a new document is created.
• Decrement the count property of counts document when a document is deleted.

w.r.t price (Document Read = 1) and fast data retrieval the above solution is good.