C++ - Is it possible to implement memory leak testing in a unit test?

Question

I\'m trying to implement unit testing for my code and I\'m having a hard time doing it.

Ideally I would like to test some classes not only for good functionality but

Answer 1

You can use Google's tcmalloc allocation library, which provides a heapchecker.

(Note that heapchecking may add noticeable overhead to your program's performance, so you probably only want to enable it on debug builds or unit tests.)

And you asked for example code, so here it is.

Answer 2

1) After some investigation of mine and based on very nice solution (for Windows) of Chris Becke, I have made a very similar solution for Linux OS.

2)My memory leak detection goals:

Are pretty clear - detect leaks while as well:

2.1) Ideally in a precise manner - indicate exactly how many bytes were allocated YET not deallocated.

2.2) Best effort - if not exactly, indicate at a “false positive” manner (tell us about a leak even if it is not necessarily one and at the same time DO NOT miss any leak detection). It is better to be more harsh on ourselves here.

2.3) Since I am writing my unit tests in the GTest framework - test each GTest unit test as an “atomic entity”.

2.4) Take into consideration also “C-style” allocations (deallocations) using malloc/free.

2.5) Ideally - take into consideration C++ “in place allocations”.

2.6) Easy to use and integrate into an existing code (GTest based classes for unit testing).

2.7) Have the ability to “configure” main checks settings (enable/disable memory checking, etc…) for each test and/or entire test class.

3) Solution architecture:

My solution uses the inherited capabilities of using the GTest framework, so it defines a “base” class for each unit test class we will add in the future. Basically, the base class main functionalities can be divided into the following:

3.1) Run the “first” GTest style test in order to understand the amount of “extra memory” allocated on the heap in the case of a test failure.As Chris Becke mentioned in the last sentence of its answer above.

3.2) Easy to integrate - simply inherit from this base class and write your unit tests “TEST_F style” functions.

3.3.1) For each test, we can decide whether to stated otherwise perform the memory leak check or not.This is done via the SetIgnoreMemoryLeakCheckForThisTest() metohd. Note: No need to “reset” it again - it will happen automatically for the next test due to the way GTest unit tests works (they call the Ctor prior for each function call).

3.3.2) Also, if for some reason you know in advance that your test will “miss” some deallocations of memory and you know the amount - you can take advantage of the two functions in order to take this fact into considerations once performing the memory check (which, by the way, performed by “simply” subtracting the amount of memory in use at the beginning of the test from the amount of memory in use at the end of the test).

Below is the header base class:

// memoryLeakDetector.h:
#include "gtest/gtest.h"
extern int g_numOfExtraBytesAllocatedByGtestUponTestFailure;

// The fixture for testing class Foo.
class MemoryLeakDetectorBase : public ::testing::Test 
{
// methods:
// -------
public:
    void SetIgnoreMemoryLeakCheckForThisTest() { m_ignoreMemoryLeakCheckForThisTest= true; } 
    void SetIsFirstCheckRun() { m_isFirstTestRun = true; }

protected:

    // You can do set-up work for each test here.
    MemoryLeakDetectorBase();

    // You can do clean-up work that doesn't throw exceptions here.
    virtual ~MemoryLeakDetectorBase();

    // If the constructor and destructor are not enough for setting up
    // and cleaning up each test, you can define the following methods:

    // Code here will be called immediately after the constructor (right
    // before each test).
    virtual void SetUp();

    // Code here will be called immediately after each test (right
    // before the destructor).
    virtual void TearDown();

private:
    void getSmartDiff(int naiveDiff);
    // Add the extra memory check logic according to our 
    // settings for each test (this method is invoked right
    // after the Dtor).
    virtual void PerformMemoryCheckLogic();

// members:
// -------
private:
    bool m_ignoreMemoryLeakCheckForThisTest;
    bool m_isFirstTestRun;
    bool m_getSmartDiff;
    size_t m_numOfBytesNotToConsiderAsMemoryLeakForThisTest;
    int m_firstCheck;
    int m_secondCheck;
};

And here is the source of this base class:

// memoryLeakDetectorBase.cpp
#include <iostream>
#include <malloc.h>

#include "memoryLeakDetectorBase.h"

int g_numOfExtraBytesAllocatedByGtestUponTestFailure = 0;

static int display_mallinfo_and_return_uordblks()
{
    struct mallinfo mi;

    mi = mallinfo();
    std::cout << "========================================" << std::endl;
    std::cout << "========================================" << std::endl;
    std::cout << "Total non-mmapped bytes (arena):" << mi.arena << std::endl;
    std::cout << "# of free chunks (ordblks):" << mi.ordblks << std::endl;
    std::cout << "# of free fastbin blocks (smblks):" << mi.smblks << std::endl;
    std::cout << "# of mapped regions (hblks):" << mi.hblks << std::endl;
    std::cout << "Bytes in mapped regions (hblkhd):"<< mi.hblkhd << std::endl;
    std::cout << "Max. total allocated space (usmblks):"<< mi.usmblks << std::endl;
    std::cout << "Free bytes held in fastbins (fsmblks):"<< mi.fsmblks << std::endl;
    std::cout << "Total allocated space (uordblks):"<< mi.uordblks << std::endl;
    std::cout << "Total free space (fordblks):"<< mi.fordblks << std::endl;
    std::cout << "Topmost releasable block (keepcost):" << mi.keepcost << std::endl;
    std::cout << "========================================" << std::endl;
    std::cout << "========================================" << std::endl;
    std::cout << std::endl;
    std::cout << std::endl;

    return mi.uordblks;
}

MemoryLeakDetectorBase::MemoryLeakDetectorBase() 
    : m_ignoreMemoryLeakCheckForThisTest(false)
    , m_isFirstTestRun(false)
    , m_getSmartDiff(false)
    , m_numOfBytesNotToConsiderAsMemoryLeakForThisTest(0)
{
    std::cout << "MemoryLeakDetectorBase::MemoryLeakDetectorBase" << std::endl;
    m_firstCheck = display_mallinfo_and_return_uordblks();
}

MemoryLeakDetectorBase::~MemoryLeakDetectorBase() 
{
    std::cout << "MemoryLeakDetectorBase::~MemoryLeakDetectorBase" << std::endl;
    m_secondCheck = display_mallinfo_and_return_uordblks();
    PerformMemoryCheckLogic();
}

void MemoryLeakDetectorBase::PerformMemoryCheckLogic()
{
    if (m_isFirstTestRun) {
        std::cout << "MemoryLeakDetectorBase::PerformMemoryCheckLogic - after the first test" << std::endl;
        int diff = m_secondCheck - m_firstCheck;
        if ( diff > 0) {
            std::cout << "MemoryLeakDetectorBase::PerformMemoryCheckLogic - setting g_numOfExtraBytesAllocatedByGtestUponTestFailure to:" << diff << std::endl;
            g_numOfExtraBytesAllocatedByGtestUponTestFailure = diff;
        }
        return;
    }

    if (m_ignoreMemoryLeakCheckForThisTest) {
        return;
    }
    std::cout << "MemoryLeakDetectorBase::PerformMemoryCheckLogic" << std::endl;

    int naiveDiff = m_secondCheck - m_firstCheck;

    // in case you wish for "more accurate" difference calculation call this method
    if (m_getSmartDiff) {
        getSmartDiff(naiveDiff);
    }

    EXPECT_EQ(m_firstCheck,m_secondCheck);
    std::cout << "MemoryLeakDetectorBase::PerformMemoryCheckLogic - the difference is:" << naiveDiff << std::endl;
}

void MemoryLeakDetectorBase::getSmartDiff(int naiveDiff)
{
    // according to some invastigations and assumemptions, it seems like once there is at least one 
    // allocation which is not handled - GTest allocates 32 bytes on the heap, so in case the difference
    // prior for any further substrcutions is less than 32 - we will assume that the test does not need to 
    // go over memory leak check...
    std::cout << "MemoryLeakDetectorBase::getMoreAccurateAmountOfBytesToSubstructFromSecondMemoryCheck - start" << std::endl; 
    if (naiveDiff <= 32) {
        std::cout << "MemoryLeakDetectorBase::getSmartDiff - the naive diff <= 32 - ignoring..." << std::endl;
        return;
    }

    size_t numOfBytesToReduceFromTheSecondMemoryCheck = m_numOfBytesNotToConsiderAsMemoryLeakForThisTest + g_numOfExtraBytesAllocatedByGtestUponTestFailure;
    m_secondCheck -= numOfBytesToReduceFromTheSecondMemoryCheck;
    std::cout << "MemoryLeakDetectorBase::getSmartDiff - substructing " << numOfBytesToReduceFromTheSecondMemoryCheck << std::endl;
}

void MemoryLeakDetectorBase::SetUp() 
{
    std::cout << "MemoryLeakDetectorBase::SetUp" << std::endl;
}

void MemoryLeakDetectorBase::TearDown() 
{
    std::cout << "MemoryLeakDetectorBase::TearDown" << std::endl;
}

// The actual test of this module:


TEST_F(MemoryLeakDetectorBase, getNumOfExtraBytesGTestAllocatesUponTestFailureTest) 
{
    std::cout << "MemoryLeakDetectorPocTest::getNumOfExtraBytesGTestAllocatesUponTestFailureTest - START" << std::endl;

    // Allocate some bytes on the heap and DO NOT delete them so we can find out the amount 
    // of extra bytes GTest framework allocates upon a failure of a test.
    // This way, upon our legit test failure, we will be able to determine of many bytes were NOT
    // deleted EXACTLY by our test.

    std::cout << "MemoryLeakDetectorPocTest::getNumOfExtraBytesGTestAllocatesUponTestFailureTest - size of char:" << sizeof(char) << std::endl;
    char* pChar = new char('g');
    SetIsFirstCheckRun();
    std::cout << "MemoryLeakDetectorPocTest::getNumOfExtraBytesGTestAllocatesUponTestFailureTest - END" << std::endl;
}

Finally, a sample "GTest-based" unit test class that uses this bases class and illustrates usages and several different POC's (proof of concept) to all sort of different allocations and verification if we are able (or not) to detect the missed de-allocations.

// memoryLeakDetectorPocTest.cpp
#include "memoryLeakDetectorPocTest.h"
#include <cstdlib>  // for malloc

class MyObject 
{

public:
    MyObject(int a, int b) : m_a(a), m_b(b) { std::cout << "MyObject::MyObject" << std::endl; }
    ~MyObject() { std::cout << "MyObject::~MyObject" << std::endl; }  
private:
    int m_a;
    int m_b;
};

MemoryLeakDetectorPocTest::MemoryLeakDetectorPocTest() 
{
    std::cout << "MemoryLeakDetectorPocTest::MemoryLeakDetectorPocTest" << std::endl;
}

MemoryLeakDetectorPocTest::~MemoryLeakDetectorPocTest() 
{
    std::cout << "MemoryLeakDetectorPocTest::~MemoryLeakDetectorPocTest" << std::endl;
}

void MemoryLeakDetectorPocTest::SetUp() 
{
    std::cout << "MemoryLeakDetectorPocTest::SetUp" << std::endl;
}

void MemoryLeakDetectorPocTest::TearDown() 
{
    std::cout << "MemoryLeakDetectorPocTest::TearDown" << std::endl;
}

TEST_F(MemoryLeakDetectorPocTest, verifyNewAllocationForNativeType) 
{

    std::cout << "MemoryLeakDetectorPocTest::verifyNewAllocationForNativeType - START" << std::endl;

    // allocate some bytes on the heap and intentially DONT release them...
    const size_t numOfCharsOnHeap = 23;
    std::cout << "size of char is:" << sizeof(char)  << " bytes" << std::endl;
    std::cout << "allocating " << sizeof(char) * numOfCharsOnHeap << " bytes on the heap using new []" << std::endl;
    char* arr = new char[numOfCharsOnHeap];

    // DO NOT delete it on purpose...
    //delete [] arr;
    std::cout << "MemoryLeakDetectorPocTest::verifyNewAllocationForNativeType - END" << std::endl;
}

TEST_F(MemoryLeakDetectorPocTest, verifyNewAllocationForUserDefinedType) 
{
    std::cout << "MemoryLeakDetectorPocTest::verifyNewAllocationForUserDefinedType - START" << std::endl;

    std::cout << "size of MyObject is:" << sizeof(MyObject)  << " bytes" << std::endl;
    std::cout << "allocating MyObject on the heap using new" << std::endl;
    MyObject* myObj1 = new MyObject(12, 17);

    delete myObj1;
    std::cout << "MemoryLeakDetectorPocTest::verifyNewAllocationForUserDefinedType - END" << std::endl;
}

TEST_F(MemoryLeakDetectorPocTest, verifyMallocAllocationForNativeType) 
{
    std::cout << "MemoryLeakDetectorPocTest::verifyMallocAllocationForNativeType - START" << std::endl;
    size_t numOfDoublesOnTheHeap = 3;
    std::cout << "MemoryLeakDetectorPocTest::verifyMallocAllocationForNativeType - sizeof double is " << sizeof(double) << std::endl; 
    std::cout << "MemoryLeakDetectorPocTest::verifyMallocAllocationForNativeType - allocaitng " << sizeof(double) * numOfDoublesOnTheHeap << " bytes on the heap" << std::endl;
    double* arr = static_cast<double*>(malloc(sizeof(double) * numOfDoublesOnTheHeap));

    // NOT free-ing them on purpose !!
    // free(arr);
    std::cout << "MemoryLeakDetectorPocTest::verifyMallocAllocationForNativeType - END" << std::endl;
}

TEST_F(MemoryLeakDetectorPocTest, verifyNewAllocationForNativeSTLVectorType) 
{
    std::cout << "MemoryLeakDetectorPocTest::verifyNewAllocationForNativeSTLVectorType - START" << std::endl;
    std::vector<int> vecInt;
    vecInt.push_back(12);
    vecInt.push_back(15);
    vecInt.push_back(17);

    std::cout << "MemoryLeakDetectorPocTest::verifyNewAllocationForNativeSTLVectorType - END" << std::endl;
}

TEST_F(MemoryLeakDetectorPocTest, verifyNewAllocationForUserDefinedSTLVectorType) 
{
    std::cout << "MemoryLeakDetectorPocTest::verifyNewAllocationForUserDefinedSTLVectorType - START" << std::endl;
    std::vector<MyObject*> vecMyObj;
    vecMyObj.push_back(new MyObject(7,8));
    vecMyObj.push_back(new MyObject(9,10));

    size_t vecSize = vecMyObj.size();
    for (int i = 0; i < vecSize; ++i) {
        delete vecMyObj[i];
    }

    std::cout << "MemoryLeakDetectorPocTest::verifyNewAllocationForUserDefinedSTLVectorType - END" << std::endl;
}

TEST_F(MemoryLeakDetectorPocTest, verifyInPlaceAllocationAndDeAllocationForUserDefinedType) 
{
     std::cout << "MemoryLeakDetectorPocTest::verifyInPlaceAllocationAndDeAllocationForUserDefinedType - START" << std::endl;
    void* p1 = malloc(sizeof(MyObject));
    MyObject *p2 = new (p1) MyObject(12,13);

    p2->~MyObject();
    std::cout << "MemoryLeakDetectorPocTest::verifyInPlaceAllocationAndDeAllocationForUserDefinedType - END" << std::endl;
}

TEST_F(MemoryLeakDetectorPocTest, verifyInPlaceAllocationForUserDefinedType) 
{
    std::cout << "MemoryLeakDetectorPocTest::verifyInPlaceAllocationForUserDefinedType - START" << std::endl;
    void* p1 = malloc(sizeof(MyObject));
    MyObject *p2 = new (p1) MyObject(12,13);

    // Dont delete the object on purpose !!
    //p2->~MyObject();
    std::cout << "MemoryLeakDetectorPocTest::verifyInPlaceAllocationForUserDefinedType - END" << std::endl;
}

The header file of this class:

// memoryLeakDetectorPocTest.h
#include "gtest/gtest.h"
#include "memoryLeakDetectorBase.h"

// The fixture for testing class Foo.
class MemoryLeakDetectorPocTest : public MemoryLeakDetectorBase
{
protected:

    // You can do set-up work for each test here.
    MemoryLeakDetectorPocTest();

    // You can do clean-up work that doesn't throw exceptions here.
    virtual ~MemoryLeakDetectorPocTest();

    // Code here will be called immediately after the constructor (right
    // before each test).
    virtual void SetUp();

    // Code here will be called immediately after each test (right
    // before the destructor).
    virtual void TearDown();
};

Hope it is helpful and please let me know if there is anything which is not clear.

Cheers,

Guy.

Answer 3

You might be able to detect memory leak on tests by providing your own implementation of new, delete, malloc and free functions, by adding memory tracking informations on allocation.

Answer 4

You can use the debug functionality right into dev studio to perform leak checking - as long as your unit tests' run using the debug c-runtime.

A simple example would look something like this:

#include <crtdbg.h>
struct CrtCheckMemory
{
  _CrtMemState state1;
  _CrtMemState state2;
  _CrtMemState state3;
  CrtCheckMemory()
  {
    _CrtMemCheckpoint(&state1);
  }
  ~CrtCheckMemory()
  {
    _CrtMemCheckpoint(&state2);
    // using google test you can just do this.
    EXPECT_EQ(0,_CrtMemDifference( &state3, &state1, &state2));
    // else just do this to dump the leaked blocks to stdout.
    if( _CrtMemDifference( &state3, &state1, &state2) )
      _CrtMemDumpStatistics( &state3 );
  }
};

And to use it in a unit test:

UNIT_TEST(blah)
{
  CrtCheckMemory check;

  // TODO: add the unit test here

}

Some unit test frameworks make their own allocations - Google's for example allocates blocks when a unit test fails, so any test block that has a fail for any other reason always also has a false positive "leak".