Hideki A. Ikeda (HidekiAI) [池田英紀] ["Tony" Ikeda] – BLog

When overriding operator new() and new[](), there are few ways to deal with it.  It should be noted that you must really ask yourself why in the hell you’d want to do this, and have a strong argument behind it. In most cases, you won’t be able to justify why.   If you really need it and your company/business needs it, consider commercial heap management, these guys “just do memory management”, your Return-On-Investment will be justified, it will save your company money in the long run because you are not tied up for months on it.  Also consider any OpenSource and academia libraries.  Do what ever you need to do to convince yourself that you don’t want to write your own memory manager!

But there are times when you want to have a fast-allocator because you realize your game-application has the patterns to which small blocks are consistently been allocated, mostly of similar size, and you justify that all combined together, they nicely fit in a Memory Page (i.e. Page = 64K), and you notice the pattern that they get deleted often.  I’m not going to discuss fast-allocators, for there are many ways to do it as well as design patterns shouldn’t be generic in my opinion (hint: search for “segmented storage” by Stroustrup on the Internet), and they should be specialized based on each applications and/or game-engine.

Operator new and delete can be overridden in class-specific and global.  There is also “placement new” which can be useful for types of memory which you do not need to call neither the destructor or freeing of the memory (such as loading the new scene, and you want to wipe out the entire buffer at once).  I think it has potential, but needing to manually (explicitly) call the destructor is a bit unconventional so care must be taken.

Class Specific operator new() and delete()

Class-specific overloads and overrides of operator new and delete works very well on both MSVC and GCC and in my opinion, least amount of hassles.  Most developers would create a base class with the operator overloads for new() and delete() (and possibly new[]() and delete[]() as well) and have all classes which they want to have it use their own memory allocation strategies to inherit this base class.  Here’s a simple example of class specific:

class CBaseMemoryObject
{
    public:
        static void *     operator new(size_t sizeInBytes);
        static void *     operator new[](size_t sizeInBytes);
        static void       operator delete(void *pAddress, size_t sizeInBytes);
        static void       operator delete[](void *pAddress, size_t sizeInBytes);
        // placement new() and delete()
        static void *     operator new(size_t sizeInBytes, void* p)        {return p;}
        static void *     operator new[](size_t sizeInBytes, void* p)      {return p;}
        static void       operator delete(void *, void*)                   {}
        static void       operator delete[](void *, void*)                 {}
};
 
class CMyObject
  : public CBaseMemoryObject
{
  ...
};
 
int main(int, char**)
{
  CMyObject * myObject = new CMyObject();  // this will call CBaseMemoryObject's new()
  ...
  delete myObject;  // this will call CBaseMemoryObject's delete()
}

What is so nice about class-specific is that if you know the patterns of how big, how often, whether it is static or dynamic, etc, you can create allocators specific to different memory patterns and have each classes inherit different models.

Global operator new() and delete() – Overrides and Overloads

What is a nightmare of course is management of different allocator base-classes or being not generic.  You therefore would have the tendencies to just write one base class, let the new() determine the size_t passed and figure out which allocator it should use.  Unless you code wisely, you’ll start to see that bottleneck in that logic of choosing which allocator to use, especially in a multi-threaded system.

I don’t think there are right ways or wrong ways, I still believe that it is application specific.

There comes a time when you’d want to override the global new() and delete(), mainly for DEBUG purpose – commonly, programmers do this so that they can track memory leaks by filenames and linenumber like so:

// Following works for both GCC and MSVC (they both support NDEBUG, __FILE__, and __LINE__)
#if defined(NDEBUG)
  // do nothing, just use the compiler's version
#else
#  define    new       new(__FILE__, __LINE__)
#  define    delete    DeleteTracker(__FILE__, __LINE__), delete
#endif

Because it is a #define macro, the above will take care of both new() and new[]() as well as delete() and delete[]().  Note that I use the “,” to separate calls to two functions DeleteTracker() and delete(), which is very careless technique to use for #define macros. For example:

#define   myMacro(x)    myFunc1(), myFunc2(x)
...
Foo(myMacro(123), Bar());  // function Foo() expects 2 args, but myMacro() expands to 3 parameters!

So that’s my disclaimer on why you shouldn’t use “,” tricks on #define macros (in fact, you shouldn’t be using #define macros at all if you can help it!)…

But it is also useful for cases where people do not use the braces. For example, if the “,” was replaced with “;” then if you had a statement such as “if (myPointer) delete myPointer;” would be not possible. By place the “,” instead, the statement will still work.

Also, note that you still need a corresponding “delete(void *, const char *, int)” to accommodate the matching “new(size_t, const char *, int)“. More on that later in the discussion of overriding and overloading delete() as a operator and as a function…

So going back to the global operator overrides and overloads for operators new/delete, the conveniences are there in a sense that you do not have to alter any existing codes that is using new/delete, but you’d have to rebuild your entire project.

One last thing to note…  Sooner or later, there will be yet another temptations and tendencies in which if you are overriding (and overloading) operators new/delete, that you’d want just one function which both the class-based and global can share/call.  The temptations is to do something like this:

class CMyMemoryManager
{
    public:
        static void * operator new(size_t size);
        ...
};
 
// Use CMyMemoryManager::new() for global new
// This WILL NOT work on some cases!
#define    new    CMyMemoryManager::new

The temptations to do this is overwhelming, because you just need to manage one code for both class-specific and global.  But the compiler error (GCC) you’d get will be this:

...filename...:42: expected unqualified-id before 'new'

Admit it! You’ve found this site because you were getting the “unqualified-id before ‘new’” and you stumbled here! *grin*

The correct solution is as follows:

class CMyMemoryManager
{
    public:
        // See #define MyNew of why we still need this
        static void * operator new(size_t size)
        {
            return(Allocate(size));
        }
 
        // Global operator (not #define MyNew) calls these
        static void * Allocate(size_t size);
        static void * AllocateArray(size_t size);
        static void * AllocateAligned(size_t size, size_t byteAlignment);
        static void   Deallocate(void * address);
        static void   DeallocateArray(void * address);
        static void   DeallocateAligned(void * address);  // MSVCRT has _aligned_malloc(), which requires _aligned_free()
};
// Globally override operator new() rather than using #define new
void * ::operator new(size_t size)
{
  // we call function CMyMemoryManager::Allocate(), not CMyMemoryManager::new (which is an operator, not a function)
  return(CMyMemoryManager::Allocate(size));
}
// and so on...

You just need to realize that new() is an operator, thus you cannot treat it like an ordinary function.  You need to therefore work around this by creating a function that is not a reserved name such as Allocate().

Ideally if you are overriding and overloading operator new() and new[]() in global level, you don’t need to do this in class level unless you are going to do #define macro so that the parametrization of the operator matches.

Caution: see the next section of why you really don’t want to override operator new() globally, but rather use #define.

Global Operator new() and STL (or other 3rd party libs)

The above example would work nicely on a super small and/or isolated project, but once you start using 3rd party libraries including STL, you’d notice that your #define new fails to compile or you do manage to compile but when you step through the STL you notice that it’s using the RTL (Runtime Library) operator new() instead.

Your gut feeling should be to let-it-be!  But there comes a time when your memory manager was integrated late in the project and you really need everything to be using your memory management system.

The preferred method should be to #define with something reasonable, then search-and-replace all your code to use that instead like so:

#ifdef NDEBUG
#    define    MyNew       new
#    define    MyDelete    delete
#else
      // NOTE: This will not work in some cases, see sections back on "unqualified-id" issue
#    define    MyNew       CMyMemoryManager::operator new
#    define    MyDelete    CMyMemoryManager::operator delete
#endif

This way, only your source codes uses your memory manager wherever “MyNew” is specified.  You also want to make sure that

In any case, be forewarned that overriding global operator new() rather than to use the #define, is not going to be a free-ride.  I’ve not yet experimented this yet on GCC side, but I’ve had the opportunity to do this on MSVCRT side (at the time of this writing, on MSVC8), and there were some linker challenges to overcome because of MSVCRT’s new() and delete() must be overridden using .def file.  But at the end of the day, whether it is STL or any 3rd party libraries that are compiled (yes, in source level and has to link against your memory manager) will transparently start using your global operator new().

Please note: I cannot go more into details right now about how I’ve gone around the linker issue, for how I discovered it (trivial as it may be) was during my work hours so any I.P. belongs to work.  Hint: My global operators are a separate .DLL.

Last but not least, although STL template containers uses placement new(), you do not have to override it (at least) in MSVCRT.

Operator delete()

Operator delete is a very interesting discussion for me in many aspects.  To some people that are encyclopedia of C++, it may be obvious but I don’t override nor overload new|delete that often so I have to constantly look it up.

In any case, one interesting thing about operator delete is that if you want to explicitly pass parameters, you have to treat it as a function rather than as an operator.  For example:

delete(myPointer, __FILE__, __LINE__);  // this will fail to compile
delete(__FILE__, __LINE__) myPointer; // fails to compile
operator delete(myPointer, __FILE__, __LINE__); // OK: treat it as a function and you can call it.

So you would think,  hey why not “#define delete(x)   operator delete(x, __FILE__, __LINE__)” so that it’ll call my delete!  Bad idea!  Think about “delete[]()” as well as the fact that you will have to go search-and-replace all the “delete p;” with “delete(p);“.  That would be so inconvinient!

Oh, also it seems that the only time your overloaded delete() will get called as an operator is if you class you are new()’ing throws an exception in its constructor.  If the class does not throw (in its constructor), then it will call either the default global operator delete() or the overridden delete.

Note that I use the term overridden delete versus overloaded delete as:

void operator delete(void * p); // overridden
void operator delete[](void *p); // overridden
void operator delete(void *p, const char * file, int line); // overloaded
void operator delete[](void *p, const char * file, int line); // overloaded

If you class throws in its constructor, then it will call the overloaded delete() when you delete the object.  The inconvenience to this is that:

• You’d have to have all your classes now have its own constructors and make it throw.
• Everywhere you call new(), you will now need to do try/catch
• Instancing none-class primitives such as int, char, etc won’t throw so they will still call the overridden delete rather than overloaded delete.

All in all, in most cases, this is OK.  To be honest, I really don’t need to care too much about overloaded delete, as long as it will call my overridden delete().  It would be nice to track who called delete (with file, line) but it’s not useful except to track double-free issues.

Another interesting problems with global operator delete() (and delete[]()) is this:

// File: MyClass.cpp
#include "MyMemoryManager.h"
 
// define a global variable that will internally do new()
const std::string  gGlobalString = "This is a global string";
 
MyClass::MyClass()
{
  ...
}
MyClass::~MyClass()
{
  ...
}
...

As you know, internally STL containers will call operator new() and delete() to dynamically allocate heap. There will come a time when you will encounter the issue that when that static variable was initialized, it had called the standard (your compiler’s version of) new(). But when the application was shutdown, it has called your MemoryManager’s delete() instead.

An easy fix would be that you first test the heap block header requested to be freed in your MemoryManager (see section below on block headers). I usually leave an extra 4-bytes watermarks as part of my header (i.e. “MMGR” as in MemoryManager, or whatever you want). If I don’t find this signature in the block header, then I would assume it must have been allocated before my MemoryManager was initialized, and just use the standard free() or ignore it (and count it as a leak that can potentially be a memory fragmentation – just hope that these chunks are small and not a large chunk!).

I normally would not recommend to call free() on these blocks though. You should leave it alone and count it as (hopefully) small fragmentation that would not get released during the lifetime of your application. Why would I leave it alone? What if it was a corrupted header due to buffer-overrun, and the reason why you couldn’t find the block header signature was because of corruptions?

In the past, at least for Microsoft’s MSVCRT based MemoryManager, I’ve even done cases where if in _DEBUG (or !NDEBUG) mode, I would call _CrtIsValidHeapPointer() first and test if that block was owned by MSVCRTD, and if so, I’d go ahead and free() it, else I’d count it as memory corruptions. Keep in mind that _CrtIsValidHeapPointer() is only valid in debug mode, so do protect it with #if defined(_DEBUG) to explicitly remind yourself that it will call an NULL function in release if you didn’t.

But that only works on debug mode, and you generally want to write code that has same characteristics of whether in debug or release, so when you are testing, you are testing apples-to-apples.

Common Bugs You May Miss

Here’s few trivial bugs you may need to consider that you may not realize…

delete() versus delete[]()

Do you realize what happens in the following code?

  // assume MyClass has destructor
  MyClass * p = new MyClass[3];
  ...
  delete p;  // should be delete[] p;

In the above case, destructor for ~MyClass() only gets called once for p[0], not three times (p[1] and p[2] ~MyClass() won’t get called). From your MemoryManager’s point of view, that’s irrelevant because you’re only managing the heap. But what if the destructor ~MyClass() was doing some clean up of other heaps? You’d miss the leaks there because it has only cleaned up once, not thrice.

double-delete

Consider the following code:

  MyClass * p1 = new MyClass;
  ...
  delete p1;  // a good practice here is to set p1=NULL
  MyClass * p2 = new MyClass;  // [1] See my description
  ...
  p1->Foo();  // valid call
  delete p1;  // [2] p1 has already been deleted!
  ...
  p2->Foo();  // [3] Exception!
  delete p2;  // [4] although p2 is valid pointer, p2 heap is already been released!

Suppose at [1], iimmediately after p1 was deleted, p2 (same sizeof()) gets allocated, and just by luck your MemoryManager returns just released block that was owned by p1 back to p2.

At [2], because the programmer did not force p1=NULL or the scope of p1 is still valid, or whatever the reasons are, you tell the MemoryManager to free p1.  Keep in mind that from heap management’s point of view, it cares less about the variable name, it only cares about the address/pointer.

At [3], you try to call method function MyClass::Foo(), but because heap of p2 is already freed, your virtual table is most likely hosed if your MemoryManager fills with no-man’s fill value (i.e. Microsoft fills it with 0xDD followed by 0xFEEEFEEE as in “free free” in debug mode – see “dbgdel.cpp” in your MSVC source codes).

What is so interesting about this dangling pointer problem to me is that it will only manifest at times when you start implementing no-man’s fill or perhaps only happens in debug builds and not in release.

For example, suppose your MemoryManager only filled the freed block with no-man’s fill in debug mode.  You get a bug written up that your application only crashes in debug but not (or rarely) in release mode.  It just happens that by luck, nothing other than p2 has requested for heap.  So at step [2], even after p1 has been freed, the data is (or “may be”) still valid.

Because of this, at step [3], calling method MyClass::Foo() may succeed…

A good practice therefore is to force p1=NULL at step [2], because it is perfectly legal to call delete() with NULL pointer (your MemoryManager should test for NULL and just bail/opt out immediately).

So you’d think, how can I trap this kind of issue? You cannot trap the culprit who has deleted your pointer (step [2]).  But you can trap the consecutive call (step [4]) if you are lucky.  I mean “lucky” because in the above example, you’d get possible/potential exceptions at step [3], so your MemoryManager won’t be processed (won’t get to step [4]).

But suppose you do get to step [4].  In which case, if your heap management is similar to most (including Microsoft’s), which uses block header (see few sections down below), in your debug mode builds, you can first test for validity of the block header.  If the block header and fences are corrupted or invalid, then you have caught the issue of invalid memory deallocation.

malloc() and free() – Aligned and Unaligned Heap

Todo: talk about need to track these allocations as well.

memset(), memcpy(), and other Utilities

Todo: talk about providing utility functions to help make sure users are informed of any buffer overruns (MSVC has nice functions such as sprintf_s() that would catch these issues).

Memory Block Headers and Fences

Todo: talk about why block headers are faster in performance compared to having a list that tracks memory blocks;  talk about why fences are needed so that detections of block headers corruptions by buffer overruns can be detected early (leave the memory “as is” and do not free corrupted blocks, just report it).

Related posts

• Operator new() and delete() [moved to articles Page] (0)
• XNA with VC8 (C++) Express Edition 2005 (0)
• Why MSVC programmers are spoiled (0)
• Visual Studio 2010 and C++’0x (0)
• Use your O/S to Multi-Process (0)