FAQ 12.08 How can new be set up to automatically flush pools of recycled objects whenever memory runs low?

FAQ 12.08 How can new be set up to automatically flush pools of recycled objects whenever memory runs low?

Applications that do a lot of freestore allocations can sometimes improve performance by using global pools of recycled objects. For example, when a dynamically allocated Fred object is no longer needed, the programmer can say p->discard() rather than delete p, and the discard() member function adds the object to a static pool of Fred objects. Then when a Fred object is needed, the programmer says Fred::create() rather than new Fred(), and the static create() member function returns the first entry from the list (or returns new Fred() if the list is empty).

Everything works great until memory runs low, at which point the pool of Fred objects needs to be flushed to free up available memory. It would be ideal if the runtime system would automatically call some routine such as Fred::flushPool() whenever new ran low on memory, since the pool could be flushed without any functional impact. For example, if someone wants to create a Wilma object and the system runs out of memory because there are too many recycled Fred objects in the Fred pool, the goal is to have the system automatically call Fred::flushPool(), which actually deletes all the Fred objects on the recycled list. We set up the Fred class with its pool of recycled objects:

 #include <new> using namespace std; class Fred { public:   static Fred* create() throw(bad_alloc);            <-- 1   virtual void discard() throw();                    <-- 2   static bool flushPool() throw();                   <-- 3 private:   Fred()  throw();                                   <-- 4   ~Fred() throw();                                   <-- 5   void init() throw();                               <-- 6   Fred* nextRecycled_;   static Fred* headRecycled_; }; void Fred::init() throw() {   // ...                                             <-- 7   nextRecycled_ = NULL; } Fred::Fred() throw() { init(); } Fred::~Fred() throw() { }                                                  <-- 8 Fred* Fred::headRecycled_ = NULL; Fred* Fred::create() throw(bad_alloc) {   if (headRecycled_ == NULL)     return new Fred();   Fred* p = headRecycled_;   headRecycled_ = headRecycled_->nextRecycled_;   p->init();                                         <-- 9   return p; } void Fred::discard() throw() {   nextRecycled_ = headRecycled_;   headRecycled_ = this; } bool Fred::flushPool() throw() {   bool stuffGotDeleted = (headRecycled_ != NULL);   while (headRecycled_ != NULL)     delete create();   return stuffGotDeleted; } 

(1) Named Constructor Idiom; see FAQ 16.08

(2) p->discard() is analogous to delete p

(3) Returns true if it freed some memory

(4) Constructor is private: so users can't say new Fred()

(5) Destructor is private: so users can't say delete p

(6) Initializes (or possibly reinitializes) the object as if it was just created

(7) This is where the constructor's logic should go

(8) The destructor doesn't need to do anything with the recycled list

(9) Reinitializes the object as if it were just created

First, notice how users are prevented from saying new Fred() or delete p. Instead users must say Fred::create() and p->discard(), respectively. The discard() member function adds the object to the recycled pool, and the create() static member function uses the recycled pool if it isn't empty. Finally the flushPool() static member function flushes the pool of recycled Fred objects, and returns a bool indicating whether anything actually was deleted.

Next, to acomplish the larger goal of having the system automatically call Fred::flushPool() whenever new runs out of memory, a special function is created that calls Fred::flushPool() (and possibly other similar pools, such as Wilma::flushPool()). This special function is known as a new handler and is called flushAllPools() in the following example. If operator new(size_t nbytes) runs out of memory, it calls this function, which tries to delete some unneeded memory. If the new handler succeeds at freeing up some storage, it simply returns to operator new(size_t nbytes), and operator new(size_t nbytes) tries the allocation again. If the new handler is unsuccessful at freeing up storage, it avoids an infinite loop by throwing an exception:

 #include <new> using namespace std; void flushAllPools() throw(bad_alloc) {   unsigned n = 0;   n += Fred::flushPool();   // Flush any other pools as well,   // e.g., n += Wilma::flushPool();   if (n == 0) throw bad_alloc();  // Nobody freed memory;                                   // prevent infinite loop } 

The final step is to register the function flushAllPools() as the official new handler. This is done using the set_new_handler() function and is normally called very early in the application's execution:

 int main() {   set_new_handler(flushAllPools);                    <-- 1   // ... } 

(1) Install the "new handler"

The rest is automatic: if someone says new Barney() and the underlying allocator (operator new(size_t nbytes)) runs out of memory, the allocator automatically calls the new handler (flushAllPools()), which flushes the Fred pool (Fred::flushPool()). If something actually was flushed, the new handler returns to operator new(size_t), which tries again. If it fails a second time, the whole process repeats. Eventually one of two things happens: either operator new(size_t) succeeds, in which case the caller who said new Barney() will never know that any of this ever happened, or flushAllPools() fails to flush anything and throws an exception (in which case the new Barney() attempt vanishes, and control goes to the appropriate catch handler; see FAQ 9.03). In either case, the users who are saying new Barney() don't know anything about the pool mechanism it is invisible to them.