From:	SMTP%"leichter@lrw.com"  9-JAN-1995 16:52:32.52
To:	EVERHART
CC:	
Subj:	re: Re: Tracking down C memory leak

From: Jerry Leichter <leichter@lrw.com>
X-Newsgroups: comp.os.vms
Subject: re: Re: Tracking down C memory leak
Message-ID: <9501091711.AA14028@uu3.psi.com>
Date: Mon,  9 Jan 95 11:31:08 EDT
Organization: Info-Vax<==>Comp.Os.Vms Gateway
X-Gateway-Source-Info: Mailing List
Lines: 44
To: Info-VAX@Mvb.Saic.Com

Since several people in this thread have commented that they had programs that
"ran just fine" under Unix, failed (at memory without bound) with the VAX C
memory allocator, but worked fine with the VAXC$xALLOC_OPT routines, it's
worth looking at the history here to understand why.

The old Unix malloc() algorithm, as a side-effect of its operation, allowed
you to realloc() a block of memory that you had just free()'d.  Not only did
this work, but it caused the memory allocator to coallesce blocks on its
free list, which could improve performance.  While these facts were never
documented (ha ha), they were used in a number of Unix programs.  To support
those programs, VAX C's malloc(), which was built on top of LIB$GET_VM, added
a small hack to ensure that this programming style would work.  (Basically,
it kept the most recently freed block of memory "in reserve", not handing
it back to LIB$GET_VM until the next free() call.)  This little hack has all
sorts of unfortunate side-effects, as has been pointed out.

The original Unix malloc() algorithm was designed on a 16-bit non-paged
machine.  It had horrible performance on paged machines with much larger
memory spaces, and was long ago replaced by better algorithms.  These
algorithms didn't need the old hack for compacting the free memory list, and
in fact no longer supported the "realloc() of free()'ed memory" trick.  Unix
programs were gradually changed not to rely on the internals of the malloc()
implementation, but rather on its documented interface.  (What a concept.)
Still, VAX C, in the name of backward compatibility, retained its little hack.
The VAXC$xALLOC_OPT routines are, in fact, completely compatible with all
contemporary (last 15 years?) Unixes, and there is no good reason not to
use them; but due to concern for upward compatibility, they remained
"optional", rather then replacing the old defaults.

In general, malloc()/free() are a poor choice for high performance programs
that do a lot of dynamic memory management.  The implementations are different
Unix systems vary from quite good to horrible.  If you really need to worry
about the time and memory your program needs, a special-purpose allocator,
probably using look-aside lists for common sizes, will win just about every
time.  While I rarely actually replace malloc()/free(), I almost never call
them directly; I define one or more private memory allocators (more than one
if I have clearly distinct uses for the memory), and simply have them call
malloc() and free().  Should they ever become bottlenecks, I can quickly
replace them.  (Actually, I generally have these routines check for a NULL
return from malloc() and abort the program.  Very few programs are written
to survive failure to get enough memory, and it's better to get a message
saying that than a message about an access violation later, when the program
tries to use the NULL pointer.)
							-- Jerry
