memory problems: Attn: Danny and Ralph

[Sudipto Banerjee]

Wow guys,

I don't know whether I should feel ashamed or honoured that my problem has
generated such a lively discussion. But I indeed am delighted to be a part
of this. About using assert(), I now see why you guys ask me not to use it
and I will follow your advice. My understanding was that since alloc problems
with new were being treated through a return of a 0 pointer, by the VC++
compiler assert was still working!

Anyway, I have changed them into an exception handling routine. But I don't
think it had anything to to do with this as far as my problem was concerned.
Take a look at the following piece of simple code. Basically I just allocate
and deallocate memory a large number of times - running the code in debug
mode in VC++ 6.0 still throws an exception in heap_alloc_dbg() for l=8, j=31,
i=10169, k=305! I am doing nothing else and am still baffled as to why this
is happening. I request a solution or lead to solve this problem in VC++6.0.
Maybe already hinted at a soln but I am naive enough on these matters to
not quite followed everything. A simpler explanation (if possible) will be
of help.

I also tried malloc-free combo and also assignment of the elements of v in
test_memory but same problem at the same place!

RALPH: If the previous code ran on your machine to completion, can you suggest
why it doesn't on mine:
Tried on Win2k, 512MB and on a WinMe, 256MB! Same problem at same place for
both machines. I have not installed Service Pack! Do you think I should do
it! Thanks.

void test_memory (void);

int main()
{
int i, j, k, l;

for (l = 0; l < 26; l++) {
for (j = 0; j < 99; j++) {
for (i = 0; i < 301; i++) {
for (k = 0; k < 17325; k++) test_memory ();
}
}
}

return 0;
}

void test_memory (void)
{
double *v;
v = new double [10];
delete [] v;
}


Couple of things: I am a statistician who needs to write code (on different
platforms) for numerical simulations that run very very long. I am by no
means a C/C++ expert and am just a user of the resources! So I really do
not have much biases for one OS over the other. But it is just that I need
to understand memory management correctly (IT IS THE MOST CRUCIAL ASPECT
OF MY CODE) so that I can incorporate them into the constructor-destructor
of classes that we always need to do. And while g++ and Borland behave according
to my expectations, VC++ does not! Clearly I need to understand VC++ better
and need everyone's help on this.

The issue to me is that VC++6.0 is creating heap problems that do not seem
to be out of a memory leakage - and I do want to understand what is happening!


Thank you all very very much for listening to my ill-constructed problems.

Sudipto.

[Ralph D. Cole]

"Sudipto Banerjee" <sudiptob@biostat.umn.edu> wrote:
>
>Wow guys,
>
>I don't know whether I should feel ashamed or honoured that my problem has
>generated such a lively discussion. But I indeed am delighted to be a part
>of this. About using assert(), I now see why you guys ask me not to use
it
>and I will follow your advice. My understanding was that since alloc problems
>with new were being treated through a return of a 0 pointer, by the VC++
>compiler assert was still working!
>
>Anyway, I have changed them into an exception handling routine. But I don't
>think it had anything to to do with this as far as my problem was concerned.
>Take a look at the following piece of simple code. Basically I just allocate
>and deallocate memory a large number of times - running the code in debug
>mode in VC++ 6.0 still throws an exception in heap_alloc_dbg() for l=8,
j=31,
>i=10169, k=305! I am doing nothing else and am still baffled as to why this
>is happening. I request a solution or lead to solve this problem in VC++6.0.
>Maybe already hinted at a soln but I am naive enough on these matters to
>not quite followed everything. A simpler explanation (if possible) will
be
>of help.
>
>I also tried malloc-free combo and also assignment of the elements of v
in
>test_memory but same problem at the same place!
>
>RALPH: If the previous code ran on your machine to completion, can you suggest
>why it doesn't on mine:
>Tried on Win2k, 512MB and on a WinMe, 256MB! Same problem at same place
for
>both machines. I have not installed Service Pack! Do you think I should
do
>it! Thanks.
>
...

When I originally posted "it ran to completion", I had screwed up the test.

The first time I ran it, I corrected the syntax and added some cout's and
other monitoring code. It ran to completion and I jumped to the conclusion
that the fault laid with the 'delete v' statement. (I jump to conclusions
a lot.)

Later, I realized I had accidently 'short-circuited' the code with my extra
code, and ran it again. This time I DID error-out at approximately the same
place you did even after making the 'delete [] v' change. I then realized
what the problem was - the Linker Heap default was being exceeeded and posted
my results.

However, I am embarrassed to admit I never changed the Heap option to the
Linker and tried again. (Your program takes hours to run...) So actually
I am "assuming" that once the Linker option is modified the program will
work.

Did you make that change?

>
>void test_memory (void);
>
>int main()
>{
> int i, j, k, l;
>
> for (l = 0; l < 26; l++) {
> for (j = 0; j < 99; j++) {
> for (i = 0; i < 301; i++) {
> for (k = 0; k < 17325; k++) test_memory ();
> }
> }
> }
>
> return 0;
>}
>
>void test_memory (void)
>{
> double *v;
> v = new double [10];
> delete [] v;
>}
>
>
>Couple of things: I am a statistician who needs to write code (on different
>platforms) for numerical simulations that run very very long. I am by no
>means a C/C++ expert and am just a user of the resources! So I really do
>not have much biases for one OS over the other. But it is just that I need
>to understand memory management correctly (IT IS THE MOST CRUCIAL ASPECT
>OF MY CODE) so that I can incorporate them into the constructor-destructor
>of classes that we always need to do. And while g++ and Borland behave according
>to my expectations, VC++ does not! Clearly I need to understand VC++ better
>and need everyone's help on this.
>
>The issue to me is that VC++6.0 is creating heap problems that do not seem
>to be out of a memory leakage - and I do want to understand what is happening!
>
>
>Thank you all very very much for listening to my ill-constructed problems.
>
>Sudipto.

I got interested in your problem because my usual assignments are Win32/Unix
distributed applications. I am continually bombarded by the 'Unix' crowd
with 'M$ bugs' that seldom are. But they often do fall into that category
best expressed with your "don't behave according to my expectations" lamentation.

Which is a real problem and a very valid one, when it comes to cross-platform
programming. (As Danny so eloquently pointed out.)

In this case I don't believe it is a 'compiler' problem but an O/S one. I
am guessing that changing the Linker Heap option will 'fix it'. But you bring
up a good point - Why does the Heap get so large?

How does one know before hand what the Heap setting should be? Maybe it is
actually a Stack problem? (There is Stack default in Win32 as well.) Are
processes effected over time on a Win32 system? (There are cases where I
have found this to be true.)

If I get a chance I will play with it some more. Unfortunately, I am at home
and only have the one machine...

Can't you come up with a problem that only takes a couple of minutes to test?
<g>

[Sudipto Banerjee]

Hi Danny and Ralph:

Thanks again for your helpful response. As I mentioned in another mail, my
understanding of dynamic memory allocation suffered a jolt. What is bewildering
to me (you can check my last post on c.general) is that even if I am allocating
and immediately deallocating memory why does a heap limit even come into
the picture? If it assigns memory correctly in the first iteration why can
it not go back and reassign in the same block as that block of memory has
been freed??????? If it works once, it should work always! That was my understanding
and that the heap size should not matter at all! My understanding and confidence
in dynamic memory allocations has suffered a BIG JOLT AFTER THIS!

But thanks for your help.

Sudipto.


"Ralph D. Cole" <nt_consulting32@SPAMhotmail.com> wrote:
>
>"Sudipto Banerjee" <sudiptob@biostat.umn.edu> wrote:
>>
>>Wow guys,
>>
>>I don't know whether I should feel ashamed or honoured that my problem
has
>>generated such a lively discussion. But I indeed am delighted to be a part
>>of this. About using assert(), I now see why you guys ask me not to use
>it
>>and I will follow your advice. My understanding was that since alloc problems
>>with new were being treated through a return of a 0 pointer, by the VC++
>>compiler assert was still working!
>>
>>Anyway, I have changed them into an exception handling routine. But I don't
>>think it had anything to to do with this as far as my problem was concerned.
>>Take a look at the following piece of simple code. Basically I just allocate
>>and deallocate memory a large number of times - running the code in debug
>>mode in VC++ 6.0 still throws an exception in heap_alloc_dbg() for l=8,
>j=31,
>>i=10169, k=305! I am doing nothing else and am still baffled as to why
this
>>is happening. I request a solution or lead to solve this problem in VC++6.0.
>>Maybe already hinted at a soln but I am naive enough on these matters to
>>not quite followed everything. A simpler explanation (if possible) will
>be
>>of help.
>>
>>I also tried malloc-free combo and also assignment of the elements of v
>in
>>test_memory but same problem at the same place!
>>
>>RALPH: If the previous code ran on your machine to completion, can you
suggest
>>why it doesn't on mine:
>>Tried on Win2k, 512MB and on a WinMe, 256MB! Same problem at same place
>for
>>both machines. I have not installed Service Pack! Do you think I should
>do
>>it! Thanks.
>>
>...
>
>When I originally posted "it ran to completion", I had screwed up the test.
>
>The first time I ran it, I corrected the syntax and added some cout's and
>other monitoring code. It ran to completion and I jumped to the conclusion
>that the fault laid with the 'delete v' statement. (I jump to conclusions
>a lot.)
>
>Later, I realized I had accidently 'short-circuited' the code with my extra
>code, and ran it again. This time I DID error-out at approximately the same
>place you did even after making the 'delete [] v' change. I then realized
>what the problem was - the Linker Heap default was being exceeeded and posted
>my results.
>
>However, I am embarrassed to admit I never changed the Heap option to the
>Linker and tried again. (Your program takes hours to run...) So actually
>I am "assuming" that once the Linker option is modified the program will
>work.
>
>Did you make that change?
>
>>
>>void test_memory (void);
>>
>>int main()
>>{
>> int i, j, k, l;
>>
>> for (l = 0; l < 26; l++) {
>> for (j = 0; j < 99; j++) {
>> for (i = 0; i < 301; i++) {
>> for (k = 0; k < 17325; k++) test_memory ();
>> }
>> }
>> }
>>
>> return 0;
>>}
>>
>>void test_memory (void)
>>{
>> double *v;
>> v = new double [10];
>> delete [] v;
>>}
>>
>>
>>Couple of things: I am a statistician who needs to write code (on different
>>platforms) for numerical simulations that run very very long. I am by no
>>means a C/C++ expert and am just a user of the resources! So I really do
>>not have much biases for one OS over the other. But it is just that I need
>>to understand memory management correctly (IT IS THE MOST CRUCIAL ASPECT
>>OF MY CODE) so that I can incorporate them into the constructor-destructor
>>of classes that we always need to do. And while g++ and Borland behave
according
>>to my expectations, VC++ does not! Clearly I need to understand VC++ better
>>and need everyone's help on this.
>>
>>The issue to me is that VC++6.0 is creating heap problems that do not seem
>>to be out of a memory leakage - and I do want to understand what is happening!
>>
>>
>>Thank you all very very much for listening to my ill-constructed problems.
>>
>>Sudipto.
>
>I got interested in your problem because my usual assignments are Win32/Unix
>distributed applications. I am continually bombarded by the 'Unix' crowd
>with 'M$ bugs' that seldom are. But they often do fall into that category
>best expressed with your "don't behave according to my expectations" lamentation.
>
>Which is a real problem and a very valid one, when it comes to cross-platform
>programming. (As Danny so eloquently pointed out.)
>
>In this case I don't believe it is a 'compiler' problem but an O/S one.
I
>am guessing that changing the Linker Heap option will 'fix it'. But you
bring
>up a good point - Why does the Heap get so large?
>
>How does one know before hand what the Heap setting should be? Maybe it
is
>actually a Stack problem? (There is Stack default in Win32 as well.) Are
>processes effected over time on a Win32 system? (There are cases where I
>have found this to be true.)
>
>If I get a chance I will play with it some more. Unfortunately, I am at
home
>and only have the one machine...
>
>Can't you come up with a problem that only takes a couple of minutes to
test?
><g>
>
>
>

[Ralph D. Cole]

"Sudipto Banerjee" <sudiptob@biostat.umn.edu> wrote:
>
>Hi Danny and Ralph:
>
>Thanks again for your helpful response. As I mentioned in another mail,
my
>understanding of dynamic memory allocation suffered a jolt. What is bewildering
>to me (you can check my last post on c.general) is that even if I am allocating
>and immediately deallocating memory why does a heap limit even come into
>the picture? If it assigns memory correctly in the first iteration why can
>it not go back and reassign in the same block as that block of memory has
>been freed??????? If it works once, it should work always! That was my understanding
>and that the heap size should not matter at all! My understanding and confidence
>in dynamic memory allocations has suffered a BIG JOLT AFTER THIS!
>
>But thanks for your help.
>
>Sudipto.
>
> ...

It is a "feature" of the M$ CRT Debug Heap routines. Apparently in order
to provide support for memory leak detection, monitoring, and logging(dump)
- freed blocks are kept unused in the debug heap's linked list. This behavior
can be changed with the _crtDbgFlag flag.

I am just guessing, but Borland probably doesn't supply the same Debug Heap
library and therefore doesn't exhibit the same behavior.

'Release' versions of your program work as expected.

[Ralph D. Cole]

I forgot to include the following URL:
<http://msdn.microsoft.com/library/de...debug_heap.asp>

Also I would like to thank you for posting this question. It has been a humbling
adventure for me. One of the problems with spending most of your time with
"higher-level" frameworks/packages and 3rd party Debugging tools (SoftIce),
is that after a while you tend to forget the basics and how to use the native
support built into the compiler.

It is also useful for me to be occasionally reminded to not "jump to conclusions"
as I seldom let "not knowing all the facts" deter me from having an opinion.