Allocating memory to minimise hard disc access

[Cheryl]

My application allocates a series of blocks of memory at runtime and I am
having problems when the process runs for a long time and I start to run
out of RAM and start accessing the hard disk. I do not know beforehand how
much memory I will need but could place an upper limit on this. The blocks
of memory are arrays of 128k of 2 byte values within a structure which forms
part of a linked list. This is 'old' code and uses malloc to allocate each
new structure as it is required. As I am acquiring each of the 2 byte values
at a constant 1M samples per second, when I start to need to use the hard
disk I am losing samples.

It has been suggested to me that I preallocate all the memory required -
i.e. the entire linked list - and write to every e.g. 4k sample to 'reserve
the memory'. The person that suggested this knows their stuff but I don't
fully understand how this works and whether I have whether I've remembered
the entire method they described.

I also wonder whether there are now better ways to handle this - using a
vector and reserving a number of elements maybe? I can certainly take this
opportunity to switch to using 'new' in place of 'malloc' unless the latter
has any advantages in this particular instance. Unfortunately, I do not have
much time to get this fixed.

(Operating System NT4; IDE Borland C++ Builder 5)

[Danny Kalev]

You have two options: either pre-allocate all the memory in advance,
read this article to see how this can be done:
http://gethelp.devx.com/techtips/cpp_pro/10min/10min0999.asp

Note however that since most OSs use virtual memory, there's no
guarantee that the memory you preallocate is actually mapped to the RAM.
It could be swapped in and out from the hard disk. Depending on your OS,
you could use API functions that lock the preallocated memory in the
RAM.

Danny

Cheryl wrote:
>
> My application allocates a series of blocks of memory at runtime and I am
> having problems when the process runs for a long time and I start to run
> out of RAM and start accessing the hard disk. I do not know beforehand how
> much memory I will need but could place an upper limit on this. The blocks
> of memory are arrays of 128k of 2 byte values within a structure which forms
> part of a linked list. This is 'old' code and uses malloc to allocate each
> new structure as it is required. As I am acquiring each of the 2 byte values
> at a constant 1M samples per second, when I start to need to use the hard
> disk I am losing samples.
>
> It has been suggested to me that I preallocate all the memory required -
> i.e. the entire linked list - and write to every e.g. 4k sample to 'reserve
> the memory'. The person that suggested this knows their stuff but I don't
> fully understand how this works and whether I have whether I've remembered
> the entire method they described.
>
> I also wonder whether there are now better ways to handle this - using a
> vector and reserving a number of elements maybe? I can certainly take this
> opportunity to switch to using 'new' in place of 'malloc' unless the latter
> has any advantages in this particular instance. Unfortunately, I do not have
> much time to get this fixed.
>
> (Operating System NT4; IDE Borland C++ Builder 5)
>
>