Allocating memory in VB (Custom strings)

[Mark Alexander Bertenshaw]

Hi -

I have been working on a couple of classes which are custom strings. Thereasons
why I want to do this are:

* The strings use ASCII internally, so no conversion is required when dealing
with pure ASCII data (such as on the web).
* I can write string processing routines which manipulate string data with
the minimum of allocation/deallocation.
* I can have a "string pointers" class based on my custom string class, thus
meaning that I don't always have to copy string data.

Currently I am using Byte arrays to allocate memory for my arrays. I am
also leveraging the Shlwapi.dll string functions so as to provide replacements
for InStr. But I have just come to my Replace method, and have just wondered
whether the ReDim Preserve calls that I must use to increase the string size
are the most efficient way to allocate new memory. Having read the chapter
on strings in Matt Curland's "Advanced Visual Basic 6", I am wondering whether
it would be better to use BSTRs instead, purely to allocate memory (he says
that 64k of BSTRs are cached). Otherwise, should I use system functions
to allocate memory directly?

--
Mark Alexander Bertenshaw
Programmer/Analyst
Chordiant Software, Inc.
Brentford
UK

[Joe \Nuke Me Xemu\ Foster]

"Mark Alexander Bertenshaw" <mark.bertenshaw@virgin.net> wrote in message <news:3c7e76d5$1@10.1.10.29>...

> for InStr. But I have just come to my Replace method, and have just wondered
> whether the ReDim Preserve calls that I must use to increase the string size
> are the most efficient way to allocate new memory. Having read the chapter

You could first locate all occurrences of the search string that
should be replaced, then you only have to resize the string once.
You'll need to resize the array of search string occurrences, but
for some reason that appears to be less expensive than resizing
strings. Of course, YMMV. These might help:

http://groups.google.com/groups?selm....supernews.com

http://groups.google.com/groups?selm....supernews.com

There's a Replace clone buried in here somewhere:

http://groups.google.com/groups?selm...%40tkmsftngp02

--
Joe Foster <mailto:jlfoster%40znet.com> Wanna buy a Bridge? <http://xenu.net/>
WARNING: I cannot be held responsible for the above They're coming to
because my cats have apparently learned to type. take me away, ha ha!

[Mark Alexander Bertenshaw]

Hi Joe -

"Joe \"Nuke Me Xemu\" Foster" <joe@bftsi0.UUCP> wrote:
>"Mark Alexander Bertenshaw" <mark.bertenshaw@virgin.net> wrote in message
<news:3c7e76d5$1@10.1.10.29>...
>
>> for InStr. But I have just come to my Replace method, and have just wondered
>> whether the ReDim Preserve calls that I must use to increase the string
size
>> are the most efficient way to allocate new memory. Having read the chapter
>
>You could first locate all occurrences of the search string that
>should be replaced, then you only have to resize the string once.
>You'll need to resize the array of search string occurrences, but
>for some reason that appears to be less expensive than resizing
>strings.

Yup - I understand that bit. In fact, you don't even need to create an array
of string occurrences. If the string is going to be shrunk then you can
go forwards through the string, detecting the string to be replaced, moving
text _backwards_ with an ever increasing offset. If the string is going
to be expanded then you must search from the end of the string backwards,
moving text _forward_. No, what I was wondering about was what type of memory
would be best to use: my current Byte array, BSTR, or memory allocated directly
from the system. I have been reading up on BSTRs in the Curland book and
on MSDN, and I am tending towards them (they're cached, they individually
have a slightly smaller overhead, it's slightly easier to convert between
Unicode and Ascii).

Inside my existing class, I am allocating blocks of bytes at a time (the
block size is changeable, but is by default 256 bytes). Thus, when appending
strings, my algorithm simply sees whether the resulting string is too big
for the buffer, expands if necessary, and the appending string is copied
into the buffer. It would be nice to know whether allocating a new string
and copying into it is faster than doing a ReDim Preserve, for instance.

--
Mark Alexander Bertenshaw
Programmer/Analyst
Chordiant Software, Inc.
Brentford
UK

[Joe \Nuke Me Xemu\ Foster]

"Mark Alexander Bertenshaw" <mark.bertenshaw@virgin.net> wrote in message <news:3c7f64e8$1@10.1.10.29>...

> "Joe \"Nuke Me Xemu\" Foster" <joe@bftsi0.UUCP> wrote:

> >You could first locate all occurrences of the search string that
> >should be replaced, then you only have to resize the string once.
> >You'll need to resize the array of search string occurrences, but
> >for some reason that appears to be less expensive than resizing
> >strings.
>
> Yup - I understand that bit. In fact, you don't even need to create an array
> of string occurrences. If the string is going to be shrunk then you can
> go forwards through the string, detecting the string to be replaced, moving
> text _backwards_ with an ever increasing offset.

I use this quite a bit myself.

> If the string is going
> to be expanded then you must search from the end of the string backwards,
> moving text _forward_.

I'm not sure how this would work. Unless you already know how many
occurrences of the search string you have, how do you know the offset?
Building and using the array might be faster than having to do two InStr
loops, especially if you can't use vbBinaryCompare.

> No, what I was wondering about was what type of memory
> would be best to use: my current Byte array, BSTR, or memory allocated directly
> from the system. I have been reading up on BSTRs in the Curland book and
> on MSDN, and I am tending towards them (they're cached, they individually
> have a slightly smaller overhead, it's slightly easier to convert between
> Unicode and Ascii).

Plain old strings might be more attractive if you can determine that
it's safe to bypass Mid$'s error-checking:

URL:http://groups.google.com/groups?selm...%40tkmsftngp05

> Inside my existing class, I am allocating blocks of bytes at a time (the
> block size is changeable, but is by default 256 bytes). Thus, when appending
> strings, my algorithm simply sees whether the resulting string is too big
> for the buffer, expands if necessary, and the appending string is copied
> into the buffer. It would be nice to know whether allocating a new string
> and copying into it is faster than doing a ReDim Preserve, for instance.

The Replace clone doubles the string, plus some, though similar
techniques are beneficial for ReDim Preserve as well. Reallocating
memory buffers of almost any sort tends to be very slow! (Could
Space$ be the culprit here?)

--
Joe Foster <mailto:jlfoster%40znet.com> On the cans? <http://www.xenu.net/>
WARNING: I cannot be held responsible for the above They're coming to
because my cats have apparently learned to type. take me away, ha ha!

[Mark Alexander Bertenshaw]

Joe -

Thanks for the reply.

"Joe \"Nuke Me Xemu\" Foster" <joe@bftsi0.UUCP> wrote:
>"Mark Alexander Bertenshaw" <mark.bertenshaw@virgin.net> wrote in message
<news:3c7f64e8$1@10.1.10.29>...
>
>> "Joe \"Nuke Me Xemu\" Foster" <joe@bftsi0.UUCP> wrote:
>
>> >You could first locate all occurrences of the search string that
>> >should be replaced, then you only have to resize the string once.
>> >You'll need to resize the array of search string occurrences, but
>> >for some reason that appears to be less expensive than resizing
>> >strings.
>>
>> Yup - I understand that bit. In fact, you don't even need to create an
array
>> of string occurrences. If the string is going to be shrunk then you can
>> go forwards through the string, detecting the string to be replaced, moving
>> text _backwards_ with an ever increasing offset.
>
>I use this quite a bit myself.
>
>> If the string is going
>> to be expanded then you must search from the end of the string backwards,
>> moving text _forward_.
>
>I'm not sure how this would work. Unless you already know how many
>occurrences of the search string you have, how do you know the offset?
>Building and using the array might be faster than having to do two InStr
>loops, especially if you can't use vbBinaryCompare.

Yes - you're absolutely right. I was actually doing multiple searches without
saving the positions, just counting the number of instances of the replaced
expression. (It might be worth testing the general algorithm with and without
the array).

>> No, what I was wondering about was what type of memory
>> would be best to use: my current Byte array, BSTR, or memory allocated
directly
>> from the system. I have been reading up on BSTRs in the Curland book
and
>> on MSDN, and I am tending towards them (they're cached, they individually
>> have a slightly smaller overhead, it's slightly easier to convert between
>> Unicode and Ascii).
>
>Plain old strings might be more attractive if you can determine that
>it's safe to bypass Mid$'s error-checking:
>
> URL:http://groups.google.com/groups?selm...%40tkmsftngp05

Heh heh! Interesting ... And I liked the comment about Space$(). Reading
the BSTR documentation, it seems that it is quite possible to allocate the
string without having to initialise it (what a waste of time!).

>> Inside my existing class, I am allocating blocks of bytes at a time (the
>> block size is changeable, but is by default 256 bytes). Thus, when appending
>> strings, my algorithm simply sees whether the resulting string is too
big
>> for the buffer, expands if necessary, and the appending string is copied
>> into the buffer. It would be nice to know whether allocating a new string
>> and copying into it is faster than doing a ReDim Preserve, for instance.
>
>The Replace clone doubles the string, plus some, though similar
>techniques are beneficial for ReDim Preserve as well. Reallocating
>memory buffers of almost any sort tends to be very slow! (Could
>Space$ be the culprit here?)

Re. Space$(): Yeah - could be.

In fact, I have decided to bypass all the usual suspects, and use the shlwapi.dll
string functions. And why not? Most people have IE 4 and above now! Use
those APIs if you dare ...

>--
>Joe Foster <mailto:jlfoster%40znet.com>

--
Mark Alexander Bertenshaw
Programmer/Analyst
Chordiant Software, Inc.
Brentford
UK