Re: Strings and memory management
Brian,
> Yes. that sounds like a better option. I should probably learn
> how to do that, but until now have only been programming with
> standard C++ features.
Like I said, we do this type of thing all the type for ourselves and various
clients.
> Any suggestions on how best to write to file? i.e., is it
> better to concatenate a large string and write that rather than
> writing separate small string?? Is there a optimal string
> length to write at a time?
Disk access is always slow. For the most part, any code that must be as fast
as possible will not include disk access. There are times where the data is
organized in such a way that it can be written in a large chunk without a
bunch of conversions. But for many cases, the benefits do not justify
worrying about it.
--
Jonathan Wood
SoftCircuits Programming
http://www.softcircuits.com
Re: Strings and memory management
Hey, rent a sense of humor! What makes you think I don't understand it?
--
Jonathan Wood
SoftCircuits Programming
http://www.softcircuits.com
Michael (michka) Kaplan <former_mvp@spamfree.trigeminal.nospam.com> wrote in
message news:3968e39a@news.devx.com...
> (people who do not understand geek talk should not criticize it!)
>
> --
> MichKa
> "Cause it's a bittersweet symphony, thats life..." -- The Verve
>
> random junk of dubious value, at the multilingual,
> no scripts required, http://www.trigeminal.com/
>
> "Jonathan Wood" <jwood@softcircuits.com> wrote in message
> news:3968969a@news.devx.com...
> > Oren,
> >
> > > [...] I've done many a dumps, :o)
> >
> > Please keep your bathroom habits private.
> >
> > Jonathan
> >
> >
> >
>
>
Re: Strings and memory management
Hey, rent a sense of humor! What makes you think I don't understand it?
--
Jonathan Wood
SoftCircuits Programming
http://www.softcircuits.com
Michael (michka) Kaplan <former_mvp@spamfree.trigeminal.nospam.com> wrote in
message news:3968e39a@news.devx.com...
> (people who do not understand geek talk should not criticize it!)
>
> --
> MichKa
> "Cause it's a bittersweet symphony, thats life..." -- The Verve
>
> random junk of dubious value, at the multilingual,
> no scripts required, http://www.trigeminal.com/
>
> "Jonathan Wood" <jwood@softcircuits.com> wrote in message
> news:3968969a@news.devx.com...
> > Oren,
> >
> > > [...] I've done many a dumps, :o)
> >
> > Please keep your bathroom habits private.
> >
> > Jonathan
> >
> >
> >
>
>
Re: Strings and memory management
Brian,
Unfortunately as Bill confirmed VB is pretty unintelligent when it comes to
concatenation. In fact the following is slightly faster:
s = "a" & "b"
s = s & "c"
It is very sad but I actually know the reason for this. :-)
Apart from using a different language (which as you have pointed out has
it's own conversion problems) the following is (a complete waste of time)
the fastest way to concatenate 4 strings (the principle is true of three or
more)
<Code>
Private Function FastCat4(s1 As String, s2 As String, s3 As String, s4 As
String) As String
Static ptr As Long
Static ls1 As Long, ls2 As Long, ls3 As Long, ls4 As Long
ls1 = LenB(s1)
ls2 = LenB(s2)
ls3 = LenB(s3)
ls4 = LenB(s4)
ptr = SysAllocStringLenPtr(ByVal 0&, (ls1 + ls2 + ls3 + ls4) \ 2)
CopyMemory FastCat4, ptr, 4
CopyMemory ByVal ptr, ByVal StrPtr(s1), ls1
ptr = ptr + ls1
CopyMemory ByVal ptr, ByVal StrPtr(s2), ls2
ptr = ptr + ls2
CopyMemory ByVal ptr, ByVal StrPtr(s3), ls3
ptr = ptr + ls3
CopyMemory ByVal ptr, ByVal StrPtr(s4), ls4
End Function
</Code>
This is faster by about 10% for orginal string lengths of 15 characters. Of
course as the string lengths or number of strings increase so does it's
advantage.
TypeLib based definitions for the API calls are necessary in order avoid
calls to the GetLastError function that is implicit in using Declare.
[
entry("RtlMoveMemory"),
helpstring("Copies memory from source to destination by reference"),
]
void WINAPI CopyMemory(LPVOID pvDst,
LPVOID pvSrc, DWORD cbCopy);
[
entry("SysAllocStringLen"),
helpstring("Allocate String using pointers. (no last error)")
]
int WINAPI SysAllocStringLenPtr(LPTSTR sz, UINT len);
As you can see you would need to have a really, really good reason before
using this.
--
Anthony Jones
Secta Group Ltd
Re: Strings and memory management
Brian,
Unfortunately as Bill confirmed VB is pretty unintelligent when it comes to
concatenation. In fact the following is slightly faster:
s = "a" & "b"
s = s & "c"
It is very sad but I actually know the reason for this. :-)
Apart from using a different language (which as you have pointed out has
it's own conversion problems) the following is (a complete waste of time)
the fastest way to concatenate 4 strings (the principle is true of three or
more)
<Code>
Private Function FastCat4(s1 As String, s2 As String, s3 As String, s4 As
String) As String
Static ptr As Long
Static ls1 As Long, ls2 As Long, ls3 As Long, ls4 As Long
ls1 = LenB(s1)
ls2 = LenB(s2)
ls3 = LenB(s3)
ls4 = LenB(s4)
ptr = SysAllocStringLenPtr(ByVal 0&, (ls1 + ls2 + ls3 + ls4) \ 2)
CopyMemory FastCat4, ptr, 4
CopyMemory ByVal ptr, ByVal StrPtr(s1), ls1
ptr = ptr + ls1
CopyMemory ByVal ptr, ByVal StrPtr(s2), ls2
ptr = ptr + ls2
CopyMemory ByVal ptr, ByVal StrPtr(s3), ls3
ptr = ptr + ls3
CopyMemory ByVal ptr, ByVal StrPtr(s4), ls4
End Function
</Code>
This is faster by about 10% for orginal string lengths of 15 characters. Of
course as the string lengths or number of strings increase so does it's
advantage.
TypeLib based definitions for the API calls are necessary in order avoid
calls to the GetLastError function that is implicit in using Declare.
[
entry("RtlMoveMemory"),
helpstring("Copies memory from source to destination by reference"),
]
void WINAPI CopyMemory(LPVOID pvDst,
LPVOID pvSrc, DWORD cbCopy);
[
entry("SysAllocStringLen"),
helpstring("Allocate String using pointers. (no last error)")
]
int WINAPI SysAllocStringLenPtr(LPTSTR sz, UINT len);
As you can see you would need to have a really, really good reason before
using this.
--
Anthony Jones
Secta Group Ltd
Re: Strings and memory management
Brian,
You ask about optimal sizes for file writes, and stuff.
Well, a few years ago I came across a routine by Crescent (from the Internet
ToolPak - not sure if really theirs - but the here is the credit for my
solution) for 'attaching' a mimed file. The routine was called 'LoadFile' -
pretty original - but the name says it all.
I came up with an adaptation of that code, I use a 'smart size' variable and
use 4096 byte blocks (compared to their 32767byte blocks) since most of the
windows OSs use 4K for allocating, and networks use 4K for its information
interchange (usu.). If dealing with files - I use the OPEN FOR BINARY vb
method
Anyway by SmartSize it is intended for a loop of dealing with long strings
or files, simply put....
newStr = str(BlockSize, vbnullchar)
for x = 1 to numChunks
get# , , newStr
if len(buffer$) < len(NewStr) + len(buffer$) then buffer$ = buffer$ &
str(BLOCKSIZE, vbnullchar)
mid$(buffer$,x,BLOCKSIZE)=NewStr
next
newStr = str(RemainderLen, vbNullChar)
get# , , newStr
if len(buffer$) < len(NewStr) + len(buffer$) then buffer$ = buffer$ &
str(BLOCKSIZE, vbnullchar)
mid$(buffer$,x,BLOCKSIZE)=NewStr
then use the Buffer$ as you see fit.
Example - with an adaptation, I have copied 32Mb files over a 10BaseT net in
about 30seconds - During a busy day with over 200 people on the network
(which IS FAR from optimized)
"Brian Leung" <bleung@zoo.cam.ac.uk> wrote in message
news:3969b2b4$1@news.devx.com...
>
> Hi Jonathan,
> >
> >Huh? If you are trying to optmize this process, why would you convert
> >Unicode to ANSI and then back?
>
> I wouldn't want to, which is why I didn't like this idea.
>
> >
> >If you use a typelib, VB can pass the Unicode strings unchanged. I would
> >think you'd then want to work with and return Unicode strings.
>
> Yes. that sounds like a better option. I should probably learn
> how to do that, but until now have only been programming with
> standard C++ features.
>
> >
> >Also, be careful with CopyMemory (RtlMoveMemory) because it will convert
> the
> >strings to ANSI and back unless you use something like StrPtr to pass the
> >address of the Unicode data.
>
> I hadn't considered that.
>
> >
> >> I suppose the slowest step would probably be writing
> >> these strings to file rather than
> >> memory management of even large strings.
> >
> >Amen to that!
>
> Any suggestions on how best to write to file? i.e., is it
> better to concatenate a large string and write that rather than
> writing separate small string?? Is there a optimal string
> length to write at a time?
>
> Cheers,
>
> Brian
> >
> >Jonathan
> >
> >
> >
>
Re: Strings and memory management
Brian,
You ask about optimal sizes for file writes, and stuff.
Well, a few years ago I came across a routine by Crescent (from the Internet
ToolPak - not sure if really theirs - but the here is the credit for my
solution) for 'attaching' a mimed file. The routine was called 'LoadFile' -
pretty original - but the name says it all.
I came up with an adaptation of that code, I use a 'smart size' variable and
use 4096 byte blocks (compared to their 32767byte blocks) since most of the
windows OSs use 4K for allocating, and networks use 4K for its information
interchange (usu.). If dealing with files - I use the OPEN FOR BINARY vb
method
Anyway by SmartSize it is intended for a loop of dealing with long strings
or files, simply put....
newStr = str(BlockSize, vbnullchar)
for x = 1 to numChunks
get# , , newStr
if len(buffer$) < len(NewStr) + len(buffer$) then buffer$ = buffer$ &
str(BLOCKSIZE, vbnullchar)
mid$(buffer$,x,BLOCKSIZE)=NewStr
next
newStr = str(RemainderLen, vbNullChar)
get# , , newStr
if len(buffer$) < len(NewStr) + len(buffer$) then buffer$ = buffer$ &
str(BLOCKSIZE, vbnullchar)
mid$(buffer$,x,BLOCKSIZE)=NewStr
then use the Buffer$ as you see fit.
Example - with an adaptation, I have copied 32Mb files over a 10BaseT net in
about 30seconds - During a busy day with over 200 people on the network
(which IS FAR from optimized)
"Brian Leung" <bleung@zoo.cam.ac.uk> wrote in message
news:3969b2b4$1@news.devx.com...
>
> Hi Jonathan,
> >
> >Huh? If you are trying to optmize this process, why would you convert
> >Unicode to ANSI and then back?
>
> I wouldn't want to, which is why I didn't like this idea.
>
> >
> >If you use a typelib, VB can pass the Unicode strings unchanged. I would
> >think you'd then want to work with and return Unicode strings.
>
> Yes. that sounds like a better option. I should probably learn
> how to do that, but until now have only been programming with
> standard C++ features.
>
> >
> >Also, be careful with CopyMemory (RtlMoveMemory) because it will convert
> the
> >strings to ANSI and back unless you use something like StrPtr to pass the
> >address of the Unicode data.
>
> I hadn't considered that.
>
> >
> >> I suppose the slowest step would probably be writing
> >> these strings to file rather than
> >> memory management of even large strings.
> >
> >Amen to that!
>
> Any suggestions on how best to write to file? i.e., is it
> better to concatenate a large string and write that rather than
> writing separate small string?? Is there a optimal string
> length to write at a time?
>
> Cheers,
>
> Brian
> >
> >Jonathan
> >
> >
> >
>
