Strings and memory management - Page 3


DevX Home    Today's Headlines   Articles Archive   Tip Bank   Forums   

Page 3 of 3 FirstFirst 123
Results 31 to 37 of 37

Thread: Strings and memory management

  1. #31
    Jonathan Wood Guest

    Re: Strings and memory management

    Brian,

    > Yes. that sounds like a better option. I should probably learn
    > how to do that, but until now have only been programming with
    > standard C++ features.


    Like I said, we do this type of thing all the type for ourselves and various
    clients.

    > Any suggestions on how best to write to file? i.e., is it
    > better to concatenate a large string and write that rather than
    > writing separate small string?? Is there a optimal string
    > length to write at a time?


    Disk access is always slow. For the most part, any code that must be as fast
    as possible will not include disk access. There are times where the data is
    organized in such a way that it can be written in a large chunk without a
    bunch of conversions. But for many cases, the benefits do not justify
    worrying about it.

    --
    Jonathan Wood
    SoftCircuits Programming
    http://www.softcircuits.com



  2. #32
    Jonathan Wood Guest

    Re: Strings and memory management

    Hey, rent a sense of humor! What makes you think I don't understand it?

    --
    Jonathan Wood
    SoftCircuits Programming
    http://www.softcircuits.com

    Michael (michka) Kaplan <former_mvp@spamfree.trigeminal.nospam.com> wrote in
    message news:3968e39a@news.devx.com...
    > (people who do not understand geek talk should not criticize it!)
    >
    > --
    > MichKa
    > "Cause it's a bittersweet symphony, thats life..." -- The Verve
    >
    > random junk of dubious value, at the multilingual,
    > no scripts required, http://www.trigeminal.com/
    >
    > "Jonathan Wood" <jwood@softcircuits.com> wrote in message
    > news:3968969a@news.devx.com...
    > > Oren,
    > >
    > > > [...] I've done many a dumps, )

    > >
    > > Please keep your bathroom habits private.
    > >
    > > Jonathan
    > >
    > >
    > >

    >
    >




  3. #33
    Jonathan Wood Guest

    Re: Strings and memory management

    Hey, rent a sense of humor! What makes you think I don't understand it?

    --
    Jonathan Wood
    SoftCircuits Programming
    http://www.softcircuits.com

    Michael (michka) Kaplan <former_mvp@spamfree.trigeminal.nospam.com> wrote in
    message news:3968e39a@news.devx.com...
    > (people who do not understand geek talk should not criticize it!)
    >
    > --
    > MichKa
    > "Cause it's a bittersweet symphony, thats life..." -- The Verve
    >
    > random junk of dubious value, at the multilingual,
    > no scripts required, http://www.trigeminal.com/
    >
    > "Jonathan Wood" <jwood@softcircuits.com> wrote in message
    > news:3968969a@news.devx.com...
    > > Oren,
    > >
    > > > [...] I've done many a dumps, )

    > >
    > > Please keep your bathroom habits private.
    > >
    > > Jonathan
    > >
    > >
    > >

    >
    >




  4. #34
    Anthony Jones Guest

    Re: Strings and memory management

    Brian,

    Unfortunately as Bill confirmed VB is pretty unintelligent when it comes to
    concatenation. In fact the following is slightly faster:

    s = "a" & "b"
    s = s & "c"

    It is very sad but I actually know the reason for this. :-)

    Apart from using a different language (which as you have pointed out has
    it's own conversion problems) the following is (a complete waste of time)
    the fastest way to concatenate 4 strings (the principle is true of three or
    more)

    <Code>
    Private Function FastCat4(s1 As String, s2 As String, s3 As String, s4 As
    String) As String

    Static ptr As Long
    Static ls1 As Long, ls2 As Long, ls3 As Long, ls4 As Long

    ls1 = LenB(s1)
    ls2 = LenB(s2)
    ls3 = LenB(s3)
    ls4 = LenB(s4)

    ptr = SysAllocStringLenPtr(ByVal 0&, (ls1 + ls2 + ls3 + ls4) \ 2)

    CopyMemory FastCat4, ptr, 4

    CopyMemory ByVal ptr, ByVal StrPtr(s1), ls1
    ptr = ptr + ls1
    CopyMemory ByVal ptr, ByVal StrPtr(s2), ls2
    ptr = ptr + ls2
    CopyMemory ByVal ptr, ByVal StrPtr(s3), ls3
    ptr = ptr + ls3
    CopyMemory ByVal ptr, ByVal StrPtr(s4), ls4

    End Function
    </Code>

    This is faster by about 10% for orginal string lengths of 15 characters. Of
    course as the string lengths or number of strings increase so does it's
    advantage.

    TypeLib based definitions for the API calls are necessary in order avoid
    calls to the GetLastError function that is implicit in using Declare.

    [
    entry("RtlMoveMemory"),
    helpstring("Copies memory from source to destination by reference"),
    ]
    void WINAPI CopyMemory(LPVOID pvDst,
    LPVOID pvSrc, DWORD cbCopy);

    [
    entry("SysAllocStringLen"),
    helpstring("Allocate String using pointers. (no last error)")
    ]
    int WINAPI SysAllocStringLenPtr(LPTSTR sz, UINT len);

    As you can see you would need to have a really, really good reason before
    using this.


    --
    Anthony Jones
    Secta Group Ltd




  5. #35
    Anthony Jones Guest

    Re: Strings and memory management

    Brian,

    Unfortunately as Bill confirmed VB is pretty unintelligent when it comes to
    concatenation. In fact the following is slightly faster:

    s = "a" & "b"
    s = s & "c"

    It is very sad but I actually know the reason for this. :-)

    Apart from using a different language (which as you have pointed out has
    it's own conversion problems) the following is (a complete waste of time)
    the fastest way to concatenate 4 strings (the principle is true of three or
    more)

    <Code>
    Private Function FastCat4(s1 As String, s2 As String, s3 As String, s4 As
    String) As String

    Static ptr As Long
    Static ls1 As Long, ls2 As Long, ls3 As Long, ls4 As Long

    ls1 = LenB(s1)
    ls2 = LenB(s2)
    ls3 = LenB(s3)
    ls4 = LenB(s4)

    ptr = SysAllocStringLenPtr(ByVal 0&, (ls1 + ls2 + ls3 + ls4) \ 2)

    CopyMemory FastCat4, ptr, 4

    CopyMemory ByVal ptr, ByVal StrPtr(s1), ls1
    ptr = ptr + ls1
    CopyMemory ByVal ptr, ByVal StrPtr(s2), ls2
    ptr = ptr + ls2
    CopyMemory ByVal ptr, ByVal StrPtr(s3), ls3
    ptr = ptr + ls3
    CopyMemory ByVal ptr, ByVal StrPtr(s4), ls4

    End Function
    </Code>

    This is faster by about 10% for orginal string lengths of 15 characters. Of
    course as the string lengths or number of strings increase so does it's
    advantage.

    TypeLib based definitions for the API calls are necessary in order avoid
    calls to the GetLastError function that is implicit in using Declare.

    [
    entry("RtlMoveMemory"),
    helpstring("Copies memory from source to destination by reference"),
    ]
    void WINAPI CopyMemory(LPVOID pvDst,
    LPVOID pvSrc, DWORD cbCopy);

    [
    entry("SysAllocStringLen"),
    helpstring("Allocate String using pointers. (no last error)")
    ]
    int WINAPI SysAllocStringLenPtr(LPTSTR sz, UINT len);

    As you can see you would need to have a really, really good reason before
    using this.


    --
    Anthony Jones
    Secta Group Ltd




  6. #36
    Ivan Weaver Guest

    Re: Strings and memory management

    Brian,
    You ask about optimal sizes for file writes, and stuff.
    Well, a few years ago I came across a routine by Crescent (from the Internet
    ToolPak - not sure if really theirs - but the here is the credit for my
    solution) for 'attaching' a mimed file. The routine was called 'LoadFile' -
    pretty original - but the name says it all.
    I came up with an adaptation of that code, I use a 'smart size' variable and
    use 4096 byte blocks (compared to their 32767byte blocks) since most of the
    windows OSs use 4K for allocating, and networks use 4K for its information
    interchange (usu.). If dealing with files - I use the OPEN FOR BINARY vb
    method
    Anyway by SmartSize it is intended for a loop of dealing with long strings
    or files, simply put....

    newStr = str(BlockSize, vbnullchar)
    for x = 1 to numChunks
    get# , , newStr
    if len(buffer$) < len(NewStr) + len(buffer$) then buffer$ = buffer$ &
    str(BLOCKSIZE, vbnullchar)
    mid$(buffer$,x,BLOCKSIZE)=NewStr
    next

    newStr = str(RemainderLen, vbNullChar)
    get# , , newStr
    if len(buffer$) < len(NewStr) + len(buffer$) then buffer$ = buffer$ &
    str(BLOCKSIZE, vbnullchar)
    mid$(buffer$,x,BLOCKSIZE)=NewStr

    then use the Buffer$ as you see fit.

    Example - with an adaptation, I have copied 32Mb files over a 10BaseT net in
    about 30seconds - During a busy day with over 200 people on the network
    (which IS FAR from optimized)

    "Brian Leung" <bleung@zoo.cam.ac.uk> wrote in message
    news:3969b2b4$1@news.devx.com...
    >
    > Hi Jonathan,
    > >
    > >Huh? If you are trying to optmize this process, why would you convert
    > >Unicode to ANSI and then back?

    >
    > I wouldn't want to, which is why I didn't like this idea.
    >
    > >
    > >If you use a typelib, VB can pass the Unicode strings unchanged. I would
    > >think you'd then want to work with and return Unicode strings.

    >
    > Yes. that sounds like a better option. I should probably learn
    > how to do that, but until now have only been programming with
    > standard C++ features.
    >
    > >
    > >Also, be careful with CopyMemory (RtlMoveMemory) because it will convert

    > the
    > >strings to ANSI and back unless you use something like StrPtr to pass the
    > >address of the Unicode data.

    >
    > I hadn't considered that.
    >
    > >
    > >> I suppose the slowest step would probably be writing
    > >> these strings to file rather than
    > >> memory management of even large strings.

    > >
    > >Amen to that!

    >
    > Any suggestions on how best to write to file? i.e., is it
    > better to concatenate a large string and write that rather than
    > writing separate small string?? Is there a optimal string
    > length to write at a time?
    >
    > Cheers,
    >
    > Brian
    > >
    > >Jonathan
    > >
    > >
    > >

    >




  7. #37
    Ivan Weaver Guest

    Re: Strings and memory management

    Brian,
    You ask about optimal sizes for file writes, and stuff.
    Well, a few years ago I came across a routine by Crescent (from the Internet
    ToolPak - not sure if really theirs - but the here is the credit for my
    solution) for 'attaching' a mimed file. The routine was called 'LoadFile' -
    pretty original - but the name says it all.
    I came up with an adaptation of that code, I use a 'smart size' variable and
    use 4096 byte blocks (compared to their 32767byte blocks) since most of the
    windows OSs use 4K for allocating, and networks use 4K for its information
    interchange (usu.). If dealing with files - I use the OPEN FOR BINARY vb
    method
    Anyway by SmartSize it is intended for a loop of dealing with long strings
    or files, simply put....

    newStr = str(BlockSize, vbnullchar)
    for x = 1 to numChunks
    get# , , newStr
    if len(buffer$) < len(NewStr) + len(buffer$) then buffer$ = buffer$ &
    str(BLOCKSIZE, vbnullchar)
    mid$(buffer$,x,BLOCKSIZE)=NewStr
    next

    newStr = str(RemainderLen, vbNullChar)
    get# , , newStr
    if len(buffer$) < len(NewStr) + len(buffer$) then buffer$ = buffer$ &
    str(BLOCKSIZE, vbnullchar)
    mid$(buffer$,x,BLOCKSIZE)=NewStr

    then use the Buffer$ as you see fit.

    Example - with an adaptation, I have copied 32Mb files over a 10BaseT net in
    about 30seconds - During a busy day with over 200 people on the network
    (which IS FAR from optimized)

    "Brian Leung" <bleung@zoo.cam.ac.uk> wrote in message
    news:3969b2b4$1@news.devx.com...
    >
    > Hi Jonathan,
    > >
    > >Huh? If you are trying to optmize this process, why would you convert
    > >Unicode to ANSI and then back?

    >
    > I wouldn't want to, which is why I didn't like this idea.
    >
    > >
    > >If you use a typelib, VB can pass the Unicode strings unchanged. I would
    > >think you'd then want to work with and return Unicode strings.

    >
    > Yes. that sounds like a better option. I should probably learn
    > how to do that, but until now have only been programming with
    > standard C++ features.
    >
    > >
    > >Also, be careful with CopyMemory (RtlMoveMemory) because it will convert

    > the
    > >strings to ANSI and back unless you use something like StrPtr to pass the
    > >address of the Unicode data.

    >
    > I hadn't considered that.
    >
    > >
    > >> I suppose the slowest step would probably be writing
    > >> these strings to file rather than
    > >> memory management of even large strings.

    > >
    > >Amen to that!

    >
    > Any suggestions on how best to write to file? i.e., is it
    > better to concatenate a large string and write that rather than
    > writing separate small string?? Is there a optimal string
    > length to write at a time?
    >
    > Cheers,
    >
    > Brian
    > >
    > >Jonathan
    > >
    > >
    > >

    >




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
HTML5 Development Center
 
 
FAQ
Latest Articles
Java
.NET
XML
Database
Enterprise
Questions? Contact us.
C++
Web Development
Wireless
Latest Tips
Open Source


   Development Centers

   -- Android Development Center
   -- Cloud Development Project Center
   -- HTML5 Development Center
   -- Windows Mobile Development Center