Performance probs when creating large XML docs

[bparker]

I have some code that takes data from a file and puts it into an XML format.
I am using the Microsoft DOM (CreateElement) and VB. As I increase the
amount of data in the file, the longer it takes for the XML to be produced.
For example, 100 records in the file takes 4 seconds to parse into a XML
file. 600 records in the file takes 112 seconds to parse into a XML file.
Ideally, I would hope every 100 records would take 4 seconds so for 600
records it should take 24 seconds.

Any suggestions on how to improve performance

[Russell Jones]

From a performance standpoint, you're almost always better off using SAX or
simple string manipulation to build a document. Typically you should use the
DOM when you need to make repeated queries to retrieve data from an XML
document. Take a look at the MXXMLWriter object and sample application.
Although that's not a perfect example for what you're doing, maybe you'll
get some good ideas. As you can see, there's a penalty for building a large
in-memory object hierarchy.

With that said, the times you're giving seem excessive. If you post some
example code, maybe someone here can help you optimize it so you don't have
to change the technology you're using.

Look at the SDK under the
"bparker" <bparkerhsd@earthlink.net> wrote in message
news:3a9d4996$1@news.devx.com...
>
> I have some code that takes data from a file and puts it into an XML
format.
> I am using the Microsoft DOM (CreateElement) and VB. As I increase the
> amount of data in the file, the longer it takes for the XML to be
produced.
> For example, 100 records in the file takes 4 seconds to parse into a XML
> file. 600 records in the file takes 112 seconds to parse into a XML file.
> Ideally, I would hope every 100 records would take 4 seconds so for 600
> records it should take 24 seconds.
>
> Any suggestions on how to improve performance

[Greg Longtin]

That's real slow. Have you looked at cloneNode? A while back I found
that speeded things up. Create a node with all your record columns,
then clone it, stuff it, add it, clone again...

BTW, if you're simply saving to an XML file, and not re-using the XML
doc, you might just assemble a string.

Greg

"bparker" <bparkerhsd@earthlink.net> wrote in message
news:3a9d4996$1@news.devx.com...
>
> I have some code that takes data from a file and puts it into an XML
format.
> I am using the Microsoft DOM (CreateElement) and VB. As I increase
the
> amount of data in the file, the longer it takes for the XML to be
produced.
> For example, 100 records in the file takes 4 seconds to parse into a
XML
> file. 600 records in the file takes 112 seconds to parse into a XML
file.
> Ideally, I would hope every 100 records would take 4 seconds so for
600
> records it should take 24 seconds.
>
> Any suggestions on how to improve performance

[Paul]

String concatenation in VB is slow (the problem gets worse as your string
gets larger). Shove all your individual strings into array elements and then
user the JOIN command to dump the array elements into a string after the
loop (I assume your doing it that way).

Cheers.

P.


"bparker" <bparkerhsd@earthlink.net> wrote:
>
>I have some code that takes data from a file and puts it into an XML format.
> I am using the Microsoft DOM (CreateElement) and VB. As I increase the
>amount of data in the file, the longer it takes for the XML to be produced.
> For example, 100 records in the file takes 4 seconds to parse into a XML
>file. 600 records in the file takes 112 seconds to parse into a XML file.
> Ideally, I would hope every 100 records would take 4 seconds so for 600
>records it should take 24 seconds.
>
>Any suggestions on how to improve performance