XML to pass data between tiers?

[Guy Smith]

I am concidering a designing an n-tier application using XML as the way to
pass information between the layers. I would like some feedback as to my
approach, any comments and any success/failures others have had.

I'll talk out loud about what I'm concidering...

I figure XML is a viable alernative to passing ADO disconnected recordset
or string arrays between the layers. I realize the tags in XML will effectively
double or triple the size of the information being passed VS string arrays
i.e. <firstName>Jack</firstName> vs strArray(0) = "Jack". VS disconnected
recordsets I figure the size should be about the same. Is this a good assumption?
Is the size unrealistic?

To populate the objects I am concidering using SAX. When a new object (business
rule layer) is created it can retrieve it's data in XML format from the data
access layer. It will implement SAX which will allow me to populate each
attribute as the XML is parsed. Sound good? What is the processing overhead
like? Would DOM be a better approach?

I figure SAX will work well when updating the object also. Say the employee's
Name is changed and it receives the new info from the UI in XML format. SAX
can be used again to do the update. Sound good?

To persist the object to the database, I'd have to generate XML from the
object's attributes to send back to the DB. What would be a good approach
to generate the XML? I can concat strings to do the trick... anything better
out there?

So there is my general approach in a nutshell. Any comments, suggestions,
flames, etc. would be appreciated!

Guy Smith

[MarkN]

You didn't say what programming language you were using. If it is VB, dump
it and go to a MS.Net language (if you must stick with MS technologies) or
Java.

I attempted this very same thing with VB because I didn't want to pass ADO
recordsets around (an Object plus all the stuff and Objects it knows about
- alot of overhead) because of the over head and I wanted to separate out
my data access. While it worked, I created mounds of code because I had
to hand parse everything. This became a maintenance nightmare. Not than
everything else in VB didn't either.

In MS.Net I believe you can just get xml from an object without alot of hand
coding so you don't need mounds of code to serialize objects.

Anyway, I am doing most of my programming in Java and in Java I can do this
without writing any parsing code.

Here is my suggestion. Code your application without XML in the mix. Properly
separate the layers - GUI, Business, Data Access. Then decide how xml should
fit into the mix. If you are doing a (D)HTML gui then generate xml from
your business objects to use with XSL or use Webforms and skip the XML part.
If you use WebForms and your all your (logical) layers are on one physical
layer you don't need XML at all. If your data access is on another server
then use Web Services to make calls to the data access from the business
layer or use remote objects. Either way, you don't have to think about XML.


Mark

[Guy Smith]

Thanks for the feedback Mark. It is appreciated. Oh and I am using VB6.

>Anyway, I am doing most of my programming in Java and in Java I can do this
>without writing any parsing code.

How do you find the performance using XML w/Java. Do the tags create a lot
of network overheard? I was looking at some XML retrieved to populate an
object and figure the tags add probably more in the lines of 4 times the
amount of data. This makes a Variant array (~14bytes + data size) a much
more efficient alternative (ouch) to XML and ado recordsets.

Did you ever use disconnected recordset? How was the performance? I keep
reading that they are questionable...

>Here is my suggestion. Code your application without XML in the mix. Properly
>separate the layers - GUI, Business, Data Access. Then decide how xml should
>fit into the mix. If you are doing a (D)HTML gui then generate xml from
>your business objects to use with XSL or use Webforms and skip the XML part.

This is a good alernative.

Thanks again Mark
Guy

[markn]

>Oh and I am using VB6.

Go to .Net or Java .

>How do you find the performance using XML w/Java. Do the tags create a
>lot
>of network overheard? I was looking at some XML retrieved to populate an
>object and figure the tags add probably more in the lines of 4 times the
>amount of data. This makes a Variant array (~14bytes + data size) a much
>more efficient alternative (ouch) to XML and ado recordsets.

Performance is perfectly fine. I've never measured the amount of data but
I do know recordsets carry alot of extra information and in addition are
objects = much overhead. The problem with Variant arrays is the maintenance.

Keep your XML as small as you can and you should be fine.

>
>Did you ever use disconnected recordset? How was the performance? I keep
>reading that they are questionable...

I've never used them. I know of people using them and they seem to work
fine. It is more of an architectual issue and that is why I don't/didn't
use them.


Mark

[JasonL]

"MarkN" <m@n.com> wrote :

> I attempted this very same thing with VB because I didn't want to pass ADO
> recordsets around (an Object plus all the stuff and Objects it knows about
> - alot of overhead) because of the over head and I wanted to separate out
> my data access. While it worked, I created mounds of code because I had
> to hand parse everything. This became a maintenance nightmare. Not than
> everything else in VB didn't either.

Mounds of code?? I successfully implemented a system that used XML rather
than ADO recordsets to pass data to the business layer. I designed a helper
object(s) that did all the XML parsing work at each end. Total lines of code
probably ~100, though it's been awhile since I've looked at it. The other
advantage to using this helper object at each end is that if I ever want to
modify the system to use ADO.NET recordsets, or some other method instead I
change the helper object only, my business and persistence (data layer)
objects don't change at all.

Sounds to me like the maintenance nightmare is the author, not the language.

JasonL

[MarkN]

Way to go back in time.

I'm sure I could have done it differently. It was the first XML project
I had done - with no one's help. I was attempting to hide implementation
too. This is much more complicated in VB than it is in MS.Net or VB. I
should have taken the time to write my own framework and did start on it.
With a small number of objects, it isn't to bad. But I had quite a few.

>>Sounds to me like the maintenance nightmare is the author, not the >>language.

I said it became one. Didn't say why. Partly it is the language (Reflection
would help) and alot was lack of experience with what I was trying to do.


How do you map your XML elements to your business objects?


"JasonL" <langston@no_spam_chime.org> wrote:
>"MarkN" <m@n.com> wrote :
>
>> I attempted this very same thing with VB because I didn't want to pass
ADO
>> recordsets around (an Object plus all the stuff and Objects it knows about
>> - alot of overhead) because of the over head and I wanted to separate
out
>> my data access. While it worked, I created mounds of code because I had
>> to hand parse everything. This became a maintenance nightmare. Not than
>> everything else in VB didn't either.
>
>Mounds of code?? I successfully implemented a system that used XML rather
>than ADO recordsets to pass data to the business layer. I designed a helper
>object(s) that did all the XML parsing work at each end. Total lines of
code
>probably ~100, though it's been awhile since I've looked at it. The other
>advantage to using this helper object at each end is that if I ever want
to
>modify the system to use ADO.NET recordsets, or some other method instead
I
>change the helper object only, my business and persistence (data layer)
>objects don't change at all.
>
>Sounds to me like the maintenance nightmare is the author, not the language.
>
>JasonL
>
>

[JasonL]

> Way to go back in time.

Just happened by, your comment caught my eye.

> >>Sounds to me like the maintenance nightmare is the author, not the
>>language.
>
> I said it became one. Didn't say why. Partly it is the language
(Reflection
> would help) and alot was lack of experience with what I was trying to do.

Sorry, that shouldn't have been a personal attack. My only point was that
the poster shouldn't have to run out and change languages simply to
implement this feature. It is possible to do with "classic" VB without a
whole lot of work.

no hard feelings??

> How do you map your XML elements to your business objects?

It's not extremely clever but the CPropMarshal object has these two methods:

getProp(sPropertyName) As String
setProp(sPropertyName, sValue)

The business object (in a load method or something) goes something like this

Me.Name = oPropMarshal.getProp("name")
Me.Age = CInt(oPropMarshal.getProp("age"))

The getProp method handles parsing the XML for the node with the passed in
property name. So all the parsing happens in one spot.

[MarkN]

More I thought, the more you were right. Sorry for being sensitive. I should
have said a little more the first time or just taken my knocks.

I think what you've done is pretty good, maybe what I would have come up
with if I had taken the time (multiple contracts + other stuff didn't give
me time) - started to do that, and is probably the best that could be done.
Guess what I was thinking/saying is that with reflection you would not even
need to do the following:

Me.Name = oPropMarshal.getProp("name")
Me.Age = CInt(oPropMarshal.getProp("age"))


That plus getting it out and putting it back into the database turned out
to be alot of code becuase I had alot of objects and multi-level objects.
It wasn't that much in each object it was just when I put it all together.


Mark

"JasonL" <langston@no_spam_chime.org> wrote:
>> Way to go back in time.
>
>Just happened by, your comment caught my eye.
>
>> >>Sounds to me like the maintenance nightmare is the author, not the
>>>language.
>>
>> I said it became one. Didn't say why. Partly it is the language
>(Reflection
>> would help) and alot was lack of experience with what I was trying to
do.
>
>Sorry, that shouldn't have been a personal attack. My only point was that
>the poster shouldn't have to run out and change languages simply to
>implement this feature. It is possible to do with "classic" VB without a
>whole lot of work.
>
>no hard feelings??
>
>> How do you map your XML elements to your business objects?
>
>It's not extremely clever but the CPropMarshal object has these two methods:
>
>getProp(sPropertyName) As String
>setProp(sPropertyName, sValue)
>
>The business object (in a load method or something) goes something like
this
>
>Me.Name = oPropMarshal.getProp("name")
>Me.Age = CInt(oPropMarshal.getProp("age"))
>
>The getProp method handles parsing the XML for the node with the passed
in
>property name. So all the parsing happens in one spot.
>
>

[JasonL]

"MarkN" <m@N.com> wrote :
>I think what you've done is pretty good, maybe what I would have come up

Can't take full credit ... Rocky's (VBx Business Objects) UDT/Lset pattern
got me started. Many other samples, etc. led to the final outcome.

> Guess what I was thinking/saying is that with reflection you would not
even
> need to do the following:
>
> Me.Name = oPropMarshal.getProp("name")
> Me.Age = CInt(oPropMarshal.getProp("age"))

Never had the pleasure of working with Java (well more JScript than I care
to remember - but I don't count that as Java), and just starting to poke
around in dotNET. Any references or examples of how reflection would solve
this problem? Does Java have a built-in serialization method/helper class?
Or did you write your own? (just curious)

> That plus getting it out and putting it back into the database turned out
> to be alot of code becuase I had alot of objects and multi-level objects.
> It wasn't that much in each object it was just when I put it all
together.

You got me there! Indeed, that was the most painful part - reinventing much
of the ADO "wheel". It was an interesting exercise. I'm honestly considering
for future projects to change the internal implementation of CPropMarshal to
use ADO.NET. The capabilities of the DataSet intrigue me.

[MarkN]

Thinking more about it, there is a way to call methods in VB by name. Don't
know how well it works on properties. I believe that is in Rocky's book
too.


>>I think what you've done is pretty good, maybe what I would have come up
>
>Can't take full credit ... Rocky's (VBx Business Objects) UDT/Lset pattern
>got me started. Many other samples, etc. led to the final outcome.

His book got me started too. The lset thing was driving me nuts because
everytime I added a field I had to figure it out again.

>
>> Guess what I was thinking/saying is that with reflection you would not
>even
>> need to do the following:
>>
>> Me.Name = oPropMarshal.getProp("name")
>> Me.Age = CInt(oPropMarshal.getProp("age"))
>
>Never had the pleasure of working with Java (well more JScript than I care
>to remember - but I don't count that as Java), and just starting to poke
>around in dotNET. Any references or examples of how reflection would solve
>this problem? Does Java have a built-in serialization method/helper class?
>Or did you write your own? (just curious)

I used open source APIs. Castor has one (www.castor.org). Apache has one
too I think. There are others.

>
>> That plus getting it out and putting it back into the database turned
out
>> to be alot of code becuase I had alot of objects and multi-level objects.
>> It wasn't that much in each object it was just when I put it all
>together.
>
>You got me there! Indeed, that was the most painful part - reinventing much
>of the ADO "wheel". It was an interesting exercise. I'm honestly considering
>for future projects to change the internal implementation of CPropMarshal
to
>use ADO.NET. The capabilities of the DataSet intrigue me.

I've done some things with that. I found some things frustrating if I went
beyond what was generated. Couldn't find much documentation on it.

With Java (this can be done with .Net if someone wants to write it) I can
retrieve my objects from the DB (Hibernate or Castor for example) move them
from XML and back (Castor for example) send across to the client (GLUE for
example) and write very little code. The db to object conversion doesn't
go through an XML conversion first. And I pay nothing for the tools!

[MarkN]

Here is the apache one - http://jakarta.apache.org/commons/betwixt/

"MarkN" <m@n.com> wrote:
>
>Thinking more about it, there is a way to call methods in VB by name. Don't
>know how well it works on properties. I believe that is in Rocky's book
>too.
>
>
>>>I think what you've done is pretty good, maybe what I would have come
up
>>
>>Can't take full credit ... Rocky's (VBx Business Objects) UDT/Lset pattern
>>got me started. Many other samples, etc. led to the final outcome.
>
>His book got me started too. The lset thing was driving me nuts because
>everytime I added a field I had to figure it out again.
>
>>
>>> Guess what I was thinking/saying is that with reflection you would not
>>even
>>> need to do the following:
>>>
>>> Me.Name = oPropMarshal.getProp("name")
>>> Me.Age = CInt(oPropMarshal.getProp("age"))
>>
>>Never had the pleasure of working with Java (well more JScript than I care
>>to remember - but I don't count that as Java), and just starting to poke
>>around in dotNET. Any references or examples of how reflection would solve
>>this problem? Does Java have a built-in serialization method/helper class?
>>Or did you write your own? (just curious)
>
>I used open source APIs. Castor has one (www.castor.org). Apache has one
>too I think. There are others.
>
>>
>>> That plus getting it out and putting it back into the database turned
>out
>>> to be alot of code becuase I had alot of objects and multi-level objects.
>>> It wasn't that much in each object it was just when I put it all
>>together.
>>
>>You got me there! Indeed, that was the most painful part - reinventing
much
>>of the ADO "wheel". It was an interesting exercise. I'm honestly considering
>>for future projects to change the internal implementation of CPropMarshal
>to
>>use ADO.NET. The capabilities of the DataSet intrigue me.
>
>I've done some things with that. I found some things frustrating if I went
>beyond what was generated. Couldn't find much documentation on it.
>
>With Java (this can be done with .Net if someone wants to write it) I can
>retrieve my objects from the DB (Hibernate or Castor for example) move them
>from XML and back (Castor for example) send across to the client (GLUE for
>example) and write very little code. The db to object conversion doesn't
>go through an XML conversion first. And I pay nothing for the tools!
>

[JasonL]

> You got me there! Indeed, that was the most painful part - reinventing
much
> of the ADO "wheel". It was an interesting exercise. I'm honestly
considering
> for future projects to change the internal implementation of CPropMarshal
to
> use ADO.NET. The capabilities of the DataSet intrigue me.

Actually, for the record, Mark's comments got me to dig a little into .NET.
Wow! Mark, you're right. The Framework can handle all the gory details of
serialization and marshalling for you. A few simple, intuitive lines of code
(http://msdn.microsoft.com/msdnmag/is...et/net0204.asp) and it's
all done for you. So no need for my CPropMarshal class anymore (good
riddance!).

Another example that moving from "classic" VB to .NET is a paradigm shift,
not just a few syntax changes.

Thanks for piquing my curiosity Mark.

JasonL

[Joe \Nuke Me Xemu\ Foster]

"JasonL" <langston@no_spam_chime.org> wrote in message <news:3d4554e5$1@10.1.10.29>...

> > You got me there! Indeed, that was the most painful part - reinventing
> much
> > of the ADO "wheel". It was an interesting exercise. I'm honestly
> considering
> > for future projects to change the internal implementation of CPropMarshal
> to
> > use ADO.NET. The capabilities of the DataSet intrigue me.
>
> Actually, for the record, Mark's comments got me to dig a little into .NET.
> Wow! Mark, you're right. The Framework can handle all the gory details of
> serialization and marshalling for you. A few simple, intuitive lines of code
> (http://msdn.microsoft.com/msdnmag/is...et/net0204.asp) and it's
> all done for you. So no need for my CPropMarshal class anymore (good
> riddance!).
>
> Another example that moving from "classic" VB to .NET is a paradigm shift,
> not just a few syntax changes.

Class_Terminate to Finalize is certainly worse than just a syntax change.
Anyway, one would certainly hope that 20+MB of run-times should support
something someone somewhere might find somewhat useful...

--
Joe Foster <mailto:jlfoster%40znet.com> Got Thetans? <http://www.xenu.net/>
WARNING: I cannot be held responsible for the above They're coming to
because my cats have apparently learned to type. take me away, ha ha!

[Jeff Jones]

I know you posted this in May, but given the sometimes questionable response
you got, I'll give it shot:

> I am concidering a designing an n-tier application using XML as the way to
> pass information between the layers. I would like some feedback as to my
> approach, any comments and any success/failures others have had.
>
> I'll talk out loud about what I'm concidering...
>
> I figure XML is a viable alernative to passing ADO disconnected recordset
> or string arrays between the layers. I realize the tags in XML will
effectively
> double or triple the size of the information being passed VS string arrays
> i.e. <firstName>Jack</firstName> vs strArray(0) = "Jack". VS disconnected
> recordsets I figure the size should be about the same. Is this a good
assumption?
> Is the size unrealistic?
Why use XML? All the hype aside, XML is useful for exchanging data between
disparate systems. If your layers are all based on MS technology (ASP,
WebClasses, VB, SQL Server, etc) then XML would be a *huge* waste of
resources and programming time.
First of all, disconnected ADO recordsets marshal-by-value, and marshalling
is optimized specifically for ADO recordsets. Based on my own tests, ADO
recordsets - particularly shaped recordsets - performed as well as arrays
with ASP pages. Contrary to some opinions, using a disconnected ADO
recordset fits into a good OO design. ADO recordsets are easier to work
with than XML parsers, and easier to code to.

> To persist the object to the database, I'd have to generate XML from the
> object's attributes to send back to the DB. What would be a good approach
> to generate the XML? I can concat strings to do the trick... anything
better
> out there?
Another advantage of ADO recordsets is that you can update the DB with them.
However, if you want scalability, updating the DB should always use SPs (as
opposed to static SQL for maintainability purposes) from ADO connection
objects.

> So there is my general approach in a nutshell. Any comments, suggestions,
> flames, etc. would be appreciated!
No flames - sounds like you have really put a lot of thought into this. But
XML is rarely ever needed when working within the same technology.

Although you can do all this in VB6, if time and expertise permit, VB.NET
would be better. In .NET, you can marshal by value any object you build
(something you could not do in VB6).

Either way, don't be fooled by the XML hype. From what you wrote, I can't
see any reason you need to use it.

Jeff Jones
Marietta, GA