Why are we going backwards in technology?

[Clarity]

I've been studying XML/schema/SOAP/SAX2 for about 6 months now... I have some
questions for anyone who cares:

Since we all know that you cannot parse a very large file using DOM, you
are forced to use SAX... But, then you have to use a query to get to ONLY
the information you want. Then you have to sequentially get to that information
because there is no "Primary Key" or Indexes that might be built prior to
the execution of the query.

My need is this: Get to the Data, Fast, using a Standard query language
(Like SQL), that doesn't have to sequentially go through the data, that uses
indexes for speed, that has a standard for defining heirarchy, types, and
rules.
I understand that standard database systems are not easily accessed through
the 'net, but holy moly. The funny thing about standards is that there are
so many of them.
To me, this is Jurrassic Park Syndrome. We spend so much time trying to
figure out if we CAN do something without spending the time to think about
whether we should have.
To me, Everyone is using the Buzz word XML. But in real-world situations
I have come across, there is not enough support, speed, or interoperability
to provide a solid solution to Database access without having to be anal
retentive about methods, drivers, memory limitations, and conversion issues.
Maybe I'm crazy, or just ill-informed. You be the judge.

[Russell Jones]

Here's why XML is important--and it has nothing to do with the size of the
data, the method used to parse it, or the data itself. XML is important
because it provides a common file format. That's all. Of course, that is
significant. Consider the problems involved in sending data from one type of
database to a company that runs another, different database (not an uncommon
need, I'm sure). You have to save the data out as some type of delimited or
fixed-field ASCII file. Then you have to let the recipient know the data
types, sizes, defaults, and constraints. The recipient needs to write
special scripts to import the files. These scripts must be tested and
debugged--tables need to be built, the rules, and defaults must be
duplicated--and none of this is automatic.

Now consider the same scenario with XML. The donor saves the table in XML
format and provides a schema (both of which are automatically generated by
the database). The recipient imports the data--all validated against the
schema, and the database itself generates all the tables, rules, and
defaults.

If you are the owner of a company, which scenario would seem less costly and
error-prone to you?

Sure, XML is an inefficient data format, and doesn't provide high
performance for large files--but you don't need it to. When you have large
amounts of data, and you need to get to it very quickly, put it in a
relational database--they're optimized for that type of data access. But
remember XML when you need to get the data back in usable form to the people
who need it.

Here's another reason. The SQL and table schema required to retrieve and
store hierarchical and irregular data is horrendously complex, not very
efficient, and highly inflexible. There's a great deal of hierarchical and
irregular data in the world. Any technology which simplifies working with
that kind of data is welcome--especially if it turns out to be a standard
method for working with both regular and irregular data types.

Finally, there's no particular reason why you can't index XML files so that
they're just as fast as any other kind of data-storage mechanism--in fact,
there are companies that advertise these capabilities now. So while the
methods and tools may not be mature yet, give XML the benefit of the many
years of development that relational databases have had, and it may evolve
to be a reasonable competitor, even with large data sets.

I hope this helps.


"Clarity" <thewarnk@flash.net> wrote in message
news:3a3e8672$1@news.devx.com...
>
> I've been studying XML/schema/SOAP/SAX2 for about 6 months now... I have
some
> questions for anyone who cares:
>
> Since we all know that you cannot parse a very large file using DOM, you
> are forced to use SAX... But, then you have to use a query to get to ONLY
> the information you want. Then you have to sequentially get to that
information
> because there is no "Primary Key" or Indexes that might be built prior to
> the execution of the query.
>
> My need is this: Get to the Data, Fast, using a Standard query language
> (Like SQL), that doesn't have to sequentially go through the data, that
uses
> indexes for speed, that has a standard for defining heirarchy, types, and
> rules.
> I understand that standard database systems are not easily accessed
through
> the 'net, but holy moly. The funny thing about standards is that there
are
> so many of them.
> To me, this is Jurrassic Park Syndrome. We spend so much time trying to
> figure out if we CAN do something without spending the time to think about
> whether we should have.
> To me, Everyone is using the Buzz word XML. But in real-world situations
> I have come across, there is not enough support, speed, or
interoperability
> to provide a solid solution to Database access without having to be anal
> retentive about methods, drivers, memory limitations, and conversion
issues.
> Maybe I'm crazy, or just ill-informed. You be the judge.
>

[Brandon Forest]

Clarity,

I understand your frustration. I have been studying XML for almost a year
now. I agree with your evaluation on it's shortcomings, but be patient tools
are coming. SQL Server 2000 can return recordsets as native XML, so you
can write stored procedures in SQL and then transport in XML. xPath it the
XML query language. It is primitive now, but I expect it to become powerful
within a couple of interations.

The key reason that XML will become the next big thing, is because it is
truly cross platform, http protocol, and firewall friendly.

"Clarity" <thewarnk@flash.net> wrote:
>
>I've been studying XML/schema/SOAP/SAX2 for about 6 months now... I have
some
>questions for anyone who cares:
>
>Since we all know that you cannot parse a very large file using DOM, you
>are forced to use SAX... But, then you have to use a query to get to ONLY
>the information you want. Then you have to sequentially get to that information
>because there is no "Primary Key" or Indexes that might be built prior to
>the execution of the query.
>
>My need is this: Get to the Data, Fast, using a Standard query language
>(Like SQL), that doesn't have to sequentially go through the data, that
uses
>indexes for speed, that has a standard for defining heirarchy, types, and
>rules.
>I understand that standard database systems are not easily accessed through
>the 'net, but holy moly. The funny thing about standards is that there
are
>so many of them.
>To me, this is Jurrassic Park Syndrome. We spend so much time trying to
>figure out if we CAN do something without spending the time to think about
>whether we should have.
>To me, Everyone is using the Buzz word XML. But in real-world situations
>I have come across, there is not enough support, speed, or interoperability
>to provide a solid solution to Database access without having to be anal
>retentive about methods, drivers, memory limitations, and conversion issues.
>Maybe I'm crazy, or just ill-informed. You be the judge.
>

[Clarity]

Thanks for your insight.

In your travels, please tell all "XMLr's" to stop creating monster data files
that are expected to be used as a database. I think the problem lies in
the fact that many people do not understand that XML is great for a "transfer
mechanism" that is schema based. They try to maintain a data store of all
records and expect the queries and record manipulation to be performed on
that XML file. As you stated... This is highly ineffecient-for now. I also
agree that Indexes could be created, but is that at the client side or the
server side? And if it is the client side, is the index up to date?...


NEWAy, I've been doing some work with SQL Server 2000 recently and hit the
Brick wall of trying to convert Large XML Sources to a recordset (via the
bulk conversion). An XPath query is performed and files as little as 190K
were dying. I was forced to try different storage variables in the Stored
Procedure I wrote... After much frustration, I found that a "Beta" version
of XML for SQL 2000 was released on Dec 7, 2000 that addressed Bulk XML Processing...
Still testing this, but I'm still skeptical.
Another issue I have is that XML Owners will provide the XML Data file without
DTD (Implied Schema), so it is up to the recipient to generate a table on
their own for Recordset manipulation. I equate this to having to perform
a DCOM Call without the parameters.

Say for example I was going to send you a bitmap image of something. In
many graphical formats, the first information I may include within the file
is a height and width, or some bounded coordinate data to descibe the image
size. Notice the first items I'm sending are attributes about the file I'm
sending. Kinda like ZMODEM if your from the old school. It's nice that
an XML file contains versioning information so I cound at least make sure
I can parse it. I wish that for so many standards it would always include
a schema standard version name and number. Thank God for the W3C! This
way the XML recipient could validate the incoming data to against a local
schema, or optionally send back some kind of reply stating it's compatible
version or failure. The next action would be for the sender to comply or
send a new schema/DTD.

Simply Put: when a record is sent that does not contain all of the information
of a full record, then the missing tags (Fields) are never defined in your
destination recordset. Next action item: Notify all XML Data owners (in
the real, perfect world) to always provide updated schema to the recipient
prior to sending.

I'm relying on communications to sources such as yourself. Thanks again!
Any further input is greatly appreciated.


"Russell Jones" <arj1@northstate.net> wrote:
>Here's why XML is important--and it has nothing to do with the size of the
>data, the method used to parse it, or the data itself. XML is important
>because it provides a common file format. That's all. Of course, that is
>significant. Consider the problems involved in sending data from one type
of
>database to a company that runs another, different database (not an uncommon
>need, I'm sure). You have to save the data out as some type of delimited
or
>fixed-field ASCII file. Then you have to let the recipient know the data
>types, sizes, defaults, and constraints. The recipient needs to write
>special scripts to import the files. These scripts must be tested and
>debugged--tables need to be built, the rules, and defaults must be
>duplicated--and none of this is automatic.
>
>Now consider the same scenario with XML. The donor saves the table in XML
>format and provides a schema (both of which are automatically generated
by
>the database). The recipient imports the data--all validated against the
>schema, and the database itself generates all the tables, rules, and
>defaults.
>
>If you are the owner of a company, which scenario would seem less costly
and
>error-prone to you?
>
>Sure, XML is an inefficient data format, and doesn't provide high
>performance for large files--but you don't need it to. When you have large
>amounts of data, and you need to get to it very quickly, put it in a
>relational database--they're optimized for that type of data access. But
>remember XML when you need to get the data back in usable form to the people
>who need it.
>
>Here's another reason. The SQL and table schema required to retrieve and
>store hierarchical and irregular data is horrendously complex, not very
>efficient, and highly inflexible. There's a great deal of hierarchical and
>irregular data in the world. Any technology which simplifies working with
>that kind of data is welcome--especially if it turns out to be a standard
>method for working with both regular and irregular data types.
>
>Finally, there's no particular reason why you can't index XML files so that
>they're just as fast as any other kind of data-storage mechanism--in fact,
>there are companies that advertise these capabilities now. So while the
>methods and tools may not be mature yet, give XML the benefit of the many
>years of development that relational databases have had, and it may evolve
>to be a reasonable competitor, even with large data sets.
>
>I hope this helps.
>
>
>"Clarity" <thewarnk@flash.net> wrote in message
>news:3a3e8672$1@news.devx.com...
>>
>> I've been studying XML/schema/SOAP/SAX2 for about 6 months now... I have
>some
>> questions for anyone who cares:
>>
>> Since we all know that you cannot parse a very large file using DOM, you
>> are forced to use SAX... But, then you have to use a query to get to
ONLY
>> the information you want. Then you have to sequentially get to that
>information
>> because there is no "Primary Key" or Indexes that might be built prior
to
>> the execution of the query.
>>
>> My need is this: Get to the Data, Fast, using a Standard query language
>> (Like SQL), that doesn't have to sequentially go through the data, that
>uses
>> indexes for speed, that has a standard for defining heirarchy, types,
and
>> rules.
>> I understand that standard database systems are not easily accessed
>through
>> the 'net, but holy moly. The funny thing about standards is that there
>are
>> so many of them.
>> To me, this is Jurrassic Park Syndrome. We spend so much time trying
to
>> figure out if we CAN do something without spending the time to think about
>> whether we should have.
>> To me, Everyone is using the Buzz word XML. But in real-world situations
>> I have come across, there is not enough support, speed, or
>interoperability
>> to provide a solid solution to Database access without having to be anal
>> retentive about methods, drivers, memory limitations, and conversion
>issues.
>> Maybe I'm crazy, or just ill-informed. You be the judge.
>>
>
>

[Clarity]

Thanks for your reply,

Please see my response to this topic to Russell Jones re: SQL 2000 and it's
XML--->SQL Recordset conversion. I've got Code if your interested! As stated,
I rely on the valuable information of individuals such as yourself. Keep
it coming!

"Brandon Forest" <Brandon_Forest@ZDNetonebox.com> wrote:
>
>Clarity,
>
>I understand your frustration. I have been studying XML for almost a year
>now. I agree with your evaluation on it's shortcomings, but be patient
tools
>are coming. SQL Server 2000 can return recordsets as native XML, so you
>can write stored procedures in SQL and then transport in XML. xPath it
the
>XML query language. It is primitive now, but I expect it to become powerful
>within a couple of interations.
>
>The key reason that XML will become the next big thing, is because it is
>truly cross platform, http protocol, and firewall friendly.
>
>"Clarity" <thewarnk@flash.net> wrote:
>>
>>I've been studying XML/schema/SOAP/SAX2 for about 6 months now... I have
>some
>>questions for anyone who cares:
>>
>>Since we all know that you cannot parse a very large file using DOM, you
>>are forced to use SAX... But, then you have to use a query to get to ONLY
>>the information you want. Then you have to sequentially get to that information
>>because there is no "Primary Key" or Indexes that might be built prior
to
>>the execution of the query.
>>
>>My need is this: Get to the Data, Fast, using a Standard query language
>>(Like SQL), that doesn't have to sequentially go through the data, that
>uses
>>indexes for speed, that has a standard for defining heirarchy, types, and
>>rules.
>>I understand that standard database systems are not easily accessed through
>>the 'net, but holy moly. The funny thing about standards is that there
>are
>>so many of them.
>>To me, this is Jurrassic Park Syndrome. We spend so much time trying to
>>figure out if we CAN do something without spending the time to think about
>>whether we should have.
>>To me, Everyone is using the Buzz word XML. But in real-world situations
>>I have come across, there is not enough support, speed, or interoperability
>>to provide a solid solution to Database access without having to be anal
>>retentive about methods, drivers, memory limitations, and conversion issues.
>>Maybe I'm crazy, or just ill-informed. You be the judge.
>>
>

[Richard]

>>To me, this is Jurrassic Park Syndrome. We spend so much time trying to
figure out if we CAN do something without spending the time to think about
whether we should have.<<

XML doesn't replace SQL and RDBMS. It supplements it. In one sense it is
similar to a MS ADO disconnected recordset that is a generic object than
can be passed among tiers and disparate platforms (try passing an ADO RS
into an EJB method).

When coding for the web, there has been a need for a common disconnected
recordset supported by both Netscape and IE and runnable on Unix, Macs, and
Windows (write once, deploy many). XML is the proposed solution.

There has been a need to have a uniform object type that call directly to
component methods/behaviors without JSP or ASP. This data type must be supported
by many languages and platforms. XML will be that object type.

Many programs use configuration and ini files but there has never been a
set format (outside of text). XML fills that role.

It's a good generic solution that can be overused. But, it's a great start
and I heven't touched on it as a standard to replace EDI X-12 docs or it's
potential as uniform printer script, or the grace in which XSL can transform
XML data, HTML templates, CSS, and script into a well-defined page assembly
framework.

[Mike Wilkes]

Without addressing your specifics, let me say that in general I agree with
you. XSL, for instance, is one of the ugliest things I've seen since Perl.
I don't know why we keep inventing "languages" (and use the term loosely)
that are on the same or lower level that what we already have.

Personally, I'm waiting for a language "above" Visual Basic that allows me
to write business solutions using standard objects without having to code
every class myself.

Given the "state of the art" in Microsoft thinking, however, I'm not holding
my breath...

Michael

[TheAnswer]

The point I think you're missing (or that those you've been studying are missing)
is that XML is NOT a good way to store data... It IS a good way to TRANSFER
data. You use SQL commands through SOAP to ask an SQL server for the data
you want; it uses all it's magical indexes and whatever to find the data,
wrap it up as XML and hand it back to you.

"Clarity" <thewarnk@flash.net> wrote:
>
>I've been studying XML/schema/SOAP/SAX2 for about 6 months now... I have
some
>questions for anyone who cares:
>
>Since we all know that you cannot parse a very large file using DOM, you
>are forced to use SAX... But, then you have to use a query to get to ONLY
>the information you want. Then you have to sequentially get to that information
>because there is no "Primary Key" or Indexes that might be built prior to
>the execution of the query.
>
>My need is this: Get to the Data, Fast, using a Standard query language
>(Like SQL), that doesn't have to sequentially go through the data, that
uses
>indexes for speed, that has a standard for defining heirarchy, types, and
>rules.
>I understand that standard database systems are not easily accessed through
>the 'net, but holy moly. The funny thing about standards is that there
are
>so many of them.
>To me, this is Jurrassic Park Syndrome. We spend so much time trying to
>figure out if we CAN do something without spending the time to think about
>whether we should have.
>To me, Everyone is using the Buzz word XML. But in real-world situations
>I have come across, there is not enough support, speed, or interoperability
>to provide a solid solution to Database access without having to be anal
>retentive about methods, drivers, memory limitations, and conversion issues.
>Maybe I'm crazy, or just ill-informed. You be the judge.
>