XML files vs Database

I have to take a design time decision. We are developing an enterprise application
based on COM/COM+ that already makes use of SQL Server. Our application supports
some standard documents that are XML-based and are stored in the form of
files. Our application requires frequent read/write operations on these documents.
Now there are two techniques that may be followed:

One of them is to continue using the physical files as documents, and use
parsers like SAX, XPATH, DOM etc to manipulate the documents. The drawbacks
would be very slow read/write operations, large memory consumption, and concurrent
accesses to these files to support multiple users at a single time would
be cumbersome to manage. I cannot think of any advantage except that the
data would exist physically separate from each other in separate files.

The other alternative is to map the hierarchy of our xml document to a hierarchy
of tables in our database. The data from the xml files can be parsed and
dumped into those tables in database at the time when the file is imported
into the system, any subsequent read/write operation would now require simple
queries on the database. Anytime a user demands a physical file, it can be
built on the fly from the stored state in the database. The database option
would also allow to utilize the inherent transaction management support which
will solve the problems of concurrent accesses to the documents. This would
also result in simpler logic and less complex, efficient and traditional
code, which may have a significant effect over the overall performance of
our application. The drawbacks, as feared by some of my teammates, would
likely be increased size of the database(which might be coz of redundant
data), speed(Well, I strictly oppose this argument, since I believe that
speed of the operations on the database would still be faster than manipulating
the physical files themselves, since commercial databases are highly optimized
form of data storage).

I am not here to argue over the use of database or xml files as a means of
document storage in an enterprise application, even though I am in favor
of going with the 2nd approach. I have to see things from cost and benefit
point of view, since we have limited resources, and we cannot go for adventures
like writing a customized database system of our own using xml files as the
storage mechanism, and I want to seek a solution which is optimized and efficient
enough to be incorporated in our enterprise-level application and at the
same time requiring less time and resources. Well these are my points of
view, I might be missing something or may be wrong altogether. I need some
input from you guys who have been though this kind of situation before, and
what you prefer would be an ideal solution in this situation. Your help will
be appreciated.

Thanking you
Mansoor Ali Khan

Let me reelaborate my stand. Actually, as i have mentioned, our application
operates on data which is stored in the form of a xml-based standard called
PDX(Product Data Exchange). PDX packages come in the form of xml files, and
currently our application supports importing of these files into our system
from other tools/applications, thus we CANNOT avoid use of xml files altogether.
But we do have the option to use database for representing this data for
internal usage to avoid compexity and redundancy in our code as well as to
improve performance of our system, as it is a very critical factor for any
enterprise-level application. And any time we need to export a PDX package,
we can do it instantly by building an xml file from the state stored in database.
My question is that what are the advantages/disadvantages of each of the
approach I had mentioned earlier.

"Mansoor" <mansooralikhan78@hotmail.com> wrote:
>
>I have to take a design time decision. We are developing an enterprise application
>based on COM/COM+ that already makes use of SQL Server. Our application
supports
>some standard documents that are XML-based and are stored in the form of
>files. Our application requires frequent read/write operations on these
documents.
>Now there are two techniques that may be followed:
>
>One of them is to continue using the physical files as documents, and use
>parsers like SAX, XPATH, DOM etc to manipulate the documents. The drawbacks
>would be very slow read/write operations, large memory consumption, and
concurrent
>accesses to these files to support multiple users at a single time would
>be cumbersome to manage. I cannot think of any advantage except that the
>data would exist physically separate from each other in separate files.
>
>The other alternative is to map the hierarchy of our xml document to a hierarchy
>of tables in our database. The data from the xml files can be parsed and
>dumped into those tables in database at the time when the file is imported
>into the system, any subsequent read/write operation would now require simple
>queries on the database. Anytime a user demands a physical file, it can
be
>built on the fly from the stored state in the database. The database option
>would also allow to utilize the inherent transaction management support
which
>will solve the problems of concurrent accesses to the documents. This would
>also result in simpler logic and less complex, efficient and traditional
>code, which may have a significant effect over the overall performance of
>our application. The drawbacks, as feared by some of my teammates, would
>likely be increased size of the database(which might be coz of redundant
>data), speed(Well, I strictly oppose this argument, since I believe that
>speed of the operations on the database would still be faster than manipulating
>the physical files themselves, since commercial databases are highly optimized
>form of data storage).
>
>I am not here to argue over the use of database or xml files as a means
of
>document storage in an enterprise application, even though I am in favor
>of going with the 2nd approach. I have to see things from cost and benefit
>point of view, since we have limited resources, and we cannot go for adventures
>like writing a customized database system of our own using xml files as
the
>storage mechanism, and I want to seek a solution which is optimized and
efficient
>enough to be incorporated in our enterprise-level application and at the
>same time requiring less time and resources. Well these are my points of
>view, I might be missing something or may be wrong altogether. I need some
>input from you guys who have been though this kind of situation before,
and
>what you prefer would be an ideal solution in this situation. Your help
will
>be appreciated.
>
>Thanking you
>Mansoor Ali Khan

"Mansoor" <mansooralikhan78@hotmail.com> wrote:
>
>I have to take a design time decision. We are developing an enterprise application
>based on COM/COM+ that already makes use of SQL Server. Our application
supports
>some standard documents that are XML-based and are stored in the form of
>files. Our application requires frequent read/write operations on these
documents.
>Now there are two techniques that may be followed:
>
>One of them is to continue using the physical files as documents, and use
>parsers like SAX, XPATH, DOM etc to manipulate the documents. The drawbacks
>would be very slow read/write operations, large memory consumption, and
concurrent
>accesses to these files to support multiple users at a single time would
>be cumbersome to manage. I cannot think of any advantage except that the
>data would exist physically separate from each other in separate files.
>
>The other alternative is to map the hierarchy of our xml document to a hierarchy
>of tables in our database. The data from the xml files can be parsed and
>dumped into those tables in database at the time when the file is imported
>into the system, any subsequent read/write operation would now require simple
>queries on the database. Anytime a user demands a physical file, it can
be
>built on the fly from the stored state in the database. The database option
>would also allow to utilize the inherent transaction management support
which
>will solve the problems of concurrent accesses to the documents. This would
>also result in simpler logic and less complex, efficient and traditional
>code, which may have a significant effect over the overall performance of
>our application. The drawbacks, as feared by some of my teammates, would
>likely be increased size of the database(which might be coz of redundant
>data), speed(Well, I strictly oppose this argument, since I believe that
>speed of the operations on the database would still be faster than manipulating
>the physical files themselves, since commercial databases are highly optimized
>form of data storage).
>
>I am not here to argue over the use of database or xml files as a means
of
>document storage in an enterprise application, even though I am in favor
>of going with the 2nd approach. I have to see things from cost and benefit
>point of view, since we have limited resources, and we cannot go for adventures
>like writing a customized database system of our own using xml files as
the
>storage mechanism, and I want to seek a solution which is optimized and
efficient
>enough to be incorporated in our enterprise-level application and at the
>same time requiring less time and resources. Well these are my points of
>view, I might be missing something or may be wrong altogether. I need some
>input from you guys who have been though this kind of situation before,
and
>what you prefer would be an ideal solution in this situation. Your help
will
>be appreciated.
>
>Thanking you
>Mansoor Ali Khan