XML files vs Database


DevX Home    Today's Headlines   Articles Archive   Tip Bank   Forums   

Results 1 to 9 of 9

Thread: XML files vs Database

Hybrid View

  1. #1
    Mansoor Guest

    XML files vs Database


    I have to take a design time decision. We are developing an enterprise application
    based on COM/COM+ that already makes use of SQL Server. Our application supports
    some standard documents that are XML-based and are stored in the form of
    files. Our application requires frequent read/write operations on these documents.
    Now there are two techniques that may be followed:

    One of them is to continue using the physical files as documents, and use
    parsers like SAX, XPATH, DOM etc to manipulate the documents. The drawbacks
    would be very slow read/write operations, large memory consumption, and concurrent
    accesses to these files to support multiple users at a single time would
    be cumbersome to manage. I cannot think of any advantage except that the
    data would exist physically separate from each other in separate files.

    The other alternative is to map the hierarchy of our xml document to a hierarchy
    of tables in our database. The data from the xml files can be parsed and
    dumped into those tables in database at the time when the file is imported
    into the system, any subsequent read/write operation would now require simple
    queries on the database. Anytime a user demands a physical file, it can be
    built on the fly from the stored state in the database. The database option
    would also allow to utilize the inherent transaction management support which
    will solve the problems of concurrent accesses to the documents. This would
    also result in simpler logic and less complex, efficient and traditional
    code, which may have a significant effect over the overall performance of
    our application. The drawbacks, as feared by some of my teammates, would
    likely be increased size of the database(which might be coz of redundant
    data), speed(Well, I strictly oppose this argument, since I believe that
    speed of the operations on the database would still be faster than manipulating
    the physical files themselves, since commercial databases are highly optimized
    form of data storage).

    I am not here to argue over the use of database or xml files as a means of
    document storage in an enterprise application, even though I am in favor
    of going with the 2nd approach. I have to see things from cost and benefit
    point of view, since we have limited resources, and we cannot go for adventures
    like writing a customized database system of our own using xml files as the
    storage mechanism, and I want to seek a solution which is optimized and efficient
    enough to be incorporated in our enterprise-level application and at the
    same time requiring less time and resources. Well these are my points of
    view, I might be missing something or may be wrong altogether. I need some
    input from you guys who have been though this kind of situation before, and
    what you prefer would be an ideal solution in this situation. Your help will
    be appreciated.

    Thanking you
    Mansoor Ali Khan

  2. #2
    Mansoor Guest

    Re: XML files vs Database



    Let me reelaborate my stand. Actually, as i have mentioned, our application
    operates on data which is stored in the form of a xml-based standard called
    PDX(Product Data Exchange). PDX packages come in the form of xml files, and
    currently our application supports importing of these files into our system
    from other tools/applications, thus we CANNOT avoid use of xml files altogether.
    But we do have the option to use database for representing this data for
    internal usage to avoid compexity and redundancy in our code as well as to
    improve performance of our system, as it is a very critical factor for any
    enterprise-level application. And any time we need to export a PDX package,
    we can do it instantly by building an xml file from the state stored in database.
    My question is that what are the advantages/disadvantages of each of the
    approach I had mentioned earlier.

    "Mansoor" <mansooralikhan78@hotmail.com> wrote:
    >
    >I have to take a design time decision. We are developing an enterprise application
    >based on COM/COM+ that already makes use of SQL Server. Our application

    supports
    >some standard documents that are XML-based and are stored in the form of
    >files. Our application requires frequent read/write operations on these

    documents.
    >Now there are two techniques that may be followed:
    >
    >One of them is to continue using the physical files as documents, and use
    >parsers like SAX, XPATH, DOM etc to manipulate the documents. The drawbacks
    >would be very slow read/write operations, large memory consumption, and

    concurrent
    >accesses to these files to support multiple users at a single time would
    >be cumbersome to manage. I cannot think of any advantage except that the
    >data would exist physically separate from each other in separate files.
    >
    >The other alternative is to map the hierarchy of our xml document to a hierarchy
    >of tables in our database. The data from the xml files can be parsed and
    >dumped into those tables in database at the time when the file is imported
    >into the system, any subsequent read/write operation would now require simple
    >queries on the database. Anytime a user demands a physical file, it can

    be
    >built on the fly from the stored state in the database. The database option
    >would also allow to utilize the inherent transaction management support

    which
    >will solve the problems of concurrent accesses to the documents. This would
    >also result in simpler logic and less complex, efficient and traditional
    >code, which may have a significant effect over the overall performance of
    >our application. The drawbacks, as feared by some of my teammates, would
    >likely be increased size of the database(which might be coz of redundant
    >data), speed(Well, I strictly oppose this argument, since I believe that
    >speed of the operations on the database would still be faster than manipulating
    >the physical files themselves, since commercial databases are highly optimized
    >form of data storage).
    >
    >I am not here to argue over the use of database or xml files as a means

    of
    >document storage in an enterprise application, even though I am in favor
    >of going with the 2nd approach. I have to see things from cost and benefit
    >point of view, since we have limited resources, and we cannot go for adventures
    >like writing a customized database system of our own using xml files as

    the
    >storage mechanism, and I want to seek a solution which is optimized and

    efficient
    >enough to be incorporated in our enterprise-level application and at the
    >same time requiring less time and resources. Well these are my points of
    >view, I might be missing something or may be wrong altogether. I need some
    >input from you guys who have been though this kind of situation before,

    and
    >what you prefer would be an ideal solution in this situation. Your help

    will
    >be appreciated.
    >
    >Thanking you
    >Mansoor Ali Khan



  3. #3
    Kirk Allen Evans Guest

    Re: XML files vs Database

    You can easily use the SQLXML provider [1] to insert the entire contents of
    the XML document into SQL Server using one of several approaches
    (Updategrams or Diffgrams). If you expose the Updategram or Diffgram
    directly through a SQL Server virtual directory tempate, you might also be
    able to convince the client to directly post the XML to your IIS server for
    processing (long shot, but it would put the burden of shredding the document
    on them instead of you).

    If you don't want to use Diffgrams or Updategrams, then you can also use the
    OPENXML keyword in SQL Server to shred the contents of the XML file into the
    database, shoving the data from the XML file into respective relational
    tables (no need to design hierarchically). Finally, you can use SQL
    Server's XML Bulk Load to load the contents of the XML file into SQL Server.

    Now that the data is in SQL Server and stored relationally, you can
    manipulate the data using traditional transactional designs. The database
    takes care of concurrency for you, it is just up to you to design and
    performance test the transactional logic and test for lock conditions.

    When someone wants to retrieve data as XML instead of as a relational
    rowset, you can use the FOR XML keyword to retrieve results from SQL Server.
    You can use FOR XML with ADO fairly easily by using the Stream object:
    there are several examples in books online. Again, this also gives you the
    capability of exposing your SQL Server over HTTP using templates because you
    can use FOR XML queries over the web: the end user simply thinks they are
    retrieving an XML document, where you are really using that XML document to
    query SQL Server.

    FWIW, I cover many of these techniques in my book, "XML and ASP.NET." I
    also have several samples using these techniques that I will be posting over
    the next week to my website, www.xmlandasp.net.

    --
    Kirk Allen Evans
    http://www.xmlandasp.net
    "XML and ASP.NET", New Riders Publishing
    http://www.amazon.com/exec/obidos/ASIN/073571200X


    "Mansoor" <mansooralikhan78@hotmail.com> wrote in message
    news:3d578163$1@10.1.10.29...
    >
    > I have to take a design time decision. We are developing an enterprise

    application
    > based on COM/COM+ that already makes use of SQL Server. Our application

    supports
    > some standard documents that are XML-based and are stored in the form of
    > files. Our application requires frequent read/write operations on these

    documents.
    > Now there are two techniques that may be followed:
    >
    > One of them is to continue using the physical files as documents, and use
    > parsers like SAX, XPATH, DOM etc to manipulate the documents. The

    drawbacks
    > would be very slow read/write operations, large memory consumption, and

    concurrent
    > accesses to these files to support multiple users at a single time would
    > be cumbersome to manage. I cannot think of any advantage except that the
    > data would exist physically separate from each other in separate files.
    >
    > The other alternative is to map the hierarchy of our xml document to a

    hierarchy
    > of tables in our database. The data from the xml files can be parsed and
    > dumped into those tables in database at the time when the file is imported
    > into the system, any subsequent read/write operation would now require

    simple
    > queries on the database. Anytime a user demands a physical file, it can be
    > built on the fly from the stored state in the database. The database

    option
    > would also allow to utilize the inherent transaction management support

    which
    > will solve the problems of concurrent accesses to the documents. This

    would
    > also result in simpler logic and less complex, efficient and traditional
    > code, which may have a significant effect over the overall performance of
    > our application. The drawbacks, as feared by some of my teammates, would
    > likely be increased size of the database(which might be coz of redundant
    > data), speed(Well, I strictly oppose this argument, since I believe that
    > speed of the operations on the database would still be faster than

    manipulating
    > the physical files themselves, since commercial databases are highly

    optimized
    > form of data storage).
    >
    > I am not here to argue over the use of database or xml files as a means of
    > document storage in an enterprise application, even though I am in favor
    > of going with the 2nd approach. I have to see things from cost and benefit
    > point of view, since we have limited resources, and we cannot go for

    adventures
    > like writing a customized database system of our own using xml files as

    the
    > storage mechanism, and I want to seek a solution which is optimized and

    efficient
    > enough to be incorporated in our enterprise-level application and at the
    > same time requiring less time and resources. Well these are my points of
    > view, I might be missing something or may be wrong altogether. I need some
    > input from you guys who have been though this kind of situation before,

    and
    > what you prefer would be an ideal solution in this situation. Your help

    will
    > be appreciated.
    >
    > Thanking you
    > Mansoor Ali Khan




  4. #4
    Jim Slade Guest

    Re: XML files vs Database


    Mansoor-
    How big are your PDX files?
    Do you need granular access to the information in the files, or is your application
    accessing the entire document every time? Is the schema fixed, or can it
    vary?
    If the documents are big or will get bigger, and you need performance (granular
    access) AND extensibility, you might want to look at using a native XML datastore
    alongside (on top) of SQL server. You expressed the tradeoffs eloquently,
    and if the future of your applicatoin is with PDX, you will continue to struggle
    with the tradeoffs you described. Relational mapping can get verbose and
    heavy (and does not lend itself to extensibility), and the overhead of just
    using the files can grind performance to a halt as documents grow, not to
    mention loss of all the traditional database features.
    Jim

    "Mansoor" <mansooralikhan78@hotmail.com> wrote:
    >
    >
    >Let me reelaborate my stand. Actually, as i have mentioned, our application
    >operates on data which is stored in the form of a xml-based standard called
    >PDX(Product Data Exchange). PDX packages come in the form of xml files,

    and
    >currently our application supports importing of these files into our system
    >from other tools/applications, thus we CANNOT avoid use of xml files altogether.
    >But we do have the option to use database for representing this data for
    >internal usage to avoid compexity and redundancy in our code as well as

    to
    >improve performance of our system, as it is a very critical factor for any
    >enterprise-level application. And any time we need to export a PDX package,
    >we can do it instantly by building an xml file from the state stored in

    database.
    >My question is that what are the advantages/disadvantages of each of the
    >approach I had mentioned earlier.
    >
    >"Mansoor" <mansooralikhan78@hotmail.com> wrote:
    >>
    >>I have to take a design time decision. We are developing an enterprise

    application
    >>based on COM/COM+ that already makes use of SQL Server. Our application

    >supports
    >>some standard documents that are XML-based and are stored in the form of
    >>files. Our application requires frequent read/write operations on these

    >documents.
    >>Now there are two techniques that may be followed:
    >>
    >>One of them is to continue using the physical files as documents, and use
    >>parsers like SAX, XPATH, DOM etc to manipulate the documents. The drawbacks
    >>would be very slow read/write operations, large memory consumption, and

    >concurrent
    >>accesses to these files to support multiple users at a single time would
    >>be cumbersome to manage. I cannot think of any advantage except that the
    >>data would exist physically separate from each other in separate files.
    >>
    >>The other alternative is to map the hierarchy of our xml document to a

    hierarchy
    >>of tables in our database. The data from the xml files can be parsed and
    >>dumped into those tables in database at the time when the file is imported
    >>into the system, any subsequent read/write operation would now require

    simple
    >>queries on the database. Anytime a user demands a physical file, it can

    >be
    >>built on the fly from the stored state in the database. The database option
    >>would also allow to utilize the inherent transaction management support

    >which
    >>will solve the problems of concurrent accesses to the documents. This would
    >>also result in simpler logic and less complex, efficient and traditional
    >>code, which may have a significant effect over the overall performance

    of
    >>our application. The drawbacks, as feared by some of my teammates, would
    >>likely be increased size of the database(which might be coz of redundant
    >>data), speed(Well, I strictly oppose this argument, since I believe that
    >>speed of the operations on the database would still be faster than manipulating
    >>the physical files themselves, since commercial databases are highly optimized
    >>form of data storage).
    >>
    >>I am not here to argue over the use of database or xml files as a means

    >of
    >>document storage in an enterprise application, even though I am in favor
    >>of going with the 2nd approach. I have to see things from cost and benefit
    >>point of view, since we have limited resources, and we cannot go for adventures
    >>like writing a customized database system of our own using xml files as

    >the
    >>storage mechanism, and I want to seek a solution which is optimized and

    >efficient
    >>enough to be incorporated in our enterprise-level application and at the
    >>same time requiring less time and resources. Well these are my points of
    >>view, I might be missing something or may be wrong altogether. I need some
    >>input from you guys who have been though this kind of situation before,

    >and
    >>what you prefer would be an ideal solution in this situation. Your help

    >will
    >>be appreciated.
    >>
    >>Thanking you
    >>Mansoor Ali Khan

    >



  5. #5
    Vijay Gummadi Guest

    Re: XML files vs Database


    Folks,

    An approach that is working well for us is storing the XML as text in
    SQL Server and retreiving it as required. We didn't want to deal with XML
    files (management, I/O and such) or introduce another database that our client
    may have to purchase/license/install. In addition, we wanted to avoid the
    overhead of parsing the XML to convert to store in a relational system and
    vice versa.

    Other benefits include not having to manage User based locking and transaction
    management since all of these are supported by the database.

    Any comments on this approach?

    Vijay

    "Kirk Allen Evans" <nntp@xmlandasp.net> wrote:
    >You can easily use the SQLXML provider [1] to insert the entire contents

    of
    >the XML document into SQL Server using one of several approaches
    >(Updategrams or Diffgrams). If you expose the Updategram or Diffgram
    >directly through a SQL Server virtual directory tempate, you might also

    be
    >able to convince the client to directly post the XML to your IIS server

    for
    >processing (long shot, but it would put the burden of shredding the document
    >on them instead of you).
    >
    >If you don't want to use Diffgrams or Updategrams, then you can also use

    the
    >OPENXML keyword in SQL Server to shred the contents of the XML file into

    the
    >database, shoving the data from the XML file into respective relational
    >tables (no need to design hierarchically). Finally, you can use SQL
    >Server's XML Bulk Load to load the contents of the XML file into SQL Server.
    >
    >Now that the data is in SQL Server and stored relationally, you can
    >manipulate the data using traditional transactional designs. The database
    >takes care of concurrency for you, it is just up to you to design and
    >performance test the transactional logic and test for lock conditions.
    >
    >When someone wants to retrieve data as XML instead of as a relational
    >rowset, you can use the FOR XML keyword to retrieve results from SQL Server.
    >You can use FOR XML with ADO fairly easily by using the Stream object:
    >there are several examples in books online. Again, this also gives you

    the
    >capability of exposing your SQL Server over HTTP using templates because

    you
    >can use FOR XML queries over the web: the end user simply thinks they are
    >retrieving an XML document, where you are really using that XML document

    to
    >query SQL Server.
    >
    >FWIW, I cover many of these techniques in my book, "XML and ASP.NET." I
    >also have several samples using these techniques that I will be posting

    over
    >the next week to my website, www.xmlandasp.net.
    >
    >--
    >Kirk Allen Evans
    >http://www.xmlandasp.net
    >"XML and ASP.NET", New Riders Publishing
    > http://www.amazon.com/exec/obidos/ASIN/073571200X
    >
    >
    >"Mansoor" <mansooralikhan78@hotmail.com> wrote in message
    >news:3d578163$1@10.1.10.29...
    >>
    >> I have to take a design time decision. We are developing an enterprise

    >application
    >> based on COM/COM+ that already makes use of SQL Server. Our application

    >supports
    >> some standard documents that are XML-based and are stored in the form

    of
    >> files. Our application requires frequent read/write operations on these

    >documents.
    >> Now there are two techniques that may be followed:
    >>
    >> One of them is to continue using the physical files as documents, and

    use
    >> parsers like SAX, XPATH, DOM etc to manipulate the documents. The

    >drawbacks
    >> would be very slow read/write operations, large memory consumption, and

    >concurrent
    >> accesses to these files to support multiple users at a single time would
    >> be cumbersome to manage. I cannot think of any advantage except that the
    >> data would exist physically separate from each other in separate files.
    >>
    >> The other alternative is to map the hierarchy of our xml document to a

    >hierarchy
    >> of tables in our database. The data from the xml files can be parsed and
    >> dumped into those tables in database at the time when the file is imported
    >> into the system, any subsequent read/write operation would now require

    >simple
    >> queries on the database. Anytime a user demands a physical file, it can

    be
    >> built on the fly from the stored state in the database. The database

    >option
    >> would also allow to utilize the inherent transaction management support

    >which
    >> will solve the problems of concurrent accesses to the documents. This

    >would
    >> also result in simpler logic and less complex, efficient and traditional
    >> code, which may have a significant effect over the overall performance

    of
    >> our application. The drawbacks, as feared by some of my teammates, would
    >> likely be increased size of the database(which might be coz of redundant
    >> data), speed(Well, I strictly oppose this argument, since I believe that
    >> speed of the operations on the database would still be faster than

    >manipulating
    >> the physical files themselves, since commercial databases are highly

    >optimized
    >> form of data storage).
    >>
    >> I am not here to argue over the use of database or xml files as a means

    of
    >> document storage in an enterprise application, even though I am in favor
    >> of going with the 2nd approach. I have to see things from cost and benefit
    >> point of view, since we have limited resources, and we cannot go for

    >adventures
    >> like writing a customized database system of our own using xml files as

    >the
    >> storage mechanism, and I want to seek a solution which is optimized and

    >efficient
    >> enough to be incorporated in our enterprise-level application and at the
    >> same time requiring less time and resources. Well these are my points

    of
    >> view, I might be missing something or may be wrong altogether. I need

    some
    >> input from you guys who have been though this kind of situation before,

    >and
    >> what you prefer would be an ideal solution in this situation. Your help

    >will
    >> be appreciated.
    >>
    >> Thanking you
    >> Mansoor Ali Khan

    >
    >



  6. #6
    Merijn Boom Guest

    Re: XML files vs Database

    For me i't sounds a bit double, because why would you store xml (which is
    OO) in a database which is OO as well, I mean XML is great as a transport
    layer between applications but isn't it wiser to store data as flat as
    possible (performance).

    Another thing is that with this approach you mention you don't properly
    design your database (because you propably use text fields for storing data
    so no guaranties about the data which is stored (you do not make a diffrence
    between integers, currencies, data etc etc)

    My approach for this problem would be creating the database, using stored
    procedures and commands which transfer the xml data (whcih can be formed
    whith stylesheets and templates) to the application you want to use this
    data in.

    With this approach you'll be able to create difrences between data format's,
    you have protection of your database and you use xml which is a beautifull
    "transport language"

    Good luck,


    --
    Merijn Boom
    Senior Developer
    Con-X-Com
    Connecting Computers


    "Vijay Gummadi" <vgummadi@wayne.edu> schreef in bericht
    news:3d5c2743$1@10.1.10.29...
    >
    > Folks,
    >
    > An approach that is working well for us is storing the XML as text in
    > SQL Server and retreiving it as required. We didn't want to deal with XML
    > files (management, I/O and such) or introduce another database that our

    client
    > may have to purchase/license/install. In addition, we wanted to avoid the
    > overhead of parsing the XML to convert to store in a relational system and
    > vice versa.
    >
    > Other benefits include not having to manage User based locking and

    transaction
    > management since all of these are supported by the database.
    >
    > Any comments on this approach?
    >
    > Vijay
    >
    > "Kirk Allen Evans" <nntp@xmlandasp.net> wrote:
    > >You can easily use the SQLXML provider [1] to insert the entire contents

    > of
    > >the XML document into SQL Server using one of several approaches
    > >(Updategrams or Diffgrams). If you expose the Updategram or Diffgram
    > >directly through a SQL Server virtual directory tempate, you might also

    > be
    > >able to convince the client to directly post the XML to your IIS server

    > for
    > >processing (long shot, but it would put the burden of shredding the

    document
    > >on them instead of you).
    > >
    > >If you don't want to use Diffgrams or Updategrams, then you can also use

    > the
    > >OPENXML keyword in SQL Server to shred the contents of the XML file into

    > the
    > >database, shoving the data from the XML file into respective relational
    > >tables (no need to design hierarchically). Finally, you can use SQL
    > >Server's XML Bulk Load to load the contents of the XML file into SQL

    Server.
    > >
    > >Now that the data is in SQL Server and stored relationally, you can
    > >manipulate the data using traditional transactional designs. The

    database
    > >takes care of concurrency for you, it is just up to you to design and
    > >performance test the transactional logic and test for lock conditions.
    > >
    > >When someone wants to retrieve data as XML instead of as a relational
    > >rowset, you can use the FOR XML keyword to retrieve results from SQL

    Server.
    > >You can use FOR XML with ADO fairly easily by using the Stream object:
    > >there are several examples in books online. Again, this also gives you

    > the
    > >capability of exposing your SQL Server over HTTP using templates because

    > you
    > >can use FOR XML queries over the web: the end user simply thinks they

    are
    > >retrieving an XML document, where you are really using that XML document

    > to
    > >query SQL Server.
    > >
    > >FWIW, I cover many of these techniques in my book, "XML and ASP.NET." I
    > >also have several samples using these techniques that I will be posting

    > over
    > >the next week to my website, www.xmlandasp.net.
    > >
    > >--
    > >Kirk Allen Evans
    > >http://www.xmlandasp.net
    > >"XML and ASP.NET", New Riders Publishing
    > > http://www.amazon.com/exec/obidos/ASIN/073571200X
    > >
    > >
    > >"Mansoor" <mansooralikhan78@hotmail.com> wrote in message
    > >news:3d578163$1@10.1.10.29...
    > >>
    > >> I have to take a design time decision. We are developing an enterprise

    > >application
    > >> based on COM/COM+ that already makes use of SQL Server. Our application

    > >supports
    > >> some standard documents that are XML-based and are stored in the form

    > of
    > >> files. Our application requires frequent read/write operations on these

    > >documents.
    > >> Now there are two techniques that may be followed:
    > >>
    > >> One of them is to continue using the physical files as documents, and

    > use
    > >> parsers like SAX, XPATH, DOM etc to manipulate the documents. The

    > >drawbacks
    > >> would be very slow read/write operations, large memory consumption, and

    > >concurrent
    > >> accesses to these files to support multiple users at a single time

    would
    > >> be cumbersome to manage. I cannot think of any advantage except that

    the
    > >> data would exist physically separate from each other in separate files.
    > >>
    > >> The other alternative is to map the hierarchy of our xml document to a

    > >hierarchy
    > >> of tables in our database. The data from the xml files can be parsed

    and
    > >> dumped into those tables in database at the time when the file is

    imported
    > >> into the system, any subsequent read/write operation would now require

    > >simple
    > >> queries on the database. Anytime a user demands a physical file, it can

    > be
    > >> built on the fly from the stored state in the database. The database

    > >option
    > >> would also allow to utilize the inherent transaction management support

    > >which
    > >> will solve the problems of concurrent accesses to the documents. This

    > >would
    > >> also result in simpler logic and less complex, efficient and

    traditional
    > >> code, which may have a significant effect over the overall performance

    > of
    > >> our application. The drawbacks, as feared by some of my teammates,

    would
    > >> likely be increased size of the database(which might be coz of

    redundant
    > >> data), speed(Well, I strictly oppose this argument, since I believe

    that
    > >> speed of the operations on the database would still be faster than

    > >manipulating
    > >> the physical files themselves, since commercial databases are highly

    > >optimized
    > >> form of data storage).
    > >>
    > >> I am not here to argue over the use of database or xml files as a means

    > of
    > >> document storage in an enterprise application, even though I am in

    favor
    > >> of going with the 2nd approach. I have to see things from cost and

    benefit
    > >> point of view, since we have limited resources, and we cannot go for

    > >adventures
    > >> like writing a customized database system of our own using xml files as

    > >the
    > >> storage mechanism, and I want to seek a solution which is optimized and

    > >efficient
    > >> enough to be incorporated in our enterprise-level application and at

    the
    > >> same time requiring less time and resources. Well these are my points

    > of
    > >> view, I might be missing something or may be wrong altogether. I need

    > some
    > >> input from you guys who have been though this kind of situation before,

    > >and
    > >> what you prefer would be an ideal solution in this situation. Your help

    > >will
    > >> be appreciated.
    > >>
    > >> Thanking you
    > >> Mansoor Ali Khan

    > >
    > >

    >




  7. #7
    Carl Gregory Guest

    Re: XML files vs Database


    Having had a similar design decision myself, I appreciate your dillema!
    In the end, our decision (to store the data in SQL server, and extract XML
    when needed) was quite heavily based on the skillset within our organisation
    (loads of database experience, not much XML experience).
    As well as that though, we did look at the pros and cons of each solution.
    The major plus of using the database, and extracting XML as needed (currently
    by use of FOR XML queries) was the inherent flexibilty, we can produce XML,
    but we can also allow 'traditional' client server access to the data, produce
    reports/statistics etc and have efficient searching.
    Storage requirements in a well designed relational database are also considerably
    less, but this is not really an issue, disk space is cheap!
    I'd be interested if anyone else has gone the other way, and used an XML
    document repository approach (Or the middle way, and used a native XML database)and
    what their experiences are.

    Carl

  8. #8
    Merijn Boom Guest

    Re: XML files vs Database

    Well the reason why I sudgested the way as described above was that I did a
    project with TeamSite (a content management system) where we had to keep up
    to a 1000000 html pages for the EMEA region.

    TeamSite uses xml in diffrent ways but as well to store this data. The
    problem is that storing and processing these amount of files (especially
    over the internet) and working with hudge files (not the amount but the
    size) it gives you an awfull lot of problems.

    We couldn't search properly through xml based files because the file system
    is quite slow (compared to a well designed relational database), so what we
    desided to do was to use the xml as a tranporting layer (this is the way
    Microsoft intend to do with .Net, besides a lot of other things as well),
    you can see xml in that way a bit like the middle tier protocol.

    For this project it worked fine. For other projects we sometimes use xml
    files to store data but we always make the decision and warn our clients
    that the amount of data often requires a database, instead of an OO file
    system.

    Hope it gave you a bit more the impression,

    Regards,

    Merijn Boom
    "Carl Gregory" <carl.gregory@hants.gov.uk> schreef in bericht
    news:3d65eed7$1@10.1.10.29...
    >
    > Having had a similar design decision myself, I appreciate your dillema!
    > In the end, our decision (to store the data in SQL server, and extract XML
    > when needed) was quite heavily based on the skillset within our

    organisation
    > (loads of database experience, not much XML experience).
    > As well as that though, we did look at the pros and cons of each solution.
    > The major plus of using the database, and extracting XML as needed

    (currently
    > by use of FOR XML queries) was the inherent flexibilty, we can produce

    XML,
    > but we can also allow 'traditional' client server access to the data,

    produce
    > reports/statistics etc and have efficient searching.
    > Storage requirements in a well designed relational database are also

    considerably
    > less, but this is not really an issue, disk space is cheap!
    > I'd be interested if anyone else has gone the other way, and used an XML
    > document repository approach (Or the middle way, and used a native XML

    database)and
    > what their experiences are.
    >
    > Carl




  9. #9
    Guest

    Re: XML files vs Database


    Mansoor,
    Have you thought of using a varchar string to store the doc in sql server
    rows (max size 8064 bytes). If the docs are huge then break them up into
    smaller rows.

    Padraic

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
HTML5 Development Center
 
 
FAQ
Latest Articles
Java
.NET
XML
Database
Enterprise
Questions? Contact us.
C++
Web Development
Wireless
Latest Tips
Open Source


   Development Centers

   -- Android Development Center
   -- Cloud Development Project Center
   -- HTML5 Development Center
   -- Windows Mobile Development Center