Best way to store XML data in a MySQL database, with some specific requirements

Question

I am receiving XML data from a service. The test data I am receiving back has about 300 XML nodes, clearly far too many to create individual rows for in a MySQL database.

<

Answer 1

A lot depends on what you want to do with the data - if you want to search for stuff within the XML, then decomposing it into a tree will give much better query performance.

500Mb is not a huge amount of data - the issues are all about how you reference it and search it. If it's just for archiving purposes or you never need to search inside the XML, then compressing it then (e.g.) base64 encoding will reduce this down to less than 80Mb

Answer 2

The best way is to not store XML in the DB, but I have history with that particular issue.

Just store it as TEXT. 500 MB is nothing for MySql, especially with TEXT datatypes, since those aren't stored in the row buffer.

Answer 3

You could create a blob column (i.e. mediumtext column). Instead of inserting XML purely as strings in the DB, you could zip the XML, then store in MySQL.

When you read from MySQL, you unzip it again. Since XML is text you'll get very high compression rates (close to 80% compression). The thought process being, disk IO takes a lot longer time than compression/un-compression which is predominantly Processor bound.

The downside being you will no longer be able to query or do full text search using SQL....

Answer 4

Try to use LOAD XML command. This statement is available in MySQL 5.5.