maximum size for xml files

Question

whats your rule of thumb for maximum size for xml files.

What if I ask maximum no of records, and one record have 10 values for instance? Another condition is we are

Answer 1

Even though not all parsers read the whole file into memory, if you really need a rule of thumb I would say not bigger than half of your available ram. Anything bigger will likely be way too slow :)

Answer 2

I worked on a project in 2010 where by i had to move a newspaper website from Typo 3 to Drupal 7 and the fastest way around at the time was to export all the content as xml and then parse them into drupal(Xpath). We tried doing it in one go but we had problems at 4Gigs .. So we divided the xmls per year and had each file have less time to parse and the file size in MBs and that went fine. Other reads suggest that the max file may depend on your ram.

Answer 3

There is no limit of XML file size but it takes memory (RAM) as file size of XML file, so long XML file parsing size is performance hit.
It is advised to long XML size using SAX for .NET to parse long XML documents.

Answer 4

I don't think you should have a rule of thumb for maximum size for data, be it XML or anything else. If you need to store multiple gigabytes of data, then you store that data. What makes a difference is what API you use to process that data. However, XML may not be your best bet if your data set is very large. In those cases a relational or XML database will probably work better than a single XML file.

Answer 5

There isn't any. There are maximum sizes for files that depend on the file system you are using, though.

Answer 6

This may be not the thing you want to hear, but... If you're thinking about the size of your XML files, chances are you should use a database instead of files (even if they are not flat files but structured like XML). Databases are highly optimized for efficient storage of huge masses of data. The best algorithms for retrieving data are in the code base of databases.