Flat file vs database - speed?

问题

I'm making a chat program, and I need a place to store messages. The client would contact the server every x seconds with the last received message id, and the server would find all messages with a id higher than that, in the rooms that the client has joined.

As I'm not going to store things forever, I'm thinking of using flat files (one per room, as well as direct messages) with only the last 40 or so messages. However I think with comparing numbers a database would be faster.

What method of data storage should I use?

回答1:

The flat file may be a bit faster, but it will end up being more buggy in the long run, because instead of just doing a SELECT * FROM messages WHERE room=nnn AND ID > yyy, you will have to load the file, parse it, scan each row for the message ID, go to the right message, and then read it out.

That's only the first problem. A text file doesn't support writing by multiple users (what if two people are posting to the same room at the same time?), and can easily become corrupt.

All things considered, I'd say it's better across the board to use a DB, even if it's something simple like SQLite, which has excellent PHP support. However, consider the many-users condition, MySQL is probably a much better choice. Also, MySQL has excellent caching capabilities, so most of the time, the recent data will come directly from RAM, and will be served faster than you could scan a text file in PHP either way.

回答2:

Seems you don't have to keep your chat history at all, so why are you considering to use a permanent data storing(database or flat-file) method? You can do this using in memory caching software like Memcashed which faster than database or flat-file.

回答3:

In general, a database would be far faster, since an index would let you go directly to the records the server has to send.

OTOH, with just 40 or so messages, it's quite likely that would fit in RAM, and with so few records, even the simplest linear-search routine would be far faster than a single HD access.

Still, using a database is so much easier, I'd use it for simplicity and not for speed. Also, if you have a lot of simultaneous rooms, writing it all yourself means a lot of opportunities for small, trivial bugs that delay development unnecessarily.

Just go with a database.

回答4:

Ill throw my hat in the ring here even though this is an old question. For speed, reliability, and ease of use a DB is the obvious easy choice... With one major caveat that a lot of people are overlooking, and that is most shared hosts (the most common form of web hosting) only allow 15 or so connections at once, even VPS's usually only allow 100-200, dedicated being 500 or more. What that means is if you have (n) users pooling every (s) seconds those connections are going to get eaten up fast, even faster if you are also running any sort of CMS. Being in the middle of developing my own chat room code on a VPS I am also facing these issues myself.

So far my method has been this.

• Make sure to pass a lastMessageReceived variable to limit the response.
• If a public chatroom pass a timestamp filter as well as the above
• if at all possible use a DB caching engine such as MySQLnd with query caching enabled and TTL set to whatever your pooling rate is.
• Don't get crazy with your pooling rate 1-2 second intervals may seem neat and snappy, but it'll kill your connection count. Dropping that to 5s or even more wont really make a huge difference and users probably wont notice, and your server load will be much lighter. Even consider a variable pooling rate that raises itself during high load.
• Write your ajax to use timeout instead of interval for its pooling and place the timeout call in the ajax success callback, this prevents the requests from stacking up during peak ussage.
• And the biggest one, if using a shared chat room with many users, write your own code to cache the SQL query into a json file and serve that to ajax requests, and write some custom TTL code to check its age and re-populate it as needed during requests, CRON would be great here if your host allows. Age checking a file and redirecting an AJAX request to it is a higher level function with very little server overhead especially compared to querying a DB. And don't parse the file in PHP looking to filter out old messages, store the file with the first message in the filename such as chat_243.json and save it as already formatted json, then just serve up the entire file if a request comes into the php with lastMessageReceived = 243. Since this will create several files you will need a function to clean up files older than (m) minutes, but this is also light duty work for the server.

There are also options like DB engines designed for chat and sockets (node.js), but those require more server tweaking than typical hosting accounts allow, for my purposes I have been writing my chat-room keeping in mind the idea that it may get deployed to a shared server at some point.

来源：https://stackoverflow.com/questions/6853482/flat-file-vs-database-speed