is having millions of tables and millions of rows within them a common practice in MySQL database design?

Question

I am doing database design for an upcoming web app, and I was wondering from anybody profusely using mysql in their current web apps if this sort of design is efficient for

Answer 1

Having millions of rows in a table is perfectly normal and MySQL can handle this easily, as long as you use appropriate indexes.

Having millions of tables on the other hand seems like a bad design.

Answer 2

In addition to what others have said, don't forget that finding the right table based on the given table name also takes time. How much time? Well, this is internal to DBMS and likely not documented, but probably more than you think.

So, a query searching for a row can either take:

1. Time to find the table + time to find the row in a (relatively) small table.
2. Or, just the time to find a row in one large table.

The (2) is likely to be faster.

Also, frequently using different table names in your queries makes query preparation less effective.

Answer 3

1 - Definitely not. Almost anyone you ask will tell you millions of tables is a terrible idea.

2 - Millions of ROWS is common, so just fine.

3 - Probably terribly, especially if the queries are written by someone who thinks it's OK to have millions of tables. That tells me this is someone who doesn't understand databases very well.

4 - See #3

5 - Impossible to tell. You will have a lot of extra overhead from the extra tables as they all need extra metadata. Space needed will depend on indexes and how wide the tables are, along with a lot of other factors.

In short, this is a very very very seriously bad idea and you should not do it.

Answer 4

If you are thinking of having millions of tables, I can't imagine that you actually designing millions of logically distinct tables. Rather, I would strongly suspect that you are creating tables dynamically based on data. That is, rather than create a FIELD for, say, the user id, and storing one or more records for each user, you are contemplating creating a new TABLE for each user id. And then you'll have thousands and thousands of tables that all have exactly the same fields in them. If that's what you're up to: Don't. Stop.

A table should represent a logical TYPE of thing that you want to store data for. You might make a city table, and then have one record for each city. One of the fields in the city table might indicate what country that city is in. DO NOT create a separate table for each country holding all the cities for each country. France and Germany are both examples of "country" and should go in the same table. They are not different kinds of thing, a France-thing and a Germany-thing.

Here's the key question to ask: What data do I want to keep in each record? If you have 1,000 tables that all have exactly the same columns, then almost surely this should be one table with a field that has 1,000 possible values. If you really seriously keep totally different information about France than you keep about Germany, like for France you want a list of provinces with capital city and the population but for Germany you want a list of companies with industry and chairman of the board of directors, then okay, those should be two different tables. But at that point the difference is likely NOT France versus Germany but something else.

Answer 5

SQL Server has many ways you can support large tables. You may find some help by splitting your indexes across multiple partitions (filegroups), placing large tables on their own filegroup, and indexes for the large table on another set of filegroups.

A filegroup is basically a separate drive. Each drive has its own dedicated read and write heads. The more drives the more heads are searching the indexes at a time and thus faster results finding your records.

Here is a page that talks in details about filegroups.

http://cm-bloggers.blogspot.com/2009/04/table-and-index-partitioning-in-sql.html

Answer 6

Millions of rows is perfectly normal usage, and can respond quickly if properly optimized and indexed.

Millions of tables is an indication that you've made a major goof in how you've architected your application. Millions of rows times millions of tables times 80,000 users means what, 80 quadrillion records? I strongly doubt you have that much data.