Partitioning mySQL tables that has foreign keys?

Question

What would be an appropriate way to do this, since mySQL obviously doesnt enjoy this. To leave either partitioning or the foreign keys out from the database design would not

Answer 1

It depends on the extent to which the size of rows in the partitioned table is the reason for partitions being necessary.

If the row size is small and the reason for partitioning is the sheer number of rows, then I'm not sure what you should do.

If the row size is quite big, then have you considered the following:

Let P be the partitioned table and F be the table referenced in the would-be foreign key. Create a new table X:

CREATE TABLE `X` (
    `P_id` INT UNSIGNED NOT NULL,
        -- I'm assuming an INT is adequate, but perhaps
        -- you will actually require a BIGINT
    `F_id` INT UNSIGNED NOT NULL,
    PRIMARY KEY (`P_id`, `F_id`),
    CONSTRAINT `Constr_X_P_fk`
        FOREIGN KEY `P_fk` (`P_id`) REFERENCES `P`.`id`
        ON DELETE CASCADE ON UPDATE RESTRICT,
    CONSTRAINT `Constr_X_F_fk`
        FOREIGN KEY `F_fk` (`F_id`) REFERENCES `F`.`id`
        ON DELETE RESTRICT ON UPDATE RESTRICT
) ENGINE=INNODB CHARACTER SET ascii COLLATE ascii_general_ci

and crucially, create a stored procedure for adding rows to table P. Your stored procedure should make certain (use transactions) that whenever a row is added to table P, a corresponding row is added to table X. You must not allow rows to be added to P in the "normal" way! You can only guarantee that referential integrity will be maintained if you keep to using your stored procedure for adding rows. You can freely delete from P in the normal way, though.

The idea here is that your table X has sufficiently small rows that you should hopefully not need to partition it, even though it has many many rows. The index on the table will nevertheless take up quite a large chunk of memory, I guess.

Should you need to query P on the foreign key, you will of course query X instead, as that is where the foreign key actually is.

Answer 2

I would strongly suggest sharding using Date as the key for archiving data to archive tables. If you need to report off multiple archive tables, you can use Views, or build the logic into your application.

However, with a properly structured DB, you should be able to handle tens of millions of rows in a table before partitioning, or sharding is really needed.