Migrating databases using phpMyAdmin's tracking mechanism

Question

In a development database, I have phpMyAdmin Tracking enabled on all tables. It logs all the changes I make to the tables\' structures (in this case I\'m not interested in d

Answer 1

I've had some success with MySQL Workbench:

Import (reverse engineer) your dev database into workbench. You can do this by either exporting your schema to an SQL file and loading it into workbench, or workbench will get the schema directly from the server.

Next, generate your diff file with the "Synchronise model" option. You select the production database, then which tables to sync, and workbench generates an SQL file you can run to sync both models.

A word of caution: the first time, there will likely be quite a few apparently uneeded changes while the DB is updated to workbench "style". For subsequent updates, the tool is rather reliable, though I would never let an automated tool have free range over my production DB ;-)

Always check the SQL file for errors, in some cases, dropping a column then adding another of the same name but different type will generate an alter column which will fail.

Answer 2

The algorithm for parsing the BLOB field of the "pma_tracking" table is located in the getTrackedData method of the PMA_Tracker class, in the libraries/Tracker.class.php source file.
Starting from that code, I've written a simple PHP script to extract all the data definition statements (except the "DROP TABLE" statements) from the "pma_tracking" table.
For example, suppose that you want to get the list of all the changes of all the tables of the "test" database since version "1":

<?php

$link = mysqli_init();

// Adjust hostname, username, password and db name before use!
$db = mysqli_real_connect($link, "localhost", "myuser", "mypass", "phpmyadmin") 
      or die(mysqli_connect_error());

// Adjust also target db name and tracking version
$db_name = "test";
$version = "1";

$sql = "SELECT schema_sql FROM pma_tracking 
         WHERE db_name='{$db_name}' AND version>='{$version}' 
         ORDER BY version,date_created";
$result = mysqli_query($link, $sql) or die(mysqli_error($link));
while ($myrow = mysqli_fetch_assoc($result)) {
    $log_schema_entries = explode('# log ',  $myrow['schema_sql']);
    foreach ($log_schema_entries as $log_entry) {
        if (trim($log_entry) != '') {
            $statement = trim(strstr($log_entry, "\n"));
            if (substr($statement, 0, 11) != "DROP TABLE ") {
                echo "{$statement}\n";
            }
        }
    }
}

?>

By redirecting the script output on a file, you'll obtain a SQL commands file with (almost) all the statements needed to replicate the schema changes on the target (eg. production) database; this file must be executed by specifying the "-f" (force) MySQL option:

-f, --force Continue even if we get an SQL error.

By doing so, MySQL will ignore all the "Table already exists" error that will be thrown each time that a CREATE TABLE statement for an existing table is encountered, thus creating only the tables that still does'nt exist in the target database.
This kind of approach obviously has some drawbacks:

1. ALL the DROP TABLE commands will be ignored (not only those automatically inserted from phpMyAdmin) so, if you have deleted a table in the source database, that table won't be deleted in the target database.
   
2. ALL the script errors will be ignored, so it may not be 100% affordable.

A final word of advice: always do a full backup of your target database before proceeding!

Answer 3

I don't have anything that creates an incremental diff between two databases but here's the script I use to compare two MySQL databases:

<?php
//------------------------------------------------------------------------------
// Define the variables we'll be using.
//------------------------------------------------------------------------------
$db1_con = NULL;
$db1_constraints = array();
$db1_dbname = 'db1';
$db1_host = 'localhost';
$db1_password = 'password1';
$db1_tables = array();
$db1_username = 'username1';

$db2_con = NULL;
$db2_constraints = array();
$db2_dbname = 'db2';
$db2_host = '123.123.123.123';
$db2_password = 'password2';
$db2_tables = array();
$db2_username = 'username2';

//------------------------------------------------------------------------------
// Connect to the databases.
//------------------------------------------------------------------------------
try{
    $db1_con = new PDO("mysql:host=$db1_host;dbname=information_schema", $db1_username, $db1_password);
    $db1_con->setAttribute(PDO::ATTR_EMULATE_PREPARES, FALSE); // Try to use the driver's native prepared statements.
    $db1_con->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION); // Let's use exceptions so we can try/catch errors.
}catch(PDOException $e){
    echo "<p>Connection failed for $db1_host: " . $e->getMessage() . '</p>';
    exit;
}

try{
    $db2_con = new PDO("mysql:host=$db2_host;dbname=information_schema", $db2_username, $db2_password);
    $db2_con->setAttribute(PDO::ATTR_EMULATE_PREPARES, FALSE); // Try to use the driver's native prepared statements.
    $db2_con->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION); // Let's use exceptions so we can try/catch errors.
}catch(PDOException $e){
    echo "<p>Connection failed for $db2_host: " . $e->getMessage() . '</p>';
    exit;
}

if (NULL !== $db1_con && NULL !== $db2_con){
    echo "<h2>Column Analysis</h2>";
    $sql = 'SELECT * FROM information_schema.COLUMNS WHERE TABLE_SCHEMA = ? ORDER BY TABLE_NAME, ORDINAL_POSITION';
    $statement1 = $db1_con->prepare($sql);
    $statement1->bindValue(1, $db1_dbname);

    $statement2 = $db2_con->prepare($sql);
    $statement2->bindValue(1, $db2_dbname);

    if (TRUE === $statement1->execute()){
        while ($row = $statement1->fetch(PDO::FETCH_ASSOC)){
            $db1_tables[$row['TABLE_NAME']][$row['COLUMN_NAME']] = array();
            foreach ($row AS $key => $value){
                $db1_tables[$row['TABLE_NAME']][$row['COLUMN_NAME']][$key] = $value;
            }
        }
    }

    if (TRUE === $statement2->execute()){
        while ($row = $statement2->fetch(PDO::FETCH_ASSOC)){
            $db2_tables[$row['TABLE_NAME']][$row['COLUMN_NAME']] = array();
            foreach ($row AS $key => $value){
                $db2_tables[$row['TABLE_NAME']][$row['COLUMN_NAME']][$key] = $value;
            }
        }
    }

    foreach ($db1_tables AS $table => $info){
        if (!isset($db2_tables[$table])){
            echo "<p>Table <strong>$table</strong> does not exist in the SECOND database!</p>";
        }else{
            foreach ($info AS $column => $data){
                if (!isset($db2_tables[$table][$column])){
                    echo "<p>Column <strong>$column</strong> does not exist in table <strong>$table</strong> in the SECOND database!</p>";
                }else{
                    if (count($data)){
                        foreach ($data AS $key => $value){
                            if ($db1_tables[$table][$column][$key] !== $db2_tables[$table][$column][$key]){
                                echo "<p>Column <strong>$column</strong> in table <strong>$table</strong> has differing characteristics for <strong>$key</strong> (". $db1_tables[$table][$column][$key] ." vs. ". $db2_tables[$table][$column][$key] .")</p>";
                            }
                        }
                    }
                }
            }
        }
    }

    foreach ($db2_tables AS $table => $info){
        if (!isset($db1_tables[$table])){
            echo "<p>Table <strong>$table</strong> does not exist in the FIRST database!</p>";
        }else{
            foreach ($info AS $column => $data){
                if (!isset($db1_tables[$table][$column])){
                    echo "<p>Column <strong>$column</strong> does not exist in table <strong>$table</strong> in the FIRST database!</p>";
                }else{
                    if (count($data)){
                        foreach ($data AS $key => $value){
                            if ($db2_tables[$table][$column][$key] !== $db1_tables[$table][$column][$key]){
                                echo "<p>Column <strong>$column</strong> in table <strong>$table</strong> has differing characteristics for <strong>$key</strong> (". $db2_tables[$table][$column][$key] ." vs. ". $db1_tables[$table][$column][$key] .")</p>";
                            }
                        }
                    }
                }
            }
        }
    }
    echo "<h2>Constraint Analysis</h2>";

    $sql = 'SELECT * FROM information_schema.KEY_COLUMN_USAGE WHERE TABLE_SCHEMA = ? ORDER BY TABLE_NAME, ORDINAL_POSITION';
    $statement1 = $db1_con->prepare($sql);
    $statement1->bindValue(1, $db1_dbname);

    $statement2 = $db2_con->prepare($sql);
    $statement2->bindValue(1, $db2_dbname);

    if (TRUE === $statement1->execute()){
        while ($row = $statement1->fetch(PDO::FETCH_ASSOC)){
            foreach ($row AS $key => $value){
                $db1_constraints[$row['TABLE_NAME']][$row['COLUMN_NAME']][$key] = $value;
            }
        }
    }

    if (TRUE === $statement2->execute()){
        while ($row = $statement2->fetch(PDO::FETCH_ASSOC)){
            foreach ($row AS $key => $value){
                $db2_constraints[$row['TABLE_NAME']][$row['COLUMN_NAME']][$key] = $value;
            }
        }
    }

    foreach ($db1_constraints AS $table => $info){
        foreach ($info AS $column => $data){
            if (isset($db2_constraints[$table][$column])){
                if (count($data)){
                    foreach ($data AS $key => $value){
                        if ('CONSTRAINT_NAME' !== $key && $db1_constraints[$table][$column][$key] !== $db2_constraints[$table][$column][$key]){
                            echo "<p>Column <strong>$column</strong> in table <strong>$table</strong> has differing characteristics for <strong>$key</strong> (". $db1_constraints[$table][$column][$key] ." vs. ". $db2_constraints[$table][$column][$key] .")</p>";
                        }
                    }
                }
            }else{
                echo "<p>Column <strong>$column</strong> in table <strong>$table</strong> is missing a constraint in the SECOND database!</p>";
            }
        }
    }

    foreach ($db2_constraints AS $table => $info){
        foreach ($info AS $column => $data){
            if (isset($db1_constraints[$table][$column])){
                if (count($data)){
                    foreach ($data AS $key => $value){
                        if ('CONSTRAINT_NAME' !== $key && $db2_constraints[$table][$column][$key] !== $db1_constraints[$table][$column][$key]){
                            echo "<p>Column <strong>$column</strong> in table <strong>$table</strong> has differing characteristics for <strong>$key</strong> (". $db2_constraints[$table][$column][$key] ." vs. ". $db1_constraints[$table][$column][$key] .")</p>";
                        }
                    }
                }
            }else{
                echo "<p>Column <strong>$column</strong> in table <strong>$table</strong> is missing a constraint in the FIRST database!</p>";
            }
        }
    }
}
?>

Edited to add code that shows differences in constraints as well.

Answer 4

I'm not too familiar with SQL tools, so I cannot recommend anything to help you out there, but I can try and help with a custom workflow...

1. Create a table called structure_log
2. Create a PHP script called print_stucture.php that prints whatever info you desire to a file on the server, saves the file as a timestamp (this will be your version number), and saves the name in the structure_log table
3. Create a crontab that runs print_structure.php however often you desire
4. Create a PHP script called delete_dups.php that grabs the last two records from your structure_log table, compares those two files, and if they are the same (representing no change to structures), deletes the one with the latest timestamp (filename) and removes that record from the structure_log table
5. Create a crontab that runs delete_dups.php half as often as the one that runs print_structure.php

This will make a versioning folder on your server. You can manually run the print_structure.php script whenever you desire and compare it against the latest version log you have in your server folder to see if your database you just ran it on, is the same as the last time the version check was ran.

Answer 5

I don't know how you could solve this problem using phpMyAdmin, but there are other tools that might help you achieve the effect your looking for. Liquibase is one of them. I've used it some times in the past and it was pretty good. It takes a little to get the hang of it, but I think it might help you.