ALTER TABLE ADD COLUMN IF NOT EXISTS in SQLite

Question

We\'ve recently had the need to add columns to a few of our existing SQLite database tables. This can be done with ALTER TABLE ADD COLUMN. Of course, if the table has alre

Answer 1

select * from sqlite_master where type = 'table' and tbl_name = 'TableName' and sql like '%ColumnName%'

Logic: sql column in sqlite_master contains table definition, so it certainly contains string with column name.

As you are searching for a sub-string, it has its obvious limitations. So I would suggest to use even more restrictive sub-string in ColumnName, for example something like this (subject to testing as '`' character is not always there):

select * from sqlite_master where type = 'table' and tbl_name = 'MyTable' and sql like '%`MyColumn` TEXT%'

Answer 2

All these answers are fine if you execute one line at a time. However, the original question was to input a sql script that would be executed by a single db execute and all the solutions ( like checking to see if the column is there ahead of time ) would require the executing program either have knowledge of what tables and columns are being altered/added or do pre-processing and parsing of the input script to determine this information. Typically you are not going to run this in realtime or often. So the idea of catching an exception is acceptable and then moving on. Therein lies the problem...how to move on. Luckily the error message gives us all the information we need to do this. The idea is to execute the sql if it exceptions on an alter table call we can find the alter table line in the sql and return the remaining lines and execute until it either succeeds or no more matching alter table lines can be found. Heres some example code where we have sql scripts in an array. We iterate the array executing each script. We call it twice to get the alter table command to fail but the program succeeds because we remove the alter table command from the sql and re-execute the updated code.

#!/bin/sh
# the next line restarts using wish \

exec /opt/usr8.6.3/bin/tclsh8.6  "$0" ${1+"$@"}
foreach pkg {sqlite3 } {
    if { [ catch {package require {*}$pkg } err ] != 0 } {
    puts stderr "Unable to find package $pkg\n$err\n ... adjust your auto_path!";
    }
}
array set sqlArray {
    1 {
    CREATE TABLE IF NOT EXISTS Notes (
                      id INTEGER PRIMARY KEY AUTOINCREMENT,
                      name text,
                      note text,
                      createdDate integer(4) DEFAULT ( cast( strftime('%s', 'now') as int ) ) ,
                      updatedDate integer(4) DEFAULT ( cast( strftime('%s', 'now') as int ) ) 
                      );
    CREATE TABLE IF NOT EXISTS Version (
                        id INTEGER PRIMARY KEY AUTOINCREMENT,
                        version text,
                        createdDate integer(4) DEFAULT ( cast( strftime('%s', 'now') as int ) ) ,
                        updatedDate integer(4) DEFAULT ( cast( strftime('%s', 'now') as int ) )
                        );
    INSERT INTO Version(version) values('1.0');
    }
    2 {
    CREATE TABLE IF NOT EXISTS Tags (
        id INTEGER PRIMARY KEY AUTOINCREMENT,
        name text,
        tag text,
        createdDate integer(4) DEFAULT ( cast( strftime('%s', 'now') as int ) ) ,
        updatedDate integer(4) DEFAULT ( cast( strftime('%s', 'now') as int ) ) 
        );
    ALTER TABLE Notes ADD COLUMN dump text;
    INSERT INTO Version(version) values('2.0');
    }
    3 {
    ALTER TABLE Version ADD COLUMN sql text;
    INSERT INTO Version(version) values('3.0');
    }
}

# create db command , use in memory database for demonstration purposes
sqlite3 db :memory:

proc createSchema { sqlArray } {
    upvar $sqlArray sql
    # execute each sql script in order 
    foreach version [lsort -integer [array names sql ] ] {
    set cmd $sql($version)
    set ok 0
    while { !$ok && [string length $cmd ] } {  
        try {
        db eval $cmd
        set ok 1  ;   # it succeeded if we get here
        } on error { err backtrace } {
        if { [regexp {duplicate column name: ([a-zA-Z0-9])} [string trim $err ] match columnname ] } {
            puts "Error:  $err ... trying again" 
            set cmd [removeAlterTable $cmd $columnname ]
        } else {
            throw DBERROR "$err\n$backtrace"
        }
        }
    }
    }
}
# return sqltext with alter table command with column name removed
# if no matching alter table line found or result is no lines then
# returns ""
proc removeAlterTable { sqltext columnname } {
    set mode skip
    set result [list]
    foreach line [split $sqltext \n ] {
    if { [string first "alter table" [string tolower [string trim $line] ] ] >= 0 } {
        if { [string first $columnname $line ] } {
        set mode add
        continue;
        }
    }
    if { $mode eq "add" } {
        lappend result $line
    }
    }
    if { $mode eq "skip" } {
    puts stderr "Unable to find matching alter table line"
    return ""
    } elseif { [llength $result ] }  { 
    return [ join $result \n ]
    } else {
    return ""
    }
}
               
proc printSchema { } {
    db eval { select * from sqlite_master } x {
    puts "Table: $x(tbl_name)"
    puts "$x(sql)"
    puts "-------------"
    }
}
createSchema sqlArray
printSchema
# run again to see if we get alter table errors 
createSchema sqlArray
printSchema

expected output

Table: Notes
CREATE TABLE Notes (
                      id INTEGER PRIMARY KEY AUTOINCREMENT,
                      name text,
                      note text,
                      createdDate integer(4) DEFAULT ( cast( strftime('%s', 'now') as int ) ) ,
                      updatedDate integer(4) DEFAULT ( cast( strftime('%s', 'now') as int ) ) 
                      , dump text)
-------------
Table: sqlite_sequence
CREATE TABLE sqlite_sequence(name,seq)
-------------
Table: Version
CREATE TABLE Version (
                        id INTEGER PRIMARY KEY AUTOINCREMENT,
                        version text,
                        createdDate integer(4) DEFAULT ( cast( strftime('%s', 'now') as int ) ) ,
                        updatedDate integer(4) DEFAULT ( cast( strftime('%s', 'now') as int ) )
                        , sql text)
-------------
Table: Tags
CREATE TABLE Tags (
        id INTEGER PRIMARY KEY AUTOINCREMENT,
        name text,
        tag text,
        createdDate integer(4) DEFAULT ( cast( strftime('%s', 'now') as int ) ) ,
        updatedDate integer(4) DEFAULT ( cast( strftime('%s', 'now') as int ) ) 
        )
-------------
Error:  duplicate column name: dump ... trying again
Error:  duplicate column name: sql ... trying again
Table: Notes
CREATE TABLE Notes (
                      id INTEGER PRIMARY KEY AUTOINCREMENT,
                      name text,
                      note text,
                      createdDate integer(4) DEFAULT ( cast( strftime('%s', 'now') as int ) ) ,
                      updatedDate integer(4) DEFAULT ( cast( strftime('%s', 'now') as int ) ) 
                      , dump text)
-------------
Table: sqlite_sequence
CREATE TABLE sqlite_sequence(name,seq)
-------------
Table: Version
CREATE TABLE Version (
                        id INTEGER PRIMARY KEY AUTOINCREMENT,
                        version text,
                        createdDate integer(4) DEFAULT ( cast( strftime('%s', 'now') as int ) ) ,
                        updatedDate integer(4) DEFAULT ( cast( strftime('%s', 'now') as int ) )
                        , sql text)
-------------
Table: Tags
CREATE TABLE Tags (
        id INTEGER PRIMARY KEY AUTOINCREMENT,
        name text,
        tag text,
        createdDate integer(4) DEFAULT ( cast( strftime('%s', 'now') as int ) ) ,
        updatedDate integer(4) DEFAULT ( cast( strftime('%s', 'now') as int ) ) 
        )
-------------