SQL Server history table - populate through SP or Trigger?

Question

In my SQL Server backend for my app, I want to create history tables for a bunch of my key tables, which will track a history of changes to the rows.

My entire appli

Answer 1

One issue to be very careful about is to identify your intended use cases for this table, and make sure it's constructed properly for that purpose.

Specifically, if it's for an operational audit trail for stakeholders, that's quite different from before-and-after snapshots of record changes in tables. (In fact, I have a difficult time imagining a good use for record changes, other than debugging.)

An audit trail normally requires, at minimum, a user id, a timestamp, and an operation code - and probably some detail about the operation. Example - change the ordered quantity on a line item on a purchase order.

And for this type of audit trail you do not want to use triggers. The higher in the BR layer you embed the generation of these events, the better.

OTOH, for record-level changes, triggers are the right match. But it's also often easier to get this from your dbms journaling files.

Answer 2

Use triggers for this. This means that any changes, regardless of source, will be reflected in the history table. It's good for security, resilient to failure modes like people forgetting to add code to update the history table and so forth.

There is not likely to be any particular speed difference in either for this type of operation as execution time will be dominated by the I/O.

Answer 3

I prefer to use triggers for audit tables because triggers can capture all updates, inserts and deletes, not just the updates, inserts and deletes invoked through certain stored procedures:

CREATE TRIGGER [dbo].[tr_Employee_rev]
ON [dbo].[Employee]
AFTER UPDATE, INSERT, DELETE
AS
BEGIN
    IF EXISTS(SELECT * FROM INSERTED) AND EXISTS (SELECT * FROM DELETED)
    BEGIN
        INSERT INTO [EmployeeRev](EmployeeID,Firstname,Initial,Surname,Birthdate,operation, updated, updatedby) SELECT inserted.ID, inserted.Firstname,inserted.Initial,inserted.Surname,inserted.Birthdate,'u', GetDate(), SYSTEM_USER FROM INSERTED
    END 

    IF EXISTS (SELECT * FROM INSERTED) AND NOT EXISTS(SELECT * FROM DELETED)
    BEGIN
        INSERT INTO [EmployeeRev](EmployeeID,Firstname,Initial,Surname,Birthdate,operation, updated, updatedby) SELECT inserted.ID, inserted.Firstname,inserted.Initial,inserted.Surname,inserted.Birthdate,'i', GetDate(), SYSTEM_USER FROM INSERTED
    END

    IF EXISTS(SELECT * FROM DELETED) AND NOT EXISTS(SELECT * FROM INSERTED)
    BEGIN
        INSERT INTO [EmployeeRev](EmployeeID,Firstname,Initial,Surname,Birthdate,operation, updated, updatedby) SELECT deleted.ID, deleted.Firstname,deleted.Initial,deleted.Surname,deleted.Birthdate,'d', GetDate(), SYSTEM_USER FROM DELETED 
    END
END

I use SQLServer to generate the SQL for the revision tables instead of hand coding it. This code is available on https://github.com/newdigate/sqlserver-revision-tables

Answer 4

Triggers. Here is my approach:

1. Create one audit table for each critical table that requires audit trial
2. Audit table will include all columns from source table + columns audit record info such as who, when and the action
3. Trigger for UPDATE and DELETE only, the INSERT operation will have the pristine record in source table itself
4. Before update or delete, copy the original record + audit info to audit table
5. (Optionally - for UPDATE only:) To know which column got updated, use either UPDATE(ColumnName) or COLUMNS_UPDATED() built in SQL function to determine the affected columns

Auditing this way keeps the current status in the source table and all the history in audit table and easily identified by the key columns.

Answer 5

We have a third party tool ApexSQL Audit that we used to generate triggers.

Here is how triggers look like in the background and how data is stored. Hopefully guys will find this useful enough to reverse engineer the process. Its a bit different from what Ian Boyd showed in his examples because it allows each columns to be audited separately.

Table 1 – holds transaction details (who, when, application, host name, etc)

CREATE TABLE [dbo].[AUDIT_LOG_TRANSACTIONS](
    [AUDIT_LOG_TRANSACTION_ID] [int] IDENTITY(1,1) NOT NULL,
    [DATABASE] [nvarchar](128) NOT NULL,
    [TABLE_NAME] [nvarchar](261) NOT NULL,
    [TABLE_SCHEMA] [nvarchar](261) NOT NULL,
    [AUDIT_ACTION_ID] [tinyint] NOT NULL,
    [HOST_NAME] [varchar](128) NOT NULL,
    [APP_NAME] [varchar](128) NOT NULL,
    [MODIFIED_BY] [varchar](128) NOT NULL,
    [MODIFIED_DATE] [datetime] NOT NULL,
    [AFFECTED_ROWS] [int] NOT NULL,
    [SYSOBJ_ID]  AS (object_id([TABLE_NAME])),
  PRIMARY KEY CLUSTERED 
  (
       [AUDIT_LOG_TRANSACTION_ID] ASC
  )
)

Table 2 – holds before/after values.

CREATE TABLE [dbo].[AUDIT_LOG_DATA](
   [AUDIT_LOG_DATA_ID] [int] IDENTITY(1,1) NOT NULL,
   [AUDIT_LOG_TRANSACTION_ID] [int] NOT NULL,
   [PRIMARY_KEY_DATA] [nvarchar](1500) NOT NULL,
   [COL_NAME] [nvarchar](128) NOT NULL,
   [OLD_VALUE_LONG] [ntext] NULL,
   [NEW_VALUE_LONG] [ntext] NULL,
   [NEW_VALUE_BLOB] [image] NULL,
   [NEW_VALUE]  AS (isnull(CONVERT([varchar](8000),      [NEW_VALUE_LONG],0),CONVERT([varchar](8000),CONVERT([varbinary](8000),substring([NEW_VALUE_BLOB],(1),(8000)),0),0))),
   [OLD_VALUE]  AS (CONVERT([varchar](8000),[OLD_VALUE_LONG],0)),
   [PRIMARY_KEY]  AS ([PRIMARY_KEY_DATA]),
   [DATA_TYPE] [char](1) NOT NULL,
   [KEY1] [nvarchar](500) NULL,
   [KEY2] [nvarchar](500) NULL,
   [KEY3] [nvarchar](500) NULL,
   [KEY4] [nvarchar](500) NULL,
PRIMARY KEY CLUSTERED 
 (
    [AUDIT_LOG_DATA_ID] ASC
)
)

Insert trigger

I’m not showing triggers for update because they are quite long and have the same logic as this one.

CREATE TRIGGER [dbo].[tr_i_AUDIT_Audited_Table]
ON [dbo].[Audited_Table]
FOR INSERT
NOT FOR REPLICATION
As
BEGIN
DECLARE 
    @IDENTITY_SAVE              varchar(50),
    @AUDIT_LOG_TRANSACTION_ID       Int,
    @PRIM_KEY               nvarchar(4000),
    @ROWS_COUNT             int

SET NOCOUNT ON
Select @ROWS_COUNT=count(*) from inserted
Set @IDENTITY_SAVE = CAST(IsNull(@@IDENTITY,1) AS varchar(50))

INSERT
INTO dbo.AUDIT_LOG_TRANSACTIONS
(
    TABLE_NAME,
    TABLE_SCHEMA,
    AUDIT_ACTION_ID,
    HOST_NAME,
    APP_NAME,
    MODIFIED_BY,
    MODIFIED_DATE,
    AFFECTED_ROWS,
    [DATABASE]
)
values(
    'Audited_Table',
    'dbo',
    2,  --  ACTION ID For INSERT
    CASE 
      WHEN LEN(HOST_NAME()) < 1 THEN ' '
      ELSE HOST_NAME()
    END,
    CASE 
      WHEN LEN(APP_NAME()) < 1 THEN ' '
      ELSE APP_NAME()
    END,
    SUSER_SNAME(),
    GETDATE(),
    @ROWS_COUNT,
    'Database_Name'
)

Set @AUDIT_LOG_TRANSACTION_ID = SCOPE_IDENTITY()    

--This INSERT INTO code is repeated for each columns that is audited. 
--Below are examples for only two columns
INSERT INTO dbo.AUDIT_LOG_DATA
(
    AUDIT_LOG_TRANSACTION_ID,
    PRIMARY_KEY_DATA,
    COL_NAME,
    NEW_VALUE_LONG,
    DATA_TYPE
    , KEY1
)
SELECT
    @AUDIT_LOG_TRANSACTION_ID,
    convert(nvarchar(1500), IsNull('[PK_Column]='+CONVERT(nvarchar(4000), NEW.[PK_Column], 0), '[PK_Column] Is Null')),
    'Column1',
    CONVERT(nvarchar(4000), NEW.[Column1], 0),
    'A'
    , CONVERT(nvarchar(500), CONVERT(nvarchar(4000), NEW.[PK_Column], 0))
FROM inserted NEW
WHERE NEW.[Column1] Is Not Null

 --value is inserted for each column that is selected for auditin
INSERT INTO dbo.AUDIT_LOG_DATA
(
    AUDIT_LOG_TRANSACTION_ID,
    PRIMARY_KEY_DATA,
    COL_NAME,
    NEW_VALUE_LONG,
    DATA_TYPE
    , KEY1
)
SELECT
    @AUDIT_LOG_TRANSACTION_ID,
    convert(nvarchar(1500), IsNull('[PK_Column]='+CONVERT(nvarchar(4000), NEW.[PK_Column], 0), '[PK_Column] Is Null')),
    'Column2',
    CONVERT(nvarchar(4000), NEW.[Column2], 0),
    'A'
    , CONVERT(nvarchar(500), CONVERT(nvarchar(4000), NEW.[PK_Column], 0))
    FROM inserted NEW
    WHERE NEW.[Column2] Is Not Null
End

Disclaimer: I’m not affiliated with Apex in any way but I do use their tools in my current job.