Exposing SQL Server database objects as files in a file system

Question

There's more than one file system

Most version control tools operate on the local disk file system.

Database objects for most relational database systems do exist in a file system, inasmuch as there is a textual name identifying the object and the creation script can be retrieved or at least generated using this name.

But it isn't the local disk file system, and as a result they are invisible to tools like CVS or SVN, which operate strictly on the local disk file system.

In order for SVN to be applied to database objects, they must be replicated into the local disk file system, and changes to the local disk file system must be replicated to the database.

Different mode of use

Unlike source code, of which each developer maintains a private working copy, developers tend to work on a shared database on a server somewhere on the network. While Visual Studio provides direct support for mount-on-demand project-local copies of the database, developers have shunned this facility because there is no convenient and reliable way to merge changes.

However, once changes to the database structure are managed by a copy-merge version control system like CVS or SVN, propagation and merging will be mostly automatic (bar conflicts) and there is no longer any reason to share a database.

Ruling out SCC as an option

Microsoft SQL Management Studio supports version control for anything that implements the SCC spec. Microsoft only lists VSS (blech) but Google reveals a plethora of options. However, SCC is all about locking - double blech.

Replicating between file systems

The whole question now devolves to one of replicating between file systems. CodePlex contains an implementation for VS2005/SQL2005 but it doesn't work with VS2008/SQL2008.

At this point I think the underpinning question of "how should I go about this" has been satisfactorily addressed, although I'm not sure how to award points.

Thank you to all concerned for your input.

Some concrete questions do arise, mostly to do with how to script out various types of schema object.

• How to extract create and alter scripts in dependency order for
  • table
  • view
  • stored procedure
  • function
  • trigger
  • index
  • foreign key
• How to extract table population scripts in dependency order
• How to efficiently detect changes to the schema (in the absence of triggers on sys.objects it will be necessary to poll; this had better be fast and cheap)

Detecting changes

It has come to my attention that it is possible to bind actions to changes in schema using policies. There remain the questions of dependency ordering and of how to script a table creation statement

Answer 1

We use Red Gate to compare current schema against the scripted files stored in SVN for baselines, versioning etc

However, our master reference is actually a restored copy of production. This is our baseline and should correspond to SVN. It's part of deployment process to commit the master scripts to SVN, which Red Gate does usefully: it only changing the files for changed objects.

We further separate our working scripts and release scripts (changes only), so we always have a master DB and a baseline in SVN. We only use scripts for development though.

Database source control is good stuff, but it's challenging to implement because of the nature of what a SQL Server object is: a row or 3 in some tables...

Answer 2

at a very simplistic level you could write a windows service that watches the file system and parses the files in a aprticuler directory and applies them to the db. An simillar mechinsim using the SQL server broker (or just triggers and xp_commandshell) could be used to write the reverse.

Answer 3

Over the last six months or so I have been developing a tool called ShiftSchema that I think is relevant to your question.

ShiftSchema uses database triggers to synchronize SQL Server 2005 and 2008 database objects with files on disk that are suitable for storing in a version control system. It also monitors the filesystem for changes (when you update from the repository and get schema changes committed by another developer) and will push those changes into your personal development database.

It does support synchronizing data but that feature is really aimed at small amounts of data such as look up tables.

It also has a facility for comparing two databases (either in the RDBMS or on disk) and generating a DDL script to synchronize them.

ShiftSchema is intended to be used in a development environment where each developer has their own personal development database.

If you're interested, the website link in my profile points to the ShiftSchema site.

Answer 4

Red Gate is building SQL Source Control, integrating with SSMS to provide both committing to and retrieving from source control (in the background we link the database objects to the respective creation SQL files held in source control). Although we'd recommend that each developer uses their own development copy of the database we plan to support the model whereby this is shared, although this comes with the drawback that any developer can break the database for everyone at a moment's notice.

We hope to release the tool in the first half of 2010. If you'd like to know more, or sign up to our Early Access Program, please visit the following link:

http://www.red-gate.com/products/SQL_Source_Control/index.htm

Kind regards,

David Atkinson, Product Manager, Red Gate Software

Answer 5

w.r.t the changes made directly into the SQL server by other users: I dont know how practical this is for you, but it may be a good idea to make ALL changes via SQL scripts, not indivdually, directly on to the server. These SQL scripts can be numbered and placed in source control of your choice. To get a finer control of deployed changes, you can have everychange script paired with a rollback script that can be used if need be.

You will of course need to educate the users, put some controls in place etc., and also tweak the deployment process so that only approved changes flow to the database environment via the scripts. Just a thought though.

Answer 6

Do you necessarily need to track EVERY change made to an object or just the last one? We wrote a solution in C# which works against TFS in that we have a baseline of all the SQL objects in the database and then using the methods from Microsoft.SqlServer.Management.Smo we just go thru each database object and compare the 'working set' to the server version. We run it at night as part of our evening processing and it takes about 15 minutes to go thru the entire server of 9 databases. We've found that it works great, doesn't involve any direct modification to SQL servers/databases and it works for SQL 2005/2008. It generates a report that gets mailed out to our database admin letting them know what objects have changed, then allows them to go thru TFS and see whats what.

I had originally started here; http://odetocode.com/blogs/scott/archive/2008/02/02/versioning-databases-change-scripts.aspx

but found that what I was looking for wasn't so much a way to push changes to a server but to simply know the changes. The blog link has a few decent suggestions, might be helpful hopefully.

regards.