问题
Background
We are in the process of upgrading a legacy import tool, what it does is it moves data from one database attached to SQL Server to a 2nd database on the same server with a different schema performing the translations and mapping along the way.
Here is a example to help explain what is happening
Say the source database has one table called Client_Info and the destination table has two tables named Clients and Cities
Source.dbo.Client_Info
+-----------+----------+----------+-------+
| FirstName | LastName | City | State |
+-----------+----------+----------+-------+
| John | Smith | Richmond | VA |
| Jeff | Walters | New York | NY |
+-----------+----------+----------+-------+
Dest.dbo.Clients
+-----------+-------------+--------------------------------------+
| FirstName | LastName | CityGuid |
+-----------+-------------+--------------------------------------+
| Scott | Chamberlain | 07d954bf-3214-4df4-b640-48c27db2b1ed |
+-----------+-------------+--------------------------------------+
Dest.dbo.Cities
+--------------------------------------+----------+-------+
| CityGuid | CityName | State |
+--------------------------------------+----------+-------+
| 07d954bf-3214-4df4-b640-48c27db2b1ed | Richmond | VA |
+--------------------------------------+----------+-------+
After the merge I would expect the destination to look like this
Dest.dbo.Clients
+-----------+-------------+--------------------------------------+
| FirstName | LastName | CityGuid |
+-----------+-------------+--------------------------------------+
| Scott | Chamberlain | 07d954bf-3214-4df4-b640-48c27db2b1ed |
| John | Smith | 07d954bf-3214-4df4-b640-48c27db2b1ed |
| Jeff | Walters | 98a75f88-eeaa-49ba-b464-2ac988a7b093 |
+-----------+-------------+--------------------------------------+
Dest.dbo.Cities
+--------------------------------------+----------+-------+
| CityGuid | CityName | State |
+--------------------------------------+----------+-------+
| 07d954bf-3214-4df4-b640-48c27db2b1ed | Richmond | VA |
| 98a75f88-eeaa-49ba-b464-2ac988a7b093 | New York | NY |
+--------------------------------------+----------+-------+
Currently it is a VB6 project and we just use hard coded SQL statements shuffling records around using temporary #t tables and populate the GUID columns with the existing or new values as necessary.
We have since moved to being a C# .NET shop and it has been deemed time to update the import tool to C# due to it being harder and harder to make changes to the tool when the destination database changes when new versions of our software comes out (its a fight just to get Visual Studio 6 installed and working on windows 8).
My question
Is a ORM tool like NHibernate the right tool for the job? None of us have really used a ORM before (and we are not the development team that writes the software that talks to the new database for day to day use, we are just in charge of the migration of the old database to the new database and the "real developers" tells us what changes to the schema they make each version). I am not too sure about using a ORM as I think ORMs are used for things like CRUD operations to a client, not on server database migrations like this.
The thing I think that would be the "proper" way to do it is using SSIS, but no one in my department is familiar with it and my supervisor feels it would take too much time and resources to have everyone learn another language to maintain it (this migration tool is what is run after the step I was talking about in my other question).
The main thing I am looking for is a way to do this migration with:
- Large volumes of data
- Allow setting of some columns during the transfer (like re-using foreign key GUIDs)
- Easy to change as the destination database schema changes
- Done preferably in C# or SQL.
What kind of tools should I be looking for with those kinds of requirements?
I ask this question as I don't think a ORM is the right thing to do, but I am not sure what I should use instead. Other than SSIS (that has been taken off the table for being too different) I don't know what to do other than just using hard coded SQL statements, but that breaks the "Easy to change" requirement in my mind.
回答1:
SSIS is the perfect tool to do this, hands down.
An ORM is only for CRUD operations(as you mentioned correctly) in applications, and has a high probability of raising serious concerns for large data transfers. Most ORMs are not even suggested for CRUD operations involving large number of rows, forget database level data migration. ORMs are mostly used for ease-of-coding for persistent data connections between a software application and a database.
SSIS on the other hand is made for ETL (Extract Transform Load), at database, datawarehouse levels, safely. The migration occurs at a significantly higher rate as well, as compared to stored procedures.
Another important thing I wanted to add, is the fact that SSIS is super easy(from my experience). Most operations involve drag and drop of ETL controls on the Visual Studio designer and then configuring the data types on the configuration screens. Unless you really love to write code, or in extremely complex scenarios, you would be well off with just that and a few data type Transformation(T) snippets.
I understand, the boss thinks of it as an unnecessary investment currently. However, SSIS is Microsoft's trump card in the data warehousing scene. Looking at your current requirements, it is exactly what your organization needs. By the experience of it in our organization, its an investment that will prove worth every penny as long as Microsoft lives.
回答2:
I also wouldn't use a full blown ORM, but a micro ORM like Dapper is a great for tasks like this(among other things). Super fast and you run it all pretty close to the metal for high performance and ease of use if you are familiar with TSQL and c# it is a snap to use. ( you can be productive in 15 minutes)
Just finished a similar project, using it to move data from server to server and it work and performed like a champ.
https://code.google.com/p/dapper-dot-net/
回答3:
An ORM is definitely not the right tool, as you rightly point out they are for OLTP applications.
Given that SSIS is off the table (it is the right choice were it an option), I'd consider looking at Rhino ETL. Very flexible, and you can use SqlBulkCopy with it which obviously you should consider doing here. It's open source, too boot.
来源：https://stackoverflow.com/questions/17137840/is-a-orm-the-right-tool-to-use-for-migrating-data