What is the best practice for data conversion between applications [closed]

问题

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.

Closed 7 years ago.

I wonder if this might be a too subjective question for stackoverflow but ill give it a go anyway.

Is there a common/best practice for data migration between applications? Lets say I have Application A written in Java/J2EE and connected to a PostgreSQL database and Application B written in Ruby/Rails and connected to a MySQL database.

I want to migrate my data from Application A to Application B, the table structure and thus the datamodel of A is completely different from B. So I want to extract information from A, change its structure and insert it into B.

Also I have existing information in application B which has a relation with information from application A, based for example on a ID common in both applications

I tried writing a few fancy sql scripts but that goes nowhere fast.

The last time I faced a project like this I just wrote big chuck of code to handle the migration. Is there maybe a best practice for this I wonder? I reckon this is a job done quite often by developers. Maybe there are tools or frameworks available?

回答1:

Probably not a single best practice, but once you pick an approach, a collection of best practices.

One strategy is to bring the data in the same model (or very close) in the destination platform and then transform within the destination platform.

For instance, if the destination was SQL Server, I would create another database on the destination server with straight data copies from tables to tables (data types are the main thing you are watching out for there) and simply use queries against database2.user.table_names in order to populate the destination data model.

This eliminates issues with heterogenous source/destination in whatever choice of ETL tool you might be using and allows you to make some additional indexes on the database2 which might be optimal for your conversion.

Also, your conversion would be in straight SQL, allowing a joins to both source and destination simultaneously without any inter-server latency or bandwidth.

If you have binary data in your tables or anything like that, obviously things get a lot more complicated.

回答2:

Broad question, broad answer?

1. Recreate the data model in DatabaseB, without auto-increment, etc
2. Copy all appropriate data over
3. Process, manipulate, etc, to your hearts content

Scales to automated processes by allowing step 1 to be based on the current contents of both the original and copy.

来源：https://stackoverflow.com/questions/8874016/what-is-the-best-practice-for-data-conversion-between-applications