Why Duplo
Used Techniques
The different possible techniques that can be used to transfer a data from one database table to another: File Transfer, Data Pumping, DB Replication and Synchronization
File Transfer
Even with the technological limits that characterize this technique, this is the most common method used for a long time, in fact it can be defined like the classical technique, used before other techniques were available. During file transfer process the data update event starts in consequence of a command and is therefore driven by a specific user run action or by a scheduled task. The database table is exported to a flat text file (usually performed by a procedure), transferred to the target file system and imported by another procedure. The most advanced file transfer methods are those in which only the variations occurred from the last transfer are loaded to the target system, with some more logic inside the procedures permitting the transfer of the varied data only.
Limits:
-
During every step the task is exposed to possible errors which are difficult to be tracked
-
The task is activated on command, and therefore doesn's react on specific events of the database
-
It is expensive because there is a need of writing three different procedures (extract, transfer, upload)
-
Maintenance is difficult. The different procedures and the possible variations of the information system, like new releases and upgrades, often causes problems
Today this technique is quite non more used in new projects, because it’s expensive and inefficient, but there is still a lot of installed applications working in this way.
Data pumping
It is the main method in data warehouse applications. This technique takes the data in a massive way and moves them massively to destination. Even in this case, the event takes place on command (and then is guided by a specific user action or by a scheduling).Main features:
-
it's able to transform and normalize the data during the transfer process (eg. from different anagraphical tables data are carried on a single denormalized table)
-
SQL language is used for data transformation
-
Replaces completely the data in the target table
-
Moves large amounts of data penalizing network performance
-
It is unidirectional
-
It is faster to implement than File Transfer.
-
Still requires considerable programming tasks to get to the final result
-
Requires dedicated tables during upgrade process
Limits:
Tools of date pumping can also be very sophisticated but they present a series of limitations:
-
when an ERP database needs to be updated it is not possible to update only the variations.
-
the data transformation is strongly linked to the programming activities of the tool, what requires the help of experienced staff to acheive the objectives.
DB Replication
Database replication is originated from the necessity to have a copy of the data in a different site than the company site. Many databases replicate tables only if they are equal and are replicated to the same database engine (such as SQL Server and SQL Server).
Limits:
This mode it is adequate to handle disaster recovery, but cannot be used for other kind of applications, for example, capture and apply the variations from one DB to another.
Synchronization
The data synchronization represents the best of technology as it combines the features of DB replication (applying variations) with those of Data Pump (features of data transformation). Synchronization allows in fact the in real time updating of a single record, becoming the ideal tool for ERP’s transactions and for mission-critical transactions.
As it works exclusively on modified records in real time, the tool can be used for any type of transaction, both management and data warehouse type.
As now, therefore, the choice of data synchronization technique is certainly the best in all areas of business applications, both from the point of view of performance to costs.
See also: