Data Migration Template

The Data Migration Template Job (Job: TMigrate) is the first in our task-specific template Jobs.

As well as using this template as the basis for your data migrations, you may also use it as the basis for other similar activity.

This Job has been designed to manage the orchestration of the following activity, and includes checkpointing. To use this template, you will need to install additional checkpointing Components. For more information on these, read this article on checkpointing.

It is recommended that you read the article Data Migration Framework, as this template Job has uses the concepts that are described in this Data Migration Framework.

  • Acquisition
  • Identification
  • Profiling
  • Quality Control
  • Preparation
  • Finalisation
  • Loading

Although these are orchestrated in this template as linear activity, it is likely that this will not be the case in your real-world data migration. You may rearrange these to suit your own requirements and it is likely that you will also have multiple phases for some of these activities.

The data migration template (Job: TMigrate) is available with Talend Framework version 1.6 and later, and is located in the following repository folder.

Job Designs/Framework/Template/Migrate

For installation testing, simply run the Job TMigrate. If you receive errors relating to missing checkpointing Components, read this article on checkpointing.

Job: TMigrateOrchestrate

As you may already know, the general structure of a framework Job is: –

Template->TemplateProcess->TemplateChild

If you are not yet familiar with creating a Talend Job using this framework, then I’d strongly recommend reading the article My First Framework Job, now.

The structure of the Data Migration Template is: –

TMigrate->TMigrateProcess->TMigrateOrchestrate->TMigrate*

As with most framework development, the only modifications that have been made to Job: TMigrate and Job: TMigrateProcess, is to define the child Jobs that they call. In all other respects, they are identical to Job: Template and Job: TemplateProcess, respectively.

Job: TMigrateOrchestrate, derived from Job: TemplateChild, is where things start to get more interesting as this Job has been designed to manage the overarching orchestration of the entire process.

The following screenshot shows Job: TMigrateOrchestrate. In essence, there are seven checkpointed child Job, that perform the seven migration tasks that have been previously described in this article.

Job: TMigrateOrchestrate
Job: TMigrateOrchestrate