Framework Context Groups

The Talend Framework provides a number of Context Groups that allow you to interactive with the framework and control its operation.

Core Context Group

  • Framework
  • FileSystem
  • JobPersistence
  • SendMail (tSendMail support)
  • Orchestration
  • LibLoadCVMap

Extra Context Groups

Extra Context Groups are provided as examples, for configuring Talend Components. None if these are required by the core Talend Framework.

  • FTPConnection
  • MySQLSharedConnection
  • POP
  • SalesforceConnection
  • SOAP
  • SQLiteConnection

Context is a fundamental concept of Talend. If you are not yet familiar with Talend and Context, read this article, now.

Generally speaking, you do not need to change the default values that are provided within the Talend Repository. You would normally alter these values from within Context Files, as described in this article.

There are some exceptions to this rule, and these will be discussed in this article. An example of a value that you may wish to amend within the Talend Repository is the Base Directory. This is to allow you to easily specify an alternative Base Directory, for when you run Jobs from within the Talend Designer, usually when you are running Jobs within your development environment.

Context Group: Framework

The Framework Context Group includes a number of parameters that are fundamental to the operation of the framework.

ParameterDefault
Value
Description
frameworkThis value is set by the Framework during initialisation and should not be modified.
debugfalseThis flag allows you to switch debugging on, or off. This flag may be used by the Framework and your own code.
enableShowContexttrueWhen Context is read from a file, the values may be written to the console output. Any values where the key contains the pattern "password", will be masked.
confirmExecutionfalseIf you are running your Jobs through Design Studio, or from a Command Propmt in foreground, you may optionally request that the user confirms execution of the Job. A simple Yes/No Dialog Box will be displayed. You should not use this option, if your Job is to be run in background, or placed on a schedule.
confirmExecutionMessageYou are exeucting the Job %s in the %s Context.\n\nAre you sure you want to continue?You may choose your own message, when requesting that the user confirms execution of the Job (see confirmExecution). You must provide place holders for the parameters jobName and contextStr.
enableLockFiletrueA lock file is a simple mechanism that prevents two instances of the same Job, from running at the same time. You may use thiss etting inconjunction with any Job Scheduling software that you may be using.
enableErrorOnLockFileExiststrueIf a Job runs, but a lock file already exists (see enableLockFile),this will be considered an error. There may be circumstances where you want the Job to silently close down, without reporting an error condition. In these instances, you may set this value to false.
deleteLockFileInHandledErrorfalseUnder normal operation, a Lock File (see enableLockFile) will be deleted in a Handled Error. You may choos to delete these. The default CHILD_RETURN_CODE for your own Job (see TemplateChild) is 0=SUCCESS or 4=ERROR. If you want errors in your own Job to be considered a Handled Error, you must exit with a return code of 1-3.
isHeadlessfalseAllows you to specify if the Job is being run headless. This flag should be set to true when the Job contains User Input Dialogs that you want to disable for headless operation.
enableJobExecutionReportingtrueOn completion of Job, the library Job LibJobReporting will be executed. This will report on all monitored statistics. This report is presented as an Excel Spreadsheet. If you do not use Excel, you may view this using Open Source products such as Libre Office.
enableEmailErrorReportingOnErrortrueIf an error is encountered during execution of your Job, the framework is able to send an alert via Email. You will beed to define the correct SMTP Email settings (see SendMail).
attachExecutionReportOnErrortrue
enableEmailOnSuccessfalseIf your Job executes successfully, the framework is able to send an alert via Email. You will beed to define the correct SMTP Email settings (see SendMail).
attachExecutionReportOnSuccesstrue
localiseArchiveDirtrueSpecifies if the Archive Directory (see archiveDir) should be localised.

Files placed in a localised directory are usually retained for a period of time, to allow for historical reporting and analysis.

This directory may be automatically localised to the instance of a Job execution, by appending a sub-directory to the path, consisting of the Job Name, together with the date and time of execution, for example: -

.../archive/MyJob.20151021.102251
localiseReportingDirtrueSpecifies if the Reporting Directory (see archiveDir) should be localised.

Files placed in a localised directory are usually retained for a period of time, to allow for historical reporting and analysis.

This directory may be automatically localised to the instance of a Job execution, by appending a sub-directory to the path, consisting of the Job Name, together with the date and time of execution, for example: -

.../reporting/MyJob.20151021.102251
archiveDirRetentionDays-1Specifies the number of days that archived files should be retained for. A negative number indicates that there is no pruning.

If archive directories have been localised (see localiseArchiveDir), then deletion will occur at directory level, rather than deleting individual files)
reportingDirRetentionDays90Specifies the number of days that reporting files should be retained for. A negative number indicates that there is no pruning.

If reporting directories have been localised (see localiseReportingDir), then deletion will occur at directory level, rather than deleting individual files)
outputDirRetentionDays-1Specifies the number of days that output files should be retained for. A negative number indicates that there is no pruning.
workingDirRetentionDays90Specifies the number of days that working files should be retained for. A negative number indicates that there is no pruning.
tmpDirRetentionDays7Specifies the number of days that temporary files should be retained for. A negative number indicates that there is no pruning.
logDirRetentionDays30Specifies the number of days that log and statistics file should be retained for. A negative number indicates that there is no pruning.
processingErrorRecipientSpecifies a delimited list of Email recipients, for receiving processing errors. This list may be delimited by either a comma or semi-colon.
systemErrorRecipientSpecifies a delimited list of Email recipients, for receiving system errors. This list may be delimited by either a comma or semi-colon. If a recipient is not provided, system errors will be reported to SendMail.sendMailTo.

Context Group: FileSystem

ParameterDefault
Value
Description
baseDir#HOME/project/talend/#CONTEXTSTR/#PROJECTNAME/#JOBNAMEThe Base Directory identifies the location where all of the files related to your Job will be located. You may change individual locations, as required (see FileSystem Context Group). The tokens #HOME, #PROJECTNAME, #JOBNAME and #CONTEXTSTR will be replaced, at runtime, with an appropriate value. If you want to change the value of baseDir, you may set it in a Context File, or pass it as a runtime context parameter. You may also change this value in the Framework Context Group; however, you will need to re-apply this change, each time you import a new version of the framework.
archiveDir#BASEDIR/archiveSpecifies the default location for the archive directory.

This value is not used by the framework. It is a common directory name that you may use for your own requirements.

This directory may be automatically localised (see localiseArchiveDirectory) to the instance of a Job execution, by appending a sub-directory to the path, consisting of the Job Name, together with the date and time of execution, for example: -

.../archive/MyJob.20151021.102251
binDir#BASEDIR/binSpecifies the default location for the bin directory.

This value is not used by the framework. It is a common directory name that you may use for your own requirements.

If you are building your Jobs for command-line, or scheduled execution, then this directory is a suitable location.
contextDir#BASEDIR/contextThe Context Directory identifies the location that will contain the context file that is used at runtime. You may specify this value at runtime, by passing a new value as a context parameter. This value may not be specified in a context file. You may also change this value in the Framework Context Group; however, you will need to re-apply this change, each time you import a new version of the framework.

If it does not already exist, this directory will be automatically created, when you first run your Job.
databaseDir#BASEDIR/databaseSpecifies the default location for the database directory.

This value is not used by the framework. It is a common directory name that you may use for your own requirements.
gLibDir#HOME/project/talend/#CONTEXTSTR/libSpecifies the default location for the global library directory.

This value is not used by the framework. It is a common directory name that you may use for your own requirements.

If it does not already exist, this directory will be automatically created, when you first run your Job.

This is a general-purpose library directory that may be located outside of your Job's baseDir. This allows you to conveniently share library files between Projects and Jobs.
inputDir#BASEDIR/inputSpecifies the default location for the input directory.

This value is not used by the framework. It is a common directory name that you may use for your own requirements.

If it does not already exist, this directory will be automatically created, when you first run your Job.
libDir#BASEDIR/libSpecifies the default location for the library directory.

If it does not already exist, this directory will be automatically created, when you first run your Job.

This is a general-purpose library directory that is used by the framework; however you may also use for your own requirements.

Id you have enabled Job Reporting (see enableJobReporting), a default Job Reporting specification (JobReporting.xls) will be created in this directory.
lockDir#BASEDIR/lockSpecifies the default location forf the lock directory.

If it does not already exist, this directory will be automatically created, when you first run your Job.

The framework uses this directory for storing Lock Files (see enableLockFile). You also place Stop Files (see enablePersistence), should you wish to stop a Persistent Job.
logDir#BASEDIR/logSpecifies the default location for the log directory.

This value is not used by the framework. It is a common directory name that you may use for your own requirements.

If you are building your Jobs for command-line, or scheduled execution, then this directory is a suitable location for the log files.
outputDir#BASEDIR/outputSpecifies the default location for the output directory.

This value is not used by the framework. It is a common directory name that you may use for your own requirements.

If it does not already exist, this directory will be automatically created, when you first run your Job.
paramDir#BASEDIR/paramSpecifies the default location for the parameter directory.

This value is not used by the framework. It is a common directory name that you may use for your own requirements.
reportingDir#BASEDIR/reportingSpecifies the default location for the reporting directory.

If it does not already exist, this directory will be automatically created, when you first run your Job.

This is a general-purpose library directory that is used by the framework; however you may also use for your own requirements.

Id you have enabled Job Reporting (see enableJobReporting), timestamped reports will be written to this directory.

This directory may be automatically localised (see localiseArchiveDirectory) to the instance of a Job execution, by appending a sub-directory to the path, consisting of the Job Name, together with the date and time of execution, for example: -

.../reporting/MyJob.20151021.102251
statsDir#BASEDIR/statsThe Statistics Directory identifies the location that will contain the statistics files that are collected by Talend at runtime. You should specify this value, as described in the article on Project Properties. You may also specify this value at runtime, by passing a new value as a context parameter. This value may not be specified in a context file.
targetDir#BASEDIR/targetSpecifies the default location for the target directory.

This value is not used by the framework. It is a common directory name that you may use for your own requirements.
tmpDir#BASEDIR/tmpSpecifies the default location for the temporary directory.

This value is not used by the framework. It is a common directory name that you may use for your own requirements.
workingDir#BASEDIR/workingSpecifies the default location for the working directory.

This value is not used by the framework. It is a common directory name that you may use for your own requirements.

Context Group: JobPersistenceGroup

ParameterDefault
Value
Description
enablePersistencefalseNormally, your Job will run once, and then exit. If you want your Job to run persistently, set this value to TRUE. As well as having persistently running logic, you may also have, optional, run-once logic (see TalendProcess). A persistent Job will stop, if its execution takes it outside of its run-window (see runBetweenStart and runBetweenEnd). If you need to stop a persistent Job, you should place a stop file (see Stop Files) in the Lock Directory (see lockDir).
persistenceMaxIterations99999999Specifies the maximum number of iterations, for a persistent Job (see enablePersistence).
runBetweenStart00:00Used in conjuntion with runBetweenEnd, specifies the hours and minutes (HH:mm) between, which, the Job is permitted to run. The Job will always start and finish, in the usual manner, it is only the user-defined logic that will not execute. If Job Reporting is enabled (see enableJobReporting), reports will still be produced. If a Job is running in a persistent state (see enablePersistence), this run-window will be tested after each execution of the user-defined logic. If both of these values are set to "00:00", the Job may run at any time. You may use these setting inconjunction with any Job Scheduling software that you may be using.
runBetweenEnd00:00See runBetweenStart
persistenceSleepBetweenExecutions30If a Job is running persistently, this parameter specifies the number of seconds that the Job should sleep, on successful completion of executing the user-defined logic.