How to configure Database connection for production environment in Pentaho data integration Kettle transformation

Question

I designed a ktr file for transformation. I need to configure the database connection details of production environment. How can I do this? Any suggestions?

Answer 1

I use environment variables.

KETTLE_HOME
KETTLE_JNDI_ROOT
PATH=$PATH:$KETTLE_HOME

Kettle home is just a link to directory. By default i have directory specially devoted to data-integration suite. It contains several versions of kettle.

Example

/opt/kettle/data-integration-4.4.0 (few old jobs made like several years ago)
/opt/kettle/data-integration-5.2.0 (currenly used)
/opt/kettle/data-integration-6.0.0 (on test)

then there is a link to current default (something like alternatives in debian). It helps to have several versions of kettle on same machine.

ln -s /opt/kettle/data-integration-5.2.0 /opt/kettle/data-integration

now

/opt/kettle/data-integration 

is a link to main suite which will be used by default. This is made for jdk compbility reasons. Since 4.4.0 is jdk6 based, 5.2.0 jdk7 (and doesn't work on jdk8 since there is a bug in xstream xml library, but may be fixed already, 6.0 branch works well on jdk8 even it is built on jdk7)

Variables configured in /etc/profile. File jdbc.properties is located in

$KETTLE_HOME/simple-jndi 

and shared by all jobs. This file, with connection settings, generated by puppet if network configuration changes. If administrators do something, puppet automatically will make new file in case of migrations.

For connection definitions I use naming rules

 main       -connection to productive database
 main_slave -connection to slave database (read-only)
 test       -connection to test database (on separate machine)
 test_slave

and so on ...