A Guide to Simplifying InfoSphere DataStage Change Management
Managing the change of DataStage components can often become a test of wills. Picture fitting a square peg in a round hole, and you might start to get the idea. The complexity of what is required to promote components from one environment to another during the development, testing, and eventual implementation in the production environments, should be and is often a controlled activity.
Ideal for those involved in the development, administration, and change management of the DataStage environment and programs, this paper provides guidance on simplifying the DataStage change management process and helps you understand what's required to promote DataStage objects from one environment to another, or in the worst case, disaster recovery.
The purpose of this white paper is to provide some guidance for simplifying the DataStage change management process while demystifying what is required to promote DataStage objects from one environment to another, or in the worst case, disaster recovery. DataStage project organization and design methodologies will also be discussed. The actual nuts and bolts of how the tools function are contained in courses and product documentation.
DataStage change management-the management and movement of programs, scripts, binaries, stored procedures, shared objects, environment settings, etc.-is nothing new and is essential to all software processes. Through a lack of understanding, managing the change of DataStage components can often become a test of wills. Picture fitting a square peg in a round hole, and you might start to get the idea. The complexity of what is required to promote components from one environment to another during the development, testing, and eventual implementation in the production environments, should be and is often a controlled activity. The old saying, “Without control there is little need for performance,” applies here.
The responsibility of promoting upgraded or new software components is often accomplished by teams that have this specific responsibility. Part of the purpose for having a change management engineer or team is to enforce standards and make development teams think early on about how to simplify the process of migration and configuration, and in the worst case, rollback. Although developers often play a role, the goal is for the process to be repeatable by others in the organization. Developers should not be involved in this process unless troubleshooting is required. Lack of understanding or knowledge about the components and the breadth of a DataStage installation stretch the abilities of many change management models.
The audience for this paper is anyone involved in the development, administration, and change management of the DataStage environment and programs.
The DataStage Import and Export Process
Historically the DataStage import and export happened via one mechanism, the DataStage client, where import or export was most often interactive. Prior to Version 8.0, the first IBM release, import and export occurred in the DataStage Manager client; today it is the DataStage Designer client. Many thought this odd since DataStage is a server-centric application. The engine sits on a big computer somewhere, often UNIX, sometimes Windows, and now increasingly on Linux. Development is done on a Windows client that communicates with the engine. Most client management teams weren’t fond of this if they were administering an instance other than Windows. And the fact that this was interactive was of great concern. A command-line utility was developed. While better, it was still on the Windows client. This endured for quite some time despite constant complaints by consultants and administrators about the shortcoming.
A new and improved service called the DSXImportService now allows you to script an import on the server regardless of platform. So, things are beginning to move in the right direction, which has made people familiar with this service much happier. In summary, a completed DataStage component(s) can be exported to a file, typically known as a dsx file. This file is a snapshot of the component that can be promoted to another project on the same or other server. It is not part of any source code control system, but it can be imported into a source code control system of your choice since it is a text file. This doesn’t give you check-in and checkout ability from the development environment, but at least files are safe from change. Version control with check-in and checkout ability has not always been available to the DataStage development environment, but it is now with the support of Eclipse workspace-based source code control systems such as ClearCase or Rational Team Concert. These are used with the Information Server Manager with source code control integration on the developers’ workstation.