Project Delivery Process D604

D604 - Finalise Data Conversion Design

DEFINITION

Finalise the design of the extraction, validation, control and input systems for data conversion.

SUMMARY

The purpose of this task is to finalise design and development of automated procedures to manage the conversion of data from the form in which it currently exists to the form required by the new application.

PATH PLANNING GUIDANCE

This process is optional and has been written for existing Application Software Implementations where existing data will be converted by automated methods.

DEPENDENCIES

Prerequisites (Finish-Finish):

• Data Conversion Strategy  (D180)
• Dependent procedures (Finish-Finish):
• Instigate non-Application Software parallel tasks (D650) - where programming work will be outside the scope of the project team
• Modify existing non-Business application software (D656) - for components of the work not using Application Software facilities.
• Modify package software (D658) - for components of the work using Application Software facilities.

Dependent procedures (Finish-Start):

• Build of corresponding programs / modules etc

RECEIVABLES

• Definition of Requirements (DoR)
• Delivery Approach Definition (DAD) or similar definition of conceptual design
• Technical Plan IP - see process D110Data Model - see process D170
• Data Conversion Strategy IP - see process D180.
• •Documentation for the other application(s) involved
• Site standards and procedures
• Business enterprise model
• Business process specifications

DELIVERABLES

• Conversion design - Technical Implementation Paper per conversion

TOOLS

• Application Development Standards - Coding Standards
• Application Development Standards - Naming Standards
• Application Development Standards - File Definition Standards
• Skeleton Implementation Paper
• Guidelines: “Modelling Techniques”
• Guidelines: “Custom Development”
• IP Guidelines: “IP-Data Conversion”
• IP Guidelines: “IP-Database”
• IP Guidelines: “IP-Languages”
• IP Guidelines: “IP-Handover”
• Examples: “Interfaces and output files worksheet”
• Examples: “Conversion Worksheet”
• Guidelines: “Application Software development utilities”

DETAILED DESCRIPTION OF TASKS

Requirements

The requirements for data conversion may be found in the earlier requirements documents, conceptual design (Delivery Approach Definition), the Data Conversion Strategy IP (see Process D180), or areas of specific functional design during the design/prototyping processes (see Process D400).  These needs should be confirmed and reviewed as appropriate to ensure that the final details are frozen for the build processes.  Note that prototyping work is often continuing at this time, so it can be important to check that the conversion requirements are stable before finalising the design.

Remember that old data is almost NEVER:

• complete
• of good consistent quality,
• in the right format,
• in the right sequence,
• using the right (new) codes,
• one old record for one new record,
• without duplicates
• readable directly without a new extraction program.

A large part of the requirement may be to extract, manipulate, validate, correct, sort and control the data.

Options

Full options should be considered, for example:

• custom built programs
• use of package facilities
• use of interfacing tools
• use of manual re-entry.

Consider and report on the relative merits of these approaches, particularly in terms of costs, benefits, lead times, resource requirements, quality and risk.  Remember that the conversion routines may only be used one time, and it may be legitimate to contemplate lower standards, lower quality and higher risks provided the resulting data load is reviewed for accuracy and manual corrections are allowed.

Recommendation

Agree the preferred course of action for each data conversion with the responsible managers in the client organisation (and/or external bodies where appropriate).

Detail of conversion design (where automated techniques are to be used)

Initial data conversions are one-time data loads into the Application Software that do the initial data population of the Application Software data from legacy systems.  Techniques may be identical to those used in permanent interfaces - see Process D602 for guidance on interface design options.

Data extraction may often be performed using features of the old system or defined in the same language as the old systems as it will be necessary to access the existing data, preferably using standard routines.   Validation, correction, sorting, reformatting, translation and control functions may be performed using whichever tools and techniques are most appropriate.  Note that Consultants may have access to certain conversion and interfacing tools.

Batch Input Program Processing Options

Run Mode	Processing Dispatched To	Runtime Limit	Speed	Use
Process / foreground	Dialog Process	5 minutes (clock time)	Slowest	Use during development & debugging
Display errors only	Dialog Process	5 minutes (clock time)	Next Fastest	Use during development
Background	Background Process	No limit. Jobs run until they finish.	Fastest	Use during development & unit test

The following performance improvement activities should be initiated:

• Identify and verify Application Software configuration modifications to optimise performance of initial data conversions.
• Verify server configuration modifications to optimise performance of initial data conversions. Note that modifications may be possible on a one-off basis, just while the conversion is being run.  Associated pros and cons should be considered before deciding to implement:
• Increasing memory cache
• Turning off logging during loads
• Pre-sorting data loads
• Dropping indexes before loads
• Enlarging the redo log buffers files
• Physically locating tables with many updates or inserts on separate disks