Project Delivery Process D180

D180 - Data Conversion Strategy

DEFINITION

Analyse the needs for existing data to be converted and the availability of that data, its integrity, needs for supplementing or improving the data etc.  Based upon this, define and agree an approach to data conversion.

SUMMARY

Data from existing systems or business processes is usually required to be converted and transferred to the new system.  The needs for this should be apparent from a review of the overall implementation strategy defined in the Delivery Approach Definition (DAD).  If appropriate, further investigation may be performed to identify the needs for data to be converted.

Data availability and integrity is often wrongly assumed during design work.  In fact, existing data is rarely of good quality and often is insufficient for the needs of the new system.  A study should be made of the availability and integrity of the data.  Recommendations will be made for securing data as required.

PATH PLANNING GUIDANCE

Optional - may be used where existing data will be brought into the new system.

DEPENDENCIES

Prerequisites (Finish-Start):

• Delivery Approach Definition (DAD) or similar high-level definition of the design

Prerequisites (Finish-Finish):

• Definition of the requirements for data in the new system

Dependent procedures (Finish-Start):

• Design and development of conversion and interfacing programs (in Processes D600/D604 and D650/D656/D658)

RECEIVABLES

• Delivery Approach Definition (DAD) or similar definition of high-level design
• Current  Data Dictionary (if any)
• Data dictionary, data definitions or data models supplied with the package (if any)

DELIVERABLES

• Data Conversion Strategy IP (or BIP)
• Revised Segment Plan (if appropriate)

TOOLS

•        None

DETAILED DESCRIPTION OF TASKS

Defining the approach

An Implementation Paper (IP) or Brief Implementation Paper (BIP) should be used to define the requirements, issues and agreed approach.  Where the project is complex with different departments being involved, an IP is to be preferred as it allows full consideration and acceptance of the issues and the decisions.

The Implementation Paper may identify the:

• Requirements for data - data items, quality / accuracy required, extent to which items are mandatory
• Options available, e.g. various sources, ways of supplementing the data, ways of validating and correcting data, ways of converting and/or loading the data etc
• Recommended approach
• Details of how to implement that approach

Requirements and issues

A study may be required to identify key items of data required to meet the various functional requirements and designs.  Very often, this evaluation will need to take place in parallel with the detailed design of the functions.

The study will note the required functional content of the information, requirements for detail and/or summarisation, requirements for timing, likely average/peak volumes, priorities, importance, etc.  The study would normally be based upon information already established during the Requirements and Selection Segment, eg the corporate data model, detailed requirements, the System Vision, the Delivery Approach Definition (DAD), etc.  It is, however, common to find that requirements become further detailed or amended as a result of detailed design tasks.  Accordingly, detail should be kept at a high level unless vital to the conversion strategy.

The requirements should include the extent to which accuracy and completeness of data is necessary.  For example, historic management information may not need to be totally accurate, but the records of employees’ salaries must be absolutely correct.

These requirements for availability, accuracy and completeness of source data should be compared with the actual availability and condition of the data.  This will lead to the identification of requirements to improve, modify or supplement the existing data.

Options

Options to address the requirements for complete accurate data should be established.  These may include:

• Currently available data is acceptable
• Manual validation and correction
• by the user departments
• using outside clerical assistance
• Automated validation / manual correction
• by the user departments
• using outside clerical assistance
• Automated correction (eg postal codes)
• Ignore current data and collect new data from scratch (eg full inventory of assets)
• by the user departments
• using outside clerical assistance
• Convert / supplement data using tables etc.

Options to convert or load the data should be established.  These may include:

• Automated conversion using custom programs
• developed by the project team
• developed by the user organisation’s MIS department
• developed by the package vendor
• developed by a third party contractor
• Automated conversion using package facilities
• Manual data entry using the package
• by the user departments
• using outside clerical assistance
• Manual data entry in some other computer system followed by automated transfer
• by the user departments
• using outside clerical assistance

Recommended Approach

The benefits, costs, risks and timescales for the different options are reviewed and an approach is agreed.  Details of the overall approach will be given.   In many cases these will lead to additional custom programming and/or clerical work.

General details of how the data will be obtained, corrected, supplemented and converted should be stated.  This may include full details of manual procedures.  Detailed technical requirements should be made the subject of separate Technical Implementation Papers - see Processes D600 and/or D604.  The development, testing and implementation of these will then be undertaken - see Process D650 and/or D656/D658.  Where the recommended process is entirely manual, it may subsequently be considered and detailed in a normal Implementation Paper if further detail is necessary - see process D400 or D450.

The overall data conversion strategy should be reviewed and signed off by the project’s sponsor and other relevant managers within the client organisation.

Where necessary, appropriate changes should be made and agreed for the Segment Plan.

Project Delivery Process D170

D170 - Data Modelling

DEFINITION

Build and/or maintain data dictionary and/or data model.

SUMMARY

A data model may be produced to show the data and the relationships between the various data entities involved in the business solution.  This may be at a high summary level, or at a detailed level depending on the wishes of the client organisation and the needs of the project team.  Some or all of the model may already exist as a result of:

• work done during the requirements and selection segments,
• the client organisation’s pre-existing enterprise data models, or
• standard models reflecting the structure of the chosen packages.

In general Application Software Implementation Projects do not require that a full data model be created. Most modern packages have internal data dictionaries sufficient for their own needs, and several have published data models electronically or on paper.  This process may be required:

• to check whether the chosen package will provide the needed coverage of data items to fulfill the client organisation’s needs,
• to define which possible amendments have to be made to the standard model for the package,
• where there is an external need for a data model or data dictionary, e.g. because the data will be integrated in a corporate database, or
• simply because it is normal practice within the organisation.

Normally a formal approach would be used, either using the approach as used within the specific package, or a general approach depending on the organisation’s normal practice.

PATH PLANNING GUIDANCE

Optional - may be used where required by the client, or where the technical solution is part of a larger database, or where an enterprise data model is in use.

DEPENDENCIES

Prerequisites (Finish-Start):

• Delivery Approach Definition (DAD) or similar high-level definition of the design.

Prerequisites (Finish-Finish):

• all data-related aspects of the design should be completed before the data model can be considered fixed and final.

Dependent procedures (Start-Start):

• design of data-related aspects of the system should be performed with the benefit of the provisional data model.

RECEIVABLES

• Delivery Approach Definition (DAD) or similar definition of high-level design
• Current corporate/enterprise Data Model or Data Dictionary (if any)
• Data dictionary, data definitions or data models supplied with the package (if any)

DELIVERABLES

• Data Model (DM) - new or updated - as required
• Data Dictionary (DD) - new or updated - as required
• Updates to the package’s standard data model facility
• Statement of differences in standard package model to client organisation’s desired model

TOOLS

• Guidelines:  Modelling Techniques

DETAILED DESCRIPTION OF TASKS

Using a data model or data dictionary

Data models and data dictionaries are methods used to record the precise nature of data (including its format, use, ownership etc) and the relationship between items of data.  They are frequently used in custom-developed systems, particularly where the application uses a database.  Having a clear, controlled definition of the corporate data is essential to ensure the coherence and integrity of parts of a system.  Where integrated solutions are used, they are of even greater value in ensuring the coherence of all the individual components.

The exact format of a data model or data dictionary will depend on its origin, however, typically, data models are shown in the form of “entities” and “relationships” in an “Entity/Relationship” model otherwise known as a Bachmann diagram after its originator.  Most methodologies use a version of this style of diagram, although they will normally show cosmetic differences and be known by different names.  A data dictionary is more frequently in the format of a table in alpha-numeric sequence listing all recorded types of data plus the data attributes.  It can normally be used to introduce data definitions automatically into programs and other accesses to data.

In a simple package-focused implementation the data design of the application is often fixed at a technical level and straightforward at the business function level.  It is often of limited benefit to produce or use data models or data dictionaries in such cases.  It is important that the Project Team does not waste time in data modelling where there will be no particular benefit derived from it.

It may be important to check whether “knockout” mandatory and business critical requirements can be fulfilled with a specific package.  The needs for specific data should be looked at carefully to ensure that they result from actual business needs and not just from current business practices or arbitrary aspects of data analysis and design.

In some packages the data model used in the application can be customised individually to the client organisation’s needs.  The package will then provide an integrated environment to maintain data definitions and specify at which places data will be used.  By using a data dictionary, individual amendments can be added to the standard data model provided.

Reasons for using data modelling techniques may include:

• required by the client due to internal standards,
• the package has to be customised to provide the essential functionality,
• where the overall solution will be formed by several integrated components, only some of which are packages,
• where the technical solution uses, or is part of, a larger corporate database and overall control must be maintained,
• where an enterprise data model is in use to maintain a high-level view of how the organisation is composed in terms of data.

Levels of modelling

There are different levels (ie degrees of detail) at which data modelling may be performed.  Different levels serve different purposes.

High-level data models concentrating on key corporate data are used for high-level business process design.  They allow an overall view of the organisation to be described in terms of its data.  

Detailed data models may show every aspect of every item of data.  This is unlikely to be of benefit unless required for the integration of databases or for detailed programming purposes.

Data models can be used to show the current picture, ie the starting point for the project,  or to document the designs for the new system.

Data dictionaries normally contain detailed definitions of data format and usage.  They allow the precise definition of how data is accessed and used.  They can be used both as a design documentation aid and as a development aid as they will normally be used automatically by the software, saving coding time and ensuring consistent definition of the data structures.  For example, a data dictionary provides data definitions for:

• table structures
• single data items like data fields including a separate help text which explains the meaning and technical format of the data item,
• domains which contain common characteristics of a group of data items (eg all cost centres have common characteristics).

Method

Where data modelling is to be used, there will normally be some existing techniques already in use - either within the client organisation or as used by the package supplier.  Accordingly, the best method will normally be decided by the Project Team based upon the approach most beneficial and/or appropriate for the Project.  Such decisions should be agreed with the client organisation, especially the MIS department, preferably before the project starts as part of the Path Planning / Segment Planning processes.

The techniques used will normally be drawn from a recognised systems analysis methodology such as SSADM (the UK’s government standard approach), Merise (the French standard approach), or a package-specific approach.

Typical uses of a data model or data dictionary during the Application Software Implementation Project  process might be:

Requirements

• as a record of the structure of the existing systems and data
• to document how data needs to be structured in the new system (nb - this should be restricted to genuine logical needs and not be used to specify specific physical design approaches)

Selection

• to compare the data structures of the proposed solutions against the client organisation’s needs

Implementation Strategy and Delivery Approach Definition (DAD)

• High-level definition of existing corporate data model
• High-level definition of required data model

During a design and development

• Analysis of existing systems
• Design of target system(s) including integration / relationships between systems
• Documentation of designs / database etc
• For use in freestanding enquiry facilities or reporting tools (eg SQL enquiry)

Project Delivery Process D167

D167 - Consider and Review Needs for Custom Development / Modifications

DEFINITION

Consider and review needs for custom development or custom modifications to the package software or to other applications.

SUMMARY

Custom development or custom modifications to the package software or to other applications may provide a way to extend the functionality of the basic package to meet the business  needs.  Such development often represents higher costs and risks, coupled with extended timescales.  The full need is reviewed and the options are assessed.  Recommendations are formulated and agreed.

PATH PLANNING GUIDANCE

This process is optional.  It is used where custom development may be required.

DEPENDENCIES

Prerequisites (Finish-Finish):

• Implementation Strategy - Process S700 / S710
• Approach to integration - Process D160
• Data Conversion Strategy - Process D180

Prerequisites (Finish-Finish):

• Create detailed specifications - processes D600, D602, D604, D606

RECEIVABLES

• Delivery Approach Definition (DAD)
• Approach to integration IP (D160)
• Data Conversion Strategy IP (D180)

DELIVERABLES

• Use of Custom Development IP

TOOLS

• (none)

DETAILED DESCRIPTION OF TASKS

Custom development may be used to fill gaps in the functionality of the package software when addressing the business needs.  This process provides an initial examination of the needs and considers the options which may be available.  The advantages and disadvantages of each approach are evaluated leading to recommended approaches.  These would then be discussed and agreed with the client organisation.  The findings are presented in the form of an implementation paper.

It should always be a first preference to avoid the use of custom development.  There are several reasons for this:

• Use of custom development techniques invalidates the “parallel approach” assumptions.  The great efficiencies of the IDDI (Integrated Design Development and Implementation) approach cannot be taken advantage of.
• Custom development often takes a long time and can delay the project.  It may require the services or co-operation of bodies outside the project team, eg
• the package supplier
• the client organisation’s IT department
• development team’s for other software applications concerned.

In addition, custom modifications to the package software have certain significant drawbacks:

• The supplier is unlikely to provide full support for the application unless a fault can be reproduced on an unaltered version of the software
• Bug fixes may clash with the changes made by the project
• Documentation and help information may not match the way the system works
• Upgrades can be very difficult to accommodate.  Often it is necessary to re-analyse and repeat all the changes applied to the earlier system.

Where custom work is vital, an option to consider might be to contract the package supplier to undertake the work and to incorporate it in future releases.  This can substantially reduce the current and future risk.

There will be options regarding how the custom work can be undertaken.  For example, modifications would preferably be made using “user exits” in the package modules and using the package’s in-built language facilities. Most packages also have version control systems which should be used to ensure that the custom changes are controlled and maintained properly.

The  recommendation will be formulated on the basis of the projected costs and risks in comparison with the business benefit that can be achieved by conducting the work.  This is essentially a business decision which should be taken by the client organisation.