Project Selection Process S200

S200 - Identify Issues

DEFINITION

Identify, investigate and review the key issues which differentiate between the various vendors’ proposals received.

SUMMARY

This process continues throughout the evaluation process.  It commences with an initial evaluation of the main issues in the vendors’ responses.  All business critical areas where zero scores were achieved are listed.  Any other issues or doubts arising from the response are also listed.  These issues are logged and discussed within the selection team.  As the evaluation processes proceed, issues are used to identify areas requiring further investigation.  The outstanding issues will be a major factor in the final evaluation of the options.

The zero scores list and the issues list are used for:

• identification of significant problems with proposed solutions,
• identification of key differences between the vendors’ proposals,
• determination of items to be discussed during contacts with the vendor and reference users,
• identification of further actions required,
• discussion of differentiating considerations during the final evaluation,
• preparation of the selection report.

PATH PLANNING GUIDANCE

This process is optional - it is common practice unless selection is simple and a small team is involved.

DEPENDENCIES

This process normally runs throughout the evaluation of tenders received in response to the Invitation to Tender (ITT).

Concurrent procedures:

• Receipt of tenders (S190)
• Confer with Vendors (S210)
• Consult with Reference Users (S220)
• Package Fit Tests (S180)
• Agree final marks and issues (S230)

RECEIVABLES

• information concerning tenders and their fit with the organisation’s needs

DELIVERABLES

• Issues lists / logs
• Zero scores list

TOOLS

• Examples: Selection Issues List
• Examples: Zero Scores List

DETAILED DESCRIPTION OF TASKS

It is valuable to focus on key issues when evaluating the competing proposals.  Although various trends may be apparent, it is often these issues which prove to be the real differentiators.  Throughout the evaluation processes, issues should be noted as they are identified.  Where appropriate, actions can be targeted to explore these issues further.

There are two types of issues list kept:

• Zero Scores List - notes all “business critical” requirements which cannot be met (and are by definition a significant issue regarding that proposed solution).
• Selection Issues List - a list or log of all other issues as they arise.

Zero Score List

Three classes of criticality may have been identified (see Process S070):

• Knockout - if one of these cannot be met the entire proposal will have been rejected as non-compliant,
• Business Critical - if one of these cannot be met it will have a severe impact on the value of the proposed system and/or may present difficulties which require alternative approaches,
• Desirable - these are requirements which are of genuine value but which would not result in severe problems.

Based on this definition, proposals with “Knockout” requirements should not have reached the detailed evaluation stage.  “Desirable” requirements are relevant only to the overall scoring and evaluation of the proposals.  Only “Business Critical” requirements present key issues that merit individual consideration.

During the evaluation of the vendors’ responses a separate list is made to highlight all business critical requirements which cannot be met - ie which have been given a zero score.  This list will be reviewed and discussed by the full selection team.  Where appropriate, an analysis will  be added showing the potential impact of the issue along with any costs or recommended alternative approaches.  If the facts need further investigation or the alternative course needs to be investigated, specific actions may be identified.

The Zero Score List will form an important part of the final evaluation discussions.  In effect these issues will be double counted in the process as they count towards the quantified compliance scores and are also discussed as individual issues.  To help in this evaluation, it may be possible to categorise their severity.

The precise format may be varied to meet specific needs.  It is not necessary to have an individual form to represent each item on the list.

Selection Issues List

Any other significant issues are logged as they arise.  Some may arise directly from the evaluation of the tenders but not relate to a “Zero Score”.  Others will arise during the course of the evaluation, from discussions within the team, from reference user site visits, form the vendors’ demonstrations etc.

Where appropriate an analysis should be added to state the expected impact and costs relating to the issue and to recommend a course of action.  Actions may also be defined to attempt to clarify the issue.

Examples of selection issues and the actions which might be taken include:

Issue	Action
Doubt regarding specific answers in the proposal	Confirm response with vendor and get them proved at the demonstration; get corroboration from reference users.
Proposed alternative approach to a requirement	Review whether the alternative would reasonably meet the underlying need.
Use of customised add on or modifications	Investigate the full implications of accepting a non-standard solution
Incomplete or insufficient information available	Requests for further information or answers; analyse manuals; interview experts
Specific strengths or weaknesses identified	Get these highlighted in questions and answers during vendor and reference user contacts
Lack of understanding of specific parts of the proposed approach	Get these aspects covered in the demonstrations

As the evaluation processes proceed, issues are used to identify areas requiring further investigation. The identified issues should be used in setting the agenda for contacts with the vendors and with reference users.

The outstanding issues will be a major factor in the final evaluation of the options.  The list is used to identify the significant remaining problems with each solution and thus to identify key differentiators between the competing solutions.  These would be fully considered by the evaluation team in making the final decision.  The issues would also be used in the preparation of the final report.

It is possible to use the generic issues resolution mechanism that has been defined as part of the project’s management and control.  Such a method is, however, not directly designed for this purpose and tends to be unnecessarily complex.  The requirement can normally be met by the maintenance of a simple table of selection issues without individual control forms and without the tracking of follow-up activities.