Project Selection Process S530

S530 - Evaluate Responses

DEFINITION

Receive and evaluate responses to the Invitation to Tender.

SUMMARY

Log and acknowledge tenders or proposals received from vendors in response to the Invitation to Tender (ITT).

Collate the information received for use in the evaluation process and in preparation for the production of the selection report.

Evaluate responses in terms of compliance with the key requirements.  Scoring techniques may be used where required, but it is generally recommended that they be avoided where a very fast approach is required.

Any omissions or queries may (at the discretion of the team) be referred back to the vendor for clarification.

PATH PLANNING GUIDANCE

This process is normal practice in Requirements / Selection Fast Track.

DEPENDENCIES

Prerequisites (Finish-Start):

• Issue Invitation to Tender (S520)

Dependent procedures (Finish-Finish):

• Confer with Vendors (S540);
• Consult with Reference Users (S550);
• Select preferred vendor (S560)

RECEIVABLES

• completed Requirement Matrices
• vendors’ responses

DELIVERABLES

• list of key requirements which cannot be met by the vendors
• assessment of other selection factors

TOOLS

• Hardware/Software Comparison Worksheet;
• Vendor Comparison Worksheet;
• Cost Comparison Worksheet;
• System Quality Chart;
• Vendor Quality Chart;
• System Quality/Vendor Quality Chart;
• Cost Summary Chart;
• Application Comparison Worksheet;
• Hardware Comparison Worksheet;
• Vendor Comparison Worksheet.

DETAILED DESCRIPTION OF TASKS

Receive and Acknowledge Responses

Some time after issuing the ITT it may be a good idea to make sure that the vendors have received the ITT are planning to respond to it.  As the deadline approaches it may be useful to contact the vendors and check that they will be able to meet the deadline.  In some cases it may be found that all vendors are struggling to meet the timescale and it might be appropriate to allow an extension.  Any changes in the timescale or other arrangements should be communicated to all vendors.

As the vendors submit their responses it is good practice to note their arrival and confirm this to the vendor.  Either a simple letter or a telephone call will normally suffice.  The team should keep note of which proposals have been received and who has them.  As the copies arrive they may be distributed to the personnel who will be reviewing them.

An initial check should be made on the format of the proposals to see that the basic rules have been followed and that it will be possible to assess them.  In a few cases, vendors will have failed to appreciate the need to comply with the required format and it is good practice to give them a second chance to comply before rejecting their submissions.

Review Proposals for Compliance with key Requirements

The purpose of the evaluation process is to identify requirements which could not be met satisfactorily by the respective solutions so that these areas could be investigated in more detail during vendor demonstrations.

The objective is to prepare a list of issues which might affect the selection decision.  Scoring can be relevant on these “Fast Track” selection exercises, but it is not compulsory and may be avoided in the interest of time.  Scoring is particularly relevant if there is difficulty in differentiating between solutions.  Where appropriate, the techniques described in the “Full Selection” approach may be used.

Collating the Information

It is usual to enter all the responses manually since the key requirements list used in Requirements / Selection Fast Track is generally brief.

It is more useful to create tables showing the comparison of key facts.  For example - hardware requirements, cost comparisons, vendors' capabilities.  Where specific tables were used in the Invitation to Tender the data collected may easily be tabulated (see the example layouts in Examples:  Request For Proposal and the example worksheets shown below).  This can be particularly useful regarding the costings of the various proposals.

The collected data may be grouped into tables showing the details of each vendor in separate columns.  The following examples are included in the toolkit:

Application Comparison Worksheet

Collates information from each vendor about the proposed applications.  Holds basic summary information such as release date and number of installations.

Hardware Comparison Worksheet

Collates information from each vendor about the proposed hardware configuration and operating environment.  This example assumes the hardware is to be procured for the new system.  A similar format could be used to collect facts relating to the degree of expected usage of an existing hardware configuration.

Vendor Comparison Worksheet

Collates basic information from each vendor about their organisation, references, technical support capabilities and technology infrastructure etc.  It gives a feel for the reliability of the vendor as a future business partner.

Cost Comparison Worksheet

Collates responses from each vendor about the expected costs for each alternative over five years.

Cost Summary Chart

Summarises and charts the estimated cost responses from the Cost Comparison Worksheet.

Dealing with Proposed Modifications

Vendors may volunteer solutions including proposed modifications to their standard products or including custom programmed add-ins. Such proposals should not be rejected out of hand, but should be assessed in terms of:

• Practicality - is it a practical option?
• Elapsed time - how long will it take, is there enough time, will it affect the delivery date?
• Staffing requirements - are there sufficient human resources to cope with the customisations and/or modifications?
• Risk - will the reliance on customisation unduly increase the project’s risks, e.g. risk of errors, risk of delays, risk of additional costs?
• Costs - what is the effect on the project’s costing and the overall cost/benefit analysis for the new system?
• Future support - will the changes or customised components make it difficult or expensive to maintain the system and, in particular, to upgrade to future releases of the packaged components?

Generally speaking, a solution relying on modification is less satisfactory than one where the requirements can be met within the standard facilities of the packages.  Due consideration should be given to these factors in the assessment.  The scoring and/or other comparisons may reflect a lower level of compliance but this will not result in a significant numerical difference.  Required modifications should be noted.  Their full effect should be included in the cost comparisons, and they should be discussed as specific issues during the final evaluations.

Dealing with Alternatives

Sometimes a vendor will offer alternative solutions or options for some of the requirement.  If possible, the preferred approach should be decided early and the full comparison made on that basis.  If necessary, the options may be assessed separately, but this will often mean treating the permutations of solutions as entirely separate proposals.

Make sure that options are not mixed up in the assessment.  For example, if the functionality was assessed on the basis of a specific option being included then the cost comparison should also be made on that basis.