Package Evaluation & Selection Phase

In this phase, requirements for the package are formally identified, based on the system analysis. Using the requirements document, the field of candidates is narrowed to five or less, and the selected candidates are brought in for unscripted demos. Finalists are selected based on the unscripted demos.

After the finalists are selected, an assessment document is created that will be used to assess the finalists, requests for information are sent to the vendors, and scripts or scenarios for the final demonstrations are created. The demonstrations are held, and the assessments are collated and weighed against the requirements. At the end of this phase, a recommendation is made on which package, if any, should be purchased.

The primary participants in this phase are the core team identified in the Preliminary Planning Phase, and any extended team members with the required knowledge. From the business area, there should be representatives of both management and staff, since each brings unique insights to the evaluation. From the technical area, there should be expertise in application development, architecture, data administration, networking, and so on.

Deliverables

Requirements Document This is a list of the package requirements based on the results of the system analysis. Neither sample nor template is available yet.
Assessment Report This document is a form that will be used to evaluate each package during the scripted demo. Neither sample nor template is available yet.
Demonstration Scripts/Scenarios These documents list the specific functions or tasks that must be included in the final demonstration by each participating vendor. Neither sample nor template is available yet.
Methodology Compliance Form This form is initialized by the project team, and completed by a methodology representative who has reviewed the project documentation and found it acceptable. It is completed in Word. At this time, no sample is available.
Package Recommendation This form is completed by the project team, and contains the recommendation for the package to be purchased. The form is completed in Word. ) At this time, no sample is available.
Recommendation Form This form is completed by the project team, and contains the recommendation to the approving authority on whether or not the project should continue. The form is completed in Word. At this time, no sample is available.

Steps

1. The project team reviews the documentation from the previous phases.
2. The team creates a requirements document, detailing what the users need from the system, as well as other, more technical requirements for the system.
3. The team creates an assessment form to be used in evaluating each package that is demonstrated. The assessment form should include space for comments, as well as a Has/Doesn't Have checkbox for each item to be evaluated. It is also useful to indicate whether each item is required or simply would be nice to have. Keep in mind that some questions should address how easy it is to work with the vendor, how good is the support they offer, and whether they will provide contacts for recommendations.
4. The team schedules unscripted demos for each package selected in the Market Research phase. If significant new information has been discovered since the initial demo of the market leader, the team may want to call that vendor back for a second demo.
5. The team evaluates each package, based on the assessment form, and determines which packages are to be called back for scripted demos. It might be useful to create an Excel spreadsheet for this evaluation.
6. Based on the evaluations, the team (or a designated member of the team) writes an assessment report, which includes the pros and cons of each package and the recommendation for finalists.
7. Based on the requirements document and any new information that came out of the unscripted demos, the team writes and sends a request for information (RFI) or request for proposal (RFP) to each of the finalists.
8. The entire extended team meets to determine which functions should be included in the scripted demos. Smaller subgroups can then write the scripts, being sure to capture all the business steps in the function. The scripts should also try to bring out any underlying issues with data storage, if possible.
9. The scripted demos are scheduled with the vendor, who is advised to expect the script, and the scripts are sent to the vendor contact. All the business members of the team should attend each demo, and the technical members of the team should be well represented as well. It is also useful to invite the executive sponsor, project sponsor, and project steering committee to the demos.
10. You can use the existing assessment form to evaluate the scripted demos, however, it might be useful to create a new assessment form for the scripted demos -- the vendors should be showing the same information, and you can get more detailed in your requirements here. Whether the assessment form is new or old, each team member should fill out an assessment form for each scripted demo.
11. After the final scripted demo, the team meets to review the packages. The assessment forms should be used for this, and an Excel spreadsheet to evaluate the differences might be useful. It is also wise to leave room in the assessment process for instinct and gut feelings. When the team has determined which package they think is best (most closely aligned with the requirements), they complete the package recommendation form.

    Note: If the team determines that none of the packages provide the necessary functionality, the team can recommend that no package be purchased. The team must then also recommend whether the preferred option is to build a system (in-house or off-site), or make do without a new system.

12. The project team forwards the deliverables to the methodology representative for review. The methodology representative reviews the documents for compliance, and completes the compliance form. If the deliverables conform to methodology requirements, they are returned with the completed compliance form to the team, which then proceeds with the next step.

If the deliverables do not conform to the methodology requirements, the deliverables and the compliance form are returned to the team with an explanation of what is wrong. The team then corrects the deliverables as required and returns them to the methodology representative for a follow-up review. When the reviewer approves the deliverables, the team continues with the next step.

13. The project team writes a recommendation indicating whether they think the project should continue or not, using the recommendation form. The recommendation form is attached to the updated project plan and any other project documentation (including the package recommendation form), and forwarded to the approval authority for review.
14. The approval authority decides whether or not to move forward with the project.