Research Methodology

How an Overall Performance Rating is Calculated

Software & Medical Equipment

The overall performance rating is calculated from individual scores for each of 40* criteria. The 40 criteria consist of 28 Performance Indicators (PI), rated on a scale of 1-9, where 1=poor and 9=excellent, and 12 Business Indicators (BI) where the rating is a “yes” or “no”. The PIs account for 70 percent of the rating and the BI (scored on the percent of “yes” responses) account for the remaining 30 percent of the rating. Each question is weighted equally. The total score is based on a 100-point scale.

Key questions that are covered in software evaluations follow:

How long with product? Money's worth Would you buy it again Vendor is improving Vendor executive involvement Contract is complete Keeps all promises Product works as promoted Product has needed functionality System response time Quality of implementation staff Implementation within budget/cost Quality of telephone/web support Proactive service Quality of releases or updates Quality of custom work Technology easy to implement & support Quality and timeliness of interfaces Helps your job performance Recommend to a friend/peer

Lived up to expectations Worth the effort Part of long-term plan Delivery of new technology Contracting experience Avoids nickel and diming Good job selling Product quality rating Ease of use Quality of implementation Implementation on time Quality of training Support costs as expected Production errors addressed quickly Timely enhancement releases Quality of documentation Supports integration goals 3rd party prod. works w/ vendor prod. Overall satisfaction with vendor/product Average overall vendor ranking

Professional Service Firms

The overall performance rating is calculated from individual scores, each with equal weighting, from 17 indicators. Twelve (12) performance indicators, rated on a scale of 1-9 (where 1 = poor and 9 = excellent) account for 70.6 percent of the score. Five (5) “yes” or “no” business indicators (scored on percentage of “yes” responses) account for the remaining 29.4 percent of the score. The total score is based on a 100 point scale. Two additional scoping questions (which are not included in the total score) complete the evaluation, for a total of 19 criteria.

Key questions that are covered in professional service firm evaluations:

When was the project completed? Money's worth Would you buy it again Contracting experience Keeps all promises Service delivery on time Quality of services staff Helps your job performance Recommend to a friend/peer Average overall ranking

Lived up to expectations Worth the effort Executive involvement Avoids nickel and diming Services delivered as promoted Services delivered within budget/cost Real problem resolution Overall satisfaction Overall costs

Questions? Access Methodology FAQs