With KLAS Alert you will be the first to know the latest information on the performance of your healthcare IT products.
On behalf of the KLAS team, I want to introduce you to KLAS Alert, a faster way to communicate performance to the market. Many of our provider partners have asked for more real-time analysis. KLAS Alert will provide the platform for sharing new, critical trends in the healthcare IT market.
Every month KLAS will scan the more than 25,000 surveys collected annually and alert the market to significant findings. These alerts will be based on specified parameters as outlined in the alert criteria section below. Each possible alert is reviewed and clarified before our monthly publication.
KLAS Alert will have a phased rollout that will begin with surveillance on the most requested products. Within six months, all software products will be under continual surveillance, with new alerts publishing online the 15th of every month.
We cheer those positive alerts that will be coming and hope to share challenges earlier, allowing for quicker detection and action.
Adam Gale
President, KLAS
KLAS Alert Examples
Product Improvements Push Scores Up: Product improvements with successive versions have helped to improve ratings among Vendor X customers that KLAS interviewed over the past 12 months, driving the vendor's overall KLAS rating from a 79 to an 83. One provider told KLAS: "The upgrades do seem to include enhanced features and not just upgrades for upgrade's sake. The upgrades bring with them things that we want, which has been good because it has brought on things that we really needed and wanted. Vendor X is doing a good job on the development side of things. Our task is to now keep things up to date over at our site and make the upgrades work and fit the best."
Newly Live Customers Report Frustration: On average, the seven newly live Product X customers KLAS interviewed rate Vendor X a 57.5, well below the KLAS HIT Software average. Many of these organizations report frustrations dealing with Vendor X's support. One IT manager reported, "We recently went live with Product X, and the functionality has been adequate. However, we have noticed a lack of commitment from Vendor X in regard to support. The turnaround time from when we enter in tickets to when they are resolved is poor. I have some tickets still out from a month ago. The average ticket takes at least a couple of weeks to be resolved, if not longer. That is the biggest frustration for me."
Differences in Satisfaction: Clinicians rate Product X significantly higher than do IT professionals (88.9 versus 77.6). One CIO commented on the strengths and weaknesses of the purchase: "Product X is great as a standalone system. All of our staff members love Product X and think it is very appealing. I even find it to be easy to work with, and I am not an ED physician. The problem is that the world is trying to connect. The interface from Product X to our EMR has a lot of challenges. We have to have a portal for the physicians to see the Product X data."
Alert Criteria
Just as a decision support alert can be complex in nature and based on multiple parameters, the same is often the case with KLAS Alerts. KLAS does not publish every alert that is found in our scans, but rather reviews and validates every possible alert before publication.
| Newly Live Alert |
Newly live customers (live less than 12 months) rate a product on average in the bottom ( ) or top ( ) quartile as compared to average ratings of newly live customers across all software products KLAS monitors. Required minimum of at least 3 newly live organizations surveyed. |
The 8 newly live customers KLAS has interviewed rate Product X a 91. Customers report being exceptionally pleased with the product's clinical usability. |
| Code Quality Alert |
A product is in the bottom () or top () quartile for the number of bugs reported per customers interviewed. Required minimum of at least 15 customers interviewed. |
Of Vendor X's surveyed customers, 15 have specifically mentioned code-quality issues, including 4 customers reporting concerns in the latest release. |
| Stratification Alert |
A significant difference in ratings is observed between respondent types in the KLAS sample (as indicated by the Kruskal-Wallis non-parametric ANOVA). |
The C-level respondents surveyed rate Product X 8 points higher on average than do department directors and managers, in large part because of integration benefits. |
| Cost/Value Alert |
Multi-parameter alert type triggered by (1) low or high ratings for the "Money's Worth" rating question, (2) the proportion of customers reporting that a vendor engages in nickel-and-diming, and (3) the proportion of customers specifically complaining about vendor costing practices. Required minimum of at least 15 organizations surveyed. |
Product X qualifies as a high-value product because customers rate an 8.1 out of 9 for receiving their money's worth; 95% of customers interviewed said that Vendor X avoids nickel-and-diming practices, and no customers have reported frustrations with vendor costing practices. |
| Regional Differences Alert |
A significant difference is observed in ratings between geographical regions in KLAS' sample (as measured by the Kruskal-Wallis non-parametric ANOVA). |
Customers in the Northeastern United States rate much higher than do other regions interviewed. Differences in account management are evident from different regions surveyed. |
| Score Change Alert |
A significant improvement or decline in KLAS scores is observed (as measured by the Wilcoxon Rank Sum Test). |
The overall score for Product X has gone up 8 points over the past year. Customers point to recent version improvements as well as better customer communication. |
| Version Alert |
Any significant feedback from customers on the newest version. Required minimum of at least 3 new-version organizations surveyed. |
The 6 organizations KLAS has spoken to on v.9.4 rate well below the overall product average score. All 3 customers cited poor code quality and lacking support as the root of their difficulties. |
| Scalability Alert |
A significant difference is observed in ratings between surveyed organization types or size groups (as measured by the Kruskal-Wallis non-parametric ANOVA). |
The 8 hospitals KLAS spoke to over 400 beds rate significantly lower than their smaller-hospital peers. Customers consistently pointed to product scalability as an issue. |
| Specialty Scoring Differences Alert (Ambulatory Only) |
A significant difference is observed in ratings between ambulatory practice specialties is observed (as measured by the Kruskal-Wallis non-parametric ANOVA). |
The high ratings come from primary care practices that KLAS has spoken with, which rate Product X 15 points higher than do specialty and multi-specialty practices. |
| Optimism Alert |
The average difference between customer ratings for "Overall Satisfaction" and "Forecasted Overall Satisfaction," is below -.1 () or above .4 (), indicating high or low optimism for the product's future. Required minimum of at least 20 organizations interviewed. |
Many customers report optimism with Vendor X's future development plans, specifically with planned changes in v.4.6. |
| Turnover Alert |
Over 25% of a vendor's customers interviewed have future plans to leave a product. (Customers rate "No" to "Part of Long-Term Plans.") Required minimum of at least 15 organizations interviewed. |
Often frustrated by high support costs and slower support, over 40% of organizations surveyed report plans of looking to replace Product X, with at least 5 organizations looking to replace in the next 6 months. |
| Not Validated in Large Hospitals Alert |
KLAS has validated less than 3 organizations with over 300 beds using a product that could possibly be marketed to larger hospitals. |
While Product X rates very highly, it should be noted that all hospitals KLAS has surveyed are under 300 beds. |
| Buyer's Remorse/Lack of Buyer's Remorse Alert |
Either 100% of customers would buy the product again (), or over 25% would not buy the product again (). Required minimum of 20 organizations needed. |
Of the 27 organizations surveyed using Product X, all reported that they would purchase the product again if they started fresh today. |
| Few New Go-Lives Alert |
KLAS has validated two or fewer newly-live organizations in the past two years. Required minimum of 20 organizations needed. |
After speaking to 82 organizations, KLAS has not validated any sites to come live in the past two years. |
| Regional Product Alert |
Over 60% of surveys collected come from one geographic region of the United States. |
28 of 39 organizations surveyed using Product X were located in the Southeastern United States. |
| Executive Alert |
Published at the request of KLAS executives. Reasoning for the alert is made clear in the alert write-up. |
Ten customers recently surveyed have expressed frustration that Product X was not CCHIT certified for 2010. |
| Sunset Alert |
The vendor has announced publicly that a product will be sunset. |
Vendor X has announced on January 1, 2011 that support for Product X will end January 1, 2012. |
Frequently Asked Questions:
Q: Who can see alerts?
A: All participating providers can access KLAS Alerts. For 2011, all vendors with a current Platinum KLAS account can see alerts published about their products. Vendors with access to Platinum KLAS Plus will be able to view all software alerts.
Q: How can alerts help my organization and our performance?
A: KLAS Alerts are intended to alert providers to both positive and negative vendor performance. KLAS observes that some vendors stop improvement efforts short because those efforts are not seen quickly enough in the market to impact market perceptions. Please work with us to validate your improvements so that the market can take notice.
Q: How long does an alert remain online for providers and vendors to view?
A: Alerts remain in the database for six months, and then are viewable only in the alerts archive. When an alert is sunsetted, if the product still meets the requirements for that alert, a replacement alert will be immediately published.
Q: What data window are alerts based on?
A: All alerts are based on a 12-month data window; however, to add better clarity for providers, KLAS will occasionally comment on longer-term score trends.
Q: When are alerts published?
A: Alerts will be published on the 15th of the month for providers to see. As a courtesy, KLAS will publish alerts for vendors to see 48 hours before. For February only, alerts will be published to vendors on the 14th, with a general publish date of the 21st.
Q: Do all products have alerts?
A: If a product does not fit any alert criteria, no alerts will be published. KLAS products are under constant monthly surveillance, and when criteria are met for any alert, that alert will be published. If a product has not qualified for any alert in the past six months, no alerts will appear.
Q: Will KLAS start surveillance of all products at once?
A: No, KLAS Alerts will be viewable for software products first starting in February, followed by medical equipment and professional services after August of 2011. KLAS prioritizes products on which to begin surveillance by how often those products are viewed by providers online.
Q: What does surveillance mean?
A: When a product is under active KLAS surveillance, each month the product is evaluated in order to understand if the product qualifies for any KLAS Alerts not currently published. KLAS will have a staged go-live strategy for starting products under surveillance with the most viewed products entering surveillance first.
Q: When will my software products enter surveillance?
A: See alert publish dates for software products.
Q: Will we see alerts for professional services and medical equipment?
A: We will start surveillance of professional services and medical equipment in the latter half of 2011.
Q: Are all alerts negative in nature?
A: No. Alerts fall into one of three categories: , , or  . are alerts that are (1) not significantly negative or positive or (2) an of minimal impact.
Q: Each alert has a write-up, but in that paragraph I don't see a detailed justification as to why the alert was published.
A: KLAS' intention with alert write-ups is to educate providers and vendors about what KLAS is observing, not to necessarily justify an alert by citing sample size, p-values, etc. All of these details are kept on file and can be obtained for any published alert from your KLAS representative.
Q: Will KLAS Alerts replace KLAS in-depth reports?
A: No. There will still be one place for providers and vendors to read deeper KLAS analysis on a specific topic in these in-depth reports.
Q: What criteria are used to justify a KLAS Alert?
A: See criteria table above.
|