My post of June 18, 2007, in which I wrote about the impossibility of calculating an ROI for knowledge management projects, elicited a long comment from Dr. Dan Kirsch, CKM, MKMP, CKMI COO & Board Member Knowledge Management Professional Society (KMPro). I have shortened his remarks just a bit, but otherwise offer them in their entirety.
“Firstly, ROI is really a pretty lousy “metric” to begin with. Sure lots of folks use it — but it is a quick ratio and there are more games to play with ROI than with employee performance appraisals and curves. In short, tell me what you’d like ROI to be and I can come up with a dozen legitimate ways to game it to accomplish that goal. Secondly, most organizations that insist upon ROI metrics from something like KM don’t really “get it” to begin with. Implementation of KM is all about ….organizational change. If you’re not currently creating, transferring and utilizing knowledge…then you’re implementing a change to get the organization to do so. And the significance of that is that an organizational change cycle is typically going to be somewhere in the neighborhood of 5-8 years. So in short, you cannot sit down and begin KM implementation today and utilize traditional ROI measures which seek to … prove that return in 6-12 months. One reason that this takes place with regard to KM is that most organizations still don’t seem to understand that KM is NOT about implementing IT (information technology).
Now having said that, let me suggest that there ARE in fact great ways to measure a sense of ROI in KM. We use things like “Knowledge Value-Added” (KVA) for that. KVA is based on the basic concept that if the knowledge within a process increases and that results in a increase in the outcome, then you can in fact measure the contribution that knowledge has to the process.
But related to things that would for example, impact the legal profession, there is a much easier way to measure the value of knowledge management. We pretty well accept (in the KM community) that about 50% of the time a “knowledge worker” (defined as someone who develops or uses knowledge — think for example, any documentation, briefs, etc.) will recreate their own document because they are unable to find it on the shared drive. Based on that you’re looking at a potential 50% rework rate on knowledge documents. It doesn’t take an overactive imagination to realize that you can readily demonstrate a return. … [Further] despite having spent thousands and thousand on one search appliance after another…we still see search failure rates as high as 95%. Search “failure” can be defined as failing to find the exact results you need in the first set of returned results …. Again, a lack of knowledge management. The search appliances are much less effective when the documents themselves aren’t properly keyworded. Both PDF and MS Word documents — the two most frequently found file types in a knowledge-base (add in Word Perfect for things related to courts that still utilize that) are typically saved to the knowledge-base without consideration given for including keywords to the document meta tags ….