I went to his session this morning. Here is my dump:
- We live in a knowledge economy and more people learn in companies than in schools. However there are no peer reviewed studies of ROI in companies. That is worrying.
- It is not that ROI cannot be done, but it is very complex to do and not necessarily meaningful. A quick poll of the audience says 71% say their companies do not have an accurate ROI measurement in place.
- Learning is accounted for under GAAP as an expense, even if we talk about it as an investment in people
- Some reasons why ROI is not what we necessarily need: ROI is outcome based but in a learning organization for example it is the process of continuous improvement that matters. For ROI you need to define, measure and monetize all variables that make up the benefits and the costs of learning, while controlling for all variables except for learning. What goes into ROI is also very specific to the company and context.
- All things being equal, what is the impact of learning? Well guess what, all other things that affect performance are not equal, performance is affected by lots of things we cannot control for in the calculation.
- What we are actually are talking about is the IMPACT of learning, and evidences of that instead of the financial number related to outcome benefits and associated costs ratio.
- So we should all be like researchers and find evidence and validate that in a scientific way. Work with the evidence you can gather or with what is easily quantifiable.
- Americans are typically asking a lot of questions and interrupt the flow of the presentation and go into side discussions much more than you will see happening on European conferences. That is not necessarily bad, but at this point I lost it so I don't really know what we could do instead ROI. We did not get to the end or half of the slides, but they will be on the learningwiki.com site later on so maybe I'll get the point of the session later.
At the end of the session I'm still as confused about the ROI debate as before. I have more reasons to believe that ROI is not the unifying answer. But I still have no idea as to what other evidences to replace it with. So hopefully I'll get it later and in the mean time stick to Kirkpatrick.