shopify
analytics ecommerce

Evaluation

Goal setting: making it public or keeping it private?

Some really interesting research that was carried out last year challenges a fairly fundamental piece of advice facilitators hand out during programmes i.e. disclosing your goals in a public setting will make them more likely to happen.

The research is summarised nicely on Psyblog.

The research acts as a helpful reminder that objectives need to be accompanied by other thinking and behaviour:

  • how do I make my objective achievable?
  • why is the objective important to me?
  • how will I feel if I achieve it?
  • what actions do I need to take?
  • who can help me achieve my objective?
  • what will get in the way?
  • how will I celebrate?!

Evaluation of learning and development

As well as measuring the impact of training, a recent article published on the Fitzpatrick website, argues that organisations should also measure the commitment and confidence of participants to apply what they learn. The article suggests that using new ideas and changing behaviour at work requires confidence and commitment. Just as perhaps changing ones dress style or appearing for the first time in public with a radical new haircut, changing to new behaviours can be daunting and requires courage.

You can ask people to rate how committed they feel about applying learning at the end of a workshop and we at Interaction always do this. Because our programmes are highly active participants have a chance to try out new behaviours and ideas and gain confidence through practising before they return to work. The challenge is whether the workshop accurately reflects work so we take time to understand our clients to make sure that our programmes truly reflect reality.

Here’s more about how we do this.

Back from the AGR Annual Conference

We had another good experience this year at the annual AGR Conference.  Thanks to all those who came to see us on our stand.  It was great meeting new people and catching up with people we know already.

Thanks to all those people who spent time filling in our survey on internships.  We will collate the results and send them out to you shortly.

The conference provided useful insights as to how recruiters and developers are reacting to the economic changes.  What was pleasing to hear was most organisations had maintained their graduate development programmes through some pretty challenging times.  It demonstrates the commitment from organisations to recruiting a population of people, who with the right development can provide value to the organisation at different levels.

The AGR survey results indicated that most organisations are using retention as a way of measuring a programmes value.  This is a useful measure as long as the organisation is retaining the right graduates.  Not all graduates flourish as expected and during these uncertain times graduates are more likely to remain at an organisation that might not be right for them.

Getting data that is meaningful and helpful does require commitment and some sophistication.  If recruiters are absolutely clear about why and who they are recruiting and developers are clear about the programmes immediate and future outcomes then good evaluation can be achieved.

Read more on our graduate case studies here - Interaction graduate case studies

Donald Kirkpatrick on learning evaluation

It’s been almost 50 years since Kirkpatrick gave us his “four levels”, Reaction, Learning, Behaviour and Results, to help us evaluate learning interventions.

This interview of Kirkpatrick in Training Zone is interesting to read, as are the questions put to him by readers. When questioned about the “newer” concept of ROI in training, he seems to prefer to stick with his model and the idea that “the Board” will judge results against their expectations.

How do the two approaches differ in reality? In our experience, when an organisation is constantly looking for the impact, they also invest more in follow-up and feedback, and this in itself has a huge positive impact on results. This clarity of purpose also informs delegates and helps them to work out their contribution to the organisation.

We’ve always found that clearly understanding the business impact a client is looking for makes for a fruitful and long lasting relationship. Our ultimate goal is to make a positive contribution to the business. One notable example of where we and the client made a big impact by focusing on business results is here, or you can search our website for more related content.

 

Measuring the impact of training

More and more we are asked to demonstrate the value of our training - particularly at the early stages in a client relationship . Organisations are obviously focusing on the payback of investment in training and rightly so. The interesting challenge for us is that whilst we can agree measures at the start of a project with a client, and always gather feedback at the end of each programme, we are heavily reliant on the client to measure the longer term impact of what we do.

The other interesting challenge for us is that the degree to which we are put through our paces during the sales stage with new clients with regard to return on investment does not always match the level of interest that clients have once the training has been delivered. This is understandable - it takes time and effort to measure the impact of training. A few years ago we carried out a small piece of research amongst our clients and found that many organisations had made a conscious decision not to evaluate training because it took too much resource.

However, we do have clients who do evaluate training very effectively. One of our key clients uses a simple questionnaire to gather data from line managers and participants at three month intervals post programmes. The questionnaire asks managers a range of questions such as the extent to which they have seen a noticeable change in the team member’s way of working. They also ask them to rate the team member’s capability before and after the training programme. There is a separate questionnaire for participants which asks them what they are doing as a result of the programme. They are also asked to rate the benefit of attending the programme in terms of value for money.

Whilst simple and straightforward, this approach delivers valuable data. Comments that respondents add are very useful as they provide specific information about what they are doing as a direct result of the training. They also identify organisational issues that help or hinder the transfer of learning - such as the level of support from line managers or current climate within the organisation.