Archive for the ‘Topic – Evaluation’ Category

Purposeful Objectives

Wednesday, September 26th, 2012

For effective learning to take place, what is the most important part of the process? The accurate specification, the quality of the performance improver (aka trainer, coach), the attitude of the learner, the quality of the evaluation, or something else?

I’m not sure it’s possible to make a robust argument for one particular aspect being more important or influential than any other – they are all integral to bringing about effective performance improvement.

Having said that, I do think there is one part of the process that links all the parts together. And it is an area that is often not given the priority or importance it warrants. It is the ‘learning objectives’ or ‘learning outcomes’.

When I used to train trainers, and then assess the lessons they delivered, it was the single most problematic area. Where the trainers weren’t clear on their objectives, or their methodology didn’t meet the objective (usually because they had committed the cardinal sin of choosing a methodology before arriving at their objective) they often came unstuck. Where they had a well structured objective together with a matching methodology, they had clarity of purpose – and the sessions generally went so much more smoothly for all concerned.

Well-formed objectives assure the client that the specification of their needs is clearly understood, they inform the trainer what is to be covered and to what level, they let the delegates know what they are there for and they let the evaluator know what they need to measure.

An well written learning objective needs to achieve two key criteria. Firstly, it needs to say what is to be achieved by the delegates, and secondly it needs to say to what level.

This is where we need to thank Benjamin Bloom for his Cognitive Domain Taxonomy of Learning. He developed his theory in the 1950s, together with work on the Psychomotor and Affective Domains. A Taxonomy is an “ordered list” and the Cognitive Domain describes the various stages and levels a person passes through as they develop greater knowledge and learning around a specific area.

The left hand column of the table below describes the six levels of Bloom’s Taxonomy relating to the Cognitive Domain.

Level Possible measurable
words
Knowledge Label; Identify; Recite; List; Name
Comprehension Explain; Summarise; Illustrate; Give examples; Distinguish between
Application Organise; Apply; Produce; Show
Analysis Differentiate; Compare; Prioritise; Categorise; Classify
Synthesis Compose; Construct; Hypothesise; Design; Combine
Evaluation Evaluate; Make a judgement; Recommend

The right hand column gives some of the words that can be used in writing objectives to ensure that they are both measureable and pitched at the appropriate level. Let me give an example.

An organisation wants its staff to learn about its Performance Management processes. Some staff who are perhaps not included in the process may need only to have ‘Knowledge’ of it. Those who are appraised probably require the “Comprehension” level, and managers will need to have the “Application” level in order to carry out performance reviews. Other staff in HR may require the higher levels within Bloom’s Taxonomy.

These first three levels should generate three different learning objectives which could be, for example, that by the end of the session the delegates are able to:

  1. Identify the organisation’s Performance Management processes (Knowledge)
  2. Summarise the organisation’s Performance Management processes (Comprehension)
  3. Apply the organisation’s Performance Management processes (Application)

These different objectives dictate the levels to be achieved, and therefore the methodologies to be used and consequently the duration of any learning intervention. To achieve the first objective could take 10 minutes, to achieve the third could take 2 days – a significant time and cost differential for a one word difference in the objective.

Moving finally to our evaluator. He or she can then ask delegates, either verbally or in writing, to “Summarise the organisation’s Performance Management processes” – and the answer will tell us whether the intervention has achieved its stated objective. When reported back, the client has clarity as to whether their requirement has been met, as does the trainer.

Paul

Management Gone Missing?

Monday, September 10th, 2012

I saw this comment on a learning and development forum recently – “I recently ran a management course for new and existing managers at similar levels, it is a course which all managers new to the organisation are told about and attend with agreement from their line manager. I had a number of managers who were negative and felt they shouldn’t have attended as they had a number of years’ experience as managers already. The other delegates were newer and did want to be there. I found I really struggled to turnaround / manage the really negative individual and the other few who weren’t happy to be there. Due to this I really feel I didn’t deliver the best course I could have done as, whilst I have experienced the odd 1 or 2 negative delegates in the past on various courses, I’ve never experienced such constant negativity which I didn’t seem to be able to have any influence over.”

Following the post were a number of suggestions from respondents, including:

  • Having individual discussions with delegates prior to their arrival
  • Encourage delegates to explain their concerns
  • If people don’t want to stay, don’t make them
  • Tailor the course more to their particular needs
  • Ask them what they want to get out of the course
  • Meet with their managers post-course

What surprised me, though, were that all the responses focused on what to do in the classroom environment, or post-course. In order to implement a long term solution, there needs to be wider and more systematic activity.

What can be done to ensure that such instances are minimised in the first place? What appear to be missing are effective learning and development management interventions.

For learning in the workplace to be effective, there needs to be a tripartite approach comprising the delegate, the delegate’s line management and the learning provider. And in terms of the learning provider, this means both the trainer and – importantly – their management.

Within the post there is no mention of 3 key activities or processes that I would expect to see.

1. Why is each delegate attending the course? They have been nominated by their manager – what are the reasons for the manager requiring them to attend? Managers having pre and post-course meetings with their attending delegates is one of the areas often examined during an Investor in People (IiP) assessment – because it makes a difference.

If this process doesn’t happen, it should. If it does happen, it should be recorded – and most     importantly be available to the trainer. This not only helps in terms of dealing with the issues    in the person’s post – by being able to explain to or remind them of why they are there – but it helps in terms of planning a more meaningful event for the delegates – and will generally mean an increased return on investment (ROI).

2. How might the use of a Student Charter or Learning Agreement help this situation? If such a process is in place, it means that the internal customer (the delegate and their line management) knows what to expect from the internal provider (the learning and development function). It should also include what the internal provider requires from its delegates – in terms of participation, input and behaviours – together with the consequences for a delegate who chooses not to meet this level of expectation.

Such an agreement should have been agreed across the business at senior management level – and so it is then easy for a trainer to use and implement without fear of having overstepped the mark. Transparency for all concerned.

And as a consequence, those who want to be there and want to learn can do so.

3. Finally, the person’s post on the forum indicates that this course has been delivered previously. What evaluation data has been generated?  And I don’t mean end of course or Level 1 data – I mean Level 3 or above, good quality data.  How is it improving the performance of those who are attending? How have previous delegates been able to use what they have learned? How is the course improving organisational performance?  What is the organisation’s return on its investment?

This information should be gathered from the delegates between 2 and 4 months after the completion of the programme. Both the quantitative and qualitative information can then be used by the trainer to illustrate what delegates can gain from attending – thus also demonstrating the overall value of the internal learning and development function to the business.

I have to say, I really feel for the trainer. I’ve been in that position and it’s not a lot of fun. Here is a trainer wanting to deliver a quality product, but some of the delegates are apparently thwarting his or her efforts.

And I get the impression that perhaps in this situation they are not been sufficiently supported by their learning and development management. The activities I have outlined above would both assist the trainer and probably make their role more enjoyable – and, above all, benefit the business.

Paul

 

7 Steps to L&D Heaven

Saturday, May 14th, 2011

In my last post I outlined seven questions that I always use when determining exactly what a client’s needs are. I said in this post I would explain why I use – and document – these questions, and why I think they are important.

The basis for them is that in my experience a little time spent at this stage pays dividends later in the process. This is because sometimes clients:

  • Spend too little time critically thinking about their true learning requirements.
  • Do not appreciate the true cost of the investment in people’s development that they are asking for.
  • Have no ownership of the rest of the process if not involved from the start and – importantly for the L&D provider – no accountability if it doesn’t achieve what it was supposed to achieve.

What is the identified performance need? Too often clients only want to talk in terms of a solution, and that solution is often a course. This is understandable to some degree as they are busy people and it will often seem relatively straight forward to them. This question, and any subsequent clarification questions, are intended to take us back to basics. This will help me to build a product or solution based on a firm foundation of a properly identified need.

Why is a solution required (how will it add value, and what would happen if it did not exist)? By asking this, I am ensuring that the learning is required – in other words, once completed, the person will be able to add additional value to the organisation. They will have new skills. It helps us start to identify what the intended return on investment (ROI) will be. If the client is unable to specify how it will add value, is it really a development need that the organisation should support?

How will the solution improve the organisation’s performance against its identified goals? Not only do we need to ensure that the person will be able to add value after completing their learning, we must ensure that this value is what the organisation currently values – is it a company priority? The majority of L&D functions receive more requests than they can handle – identifying which will support current organisational priorities is one way of prioritising them.

Which organisational competencies does the solution aim to address and to what level? This enables the solution to be linked to any relevant competency frameworks or appraisal process. In ascertaining the correct level, it enables the L&D professional who takes responsibility for designed the solution to be able to pitch it at the appropriate level.  If the eventual solution is publicised more widely, it also assists other clients to understand the level of the event.

What are the target staff groups /  teams / Departments for this solution? We have to ensure that the target audience is the correct audience to deliver this business improvement. As we all know, there are some people who try to get on courses for the sake of going on a course (and others who we can never get near one!). This question is intended to help me critically examine exactly who should be receiving it – thus offering it to all who have the need, yet not making it available to those who will not be able to use it to add organisational value.

How will the effectiveness of the solution be measured in the workplace? This links back to the return on investment (ROI) seed which we planted earlier in the conversation.  We need to clearly specify this so that anyone involved in the delivery and receipt of the solution is clear as to how they will add value. Also, so that the evaluator can check the effectiveness of the solution in the future. For example, let’s say the solution is in the contact centre environment, and it is intended to reduce a team of advisors’ Average Handling Times (AHT) and their Transfer Rates for calls to the contact centre. We need to measure these in terms of the average length of calls and percentage transfer rate prior to the interventions so that we have the base data. At a given time after the delivery of the solution, we can then measure again and quantify whether the required improvements have been made, and if so by how much – which can then be converted to a monetary value relatively easily. If this measurement, or the decision on how to measure, is not undertaken at the start of the process, we cannot quantify the success of the intervention.

What are the clearly stated, measurable objectives for the solution? This will usually  need to be completed by the L&D specialist as our clients are unlikely to be familiar with Bloom’s Taxonomies of learning, and in particular the levels.  Objectives need to be measurable so we need to ensure we use words that can be evaluated against. This means that when we say, “By the end of the session delegates will be able to …”, we should avoid words like ‘understand’ or ‘comprend’. Instead, we should use words such as ‘outline’ (Knowledge), ‘explain’ (Comprehension) and ‘apply’ (Application).

Finally, we need our sponsor to sign off a documentary record of what we have agreed. It can sometimes feel a little onerous to do this, and sponsors may be reluctant to engage in this process – if this happens I start to be a little concerned. Having them sign off the process is all about them agreeing that the answers to the questions are correct, and that the description of the need, the way it will be measured and the proposed learning objectives are all recorded accurately. If there are disagreements, then amendments can be made. If not, it can be agreed, signed and then passed to L&D professionals in order for them to suggest and develop appropriate solutions.

And the potential for significant added value will be greatly increased!

Paul

So tell me what you need, what you really, really need

Monday, April 25th, 2011

I’m all for the phrase “Make it happen”, but we always have to be clear on the ‘it’. Sometimes organisations are so keen to make ‘it’ happen, the priority can become “Make something happen” which can often be one step forward and two steps back. It can also damage relationships and cause waves in previously tranquil waters. And it can waste serious amounts of money.

The area of learning and development is one of the best – or worst, dependent upon how you look at it – examples of this.

I can think of several occasions where training interventions have been delivered and they have not delivered the required results in the workplace. In several of those instances the claim has been that it was “Training’s fault because they didn’t train the right things”. Hmmm. Perhaps it was, or perhaps it wasn’t. It is, however, relatively simple to introduce a process that minimises this potential and maximises the value from an organisation’s investment in learning. As with so many successful projects, the primary requirement is for the relevant people to take the required responsibility and accountability.

Learning and development professionals (the contractors) need to slow down the commissioning process. The commissioning process is the stage where the internal or external client identifies what needs to change. And this will only work if the responsibility is in the right place.

The client has to take responsibility for job descriptions, core skills and identifying individuals’ learning and development needs. The contractor’s role is to analyse the client’s needs and develop appropriate and cost-effective interventions, deliver the interventions, assess the competence of delegates and check the intervention met its stated objectives. Finally, we arrive at the evaluation of the changes in the workplace as a result of the intervention – which is the client’s responsibility.

The start of this cycle used to be called the Training Needs Analysis (TNA). However, it is increasing being broken down into two activities called the Performance Needs Analysis (PNA) and the Learning Needs Analysis (LNA). Whilst it is the responsibility of the client, the contractor needs to assist them in this process. When we are closely involved in matters, we often lose our objectivity and sometimes make assumptions – this is what can happen for a client and so we need to help them avoid this by taking them through a structure process.

This process can be as detailed as everyone wants to make it. Having said that, if learning and development professionals use too complex a process, they will lose credibility with the client. The complexity of the process should also be driven by the size of the audience – in other words, if 20 people need to improve their performance in a particular area, the process should be less arduous than if 2,000 people require it – but both require a process to ensure we make the right thing happen.

As a start, I use seven questions. These are:

  1. What is the identified performance need?
  2. Why is a solution required (how will it add value, and what would happen if it did not exist)?
  3. How will the solution improve the organisation’s performance against its identified goals?
  4. Which organisational competencies does the solution aim to address and to what level?
  5. What are the target staff groups /  teams / Departments for this solution?
  6. How will the effectiveness of the solution be measured in the workplace?
  7. What are the clearly stated, measurable objectives for the solution?

From experience, I know that these questions can come across as threatening to clients. This is particularly the case in immature organisations – in other words, organisations where discussions around learning and development activities are often packaged in terms of the client stating they want a course on a particular aspect, rather than rational, objective and meaningful discussions between the client and contractor on what they need to achieve together.  Consequently, my preferred approach is to let the client have the questions prior to us meeting face to face or us having a further phone conversation. This gives them the opportunity to identify what they need – what they really need.

When learning and development professionals get this process right, they are well on their way to making a meaningful impact for the organisation. They will be able to demonstrate their value in improving performance.

In my next post, I will explain a little more about each of these seven star questions – and why, when you have completed the process, you need to get your sponsor’s sign-off.

Paul

Misleading Matters

Saturday, April 2nd, 2011

I was having a lovely email exchange with a person yesterday who has recently moved into learning and development management. She was explaining how she now needed to be more strategic, she has real plans for her company and she was so passionate about what she wanted to achieve. It was an inspiring exchange!

It started me thinking about if I was in such a position again. If I wanted to develop a culture that was as performance focused as possible and able to demonstrate how it added quantifiable value to the organisation, what are the words or phrases that would most hinder that culture shift, and what would I seek to replace them with? How could I demonstrate clarity of direction? Here are my top five:

Training Manager – that would be the first one to go. For me, the title signifies that the post-holder is purely focussed on what happens whilst a delegate is attending a course or undertaking an e-learning package. I’d want to be the Learning and Development Manager – or even the Performance Improvement Manager – thus signifying that I have a proactive interest in engaging with the managers and delegates pre and post event, and ascertaining the value of our contribution.

Abstraction – So often I used to be told at meetings that training was an abstraction, and the person making the judgment argued they couldn’t afford such an abstraction (in some companies the word ‘overhead’ is also used). I would immediately retort – sometimes interrupting the person, especially if they didn’t get the message the first time – that the learning and development that I managed and delivered was an investment (not an abstraction). If people get used to referring to your activities as an abstraction, the activities have the potential to become so. Take the opportunity to change the culture and make people think differently about what you are doing.  Eventually others will call it an investment, and then you know the culture can change and you will be viewed very differently.

Time Management Course – Why advertise that you are going to do something that is clearly impossible? Sailors don’t claim to manage the tides, and weather reporters don’t claim to manage the weather – we may wonder what planet they were coming from if they did. So what reasons do trainers and training managers have for claiming they can help people manage time?  We know what the tides will do, and can see and feel (and predict to some degree) the weather, so we manage ourselves accordingly. The same goes for time. Call such events Task Management or Task Prioritisation Courses – you will be surprised how differently people think about the event from the outset, because you are being clear about what will be achieved. Have a look at all your offerings – do they really do what they say on the tin?

Problem Person – I have often been faced with someone seeking assistance as they have a ‘Problem Person’ to deal with.  I bet Carl Rogers turns in his grave every time he hears this. If people are viewed in this way, they will potentially always be a problem. Such an individual is a person. A person who has a problem – a problem which you may well be able to assist them with. Where the manager thinks they have a ‘Problem Person’ they will more than likely become one.

Can you organise a (whatever) skills course, please? – The answer to this is ‘No’ – well not on this information, anyway. Operational managers are busy people and also may not be aware of all the ways that development needs can be met. You need to find out more – a lot more … how was the need identified, how many people does it apply to, how do we know it applies to all of them, why do all of them need it, what opportunities will there be for all these people to use the new skill, and so on. Get to the heart of the matter. You will receive so many ill-defined needs – and asking these questions (and others) will help generate a performance improvement partnership between your function and the rest of the organisation. And if you don’t ask the questions, and the learning and development intervention doesn’t work, the operational manager will make it public as to whose fault they think it is – and that will not do your culture shift (or reputation) much good!

So those are my top five – are there any that you would add?

Paul

Can training ever be exceptional?

Sunday, March 27th, 2011

I saw a question on a forum yesterday that got me thinking. The question was, “I’m interested to know how folk define ‘exceptional training’ – what makes training outstanding, and sets it apart from an everyday training course?”

As a trainer, what have I delivered that has been exceptional? I can think of many sessions that I have been pleased with, and that have gone better than others that I have delivered. I can also think of ones that I have thought have been very powerful in the learning that has been generated for the delegates. But I don’t think any of that necessarily makes them ‘exceptional’.

Then I considered what training I had attended that had been exceptional. What came to mind was not training that had been exceptional, but exceptional learning that I had taken from sessions or courses – and the two, I think, are completely different.

For training to be exceptional (if, in fact, it can be) it will depend upon how the delegates respond to it – and not necessarily whilst they are experiencing it, but how they respond back in their workplaces and lives in the weeks, months and years to come.

For example, I was fortunate to be able to attend the Work Foundation’s Runge Effective Leadership Programme about 10 years ago. There was a session on how to create a Vision for a Company or Department. I cannot remember the name or much about the man who delivered the session. It wasn’t a particularly innovative or interactive session, and at the end of the one-week Programme it wasn’t in my ‘top three’ of the sessions I had attended. When I look back on it, however, I realise how valuable it was to me and how much I have used it since. It has been a foundation, a starting point, for many of my successes since then. But I really couldn’t class it as exceptional training.

Which supports my view that only delegates can really say whether such experiences are exceptional, and they can only really do that further down the line. Perhaps there can be ‘exceptional learning’?

So as a starting point in trying to answer the original question how about this – exceptional learning occurs when 100% of the full cost of the learning event is recovered within 12 months of the event – in other words, the person or company acquires a full return on its investment (ROI) within that time frame. I think that could then be classed as ‘exceptional’.

The first step in seeking to achieve this is for trainers to stop thinking of themselves as trainers, but to think of themselves as Performance Improvers – who can play their part in achieving such improvements by effective training methodologies. By reframing their role, their thought processes are likely to have a greater focus on how they create exceptional performance improvements – which then brings the realisation that it is as much about the part the delegates play as the trainer.

It includes getting the right people on the right events in the first place, and motivating and developing them once they get back to the workplace.

The trainer – or Performance Improver – plays their important part in such achievements, but others also have very important parts. And all must play their parts well to achieve exceptional learning.

Unless, of course, you think differently?

Paul

Who’s giving training a bad name?

Wednesday, March 9th, 2011

I sometimes wonder how people are appointed to positions as trainers. Do some organisations see it as a position that people are moved if they might be good at it, and are then left to get on with it – without any training or support? Do businesses think training is such a waste of time that they just employ anyone in the position? Whatever the reason, there appear to be too many companies or trainers giving training a bad name.

I see requests for assistance from trainers on various forums, but I’m not sure how they expect people to be able to respond with targeted and focussed suggestions. I’m meaning where people are asking others to suggest methodologies. Here are a couple I have recently seen:

“I have a four-hour workshop to give on presentations skills. I see the same participants the following week and they have to give a five-minute using the information they’ve learned in just four hours. Any recommendations? No technology available for them except for one laptop. They do not have laptops themselves.”

“I work in a manufacturing facility and I am about to conduct management development training for supervisors/managers to teach them more about ‘HR in their jobs’.   Does anyone have any good icebreakers that will compliment this training?”

Do these same people (or organisations) say, “I’m going to buy a car, which one would you recommend?”, or “I’m going on holiday and need a coat – which do you think I should buy?” I think not. So why do it for training?

Trainers asking such questions cannot expect to receive any sort of meaningful advice or assistance. And if they do use what is offered, does the organisation realise how much money it is potentially wasting?

Let’s say we have 12 people in our group, and each receives a salary of £100 per day. That means with salary on-costs they probably cost their organisation £120 per day. That’s £1,440 in wages for attending a day’s training. Add another £200 for the trainer’s delivery and preparation time, £100 for the administration of getting the people on to the course, £50 for room hire and £10 for photocopying costs – that gives us a total cost of £1,800 for the day. There may also be travel and meal costs, but we’ll leave it at this for the purpose of the exercise.

If we have 6.5 hours contact time, it means that each hour of delivery is costing our organisation approximately £277.00. Or £4.62 per minute.  So a half hour opening exercise comes in at £138.

And if it costs £138 it needs to be focussed – not focussed on the ideas of others who have no knowledge of the organisation, but focussed on operational effectiveness, focussed or organisational goals, focussed on delegate needs and focussed on extracting a return on the investment. Effective evaluation.

As effective trainers know, evaluation starts at the planning stage. An evaluation of the needs of the attendees and the organisation. Once a Training Needs Analysis of some sort has been undertaken, planning, validation, delivery and further evaluation follow. This evaluation can show the organisational and operational improvements that have been achieved as a result of the training. Very satisfying, and likely to lead to the increased credibility of the trainers and L&D function.

Training people effectively is costly. Training people ineffectively is even more costly as we reduce (or fail to get any) the return on the investment.

Or have I got it all wrong?

Paul

Sell your crunch, not your apples

Saturday, December 4th, 2010

Where does the L & D function fit within your organisation? The importance of this cultural fit cannot be underestimated when leading a L & D function. In my experience, too many L & D functions have existed to deliver training – that is what they have been seen to do, and that is what they have done – and this approach gives them little credence with operational managers.

I once had concerns about the impressions that my unit was giving to the remainder of the business. At a managers’ meeting I took the opportunity to ask a few (what I thought to be) fundamental questions to see whether they were espousing my stated aims for the unit. They all managed trainers, so firstly I asked what they saw to be the overall role of the trainer. The first answer was, “To deliver training”. This was closely followed by answers indicating that they needed to be able to write lesson plans and that they should be able to explain points clearly to delegates. Two more responses followed in a similar vein. The sixth response was, “They need to get the delegates to be more effective”. That’s what I was looking for – a link to improving performance.

After a few other responses, I then asked, “What are trainer shoes for?” The first response was, “to help people run faster”. Other verbal offerings built on this performance related theme.

I then asked what they thought was the overall role of a horse trainer. The first respondent indicated that her view was that they were there to enable horses to win races. A second added that perhaps it was to enable both horse and rider to win races.

“So”, I asked, in a John Humphreys sort of way, “Why is it that you view trainer shoes and horse trainers as being there to improve performance, yet you see that the primary purpose of our trainers is to deliver training?”. This was followed by a silence.

This short exchange had confirmed my concerns and highlighted a fundamental issue in leading a successful Learning and Development Unit. If its own staff see the focus as being delivering training, this will be apparent to others in everything they do – most of all to the already potentially sceptical operational managers.

I personally would consider changing the job title of “Trainer” to “Performance Improver”, however, – at least initially – I think this would cause problems for others to understand what the role is about. It is fundamental, however, that the trainer views their role in this way. They should not see themselves as people with training skills who use these skills to improve performance; they need to see themselves as performance improvers who achieve this by using sound training skills. This may appear to be a play on words, but I don’t believe this to be the case – it is a way of the person constantly being reminded of the primary purpose of their role.

When I used to train trainers, we engaged in some lengthy discussions which felt very important and fundamental to the role – in retrospect, I am now disappointed to admit that they were akin to navel gazing – they were all about the minutiae of training delivery which, on reflection, had little impact on delivering greater operational performance.

It is only by demonstrating this focus on improving performance that you will earn credibility and respect, together with a place around the table when the all important annual budget is being allocated – and then the more improvements you can show to have been involved in, the better your chance will be of improving your share of the budget.

All this can be achieved by a number of key activities. These include:

  • Engaging your internal customers in deciding learning priorities
  • Explicitly linking all learning to business objectives
  • Explicitly linking all learning to improving performance
  • Measuring your team’s performance in terms of operational performance improvements rather than / in addition to the quality of the training delivered
  • Clearly communicating your successes to the business
  • Where appropriate, measuring your impact by undertaking Return on Investment (ROI) evaluation activity to demonstrate the financial return for the business’s investment.

The overall culture of a unit is probably the most difficult aspect to change – but ultimately the most fundamental and rewarding. By implementing these processes, we can start to change the mindset of our team. We find out who wants to make the journey with us, and who perhaps does not.

Paul

Deliver Outcomes, not Outputs

Friday, November 12th, 2010

Where at all possible, I try and steer clear of discussing governmental politics. This is because I don’t really want to be perceived to have particular party allegiances. But rules, as with policies and directions from Sat-navs, are there to be broken where there is good reason – they exist to provide effective guidance for the majority of situations as opposed to the answer for all situations.

I am very interested by the Business Plans that the ruling coalition has published this week (I feel like a very sad person as I write that, but please bear with me …).  It’s the approach behind some of the Plans that I find particularly interesting.

When the coalition first came to power, they talked about financially rewarding private contractors for ‘housing’ prisoners and stopping them reoffending – whereas currently they are paid for merely ‘housing’ them for the duration of their sentences.  As someone who is very keen on the use of outcomes rather than outputs, I found this very refreshing.

More detail on this approach has now been published – it is within the Ministry of Justice Business Plan which was published last Monday. The first two objectives within that particular Business Plan are:

1.1 Develop an overall strategy for the ‘rehabilitation revolution’ for adults and youths, including paying local private and voluntary organisations by results.

1.2 Introduce payment by results schemes, working with local, voluntary and private sector organisations that specialise in the rehabilitation of offenders.

I am interested in this approach as I believe that society as a whole should move more towards an outcome focussed approach to business. This is particularly true of learning and development providers – and clients.

If a client seeks and purchases a course, event or programme to address a particular area or aspect of business, it is likely that it will be delivered and there may well be a short end of event questionnaire. It is rare, in my experience, that there is much further evaluation of the benefits of the event.

So, how about the client publicises the issues or skills they require addressing on the event and these are then quantified into measurable outcomes (not outputs).  The provider who then gains the contract only gets paid once these targets have been achieved – perhaps 6 months after the event. The probable consequences would include:

  • A greater focus on outcomes as opposed to outputs.
  • Only providers who have confidence in their abilities bidding for the contract.
  • Clients ensuring and demonstrating value for their money.
  • Far greater interest by clients, providers and delegates’ line managers as to how the learning is put into practice in the workplace.

Who wouldn’t want these consequences?

So why does this approach generally not happen at the moment?

Paul

Ofsted – head in the sandpit?

Sunday, November 7th, 2010

As we focus on where we could save money and where we waste money in the UK in order to balance the books, I am becoming increasingly unsure of why Ofsted continues to exist – at least in its current form.

Earlier this year (15 August 2010) I blogged about my thoughts on what Chris Woodhead, the former Chief Inspector of Ofsted (Office for Standards in Education) had been saying about schooling. In The Times (25 July 2010), whilst talking about English Literature classes, he wrote, “The aim should not be to develop ‘critical thinking skills’”, and added that children, “should not be encouraged to express their opinions on the texts. Who cares what they think or feel?” I find that harder to believe every time I read it. And, incidentally, I care what they think and feel.

His more current collegues were also involved in the tragic Baby Peter case. You may recall that Ms Shoesmith’s was the Director of Children’s Services in Haringey at the time, and she appealed against unfair dismissal by the Council. During the appeal Ofsted suddenly discovered thousands of pages of undisclosed evidence which they had previously said could not be located. These contained snippets such as notes relating to Ofsted inspectors being told to delete all emails on the case, and evidence that an additional finding was popped in stating, “There is insufficient strategic leadership and management oversight …”  Ofsted had also given Haringey’s Children’s Services a glowing 3-star rating only a few weeks after Baby Peter had died. The subsequent revelations didn’t make things appear that 3-star to others.

And last week Ofsted again hit the headlines for their part in the awful Little Ted Nursery case. The Plymouth Safeguarding Children Board completed the Serious Case Review into the Nursery in March 2010, and sent it to Ofsted as is required. Apparently, Ofsted didn’t respond until October 2010, thus delaying the publishing of the Report until last week.

The purpose of a Serious Case Review is to ensure that lessons are learned and practices are changed so that incidents such as this should not recur – so that society safeguards and promotes the welfare of children. How is taking 7 months to respond to a report an effective way of protecting children? I thought that there must be a good reason for this delay, so I went to its website and had a look at the ‘News’ section. Not a mention – even though they’ve had 7 months to think about it.

Paragraph 5.44 of the Serious Case Review states, “It has become clear from this review that whilst the Early Years Service had many concerns about the nursery there was no formal mechanism for informing Ofsted, since they did not reach the threshold of a breach of regulations. Similarly Ofsted had no means of discussing with Early Years the support need of the nursery. It is notable that whilst Early Years had the nursery identified as red or amber on its own rating system, Ofsted inspections were good or satisfactory”. Again, Ofsted making a judgement that is far removed from reality.

So why is this happening? There is a theme, and that is the ‘tick-box’ culture that Ofsted appears to support. This was noted by Deborah Orr when she wrote in the Guardian last year (26 November 2009). “Does Ofsted’s  tick-box culture deserve the criticism that has been heaped on it this week? I do believe so. I have before me an inspection report on a London primary school, rated outstanding. It scores the highest grade (out of four) in every one of 33 categories, except two. What are the two categories that this school fails to achieve perfection in? Oh, just ‘standards achieved by learners’ and ‘how well learners develop . . . skills that will contribute to their economic well-being’. Little things, of no great consequence in a school.”

What’s the common denominator in these last three examples? Ofsted have awarded ratings that are far higher than the services or institutions warrant – when looking at it from the perspective of society rather than ticked boxes.

Exactly. Because what generally happens when you create a tick-box culture? You measure things that are easy to measure – because that’s what ticking boxes is about. And that in turn makes unimportant things suddenly important – because they are generally easier to measure when using boxes to be ticked. And so people stop concentrating on the important things – because what gets measured generally happens.  And so the truly important matters are not important any longer – well, not to the inspectors and those who are being inspected and want to achieve a good rating.

These examples are three occasions where Ofsted have given ratings far higher than they deserve.

Earlier this year Zenna Atkins announced she would be standing down as Chair of Ofsted on 31 August 2010.  When she spoke of her decision to stand down, she said, “The work of Ofsted not only costs the taxpayer a third less than when undertaken by four separate inspectorates but we are also supporting improvement in a more efficient and effective way. Front-line observation and engagement with children, parents and leadership teams are now at the heart of all new inspection arrangements, sweeping away any inherited tick box culture”.

I’m not convinced, Zenna.

In my view, Ofsted is not providing value for money – even if it is costing a third less – and it needs a complete overhaul. More important than it not providing value, it is not measuring the right things, and most importantly it is not effectively protecting children or supporting their development. And it appears to be institutionalised in its processes and so I wonder whether it will be able to change. I’d scrap it.

It is great as a case study of what can go wrong when tick-box processes are created. But, sadly, in my opinion, that’s all it’s currently good for.

Paul