Words: 9602
Learning and Performance Development in Asia: A Study of the Evaluation of McDonald’s & BreadTalk’s Training Programs Executive Summary This report reviews the current methods on evaluating training programs by BreadTalk, with an aim of improving its evaluation methods. Through a search of literature and an analysis of current methods, the report summarises the latest thinking on evaluating training programs and provides insights on how BreadTalk can improve the quality and management of its evaluations. A comparison with an overseas company, McDonald’s, was also done to provide suggestions for improvement for BreadTalk’s evaluation methods.
There are three key objectives of the report: first, to identify models, frameworks, methodologies, and approaches as well as their uses and feasibility in evaluating training programs; second, to summarize how the quality and effectiveness of these evaluations are assessed in theory and in reality, and mainly, what cutting-edge methods and approaches other organisations use to maintain the quality and effectiveness of their evaluations; and, third, to outline some common practices of corporate evaluation units. Latest models.
The review found that the reigning framework for evaluating training programs – and the subsequent models, methods and approaches – is one outlined more than forty years ago by Donald Kirkpatrick. His framework evaluates training programs in terms of four levels – reaction, learning, behavior, and results. Subsequently, Jack Philip’s five level Return On Investment (ROI) model became the conventional model for evaluating training. Basically it is similar to Kirkpatrick’s framework, with an additional level, the calculation of ROI. Other models analysed include the CIRO model and a few others.
Critiques of the models are also mentioned. Quality and effectiveness. The report found that there were no new, cutting-edge methods and approaches beyond the Kirkpatrick framework used by the two organisations to maintain the quality and effectiveness of training program evaluations. In general, most organisations rely on simple evaluations of participants’ perception of learning. Common management practices. The role and management of evaluations of training programs are similar within organisations, and the responsibilities of the management teams are similar. It is common to use annual work plans and quarterly eports to develop work objectives and results and to report on progress toward those results. The organisations that appear to be the most effective in implementing training programs and in enhancing the impact of recommendations from evaluations are those that fully integrated training, evaluation, and administration in one unit and/or have developed strong partnerships with operational managers within the organisations (Evans, March 2007). Table of Contents Executive Summary1 1. Introduction2 1. 1 Description of BreadTalk2 1. 2 Significance of Training Evaluation2 1. General Trends of Training Evaluation2 1. 3 Training Evaluation trends in Singapore2 1. 4 Methodology2 2. Literature Review2 2. 1 Definition of Training Evaluation2 2. 2 Theories and Models in Training Evaluation2 2. 2. 1 Kirkpatrick’s Four Levels of Evaluation2 2. 2. 2 Jack Philip’s Five Level Return On Investment (ROI) Model2 2. 2. 3 CIRO (Context, Input, Reaction, Outcome) Model2 3. Current Practices of BreadTalk2 3. 1 Challenges Faced and Strategies Taken2 3. 2 Future Path2 4. Summary and Recommendations2 4. 1 McDonald’s Evaluation Strategies2 4. 2 Comparing BreadTalk with MacDonald’s2 . 3 Our recommendations to BreadTalk2 4. 3. 1 Kirkpatrick’s four level Evaluation Model2 4. 3. 2 Jack Philip’s ROI model2 4. 3. 3 Making Evaluation Work in Your Organisation2 4. 4 Recommendations for McDonald’s2 4. 5 Limitations2 5. Conclusion2 Bibliography2 Appendix A – The Success Case Method2 Appendix B – MOM report on Employee Supported Training: Charts2 Appendix C – Phone Interview with Mr Ng Kwee Lam, HOD of Training, BreadTalk on Monday, 20th October 2008 @ 5. 20pm2 Appendix D – Kirkpatrick’s Four Levels of Evaluation2 Appendix E – The Return on Investment (ROI) Model2
Appendix F – Different Strokes for different folks (Cross, 2001)2 Appendix G – When and How to Evaluate2 1. Introduction 1. 1 Description of BreadTalk BreadTalk opened in Singapore in July 2000 with a revolutionary take on the bread industry. Entering the market with its new concept of a boutique bakery, it shook up the bakery industry with its innovative and creative products. Currently the market leader in the new concept bakery, BreadTalk was listed in June 2003 on the Singapore Exchange, giving customers a stake in the popular bakery (BreadTalk – Boutique Bakery: A Contemporary Lifestyle Concept).
It was able to gain a strong foothold in Singapore due to its competitive strengths (BreadTalk, 2008). The first strength is its unique and branding, in which it is able to distinguish itself from traditional bakeries. Next is its wide range of products, in which its success is attributed to the different varieties o f breads, buns, pastries and cakes. The third strength is its strategic locations, which are accessible, thus attracting many potential customers.
Lastly, BreadTalk has a pool of experienced management team which has many years of experience in the food and beverage industry. BreadTalk has also achieved numerous awards testifying to its branding and business success. 1. 2 Significance of Training Evaluation Effective training programs remain on management’s continuum for producing organisational value and increasing people value (McFarlane, 2006; Rossett, 2007). Valid evidence of the training impact on our organisation’s bottom line has to be provided to validate training as a business tool. By ddressing basic concepts using a framework of inquiry methods, results can be uncovered, measured and documented, making a compelling case for improving training impacts and bottom line results (Appendix A) (Brinkerhoff, May 2006), thus validating training as a business tool and justifying its costs (M. Osman-Gani, 2008). Reasons for evaluating training (pg 199): 1. Because companies have made large dollar investments in training and education and view training as a strategy to be successful, they expect the outcomes or benefits related to training to be measurable. . Common use of engagement builds accountability for managers and provides a valued progress measure for the entire company. 3. Provides a way to understand the investments that training produces and provides information needed to improve training. 4. Provides the data needed to demonstrate that training does offer benefits to the company. 5. Formative evaluation: Evaluation of training that takes place during program design and development. Conducted individually or in groups before the program is made available to the company.
Pilot testing is a form of formative evaluation. a. Ensures that training program is well organized and runs smoothly b. Trainees learn and are satisfied with the program. 6. Summative evaluation: Conducted to determine the extent to which trainees have changed as a result of participating in the training program. 1. 3 General Trends of Training Evaluation With increasing expenses involved in conducting training programs, many corporations are very interested in measuring their concrete outcomes to ensure that the benefits are justified.
In the past, the success of a training program was measured by how many classes were held and the number of employees trained. If the numbers were high, then the training was considered successful. In the current workplace, companies want to know to what degree the training assisted in the transfer of new skills to the job and to what extent individual and group performance improves. In fact, the movement toward out-based training is so strong in some companies, the professionals who provide such training refer to themselves as “performance improvement specialists” (Dutkowsky). 1. Training Evaluation trends in Singapore Ministry of Manpower’s Research and Statistics Department gave out a report on Employer Supported Training in September 2007. The report provides details which cover statistical information to the labour market and facilitate decision-making within Singapore. It also includes information about training evaluation. Some of the charts were reproduced in this report and attached in Appendix B. Exhibit B1 shows that the proportion of training-providing establishments that evaluated the effectiveness of training dropped to 56% in 2006, following an upward blip in 2005.
The larger establishments were more likely to undertake training evaluation than those smaller establishments. Exhibit B2 shows the common evaluation procedures and we can see that the most common evaluation method is getting feedback from HR/Line managers or supervisors. One other method of evaluation is measuring the job satisfaction, which is consistent with Kirkpatrick’s level 1 evaluation. Exhibit B3 shows the reasons for not having evaluation procedures. From the chart, we can see that the main reason given is that it is not a priority, which has a percentage of 43%.
It is useful to note at this point in time that only a minority indicated employee resistance (3. 8%). 1. 4 Methodology The research process started off with a secondary research based on the current performance of the companies of interest. This research uncovered general description of the organisations, their business strategies and the significance of the Human Resource Development/ Training & Development (HRD/T) for their goal achievement. In particular, we wanted to look into their training evaluation efforts and the impact of training evaluation on their training programs.
Some of the tools which we used included corporate materials such as websites, annual reports, brochures, media releases as well as other published secondary data. Such information was also used to refine our primary research procedure. Next, in our Literature Review, our group went beyond the textbooks to discover alternative frameworks which have been developed about evaluation of training program. This section of the report first draws out the scope of our project by defining the concept of training evaluation. Next, it looks at the mportance and reasons for conducting training evaluation. Finally and most importantly, major theories and frameworks utilized in training evaluation are discussed. These theories can be obtained from peer-reviewed scholarly journals and provided more perspectives on how we may handle the research problem. Our preliminary research on electronic databases returned more than 100 journal articles. However, we narrowed down our findings to only four of the most influential and recent frameworks which have been thought to have major contributions in the academic arena of training evaluation.
These findings were then cross-referred with other journal articles to enrich the content and provide a more holistic view of the full models, including the shortcomings and applications. While this aspect of the research is not exhaustive, it provides a comprehensive view of the current trends in training evaluation. After the secondary research, our first task of the primary research was to collect more information from the companies regarding their training evaluation practices. An interview (Appendix C) was conducted with the Head of Department (HOD) from BreadTalk.
From the interview, qualitative data were obtained and analysed. Data analysis was done by exploring the relationship between the current practices and the training outcomes of the companies measured by the performance. From this stage, we were able to identify the shortcomings that these companies had in their training evaluation method. We were also able to tease out some of the major challenges faced by these companies when they attempt to perform training evaluation. Lastly, based on our literature review and data collected, we were able to point out several recommendations to enhance the training evaluation.
The challenge was to fit the recommendations into the specific context of the company, taking into consideration their different organisational structures. The final part of the report would conclude with some of the learning points and implications that HRD professionals reading this report could take away with, in order to make their training evaluation a more effective one. 2. Literature Review 2. 1 Definition of Training Evaluation Training evaluation is the systematic collection of relevant outcomes of the training program to aid in the selection, adoption, value or modification of he program, after determining its effectiveness (M. Osman-Gani, 2008). It involves two components: Formative evaluation and Summative Evaluation (Noe, 2008). The former refers to the evaluation of training that takes place during the program design and development; while the latter refers to evaluation conducted to determine the extent of change in the trainees as a consequent of participating in the training. As such, even though training evaluation is common in the last phase of the training process, training evaluation takes place both before and after the completion of the training program (Noe, 2008). . 2 Theories and Models in Training Evaluation While the importance of HRD has risen over the past years, the rising of new techniques or frameworks did not manage to keep up. It is interesting to note that through our research, the numbers of new theories and models regarding training evaluation remains dismal. Although the importance of training evaluation has risen, many HRD professionals do not view measurement and evaluation as a core area of expertise. One major finding was that most human resource development professionals do not have a predisposition towards measurement and evaluation.
In this project, two of the most influential models will be emphasized on as the major frameworks of training evaluations. These frameworks refer to the 4-Level Model by Kirkpatrick, the Return on Investment model by Jack Philips. These are also highlighted as major influences of the training evaluation methods employed by McDonald’s and Breadtalk. In addition, various others techniques for training evaluation will be briefly mentioned to provide a holistic view of the alternatives and recommendations to be offered in our analysis. 2. 2. 1 Kirkpatrick’s Four Levels of Evaluation
Perhaps the most influential evaluation methodology is Donald Kirkpatrick invention of the four levels of evaluation- Reaction, Learning, Behaviour and Results. This taxonomy was invented more than 50 years ago in 1959, but still garners wide support and usage due to its simplicity. Exhibit D2 in Appendix D shows how the four levels of the fit together in the evaluation process. The author, in his original works, suggested that there were three major reasons to evaluate a training program: to determine if a program should be continued; to improve a program; and to ensure learning compliance Kirkpatrick D. , 1993). Recently, his son and him worked together and came up with three other important reasons to utilize the four levels of evaluation. They are: Maximizing the value of training; aligning training with strategy; and demonstrating the value of strategy (Kirkpatrick J. , Aug 2007). The four levels of the evaluation are explained in greater details below. A summary of the findings can be found in Exhibit D1 of Appendix D. Level 1: Reaction Evaluation at this level measures how the trainees react to the training.
This level is often measured with attitude questionnaires, otherwise known as “smile sheets” that are distributed after the training. Trainees may also be interviewed at the end of the training program. In short, this level measures one thing: the learner’s perception and attitudes towards the course. As Kirkpatrick himself has mentioned, “Reaction may best be considered as how well the trainees liked a particular training program. “(1993). Hence, in measuring reaction, trainers are able to find out the how best to promote the learning process.
The aim of evaluation at the “Reaction level” is to build or modify the design of a learning package, thus helping the learners to close a performance gap. However, this level is not indicative of the training’s performance potential as it does not measure what new skills the learners have acquired or what they have learned that will transfer back to the working environment. This will be addressed in level 2 of the framework. Level 2: Learning This level of evaluation addresses the main question of whether the participants learnt anything.
Focus is placed on the kind of knowledge acquired, the skills developed or enhanced, and the attitudes that have been changed (M. Osman-Gani, 2008). This requires post-testing effectively combined with pre-testing, so that one can differentiate between what they already knew prior to training and what they actually learned during the training program. Measuring the learning that takes place in a training program is important in order to validate the learning objectives. This knowledge allows a judgment to be made about the learner’s capability for performance and further helps to define what the learning must produce.
For the learner, however, such assessments are statistical instruments that normally poorly correlate with the realities of performance on the job (Clark, 2008). Thus, the next level exists to assure that the learning transfers to the job. Level 3: Behaviour This level of evaluation determines if the learners can actually put their newly acquired knowledge and skills to use on the job. Sadly, it is not often performed in reality. This evaluation involves testing the students capabilities to perform learned skills while on the job, rather than in the classroom.
Level three evaluations can be performed formally (testing under actual working conditions) or informally (observation) (Clark, 2008). It determines if the correct performance is now occurring by answering the question, “Do people use their newly acquired learning on the job? ” Measuring performance is important as the main purpose of training is to improve results by having the applying new skills which they have learnt to the job. Thus, the actual Level Three measurements will typically involve someone closely involved with the learner, such as a supervisor, in order to accurately measure performance at a job.
Although it takes a greater effort to collect this data its value is important to the training department and organisation as the data provides insight into the transfer of learning from the classroom to the work environment and the barriers encountered when attempting to implement the new techniques learned in the program. Level 4: Results This is the final impact that occurs to an organisation after training. It measures the training program’s effectiveness, that is, “What impact has the training achieved? ” These impacts can include such items as monetary, efficiency, moral, teamwork, etc.
As a result, this level does not look at any one individual but the overall effect of all individuals in the organisation (M. Osman-Gani, 2008). While it is often difficult to isolate the results of a training program, it is usually possible to link training contributions to organisational improvements. Collecting, organizing and analysing level four information can be difficult, time-consuming and more costly than the other three levels, but the results are often quite worthwhile when viewed in the full context of its value to the organisation. Critiques of Kirkpatrick’s Four- Level Model
Like all models, Kirkpatrick’s Four- Level Model of Evaluation has its fair share of criticisms. One of the fiercest criticisms came from Ed Holton in 1996, which stated that the four levels are flawed- that it is merely taxonomy because there is no casual relationship between the various models (Holton, 1996). In an interview conducted in 2004 with Jack Philips, Donald Kirkpatrick responded by contending that he saw no value-add in research that showed correlation between the levels, as these do not add any insights to training and development (Kirkpatrick J.
P. , 2004 ). Reeves and Hedberg (2003) also criticized Kirkpatrick’s approach as overly simplistic for the following reasons: Control groups are rarely feasible in education or training contexts; paper-and-pencil tests lack reliability and validity in measuring KSAs; 100% response rate is unrealistic; and valuation results are not the only input to decision-making within an organisation. In the experience of Reeves and Hedberg, results are often influenced by many factors other than training.
They advised against concentrating too much effort in one evaluation strategy and advocate collecting data from multiple sources using multiple methods. As we move from level one to level four, the evaluation process becomes more difficult and time-consuming; however, it provides information that is of increasing value. Each evaluation level should be used to provide a cross set of data for measuring training program. Due to the rigorous nature of obtaining valuable data, many companies do not go to the fourth level of the level, and they are thus limited of the strength of the evaluation.
This is a challenge that HRD professionals have to overcome in their training evaluation process. This will be touched on with greater details later in this report. 2. 2. 2 Jack Philip’s Five Level Return On Investment (ROI) Model Return on Investment, or ROI, is a form of evaluation that allows organisations to find out if a training program has been profitable for the company (Adelgais). It was added to Donald Kirkpatrick’s four-level framework to measure the monetary benefits of the program as compared to the cost of implementation (Philips, April 1996).
The ROI model helps to demonstrate program success and impact of training and educational programs, assess the value of a training and development program, and many other uses as well (Phillips, August 2007). Many organisations have used this framework to help them in their evaluation process. Steve Mapp, Senior Manager, Learning and Quality at Loyds TSB was even commented that the Philips methodology was valuable not just for the fifth, monetary, level of evaluation but also for the metrics and templates that help practitioners with the evaluation process (Steve Mapp, 2007).
The first four levels of this model are the same as the Kirkpatrick’s four-level model, that is, reaction, learning, behavior and results. The last level, level five, ROI can be calculated by converting productivity and quality improvements to monetary values and comparing these to the full program costs to yield an ROI value (Philips, April 1996). To calculate ROI, a process model is required. This involves four sections, as can be seen in Figure 1 (Appendix E) (Certification in the ROI Methodology, 2006).
The first section, evaluation planning, requires the development of objectives of solutions and evaluation plans and baseline data. Basically it is the inputs or indicators required. The second section, data collection, includes both hard data and soft data. Methods of collection include follow-up questionnaires, action plans, performance contracts, performance monitoring, and program assignments (Dave Jennings, November 2001). In the third section, data analysis, it begins with the deliberate attempts to isolate the effects of training on the data items (Philips, April 1996).
This is because many factors influence performance data after training. Thus this step is necessary to determine the amount of improvement directly related to the training program, and also to maintain credibility (Dave Jennings, November 2001). Next, data are converted to monetary values and compared against program costs to determine the program’s monetary benefits (Dave Jennings, November 2001). Following that, ROI is calculated by using the net program benefits divided by total program costs (Dave Jennings, November 2001).
The net program benefits include “hard benefits”, which refer to results that can be easily converted to monetary values; and “soft benefits”, which are more difficult to convert. “Hard benefits” can be in the form of increased productivity (units produced, items sold, forms processed, tasks completed, etc. ), improved quality (less scrap, less waste, less rework of product, less defects, etc. ), reduced turnover, reduction in lost-time injuries, reduction in workers compensation insurance claims and increased in customer satisfaction (Consulting).
The model also recognizes that there should be intangible benefits that will be presented along with the ROI calculation (Philips, April 1996). This arises when it is not practical or credible to convert hard and soft data to monetary values. Training results, such as improved communication, enhanced corporate image, improved conflict resolution, increased sensitivity to human diversity, improved employee morale and increased employee loyalty are harder to assign a monetary value and these are often referred to as “soft benefits” or otherwise known as intangible benefits (Consulting).
Although not everything can be reduced to dollars, the ROI model provides an effective tool for gauging the value of the organisation’s training program; and when used correctly, ROI can be ideal for proving the tangible value of the professional development programs (Dave Jennings, November 2001). The possibility that an organisation will continue to invest in training depends greatly on the benefits it receives from that training. In the past organisations have viewed training as a necessary expense rather than an investment.
Emphasis was on cutting the cost of this expense by making it more efficient. However, now organisations are beginning to view training as an investment where knowledge and skills are being viewed on equal basis with monetary assets. Thus obtaining an accurate measure of ROI is a powerful method for demonstrating the effectiveness of the training program (Consulting). Critiques of Jack Philip’s ROI model There has been rising interest in measuring monetary benefits (ROI) over the last few years and this phenomenon can be explained with the help of the following factors: •The growing size of HRD budget Measuring output due to pressure from Total Quality Management to emphasise on effectiveness •Business mindset of HRD managers that emphasise economic issues •Threat of outsourcing, where HRD has to proof its contribution to the organisation ROI can help to measure the value that training and HRD function contribute to the organisation. There are critics, however, that expressed their views against the use of ROI, stating that it can be time consuming and complex to carry out. (Redford, 2007) 2. 2. 3 CIRO (Context, Input, Reaction, Outcome) Model In 1970, Warr, Bird and Rackham presented another four-level framework based n Kirkpatrick’s four-level framework. CIRO stands for context, input, reactions and outcome. They believe that before assessing reactions and outcomes, there needs to be an analysis of the context and inputs (Travis K. Brewer, August 2007). The context, input, and outcome evaluations in this model are essentially the same as the context, input, and product evaluations in the CIPP (Context, Input, Process, Product) model, but CIRO emphasizes trainee reaction as a source of information to improve the training program (Human Resource Development). The first step is context evaluation.
It seeks to measure the context within which a training program takes place. It scrutinizes the way performance needs were identified, learning objectives were established, and the way the objectives link to and support the necessary competencies. In addition, it ought to consider how these components of the program reflect the culture and structure of the organisation (Harris). The second step is input evaluation. This refers to the measurement of a number of inputs to a training program, with a view to assisting managers in the process of identifying those which will be most cost-effective.
It focuses on the resources needed to meet performance needs and the content and delivery methods that allow the training to be achieved (Harris). The third step, reaction evaluation, is similar to the Kirkpatrick’s model, which aims to determine how the trainees reacted to the training program. This type of evaluation draws on the subjective opinions and experiences of trainees about the training program and how it might be improved (Harris). Finally, the outcome evaluation measures the training and development outcomes against the benchmark of the program’s objectives.
It focuses on the achievements gained from the activity and is assessed at three levels: immediate, intermediate and ultimate evaluation (Hogan, Fall 2007). Immediate evaluation attempts to measure changes in knowledge, skill, or attitude after training and before a trainee returns to the job. Intermediate evaluation refers to the impact of training on job performance and how learning is transferred back into the workplace (Santos, 2003). Lastly, ultimate evaluation attempts to assess the impact of training on departmental or organisational performance in terms of overall results (Hogan, Fall 2007).
The main strength of the CIRO model is that the objectives (context) and the training equipment (input) are considered. Also, it focuses on measurements both before and after the training has been carried out (Tennant, 2002). 3. Current Practices of BreadTalk A phone interview conducted with Mr Ng Kwee Lam, the Head of Department (HOD) of Training from BreadTalk has allowed us to better understand its training programs and how they carry out their evaluation methods. According to Mr Ng, BreadTalk has training programs catered to all the new employees depending on their positions.
For example, sales training will be given to employees who are in the sales line. Most of their current training programs are conducted in the form of off-job training. This is carried out in classroom environment, where trainees attend class session and listen to the trainers. One interesting point to note is that BreadTalk does not hire external consultants to do the training. Instead, they rely on senior employees who have attended similar training programs to train and coach the new employees. One possible reason could be the cost of hiring of external vendors to conduct training.
Furthermore, it would be more trustworthy and reliable to have BreadTalk’s own employees to conduct the training since they will impart the relevant knowledge to the new employees. According to Mr Ng, plans to modify current off-the-job training programs into on-the-job training programs have been made and will most probably be launched by early next year. These on-the-job training will be carried out on-site, where new employees will be trained at the respective outlets they are working in. Each of them will own a record tab, where senior employees who trained them will write comments regarding their progress.
The duration of each training session will also be recorded. After a period of time, a post-training assessment will be done to evaluate the performance of the employee. Assessment will be done in the form of observation of the trainees’ skills and also through the trainees’ interaction with the customers. From there, the performance of the trainees can be evaluated based on their improvement. In general, BreadTalk does not have any form of evaluation as it is currently still in the transition stage.
However, they do use the Kirkpatrick’s four-level model of evaluation at the first and second level. At the first level, which is Reaction, BreadTalk uses a happy sheet, where trainees were asked to provide feedback on the training program in terms of the following factors: the training environment, the trainer and the training materials provided. During learning, the second level of Kirkpatrick’s four-level model, tests, in the form of multiple choice questions are given to find out what trainees have learnt during the training course.
Hence we can see that currently BreadTalk only evaluates its training programs at the first two levels of the Kirkpatrick’s four-level model. 3. 1 Challenges Faced and Strategies Taken When asked about challenges faced when implementing these evaluation programs, Mr Ng did not explicitly mention any challenges targeted at the evaluation programs but did point out one challenge faced when implementing training programs. This challenge was language problem. Initially, training programs were all conducted in English.
However, BreadTalk realized that they had employees who were Mandarin speaking. To tackle this problem, they had to do translation for these employees, which was very troublesome and time-consuming. It was some time later when they finally conducted training programs in both English and Mandarin, so as to cater to all employees. However, there are many common challenges HRD personnel face when implementing training evaluation programs. Deborah Gwinner cited the following ten reasons why companies do not effectively carry out training evaluation (Abernathy).
The reasons are lack of planning, assumption that training is a cost rather than an investment, lack of sponsorship, lack of budget, lack of appropriate resources, lack of understanding of what is important to measure, evaluation techniques are not accurate in capturing human performance, lack of valid measurements, lack of data collection, and finally, lack of data analysis and summary. Lack of experience in evaluation could be one of the key factors for the above reasons. It could also be due to the lack of awareness of the availability of evaluation models or the complexities involved in the training evaluation processes.
These reasons explain why BreadTalk does not effectively carry out training evaluation. 3. 2 Future Path So far, we have not really go into detail of BreadTalk’s training evaluation program as it is currently in the transition stage and many issues and new plans are still uncertain and sensitive to divulge just as yet. But we can expect BreadTalk to have more in-depth evaluation of their training programs. As mentioned by Mr Ng, BreadTalk plans to launch the use of Kirkpatrick’s four-level model at all four levels in two to three months time.
Jack Philip’s ROI model will also be implemented then. In fact, according to Mr Ng, BreadTalk already has prepared all the instruments and materials required for evaluation at all the four levels. Based on what we have obtained from BreadTalk’s website and the phone interview with Mr Ng, we will propose some recommendations in the next section. 4. Summary and Recommendations 4. 1 McDonald’s Evaluation Strategies McDonald’s is the leading global foodservice retailer with more than 30,000 restaurants serving 52 million people in more than 100 countries each day.
More than 70% of McDonald’s restaurants worldwide are owned and operated by independent local men and women (McDonald’s Corporation). In the UK, it employs more than 60,000 people and serves over 1. 5 million customers daily in excess of 1,200 restaurants. The McDonald’s service culture was based on speed and convenience. The need, while retaining those elements, was to enhance the existing hospitality culture to exceed customer expectations. This led the company to design and implement a customer care program in 2004, consisting of two phases – Hospitality Plus workshops, and the Customer Care course.
To assess the impact of the training programs, surveys were conducted to gather feedback from participating staff and their managers. Almost all the comments were positive; staff had learnt more about hospitality and the standard of customer care. By the end of the third quarter 2005, turnover of staff had fallen by more than 8% compared to the same period the year before, implying that Return On Investment (ROI) had improved (Pollitt, 2007). Thus we can see that systems for monitoring and evaluating training programs are essential in multi-unit leisure retail organisations.
Finally, it can be noted that McDonald’s is a company that makes a major commitment to training as a means of achieving strategic objectives to increase customer and employee satisfaction, lowering operating costs and increasing market share (Andrew Eaglen, 2000). Rob Griffiths, a training consultant, commented that McDonald’s operates in a competitive environment because customers increasingly vary their choice of eating experience. The McDonald’s service culture was based on speed and convenience and at one time, McDonald’s UK had the best service standards in the quick-service restaurant (QSR) sector.
In 2004, the company wanted to re-establish this competitive advantage in order to enhance the existing hospitality culture to exceed customer expectations. This led the company to design and implement a customer care program, designed to address the training needs of existing customer-care staff and to develop future customer-care staff. The program was launched in two phases – Hospitality Plus workshops, and the Customer Care course. Each phase was designed to ensure that customer-care staff would learn more about hospitality and improve interaction with customers (Pollitt, 2007).
Phase 1 Before the start of Phase, a Hospitality Plus workshop pilot was successfully conducted and a “train the trainer” session was held before starting the real training (Pollitt, 2007). Phase 2 The second phase of the program was rolled out to regional departments through train-the-trainer sessions. Workbooks and knowledge tests based on information contained in the workbook were prepared for the staff. In addition, a one-day customer-care course was also delivered (Pollitt, 2007). Assessing the impact
To assess the training program, surveys were sent to participating staff and managers and it was found that almost all of the staff responded positively towards the training program, commenting that they learnt more about showing great hospitality to customers and that they find the workbook useful. Their managers also agreed that ever since the training program had begun, the standard of customer care had improved. To assess he long-term effect of Phases 1 and 2, another survey was sent to 88 other restaurants in the southern region, whose staff had attended the customer care training.
The survey aimed to gauge the change in participants’ behaviour since completing the workbook and attending the course. The results of the survey showed that most of the staff found the workbook informative and useful; and the course enjoyable. Mystery-shopper surveys also indicated that staff presence in the restaurant customer areas had improved, and that staff were friendlier and engaged in better interaction with customers (Pollitt, 2007). To measure the wider, long-term impact of the training program, McDonald’s used feedback rom “Fast Track”, a regular telephone-based survey of customer opinion. Fast Track revealed that between the last quarter of 2004 and the third quarter of 2005, the number of customers agreeing that McDonald’s service was friendly and courteous rose 9%, while the number believing that the service was fast and efficient increased 10% (Pollitt, 2007). Through using surveys as a form of training programs evaluation, it can be concluded that the these programs had not only improved levels of customer care in McDonald’s, but have also reduced turnover among customer-facing employees.
Furthermore, the workbook and the customer care course continued for new employees as training represents real value for money, given the number of people trained and the cultural impact the company has gained (Pollitt, 2007). 4. 2 Comparing BreadTalk with MacDonald’s In coming up with recommendations for BreadTalk, we have looked at the strategies that BreadTalk with another big player in the Food and Beverage industry, McDonald’s. McDonald’s seems to have been efficient in implementing the Kirkpatrick’s models up to the point of calculating ROI.
While we want to adapt the uses of McDonald’s strategies, we would also like to do a comparison between the two corporations. Both companies are in the food industries, selling ready-made food to customers. The main concerns for these companies would be the hygiene and sales. Also, both McDonald’s and BreadTalk have a global presence, although McDonald’s is apparently more influential, with outlets all over the world. BreadTalk’s outlets are mainly situated in the Asia Pacific region. While both companies use Kirkpatrick Model to evaluate the training programs, McDonald’s seems to be able to implement it to a higher level.
BreadTalk, on the other hand, seems to lack the experience in doing so, and they only evaluate on the Reaction and Learning levels. Further, in the time of transition, BreadTalk may face difficulties in implementing higher levels of the Models. The competition for both companies is highly intensive, as consumers usually like to vary their taste. Our firm believes that in this aspect, BreadTalk faces an even bigger challenge, particularly in Singapore, where bread is consumed more as a commodity rather than a brand.
BreadTalk needs to compete with other bakeries and confectionaries and to consistently justify the higher price they charge for their products. This is especially significant when the company is pitted against bakeries in the heartland neighbourhood stores. In summary, the comparison between these two companies show that there are a lot of similarities between them, and thus BreadTalk may ride on the success of McDonald’s to adopt their evaluation methods of their training programs. 4. 3 Our recommendations to BreadTalk
Based on the interview with Mr Ng, it is already established that BreadTalk is in the midst of moving towards a more wholesome Kirkpatrick Evaluation, with the possibility of implementing ROI in future. Our recommendations would come at the right time for them to conduct even better evaluation in future. 4. 3. 1 Kirkpatrick’s four level Evaluation Model a. Measuring reaction: Identify the needs of the non-English speakers and construct special programs to enhance their learning b. Measuring behaviour: Mystery shoppers c. Measuring results: Reduction in customer complaints, increasing in sales d.
Measuring results: Solomon Four Group Models e. Going beyond the cognitive and skill-based outcomes to measure the affective outcomes. 4. 3. 2 Jack Philip’s ROI model Since BreadTalk has plans to evaluate its training programs using the ROI model, it would be useful to know that different levels of management make different sorts of decisions, so it is appropriate that they use different measures of ROI. Appendix F shows a table of which function of an organisation should focus on what kind of measurement. 4. 3. 3 Making Evaluation Work in Your Organisation
Before BreadTalk embarks on its new evaluation system, we would like to highlight the possibility of resistance to evaluation. According to this book “Making Training Evaluation Work” (Jack J. Philips, 2004), it states that learning and development staff members and others closely associated with their programs usually resist evaluation for two reasons: 1. Anticipation of the time needed to implement the process. 2. Concern about the consequences of the final outcomes The book also provides building blocks to overcoming resistance. 4. 3. 3. 1 Identify a champion A leader must be identified as he or she serves as a champion for evaluation.
The evaluation leader is usually a member of the learning and development staff who has this responsibility full time in larger in larger organisations or part time in smaller organisations. This leader must understand the intent of evaluation, including ROI, and sees the vast potential for its contribution. Of course, more importantly, this leader must be willing to show and teach others. 4. 3. 3. 2 Assign Roles and Responsibilities Determining specific responsibilities is important to prevent the confusion created when individuals are unclear about their evaluation assignments.
Most responsibilities apply to two broad groups. The first group is the entire learning and development staff. Some of the specific responsibilities include: •Ensuring that the needs assessment includes specific business impact measures •Developing specific learning, application, and business impact objectives for each program •Focusing the content of the program on performance improvement •Ensuring that exercises, tests, case studies, and skill practices relate to the desired objectives •Keeping participants focused on program objectives Communicating rationale and reasons for evaluation •Assisting in follow-up activities to capture the appropriate data •Providing assistance for data collection, data analysis, and reporting •Developing plans for data collection and analysis •Presenting evaluation data to a variety of groups •Assisting with the design of instruments Assignment of responsibilities keeps evaluation from becoming disconnected from major learning and development activities. The second group to which responsibilities should be assigned is the technical support function.
For this group, responsibilities revolve around several key areas: •Designing data collection instruments •Providing assistance for developing an evaluation strategy •Coordinating a major evaluation project •Analysing data, including specialized statistical analyses •Interpreting results an making specific recommendations •Developing an evaluation report or case study to communicate overall results •Presenting results to critical audiences •Providing technical support in any phase of evaluation 4. 3. 3. 3 Assess the Climate
The use of an assessment process provides an understanding of current status. The organisation can then plan for changes and pinpoint particular issues that need support as the measurement and evaluation process is implemented. 4. 3. 3. 4 Establish Evaluation Targets Specific targets for evaluation measures and projects are necessary to make progress with measurement and evaluation. Targets enable the learning and development staff to focus on the improvements needed within specific evaluation categories.
The few points mentioned above are not exhaustive. There are other implementations as listed below: •Develop a project plan •Revise policies and procedures •Involve the learning and development staff •Teach the staff •Select programs for evaluation •Report progress •Prepare management team •Remove the barriers For lasting values, measurement and evaluation must be perceived as routine, not a one-time event or an add-on process. It must be considered early and often in the learning and development cycle.
Having said that, from the Employer Supported Training report in Appendix B, Exhibit B3, we can see that resistance to training evaluation is not the main problem and thus if evaluation of the employee performance shows good results, there should be no concerns regarding resistance to evaluation. Thus we hope that through the recommendations, BreadTalk will be more confident and ready to implement their new evaluation system at a full scale. 4. 4 Recommendations for McDonald’s
As this is the expansion period for McDonald’s to place their evaluation program in the global context, McDonald’s need to work on the following: a. Starting small in countries that may lack the infrastructure to do a full fledged ROI analysis like that in US. This can be done by evaluating at lower levels of the Kirkpatrick’s models first, before moving up to the ROI. b. Trained evaluation officers can be sent to these countries so that the evaluation tools of US may be adapted, or new evaluation tools can be implemented. In this endeavour, McDonald’s must also make sure that cultural differences are reconciled.
To minimize the cultural incompatibility of the training tools, McDonald’s can segment their whole market into regions with close cultural similarities. In this case, an evaluation tool can be used across a region without the need for expensive and time-consuming development and adaptation for every single country. 4. 5 Limitations 1. As mentioned by Mr Ng, BreadTalk is currently undergoing a transition into the Kirkpatrick evaluation. The results obtained may be inaccurate or inapplicable simply because the tools are not refined enough to capture the essence of training.
As such, the long term benefits of higher levels of Kirkpatrick and ROI may not be observed immediately. 2. As BreadTalk is expanding into the Asia region, the presence of global franchises and businesses will result in a currency issue. BreadTalk has to account for the currency differences if they are to implement the training on a regional basis. This is important for the calculation of ROI as well. 3. Measuring ROI may be too costly to take place; the company needs to budget for the measurement of ROI to justify its existence in improving quality of the training programs. 4.
The company needs to ensure that there is seamless co-ordination between the accounting and HR functions to ensure effective measurement of performance due to training intervention. 5. Conclusion We have been talking about training evaluation, but Martin Schmalenbach argues that there are times when perhaps evaluation should not take place. Training may need to take place in a hurry, for whatever good reasons, to the extent that the elapsed time needed to do the evaluation properly is too long to be able to directly influence the decision as to whether to commit resources to the training or not.
There is one good reason to do the evaluation anyway, and that is to develop some data and that can help to validate the decision taken and so support organisational learning about how to respond to such scenarios in the future. Exhibit G1 in Appendix G shows when training evaluation is needed and when it is not needed (Schmalenbach, 2005). Martin Schmalenbach also gave us an idea on which approach is suitable for which type of audience, and this is shown in Exhibit G2 of Appendix G (Schmalenbach, 2005). A final note, not everyone agrees that the Kirkpatrick’s model should be used for evaluation. Watson) Elwood Holton argues that it is not even a model, but rather merely a taxonomy. Holton then carry on to propose a new model for evaluation of training that, unlike Kirkpatrick’s four-level system will “account for the impact of the primary intervening variables such as motivation to learn, trainability, job attitudes, personal characteristics, and transfer of training conditions. ” Some of the differences between this model and Kirkpatrick’s model are: 1. Absence of reactions (level one) as a primary outcome. 2. Individual performance is used instead of behavior. 3.
The inclusion of primary and secondary influences on outcomes. Three primary learning outcome measures are proposed: 1. Learning: achievement of the learning outcomes desired in the intervention. 2. Individual performance: change in individual performance as a result of the learning being applied on the job. 3. Organizational results: consequences of the change in individual performance. And, according to the model, the three primary influences on learning are: 1. Trainee reactions 2. Motivation to learn 3. Ability This model has been proposed but needs to be tested, Holton says.
A simpler model may emerge from such testing; for example, perhaps measuring only primary intervening variables will be sufficient, or perhaps only a few key variables within each category should be measured. Bibliography Abernathy, D. J. (n. d. ). Thinking outside the evaluation box. ASTD website. Adelgais, S. (n. d. ). Encyclopedia of Education Technology. Retrieved from Return on Investment – An Evaluative Framework: http://coe. sdsu. edu/eet/articles/roi/index. htm Andrew Eaglen, C. L. (2000). The benefits of training in leisure retailing: a case study of McDonald’s restaurants. Strategic Change , 333.
BreadTalk – Boutique Bakery: A Contemporary Lifestyle Concept. Economic Development Board . BreadTalk. (2008). Competitive Strengths. Retrieved from BreadTalk: http://breadtalk. listedcompany. com/strengths. html Brinkerhoff, R. O. (May 2006). Getting Real About Evaluation. Training & Development , 24. (2006). Certification in the ROI Methodology. ROI Institute. Chapman, A. (1995). Retrieved September 24, 2008, from Donald L Kirkpatrick’s Training Evaluation Model : http://www. businessballs. com/kirkpatricklearningevaluationmodel. htm Clark, D. (2008, July 9). Instructional System Design Handbook.
Retrieved September 22, 2008, from Instructional System Design – Evaluation Phase – Chapter VI: http://www. nwlink. com/~donclark/hrd/sat6. html#kirkpatrick Consulting, k. Calculating the ROI of Training. Cross, J. (2001). A Fresh Look at Return on Investment. Internet Time Group . Dave Jennings, C. (November 2001). Proving Your Worth. Professional Development Forum Online. Dutkowsky, S. (n. d. ). Trends in Training and Development – THE NEW ECONOMY, TRAINING IN U. S. COMPANIES, WHO DOES THE TRAINING IN CORPORATIONS? Retrieved from STATEUNIVERSITY. COM: http://careers. stateuniversity. com/pages/852/Trends-in-Training-Development. tml Evans, D. (March 2007). Are There New Approaches to the Evaluation of Learning Programs? WBI Evaluation Studies. Harris, F. Evaluation models and methods. In F. Harris, The Role of Capacity-Building in Police Reform. Hogan, R. L. (Fall 2007). THE HISTORICAL DEVELOPMENT OF PROGRAM EVALUATION: EXPLORING THE PAST AND PRESENT. Online Journal of Workforce Education and Development , 9 – 10. Holton, E. (1996). The Flawed Four-Level Evaluation Model. Human Resource Quartely , 21. Human Resource Development. (n. d. ). Retrieved from http://www. pondiuni. org/DDE/Human%20Resource%20Development. pdf Jack J. Philips, P. P. (2004).
Making Evaluation Work in Your Organization. In P. P. Jack J. Philips, Making Training Evaluation Work. American Society for Training and Development. Kirkpatrick, D. (1993). Evaluating Training Programs. Kirkpatrick, J. P. (2004 , January). The Evaluation Heavyweight Match. (D. Stoel, Interviewer) Kirkpatrick, J. (Aug 2007). The Hidden Power of Kirkpatrick’s Four Levels. Training & Development , 34. M. Osman-Gani, A. (2008). Performance Development & Training with Asian Perspective. Singapore: McGraw-Hill Education (Asia). Manpower, M. R. (September 2007). Employer Supported Training 2006. MOM. McDonald’s Corporation. n. d. ). Retrieved from About McDonald’s: http://www. mcdonalds. com/corp/about. html Noe, R. A. (2008). Employee Training & Development. Singapore: The McGraw-Hill Education (Asia). Philips, J. J. (April 1996). Measuring ROI: The Fifth Level of Evaluation. Technocal & Skills Training , 1. Phillips, J. J. (August 2007). Using ROI to Demonstrate HR Value in the Public Sector: A Review of Best Practices. Strategic Planning , 14. Pollitt, D. (2007). McDonald’s serves up better customer care and lower employee turnover: Two-stage training for more than 4,000 staff. Human Resource Management International Digest , 23-26.
Redford, K. (2007, June 19). Measuring return on investment (RoI) in training could be pointless. Retrieved from Personneltoday. com: http://www. personneltoday. com/articles/2007/06/19/41279/measuring-return-on-investment-roi-in-training-could-be-pointless. html Santos, A. &. (2003). Employees perceptions and influence on training. Human Resource Management Journal , 27-45. Schmalenbach, M. S. (2005, January 24). trainingzone. co. uk. Retrieved from Training Evaluation: http://www. trainingzone. co. uk/cgi-bin/item. cgi? id=136034 Steve Mapp, S. M. (2007, July). Evaluation and ROI in Lloyds TSB. Tennant, C. B. 2002). The design of a training program measurement model. Journal of European Industrial Training , 230-241. Travis K. Brewer, A. B. (August 2007). USE OF PHILLIPS’S FIVE LEVEL TRAINING EVALUATION AND RETURN ON INVESTMENT FRAMEWORK IN THE U. S. NON-PROFIT SECTOR. Watson, J. S. (n. d. ). Evaluation of Training. Retrieved from www. ispi. org/ispi-cpc/resrcs/Evaluatn. doc Appendix A – The Success Case Method Appendix B – MOM report on Employee Supported Training: Charts Exhibit B1: Proportion of Training-providing Establishments that Evaluated Effectiveness of Staff Training, 2003-2006 (Manpower, September 2007)
Exhibit B2: Common Evaluation Procedures, 2003 – 2006 (Manpower, September 2007) Exhibit B3: Main Reasons (%) for not having Evaluation Procedures, 2003 – 2006 (Manpower, September 2007) Appendix C – Phone Interview with Mr Ng Kwee Lam, HOD of Training, BreadTalk on Monday, 20th October 2008 @ 5. 20pm HQ: Hello Mr Ng, I’m representing a group of my friends and myself. We’re students from NTU and I believed my friend, Sara has talked to you earlier this morning. Just a bit of background of our project. We will be covering on the evaluation of the training programs of the company we’re working on.
I have some questions which I hope you can assist us in. Thank you. Mr Ng: Yup, ok sure. HQ: What kind of training programs does BreadTalk offer for its employees? Mr Ng: We have different kind of training programs for different positions. For example, for the sales personnel, we offer sales training. HQ: How are these training programs administered? Mr Ng: Currently, most of our training programs are conducted in the form of off-job training, where trainees attend classes in classroom environment and listen to the trainers. HQ: So who are these trainers? Are they internal or external consultants?
Mr Ng: We do not hire external consultants to give training. Instead, we offer house-in training, where senior employees who have gone through training will train the new employees. HQ: You mentioned that these training programs are off-job. Does BreadTalk have any plans to change this form of training? Mr Ng: Yes, certainly. In fact, we have already made modifications to some of our training programs to on-job training. HQ: So how are these on-job trainings being carried out? Mr Ng: Well basically, they will be carried out on-site where they will be trained at the respective outlets they are working in.
They will each have a record tab, where names of the senior employees who train them and the duration of the training will be noted down. After some time of training, an assessment will be done. HQ: Do you mean like a post assessment? How will it be done? Mr Ng: Yes, it will be a post assessment of the training. Basically, assessment will be done by the senior employees in the form of observation of their skills and also through the trainees’ interaction with the customers. From there, we will assess the trainees’ performance.
HQ: Well, now that we’re talking about assessment of trainees’ performance, does BreadTalk engage in any form of evaluation of the effectiveness of the training programs, like ROI? Mr Ng: Well, we do not really have any form of evaluation as we’re now at the transition stage, but we have plans to use models like Kirkpatrick’s 4-levels model to evaluate at all 4 levels or in fact Philip’s ROI model. HQ: So how then do you measure the effectiveness of current training programs? Do you evaluate at any of the levels of the Kirkpatrick’s model? Mr Ng: We do evaluate at the first and second levels.
At the first level, people usually call it the happy sheet, where trainees were asked to give feedback on the training environment, the trainer himself/herself, and also the training materials provided. For level 2, tests will be given in the form of MCQ to find out what they have learnt from the training course. HQ: In the event of carrying out these evaluation programs, are there any problems that you encountered? And how did BreadTalk solve them? Mr Ng: Yes, definitely there were problems. One of the most fundamental challenges we faced is language problem. Initially we thought that all training programs should be conducted in English.
However we realized that we have employees who were Mandarin-speaking. Thus we had to do translation for them. Some time later, we realized that we had to conduct our training in Mandarin as well to cater to these Mandarin-speaking trainees. When it comes to evaluating training programs, I think most of the companies also face similar problems of the lack of planning, budget and experience in evaluation. HQ: You mentioned about plans to use Kirkpatrick’s and Jack Philip’s models. When will this take place? Mr Ng: Hopefully in 2-3 months time, we will be implementing the new system of evaluation.
In fact, we have all the instruments and materials prepared and ready to use for evaluation at all the four levels of Kirkpatrick’s model. HQ: Well, that’s all I have for you Mr Ng, we really thank you for your time and your help. Mr Ng: All the information that I’ve said so far are the rough idea of what we will be doing. Because we’re now still in the transition stage, many issues are still uncertain and sensitive thus I cannot answer all your questions. Perhaps by next year, we can have more detailed explanations and answers for you. Appendix D – Kirkpatrick’s Four Levels of Evaluation
Exhibit D1: Summary of Kirkpatrick’s Four Levels of Evaluation (Chapman, 1995; M. Osman-Gani, 2008) Evaluation type Evaluation description and characteristicsExamples of evaluation tools and methodsRelevance and Practicability Level 1: ReactionHow the delegates felt about the training or learning experience. Did the trainees like and enjoy the training? Did they consider the training relevant? Did they like the venue, the style, timing, domestics, etc? Level of participation, ease and comfort of experience. Perceived practicability and potential for applying the learning. 1. ‘Smile sheets’, feedback forms. 2.
Verbal reaction, post-training surveys, interviews or questionnaires. 3. Online evaluation or grading by delegates. 1. Can be done immediately the training ends. 2. Very easy to obtain reaction feedback 3. Not expensive to gather or to analyse for groups. 4. Qualitative data may be difficult to analyse. Level 2: LearningThe measurement of the increase in knowledge due to the training program. Did the trainees learn what was intended to be taught? Did the trainee experience what was intended for them to experience? What is the extent of advancement or change in the trainees after the training, in the direction or area that was intended? . Assessments or tests before and after the training. 2. Performance or skill practices, simulations1. Simple to set up; clear cut for quantifiable skills 2. Less easy to assess attitudinal development Level 3: BehaviourThe extent of applied learning back on the job. Did the trainees put their learning into effect when back on the job? Was there noticeable and measurable change in the activity and performance of the trainees back in their roles? Was the change in behavior and new level of knowledge sustained? Would the trainee be able to transfer their learning to another person?
Is the trainee aware of their change in behavior, knowledge, skill level? 1. Observation and interview over time are required to assess change, relevance of change, and sustainability of change 2. Self-assessment using carefully designed criteria and measurements 3. Assessments can be designed around relevant performance scenarios, and specific key performance indicators or criteria 1. Measurement of behavior change requires cooperation and skill of line-managers. Difficult to control. Level 4: ResultsThe effect on the business or environment by the trainee.
Business or organisational key performance indicators: Sales volumes, values, percentages, timescales, return on investment. Other quantifiable aspects of organisational performance: Numbers of complaints, staff turnover, attrition, failures, wastage, non-compliance, quality ratings, achievement of standards and accreditations, growth, retention, etc. 1. Measures are already in place via normal management systems and reporting – the challenge is to relate to the trainee’s input and influence. 2. Identify and agree accountability and relevance with the trainee at the start of the training, so they understand what is to be mea