Words: 2908
The Role of Evaluation in Vocational Technical Education An Assignment By Achukwu Chimezie B 2009196024F Submitted Impartial fulfilment of the course TED 631 Evaluation in Industrial Education and Technology Department of Vocational Education Nnamdi Azikiwe University, Awka Lecturer: Prof. I. T. Eze Date: 10th August, 2010 Abstract Evaluation in education involves the collection of data and the use of such data to assess the effectiveness or quality of a programme or performance.
Programme in education is established for some purpose and it is the function of programme evaluation to determine the extent to which the purpose of this programme is achieved. Vocational technical education (VTE) like any other programme is no exception. This paper discusses what is evaluation as well as its steps and main roles. The three purposes was not left out and its roles in vocational Technical education. It also provides an opportunity to look at the distinction between formative and summative evaluation, which has become a widely accepted distinction within evaluation methodology.
INTRODUCTION Many people think of evaluation as taking a snapshot of outcomes at the end of a program to prove that it worked or failed. These same people don’t hold evaluation in much regard because they feel they are getting too little information too late in the day, especially if their program fell short of expectations or made no difference at all. Evaluation can, and should, however, be used as an ongoing management and learning tool to improve a project or programme effectiveness. Well-run organizations and effective programs are those that can demonstrate the achievement of results.
Results are derived from good management. Good management is based on good decision making. Good decision making depends on good information. Good information requires good data and careful analysis of the data. These are all critical elements of evaluation. Evaluation refers to a periodic process of gathering data and then analyzing or ordering it in such a way that the resulting information can be used to determine whether your organization or program is effectively carrying out planned activities, and the extent to which it is achieving its stated objectives and anticipated results.
Managers or school administrators can and should conduct internal evaluations to get information about their programs so that they can make sound decisions about the implementation of those programs. Internal evaluation should be conducted on an ongoing basis and applied conscientiously by administrators at every level of an organization in all program areas. In addition, all of the program’s participants (Administrative personnel, External personnel, Teachers and students) should be involved in the evaluation process in appropriate ways.
This collaboration helps ensure that the evaluation is fully participatory and builds commitment on the part of all involved to use the results to make critical program improvements. Although most evaluations are done internally, conducted by and for program managers and staff, there is still a need for larger-scale, external evaluations conducted periodically by individuals from outside the program or organization.
Most often these external evaluations are required for funding purposes or to answer questions about the program’s long-term impact by looking at changes in demographic indicators such as graduation rate or performance of students. In addition, occasionally a manager may request an external evaluation to assess programmatic or operating problems that have been identified but that cannot be fully diagnosed or resolved through the findings of internal evaluation. Program evaluation, conducted on a regular basis, can greatly improve the administration and effectiveness of an institute and its programs.
To do so requires understanding the differences between monitoring, research and evaluation, making evaluation an integral part of regular program planning and implementation, and collecting the different types of information needed by managers at different levels of the organization. Research, Evaluation and Monitoring. The main difference between research, evaluation and monitoring is that research is usually conducted with the intent to generalize the findings from a sample to a larger population.
Evaluation, on the other hand, usually focuses on an internal situation, such as collecting data about specific programs, with no intent to generalize the results to other settings and situations while monitoring, though different from them go hand in hand and are usually taken together. Monitoring is basically getting aware of a system, people or an organization. It is observing a certain situation for any changes that occur over a period of time. In other words, research generalizes, evaluation particularizes and monitoring observe.
What Is Evaluation? As defined by the American Evaluation Association, evaluation involves assessing the strengths and weaknesses of programs, policies, personnel, products, and organizations to improve their effectiveness. -Wiki Evaluation is the process of determining significance or worth, usually by careful appraisal and study. Evaluation is the analysis and comparison of actual progress vs. prior plans, oriented toward improving plans for future implementation.
It is part of a continuing management process consisting of planning, implementation, and evaluation; ideally with each following the other in a continuous cycle until successful completion of the activity. Evaluation is the process of determining the worth or value of something. This involves assigning values to the thing or person being evaluated. Evaluation in education according to Okoro M. O. (2000) involves the collection of data and the use of such data to assess the effectiveness or quality of a programme or performance.
From these definitions of evaluation, one can see that evaluation has an important role to play in the sustainability and successful operation of vocational technical education programme in our institutions of learning. Below, is an overview of the steps of a “typical” evaluation process. The Main Roles of Evaluations People working in the field of programme evaluation have attempted to summarise this wide range of roles into categories so that it is easier to write about and discuss evaluation purposes.
While there is always some difference between the approaches adopted by different authors there is also a general acceptance of the main purposes for which evaluations are conducted. Although we look at only two here, many different taxonomies of evaluation have been proposed in the literature evaluations as: Babbie and Mouton (2001) use the work of Michael Patton to define three main purposes of evaluations as: 1. To make judgements (judgement-orientated) 2. To make improvements (improvement-orientated) . To generate knowledge (knowledge-orientated). Similarly but with one addition, Francis Rubin (1995) defines four categories: 1. To improve performance 2. To make choices and decisions 3. To learn lessons 4. To increase accountability. We will look in more detail at Babbie and Mouton’s three categories. But before we do, we should note that the last of Rubin’s points – to increase accountability is also an important purpose of evaluation, especially in the context of quality assurance.
Evaluation is one means of assessing whether people and institutions do what they have been assigned to do, and so provides a means of checking students, staff and administrative performance, as people will have to provide answers for their actions during an evaluation process. For these kinds of reasons evaluations can sometimes be challenging to conduct, and in such cases evaluators are rarely liked by everyone! Some approaches to evaluation have explicitly sought to address this issue.
For example participatory evaluation seeks to involve programme participants in all parts of the evaluation and approaches evaluation more as learning opportunity than an opportunity for making judgements. To return to our purposes of evaluation – we will look briefly at each of the three purposes defined above. This will also provide us with an opportunity to look at the distinction between formative and summative evaluation, which has become a widely accepted distinction within evaluation methodology. 1.
Judgement-oriented evaluations Think of a situation in which you might need to conduct an evaluation of a programme that has provided teachers with computer training to assess whether they are now able to use computers and whether this has improved their teaching. In this instance you would be conducting a judgement-oriented evaluation. This category of evaluation research includes those studies that set out to determine the worth, value or success of a programme. But, how do you know if the teachers’ teaching has improved?
Central to all evaluations that set out to make judgements is the formulation of criteria against which to judge the success of the programme. A good programme would have clearly defined goals and that those goals would have been translated into measurable outcomes or critical success factors? Noted that, often, programmes do not have clear goals and measurable outcomes, and only when evaluations begin that goals are made explicit and measurable outcomes defined. A core step in conducting a judgement-oriented evaluation is to define these criteria and how they will be measured.
Since we are measuring whether a programme meets specified criteria, then we must be conducting the evaluation after the programme has been implemented or towards the end of implementation, for example, at the end of a newly introduced distance education course. This is where we meet up with the concept of summative evaluation. Summative evaluations Summative evaluation is an evaluation that takes place to make judgements about the overall success or failure of a programme, and is often related to decisions about whether the programme should be continued or not.
Summative evaluation usually takes place towards the end of a programme or after implementation. Administrators often use this kind of evaluation to test whether the programme they have funded/established has achieved the outcomes it was supposed to. The answer to this question may then determine whether further funding/running is granted or not. This kind of evaluation is also conducted to identify the lessons that can be learned from the experience and applied to future projects or programmes. 2. Improvement-orientated evaluations
As the name suggests, improvement-oriented evaluations are concerned with improving the programme while it is being implemented rather than judging how successful it has been during implementation or after completion. This kind of evaluation is usually called a formative evaluation. Here the evaluation helps to ‘form’ the programme by providing a means of assessing the strengths and weaknesses of the programme, looking at how implementation takes place, consider the response of programme participants etc.
The aim here is to identify problems as they occur so that they can be quickly corrected. Sometimes more substantial changes of approach or adjustment of plans may be needed to improve the programme. 3. Knowledge-orientated evaluations Judgement and improvement-orientated evaluations are carried out in order to provide direct input into a specific programme or to assess whether or not the programme is successful. In other words, both forms of evaluation have an applied focus.
In contrast, knowledge-oriented evaluations are conducted to improve our understanding, to help build theory and sometimes to inform policy-making processes. For example, you might evaluate a range of distance education programmes to assess which approach led to the best learning outcomes, the results of which could inform the planning of future programmes. This purpose for evaluation is much less3 common than the two described above one may likely make use of formative and summative evaluations, or a combination of these, in their own evaluation work. The Role of Evaluation in VTE
In view of the definitions, main role and steps in evaluation, the following are the roles of evaluation in vocational technical education: 1. Programme Improvement This is the most important function or purpose of evaluation. Evaluation makes it possible for data and information relating to vocational technical education students and their performance to be collected. Such collected data are used in judging the effectiveness of the programme and in detecting deficiencies in the programme that needs to be removed. Government spend huge sum of money on education.
This sum of money should produce good educational programmes for students; programmes that would serve the need of the country like VTE and bring about whatever change are expected in behaviour, character, skill level and social life of the students who pass through the educational institution. An evaluation takes care of that. 2. Evaluations provide important and action, oriented lessons about implementation and improved Delivery. Evaluations have produced powerful evidence on the feasibility of innovative programs, how they can be operated more effectively, and how newly, designed programs can be refined to improve their performance.
Projects evolve over time. What was once a coherent, discrete set of activities may have grown into a jumbled set of loosely related events. An evaluation can help clarify the purposes of the programme, allowing decision-makers to examine programme components against well-thought-out criteria. Valid comparisons of programmes and activities can be made and duplication of efforts can be limited. It is quite possible that the evaluation will uncover a gem hidden in the jumble. 3. Evaluations provide powerful evidence about impacts of vocational education.
The significance of impact evaluations is that they provide credible, actionable evidence on whether or not a program or service achieves desired effects and should continue to be used. Impact evaluations are thought by many people to be “what evaluation is. ” To the contrary, impact evaluations are simply one approach from the rather lengthy menu of evaluation. It is in examining a program that one is likely to provide an accurate assessment of the impact and outcome potential of a particular substantive strategy. If an impact evaluation is conducted prematurely, it can lead to the misleading conclusion that the approach studied is a failure, when the truth is that it was simply not well implemented. The challenge administrator is to understand when an impact evaluation is called for – and when another evaluation approach, such as an implementation evaluation, is more appropriate. Rushing to do an impact evaluation in the hope of discovering a “magic bullet” is frequently a recipe for disappointment, and a missed opportunity to understand the challenges of effective program implementation. . Evaluations provide practical information for Decision Making. Decision must be made with regard to all aspects of education and at all stage in the provision of education. It is necessary for the effective operation of VTE, that decision be made on the basis of authentic data collected as a result of evaluation processes. Evaluation assists in decision making by providing the information on the basis of which wise decision can be taken. 5. Evaluations provide crucial information on Accountability.
Accountability is not merely the process of keeping good accounts of all expenditures so that mismanagement of funds and wasteful spending can be avoided. Accountability is the process of ensuring that all educational expenditures are justified by the improved learning or other favourable outcomes that might result from the expenditure. New equipment, new building may be required by the institution for the programme. Accountability through evaluation, seek to determine the cost of the new facilities and relate such cost to benefits. 6. Evaluations clarifying programme theory
When the programme was designed initially, it was based either explicitly or implicitly on a programme theory that explained how things work or how people learn or even how organizations change. An evaluation asks those involved to revisit that programme theory. Based on experiences with the programme and information taken from research literature, the evaluation provides an opportunity to revise the programme theory. By making the programme theory explicit, the underpinnings of the programme and what makes it work will be better understood and thus, better implemented.
Staff members and students will understand why a particular set of teaching methods was selected or why the programme activities were sequenced the way they were will be more likely to actually follow the plan. They will also feel more involved in the programme if they understand the theory behind the programme more fully. 7. Personnel Improvement. Evaluation aids in ensuring that VTE personnel are well trained and are carrying out the functions that they are best suited to carry out. The skill and ability of administrative and instructional personnel determine to a large extent the quality of the programme.
Staffs therefore are assisted through evaluation to identify their strong and weak points and be encouraged to improve on their performances 8. Taking Stock Engaging in evaluation provides a precious opportunity to reflect on the programme. It is an opportunity to document where the programme has been and where it is going, and consider whether the programme is doing what its designers hoped it would do. Taking stock is more than accumulating information about the programme, it is learning through the programme.
Conclusion Looking ahead, it is clear that evaluation can and should be used in ways that reach far beyond monitoring and impact evaluations, valuable as these may be in some situations. The only barriers to using evaluations effectively are the limits on our imagination. The examples also make plain the fundamental value of evaluation for 21st century philanthropy: Evaluations create useful lessons that provide significant social benefits and advance the strategic purposes and goals of foundations.
That is the reason administrators should use evaluation as a core tool enabling them to achieve their missions. References Evaluation Definition: What s Evaluation. (2007, April). Retrieved August 9, 2010, from http://www. evaluationwiki. org/index. php/Evaluation_ Definition Okoro, M. O. (2000). Programme Evaluation in Education. Onitsha: Pacific Publishers. Priest, S. (2001). A Program Evaluation Primer. Journal of Experiential Education, 24(1), 34-40. [pic][pic][pic][pic][pic][pic][pic][pic]