Traditional approaches to assessing the effectiveness of personnel training activities are that at the end of the training (seminars, trainings, courses, schools, etc.), students, as a rule, give their assessment in the form of interviews or filling out questionnaires, answering questions and by choosing one of the proposed assessment options (points):
compliance of the training content with the expectations (needs) of students;
application of active learning methods;
application modern means training;
connection of educational activities with the workplace;
quality of handouts (workbooks, etc.);
optimal number of students in a group;
organizational conditions for conducting classes;
qualifications of teaching staff, etc.
Students give organizers and teachers a score based on their level of satisfaction with the training.
In addition, to traditional methods assessment of the effectiveness of personnel training can include observation, statistical analysis, self-report, testing, etc. Of particular interest to us is the analysis unconventional methods assessing the effectiveness of personnel training, such as:
Donald Kirkpatrick technique;
Jack Phillips technique;
biparametric assessment technique;
assessment of the effectiveness of training within complex systems assessments such as BSC, KPI;
Bloom's assessment model.
Kirkpatrick's model, described in the book “Four Steps to Successful Training,” involves assessment at four levels. These levels determine the sequence in which learning assessments are conducted. He writes: "Each level is important and affects the next level. As you move from level to level, the assessment process becomes more difficult and time-consuming, but it also provides more valuable information. No level can be skipped simply because to focus on what the coach thinks is most important." Here are the four levels according to the author:
Level 1 – Reaction
Evaluation at this level determines how program participants respond to the program. Kirkpatrick himself calls this a customer satisfaction score. He emphasizes that the reaction of the participants is a very important criterion for the success of the training, for at least two reasons.
Firstly, people one way or another share their impressions of the training with their management, and this information goes higher. Consequently, it influences decisions about continuing training.
Secondly, if participants do not respond positively, they will not be motivated to learn. According to Kirkpatrick, a positive reaction does not guarantee the successful development of new knowledge, skills and abilities. A negative reaction to training almost certainly means a decrease in the likelihood of learning.
Level 2 – Learning
Learning is defined as the change in attitudes, improvement in knowledge and improvement in skills of participants as a result of their completion of a training program. Kirkpatrick argues that changes in participants' behavior as a result of training are only possible when learning occurs (attitudes change, knowledge improves, or skills improve).
Level 3 – Behavior
At this level, an assessment is made of the extent to which the participants' behavior has changed as a result of training. Kirkpartick points out that the lack of change in participants' behavior does not mean the training was ineffective. Situations are possible when the reaction to the training was positive, learning occurred, but the behavior of the participants did not change in the future, since the necessary conditions for this were not met. Therefore, the lack of change in the behavior of participants after the training cannot be a reason for making a decision to terminate the program.
Level 4 – Results
Outcomes include changes that occurred as participants completed the training. As examples of results, Kirkpatrick cites increased productivity, improved quality, decreased accidents, increased sales, and decreased employee turnover. He insists that results should not be measured in money.
According to Kirkpatrick, assessment at this level is the most difficult and expensive. Here are some practical tips that can help you evaluate your results:
if possible, use a control group (those who did not receive training);
carry out the assessment over time so that the results become noticeable;
conduct pre- and post-program assessments (if possible);
conduct the assessment several times during the program;
compare the value of the information that can be obtained through the assessment and the cost of obtaining this information (the author believes that conducting an assessment at level 4 is not always advisable due to its high cost).
The Jack Phillips technique is the use of various formulas to measure the return on capital invested in personnel (ROI):
Estimation of investments in the HR department = HR expenses / operating expenses.
Estimation of investments in HR departments = HR costs/number of employees.
Absenteeism rate = absenteeism, unannounced absences + number of employees who quit unexpectedly.
Satisfaction indicator is the number of employees satisfied with their work, expressed as a percentage. Determined by survey method.
A criterion that reveals unity and agreement in the company. Calculated based on statistical data on productivity and assessment of labor efficiency.
McGee offers a biparametric assessment, that is, he considers the effectiveness and efficiency of training, he also introduces the concepts of effectiveness and efficiency and how their optimal combination introduces the concept of training productivity.
The methodology for measuring effectiveness should take into account the specifics of training activities, the range of which is quite wide.
If the task is to evaluate the success of events through reducing the cost of a business process (although this is only a special case - reducing costs), then the following set of formulas for assessing the effectiveness of training will be optimal.
The private economic effect (E) from a personnel training event that caused a change in the cost of a given business process can be determined as follows:
where is the cost of the business process (unit of production) before training, den. units;
– cost of the business process after training, den. units
Determining the absolute value of the effect only allows one to find out the trends, scale and direction of the effect of training (positive, neutral, negative), therefore it is advisable to compare the value of the effect with the costs aimed at training personnel.
Interpretation of the result: if E ≥ 0, therefore, success was achieved, according to at least, the goal of reducing costs has been achieved, however, it is a question of price. Therefore, it is necessary to determine net effect. The net private economic effect of personnel training activities is determined as follows:
where is the cost of the business process (unit of production) before training, den. units;
– cost of the business process after training, den. units;
– cost of the training program (company costs for maintaining the training center), den. units
Determining the absolute value of a particular effect allows you to compare the result of an activity with the costs of training (whether the benefits exceed the costs or not).
Interpretation of the result: if ≥ 0 - therefore a positive result was achieved, at least the cost reduction exceeded the costs - the activity brings a net effect.
This sequence of calculations is appropriate for assessing the effectiveness of training specifically for line personnel, for example, as a result of training activities, the speed of customer service has increased, the quality of customer service (quality can be assessed by reducing the number of errors, customer complaints) has improved, etc.
A more traditional indicator for perception and easier to interpret is the profitability indicator (let’s consider it as a typical indicator for a business - the ratio of net income to costs):
Return (return on investment in training) is expressed as a percentage.
Interpretation of the result: if > 0, therefore the type of activity is profitable, otherwise, training costs are not returned as an additional effect.
This indicator is convenient to use when comparing several options or implemented activities.
Assessing the effectiveness of training within the framework of comprehensive assessment systems such as BSC, KPI. Often assessing individual areas of company development, modern management uses complex assessment systems that include a number of private indicators that reflect the effectiveness of departments, within the framework of which models successfully apply comprehensive assessment methods, for example, determining the contribution of certain departments to the overall performance result. Among the above-mentioned models, we can note the applicability of each of them for solving the problems of assessing the effectiveness of personnel training, highlighting the subdepartment responsible for training, a certain set of target indicators that allow us to evaluate this particular division of the company.
For example, by highlighting some quantitative and qualitative indicators for a given department, assuming that personnel training activities will affect these indicators, we will be able to assess the effectiveness of these activities as a whole and for each individual episode related to training. By clearly defining the correlation of these indicators with the performance indicators of the units that have undergone training, we will be able to analyze the effectiveness of personnel training efforts in the enterprise as a whole.
These methods are difficult to use due to the breadth of the spectrum of action, however, with their skillful use, not only their intuitive clarity, adequacy and consistency are achieved, which, in general, allows us to solve the problem outlined above. The development of a system of indicators will allow not only to comprehensively evaluate, but also to monitor in real time the level of effectiveness of these activities.
Nowadays, any leader with more or less foresight devotes increased attention staff training. In order to maintain the occupied market share, and ultimately the entire business, the employer must constantly monitor the improvement of the educational level of its employees.
From this article you will learn:
- why is it necessary to evaluate the effectiveness of staff training;
- how to organize the process of assessing staff training;
- what models exist to evaluate the effectiveness of staff training;
- what four ways to use to organize an assessment of the effectiveness of staff training.
If assessing the effectiveness of staff training requires large expenses from the employer, then how to justify the expenses? As experts note, the relevance of the problem of how to evaluate the effectiveness of staff training is only growing over time. Modern universities cannot cope with business demands: there is a clear lag educational programs higher and middle educational institutions from the constantly changing requirements of companies and rapidly increasing market expectations. It so happens that an employee, a recent university graduate, comes to production with already outdated knowledge, often unsuitable for efficient growth and company development. The employer is forced to invest in staff training, especially for young professionals.
Personnel training assessment: how to organize the process
Well, the company is ready to spend money on employee development. After all, commercial activity itself involves investing money, albeit for a specific purpose - for profit. Otherwise, it is no longer a business, but a charity. This is where the difficulties of assessing the effect of preparation are revealed. To ensure that the procedure is not carried out “for the sake of evaluation,” it is important for HR managers to understand what they will do with its results and whether the efforts are worth the investment.
You can spend millions on useless personnel development, which will not bring any economic efficiency. So wouldn't it be better to give this money to an orphanage? In this case, your charity will bring at least someone tangible benefit.
What models exist to evaluate the effectiveness of staff training?
Most companies have different approaches to the question of how to evaluate the effectiveness of employee training. But almost all employers use the four-level model of American researcher Donald Kirkpatrick, described by him in 1959 in the book “Four Steps to Successful Training.” The instrument is now widespread and considered traditional.
Monitoring by HR specialists of methods for analyzing the effect of employee training used at different enterprises showed: the more levels of this model are used, the more complex the assessment procedure becomes. The complexity of assessment activities carried out using the methodology using the fourth level of the Kirkpatrick model even exceeds the costs of conducting the entire training cycle. According to the author of the technology himself, conducting training assessment at the fourth level is not always advisable due to its high cost.
Another American, J. Phillips, added a fifth level of assessment to the Kirkpatrick model in 1991 - ROI (return on investment in employee development). Today, his model has received recognition (ASTD) from the American Association for Training and Development and is successfully used around the world.
ROI is calculated using the following formula:
Together with the calculation of ROI, quite often they are involved in calculating the payback period indicator, which reflects the period of time for the return on investment invested in training. The payback period indicator is the inverse of the ROI indicator.
There is another model, it is practically unknown in our country, this is the so-called “Bloom's Taxonomy”. The model consists of three overlapping parts, spheres, which are often called KUN (knowledge, attitudes, skills):
- Cognitive sphere (knowledge);
- Emotional sphere(installations);
- Psychomotor sphere (skills).
This model is in a practical sense similar to the Kirkpatrick model, only it cannot be used to make financial assessment effectiveness of staff training.
Assessing the effectiveness of personnel training: the place of the process among the overall function of personnel development management
Assessing the effectiveness of personnel training is the final stage of personnel development management in modern organizations and enterprises. Funds spent on professional education, are considered as investments in the development of the enterprise’s personnel. These investments should bring a return in the form of increased efficiency of companies and enterprises.
Methods for assessing employee training can be distinguished by quantitative and qualitative characteristics. With the quantitative method, learning outcomes are assessed using indicators such as:
- total number of trained employees;
- number of employees undergoing training, by category;
- selected methods of advanced training;
- the amount of costs for personnel development.
Quantitative accounting of the effect of employee training is needed to form the social balance of the enterprise. However, quantitative methods will not help you analyze the results of personnel training in terms of the level of professional training and its compliance with the goals of the enterprise.
Only through qualitative methods for assessing training results will you determine the effectiveness of personnel training and its impact on technical specifications production.
Four ways in which the effectiveness of staff training can be assessed
Experts identify four main methods qualitative assessment results of professional training. In the first method, the abilities and knowledge of employees are assessed directly during or upon completion of the training course. When using the second method, professional knowledge and skills are assessed specifically in production conditions. The third method is to evaluate the impact of training on production parameters. The fourth way is the way economic assessment.
Using the first method, you can determine the degree of mastery of professional knowledge and skills. The assessment procedure usually involves the participation of only teachers and students; a classical form exam, “test situations” and so on are used.
The trainee's immediate supervisor conducts an assessment professional knowledge and skills in the production situation. The result of the acquired knowledge is assessed after a certain time interval (six or twelve months) after completion of the course. During this time period, the significance of the knowledge that the employee acquired during the training process will become apparent, and the state of “euphoria” that arose immediately upon completion of the program will pass. The use of this method will help determine the degree of practical use of the acquired knowledge.
Establishing the degree of influence of employee training on production parameters can be considered as the main assessment level that connects training results with the requirements necessary for successful work and production development. The indicators of such analysis are expressed in physical quantities:
- number of staff,
- defect rate,
- staff turnover rate and the like.
Currently unavailable complex methods analysis, the use of which would allow us to more accurately determine the degree of influence of training on each individual factor.
The economic assessment of the effectiveness of employee training is based on the feasibility of investing in personnel and human resources. The criterion for the feasibility of investing in personnel is the amount of additional net income received after the training programs are implemented. In this case:
- If the increment is greater than zero (D< C, где D – инвестиции в персонал, C – возможный долгосрочный эффект от инвестиций, наблюдаемый в производственно-коммерческом процессе), то вложение Money This staff training program pays off and is appropriate. Moreover, the lower the market rate of return on capital, the higher the expected increase in net income in the Nth year, the longer the period of time for using the acquired knowledge, the more effective the investment in personnel;
- If D > C, then investment in this program is inappropriate and there is a need to search for other areas of capital investment.
The expediency of the personnel training program is directly proportional to the period of possible use of the acquired knowledge.
The creation of some training programs does not aim at developing specific professional skills, but is aimed at developing a certain type of thinking and behavior. Measuring the effectiveness of such a program is quite difficult to do directly. After all, the results of the program are designed for a long-term period and are associated with the behavior and consciousness of people who cannot be accurately assessed. In such cases, indirect methods are used:
- conducting tests before and after training, which show the degree of increase in students’ knowledge;
- monitoring the behavior of trained employees in the workplace;
- monitoring the reactions of students during the program;
- assessment of the effectiveness of the program by students themselves using questionnaires or during open discussion.
The established criteria for assessing the effectiveness of programs should be brought to the attention of all participants (both students, trainers and process managers) even before the start of training. After the completion of the program and summing up the results, the results are reported to the personnel management service, the management of the trained employees and the employees themselves, and then are used in planning further training.
Why is it necessary to evaluate the effectiveness of staff training?
The effectiveness of staff training can be assessed using various models. After all, each of the above models has its own advantages and disadvantages. Which of them will be chosen by a specialist assessing the effectiveness of staff training will completely depend on the goals set by the company’s management.
Application of the Kirkpatrick model allows you to quickly get a clear picture of the effectiveness of training activities. Using the Bloom's Taxonomy model makes it possible to more thoroughly assess the effectiveness of training, and also allows you to choose a specific personnel training strategy. Using J. Phillips' model, you can evaluate the financial aspects of training, for example, the effectiveness of investments in personnel. Therefore, today, according to experts, the problem is not the need to evaluate the effectiveness of training or the lack thereof, but the determination of the choice of its specific algorithm.
Any training and development program is designed to promote increased efficiency or productivity of employees, the formation of such working behavior that will ensure the achievement of the goals and objectives facing the organization. Consequently, a training and development program can be considered effective if the results obtained upon completion of the training exceed the costs of its implementation.
Analyzing the impact of personnel training and development requires a lot of time and high competence of the specialists conducting this assessment. However, many organizations do not conduct such assessments, believing that any training is worthwhile. This approach has a right to exist, but it must be taken into account that training and development of personnel is an expensive undertaking to rely on chance. The training of workers must have specific results that can be assessed quantitatively or qualitatively.
According to the head of the HR department at Raiffeisenbank, for “professional training there are very specific indicators. For example, after training Excel program how much the time for preparing reports has been reduced. This figure can be easily translated into man-hours and specific amounts. Or, for example, after sales training, how much the number of meetings and signed contracts increased. When it comes to leadership qualities and personal effectiveness, it is extremely difficult to identify the ROI indicator. But probably. For us, one of the measures of a manager’s effectiveness is the indicator of employee engagement in his department and its dynamics from year to year. Sometimes the customer’s opinion is enough when his expectations are exceeded.”
To determine the effectiveness of employee training and development, monitoring of all stages of personnel training and development should be organized.
Control is the process of comparing actual results with assigned objectives. Well-established control allows timely improvement and adjustment of all stages of personnel training and development.
Types of control:
- preliminary;
- current;
- final.
In terms of the form of implementation, all types of control are similar, since they have the same goal: to promote maximum similarity between the actually obtained and planned results. A distinctive point is the timing of control activities. Characteristics of the types of control over the implementation of employee training and development programs are presented in Table. 8.1 173 .
Characteristics of types of control
Table 8.1
|
Type of control |
Characteristics of the type of control |
|
1. Preliminary |
|
|
2. Current |
|
|
3. Final |
|
Personnel training and development programs must have specific results that can be assessed quantitatively and qualitatively.
Analysis of the impact of training and development can be carried out using various tests, exams, practical tasks, etc. The assessment can be carried out by the employees themselves, specialists, company experts, as well as by their teachers or specially created commissions.
When determining the effectiveness of employee training and development, it is important to control all requirements for the organization educational process, which includes the following elements:
- program development;
- selection of teachers and trainers;
- training methods;
- composition of training workers;
- equipment for the educational process;
- input knowledge control;
- current control of knowledge;
- output control of knowledge.
Then they begin to evaluate the results of the training event (program) according to the following criteria:
- 1. Reaction of an employee who has undergone training. The opinion of the employee himself is taken into account. It is important to find out how he evaluates the training, whether he liked the proposed program, and whether the workload was excessive. As a rule, a variety of questionnaires and questionnaires are used for this.
- 2. Mastering the material. It is determined by the amount of information acquired, usually through tests, interviews or interviews.
- 3. Change in employee behavior. It is determined how the employee’s behavior has changed, to what extent he uses the knowledge and skills acquired during training, and whether there is an emotional upsurge in the team.
- 4. Operating results. Assessment of the real benefits that the organization received after training.
For example, in a travel agency, before training for sales managers, the number of tours sold per month was 100, and after training - 140. In this case, we can talk about the success of the training.
In an organization where professional learning and development is a value, the likelihood of effective training is much higher than where this aspect is not given due attention.
Of all the factors of high effectiveness of training and development, the main one is the factor that characterizes the actions of the organization’s management before, during and after training.
Senior management support for staff training and development is characterized by:
- inclusion of training and development in the long-term and operational plans of the HR service;
- preliminary assessment and clarification of the organization’s needs for training and development of employees;
- allocation of financial resources in the required amount for training and development;
- timely informing employees and their managers about the possibility and necessity of training and development;
- familiarization with training results and analysis of their effectiveness.
Some companies use the following indicators to assess personnel training and development parameters.
1. Number of company employees trained per year.
This indicator depends not only on the development of the training system, but also on the size of the company. A large company annually trains more workers than average. When setting a plan for this indicator, it should be remembered that the company has groups of employees who may undergo various training throughout the year. Such employees include managers, specialists, accountants and lawyers sent to short-term seminars and trainings on the problems of changing legislation, regulatory documents and so on. This indicator assesses the dynamics of the volume of training in a company with a relatively stable number of its personnel. If the number of the company changes significantly over the years, then to analyze the dynamics of training development, as well as to compare the company’s training indicators with other similar companies, it is necessary to use such an indicator as the ratio of the number of company employees trained per year to the total number of employees of the organization.
2. Volumetric indicator of training.
This indicator characterizes the amount of training per year per company employee. It can be measured in man-hours or man-days. Since individual learning programs or courses can last less than one day; hours can be converted into days at the rate of 1 day - 8 hours.
The practice of successful companies is characterized by the amount of training per employee ranging from 3 to 10 man-days per employee per year. IN last years There is a global trend towards a reduction in the volume of classroom training due to the expansion of computer-assisted training and distance learning.
3. Economic indicator of training.
This indicator can be assessed taking into account two main characteristics of the event (program) itself - duration and number of participants. It is used for the economic evaluation of group types of training, so in practice it should be considered as the cost of 1 day (hour) of training per student. This indicator can be called a general measure of the economic efficiency of training. The lower this indicator, the more employees can be trained at lower costs.
Expenditures on training and development are considered as investments in human resources; they should bring returns to the organization in the form of increased efficiency of its activities, profit growth, and more complete implementation of its goals and objectives. The costs of training and development of personnel can be calculated taking into account the potential benefits (Fig. 8.1).
Rice. 8.1
Example. The company carried out training center employee training. Price of one course: 26,000 rubles. (tutor’s payment, costs for preparing the premises, rent, etc.). The salary of an employee in the company is about 200 rubles/hour. Duration of training - 16 hours. The cost of employee training can be calculated using the formula
O = (DxG) + C k,
where 5 0 is the cost of training; /? - employee salary; / - course volume (in hours); P - course price. Then
200 X 16 + 26,000 = 29,200 rub.
During training hours, employees do not produce material assets Therefore, the company suffers double losses: it receives less profit due to distraction from work and spends money on training.
Training and development costs can be assessed relative to profit and relative to the fund wages, as well as per employee. Economic efficiency training is related in the same way as the price and quality of any product or service, namely: the lower the price, the lower the quality, as a rule.
For example, at PJSC Rostelecom, assessing the effectiveness of training is one of important stages personnel training.
Performance assessment is a tool for monitoring:
- the effectiveness of the methods, forms and methods of training used;
- correct identification of training needs;
- timeliness of training;
- the overall effectiveness of personnel training and development activities;
- the effectiveness of financial investments in personnel training and development;
- measuring influence human factor on the company's business results;
- increasing the transparency and controllability of the training organization.
Using training evaluation criteria, you can determine the effectiveness of in various ways estimates (Table 8.2).
After analyzing the training results using any of the listed methods, the HR officer at PJSC Rostelecom identifies the best one and, if necessary, prepares proposals for improving training.
Without assessing the effectiveness of the learning process, it is impossible to build a training and development system that provides the necessary business results. D. Kirkpatrick proposed a formula to describe the learning cycle: reaction - learning - behavior - results.
Efficiency of assessment methods
|
Criterion to be assessed |
Assessment methods |
|
|
1. Level of satisfaction of the training participant |
Training participant satisfaction level:
|
|
|
2. Degree of mastery of educational material |
Changes in knowledge, skills, attitudes and motivation of the training participant |
|
|
3. Dynamics of changes in the working behavior of a training participant, including his competencies |
Changes in the working behavior and competencies of the training participant. Complex application putting into practice the knowledge and skills acquired during the training |
|
|
4. Dynamics of changes in the training participant’s performance results |
Changes in the results of achieving the individual goals of the training participant, business indicators of the group and the structural unit as a whole after the training. |
|
|
5. Dynamics of changes in the company’s performance results and the financial effect of training |
Efficiency of investment in training |
Efficiency: program income - program costs |
He also clearly formulated the criteria for all four levels of assessment (Table 8.3).
Four levels of learning assessment by D. Kirkpatrick
Table 8.3
|
What is being assessed |
Key questions |
|
|
Level 1: "Reaction" |
How participants react to a learning event |
Did the participants enjoy the learning process? What do they plan to do with their new knowledge and skills? |
|
Level 2: "Training" |
To what extent participants have acquired knowledge, skills and formed the necessary relationships upon completion of the training event |
What skills, knowledge, attitudes have changed after training? How significant are these changes? |
|
Level 3: Behavior |
How participants apply what they learned during training to the workplace |
Did participants change their behavior in the workplace after the training? |
|
Level 4: "Results" |
To what extent have the intended results been achieved as a result of the training? |
Do participants' behavior change? positive impact on the organization? |
This model is still relevant for assessing the effectiveness of training today; in addition, experts have added a fifth level of assessment.
Level 5: Return on Investment. Evaluating effectiveness at this level allows you to get an answer to an important question for business: “What did training do for business development?”
From the point of view of experts, this model has weak and strengths. Ease of understanding for all learners, not just learning professionals strong point models by D. Kirkpatrick. Limitations of the model include its simplistic approach; assessment levels do not go beyond training, etc.
As we continue to evolve our approach to assessing learning and development, learning professionals need to change the way they work, learn how to create real business value and convincingly demonstrate the importance of those values. As a starting point, the trainer (trainer, teacher) needs to discuss with managers:
- 1) expectations of business leaders;
- 2) their understanding of success;
- 3) what observable and measurable indicators of success will be used to evaluate the outcome (level 4).
The result of the work at this stage should be the determination of the expected results. L&D professionals should then discuss with line managers which behaviors are critical to achieving the intended results (Level 3). Only then can they begin to design learning activities (levels 1 and 2).
On this basis it is possible to provide:
- effective application new skills and consolidation of changes in implementation in the workplace (level 3);
- achieving business goals (level 4);
- implementation of tasks in the field of talent management (level 4). To ensure the effectiveness of the acquired knowledge, skills and experience
training and development programs enhance business results, the employee in his workplace should receive support and recognition from managers in their implementation. Without this support, only 15% of new knowledge, skills and attitudes are successfully put into practice, reducing the value of learning to business.
The assessment methods themselves play a great role in assessing the effectiveness of training and development.
- Khripunova M. Personnel training 2016: how to teach in order to make a profit, extraordinary methods //To the HR Director. 2016. No. 2.
- Maslova V.M. Personnel management: textbook. - M.: Yurayt, 2015.
- Kirkpatrick D.L., Kirkpatrick D.D. Four steps to successful training. M.: HAR Media, 2008.
The most important aspect of building effective training in an organization is building a high-quality system for assessing the effectiveness of training. The absence of such a system can lead to:
to a significant decrease in the quality of education;
absence effective means learning management;
a general decrease in learning efficiency.
Existing models for assessing training effectiveness
The most widely used model for assessing the effectiveness of training in an organization today is the training assessment model, based on the works of Kirkpatrick and Warr, Bird and Rackham. This model for assessing the effectiveness of training has proven itself in practice.
Kirkpatrick model
Kirkpatrick views training effectiveness assessment as essential element training cycle, which includes the following stages:
identifying needs;
setting goals;
definition of subject content;
selection of training participants;
formation of an optimal schedule;
selection of appropriate premises;
selection of appropriate teachers;
preparation of audiovisual media;
program coordination;
program evaluation
Kirkpatrick cites three main reasons for conducting an assessment:
the need to justify the existence of a training department by showing how the department contributes to achieving the goals and objectives of the organization;
establishing a mechanism for deciding how to improve the training program in the future;
establishing a mechanism for obtaining information on how to improve the training program in the future.
Levels of training effectiveness assessment
The training effectiveness assessment model assumes four levels of training effectiveness assessment:
level 1: trainee reaction to the training program;
level 2: assessment of the knowledge and experience gained by the student in the training program;
level 3: workplace behavior assessment;
level 4: assessing the impact of the training program on the organization's activities.
Level 1. Students' reaction to the training program.
The first level of the training effectiveness assessment model determines how the training was perceived by students. Kirkpatrick himself calls this a customer satisfaction score. The most effective means of assessing the effectiveness of training at this level is to conduct a survey.
The questionnaires used contain a wide range of questions on all aspects of training, including:
how important the goals are;
how students evaluate the quality of the training provided;
How effective are the teaching tools used?
what additional topics they would like to include in the training program;
how comfortable it was for the listeners to work;
how the food was organized;
how the accommodation was organized;
As a result of the survey, you can evaluate:
how listeners satisfied with the quality of training and how well it meets their expectations;
how good was the training organized ;
which additional expectations they have.
The results obtained as a result of the assessment indicate the degree of satisfaction of trainees from the training provided.
Level 2. Assessment of knowledge and experience gained by the student in the training program.
The Level 2 goal is to evaluate as efficiently as possible whether its objectives were achieved during the training. The most common tool used at this level is tests to test knowledge and skills. In addition to tests, more complex testing tools are used, such as simulators, simulations, complex exercises, etc.
It is advisable to conduct testing of training participants before and after training. Conducting double testing will allow you to assess how the competence of students has changed as a result of training. By analyzing all the assessments received by students, you can evaluate how effective the training was.
At the second level, an assessment is made of the extent to which the training allowed solving the set pedagogical tasks. However, at the second level there is no assessment of how effective training is from the point of view of the functioning of the organization.
Level 3: Assessment of workplace behavior.
The third level of Kirkpatrick's performance evaluation model is responsible for collecting information to evaluate how the training provided influences the behavior of trainees in the workplace. From an organizational perspective, this is key in assessing the effectiveness of training. It is at this level that an assessment is made of what practical benefit the organization benefits from training. Were the students able to put into practice the skills, knowledge and abilities they acquired during the training?
It is advisable to evaluate the effectiveness of training at this level several times within 3–6 months after completion of training. During this time, students will be able to demonstrate how they apply the knowledge they acquired during training. Often, research at this level is carried out using special “behavior cards”, which are filled out by all participants in the training, as well as by organization specialists who can assess how the training has affected the effectiveness of their work.
Level 4. assessment of the impact of the training program on the organization's activities.
Even if an assessment of the effectiveness of training at the third level showed that the skills, knowledge and abilities obtained as a result of training are successfully used by students in the performance of their official duties, this does not mean that the training provided real benefits to the organization.
The purpose of the Level 4 assessment is to answer the question: How useful is the training provided in the organization?
First of all, it is necessary to answer how the training provided affects the key performance indicators of the company.
Assessing the effectiveness of training at level 4 is extremely rare due to the difficulty of obtaining indicators of the organization’s performance and assessing the impact of training results on it.
Here are some possible indicators efficiency of the 4th level:
Sales technology training:
increase in sales;
increasing the customer base.
technical skills training:
reducing the number of calls to support;
reducing work completion time.
Very often the assessment carried out at level 4 is divided into two sub-levels. The first sublevel assesses how training affects the company's performance. At the second sublevel, the economic efficiency of the training provided is assessed.
Distance learning and training effectiveness assessment
Distance learning (e-learning) is part of the general training conducted in the organization. And is assessed accordingly along with other forms of training used in the company, using the same assessment methods. However, it should be noted that the use of technical means used in distance learning can significantly simplify the collection of information for assessment, as well as its subsequent analysis.
Preface
1 DEVELOPED by the Educational Institution “Belarusian State University computer science and radio electronics"
PERFORMERS:
Alyabyeva I.I., electronics engineer, quality management department
Sokolovskaya E.N., business analyst, quality management department
INTRODUCED Working group on the creation and implementation of an education quality management system
2 APPROVED AND ENTERED INTO EFFECT by order of the rector
dated 02.11.2012 No. 256
3 INTRODUCED TO REPLACE MI 3.6-02-2010 (version 01)
© BSUIR
This methodological instruction cannot be replicated or distributed without the permission of the educational institution “Belarusian State University of Informatics and Radio Electronics”
Published in Russian
Purpose and scope 4
Normative references 4
Terms, designations, abbreviations 4
General provisions 5
Assessment of learning outcomes 7
Appendix B Training Assessment Questionnaire (to be completed by the employee) 10
Appendix B Training Assessment Questionnaire (to be completed by the supervisor) 11
Appendix D Evaluation of employee training effectiveness 12
Appendix E Report on assessing the effectiveness of training of department employees for the year 13
Change registration sheet 14
Approval sheet 15
1 PURPOSE AND SCOPE OF APPLICATION
This methodological instruction is a document of the university’s quality management system. Designed to assess the degree of effectiveness of personnel training and is used in structural units in order to meet the requirements of clauses 6.2.2(b,c); 7.4.1, 7.4.3 STB ISO 9001-2009.
This methodological instruction applies to all structural units of the university that send employees to training, advanced training, retraining and other types of training and self-education and is mandatory for use by managers at all levels.
2 REGULATORY REFERENCES
STB ISO 9000-2006 Quality management systems. Fundamentals and vocabulary.
STB ISO 9001-2009 Quality management systems. Requirements.
STB ISO 9004-2001 Quality management systems. Recommendations for improving activities.
TC RB 4.2-MR-17-2003 Guidelines by organization constant increase effectiveness of the quality management system.
3 TERMS, SYMBOLS, ABBREVIATIONS
This methodological instruction uses terms and definitions according to STB ISO 9000–2006, as well as the following terms with corresponding definitions:
Efficiency– the degree of implementation of planned activities and achievement of planned results.
Competence– demonstrated ability to apply knowledge and skills.
Education– a purposeful process of organizing and stimulating educational activities students to master knowledge, skills and abilities, and develop their creative abilities.
4 GENERAL PROVISIONS
4.1 A comprehensive assessment of training results is formed upon completion of all stages of the employee’s training process and is predetermined by the clarity of the goals set at each stage of training, including:
1) determination of training needs (analysis of the organization’s needs related to competence, analysis of other employee training needs, analysis of the readiness, motivation, ability of employees to learn and self-education);
2) design:
clear statement of a specific learning goal;
development of a schedule and terms of training for department employees;
planning, if necessary, coordination of training programs;
choice of forms and methods of training - courses, seminars, consultations, instruction, correspondence, distance learning etc.;
analysis of training organizations (external or internal training providers - teachers, trainers, etc.);
selection of monitoring and control methods;
methods and forms for assessing training results/outcomes by the training provider;
training financing planning;
involving employees/employees as active participants in the learning process;
provide opportunities to improve the implementation of training needs
4) assessment of learning outcomes.
Figure 1 – Learning cycle
(GOST R ISO 10015-2007 Training guidelines)
4.2 The requirements for the training provider (internal or external) are analyzed:
level of competence of teaching staff;
experience in this service sector;
a training program that meets the goals and expected results;
educational technologies used;
cost of education;
criteria and methods for assessing the effectiveness of training in order to determine: achievement of set goals, expected results, assessment of the acquired competence of a trained employee, assessment of customer satisfaction (trained employee, head of the employee's joint venture);
assessment and certification forms.
4.3 The selection of a training provider is fixed/registered and ends with the execution of an agreement/contract for training reflecting all topics, terms, costs, conditions, powers and responsibilities in relation to the training process, the procedure for resolving disputes and other articles provided for by the legislation of the Republic of Belarus (for an external supplier training).
4.4 Completion of training is documented in the form of training records.
5 EVALUATION OF LEARNING RESULTS
5.1 The main purpose of assessing learning outcomes is to confirm the achievement of learning objectives as a whole, including assessing the effectiveness of learning.
The method used to assess the effectiveness of training is a questionnaire (combined, two-level).
If a detailed analysis is necessary, you can use the multi-level model of D. Kirkpatrick, J. Phillips (additionally taking into account the economic indicator), however, it is necessary to assess the feasibility and optimality of choosing these multi-level models with associated costs.
5.2 The assessment is carried out for short-term and long-term periods.
When assessed for short term feedback information is processed from the trainee (about the practical value of training, teaching methods, resources used, as well as knowledge and skills acquired as a result of training) (Appendix B) and the immediate supervisor of the trained employee (Appendix B) about the practical value of training and proposed aspects for improvements.
Performance criteria:
student satisfaction;
students' acquisition of knowledge, skills and training;
The assessment is carried out based on the following criteria:
student satisfaction;
acquisition of knowledge, skills and qualities by students;
satisfaction from management;
impact on the organization (based on the results of analysis of proposed aspects for improvement);
procedures for monitoring the learning process.
5.3 The assessment process includes data collection, analysis and assessment of training effectiveness, and preparation of an assessment report (Appendices D, E).
The training assessment report includes the following:
applications for training (in accordance with the training plan for employees of the structural unit;
evaluation criteria, description of sources and methods;
analysis of collected data and interpretation of results;
conclusions and recommendations for improvement.
Completed and processed training assessment questionnaires are stored in structural unit before the end of the reporting period.
When nonconformities are detected, corrective actions must be taken.
The report data is input to the monitoring process, the main purpose of which is to provide objective evidence of the effectiveness of the training process and the satisfaction of the organization's training requirements. Monitoring involves analysis of the entire learning process at each of the four stages (Figure 1).
Monitoring methods may include questionnaires, testing, counseling, observation and data collection.
The algorithm for assessing the effectiveness of training is given in Appendix A.
Appendix A
Algorithm for assessing training effectiveness
Appendix B
Training Assessment Questionnaire(to be filled in by employee)
Training participant (full name)______________________________________________________________
Structural unit, position __________________________________________
Topic of training ________________________________________________________________
Date of training _____________ Venue ___________________________________
Educational institution______________________________________________________________
1. Rate the effectiveness of training on a 5-point scale (5 - excellent, 4 - good, 3 - satisfactory, 2 - bad, 1 - very bad)
|
Training assessment criteria |
Grade |
Note |
|
1. Course content meets expectations | ||
|
2. Relevance for you of the acquired knowledge (timeliness) | ||
|
3. Novelty of the information received (modernity) | ||
|
4. Compliance of the course content with the training program | ||
|
5. Clarity of the material presented | ||
|
6. Pace of classes | ||
|
7. Satisfaction with the teaching technologies used | ||
|
8. Practical value of the material, applicability for work | ||
|
9. Satisfaction with materials received | ||
|
10. To what extent did training contribute to the improvement of personal qualities? | ||
|
Total points | ||
|
Employee training assessment, Osotr, %: (Acuity = total points x 100) / 50
|
2. Your proposals for improving the activities (of an employee, department, university) taking into account the knowledge gained during training:
3. Rate the quality of teaching on a 5-point scale (5 - excellent, 4 - good, 3 - satisfactory, 2 - bad, 1 - very bad)
|
Criteria for assessing the quality of teaching |
Grade |
Note |
|
1. Availability of presentation of material | ||
|
2. Teaching technologies used (practical tasks, work in groups, use of modern technical means) | ||
|
3. Assessment of the teacher’s level of competence | ||
|
4. Communication skills of the teacher, ability to establish contact with the audience | ||
|
Total points | ||
|
Assessment of teaching quality, Kpr, %: (Kpr = total number of points x 100) / 20
|
4. Your wishes for improving teaching _______________________________________________________________________________5. Your desire for further training: a) on the subject of the training provided Yes / No
b) on another topic Yes / No
__________________ ________________
Date Signature
Appendix B
Training Assessment Questionnaire(to be filled in by the manager)
Structural subdivision_____________________________________________________
Manager (full name)___________________________________________________________
Training participant (full name), position__________________________________________________________
Topic of training (seminar)_______________________________________________________________
When and by whom the training was conducted ______________________________________________
|
Criteria for evaluation |
Grade |
Notes |
|
1. Practical value of knowledge acquired during training for a department or university | ||
|
2. The value of proposals made by the employee to improve the work of the employee/division/university | ||
|
3. To what extent did the training contribute to the development of workers’ skills necessary for work? | ||
|
4. To what extent did training contribute to the development of personal qualities necessary for work in employees? | ||
|
Total points | ||
|
Overall rating, Oruk, % (total number of points x100) / 20
|
2. What other knowledge and skills is necessary for an employee to successfully complete his task? job responsibilities: your suggestions for improving the activities (department, division) taking into account the knowledge gained at the seminar:
__________________________________________________________________________________________________________________________________________________________
3. What training is still needed for this employee (on what topic):
__________________________________________________________________________________________________________________________________________________________
4. Your wishes for improving education at the university:_________________ ________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
__________________ ________________
Date Signature
Appendix D
Assessing the effectiveness of employee training
The sum of the weighting coefficients is 1.
The column “Criterion value” is filled in:
according to Section 1 of the Questionnaire (for an employee) of Appendix B
according to Section 1 of the Questionnaire (for the manager) of Appendix B
The effectiveness of the criterion is defined as the product Weighting factor x Criterion value (%)
Appendix D
Report on assessing the effectiveness of training of department employees for ______________ year
1. Basis for the training _____________________________________________________ training schedule for ___, order No.___ dated _____, other basis
2. Performance evaluation criteria:
1) Percentage of personnel trained:
P 1 = K 1 / K 2 100, %
K 1 – number of trained workers,
K 2 – total employees planned according to the training schedule
2) Evaluation of training effectiveness by employees:
Р 2 = ∑О employee/number of questionnaires, %
3) Evaluation of training effectiveness by department heads:
Р 3 = ∑О hands/number of questionnaires, %
Overall effectiveness of employee training:
Рtot = ∑Р i / number of questionnaires, %
3. Assessment of teaching quality:
K = ∑K pr/number of questionnaires, %
To pr – indicators for assessing the quality of teaching (from training questionnaires filled out by employees)
____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
__________________ __________________ ______________________________
Date Signature Full name
CHANGE REGISTRATION SHEET
|
№ changes |
Date of amendment, addition and audit |
Sheet numbers |
Cipher document
|
Brief content of the change, revision note |
FULL NAME, signature
|
|
1 |
2 |
3 |
4 |
5 |
6 |
Loading...