The evaluation of training effectiveness is an action that corresponds to the third level of the well-known Kirkpatrick scale, the most universally accepted training evaluation model. Indeed, in the referred scale, the first level is that of "reaction" or satisfaction of the participant; the second level is that of "learning"; the third level is that of "effectiveness", also known as application; and the fourth level is the "impact" level.
• Reaction level: impression made by the course on the participant, regarding the content, the probable applicability of the course, the instructional support material (manual, visuals and any other material used), as well as the preparation and performance of the facilitator, and complementary aspects (learning environment, snacks, etc.).
• Level of learning: assimilation by the participant of the course content (how much he learned in the learning experience).
• Level of effectiveness: real possibility of applying what has been learned in the job performance of the participant. Also called “transfer of learning.”
• Impact level: to what extent training influences the generation of changes in the work process, which will therefore affect business results.
There is an alternative model, generated from the development of Kirkpatrick, called “Kirkpatrick & Philips model”, which contains a fifth level, referred to as “return on investment” (ROI). The following is a summary outline of the Kirkpatrick model, complemented by the evaluation resources:
Obviously, various methodological options can be mentioned to undertake this task, from the easiest and simplest to the most complex.
In the present essay, with the intention of sharing a viable and easily applicable solution to a problem that represents a methodological stumbling block for most professionals in companies, we present a proposal that can be crystallized in an instrument to undertake the aforementioned activity: evaluate the effectiveness of the training provided.
Methodological premises:
The proposal we offer is based on the following methodological premises:
• The effectiveness of the training provided is directly associated with the application of the knowledge received and the skills / abilities developed, specifically or specifically in the job or work environment.
• Application equals performance; therefore, the measurement should be based on indicators of "input performance" and "output performance", specifically linked to the content of the course or training activity.
• The evaluation of the effectiveness of the training provided is specific; that is, it is required to have information, in terms of measurement items, that correspond to the particular content of each training.
First step: generate a base form
Prepare a form, with a matrix structure, for each training action, containing the specific learning objectives of the course or training activity, with a column to record the “before” data and another column for the “after” data. It must contain no less than four and no more than seven objectives.
Application method
Based on the objectives or the content of the training action, the “expected results” of the training must be considered. Being clear about the specific objectives of the course, we proceed to evaluate the previous and subsequent performance of each participant or group of participants. Next, each individual's performance is “graded” “BEFORE” the training. At least three months after training, the performance of the participant or group of participants is re-scored.
Quantification of the result
In order to quantify the result obtained, we have two methodological options:
to. Simple comparison between input (before) and output (after) data, which should normally be positive (with the output data prevailing); This first option is called “simple factorial scale”; and
b. Use of indices expressed quantitatively as an "integrated factor scale".
In both cases or options, a previous semantic scale can be used, determined by the letters A, B, C, where A means “adjusts to what is expected” (performance expectation or desired result), B means “requires improvement to adjust to expected ”and C means“ deficient or does not conform to expected ”. Then, we proceed arbitrarily and on the basis of expert opinion, to assign numerical values to "A", "B" and "C", which must be corresponding or of logical configuration. A recommended way could be to assign each "A" a value of 5, each "B" a value of 3 and each "C" a value of 1.5 (the value zero is not considered because it would be the "absolute deficiency")., which is an impossible theoretical value, for reasons that we will not express here). When carrying out the respective multiplication and the sum of the partial results,input and output results will be obtained.
For the calculation of the indices, both for option "a" and for "b", we can use an Excel Spreadsheet, proceeding as follows:
• The “input performance” index (before) is calculated by dividing the “input total” by the total of selected items (which should not be less than 4 nor more than 7 aspects); that is, between the total of qualifications made.
• The “exit performance” index (after) is calculated by dividing the “exit total” by the total of selected items (which should not be less than 4 nor more than 7 aspects); that is, between the total of qualifications made.
• The effectiveness index results from the difference (subtraction operation) between the two previous indices.
Interpretation of results
According to a pilot simulation or by means of test data runs, from the consideration of the arbitrarily chosen numerical values (5, 3 and 1.5) the " performance index ", whether it is input or output, is a number to move, in the following range or numerical environment:
OPTIMAL: 4.4 to 5.0
ACCEPTABLE: 4.0 to 4.35
DEFICIENT: less than 4
Likewise, the "effectiveness index" is a figure that moves, from the selected values, in the following range or numerical environment:
HIGH: from 1.90 to 2.5
MEDIUM: from 1.6 to 1.85
LOW: less than 1.6
