Measurement and evaluation of labor competencies

A Theoretical and Methodological Contribution for the Design of Measurement Instruments and the Evaluation of Competences.

This essay is contextualized within the psychological approach to competences, based on the contributions of Dr. David Mc Clelland (1973). In it, we refer to the issues of measurement and evaluation, and the criteria inherent in the design of instruments to measure and evaluate competencies. In short, it collects in a somewhat superficial way, certain experiences of professional consultants, without pretending to be a scientific contribution on the subject.

The competition: the concept

The word "competence" has been used as a key concept in various theoretical formulations, for more than forty years, first in the discipline of curricular planning (educational approach). The word has also been used in the functional approach of British origin, also known as standardization and certification of labor competence. In both approaches, it is conceived as the quality of being competent. But, the boom in skills in the business world comes after the famous findings of Dr. David Mc Clelland in 1973. In this approach, skills refer to personal attributes to achieve success in the performance of specific roles.

As a result of the above, there are several meanings associated with the word competence, with different methodological applications and implications. In conclusion, there is no concept of competence, understood as a category of analysis, with a universally accepted meaning. The reality is that there are several conceptions or meanings given to this word.

Personnel management by competencies

Since the early 1980s, the concept of "competition" made its appearance in the business environment, as a new guiding and modernizing methodological option for personnel management, driven by the findings of Dr. David Mc Clelland.

Since then, the application of the competencies approach has been progressively spreading in many companies, as a guide to personnel management. Already in the mid-1990s, in many countries and in a multitude of companies, many attempts had been made to implement this novel approach.

From boom… to frustration

The implementation of the new model has been only partial in many companies, and has not been successful in many others due to difficulties associated with the lack of effective or practical techniques and methods to assess skills.

As a result of this limitation, frustration in such attempts became successively, because since it was not possible to determine the level of mastery and the consequent gaps in people, with respect to the competencies required by their position, the application of the approach of competencies in the selection, training, career development and compensation of personnel.

In most of the companies where the model was implemented, the inventory of corporate competencies was generated, until the formulation of the profiles or competencies models linked to the positions of the organizational structure. But, the activation of the personnel management processes by competencies, in many companies remained in a vacuum, except for some actions related to the selection of personnel where the EIB interview and other exploratory resources of the competencies were used.

First methodological attempts

The method originally used to explore and assess competencies has been the Behavioral Events and Critical Incidents Interview (BEI). It has been difficult to apply, as it requires a lot of time, effort and trained personnel. Very few companies have adopted methodological schemes offered by consulting firms, and have used the descriptive profile method based on observable behaviors, complemented with a computerized system or application, among other options. With this method, competencies are evaluated with the participation of the supervisor and the employee herself (self-evaluation), and can even be complemented with the 180 and 360 degree modalities.

However, the development of the evaluation instruments to generate the descriptive profiles, requires previously parameterizing the competences, with the support of specialists or properly trained personnel.

Main difficulties and attempted solutions

A case of special mention are technical or functional competences, for whose evaluation there are few affordable methodological options. With regard to competences of an attitudinal / social nature, nowadays there are dictionaries and, due to their almost universal applicability, there are also many already parameterized profiles relating to many competencies, which can be adopted directly or with certain adjustments. But functional competencies, much greater in number than attitudinal / social and / or generic competencies, are generally company-specific and, consequently, must be parameterized one by one, in order to generate evaluation instruments. All this requires time, effort, money and specialized personnel.

In an appreciable number of companies, functional analysis has been adopted to generate evaluation instruments, applying standardization criteria or job competence, expressed in a combination or integration of: expected results, criteria and evidence of performance.

As a consequence of all of the above, a very useful approach as a guide to personnel management has been mediated and of limited application due to the lack of adequate exploration, measurement and evaluation instruments.

The psychological or behavioral perspective

Entering the subject: the psychological or behavioral perspective originates from the research work of Dr. David Mc Clelland, from his work since the 1960s and his later experiences during the following decade, which led him to conclude that successful performance of individuals in roles or positions are linked to certain characteristics underlying the person, which he called competencies.

This approach uses the concept in a pluralized way (not competence, as the quality of being competent), to refer to the characteristics that determine the superior performance of the person. The same person possesses a varied set of competencies, which he unfolds as he assumes various roles or challenges. The competences are in the individual; they do not arise from the work process, as in the other models. In this perspective, the person is focused, with their motivations, their character traits, their self-image, their attitudes, their knowledge and their abilities.

Behavioral or behavioral?

Some authors refer to the psychological approach to competences with the qualification of behaviorist, as it is based on the analysis of observable behaviors. This qualification is not correct, because with the behaviorist word one falls into the error of linking the model of competences with the theoretical formulations of behaviorism, as a school or psychological theory, supported by the work of prestigious authors such as Pavlov, Watson and Skinner, among others, that it conceives behavior as a function of the environment. The most famous of the behavioral theories is Skinner's operant conditioning, with his thesis of positive and negative reinforcement. Far from being inspired by a behaviorist conception, the approach to competences is rather cognitive or constructivist,following methodological traces of Brunner (learning structures), Ausubel (meaningful learning), Vigotsky (mediation or mentoring factor) or Bandura (social learning).

More accurate would be to assign to this methodological approach the qualifier of behavioral. In any case, let us bear in mind that the concept of "conduct" predates Pavlov, Watson and Skinner, and refers to the concrete acts of the human being. An approach based on behavior does not have to be qualified as behavioral for it.

Competency measurement criteria

Competition is a theoretical concept, a construct: it refers to intangible aspects of reality; there are no direct references to indicate the existence of competition; It can only be referred to indirectly, through observable results, evidence or behaviors linked to ideal performance, which constitutes the validity premise of the concept.

Competences and roles

The competition is expressed in terms of the optimal performance of roles, linking these to the position structure of a company or organization. Competencies are referred to as personal attributes, which support superior performance, and are connected to the demands of the work process or positions, expressed in terms of task execution, value contribution, problem solving and of generating improvements.

Competence is the necessary condition for the successful performance of a role. In this sense, a position is understood as an integrated set of roles, such as those of planner, negotiator, communicator, leader, among others. The role of planner requires skills such as: strategic thinking, business sense and analytical thinking. The role of designer requires skills such as creative thinking, systems thinking, among others.

Mastery levels

In organizations, competencies are broken down into domain levels associated with the position structure. In this context, not all people need to display 100 percent skills. Competencies are associated with roles or positions. Obviously, a receptionist does not have to master the "customer orientation" competency to the same extent as the customer service analyst, and the customer does not have to master that competence to the same level required of the customer service supervisor. Then, mastery levels are predefined and established, depending on the scope of action and decision of each position, in a gradation that generally goes from 1 to 4 or basic, intermediate, advanced and masters.

Competition: a continuous variable

In the managerial approach, competence is perceived as a continuous variable, unlike the educational approach, which considers competence as a discrete and binary variable. It is considered continuous because it can assume various values within a structure of mastery levels, ranging from a minimum level to a maximum or master's level, depending on the demands of the positions or roles.

The positive gap

In this approach, a person is considered to be "below," "at the level of," or "above" the demands of their position. Then, something appears that is unacceptable from the curricular perspective, and it is the positive or ascending gap. This means that it is accepted as valid that a person has a mastery level greater than 100%, with respect to the demands of their current position. In this sense, the positive gap means that the person has the potential for development towards positions or positions that require a higher level of mastery, in the respective competence.

From the qualitative to the quantitative

Competence, even though it is a qualitative variable, can be measured in quantitative terms, using options other than psychometric tests. To this end, some measurement schemes have been developed, with intervals appropriate to their nature, based some on descriptive profiles or redactional sub-sets defined on the basis of observable behaviors, and others on indicators of competence domain, or items referring to individualized or specific observable behaviors.

Minimum acceptable gap

For the design of the assessment scales, it is very useful to consider the findings and contributions of Weber and Fechner, expressed in the so-called Law of Psychophysical Perceptions, mainly with the purpose of defining the minimum acceptable gap. This means, from how many points will it be considered that there is a gap in the mastery of a competence by an individual.

Probable error threshold

Applying the concept of threshold for perception of differences, of the aforementioned law, the minimum acceptable or qualifiable gap as such will be that located above the limit in which a person is capable of detecting differences, that is, close to ten percent, which was the limit detected in innumerable experiments by Weber and Fechner. Normally, between nine and ten points are chosen, and then any difference less than or equal to the established limit is rejected as an appreciation error; Of course, this applies with respect to the notion of minimum gap.

In an evaluation scale, to reduce the risk of imprecision to the detriment of the people evaluated, the minimum acceptable gap is determined in ten percent of the scale. Lower values are considered within the probable error threshold.

Scale sensitivity

It should be noted that the attempts at precision, expressed in the previous paragraphs, only apply to the case of scales based on domain indicators, which, because they are small items or units of measurement, have greater sensitivity and broaden the spectrum of values, reaching establish differences of less than ten points. In the case of scales based on descriptive profiles, the situation is different because the competence is normally disaggregated between four and eight measurement intervals. Obviously, in the latter case they can only establish differences greater than ten percent of the total scale, in wide intervals (for example, from fifteen to fifteen points).

Range width or difference intervals

Following the methodological criteria described above, the upper range of competence domain is established, on a scale of 1 to 100, which must be above 91 points, which means that whoever has a score higher than 91, will be considered A or in the maximum domain range of the competition.

Considering this same criterion, the width of the following or lower ranges is established: with a dispersion or interval between the upper and lower scores, greater than ten and less than or equal to fifteen percent. Greater than ten percent, to exceed the appreciation error threshold, and less than fifteen percent, to avoid grouping in an interval more than one unit of measurement or difference.

The job evaluation scales are established following this design criterion, with a range width between 12 and 15% with respect to the lower limit of the interval.

Evaluation or diagnostic criteria

Defining the rating of the gaps, beyond the evaluation, let us consider how the gaps are interpreted or the level of mastery resulting from the evaluation; This implies being able to define when a gap can be considered small or not very noticeable, moderate, considerable or critical. In this sense, the notion of maximum probable gap is an issue to be solved methodologically, to avoid abnormal dispersions of the data.

Let's start by "establishing the domain spectrum of the competition." We have seen ill-conceived rating scales, where gaps of more than 50% are spoken of, and are considered normal.

Let us bear in mind that if a person has a gap of 50% or more in a certain competence, and this is important for the achievement of the objectives of the position, then the gap is highly critical or we are facing an abnormal case; the respective person is badly placed in the position and should be removed from it, or would require urgent training, if it is decided to keep him in the position.

Spectrum of competition domain

To establish the spectrum of competence domain, it is necessary to start from a methodological premise, expressed by the following reasoning: if the company applies the selection of personnel, whether of external or internal origin, that determines the entry of a person to a position, on the basis of a basic professional profile, and having passed or satisfied certain minimum suitability requirements, then the notion of maximum gap should be between 40 and 45%. In other words, the minimum domain of a competition has to be 65%. Thus, a gap greater than 35% would be considered critical.

The range described, between 40 and 45%, could be established as the lower limit of the spectrum of dominance of the competition. And we would find the following spectrum: from 45% to 100%, considering that it could exceed 100 when considering that we evaluate against a level of mastery required by the position, and we understand that a person may well possess a higher mastery level in. When moving to a higher position, the level of dominance varies, decreasing, and this becomes cyclical.

Descriptive profiles

Descriptive profiles are semantic boundaries made up of paragraphs that describe a set of observable behaviors referred to domain levels, generally arranged in ascending order, on a prelative scale, which means that the domain of a higher level supposes the domain of the previous ones. In this case, the competence of a person is evaluated with respect to a competence associated with their position, comparing their performance against each descriptive paragraph, and selecting the one that best reflects the performance of the person. In this case, the positive or negative gap will result from the difference between the performance and the level of mastery exhibited.

An example of descriptive profiles is shown below:

Customer orientation

Ability to establish, maintain and develop harmonious relationships with customers, internal and external, based on the satisfaction of their requirements, within the framework of the organization's quality philosophy and business objectives. It is characterized by empathy, service attitude and business vision.

Descriptive Profiles:

B Requires training to reach level 1

1 Serve the customer properly. Their treatment of clients corresponds to the basic rules of courtesy and good treatment. Listen carefully and guide the client regarding their request for information.

2 Interprets and channels the customer's requirement towards their satisfaction. Establishes harmonious relationships with the client and understands their requirements and priorities. Guides the client on the appropriate way to address the satisfaction of their requirement. Maintains follow-up to the client's request, until it is closed.

3 Provide solutions to the customer. It acts according to the satisfaction of the client's requirement, or the solution of their problem. Inform the client clearly, sincerely and honestly. She is personally committed to solving the problem or satisfying the requirement. Maintains follow-up, until the total satisfaction of the requirement.

4 It gives added value in the relationship with the client. It is identified with the customer's need, beyond the scope of the requirement. It provides you with guidance and offers you complementary products and services to improve your results. Maintains an attitude of total availability with the client.

5 Commits to the customer. She understands the perspective and points of view of the client and is committed to their needs, to whom she provides guidance to improve their results. Supports the client in the search for solutions beyond what is established, in some cases, her area of responsibility.

Domain indicators

The domain indicators are a set of items or short sentences referring to observable behaviors, which allow evaluating the competence domain considering, one by one, the entire set of indicators. In this case, the positive or negative gap will result from the score obtained, considering whether the person's performance with respect to the indicator is A, B or C, where A could mean "fully meets or exceeds the demands of the position", B " satisfies the minimum requirements ”and C“ requires formative training to achieve the required performance ”, regarding the indicator. For each letter or other sign chosen, there will be a score, and the domain will be expressed in a total score.

The level of mastery will be expressed in the letters A, B or C. For this, an evaluation scale is generated, considering their design criteria. Considering the Weber-Fechtner law, the minimum gap will be established.

Gap rating

If we evaluate by descriptive tables, and the interval between them is 15%, then a small gap is 15% (one level below), a moderate one can be 30% (two levels below) and a larger one would be 45% (three levels below). These variations occur "in leaps". However, this can be corrected by incorporating the notions of central value and trends, that is, the person can be evaluated as follows: a. responds to the set of observable behaviors of the profile, b. responds partially, tending to the immediately preceding profile, or c. responds to the profile, tending to behaviors of the immediate higher level.

The evaluation by indicators allows greater precision in the gaps, since the resulting values can be extended in a harmonic way, without jumps between the ranges. In this methodological path, a small gap would be less than or equal to 15%, a moderate one would be greater than 15 and up to 20%, a considerable one would be between 21 and 30%, and a critical gap would be greater than 40%, understanding that never it exceeds 45%, because it is the lower limit of the spectrum.

Competition impact

A complementary methodological resource in the evaluation of competencies consists of rating each competency associated with a position according to its impact on the objectives or the results that it must achieve, as "important" and "complementary". In this way, the diagnosis is more accurate, because a considerable gap in an important competence for the position is not the same as in a complementary competence.

A contribution to assess functional skills

Next, a contribution is presented that provides guidance for the design of assessment instruments for functional competencies, through the application of the descriptive profiles method.

Functional competencies

Mastery Levels:

1 Master, at an elementary level, the theoretical, normative, methodological and operational aspects inherent to this subject. Know and follow the procedures established to carry out their work. Manages, at an acceptable performance level, computerized applications and / or work resources or tools. It manages to solve low complexity problems. It requires supervision and technical guides to carry out its work.

2 Provides ideas to improve their work and solves problems of moderate complexity inherent in this subject. Knows and applies the techniques and methods to carry out their work, adjusting to the regulatory and quality requirements. Offers suggestions for solving problems and improving methods and / or procedures. Handles computerized applications and / or methodological resources or tools, with a good level of performance. Requires little supervision.

3 Demonstrates solidity in the domain of matter. Above the previous level, it acts autonomously towards the fulfillment of work objectives and goals, without requiring specific technical guides or supervision. It solves problems of considerable complexity, which involve analysis work, calculations of medium complexity or negotiation with people from other work teams. Handles computerized applications and / or methodological resources or tools, with a high level of performance.

4 He is a specialist in the field. Above the previous level, it shows the ability to guide or advise others in the search or development of improvements, in the adaptation of systems or methodological schemes, and in the solution of complex problems that require high-level analysis, complex calculations or negotiation external.

5 It performs with notorious mastery and provides comprehensive solutions in this matter. Above the previous level, it is capable of generating innovations and acts towards the search for improvements and / or solutions applicable to the global process or business, or with the purpose of solving problems, facing potential challenges or crises, with a sense of anticipation.

With this essay, the author wanted to share in solidarity, and even in the details, with students, professionals and other people who seek answers and options, on this subject on which very little is achieved on the web and in the bibliography.