How to evaluate skills and performance?

The expectations of corporate training departments

In the first part of this article, we observed that the expectations of training departments in terms of language training results are often unrealistic, disconnected from the actual objectives of the training, and induce perverse effects on learning. The specifications in the profession’s calls for tender are full of texts where it is required, for example, that after 10 or 15 hours of training, at a rate of one hour per week, spread over a period of up to 6 months, the trainee must have progressed by half a level on the CEFR scale or by X points on the TOEIC. One only needs to consult publications by specialists in the field, such as Cambridge Assessment or The American Council on the Teaching of Foreign Languages (ACTFL), to realise that it takes many hours of guided learning to move from one CEFR level to another (150 to 300) and that fluency and spontaneity are acquired through almost daily practice. Who has the means and time for this in companies today? Moreover, most people who undertake training within their company are not concerned about their level, unless it is imposed. They rather wish to be able to participate in conferences in a foreign language, respond to emails or telephone calls, welcome visitors, give a presentation, be more at ease, or simply maintain their current level. Faced with these demands, language training professionals tend to try to accommodate everyone by allowing trainers to adapt their support to the trainee’s requests, while offering the company tools to measure micro-progression in level that have no meaning and do not reflect reality. This has contributed to discrediting the profession, as the results do not materialise.

Can we measure something other than level progression?

It would be more logical to measure the results of training according to operational objectives and communicative effectiveness rather than in terms of level. But how to proceed? The skills-based approach and the action-oriented perspective have long been adopted by certain providers and certifiers. However, language training in companies too often remains focused on achieving a higher level.

1. Language level, competence and performance

Language level is often represented by a score or letter (A1, A2, B1, B2, C1, C2). While useful, this representation is reductive. According to the CEFR, a language level mobilises linguistic, pragmatic, and sociolinguistic competences. In the professional context, one can target transversal professional language skills: ability to give a technical presentation, write emails, read a specialised article, interact in meetings, negotiate a contract, etc. Each skill can be broken down into micro-skills. As the number of variables is significant, all sorts of profiles are possible. The notion of global level is therefore relative. Two people with the same global “level” can have very different profiles and needs. Since linguistic competence is multidimensional, it is necessary to find tools other than the linear scale of level to measure progress in skills without changing the general level. “Performance” or “communicative effectiveness” is defined as the degree of mastery of a task or skill in the foreign language. Learning the language is not an end in itself, but a means to achieve operational objectives.

2. How to measure skills and performance?

It is possible to measure the mastery of professional skills, even if they are complex, by subjecting the learner to scenarios and simulations, as some tests or certifying examinations do. In the context of formative assessment, the objective is primarily pedagogical: to monitor, accelerate, and consolidate the acquisition of skills. It is therefore about designing relevant continuous assessment tools, based on tasks to be performed, and having a clear reference framework of professional language skills. A detailed and personalised programme, with milestones, tools, and clear and easy-to-adopt indicators to measure progression, is necessary. All must be adapted to the limited means allocated to language training in companies.

3. The tools

a) The framework of transversal professional language skills

A framework for listing and sequencing skills is essential. The CEFR provides a basic foundation, but must be complemented by a framework of professional skills. Objectives, programmes, and evaluations must be built from this framework.

b) The tests

External (certifying) tests measure the overall level, while tests internal to the training evaluate progression throughout the course. Internal tests can be designed by the training organisation and adapted to specific objectives.

c) The evaluation system

Targeted skills are evaluated before and after each pedagogical sequence. Each sequence includes a continuous evaluation cycle that identifies progress and readjusts the training.

d) The indicators: measurement tools and performance indicators

Instead of a linear scale, an learner’s linguistic profile can be represented in a more complex way, to better understand their strengths and areas for improvement. A simple scoring system (for example from 0 to 5) can be used to represent the achievement of objectives. The learner must also self-evaluate, which helps detect potential problems and strengthens the credibility of the system.

4. Methodology: evaluation approach and modalities

The principle is to evaluate the learner by reproducing the real conditions of language use. If one wants to evaluate the ability to interact in meetings, one conducts simulations. A pragmatic and action-centred approach is recommended.

5. Emerging trends

The future of professional language training will see increasing integration between language skills, soft skills, and hard skills. There will be more of a shift to a model where one learns a professional skill in a foreign language. Digitalisation will also simplify monitoring, continuous evaluation, and reporting, but care must be taken not to sacrifice the quality of solutions to their marketing.

Conclusion

The practices described here are not new, but remain insufficiently applied in professional training. This leads to a waste of resources and discredits language training, as the expectations of buyers are not aligned with the actual objectives of the training. It may be time to adopt a more pragmatic approach, focused on skills and performance, more in line with the real needs of the field. However, the success of a training project primarily depends on the involvement, motivation, and commitment of learners, trainers, and other stakeholders in the system.