A Look into Testing Psychometrics with two TAO Psychometricians

We sat down with our psychometricians – Hélène MAYER and Franck GISMONDI – for an interview on their role within the TAO team.

(Question) Could you give us a definition of psychometrics?

(Mayer & Gismondi) Psychometrics is a discipline designed to develop and analyze psychological and educational measurements. It is dedicated to the construction and the use of tests, questionnaires, personality assessments…

(Q) What are the advantages of creating a psychometric test?

(HM & FG)

• Psychometric tests are reliable in terms of evaluation. Hence, their scientific validation is globally acknowledged.
• Their rating is objective and affective parameters do not interfere.
• Every test taker takes his test under the same conditions.
• Psychometrics also enables to build adaptive tests.

(Q) What role do you play in the construction of a psychometric test?

(HM & FG) We may operate at different levels, i.e.:

• A test is set up in various ways according to its purpose (evaluation, diagnostic, certification, …) Our role is to walk the expert of a domain in his creation of items
• We determine a methodology to have the test taken under the same conditions for all test takers
• According psychometrics, a test has to be taken by a sample of test takers before it is deployed. This enables us to assess the items and the test. Indeed, with some statistic methods we are capable of ensuring that an item reaches its goal (for instance, by correctly measuring the chosen topic). Items with bad statistics have to be reformulated, their chronological order may be modified and if necessary, some of them may be removed while others will be suggested as a surrogate.
• Psychometric statistics validate the test itself through a study of the referential sample, calibration, validity, constancy and reliability of the test.

(Q) Franck, you took part in the realization of the Aladdin test. [Please see below for screenshots of the Aladdin test or directly import it on the TAO platform.] Could you tell us about your contribution?

(FG) I focused on the logic of the story, I checked if the items were really in relation with the sample target (i.e., youngsters).

This test consists in a framework with various items measuring some fields of knowledge such as sports and biology. I advised that each item should represent one and a single field and that each item should vary in its representation (for instance, in Aladdin, each item is performed with a different QTI item).

To process onward, we should check the validity of this test with a sample of test takers and make sure that the items measure the intended fields of knowledge. If it is not the case, they may be reformulated and retested. We’d have to check that the items enable us to differentiate the competence of all test takers (i.e., measure the items’ difficulty level). If 2 items lead to similar results, it means that one should be removed. Indeed, it does not give added value to the test and it does not differentiate the various levels of different test takers.”

We sat down with our psychometricians – Hélène MAYER and Franck GISMONDI – for an interview on their role within the TAO team.

(Question) Could you give us a definition of psychometrics?

(Mayer & Gismondi) Psychometrics is a discipline designed to develop and analyze psychological and educational measurements. It is dedicated to the construction and the use of tests, questionnaires, personality assessments…

(Q) What are the advantages of creating a psychometric test?

(HM & FG)

• Psychometric tests are reliable in terms of evaluation. Hence, their scientific validation is globally acknowledged.
• Their rating is objective and affective parameters do not interfere.
• Every test taker takes his test under the same conditions.
• Psychometrics also enables to build adaptive tests.

(Q) What role do you play in the construction of a psychometric test?

(HM & FG) We may operate at different levels, i.e.:

• A test is set up in various ways according to its purpose (evaluation, diagnostic, certification, …) Our role is to walk the expert of a domain in his creation of items
• We determine a methodology to have the test taken under the same conditions for all test takers
• According psychometrics, a test has to be taken by a sample of test takers before it is deployed. This enables us to assess the items and the test. Indeed, with some statistic methods we are capable of ensuring that an item reaches its goal (for instance, by correctly measuring the chosen topic). Items with bad statistics have to be reformulated, their chronological order may be modified and if necessary, some of them may be removed while others will be suggested as a surrogate.
• Psychometric statistics validate the test itself through a study of the referential sample, calibration, validity, constancy and reliability of the test.

(Q) Franck, you took part in the realization of the Aladdin test. [Please see below for screenshots of the Aladdin test or directly import it on the TAO platform.] Could you tell us about your contribution?

(FG) I focused on the logic of the story, I checked if the items were really in relation with the sample target (i.e., youngsters).

This test consists in a framework with various items measuring some fields of knowledge such as sports and biology. I advised that each item should represent one and a single field and that each item should vary in its representation (for instance, in Aladdin, each item is performed with a different QTI item).

To process onward, we should check the validity of this test with a sample of test takers and make sure that the items measure the intended fields of knowledge. If it is not the case, they may be reformulated and retested. We’d have to check that the items enable us to differentiate the competence of all test takers (i.e., measure the items’ difficulty level). If 2 items lead to similar results, it means that one should be removed. Indeed, it does not give added value to the test and it does not differentiate the various levels of different test takers.”