Scoring items automatically
Item scores are determined by the student’s performance in the various Interactions which make up the Items of which the Test is composed. Interactions generate individual scores which count towards the overall score of the Item. These individual scores can be tallied in different ways.
Items can be scored automatically in TAO, or manually in TAO Grader. TAO Grader provides the possibility for human scorers to score open-ended questions, and is an add-on available to Ignite and Enterprise customers. This chapter shows how to configure the automatic scoring of items.
For information on test scoring rules, see Scoring Tests.
The scoring method for an item needs to be defined during the item authoring process. There are two options for automatic scoring:
- The response of a test-taker is marked as wholly correct or wholly incorrect. In this case, no partial credit is awarded, so if part of the response is incorrect, the whole answer is marked as incorrect;
- Partial credit is awarded for the correct parts of a response, even if the response is not wholly correct.
To set up your chosen scoring method for the item in question, go to the Response window after you have created your interaction and follow the steps below.
1. In the Response Properties panel on the right, locate the Response Processing pull-down choice box offering the two options: Match Correct and Map Response.
Match Correct: With this processing option, either the question is answered correctly or incorrectly.
Map Response: With this processing option, partial credit is awarded for correct parts of the response. This is useful for questions where the answer is given in multiple parts. With the map response option, more marks can be awarded for some parts of the answer than other parts, if you consider them to be of higher importance.
2. Select the Response Processing type which is appropriate for the question.
This is going to depend to a large extent on preference, particularly with respect to partial credit.
If you select Match Correct, stop here – the scoring settings for this interaction are complete.
3. If you select Map Response, review the responses in your interaction, then determine and assign the corresponding marks to each potential response.
Partial credit can be awarded on the Inteaction canvas by entering in the score box the number of marks you wish to award for that response.
Note: For most interaction types, score boxes are located to the right of the potential responses.
4. Set the values of the Score Range fields, located in the Response Properties panel.
This is where you can specify the minimum and maximum number of marks awarded for this interaction. Its use is optional.
The minimum score indicates the minimum number of responses the Test-taker is required to select for it to be a valid answer. If the interaction involves selecting more than one response, the maximum score should reflect the total marks for all correct responses, i.e. the maximum score possible for your Interaction. If only one response is expected, the maximum should equal that of the marks in the score box with the highest number. Adjust these values as needed.
So if, for example, there are two correct responses in the question, and you have assigned 1 mark to one of them, and 2 marks to the other, the Test-taker would get a score of 3 if they are both correct (providing the maximum is set to 3 or above).
Other values in the Score Range field include Mapping Default, which contains the default value given if no specific scores are assigned to a response, and the check box used to Define Correct Response. The latter should be checked if there are specific correct responses, and left unchecked if the correctness of the answer is dependent on the sum of marks accumulated by the Test-taker in answering the interaction.
Lastly, it should be noted that TAO assigns a Response Identifier to each interaction response. It is best not to change this.
Map Response Example
In this example, there are three possible choices with the correct response having a value of 2 and the incorrect responses each having a score of -1 from the default mapping setting.
Choosing the correct response alone would give the test-taker a score of 2. If they chose the correct and an incorrect response they would receive a score of 1. And if they chose two incorrect responses they would receive a score of -2.
