Finding an item authoring platform to fit within an existing assessment ecosystem
In Switzerland, the main responsibility for education and cultural affairs resides with the cantons, the member states of the Swiss Confederation. The 26 cantonal Ministers of Education together form an administrative body, the Swiss Conference of Cantonal Ministers of Education (EDK), which coordinates educational and cultural policies at the national level.
In order to monitor and ensure a consistent implementation of educational standards in all cantons, EDK administers nationwide tests of student achievement for Grades 2, 6 and 9 in mathematics, natural science, national languages (German, French, and Italian), and foreign languages to a select sample of students. That said, the cantons are responsible for measuring the students’ individual achievements. For this purpose, some cantons host their own item banks and administer their own tests to all students within their jurisdiction. However, for those cantons who don’t have these resources, a shared item bank is beneficial.
CHALLENGE
Researching and Selecting a System That Will Work Well In A Multilingual, Decentralized Environment
Standardization of assessments is uniquely challenging in Switzerland, due to four official languages and the necessity of coordinating across 26 cantons. EDK has found proprietary online assessment systems very restraining, making customization both difficult and expensive. Openness, flexibility, and the ability to customize without hidden costs are all important to EDK as they move forward with their plan for a monitoring system for compulsory school education.
Their goals include:
- Having a tool to create items to test and monitor the implementation of nationwide learning standards
- Developing an item bank that can be shared
- Making it available to all 26 cantons
- Having an Open Source system that is compatible with existing systems, including proprietary ones
Solution
TAO, a Commercial-Grade Open Source Assessment Platform
When EDK began researching a solution for standardized testing, they looked first to European countries with experience in student achievement testing. Among them, Luxembourg was one of the most advanced, having already moved on to computer-based testing. That is where they first discovered TAO as a possible Open Source solution. After careful analysis, EDK chose TAO for several reasons:
- Open Source set them free from the restraints of expensive proprietary systems.
- TAO offered the possibility to include complex item types, in addition to standard QTI, which makes it possible to create advanced items.
- OECD (Organization for Economic Co-operation and Development) had selected TAO to conduct PISA, the Programme for International Student Assessment, a worldwide study to compare students’ proficiency in mathematics, science, and reading. This gave EDK confidence that TAO is a stable system capable of managing largescale deployments.
- The TAO team is multilingual and therefore could effectively communicate with stakeholders across all cantons, which have different language preferences.
Results
Implementing the Solution for Online Testing
At the time of this writing, EDK is in the process of creating the item bank, which it has already field-tested in some schools. They are working towards a bigger pilot in 2015, and a large assessment of 25,000 test-takers in 2016, which will enable them to make comparisons between cantons, per nationwide learning standards implemented in 2011. In 2017, they expect to test Grade 6 in Switzerland’s national languages (German, French, and Italian) as well as foreign languages.
“With an Open Source platform like TAO, we are free to do our own customization, because we can control the source code. This came as a huge relief, since proprietary systems are very restraining. The more a proprietary system was customized the more we would be dependent on it. Thankfully, with TAO we found a solution that is open and flexible for the future. It is a big step in the right direction to ensure consistent, nationwide implementation of our learning standards.”
– Dr. Vera Husfeldt, Head of Division Quality Development at the EDK since 2012.
