Interoperability Assurances: A Step by Step Guide

Creating assessment content and tests that can be imported and exported between platforms is crucial to support an institution’s evolving technology needs. Thanks to the interoperability standards set and maintained by IMS Global Learning Consortium, this exchange of test content is possible. With their open frameworks, IMS standards make it possible to build an integratabtle technology stack, or migrate assessment content to a new platform altogether without compromising or losing critical aspects of a test.

Many assessment programs are now requiring the tools that they are using for their projects to be IMS certified. In this post, we discuss the IMS QTI interoperability standard and how you can verify your project features for Interoperability Assurance against IMS validators. 

QTI Compliant Does Not Necessarily Mean Interoperable 

The IMS QTI standard in particular has become a hallmark of interoperability, where the terms are often used almost interchangeably. However, assuming a platform’s interoperability based on the mention of QTI compliance alone can cost an organization in time, effort, resources and budget. 

Many online assessment solutions claim to be QTI compliant, but are never actually validated or certified. It’s important for developers to understand the importance of IMS certification and QTI compliance in order to build truly standards-based assessment solutions and reduce errors that may arise when information is exchanged between systems. For developers, this includes testing your assessment content against IMS validators. 

Suggested Steps for Interoperability Assurance

Migrating systems and exchanging content can be complicated. A lack of software interoperability often causes valuable data to become lost in translation, or worse, it may be just unusable. Below, we’ve outlined our recommended process for ensuring QTI interoperability when developing new assessment content for a project. 

Programs should require IMS certification of the systems used for their projects, and specifically list any features not included at the required certification levels. In addition, if the project includes multiple vendors, the timeline and budget should allow for the Interoperability Assurance phase of work, which could last from weeks to months.

While the steps below detail specific actions, exchanging organizations may find that different processes are more effective for their unique contexts. This guide is a suggested approach only. 

1. Decide which features you need for your project. You may need assistance from IMS, consultants, or IMS groups to understand the specific features. We recommend that all contracts that use QTI 3 follow at least the IMS QTI Level 1 Entry Certification features. Systems using more complex feature sets should certify at Level 2 Core. All of the participating parties need to agree to the list of features, including any extra features not included in certification profiles. 
2. Create sample content (assessment content and/or PNP files) that incorporates ALL of the features included for the project. 
3. Verify that materials you create are valid QTI 3 using the official IMS validators. 
4. Exchange the content in the direction of the workflow expected for the contracted scope of work. 
5. Verify that the final endpoint (for example: import in a content management system, or candidate presentation in a delivery system) matches the expectations of the contract. Also verify that the examples use all the agreed upon features of the contract.

Note: If your project allows for the use of Assistive Technologies (AT), we recommend that users familiar with the specified AT interact with the sample content to ensure that content not only meets the technical expectations of the project, but that the content is actually accessible (perceivable, operable, understandable, robust – the WCAG 2.0 Principles of accessibility) by users of the specified AT. 

1. For any features that do not meet the expectations of the project, determine if the problem stems from the supplied source or any of downstream systems which receive and use the materials. 
2. Iterate the above process (steps 2–7) until all features are verified from all the participating systems. 
3. Report the Interoperability Assurance to the contracting agency or agencies.

— 

Interoperability is quickly becoming a baseline requirement for institutions and organizations when choosing new assessment solutions or bringing their tests online for the first time. We recommend you use the IMS standards as a backbone when building your online testing solutions. Doing so will enable flexibility for any features you create as technologies and requirements change down the line. 

Check out our recent post on using IMS standards to connect assessment to learning for more on interoperability standards.