Online assessments offer new opportunities to gain deeper insight into how test takers solve problems and help close the feedback loop between teaching and learning — both in the classroom and in professional settings. By incorporating features like multimedia, complex testing logic and custom items, you can author engaging assessments that go beyond multiple choice and mimic real-life scenarios or learning happening inside a classroom. With so many options, choosing the right assessment authoring solution can seem like a daunting task. But when you develop a framework for what your assessment needs to achieve, the features you need become more apparent. Below, we’ve outlined how to get started choosing a solution, and some of the features to look for.
What kind of test are you creating? Is it formative, summative or diagnostic? What competencies do you need to measure? What kind of tasks do you need test-takers to perform? Different types of assessments require different authoring capabilities. While a good multiple choice question can differentiate performance for content knowledge, sometimes you need test takers to respond in ways that require more complex reasoning. For example, if you are looking to create a test with high discriminating power between high and low-performing test-takers, you may want to choose a platform that supports Computer Adaptive Testing. Asking yourself these questions before you start researching assessment authoring platforms will give you a better understanding of the features and functionalities you need from your solution.
Once you’ve defined what type of test you’re creating and why, you can narrow down your focus to how. Even if you aren’t 100% sure of which functionalities you’ll need, you can still start your research by looking into assessment authoring platforms that offer flexibility, freedom and control. We’ve taken a deeper look at some of these platform features and how they can benefit you as a test author.
Whether or not you are a professional test author should not be a deterrent to being able to create digital assessments. The most flexible assessment authoring platforms allow you to create a range of assessments and item types — from complex and adaptive to simple multiple choice.
Do you need to break up your test into different parts or sub-sections? Do those sections have different time allowances or require different tools? Depending on the needs of your assessment, you might need more test preference settings that define and regulate testing sessions. Other features might include: randomized item selections, order shuffling, limiting the number of attempts, navigation settings, and test outcomes (scoring).
An assessment authoring platform that supports all QTI interactions, including Technology Enhanced Items (TEIs), gives you the most freedom to create any type of assessment. TEIs are complex assessment items that measure 21st century skills like critical thinking, problem solving and collaboration. Many of the standardized QTI interactions allow for these more complex computer enabled TEI interactions. Using an assessment platform that supports the IMS QTI Portable Custom Interaction (PCI) standard, test developers can build custom TEIs, enabling interoperability with any new or existing technology systems.
Traditional assessment is largely linear, where learners move through an assessment one question at a time until reaching the end. Branching tests, on the other hand, provide context to learning by giving choices that branch out to different consequences, mimicking real life. As a test author, you may want to create both linear and branching tests depending on the competencies you need to measure, so having the option in your platform is key.
As mentioned above a Computer Adaptive Testing (CAT) solution helps you identify a test-taker’s performance range by adapting the difficulty of a question (or test section) based on their previous answers. CAT engines use various algorithms based on item response theory (IRT) or other factors to determine mastery of a topic. They also provide highly reliable data more quickly than traditional assessments.
The downside to CAT is the requirement to create a larger pool of items across the diverse set of skills you are measuring. That large pool can however provide test security – test takers would be hard pressed to memorize all the possible questions that might be included during their particular test session, just as it is difficult to ‘steal’ all of the possible questions within the item pool.
As a test author, you spend valuable time creating unique items for your assessments. This is an investment you don’t want to lose. However, all too often, proprietary solutions do not allow for the seamless exchange of content and data from one platform to the next, leaving you to make the choice: stick with a vendor that might not be working for you, pay for expensive and time consuming content transformations, or give up your work. An interoperable assessment platform (e.g., one built on QTI open standards) ensures that your items and content are easily transferable, and not locked within your system.
Being able to easily organize your content is also an important factor when choosing an assessment authoring platform. Solutions that support metadata tagging can help you quickly navigate your content and provide rich data about the assessment as it relates to student success.
An assessment authoring platform that supports accessibility and the ability to include interactive assessment tools, like a screen reader or digital calculator, allows you to author engaging assessments that reflect a more authentic learning experience and meet the needs of a wider and diverse learner group.
Pro Tip: Assessment delivery platforms should not only support the Web Content Accessibility Guidelines (WCAG) 2.1 AA Success Criteria, but allow authors to input critical accessibility content for test takers. Some additional tools that empower learners include:
—
As educational technology becomes more advanced, the potential for test-authors to create engaging, real-world testing experiences continues to grow. By determining the purpose of your test and anticipating that your requirements may inevitably change in the future, you can choose a flexible assessment authoring platform that advances with your needs.