Best Practices for Writing Assessments in Health Education

Writing assessments is a thankless, but necessary job. Without them, you’d have no way of knowing if your students are truly comprehending the material.

While a quick Google search on the topic will yield millions of results, much of it is too vague to really be useful; “ensure that the assessment is fair and unbiased”, “provide feedback to students on their performance”. And, with so many results, where do you even start? How many articles will you need to wade through just to get a few genuinely helpful insights?

ReelDx has been writing assessments based on our patient video cases for years, and most of our tips were learned the hard way, through trial and error. But whether you want to assess comprehension of a case or reinforce other concepts, these tips should help point you in the right direction.

Without further ado, here are the lessons we’ve learned the hard way from writing dozens of assessments:

‍

‍

Download Our Guide to Writing Great Assessments

‍

‍

Format

1.Keep it short. We abide by the principle of micro-learning: each case (and assessment) should work in roughly 15-minute chunks, which will be enough to force students to focus on the material, but not so long that they lose focus or interest. Suppose a typical assessment question takes a student one minute (depending on the complexity). That means students will learn best when an assessment is 10-15 questions long (or, in our case, since they also have to review a video and supplementary materials, we try to keep our assessments to 5-7 questions).

2.Follow a consistent format. For our questions at ReelDx, we use either 4 answer choices (where one is correct) or 6 answer choices (where multiple could be correct). The reason for this is simple: if learners aren’t concentrating on the format, they’re more likely to learn from the point of the question.

3.Don’t make it easy for students to cheat. Randomize the answer choices (if possible), the order of questions, and mix up questions from a larger question bank (again, if possible).

4.On the other hand, be clear about your expectations and what a passing score entails. Is it open book or closed book (or open Google or closed Google)? Our assessments are all open Google since a) we can’t control whether students Google the information, and b) it’s more real-world for students to be able to consult any related information to help them do their jobs better when interacting with patients.

Difficulty

1.Strive for a Goldilocks level of difficulty; too easy and students will be bored, too hard and students won’t be able to answer the questions appropriately. This concept is particularly tricky for us here at ReelDx since our students run the gamut from newbies to advanced graduate students. When starting to build an assessment, we keep the “typical” learner in mind and write the questions that we think that student should be able to answer in order to demonstrate mastery of the material.

2.Assessments should move from easy to complex, from questions that measure recall to those that require learners to apply what they’ve learned. Starting simple and building allows learners to re-master the material as they work through the assessment, shoring up the basics and moving to hypotheticals.

Content

1.Don’t be too simple. True/false falls under this category.

2.Don’t try to trick learners with the question stem. That means no “all of the above, none of the above, NOT, etc.” We designed our assessments to reinforce real learning, not provide “gotchas” for students. Asking questions about what’s NOT correct just draws attention to the wrong information.

3.Stay focused on what you want students to remember. Each question should relate to the learning objectives of the lesson and the course. Often, assessments focus on details that don’t relate to the key objectives of the course.

4.Every wrong answer should have a teaching point too. Obviously, we want to guide learners to the right answer, but feedback for the wrong answer should be written in way that it draws a learner’s attention to key points in the case.

We hope you found this advice useful. I sure wish I had this guide when we started down this journey of writing assessments. It would have saved me a lot of trial and error and googling. So, good luck and let me be the only one probably to say thank you for writing a great assessment.

‍