Testing readability and measuring it: 2018-2020
Macquarie University and Biotext have collaborated in two research projects on testing the readability of health information (2018-20). Both projects were funded by the Department of Industry, Innovation and Science as Innovation Connections Grants.
#1 The first Innovation Connections project (2018-19) was largely a scoping study, setting out the parameters of readability from the language perspective and different types of readers. These correspond with the five major sections of the first project report (February 2019):
– Section 1. Linguistic aspects of readability, health literacy and health communication
– Section 2. Readability checkers used by Australian governments, and their scoring systems
– Section 3. Benchmarking readability for Australian consumers of health information
– Section 4. The reading skills of Australians who use English as a second language
– Section 5. Readability and access to medical information across the health sector
A preliminary experiment was also conducted (2018-19) on the readability of content on a Department of Health immunisation website. The experiment involved 30 parents with children at the University’s child-care centres. All had professional training and occupations, and were either first- or second- language users of English. They were tasked with finding specific information on the website, and debriefed on their experience of using it. Eye-tracking equipment was used to trace their explorations of webpages and mouseclicks as they moved to a new page. The overall time they took to complete the 7 tasks, and the correctness of their answers was recorded, as was how they experienced the website’s features, in terms of likes/dislikes, and via elicited individual comments.
#2 In the second Innovation Connections project (2019-2020), a second eye-tracking experiment was conducted with 58 university students, both first- and second-language English, to test whether differing levels of readability could be measured by means of eye movements (fixation counts). The test materials consisted of short pieces of health information on dementia, diabetes and epilepsy, of which alternative high, medium and low difficulty versions were created experimentally, by modifying their language (choice of words and sentence patterns). The participants each read just one of the modified pieces on each topic, its stylistic level selected randomly. They then responded to a set of 5 short answer questions about the three topics, to test how accurately they could understand the information through the styiistic level at which it was delivered to them. The time they took to read each text was also recorded.
Significant differences were found in the numbers of eye fixations: there were consistently higher fixation scores from low, medium to high style texts for both language groups, and in the overall times taken to read the texts at each level. But the first-language participants registered consistently fewer eye fixations and took less time to read each piece than members of the second-language group. Researchers concluded these two (number of fixations, time taken to read) are robust measures of readability, which should be further tested on more diverse populations.