26-member panel conducts independent reviews
In March 2014, 32 health and social services providers, program directors,
trainers and supervisors volunteered to assess the understandability and
actionability of Beginnings Pregnancy Guide using the PEMAT-P — Patient
Education Materials Assessment Tool for Printable Materials from the Agency
for Healthcare Research and Quality. Twenty-six completed the assessment.
The secondary purpose of the review was to implement the new PEMAT-P
and get a sense of its utility. Each of the six booklets comprising the Beginnings
Pregnancy Guide was reviewed separately by four or five individuals working
Beginnings Pregnancy Guide Earns High Marks
Understandability: 96.5% Actionability: 98%
I am particularly pleased with the nearly perfect actionability score.
Goethe said it centuries ago, “Understanding is not enough, we must act.”
Understandability: (a multisyllabic bit of Latinized jargon that is questionable
in the context of health literacy) is the quality of information that enables users
comprehend its meaning. The PEMAT-P asks reviewers to rate printable materials
on 17 factors in six categories known to affect understandability. A score of 1
indicates the factor is present; 0 indicates it is not; NA indicates the factor is not
applicable. Factor scores are expressed as the percent of possible scores of 1. The
final score is calculated as the average of reviewers’ combined scores in each category.
Here are the category scores:
Content: 94% Word Choice/Style: 96% Use of Numbers: 100%
Organization: 97% Layout/Design: 96% Visual Aids: 96%
“Overall the book looks and reads very clearly and will be very understandable
for a low level reader.”
Actionability (more jargon) is the quality of information
that enables users to take action. Reviewers score seven contributing factors. The
final score is calculated as the average of the factor scores. The Pregnancy Guide
earned 100% on five of the actionability factors.
“An additional plus is the links to other resources for specific topics.”
unsolicited comments on the materials.PEMAT-P shows good reliability Testing during development of the tool showed acceptable validity. Results of
this project suggest the tool has good inter-rater reliability, meaning that multiple
reviewers of the same materials rate the factors similarly.
About one third of the reviewers struggled with the PEMAT web page. Technical
difficulties may have discouraged some of the six who did not submit a competed form.
See Table 4.
Personally, I recommend relabeling the buttons in the top menu. I expected the PRINT
button to print something; it brings up the Printed Materials form. The bottom menu
buttons are inactive on my machine. Those buttons and the frame around the form take
up space and require printing on two pages in too-small type. I, and some others, found
the numbering on the Printable Materials form confusing; it skips items related only to
Overall, the PEMAT-P is a useful at-your-desk review that can improve materials in the
development process and weed out complex, fact heavy, concept-dense materials. It
cannot replace testing by intended users - both teachers and learners.
Kudos to the developers of the PEMAT: Michael Wolf and Cindy Brach
Thanks to the reviewers: Betsy Rubin, Lori Lake, Pamela Cho, Michelle Breuer,
Dora McKean, Kath Anderson, Joanne Martin, Tennessa Dallas-Theus, Hudelaine
Deus, Oscar Flores, Cheryl Underwood, Marisela Rosales, Kobe Rives, Alli McClennen,
Eva Perez, Lina Rooney, Elizabeth Burleson, Cynthia Smith, Denise Powell, Katie Burnett,
Leslie Munson, Mary Rosecky, Jeffrey Wynnyk, Linda Wollesen, Margarita Franco,
Find the PEMAT-P at http://www.ahrq.gov/professionals/prevention-chronic-care/improve/self-mgmt/pemat/pemat-p.pdf