We typically use the Flesch-Kincaid (F-K) method to test the grade levels of Plain Language writing for Medicaid/Medicare compliance. Flesch-Kincaid scores can easily be obtained in programs like MS Word and are accepted by all States with recommended testing methods. In this post, we’ll explain how and why we use the Flesch-Kincaid testing method. We’ll also explain how no testing method is ideal and why it matters to think just beyond complying with the required readability score.

Flesch-Kincaid Method

Before you run your F-K test, it’s important to take out certain information that interferes with the score. You should always replace information like dates or names with one syllable words or you will get inaccurate scores. Flesch-Kincaid is based on a formula that weighs heavy on sentence length and number of syllables per word. The formula for calculating F-K readability is:

This formula can be manipulated in several ways:

• Writing shorter sentences will decrease the grade level
• Writing more simple words with less syllables will decrease the grade level
• Changing distribution of number of total words versus complex words can reduce the grade level

When you look critically at readability scoring tools like Flesch-Kincaid and others, “readability” is not always a good prediction of “understand-ability”. With Plain Language, the object is to provide communication that the audience can understand the first time. A highly verbose piece can easily contain a few complex words that would make a reader stop reading. While the readability score may be at grade level, it may not be as well understood the first time by many readers for which the communication is intended.

Differentiating Factor: The Plain Language Writer

When we are writing Plain Language materials, we look beyond just obtaining a score. Plain Language Writing is not the art of getting good grade scores. Plain Language Writing is all about making communication clear. An experienced Plain Language Writer can typically work with the original copy and write the grade level down to an acceptable level. We use strategies to not only write clearly, but also may reorganize the information as well to make the flow more logical and easier to follow. Plain Language takes input from educational theories that are designed to improve outcomes. This takes understanding of the target audience, their barriers to learning and how the audience is made up (demographics, behaviors, culture, attitude, literacy levels, language, socioeconomic status and access to services).

When Plain Language writing gets reviewed by compliance, there are typically a few words that come back with changes or questions. The impact of these single words may not increase the grade level. However, we do often find that the introduction of complex words impacts “understand-ability.” When looking at better healthcare outcomes, choosing the right words despite readability scores can be a challenge. This challenge highlights the main issue with readability scores; it doesn’t necessarily promote the best writing and it really depends on the expertise and experience of the Plain Language Writer.

Choosing Flesch-Kincaid over Other Methods

Flesch-Kincaid may not be the best and only testing method out there. Currently, Medicare and Medicaid also do not stipulate a standard testing method.  Some writers feel that it gives a false positive and a test score that is actually a few grades below the actual test score. However, with the right Plain Language strategies, there is no reason that F-K readability cannot achieve high quality results. But there are other methods to consider, such as:

SMOG in Healthcare

SMOG is considered by some in the Healthcare field to be the gold standard. It’s been the recommended readability test by the National Cancer Institute.  A study on SMOG testing versus F-K testing on healthcare websites showed that SMOG scores were typically more critical and score lower than F-K. But SMOG has practical limitations. First, the testing method is not integrated in MS Word although there are automatic calculators out there. Unlike F-K testing which involves testing the entire document, SMOG only uses a samples 30 sentences. This makes testing short letters with SMOG (under 30 sentences) statistically nonviable. Plus, since SMOG only samples the first, middle and last 10 sentences of the copy, it doesn’t guarantee consistency. SMOG typically tests 2 grade levels above Flesch-Kincaid, making it tougher to reach state requirements below 6th grade. This should not be a discouragement because of grade level requirements, but there are easier ways to satisfy State requirements using Flesch-Kincaid while still achieving the health outcomes using evidence-based writing techniques.

Fry Readability Graph

Fry Readability Graph for Spanish

Readability Tests were designed for English writing. However, we also work with a lot of Spanish materials in our specialization of healthcare translations and the Fry Readability Graph has been validated in both English and Spanish. Fry has been tested to correlate highly with other tests as well (Taken from Table of Commonly Used Readability Tests). Fry also tests for number of syllables and words per sentence, but does it differently than Flesch-Kincaid. Instead, it tests the number of words and syllables per 100 words. What’s different about this formula is that you can plot different parts of 100-word text samples on an x-y graph. If you want to analyze how consistent your copy is in aggregates of 100 words, Fry is a nice way to manually achieve an in-depth analysis.

However, we never recommend converting an English piece written at an 11th grade level directly into a 6th or 4th grade level Spanish Translation. Translation is the skill of accurately conveying the original English copy. Plus, it would be a nightmare for compliance to approve a rewritten piece in Spanish. Therefore, it is always our recommendation to rewrite the English copy using professional Plain Language Writers specialized in Healthcare.

The Role of Readability Tests in Healthcare

Readability Tests are no guarantee that the copy is 100% understood by low literacy populations. Therefore, it’s important to never see Readability Scores as an outcome, but rather a guideline and a tactic to satisfy regulatory requirements. The only way to truly test for understand-ability is to do field testing of your communications.

Organizations can start small – get a few members on the phone and ask them questions about the piece. Ask for their feedback and insight.  A little testing is better than no testing at all.  For a thorough testing, 9 participants is the gold standard. A questionnaire is developed and is used with all participants who may fit a certain criteria. We do this kind of testing and the results are always invaluable.

What matters in Plain Language Writing is that the communication is written in a way that promotes desired outcomes. Metrics like drug adherence and other behavioral outcomes should really be the driving force behind any Plain Language initiative.  Once these metrics are taken into account, it becomes also clear that Plain Language is a strategy that anyone can benefit from, not just the low literacy population. By working with Plain Language Writing experts in Healthcare, you can ensure that your communications drive results instead of just satisfying regulatory needs.