Debunking ESSA Myths and Misconceptions

Adopting a new school-wide or district-wide curriculum is no small task. Stakeholders need to make sure that any new educational resources they bring in match the needs of their students and that funding is well spent on quality, evidence-based materials, which requires a lot of research.

When researching new curriculums many educators rely on resources like the evidence-level standards that were created under the Every Student Succeeds Act, better known as ESSA, to determine which programs will benefit in their schools and with their students.

But even the savviest educators and decision makers can struggle to understand the complexity of ESSA and its associated evidence levels, which can lead to misunderstandings. And those misunderstandings can lead to challenges when choosing the right curriculum.

In this blog post, we’ll discuss what ESSA is and clear up three of the most common misconceptions surrounding its evidence levels so that those misunderstandings don't pose a barrier when it comes to making informed decisions on future educational solutions.

What Is ESSA?

The Every Student Succeeds Act is a federal education law enacted in 2015. The law encourages state and district decision makers to choose evidence-based instruction materials that have been rigorously studied and proven to improve student outcomes, especially those purchased with federal funding.

But not every educator is a research expert. So how would they know if the study that produced the evidence on the curriculum they are looking to purchase is viable?

In order to provide that guidance, the U.S. Department of Education (DOE), under the ESSA legislation, created the ESSA four Levels of Evidence to help educators evaluate the quality of those studies and to give them confidence that they can trust the results. The four Levels of Evidence are Level 1 (Strong), Level 2 (Moderate), Level 3 (Promising), and Level 4 (Demonstrates a Rationale).

How an educational solution is assigned an ESSA level is determined by what kind of studies were used when researching its effects on student performance. The studies are reviewed and then categorized into the following levels:

• Level 1: Experimental Study, or a randomized controlled trial, conducted in at least two sites with 350 students where groups are randomly assigned to a treatment or control condition.
• Level 2: Quasi-Experimental Study conducted in at least two sites with 350 students where treatment and control groups are well-matched on baseline characteristics (i.e., demographics, achievement) but are not randomly assigned to a treatment condition.
• Level 3: Correlational Study includes a treatment or control group that is not randomly assigned and not well-matched on important baseline characteristics.
• Level 4: Well-Defined Logic Model & Study Underway that indicates how a program is likely to improve student outcomes. A study that meets the Promising Level or higher must be underway.

The ESSA Levels of Evidence are based on how a solution is studied by looking at five key factors: study design, results, findings from related studies, and sample size and number of sites.

Educators rely on these ESSA evidence levels to help them determine which new curriculum they will adopt. So, it’s critical that they understand what the levels mean and how they’re determined. But there may be some misunderstandings about the ESSA levels that can make it difficult to make an informed decision.

Misconception #1: ESSA Level 1 is always better than ESSA Levels 2 and 3

When we see that an educational solution is ranked in the ESSA Level 1 category, it’s easy to think that it means better. But you might be surprised to learn that is not always the case.

The experimental studies in Level 1 tend to prevail as the gold standard of research. But the other levels should not be discounted.

In fact, the quasi-experimental studies in Level 2 are identical to experimental studies in Level 1 in terms of sample size requirements and setting. In fact, Levels 1, 2, and 3 all require statistically significant, positive study results. And multiple Levels 2-4 can provide a greater breadth of insight than a single Level 1 study.

If an educational solution has a Level 1, 2, or 3 ESSA ranking at all, it means that there is evidence proving its effectiveness, which is what educators are looking for in a new solution. And even Level 4 solutions are based on a well-defined logic model built on rigorous research.

When evaluating potential educational solutions using ESSA, it’s important to remember that the evidence levels were designed to categorize the research conducted, but do not necessarily indicate how well a curriculum will do in a variety of settings. Determining those factors will require further research.

Misconception #2: The level of the study is the only criteria when evaluating a learning solution

While ESSA levels provide important information, they should not be the only criteria when evaluating the evidence of a learning solution.

A single study rated Level 1 with a sample of study participants that DO NOT reflect the population and setting for which the program is being considered, can be less relevant than several studies at various levels that DO reflect the characteristics of the district/school for which the program is intended.

Regardless of ESSA evidence level, a larger body of research across multiple sites that match the setting and population of the organization where the educational solution under review will be applied, can be stronger than a single Level 1 study.

This is why the U.S. DOE recommends that educators and administrators focus on the studies that reflect their local school setting and student population by looking at studies from multiple student populations and settings. Here are some suggestions on what to look out for:

• Be sure the evidence you find is the result of research that was conducted in a similar setting and/or population to your district. The evidence supporting a learning solution should be aligned to your district’s setting, capacities (e.g., available technology, financial means, etc.), and student demographics, such as students with disabilities, English learners, etc.
• The research you find should show student success across multiple student populations and settings. Multiple studies across various levels is preferable to a single Level 1 study showing student success in one student population and one setting.
• Be sure the research studies that have been conducted for a given learning solution are relevant and not outdated. When the research was conducted is also an important factor.

Misconception #3: All third-party reviewers are endorsed and aligned with the standards put forth by the Department of Education

Educators and administrators should also look to DOE-endorsed third-party reviewers, like the What Works Clearinghouse (WWC), as they navigate existing research and make sure programs they are reviewing include research that is current and relevant for their district.

The WWC incorporates ESSA evidence levels into its rigorous standards and is able to offer evidence of effectiveness on a wide range of programs. WWC reviews include the following: specific educational solutions and individual studies; a summary of the settings in which research was conducted; and details about the demographics of participating districts, schools, and/or students when available.

Understanding how ESSA levels work and knowing what to look for when researching new learning solutions will give educators more confidence that, when it comes time to adopt a new curriculum, they chose the right resource to fit their district’s needs and student population.

Savvas Learning Company is dedicated to conducting research that is aligned with WWC standards and uses a variety of methods to ensure its studies produce high-quality data that conscientiously reflects the unique experiences of students and teachers across the nation.

Explore the ESSA Levels of Evidence for Savvas learning solutions.