STELLA Model Indicates Strong School Leadership is Central to Closing Achievement Gap

The achievement gap—the difference in academic performance between low performing students and their high performing peers—is a hot topic in education. Closing the achievement gap is a main goal of the federal No Child Left Behind Act of 2001. Researchers continue to study variables that sustain the gap, including those normally perceived as outside the reach of the school, such as nutrition, health, parenting style, parent aspirations, childhood trauma, access to preschool, and many others. Curriculum and test developers, administrators, and policy analysts continue to draw on the available research in an effort to design and implement remedies.

Several years ago, Dr. Alan Gaynor, Associate Professor in School of Education at Boston University, decided to take a different look at the educational achievement gap. A self-described “natural systems thinker”, Gaynor wanted to focus on the K-12 school as a system.

“Karl Clauset, with whom I collaborated in the early 1980s, and I argued even then that schools are typically a reinforcing system. Kids who come in with high levels of readiness—they’ve been to preschool and their homes provide an enriched environment—are reinforced for high performance. Kids who start school with lower levels of readiness are reinforced for poor performance.”

As Gaynor points out, it’s fine to reinforce high academic performance. It’s not good, however, to reinforce poor performance. Unfortunately, on average, students who start school with high levels of readiness outperform their less ready peers throughout their school careers.

Taking a systems approach allowed Gaynor to consider the overall K-12 system and, more importantly, the underlying structures that explained the interactions between academic performance variables. “Most people have approached the achievement gap by looking at single variables or conducting multiple regression analyses,” says Gaynor. “Multiple regression employs an additive algorithm that enters data for each of the variables in the regression model, data that, themselves, are whole system data—the result of all the interactions that comprise the system, not bivariate interactions. I was interested in developing a theory about the interaction of pairs of variables in multi-variate causal loops that themselves interact to influence the behavior of the overall school system.”

Gaynor’s model includes 18 main variables and estimates the degree to which each variable impacts others. Simulations demonstrate how a typical school structure reinforces a student’s readiness level as he or she progresses from kindergarten through 12th grade.

More specifically, the model supports the theory that three self-reinforcing causal feedback loops are central to a sustained achievement gap. Those loops describe the relationship between:

• teacher expectations, student performance, and the quality of instruction, including the rigor of the content taught to students with different perceived ability and motivation
• overall student performance, school attractiveness, and the ability of the school to recruit high quality teachers
• school funding, teacher quality, and student performance

When high quality teachers expect all students to improve performance and understand how to support all levels of initial readiness, the achievement gap narrows. High quality teachers are attracted to well-funded schools with high performing students.

“The model and simulations led me to conclude that changing underlying system structures, specifically those three, critical causal loops, would lead to effective school reform,” says Gaynor. “Increasing teacher quality is essential and that relies on strong school leadership. Principals and superintendents impact teacher recruiting, curriculum development, community connections, and a host of other variables that lead to individual and school-wide academic performance improvements. Over time, strong school leadership builds teacher quality, increases community and parent interest, changes teacher expectations for students, and reduces the achievement gap.”

Having studied and taught school leadership for over forty years, Gaynor wasn’t very surprised by the behavior of the causal loops or the conclusion that strong school leadership is essential for closing the achievement gap. The model did help him identify next steps that will help him refine his conclusions.

Over time, strong school leadership builds teacher quality, increases community and parent interest, changes teacher expectations for students, and reduces the achievement gap

“The effect sizes—the magnitude of the relationship between variables—used in the model are estimates; I think they’re in the ballpark but I don’t know the precise sizes,” says Gaynor. “Sensitivity testing will help me understand how and if the model changes when effect sizes change.” Gaynor would welcome outside research that defines the effect sizes of variables included in his model but explains that organizational policies and ethical concerns make that kind of data collection difficult.

Gaynor also wants to explore the logical and functional boundaries of a school system. “Right now I’m looking at variables inside the school and seeing how they are impacted by school leadership,” he explains. “But how is a school system best defined? Can school leadership affect other important school readiness variables like parenting style, parent aspirations, and early education programs?” Gaynor anticipates conducting an expanded, systematic review of research and other literature that speaks to academic achievement variables and modeling educational systems with wider system boundaries.

STELLA makes it easy to translate qualitative understandings into formal policy models

While Gaynor feels naturally inclined toward Systems Thinking, he’d like to build his system dynamics skill set further. “My work in actual system dynamics has been on and off,” says Gaynor. “I want to learn more about important technical aspects of the methodology like sensitivity analysis and delays.”

Despite gaps in his system dynamics knowledge base, Gaynor points out that software has made it possible for him to take a systems approach to his work. “I go way back with STELLA,” he says. “STELLA makes it easy to translate qualitative understandings into formal policy models. It’s been invaluable for seeing and understanding system substructures.”

Can Systems Thinking Improve Development Efforts in Sub-Saharan Africa? Finding the answer with help from STELLA

Despite the trillions of dollars in aid that has flowed into Sub-Saharan Africa (SSA) from developed nations, the region continues to battle starvation, disease, low literacy rates, high birth rates, and a host of other intractable problems. In fact, between 1981 and 2001, extreme poverty in SSA rose 5% (UNIDO, 2004). Countries continue to struggle with high rates of HIV and AIDS. And desertification and other forms of environmental degradation are making it harder to survive in local economies that are based almost entirely on agriculture.

The coincidence of international aid and degradation of economic conditions in Africa was apparent to Viviane A. Amelewonou-Thalmensy. “I’m from Togo, West Africa but I didn’t grow up there,” says Viviane. “My father worked for the UN and we often vacationed in Togo and visited relatives. I could see that things weren’t improving.”

Educated and experienced in risk management in the banking sector, Viviane decided to pursue a PhD in development at Skema Business School in Lille, France, and to investigate the efficacy of development programs in SSA. While she was clear on the issue she wanted to address, she didn’t think about taking a system dynamics approach until meeting with her advisor, Peter Heffron, Adjunct Professor at Skema. “I had never taken a course in system dynamics, but Peter explained that it was an approach that would help me deal with the complexities of development issues.”

Interacting stocks and flows of people, natural and non-renewable resources, pollution, and other factors over time seemed like the best – even ‘only’ – way to analyze complex issues

Heffron knew first-hand that Systems Thinking is both very useful to development teams and rarely used to tackle complex development issues. “I learned about system dynamics through Limits to Growth and Beyond the Limits by Dennis Meadows, Dana Meadows and Jorgen Randers,” says Heffron. “The World 3 dynamic model was, to me, revolutionary and common-sensical at the same time. Interacting stocks and flows of people, natural and non-renewable resources, pollution, and other factors over time seemed like the best – even ‘only’ – way to analyze complex issues.”

When he joined CARE in 1979, Heffron began to study system dynamics on his own. He learned about STELLA, isee systems’ System Thinking software and understood how it could help CARE and other NGO representatives, government planners, academics and others involved in development improve projects in Honduras. "isee systems provided a team of consultants to CARE-Honduras and partner organizations for a week-long orientation to system dynamics and Systems Thinking and to facilitate a participatory, transparent strategic planning exercise, including looking into possible future scenarios based on different assumptions,” says Heffron.

Concerned about the future of development in SSA and hopeful that increased use of a system dynamics approach would improve outcomes, Viviane began to develop her PhD thesis Policy Formulation in Sub-Saharan Africa in the Context of Sustainable Development Programs: What Could a Systems Approach Contribute? Her first step was to learn more about development. She reviewed literature and case studies with key questions in mind. What are the results of SSA development policies and programs so far? What are the underlying causes of development policy/programs relative ineffectiveness? If SSA was to reach an ideal more-sustainable future, what would that future look like?

Next, she had to learn about system dynamics. Heffron offered his support and, after she made a presentation at Skema, others became interested. A one week intensive course in modeling and simulation was offered and she took it. “The course work was very tough,” says Viviane. “System dynamics required me to think in new ways about factors that influence development efforts and links between those factors. Making a causal loop diagram was simple. Thinking about how to apply real data to the loop was quite hard.”

The problems that development tackles are completely intertwined... That interconnectedness implies tremendous complexity

Developing a model would require software, another resource that wasn’t available through the university. With the help of her supervisor, Viviane contacted isee systems, explained her thesis and intention to apply system dynamics to the issue of development effectiveness, and received a license for STELLA. With those pieces in place, Viviane began to analyze development issues using a system dynamics approach.

The problems that development tackles are completely intertwined – diet and nutrition impact education, education impacts technological development, ecological conditions impact ability to grow enough food, lack of food impacts diet and nutrition, etc. That interconnectedness implies tremendous complexity. It also requires data that just doesn’t exist. Viviane decided to focus on two variables that were at the root of most development challenges; population growth and the sustainability of renewable natural resources.

The number of countries in the SSA region and many differences between them also complicated modeling efforts. Viviane further focused her study by selecting three countries, Botswana, Ghana, and Ethiopia that are diverse in terms of regional localization, current level of development, and access to resources that support additional development. Botswana is rich in diamonds and is classified as middle income country. Ghana, known as “the good student”, has an educated population, is doing well in terms of economic development, and politically progressive and democratic in relation to its neighbors. Ethiopia is a low income country and has many natural resources though it also has a very large and growing population.

Narrowing her focus to essential variables helped simplify model development but Viviane was still having trouble getting simulations to work. She used Braat’s Systems Ecological Model for Sustainable Development Analysis and the Limits to Growth 30-Year Update model published by Dennis Meadows as reference foundations.

Two sub-models, one focused on population growth rate and a second that forecasts resource consumption and regeneration, show how those factors relate on a country, village, and family level. “The more people in a given group, the more food that group has to produce,” says Viviane. “If resources don’t increase as population increases, the system loses equilibrium.”

Disequilibrium in that straightforward relationship comes with many consequences. When populations outgrow available resources, they are forced to move to other areas that offer necessities like water, food, and land. In the worst, but not uncommon, situations, people without adequate food and water succumb to disease and/or die of starvation. The consequences of the population/renewable resources relationship are pronounced in SSA where most economies are based on agricultural and fishing.

“Together, the population and renewable resources models show that infinite development is not possible in the countries studies,” says Viviane. “Everything is limited when populations increase beyond the point that renewable resources can support them.”

The process of creating models brings everyone to the table. Everyone’s perception and reality goes into the model and groups can test what-if scenarios to find and agree on policies and programs that will have the best results

If development professionals applied Systems Thinking models like Viviane’s to local problems, would they achieve better results? “As my modeling experience shows, system dynamics allows development professionals to see the overall complexity in a system and break issues down to smaller components,” says Viviane. “Models can be constantly adjusted to add new variables or reflect differences between regions, countries, or communities. The process of creating models brings everyone to the table. Everyone’s perception and reality goes into the model and groups can test what-if scenarios to find and agree on policies and programs that will have the best results. After programs are implemented, models can be run with recent data to see if reality changed over time. Were projects successful? Did they have the desired results?”

Due to the complexity of the issues development efforts try to address, programs often occur in combination. System dynamics helps aid agencies see how programs are working in cooperation or cross-purposes. Teams are able to identify high leverage points in order to optimize efforts as well as to allow long-lasting positive change. And models can be constantly revised to test new ideas.

Given all those benefits, one would imagine that Viviane would find great enthusiasm for system dynamics in the development community. Her survey of 47 development professionals, some familiar with system dynamics and some with no experience or knowledge, revealed that a majority of respondents (54%) were indeed “very likely” to recommend the approach to fellow policy makers. On the other hand, 15% were “very unlikely” to take or recommend a system dynamics approach because they consider it to be potentially more difficult than beneficial.

Because [STELLA] presents information visually, it was very easy to present my findings. Colleagues could see what questions I was asking and how I had come to my conclusions

Viviane points out that system dynamics does require a new way of thinking and there are certainly new concepts to learn and practice; but that STELLA facilitates the process. “STELLA facilitates breaking models down into layers. As I created the causal loop diagrams and stocks and flows that represented the basic systems and relationships I was studying, STELLA generated the equations and helped me run the simulations. Because it presents information visually, it was very easy to present my findings. Colleagues could see what questions I was asking and how I had come to my conclusions. They could ask questions and use the model to find answers.”

Work on her thesis left Viviane feeling realistic about the future of SSA but hopeful that system dynamics can make a difference. “I did reach an ah-ha conclusion,” she says. “Population growth and renewable resource constraints will always limit the amount and pace of development in Sub-Saharan Africa, but if we apply system dynamics to policy making and program development, we can postpone the worst consequences of that dynamic.”

“By providing STELLA and facilitating Viviane’s access to online training, isee systems contributed to her unplanned discovery, and sharing, of what is really happening in SSA,” says Heffron. “Her work demonstrates the need for planners, policy makers, and managers at all levels to use Systems Thinking principles, methods and tools to get a better handle on the interactions that are causing SSA to spiral into unsustainability.”

2012 Barry Richmond Scholarship Award

The Barry Richmond Scholarship Award was established in 2007 by isee systems to honor and continue the legacy of its founder, Barry Richmond. Barry was devoted to helping others become better systems citizens and it was his mission to make systems thinking and system dynamics accessible to people of all ages, and in all fields and professions. The award is presented annually at the System Dynamics Society Conference to an individual whose work demonstrates a desire to expand the field or to apply it to a current social issue.

It was a great pleasure to present this year’s award to Mauricio Gomez Diaz, a graduate student in MIT’s Engineering Systems Division. Mauricio’s work includes a wonderful paper sponsored by the National Institute of Health titled Unintended Effects of Changes in NIH Appropriations: Challenges for Biomedical Research Workforce Development. The paper and his model describe how a sudden rise in NIH research funding led to an overinvestment in facilities and the recruitment of PhD students by Universities, which in turn led to a more competitive grant proposal environment. More time spent writing grants means less time doing research, and ultimately a less productive workforce.

Mauricio’s work identifying the feedback loops associated with public funding and a research workforce could have implications for many other public agencies. Please join us in congratulating Mauricio!