Learning to Improve Glossary
Overview
This glossary organizes a selection of key terms used in the book, Learning to Improve: How America’s Schools Can Get Better at Getting Better, that have formal meaning. In many cases our definitions align with those established in prior writing on improvement research and networked communities. In some cases they have been modified slightly for the education context. In a few cases they are original to this work.
The terms are grouped together according to the following topics:
About Networked Improvement Communities
Colleagueship of Expertise
A community of academic, technical and clinical experts deliberately assembled to address a specific improvement problem. All involved are improvers seeking to generate strong evidence about how to achieve better outcomes more reliably.
Networked Improvement Communities
An intentionally designed social organization with a distinctive problem-solving focus; roles, responsibilities and norms for membership; and the maintenance of narratives that detail what they are about and why it is important to affiliate with them. A NIC is marked by four essential characteristics. It is:
- focused on a well specified common aim;
- guided by a deep understanding of the problem, the system that produces it, and a shared working theory to improve it,
- disciplined by the methods of improvement research to develop, test and refine interventions, and
- organized to accelerate their diffusion out into the field, and effective integration into varied educational contexts.
Network Hubs
A core group formed either as single organization or distributed across network members that carry out critical functions necessary for the support and effective operations of a networked improvement community. These functions include, but are not limited to: improvement science expertise, analytics, knowledge management, convenings, communications, and technological support.
Network Initiation Team
A team that accepts responsibility for the formation of a networked improvement community. It leads a set of processes that articulates of the problem to be solved, analyzes the system that produces current undesirable outcomes, and develops the aim statement and an initial working theory of practice improvement. The initiation team also takes the lead in securing the necessary supports for the network (both political as well as material), recruiting initial members into the community, and engaging the academic and technical expertise relevant to the specific problem to be solved.
NIC as a Scientific Community
It is organized around a shared theory and shared measures for its aim and primary drivers. Participants engage in discipline inquiries using established inquiry methods such as PDSA cycles. Promising results are subject to replication across the network to warrant claims that changes are improvements.
NIC Charter
A document that provides sustaining guidance to the distributed efforts of NIC members. It is composed of the Network’s Aim, Causal System Analysis, (often consisting of both fishbone diagrams and system improvement map), and the Working Theory of Improvement (typically represented in a driver diagram).
About Improvement Science and Quality Improvement
Continuous Improvement
Improvement research that involves multiple iterative cycles of activity over extended time periods.
Improvement Research
These are particular acts of inquiry, or projects, that aim for quality improvement.
Improvement Science
The methodology that disciplines inquiries to improve practice. Undergirding it is an epistemology of what we need to know to improve practice and how we may come to know it.
Quality Improvement
An effort to increase the capacity of an organization to produce successful outcomes reliably for different sub-groups of students, being educated by different teachers and in varied organizational contexts.
Solutionitis
The tendency to jump quickly on a solution before fully understanding the actual problem to be solved. It results in incomplete analysis of the problem to be addressed and fuller consideration of potential problem-solving alternatives. It is silo-ed reasoning—seeing complex matters through a narrow angle lens—that can lure leaders into unproductive strategies.
About Standard Work and Processes
High-Leverage Process
A process that has the following properties: 1) it consumes substantial resources, especially teacher or student time; 2) its execution and outcomes vary considerably; and 3) there are reasons to believe that changes to it might improve resource efficiency and effectiveness.
Macro-Process
A process for which the execution typically entails a sequence of more discrete micro-processes.
Micro-Process
An elemental activity or segment of work taken to achieve a particular end.
Standard Work
Regularly occurring processes that are amenable to formulation as best practice routines within a networked improvement community. The purpose of these routines is to assist educators in carry out their work by reducing the cognitive load associated with the performance of complex tasks. Such processes reduce undesirable variation in performance and free educators to better focus their attention thereby advancing quality outcomes more reliably. High leverage processes are attractive candidates for standard work.
About Systems
Causal System Analysis
An analysis that directs attention to the question, “Why do we get the outcomes that we currently do?” In working through this analysis, participants develop a shared understanding of the specific problem(s) they are actually trying to solve. The process also provides a first test as to whether a team seeking to initiate a NIC can engage productively together as a focused improvement community.
Fishbone Diagram
A tool that visually represents a group’s causal systems analysis (sometimes known as a cause-and-effect diagram or an Ishikawa diagram).
System
An organization characterized by a set of interactions among the people who work there, the tools and materials they have at their disposal, and the processes through which these people and resources join together to accomplish its work.
System Improvement Map
An analytic tool that represents what we learn through the causal system analysis about the different organizational levels (e.g., classrooms, schools and districts) and key organizational sub-systems (e.g., human resources, finance, instruction) relevant to solving the identified problem.
About Driver Diagrams and Working Theories of Practice Improvement
Change Idea
An alteration to a system or process that is to be tested through a PDSA cycle to examine its efficacy in improving some driver(s) in working theory of improvement.
Concept Framework
It provides conceptual detail and relevant research findings that form design principles for key drivers and change ideas. It also provide a conceptual basis for the development of practical measures. (Also may be referred to as design principles.)
Driver Diagram
A tool that visually represents a group’s working theory of practice improvement. The Driver Diagram creates a common language and coordinates the effort among the many different individuals joined together in solving a shared problem.
Improvement Aim
A goal for an improvement effort that answers the question What are we trying to accomplish? Improvement aims should clearly specify how much, for whom, and by when? They sit at the far left end of a Driver Diagram.
Pareto Principle
A principle anchored in a long history of organizational studies that 80% of the variability in organizational performance is often associated with only 20% of the possible causes. It aids in the selection of primary, and when needed, secondary drivers as well.
Primary Drivers
Representation of a communities’ hypothesis about the main areas of influence necessary to advance the improvement aim.
Secondary Drivers
A small set of system components that are hypothesized to activate each primary driver.
Working Theory of Practice Improvement
A small inter-related set of hypotheses about key drivers necessary for achieving an improvement aim and specific changes associated with each driver. It requires a creative blending of observations arising from the causal system analysis with relevant research that bears on this problem together with wise judgments from expert educators.
About Doing Improvement Research
Adaptive Integration
Learning how to integrate a change package into a new setting, adapting it as necessary to local conditions, while preserving fidelity to its undergirding principles and assuring local efficacy.
PDSAs (Plan-Do-Study-Act Cycles)
A pragmatic scientific method for iterative testing of changes in complex systems. Each cycle is essentially a mini-experiment where observed outcomes are compared to predictions and discrepancies between the two become a major source of learning.
Process Map
A tool for visualizing the steps in a process that can assist an improvement team in identifying gaps, strengths, and opportunities for improvement.
Run Chart
A graphical display of a some measured characteristic over time. Data from PDSA are often displayed in run charts.
Solution Systems
An intervention that consists of multiple inter-related components that must mesh well together. Formally, such interventions have a systems character. For positive effects to occur reliably coordinated improvements need to occur across all of the drivers that compose the solution system. A material weakness in any one driver can undermine the efficacy of the overall solution.
About Measurement
Balancing Measures
These measures help improvers to keep an eye on the other parts of the system that are not currently the target of improvement, but nevertheless may be affected by the changes being pursued.
Lagging Outcome Measures
Measures that are only available well after an intervention has been initiated.
Leading Outcome Measures
Measures that predict the ultimate outcomes of interest but are available on a more immediate basis.
Measurement for Accountability
Broad, general measures that aim to sort individual or organizational units into performance categories. This use introduces increased formality into the measurement process and places primacy on the reliability of individual scores. Often used as measures for improvement aims.
Measurement for Improvement
Measures that directly link to the specific drivers and work processes that are the object of change. They provide evidence for testing changes and examining hypothesized causal connections in the working theory of improvement.
Measurement for Research
Detailed measures developed by researchers to represent particular theoretical constructs. Used in academic research to test relational propositions among key constructs that form a theory. Often useful as a basis for developing practical measures.
Outcome Measure
Measures that operationalize the aim statement in the driver diagram. These data provide a way of assessing whether progress is being made on the specific problem to be solved. Accountability measures are often used here.
Practical Measurement
Data to inform improvement that is embedded in regular work. Since the intent is to inform continuous improvement, practical measures are collected frequently to assess whether positive changes are in fact occurring. Since the focus is on specific populations and contexts, the measures are framed in a language natural and comprehensible to those asked to answer them.
Primary Driver Measures
Measures associated with primary drivers. Since these drivers are intermediate outcomes in the working theory of improvement (i.e., intermediaries between process changes and leading and lagging outcomes) they play a key role in the testing of a working theory of improvement.
Process Measures
Measures that feed back valuable information about how specific processes being tested are performing under different conditions.
About Evidence for Improvement
A, B, C Level Learning
Three inter-related levels of learning, developed by Douglas Engelbart, that together form a schema for individual and organizational learning. “Level-A” learning represents the knowledge acquired by front line workers as they engage in their practice. “Level-B” learning occurs across individuals within an organization. “Level-C” learning is orchestrated by a network hub and coupled with appropriate technologies to support rapid communications across distributed sites. This confluence is what enables the network to accelerate how it learns to improve.
Evidence-Based Practice
Tools, materials, or sets of routines, typically grounded in theoretical principles, that has been subject to rigorous empirical study. Their use is warranted by results from a rigorous field trial that demonstrated that the intervention can work because it has somewhere.
Know-How
The detailed practical knowledge necessary to get good ideas actually to work in classrooms, schools, and districts.
Practice-Based Evidence
Evidence that grows from practice and can be used to improve it. This evidence, emerging from improvement research, demonstrates that some process, tool, or modified staff roles and relationships can be made to work effectively under a variety of conditions and that quality outcomes will more reliably ensue. It is the evidence of know how.