Webinar: Bryk, Gomez on Building Networked Improvement Communities in Education

What do other's feel is the role of having teacher candidates doing formal research including going through an IRB as part of this community?

Note: The responses from Carnegie are posted by me, but come from the presenters.

From Shawn Hayes 

Q: Given your response about collaborative being open, where is there room for traditional research (experimental designs and rigorous research)?  Is that no longer relevant?

This is an important and merits a detailed response—actually it could be a separate essay unto itself. So just a few quick response which hopefully will trigger further conversation.

First, let us make clear that improvement research is disciplined inquiry.  As has been argued by Don Berwick at the Institute for Healthcare Improvement and others, it is a science of improvement. It involves theory, it involves the explicit articulation of hypotheses undergirding both problem definition and any proposed solution path. It involves common measurement and design that provides a basis for empirical comparisons and contrasts. In short, it entails all of the provisions set out in the National Academy of Sciences report on Disciplined Inquiry in Education.

Second, one of the premises of improvement research is that there is much more accumulated research-based knowledge (i.e. tools, materials, ideas, practices, routines etc.) than actually gets used regularly in practice. Such "component development" has and should surely continue to be tested and refined through rigorous designs.  So there is complementarity here.

Third, embedded within the "small tests of change" characteristic of improvement research are occasions where RCTs can easily and effectively be implemented and they should be. For example, instructional innovations delivered over a digital platform to a large base of students will often provide a ready context. The overall commitment in improvement research, however is to practical design.  The improvement question (rather than an absolute commitment to one method) is in the first position and we need to constantly ask how most efficiently can we begin to generate evidence on what seems to be working (or not) as suggestive evidence for the next round of design/development work. 

Fourth and perhaps most important, the overall goal for the inquiry is different. The classic RCT answers a question of the average effect of some intervention typically in some not very random sample of sites.  Basically it tells us that an intervention can, on average, work.  Improvement research in contrast has a quality improvement focus. Its core question is what will it take, to achieve efficacy at scale for interventions for different kinds of students, advanced by different educators working under highly varied conditions. So its focus is on practical generalizability.

Questions from the Webinar

From Mary Simon:

Q: Is there a place for K-8 improvement systems in this network community?

Yes, there is a place for K-8 improvement systems. The core idea of network improvement communities is that there needs to be a group of people who are committed to a common change. This common change stems from identifying a common problem, developing measurable ways of monitoring the change in that problem, and developing common protocols of inquiry.

From Felicia Boothe:

Q: I appreciate the example of LINUX as a group of professionals that "get it".. How will the idea of sharing as an advantage be developed in a community that largely values territory?

It is true that many of our professional communities in education are territorial spaces. As we referred to in our paper, communities in general are governed by the rules and norms of membership. In the case of Network Improvement Communities, the goals of improvement impose specific demands on the rules and norms of member participation. These rules and norms of the improvement community are not just emergent hopes from the work of the community, but rather they are processes and routines that are measured and monitored by the community. In short, the improvement ethic of the community will have an impact on the territoriality.

From Geraldine Oberman:

Q: How does a new Improvement Community come into being?  How does one become affiliated with an existing Improvement Community?

Like other communities, there are a variety of ways in which an improvement community can come into being. However, the first requirement is that there be a common problem of practice around which the community may coalesce.

How one becomes affiliated with an existing community depends on the rules and norms of membership for the specific community. Network improvement communities clarify who is permitted to join, how someone might join and how a member participates. The emergence of these communities depends on "evangelizing leadership"--a small group who cultivates the initial "problem center/definition", (which of course become more detailed and specified over time), attract a few others to join in, and likely raise initiating funds. The community also needs a hub even during this early development stage.

This is the role that Carnegie is playing in forming Statway and Quantway. The latter involves a mix of technical expertise, keeper/interpreter of the organizing vision, and the judicator of the evolving rules. The hub functions as steward during the initiation period with the aim of transferring authority over time as the network moves toward becoming a self-guided institution.

From Dana Friedman:

Q: How do you address funding realities that prevent interventions from being sustained - hence no accumulated evidence.

It is difficult to ignore the funding realities that sometimes prevent accumulated evidence in our field. One way to consider this question is that network improvement communities produce intellectual property. This intellectual property derived from the work of community should remain the property of the community. With licensing agreements that appropriately incent individuals within the community and suitably benefit the community, a stream of funding could potentially carry on the work.