Carnegie at the 2019 AERA Annual Meeting

March 19, 2019

Carnegie staff will present, chair, or serve as discussants for sessions during the 2019 American Educational Research Association (AERA) Annual Meeting, April 5-9, 2019, in Toronto, Canada.

Coordinated Session

Practical Measurement for Improvement Science: Principles and Applications

Sat. April 6, 10:25–11:55 a.m., Fairmont Royal York Hotel, Mezzanine Level, Territories

Abstract: What is improvement science, and how are practitioners of this growing discipline designing and evaluating their measurement tools? Improvement science is a method that users employ to identify, understand, and solve specific problems of practice. The method requires measurements that support rapid cycles of disciplined inquiry, to detect the effects of prototyped changes. Whereas large-scale tests emphasize standardization and precision, measures for improvement science must typically be more convenient for local practitioners to develop, administer, score, interpret, and use.

Paper 1 introduces principles for practical measurement for improvement. Papers 2, 3, and 4 present examples of tools, for improving writing, mathematics, and on-time graduation, respectively. The chair, presenters, and the discussant will also identify productive tensions between modern testing standards and practical measurement for improvement. Are traditional metrics for reliability and precision relevant for measures whose purpose is to detect the effects of a prototyped change? And how should these practical measurement tools quantify growth?

This symposium integrates the ideas and memberships of two organizations deeply committed to improving the use of measurement tools in education: The National Council on Measurement in Education and Carnegie Foundation for the Advancement of Teaching.

Chair
  • Andrew Ho, Harvard University
Papers

Practical Measurement for Improvement: An Introduction
Authors: Jon Norman, Carnegie Foundation for the Advancement of Teaching; Sola Takahashi, WestEd

Developing Practical Measures to Inform Instructional Improvement Initiatives in Mathematics
Authors: Paul A. Cobb, Vanderbilt University; Kara J. Jackson, University of Washington–Seattle; Marsha M. Ing, University of California–Riverside

The Using Sources Tool: A Practical Measure for Identifying Next Instructional Steps
Author: Linda D. Friedrich, National Writing Project

Using the Freshman On-Track Indicator to Improve High Schools in Chicago
Author: Elaine M. Allensworth, University of Chicago

Discussant
  • Anthony S. Bryk, Carnegie Foundation for the Advancement of Teaching

Invited Speaker Session: Vice-Presidential Address and Panel

Developments in Education Policy Research: Building a Scholarship of Improvement

Sat. April 6, 10:25–11:55 a.m., Metro Toronto Convention Centre, 200 Level, Room 203B

Abstract: In her address as outgoing Vice President of Division L, Lora Cohen-Vogel will focus her remarks on the growing momentum for improvement-focused educational research. She will articulate the warrants or claims forwarded in both the research and funding spaces in support of the application of these approaches in education. In so doing, the address will set the stage for a discussion with panelists around an emerging “common sense” about what should be done to improve the quality of educational research.

Presenters
  • Lora A. Cohen-Vogel, University of North Carolina–Chapel Hill
  • Donald J. Peurach, University of Michigan and Carnegie Foundation for the Advancement of Teaching
  • Jennifer Lin Russell, University of Pittsburgh and Carnegie Foundation for the Advancement of Teaching
  • Ruth Lopez Turley, Rice University
  • Casey Megan White, Bill & Melinda Gates Foundation

Structured Poster Session

The Scholarship of Improvement: Building Community Around an Emerging Tradition of Practice-Focused Research

Sun. April 7, 8:00–9:30 a.m., Metro Toronto Convention Centre, 800 Level, Room 801A

Abstract: The objective of this structured poster session is to build community around the scholarship of improvement as an emerging tradition of practice-focused research. By “scholarship of improvement,” we mean research focused on understanding and advancing continuous learning and improvement with the aim of strengthening the systems, resources, and practices that support students’ success in schools. The session is structured to showcase 11 examples of the scholarship of improvement. The posters detail new methods and measurement approaches supporting collaborative, continuous improvement; research-practice partnerships aimed at improving instruction, academic learning, and socio-emotional learning; the application of improvement methods in districts, turnaround zones, and state education agencies; and shifts in federal policy establishing the context for all of the preceding.

Chairs
  • Donald J. Peurach, University of Michigan and Carnegie Foundation for the Advancement of Teaching
  • Jennifer Lin Russell, University of Pittsburgh and Carnegie Foundation for the Advancement of Teaching
Papers

Developing a System of Practical Measures, Routines, and Representations to Inform and Enhance Instructional Improvement Initiatives
Authors: Kara J. Jackson, University of Washington–Seattle; Paul A. Cobb, Vanderbilt University; Erin Craig Henrick, Vanderbilt University; Thomas M. Smith, University of California–Riverside; June Ahn, University of California–Irvine; Marsha M. Ing, University of California–Riverside; Hannah Nieman, University of Washington; Nicholas M. Kochmanski, Vanderbilt University–Peabody College; Fabio Campos, New York University; Starlie Chinen, University of Washington–Seattle; Daniela Digiacomo, University of California–Riverside; Maria Hays, University of Washington–Seattle; Emily C. Kern, Vanderbilt University; Meaghan Beth McMurran, University of California–Riverside

Evidence for Improvement: An Integrated Approach
Authors: David Gilbert Sherer, Carnegie Foundation for the Advancement of Teaching; Jon Norman, Carnegie Foundation for the Advancement of Teaching; Donald J. Peurach, University of Michigan and Carnegie Foundation for the Advancement of Teaching; Anthony S. Bryk, Carnegie Foundation for the Advancement of Teaching; Ash Vasudeva, Carnegie Foundation for the Advancement of Teaching

Applying Realist Methodology to the Study of Collaborative Capacity in Networked Improvement Initiatives
Author: David H. Eddy-Spicer, University of Virginia

Randomized Trials and Improvement Modalities: Understanding the Federal Role in Education Research
Author: Lora A. Cohen-Vogel, University of North Carolina–Chapel Hill

Continuous Improvement “On the Ground”: Lessons From Low-Performing Schools
Authors: Amanda L. Datnow, University of California–San Diego; Hayley Ryan Weddle, University of California–San Diego; Marie Lockton, University of California–San Diego

Where’s the Playbook? Common Curriculum and High School Turnaround
Authors: Adrian Mohamed Larbi-Cherif, George Washington University; Sarah Winchell Lenhoff, Wayne State University; Joshua L. Glazer, The George Washington University

Scaling Systemic Personalization Through Continuous Improvement: A Model for High School Reform
Authors: Stacey A. Rutledge, Florida State University; Elizabeth Gilliam, Florida State University; Brittany Closson-Pitts, Florida State University

Get Started—Get Better: Using Improvement Cycles to Achieve State Systemic Improvement
Author: Caryn Sabourin Ward, University of North Carolina–Chapel Hill

A Research-Practice Partnership to Transform Mathematics
Authors: Julie L. Booth, Temple University; Suzanne Donovan, SERP Institute

Improvement Science Applied to Teaching: Cases From a Randomized Trial of Fractions Lesson Study
Author: Catherine C. Lewis, Mills College

Improvement Science Applied to Improving College Courses: Learning From 17 Years of Work
Authors: James Hiebert, University of Delaware; Anne K. Morris, University of Delaware

Discussants
  • William R. Penuel, University of Colorado–Boulder
  • Jal David Mehta, Harvard University

Paper Session

Technologies to Construct Knowledge

Sun, April 7, 9:5511:25am, Sheraton Centre Toronto Hotel, Mezzanine, Chestnut West

Chair
  • Gabriela T. Richard, The Pennsylvania State University
Papers

Enhancing Learning From Educational Media Through Attention-Directing Cues
Authors: Preeti Ganesh Samudra, New York University; Susan B. Neuman, New York University; Kevin M. Wong, New York University

Explanatory and Predictive Modeling Within Improvement Science Projects
Authors: Andrew E. Krumm, Digital Promise; David Yeager, The University of Texas–Austin; Hiroyuki Yamada, Carnegie Foundation for the Advancement of Teaching

Idea Improvement Patterns in Knowledge-Building: Undergraduate and Graduate Levels
Authors: Gaoxia Zhu, University of Toronto; Matthew Moreno, University of Toronto; Andres Fernando Valencia, Universidad del Valle; Marlene Scardamalia, University of Toronto

Improvisational Structuring of Knowledge-Building Discourse in a Grade 5 Science Classroom
Authors: Jianwei Zhang, University at Albany–SUNY; Marzieh Dehghan Chaleshtori, University at Albany–SUNY; Hyejin Park, University at Albany–SUNY; GuangjiYuan, SUNY–ALBANY; Cornelia Harris, University at Albany–SUNY; He Zhou, University at Albany–SUNY; Ruijun Zhao


Symposium

Social Organization of Networked Improvement Communities: A Conceptual Framework and Empirical Studies of Its Dimensions

Sun. April 7, 3:40–5:10 p.m., Metro Toronto Convention Centre, 200 Level, Room 202B

Abstract: Structured collaborations between practitioners and researchers working towards addressing persistent problems have recently emerged in education, with limited literature around a conceptual organization and research around their effectiveness. One form of such research-practice partnerships (RPPs) that are increasingly forming—Networked Improvement Communities (NICs)—are organized around the use of improvement science methods and networked learning. This symposium offers first a conceptual framework of NIC development derived from the literature and actual cases of NICs, followed by three empirical studies building a deeper understanding of different dimensions of the framework. A discussant knowledgeable of RPPs and improvement networks will serve as a critical friend and engage the audience in discussion.

Chair
  • Donald J. Peurach, University of Michigan and Carnegie Foundation for the Advancement of Teaching
Authors

Social Organization of Networked Improvement Communities: A Conceptual Framework and Empirical Studies of Its Dimensions
Authors: Jennifer Lin Russell, University of Pittsburgh and Carnegie Foundation for the Advancement of Teaching; Anthony S. Bryk, Carnegie Foundation for the Advancement of Teaching; Edit Khachatryan, Carnegie Foundation for the Advancement of Teaching; Paul LeMahieu, Carnegie Foundation for the Advancement of Teaching; Donald J. Peurach, University of Michigan and Carnegie Foundation for the Advancement of Teaching; Jennifer Zoltners Sherer, University of Pittsburgh; Maggie Hannan

Getting Better at Getting Better: Developing Improvement Habits and Identities
Authors: Manuelito Biag, Carnegie Foundation for the Advancement of Teaching; David Gilbert Sherer, Carnegie Foundation for the Advancement of Teaching

How Participation Structures and Routines Contribute to Critical Learning Needs to Support Network Functioning
Authors: Jennifer Zoltners Sherer, University of Pittsburgh; Jennifer Elise Iriti, University of Pittsburgh; Jennifer Lin Russell, University of Pittsburgh and Carnegie Foundation for the Advancement of Teaching

Using Social Network Analysis to Inform Organizational Design in a Networked Improvement Community
Author: David Gilbert Sherer, Carnegie Foundation for the Advancement of Teaching

Discussant
  • Joshua L. Glazer, The George Washington University

Invited Speaker Session

What Can Researchers, Philanthropies, and Practitioner-Educators Do to Democratize Evidence in Education?

Sun. April 7, 3:40–5:10 p.m., Metro Toronto Convention Centre, 100 Level, Room 104A

Abstract: An enduring challenge for education research is for research itself to become democratic—that is, to foster goals of inclusion, dialogue, and deliberation through the activities of research (House & Howe, 2000; Tseng, Fleishman, & Quintero, 2018). Emerging forms of collaborative research and development show promise toward realizing this ideal, through efforts to create more equitable engagements among researchers, educators, families, and community members. This work occurs across education research in various forms or under different names, such as research-practice partnerships, community-based research, participatory action research, design research, research alliances, and engaged scholarship.

Broadly, these forms of collaborative research have many similarities. They share a commitment to centering the voices and perspectives of multiple stakeholders in education. They involve a systematic form of inquiry to understand problems and co-define possible solutions. They can also encourage the participation of those groups who have been historically or politically disenfranchised to enrich and expand the available knowledge base. All of these approaches stand in contrast to forms of research in which education stakeholders have little opportunity to define the focus of research or contribute in interpreting and making sense of findings for local needs.

Together, these approaches represent possible ways of democratizing the research enterprise through more equitable distributions of power, greater attention to diverse needs and questions, and a broader appreciation for the value of different kinds of knowledge and expertise. Yet, too often, the conversations of these traditions are disconnected from one another. The purpose of this session is to actively engage the AERA audience in a dialogue about a vision for democratization of evidence and explore their meaning to the research community moving forward in this post-truth era of civic activity.

Chairs
  • Caitlin Farrell, University of Colorado–Boulder
  • William R. Penuel, University of Colorado–Boulder
  • Jim Kohlmoos, EDGE Consulting LLC and Carnegie Foundation for the Advancement of Teaching
Participants
  • Megan Bang, The Spencer Foundation
  • John B. Diamond, University of Wisconsin–Madison
  • Ruth Lopez Turley, Rice University
  • Douglas Adam Watkins, Denver Public Schools
  • Esther Quintero, Albert Shanker Institute
Moderator
  • Vivian Tseng, William T. Grant Foundation

Structured Poster Session

Problem-Focused and Place-Based Leadership Development: The Improvement Leadership Education and Development (iLEAD) Research-Practice Preparation Partnership

Mon. April 8, 12:20–1:50 p.m., Metro Toronto Convention Centre, 800 Level, Room 801A

Abstract: There are few examples of teacher and leadership development programs that intentionally connect to place and local problems as part of its program design and purpose. This symposium presents the Carnegie Foundation’s Improvement Leadership Education and Development (iLEAD) initiative. Representing a mix of private and public universities and urban and rural districts across 10 states, iLEAD is a network of 11 research-practice preparation partnerships that use improvement science and networked improvement communities to build leadership capacity and address local and persistent problems of practice. iLEAD advances a collaborative, place-based, and problem-focused approach to build improvement capability and promote educational opportunity. This session highlights iLEAD’s shared working theory and progress indicators, and the key accomplishments, progress, and challenges of eight partnerships.

Chairs
  • Louis M. Gomez, University of California–Los Angeles and Carnegie Foundation for the Advancement of Teaching
  • David G. Imig, University of Maryland and Carnegie Foundation for the Advancement of Teaching
Papers

Improvement Leadership Education and Development (iLEAD) Research-Practice Preparation Partnership: Fordham University and New York City Region Districts
Author: Margaret Terry Orr, Fordham Graduate School of Education

Improvement Leadership Education and Development (iLEAD) Research-Practice-Preparation Partnership: The University of Virginia and Chesterfield County Public Schools
Author: David H. Eddy-Spicer, University of Virginia

Improvement Leadership Education and Development (iLEAD) Research-Practice Preparation Partnership: University of Denver and Denver Public Schools
Author: Susan Korach, University of Denver

Improvement Leadership Education and Development (iLEAD) Research-Practice Preparation Partnership: Portland State University and Portland Public Schools
Author: Tania L. McKey, Portland State University

Improvement Leadership Education and Development (iLEAD) Research-Practice Preparation Partnership: University of South Carolina and Florence One
Author: Rose M Ylimaki, University of South Carolina

High-Tech High Schools and Graduate School of Education: Transformative Adult Learning Through Improvement Science
Author: Daisy Sharrock

Improvement Leadership Education and Development (iLEAD) Research-Practice Preparation Partnership: Indiana University–Bloomington–Evansville Vanderburgh School Corporation
Author: Chad Lochmiller, Indiana University

Improvement Leadership Education and Development (iLEAD) Research-Practice Preparation Partnership: The Prince George’s County Public Schools/University of Maryland Improvement Science Collaborative
Author: Segun Eubanks, University of Maryland–College Park

Discussant
  • Donald J. Peurach, University of Michigan and Carnegie Foundation for the Advancement of Teaching

Roundtable Session

Policy, Instruction, and Effectiveness of Schools

Mon. April 8, 2:15–3:45 p.m., Metro Toronto Convention Centre, 800 Level, Hall F

Papers

Two Realities: The Relationship Between Actual School Interventions and Factors Found in Educational Effectiveness Syntheses
Author: Annemarie Neeleman, Maastricht University

Assessing Context: A Revised Theory of School and District Effectiveness
Authors: Brandy Doan, Hamilton Wentworth District School Board; Manny Figueiredo, Hamilton Wentworth District School Board

Building Bridges for Pre-K–3: Role of Policy in Districts’ Efforts to Align Instruction for Continuity Between Pre-K and Elementary Classrooms
Authors: Kelly McMahon, Northwestern University and Carnegie Foundation for the Advancement of Teaching; Seenae Chong, University of California–Berkeley; Natalie Jou, Northwestern University School of Education and Social Policy Learning Sciences

Multistate Impact: National Model as Blueprint for Increasing Teacher Instructional Quality
Authors: Tanee Hudgens, National Institute for Excellence in Teaching; Joshua H. Barnett, National Institute for Excellence in Teaching

Discussant
  • Mark A Partridge, Georgia Gwinnett College

Symposium

Testing Continuous Improvement in Practice: Learning Locally to Improve at Scale

Mon. April 8, 2:15–3:45 p.m., Metro Toronto Convention Centre, 200 Level, Room 202C

Abstract: Continuous Improvement has become a central strategy for reform in federal, state, and foundation efforts to support school improvement. Evidence from other sectors show great promise for dramatically changing outcomes in vastly different organizations. To date, while there is some evidence of impact in education, the idea is relatively new and untested. Data system requirements, processes for institutionalizing improvement processes, and approaches for leading these changes need to be better understood if this approach is to achieve its promise in education. This symposium consists of four papers, which collectively articulate a framework and required resources for continuous improvement and then provide examples from two of the nation’s stronger examples of networked improvement communities.

Chair
  • Jennifer Lin Russell, University of Pittsburgh and Carnegie Foundation for the Advancement of Teaching
Papers

Toward a Common Vision of Continuous Improvement for California
Authors: Alicia J. Grunow, Improvement Collective and Carnegie Foundation for the Advancement of Teaching; Heather J. Hough, Stanford University; Sandra Park, University of California–Berkeley and Carnegie Foundation for the Advancement of Teaching; Jason Willis, WestEd; Kelsey Krausen, University of California–Davis

Supporting Data Use for Continuous Improvement: Lessons From California
Authors: Heather J. Hough, Stanford University; Erika Byun, Stanford University; Laura S. Mulfinger, Claremont Graduate University

Using Continuous Improvement to Transform Math Outcomes: The Central Valley Networked Improvement Community
Authors: Alicia J. Grunow, Improvement Collective and Carnegie Foundation for the Advancement of Teaching; Sandra Park, University of California–Berkeley and Carnegie Foundation for the Advancement of Teaching; Christine Roberts, Tulare County Office of Education; Shelah Feldstein, Tulare County Office of Education

The CORE Improvement Community: Testing Continuous Improvement in a Network of Large Urban Districts
Authors: Alix Gallagher, Stanford University; Heather J. Hough, Stanford University; Taylor N. Allbright, University of Southern California; Kate Kennedy, University of Southern California

Discussant
  • Marc L. Stein, Johns Hopkins University

Symposium

Preparing Leaders for Continuous Improvement: Addressing Systemic Educational Inequities Through Preparation Partnerships

Tue. April 9, 10:25–11:55 a.m., Metro Toronto Convention Centre, 200 Level, Room 201C

Abstract: This symposium includes faculty from universities who have partnered with school districts to prepare and support educators to lead continuous improvement in school systems. Collectively, the papers draw upon the principles and approaches of networked improvement science, a methodology for quality improvement that is gaining prominence in both K—12 and higher education settings. The four papers broadly explore the question “What approaches to leadership preparation and/or support enable leaders to gain practical experience with continuous improvement to address systemic inequities in education?” The papers conceptualize networked improvement science, describe preparation partnerships and district practices using improvement science, and discuss programmatic implications for the education doctorate focused on improvement science.

Chairs
  • Chad Lochmiller, Indiana University
  • David H. Eddy-Spicer, University of Virginia
Papers

Cultivating Leaders for Equity: The Carnegie Foundation’s Improvement Leadership Education and Development Initiative
Manuelito Biag, Carnegie Foundation for the Advancement of Teaching

Using Improvement Science to (Re)Design Leadership Preparation: Exploring Powerful Learning Experiences Across Five University Programs
Sonya Diana Hayes, The University of Tennessee–Knoxville; Corrie Stone-Johnson, University at Buffalo–SUNY; Amy L. Reynolds, University of Virginia

Integrating Improvement Science Into Diverse District Work Streams: An Advanced Partnership’s Efforts
Marian A.Robinson, The George Washington University

Developing Improvement Science Leaders in Preparation and Practice: The University Experience
Margaret Terry Orr, Fordham Graduate School of Education

Discussant
  • Donald J. Peurach, University of Michigan and Carnegie Foundation for the Advancement of Teaching

Symposium

Recrafting Coherence in the Era of Common Core

Tue. April 9, 12:20–1:50 p.m., Metro Toronto Convention Centre, 200 Level, Room 201C

Abstract: The educational landscape has shifted since Honig and Hatch (2004) originally theorized school leaders and district personnel could “craft coherence” by strategically managing external demands based on each school’s own goals and strategies. The Common Core standards placed new demands on districts and schools by raising the bar for proficiency. Additionally, the numbers and types of systems operating to support schools in meeting these new demands have increased—including the exponential growth of CMOs, nonprofit intermediary organizations, school improvement networks, and local districts more focused on instructional improvement. Our symposium explores how these various education systems operate to continuously re-craft coherence in response to rigorous instructional expectations.

Chairs
  • Megan Duff, Teachers College, Columbia University
  • Priscilla Wohlstetter, Teachers College, Columbia University
Papers

Organizing and Managing for Excellence and Equity: The Work and Dilemmas of Instructionally Focused Education Systems
Authors: Donald J. Peurach, University of Michigan and Carnegie Foundation for the Advancement of Teaching; Maxwell Yurkofsky, Harvard University; Daniella Hall Sutherland, Clemson University; James P. Spillane, Northwestern University

The Complexity of Developing Instructional Program Coherence in Turnaround
Authors: Diane Massell, Consortium for Policy Research in Education; Joshua L. Glazer, The George Washington University

Recrafting Coherence at the Intermediary Level
Authors: Megan Duff, Teachers College, Columbia University; Angela Lyle, University of Michigan–Ann Arbor; Priscilla Wohlstetter, Teachers College, Columbia University

Instructional Guidance Infrastructures and Changing Conceptions of Teachers’ Occupational Role
Author: Clare Buckley Flack, Teachers College, Columbia University