NASSP Bulletin

December 2003
Vol. 87 No. 637, pp. 38-54

Analyzing Lists of the Characteristics of Effective Professional Development to Promote Visionary Leadership
Thomas R. Guskey

In recent years, different researchers and research agencies, teacher associations, national education organizations, and the U.S. Department of Education have published lists of the characteristics of effective professional development to guide school leaders in their improvement efforts. This study analyzed 13 of the better known of these lists to determine whether they were derived through similar procedures, based on similar frames of reference, and included the same elements or characteristics. Results show that individual characteristics vary widely in their frequency of inclusion in the lists and that no characteristic is consistently named in all lists. In addition, research evidence supporting most of the identified characteristics is inconsistent and often contradictory. Implications for leaders interested in improving professional development activities are discussed, as well as ways to enhance efforts to identify the characteristics of effective professional development.

The education reforms of the past decade brought new prominence to the role of professional development. Recognizing that schools can be no better than the teachers and administrators who work in them, policymakers emphasized professional development as a key component in nearly every education improvement plan. The recently enacted No Child Left Behind Act of 2001 (NCLB; 2002), for example, stresses the importance of high-quality professional development to guarantee that all teachers are "highly qualified" and that all students reach high levels of achievement. This emphasis compels visionary leaders to reexamine professional development efforts with a keen eye for issues of value and worth.

With this increased prominence, however, has come increased scrutiny. Professional development's tainted history of effectiveness and the lack of strong evidence showing its direct link to improvements in student learning outcomes (see, e.g., Corcoran, 1995a; Frechtling, Sharp, Carey, & Baden-Kierman, 1995; Newmann, King, & Youngs, 2000; Wang, Frechtling, & Sanders, 1999) led those same policymakers to demand assurances of quality in these endeavors. This, in turn, prompted the publication and dissemination of an assortment of "lists" describing the characteristics of high-quality, effective professional development.

Those involved in professional development research or practice, as well as school leaders at all levels, are undoubtedly familiar with several of these lists. They have been produced in recent years by researchers and research agencies (e.g., Educational Research Service and Educational Testing Service), teacher associations (e.g., American Federation of Teachers [AFT]), national education organizations (e.g., National Partnership for Excellence and Accountability in Teaching), and the U.S. Department of Education.

A quick inspection shows, however, that although some overlap exists among the identified characteristics, these lists are not identical. Some characteristics are frequently mentioned, whereas others appear unique to a particular list. Such a lack of consensus undoubtedly frustrates and confuses those responsible for designing and implementing high-quality professional development programs for educators. How can they ensure that their efforts will be effective when there appears to be little agreement about the factors that contribute to effectiveness?

To clarify these efforts and to determine the degree of consensus that exists among the individuals and organizations responsible for developing these lists, 13 of the better known lists of the characteristics of effective professional development were selected for analysis in this investigation. The analysis had three goals: The first goal was to determine whether the lists were derived in comparable ways. In other words, were they drawn from the same sources of evidence and based on similar frames of reference? The second goal was to ascertain whether specific characteristics appear on all of the lists. This would indicate strong consensus regarding the contribution of these characteristics to the effectiveness of professional development. The final purpose was to verify how well the identified characteristics correspond to the recently revised Standards for Staff Development, (National Staff Development Council [NSDC], 2001), published to help educators at all levels improve the quality of professional development endeavors.

The Lists

The lists included in the analysis are described below in chronological order, based on their publication date. They were collected from a search of the Educational Resources Information Center (ERIC) system, from a targeted Internet search, and from related professional development literature (i.e., publications from NSDC). Although all of the lists were developed within the last decade, no claim of completeness can be made for this list of lists, and other equally valid lists may exist. Nevertheless, it is believed these 13 lists comprise a fairly representative sample of those developed in recent years, and clearly rank among those best known to researchers and school leaders.

The first and earliest list considered was assembled by Thomas B. Corcoran (1995a, 1995b) as part of a project sponsored by the National Governors' Association and coordinated by the Consortium for Policy Research in Education. Corcoran reviewed papers and reports published by individuals well known in professional development circles (e.g., Griffin; Hodges; Joyce & Showers; Loucks-Horsley; Little; Price; and Zimpher & Howey) and the Standards for Staff Development: Middle Level Edition, prepared by NSDC (1994). From these sources, he identified eight guiding principles that "a number of experts and organizations have suggested (contribute to) the most promising professional development programs or policies" (1995a, p. 3).

The second was a list of nine guidelines published by AFT (1996) in Principles for Professional Development. These guidelines were derived from teachers' views of effective professional development.

The third list came from a report by Willis D. Hawley and Linda Valli (1996, 1999), who reviewed the literature on professional development and identified eight characteristics associated with effective professional development programs. According to Hawley and Valli, these eight characteristics represent "an almost unprecedented consensus … among researchers, professional development specialists, and key policymakers on a vision of ways to substantially increase the knowledge and skills of educators" (1996, p. 1).

The fourth list was based on the work of Susan Loucks-Horsley, Katherine Stiles, and Peter Hewson (1996), who examined a variety of standards and related materials as part of the Professional Development Project of the National Institute for Science Education. Their goal was to determine whether the science, mathematics, and professional development communities share a common understanding of effective professional development. This effort led to the identification of seven principles that consistently marked "the best professional development experiences for science and mathematics educators" (p. 1).

The fifth list came from a 1997 report published by the U.S. Department of Education titled, Achieving the Goals. This report included 10 principles of high-quality professional development on which "experts agree" (p. 3); however, the report provided no information as to who are these "experts."

The sixth list was part of a research summary conducted by the Educational Research Service (1998) titled, Professional Development for Teachers: Challenges and Trends. This summary drew on reports by NSDC (1994), the National Commission on Teaching and America's Future (1996), the National Foundation for the Improvement of Education (1996), and the U.S. Department of Education (1996). It extended a review by Sparks and Loucks-Horsley (1989) of the characteristics of effective professional development by articulating 16 additional, "research-based principles and standards" (p. 3).

The seventh list was developed from a research project sponsored by the National Institute for Science Education and conducted by Mary M. Kennedy (1998). For this project, Kennedy identified a pool of 93 studies that examined the effectiveness of various approaches to continuing teacher education in either mathematics or science. Kennedy limited her review, however, to the 10 investigations that included evidence of benefits to students and reached six overall conclusions about factors that contribute to this "ultimate benefit."

The eighth list came from the work of Beatrice F. Birman, Laura Desimone, Andrew C. Porter, and Michael S. Garet (2000; see also Garet, Porter, Desimone, Birman, & Yoon, 2001). These researchers surveyed a nationally representative sample of more than 1,000 teachers who participated in professional development sponsored in part by the federal government's Eisenhower Professional Development Program. As part of their national evaluation, they also conducted six exploratory case studies and 10 in-depth case studies in five states (Garet, Birman, Porter, Desimone, Herman, & Yoon, 1999). By studying the literature and analyzing the survey data, they identified three structural features and three core features that characterize effective professional development experiences.

The ninth list was developed as part of the California Professional Development Reform Initiative. In this project, Karen Kent and Carol Lingman (2000) led a working committee drawn from three partner organizations: the California Staff Development Council; the Center for the Future of Teaching and Learning, a private nonprofit organization; and the California Department of Education. After reading summaries of recent professional development reports and articles (e.g., Sparks, 1997; Sparks & Hirsh, 1997), committee members drafted "organizing themes around the nature of quality professional development for teachers" (p. 34). The group then tested these themes by gathering interview data from teachers, and finally developed a list of 10 design elements for high-quality professional development.

The tenth list stemmed from the work the National Partnership for Excellence and Accountability in Teaching (2000). Based on presentations and interviews with participants from the 1999 Conference of Teacher Professional Development and stories from several schools and districts that have been using learner-centered strategies for professional development, the Partnership developed a list of nine "research-based principles" for improving professional development.

The eleventh list came from a report prepared for the Urban Special Education Leadership Collaborative by Mary Terzian (2000) of the Education Development Center. Terzian synthesized the current literature on effective professional development for sustained school reform and outlined eight specific design features that contribute to effectiveness.

The twelfth list was derived from a study by Harold Wenglinsky (2002) of the Educational Testing Service. Wenglinsky applied multilevel structural equation modeling to data drawn from the 1996 National Assessment of Educational Progress to determine the relationship between 10 measures of professional development experienced by teachers and eighth-grade mathematics assessment scores.

The final list used for comparison was the recently revised Standards for Staff Development, published by NSDC (2001). These 12 standards, divided into subgroups of context, process, and content standards, represent a revision of an earlier set of standards developed by experts in professional development and representatives from several national education organizations. They set forth the elements of an ideal model for "staff development that improves the learning of all students" (p. 5).

Analysis

After collecting the lists, two colleagues and I analyzed the characteristics identified in each using standard content analysis procedures. This involved reading each list and grouping the noted characteristics or principles in broad, content-specific categories. Categories were labeled using general descriptors perceived by the reviewers to capture the meaning of the noted characteristics or principles. When disagreements about the classification of a characteristic on a particular list arose, the characteristic was reviewed, reexamined, and discussed until consensus among raters was reached.

The evidence from which each list was derived was also noted during the analysis. As mentioned earlier, some of the lists were based on empirical data (e.g., Birman et al., 2000; Wenglinsky, 2002), or syntheses of empirical data (e.g., Kennedy, 1998). Others were constructed as policy syntheses based on a mix of empirical data, large- and small-scale studies, and anecdotal case studies (Hawley & Valli, 1996; Loucks-Horsley et al., 1996), whereas still others were based on broad literature reviews (Educational Research Service, 1998). No attempt was made to distinguish among the lists in terms of the quality or the strength of the evidence used, however. In many cases this evidence was not described in sufficient detail in the reviewed documents to make such a determination. In addition, because all of the lists were widely disseminated, well known, and equally available to researchers, practitioners, and school leaders, all were considered comparably influential in guiding the development of modern professional development programs and activities.

Table 1 shows the results of the analysis, with the characteristic categories arranged by the frequency of inclusion of the characteristic or principle in the lists. The general description of the characteristic was used to label each. The numbers in the table represent identifying numerals assigned to characteristics based on their order of presentation in each list. The number 3 under column 1 of the references, for example, indicates that the third characteristic of effective professional development noted in the research of Corcoran (1995b), "Grounded in knowledge about teaching" (p. 3), was considered to fall under the general characteristic of "Enhancing teachers' content and pedagogic knowledge." Most characteristics were identified as having a positive influence on effectiveness. Underlined numbers represent characteristics that specific lists identified as having either no influence or a negative influence on the effectiveness of professional development activities.

Results

The results from the analysis are in some ways validating and in other ways surprising. They show, for example, that there is no relationship between the identified characteristics and the date of publication of a list. In other words, no characteristic appears to have become increasingly popular or to have fallen out of favor within this relatively narrow timeframe. Results also illustrate that although certain characteristics appear on most lists, none is included in all. Furthermore, some of the most frequently noted characteristics are included in the NSDC Standards for Staff Development, however others are not. And most surprisingly, several of the characteristics featured prominently in the NSDC Standards appear in few or none of the lists reviewed.

In considering how they were derived and the criteria used in determining effectiveness, most of the lists could be described as research based. But that research rarely includes rigorous investigations of the relationship between the noted characteristics and improvements in instructional practice or in measures of student learning outcomes. Instead, it typically involves surveys of the opinions of researchers and educators. The best that can be said, therefore, is that researchers and educators generally favor these characteristics and believe they are important, despite a lack of verifying evidence. Even the large-scale study by Birman et al. (2000) is based primarily on self-reported survey data. Only Kennedy's (1998) analysis and Wenglinsky's (2002) multilevel modeling study, 7 and 12 respectively in the table, showed a direct link between the identified characteristics and measures of student achievement.

The most frequently mentioned characteristic of effective professional development is enhancement of teachers' content and pedagogic knowledge. Eleven of the 13 lists emphasize this characteristic. Helping teachers to understand more deeply the content they teach and the ways students learn that content appears to be a vital dimension of effective professional development. Research also shows that professional development based on higher order thinking skills within a subject can be particularly effective (Wenglinsky, 2000).

At the same time, however, it is important to note that nearly all of the studies relating this characteristic to improvements in student learning focus exclusively on achievement in mathematics and/or science (e.g., Cohen & Hill, 1998, 2000; Kennedy, 1998, 1999; Wenglinsky, 2002). A few smaller studies have considered the importance of teachers' literacy content knowledge (McCutchen & Berninger, 1999; Moats, 1994), but these have focused narrowly on special education contexts. For the most part, whether this relationship is true for achievement in language arts, social studies, foreign languages, or other subject areas has yet to be thoroughly investigated.

Ten of the lists include the provision of sufficient time and other resources as essential to effective professional development. Obviously, educators need time to deepen their understanding, analyze students' work, and develop new approaches to instruction. Data gathered by Birman et al. (2000) further showed that "activities of longer duration have more subject-area content focus, more opportunities for active leaning, and more coherence with teachers' other experiences than do shorter activities" (p. 30). But significant contrary evidence exists. Kennedy's (1998) analysis showed that differences in the time spent in professional development activities were unrelated to improvements in student outcomes. How time was distributed also yielded mixed results, with no differences found in mathematics studies and only modest effects found in science investigations. Similarly, Wenglinsky's (2002) study of mathematics achievement found that among the various aspects of professional development investigated, "the amount of time is not significantly related to achievement" (p. 19). Therefore, although effective professional development clearly requires time, it also seems clear that such time must be well organized, carefully structured, and purposefully directed (Guskey, 1999).

Another consistently noted characteristic is the promotion of collegiality and collaborative exchange. Educators at all levels value opportunities to work together, reflect on their practices, exchange ideas, and share strategies and expertise. Collaboration also helps build a sense of community (Supovitz, 2002). But as Little (1990) pointed out, there is nothing particularly virtuous about collaboration per se. Individuals can collaborate to block change or inhibit progress just as easily as they can to enhance the process. Recent investigations also showed that many collaborative efforts run headlong into enormous conflicts over professional beliefs and practices for which practitioners are generally ill prepared (Achinstein, 2002). For collaboration to bring its intended benefits it, too, needs to be structured and purposeful, with efforts guided by clear goals for improving student learning.

Most of the lists and the NSDC Standards stress the inclusion of specific evaluation procedures. This emphasis may stem from growing awareness among educators at all levels of the need to gather regular formative information to guide their improvement efforts (Guskey, 2000, 2002). Of course, policymakers' new demands for accountability in professional development undoubtedly contribute to an emphasis on evaluation as well (Fuhrman, 1999; Guskey, 1998; Killion, 2002).

The majority of lists recognize the need for professional development activities to be aligned with other reform initiatives and to model high-quality instruction. These characteristics likely come from appreciation of the large-scale, comprehensive, and systemic nature of most modern education reform initiatives and from increased awareness of similarities in the learning patterns of adults and children (Consortium for Policy Research in Education, 1996; Darling-Hammond & McLaughlin, 1995; Fullan, 1993).

Most lists also stress that professional development should be school or site based, even though significant research evidence suggests this may not always be effective (see Holloway, 2000; Latham, 1998). A recent review by Corcoran, Fuhrman, and Belcher (2001), for example, found when decisions about professional development were primarily school based, "school staff members paid lip service to the use of research" and "were more interested in designs that drew on research about practices that they already felt were 'good' than in designs that were producing results" (p. 81). According to these researchers, "the decentralization of decision making appear(s) to be undermining the use of knowledge rather than promoting it" (p. 81). A well-planned, carefully organized collaboration between district-level personnel who have broader perspective on problems and site-based educators who are keenly aware of critical contextual characteristics seems essential to optimize the effectiveness of professional development (Guskey, 1996).

Building leadership capacity is included in many lists and in the NSDC Standards, reflecting current emphases on teacher leadership and the need for principals to be strong and consistent instructional leaders (Phelps & Bredeson, 2002). Several lists also note that professional development activities should be based on teachers' identified needs, even though noteworthy evidence shows that teachers rarely are able to articulate their needs. Although they have no difficulty identifying problems, dilemmas, concerns, and wants, teachers tend to describe symptoms of needs that must be diagnosed more thoroughly and interpreted more broadly (Jones & Hayes, 1980).

Despite the current emphasis in reform initiatives on improving student performance (see Linn, 2000), less than half the lists mention the importance of using careful analyses of student learning data to drive professional development activities. Likewise, fewer than half of the lists cite the importance of including follow up, ensuring that professional development activities are ongoing and job embedded, or focusing on individual and organizational improvement.

Only three lists along with the NSDC Standards stress that professional development should be based on the best available research evidence. This is particularly striking given the long-standing criticism of professional development activities that focus on fads and bandwagon movements rather than solid evidence of what works with students (Cuban, 1990; Ravitch, 2000). It may also explain the stipulation in the recently enacted NCLB legislation that only those strategies and methods "proven effective by the standard of scientifically based research should be included in school reform programs" (U.S. Department of Education, 2002, p. 2).

The remaining characteristics appear on three or fewer lists. These include the following: (a) helps to accommodate diversity and promote equity, (b) takes a variety of forms, (c) provides opportunities for theoretical understanding, (d) is driven by an image of effective teaching and learning, (e) provides for different phases of change, and (f) promotes continuous inquiry and reflection. Inconsistencies exist even in these characteristics. Wenglinsky's (2002) study showed, for example, that not all indicators of dealing with diversity and teaching special populations of students contribute substantially to improvements in student learning. Indicators related to teaching students from culturally diverse backgrounds and students with limited English proficiency were strongly related to student achievement measures, but teaching special needs students was not.

Surprisingly, no list included the NSDC Standard associated with involving families and other stakeholders. This may be because the NSDC Standards describe a broad and comprehensive list of characteristics that experts believe professional development could or should be, rather than what strong evidence supports.

Summary

This analysis of lists of the characteristics of effective professional development yielded three related conclusions. First, there appears to be little agreement among professional development researchers or practitioners regarding the criteria for effectiveness in professional development. Some define it in terms of teachers' self-reports of professional development features that influence increases in their knowledge and changes in their instructional practices. Others look for consensus in the opinions of professional development writers and researchers. Although lists of the characteristics of effective professional development based on such diverse criteria are valuable, they provide only a starting point in efforts to improve the quality of professional development programs and activities.

It could be argued, of course, that these lists were prepared for different purposes and audiences, and that those differences likely affect how the lists were configured. Some of the lists were prepared as policy documents (e.g., the U.S. Department of Education list), whereas others were prepared for audiences of professional development practitioners and school leaders (e.g., the AFT list or the NSDC Standards). As such, each reviews some portion of the existing literature and packages it for a particular audience. The needs and characteristics of each audience may have as much to do with the content of the lists as the evidence on which each is based. The differences that exist, therefore, are merely a function of how each list was prepared, not the underlying body of evidence.

To suggest that the criteria for determining the effectiveness of professional development vary depending on the intended audience implies, however, that effectiveness, like beauty, is in the eye of the beholder. How can we advance as a profession without at least a nominal level of consensus among policymakers, researchers, practitioners, and school leaders about the criteria for judging effectiveness in professional development? Such consensus brings essential focus to the goals of professional development and provides a foundation for the kinds of systematic inquiry needed to build a professional knowledge base. Without a clear sense of professional development's goals and the evidence that best reflects attainment of those goals, improvement in the quality of educators' professional development experiences is unlikely.

In order to gain authentic evidence and make serious improvements, visionary leaders must push beyond the starting point set by these initial efforts and move toward what Kennedy (1999) referred to as professional development's "ultimate benefit": demonstrable improvements in student learning outcomes. All school leaders, and especially school principals, must insist that professional development planning focus on two critical questions: How will this help our students? and What evidence will we trust to verify that it does? That evidence might include a variety of indicators of student achievement, such as assessment or test results, portfolio evaluations, marks or grades, or scores from standardized examinations. It might also include affective and behavioral indicators, such as students' attitudes, study habits, homework completion rates, or classroom behaviors. Schoolwide indicators such as attendance rates, dropout statistics, reductions in discipline problems, enrollment in advanced classes, memberships in honor societies, and participation in school-related activities might be considered as well. Significant advances in professional development will come only when researchers, practitioners, and visionary school leaders insist on this fundamental goal of improvement in student learning outcomes as the principal criteria of effectiveness.

Second, perhaps because of the lack of agreement on the criteria for effectiveness, many of the currently identified characteristics of effective professional development can be best described as "Yes, but…" statements. For example, yes, enhancing teachers' content and pedagogic knowledge is important, but existing research is limited mainly to investigations of mathematics and science instruction. Yes, professional development should provide sufficient time and other resources, but that time and those resources must be used wisely, focusing on activities that positively affect learning and learners. Yes, professional development activities should promote collegiality and collaboration, but without clear direction and purpose, individuals can collaborate to resist reform and thwart improvement efforts. Yes, professional development should include procedures for evaluation, but evaluations that focus narrowly on educators' self-reported satisfaction with professional development activities offer inadequate guidance and direction to improvement efforts. And so on.

The problem with such "Yes, but…" statements is that they frustrate policymakers and school leaders who want simple answers to their questions about effective professional development. They also tend to diminish the value of research evidence in the eyes of those seeking unambiguous statements about best practices in professional development. Nevertheless, they accurately represent the fact that nearly all professional development takes place in real-world contexts. The complexities of these varied contexts introduce a complicated web of factors that influence whether or not a particular characteristic or practice will produce the desired results (see Berliner, 2002). These nuances of context are difficult to recognize and even more difficult to take into account within the confines of a single program. Thus, programs that appear quite similar may produce different results for subtle and unanticipated reasons.

Take, for example, professional development programs specifically designed to enhance teachers' content and pedagogic knowledge-the most frequently cited characteristic of effective professional development. Schools in economically depressed areas that have trouble attracting and keeping well-qualified teachers and, as a result, have many teachers teaching subjects outside their area of certification, may benefit greatly from such programs. Conversely, schools that serve more affluent populations and have sufficient resources to attract and retain well-qualified teachers with advanced training in the subject areas they teach may see little improvement from similar programs. These real-world contextual differences profoundly influence the effectiveness of professional development endeavors.

It seems clear, therefore, that differences in communities of school administrators, teachers, and students uniquely affect professional development processes and can strongly influence the characteristics that contribute to professional development's effectiveness. Because of these powerful contextual influences, broad-brush policies and guidelines for best practice may never be completely accurate. Still, by carefully considering these contextual elements and making decisions based on specific evidence of student learning, visionary school leaders can better ensure that their professional development programs and activities will meet with success.

Third, these results show that although the promise of research-based professional development remains largely unfulfilled, it need not remain so. Analyses of student learning data typically show that greater variation exists between classrooms within a school than between schools or between districts (Kifer, 2001). In other words, within the unique context of nearly every school there are teachers who have found ways to help students learn excellently. Visionary school leaders who find ways to help these successful teachers share their practices and strategies with their colleagues in a positive and supportive environment provide a basis for highly effective professional development within that context.

The characteristics that influence the effectiveness of professional development are clearly multiple and highly complex. For this reason it may be unreasonable to assume that a single list of effective professional development characteristics will ever emerge, regardless of the quality of professional development research. Nevertheless, by agreeing on the criteria for effectiveness, considering the unique contextual elements of each school and the community of learners in that environment, and continually directing efforts toward improvements in student learning outcomes, visionary school leaders can do much to guarantee sure and steady progress in educators' and researchers' efforts to improve the quality of professional development endeavors.

References

Achinstein, B. (2002). Conflict amid community: The micropolitics of teacher collaboration. Teachers College Record, 104(3), 421-455.

American Federation of Teachers (1996). Principles for professional development (Item No. 176). Washington, DC: Author.

Berliner, D. C. (2002). Educational research: The hardest science of all. Educational Researcher, 31(8), 18-20.

Birman, B. F., Desimone, L., Porter, A. C., & Garet, M. S. (2000). Designing professional development that works. Educational Leadership, 57(8), 28-33.

Cohen, D. K., & Hill, H. C. (1998). State policy and classroom performance: Mathematics reform in California. CPRE Policy Briefs (RB-23-May). Philadelphia: Consortium for Policy Research in Education (CPRE), Graduate School of Education, University of Pennsylvania.

Cohen, D. K., & Hill, H. C. (2000). Instructional policy and classroom performance: The mathematics reform in California. Teachers College Record, 102(2), 294-343.

Consortium for Policy Research in Education (1996). Public policy and school reform: A research summary. (Rep. No. 36). Philadelphia: Author.

Corcoran, T. B. (1995a, June). Helping teachers teach well: Transforming professional development. CPRE Policy Briefs. New Brunswick, NJ: Consortium for Policy Research in Education, Rutgers University.

Corcoran, T. B. (1995b). Transforming professional development for teachers: A guide for state policymakers. Washington, DC: National Governors' Association.

Corcoran, T. B., Fuhrman, S. H., & Belcher, C. L. (2001). The district role in instructional improvement. Phi Delta Kappan, 83(1), 78-84.

Cuban, L. (1990). Reforming again, again, and again. Educational Researcher, 19(1), 3-13.

Darling-Hammond, L., & McLaughlin, M. (1995). Policies that support professional development in an era of reform. Phi Delta Kappan, 76(8), 597-604.

Educational Research Service (1998). Professional development for teachers: Challenges and trends. Arlington, VA: Author.

Frechtling, J. A., Sharp, L., Carey, N., & Baden-Kierman, N. (1995). Teacher enhancement programs: A perspective on the last four decades. Washington, DC: National Science Foundation Directorate for Education and Human Resources.

Fuhrman. S. H. (1999). The new accountability. (CPRE Policy Briefs No. RB-27). Philadelphia: University of Pennsylvania, Consortium for Policy Research in Education.

Fullan, M. G. (1993). Change forces: Probing the depths of educational reform. Bristol, PA: Falmer Press.

Garet, M. S., Porter, A. C., Desimone, L., Birman, B. F., & Yoon, K. S. (2001). What makes professional development effective? Results from a national sample of teachers. American Educational Research Journal, 38(4), 915-945.

Garet, M. S., Birman, B. F., Porter, A. C., Desimone, L., Herman, B., & Yoon, K. S. (1999). Designing effective professional development: Lessons from the Eisenhower program. Washington, DC: U.S. Department of Education.

Guskey, T. R. (1996). Jointly planning staff training. The School Administrator, 53(11), 33-34.

Guskey, T. R. (1998). The age of our accountability. Journal of Staff Development, 19(4), 36-44.

Guskey, T. R. (1999). Apply time with wisdom. Journal of Staff Development, 20(2), 10-15.

Guskey, T. R. (2000). Evaluating professional development. Thousand Oaks, CA: Corwin Press.

Guskey, T. R. (2002). Does it make a difference? Evaluating professional development. Educational Leadership, 59(6), 45-51.

Hawley, W. D., & Valli, L. (1996, Fall). The essentials of effective professional development: A new consensus. Professional Development Newsletter-ASCD Human Resource Development Program, 1-2.

Hawley, W. D., & Valli, L. (1999). The essentials of effective professional development: A new consensus. In L. Darling-Hammond & G. Sykes (Eds.), Teaching as the learning professon: Handbook of policy and practice (pp. 127-150). San Francisco: Jossey-Bass.

Holloway, J. H. (2000). The promise and pitfalls of site-based management. Educational Leadership, 57(7), 81-82.

Jones, L. L., & Hayes, A. E. (1980). How valid are surveys of teacher needs? Educational Leadership, 37(5), 390-392.

Kennedy, M. M. (1998). Form and substance in inservice teachers education. (Research Monograph No. 13). Madison, WI: National Institute for Science Education, University of Wisconsin.

Kennedy, M. M. (1999). Form and substance in mathematics and science professional development. NISE Brief, 3(2), 1-7.

Kent, K., & Lingman, C. (2000). California's course. Journal of Staff Development, 21(3), 31-36.

Kifer, E. (2001). Large scale assessment. Thousand Oaks, CA: Corwin Press.

Killion, J. (2002). Assessing impact: Evaluating staff development. Oxford, OH: National Staff Development Council.

Latham, A. S. (1998). Site-based management: Is it working? Educational Leadership, 55(7), 85-86.

Little, J. W. (1990). The persistence of privacy: Autonomy and initiative in teachers' professional relations. Teachers College Record, 91(4), 509-536.

Linn, R. L. (2000). Assessments and accountability. Educational Researcher, 29(1), 4-16.

Loucks-Horsley, S., Stiles, K., & Hewson, P. (1996). Principles of effective professional development for mathematics and science education: A synthesis of standards. NISE Brief, 1(1), 1-6.

McCutchen, D., & Berninger, V. W. (1999). Those who know, teach well: Helping teachers mastery literacy-related subject-matter knowledge. Learning Disabilities Research and Practice, 14(4), 215-226.

Moats, L. C. (1994). The missing foundations in teacher education: Knowledge of the structure of spoken and written language. Annals of Dyslexia, 44, 81-102.

National Commission on Teaching and America's Future. (1996). What matters most: Teaching for America's future. New York: Author.

National Foundation for the Improvement of Education. (1996). Teachers take charge of their learning: Transforming professional development for student success. Washington, DC: Author.

National Partnership for Excellence and Accountability in Teaching. (2000). Improving professional development. Retrieved February15, 2001, from www.npeat.org/strand2/pdprinpdf

National Staff Development Council. (1994). National Staff Development Council's standards for staff development: Middle level edition. Oxford, OH: Author.

National Staff Development Council (2001). Standards for staff development. Retrieved March 26, 2001, from www.nsdc.org/educatorindex.htm

Newmann, F., King, M. B., & Youngs, P. (2000). Professional development that addresses school capacity: Lessons from urban elementary schools. American Journal of Education, 108(4), 259-299.

No Child Left Behind Act of 2001(H.R. 1), 107 Cong., 110 (2002) (enacted).

Phelps, L. A., & Bredeson, P. V. (2002, March). Enhancing high school teacher leadership through collaborative action research: An analysis of the Education and Work Leadership Program. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA.

Ravitch, D. (2000). Left back: A century of failed school reforms. New York: Simon & Schuster.

Sparks, D. (1997). Professional development: Are you getting the results you want? California Professional Development Consortia, Restructuring Brief 14. Retrieved February 15, 2001, from www.cpdc.k12.ca.us

Sparks, D., & Hirsh, S. (1997). A new vision for professional development. Alexandria, VA: Association for Supervision and Curriculum Development.

Sparks, D., & Loucks-Horsley, S. (1989). Five models of staff development for teachers. Journal of Staff Development, 10(4), 40-57.

Supovitz, J. A. (2002). Developing communities of instructional practice. Teachers College Record, 104(8), 1591-1626.

Terzian, M. (2000). Design principles for effective professional development: A research synthesis. Newton, MA: Education Development Center.

U.S. Department of Education (1996). Building bridges: The mission and principles of professional development (pamphlet). Washington, DC: Author.

U.S. Department of Education (1997). Achieving the goals: Goal 4. Teacher education and professional development. Washington, DC: Author.

U.S. Department of Education (2002). Scientifically based research and the Comprehensive School Reform (CSR) Program. Washington, DC: Author.

Wang, Y. L., Frechtling, J. A., & Sanders, W. L. (1999, April). Exploring linkages between professional development and student learning: A pilot study. Paper presented at the annual meeting of the American Educational Research Association, Montreal, Canada.

Wenglinsky, H. (2000). How teaching matters: Bringing the classroom back into discussions of teacher quality. Princeton, NJ: Policy Information Center, Educational Testing Service.

Wenglinsky, H. (2002). How schools matter: The link between teacher classroom practices and student academic performance. Education Policy Analysis Archives, 10(2). Retrieved May 13, 2002, from http:// epaa.asu.edu/epaa/v10n12/

__________
Thomas R. Guskey is a professor in the Department of Education Policy Studies and Evaluation in the College of Education at the University of Kentucky, Lexington. Correspondence concerning this article may be sent to guskey@uky.edu.