The purpose of this research was to determine if the Pennsylvania Value-Added Assessment System Average Growth Index (PVAAS AGI) scores, derived from standardized tests and calculated for Pennsylvania schools, provide a valid and reliable assessment of teacher effectiveness, as these scores are currently used to derive 15% of the annual effectiveness rating assigned to PVAAS eligible mathematics and English language arts teachers. The research also sought to determine if teacher perceptions indicate that the validity and reliability of the PVAAS AGI score included in the Classroom Rating Tool is important. The PVAAS AGI scores, average Normal Curve Equivalent (NCE) scores, and demographic data from 260 Pennsylvania middle schools serving grades six through eight were utilized to determine the extent to which PVAAS AGI scores assigned to a school change statistically over time and if there is a relationship between the PVAAS AGI in mathematics or English language arts for each school and the percentage of economically disadvantaged, learning disabled, English language learners, and minorities attending the school. A Likert scale survey of twenty PVAAS eligible teachers of mathematics and English language arts employed in two middle schools serving grades six through eight in the same district yielded information regarding teacher perceptions. Results of this study indicated that the mean gains for PVAAS AGI scores significantly decreased from 2013 to 2016 for all schools, regardless of achievement level, in both mathematics and English language arts. The demographic variables analyzed did not impact PVAAS AGI scores for schools. Teacher morale was significantly negatively impacted by the inclusion of PVAAS AGI scores in the Classroom Rating Tool, the Pennsylvania mandated rubric utilized to measure teacher effectiveness. Additionally, teachers indicated that they believe important decisions such as changes to instructional practices and scheduling are made based on PVAAS AGI, even though teachers do not understand how it is calculated or trust the validity of PVAAS AGI scores. The literature review surfaced concerns regarding how the average NCE scores are treated in the PVAAS statistical model, as certain treatment of these scores could potentially force invalid results. The literature review also revealed that the Data Recognition Corporation (DRC), the company that designs the standardized tests from which the data utilized in the PVAAS statistical model is derived, has warned that scores at the maximum and the minimum end of the scale may not be accurate due to the design of their test. This is a matter of interest due to the fact that the SAS Institute claims that the tests utilized by their model must be able to measure the performance of students at the lowest and highest ends of the achievement spectrum, begging the question of whether it is even possible for the PVAAS model to accurately determine student growth. Finally, discrepancies in the reported grade levels for schools were found on the state data reporting site.

The Pennsylvania Educator Effectiveness System strives to place an effective teacher in every classroom. This evaluation system incorporates the use of value-added growth measures, not only at the teacher level, but also at the building level. These building level scores are included in every teacher's yearly evaluation. This study sought to determine if a relationship existed between the value-added scores and collective teacher efficacy; the faculty's collective belief that they have the capabilities to make an educational difference for their students. 120 middle school teachers from southwestern Pennsylvania were surveyed using the Collective Teacher Beliefs Scale. The study found that a relationship did not exist between the value-added scores and collective teacher efficacy. Additionally, the relationship between value-added scores and several other variables was examined. The socio-economic status of the building was found to serve as a moderator for collective teacher efficacy. Finally, this study provides administrators with examples of how to boost collective teacher efficacy and in turn improve teacher effectiveness.

Value-added assessment (VAA) systems use statistical techniques to analyze test-score data; VAA data is intended to help educators make more informed decisions about curriculum and instruction. The authors examined the rollout of Pennsylvania's VAA program, and found that, in its pilot phase, the program had little effect on student achievement and received limited use by most principals and teachers at schools participating in the program.

The purpose of this qualitative study was to investigate teachers' perceptions of their experiences with the Pennsylvania Value-Added Assessment System (PVAAS) and its use in teacher evaluation. Participants were fourth through sixth grade regular education classroom teachers in five school districts in Central Pennsylvania. Thirty-one teachers participated in the survey portion of the study. The survey was comprised of multiple-choice, Likert scale, and open-ended questions. Additionally, five teachers participated in the interview portion of the study, which required them to answer eight interview questions. An examination of the data gathered revealed that participants had an overall negative perception of PVAAS. A majority of participants did not view themselves as having a thorough understanding of PVAAS and desired additional professional development on the system. In regard to PVAAS in teacher evaluation, a majority of participants were not supportive of the use of PVAAS as a measure in teacher evaluation expressing skepticism regarding the ability of PVAAS to adequately measure a teacher's contribution to student learning.

Act 82 of 2012 established new standards for Pennsylvania's teacher evaluation system, including the incorporation of student performance measures in ratings decisions. Since 2009, approximately 35 states have amended teacher evaluation systems, with student achievement playing an increasingly prominent role. This count includes neighboring states--such as New Jersey, New York, and Ohio--which base between 40 and 50 percent of teacher effectiveness ratings on student achievement. Changes to teacher evaluation policy have been motivated in large part by U.S. Department of Education's priorities, including the issuance of waivers from certain "No Child Left Behind" requirements. States receiving a waiver, including Pennsylvania, are required to "develop and implement teacher and principal evaluation and support systems that include student achievement growth as a factor." The nonpartisan Center on Education Policy reported that ten states amended their plans for a new teacher evaluation system due to the waiver policy alone.v Likewise, the Race to the Top (RTTT) competition emphasized teacher and principal evaluation systems based on student achievement. This policy brief provides a closer look at Pennsylvania's new teacher evaluation system and the efforts of the Pittsburgh Public Schools--the state's second-largest district and an early adopter of revised evaluation standards--to implement reforms. It concludes with implications for state policymakers, district leaders, and education stakeholders.

At the request of Pittsburgh Public Schools (PPS) and the Pittsburgh Federation of Teachers (PFT), Mathematica is developing value-added models (VAMs) that aim to estimate the contributions of individual teachers, teams of teachers, and schools to the achievement growth of their students. The analyses described in this report are intended as an early step in a multi-year project that aims to produce fair, valid, reliable, and robust estimates of the contributions of individual teachers and schools in Pittsburgh to the achievement growth of their students, ultimately including as many grades, subjects, schools, and teachers as possible. In addition to scores on the Pennsylvania System of School Assessment (PSSA), this report also applies VAMs to Pittsburgh's own curriculum-based assessments (CBAs) in various subjects and courses in grades 6-10; to PSAT assessments conducted in grades 10 and 11; and to Scholastic Reading Inventory (SRI) assessments in grades 6 to 10. Composite value-added measures that incorporate results for each of the assessment outcomes relevant to each school were also created. In addition, school-level VAMs are estimated for the non-test outcomes of student attendance and credit accumulation. This report describes the methods for measuring teacher and school effectiveness using those student outcomes and the findings from the authors' analyses to date. It describes in general terms the distribution of value-added estimates for teachers and schools across the district. The findings in this report reinforce the findings of the August report (Lipscomb, Gill, and Booker, 2010) that "VAM estimates can provide meaningful information about teacher and school performance in Pittsburgh--and they confirm that VAMs can be usefully applied not only to state accountability tests and nationally normed assessments, but also to locally developed assessments and to non-test measures of student outcomes such as attendance and credit accumulation." An appendix contains supplementary tables and figures.