In this policy brief we assess the associations between teacher credentials and measures of performance— value-added estimates and evaluation ratings—for early grades reading teachers. This research is particularly valuable given the grade-level coverage of traditional testing regimes and North Carolina’s recent focus on early grades reading proficiency. We find that: (1) 1st and 2nd grade teachers with National Board Certification (NBC), higher licensure exam scores, and a reading license have significantly higher value-added estimates on the mCLASS Reading 3D exam; (2) more experienced teachers and teachers with graduate degrees have value- added estimates comparable to those of their peers; and (3) teacher experience, NBC, higher licensure exam scores, holding a reading license, and having a graduate degree all predict significantly higher evaluation ratings on the North Carolina Educator Evaluation System (NCEES). Overall, these results are an impetus for continued research to better understand the knowledge and skills underlying early grades teacher performance and to contribute to policy and staffing decisions.
Introduction
Like many states, North Carolina’s testing regime— End-of-Grade (EOG), End-of-Course (EOC), and final exams—has traditionally focused on students in upper elementary, middle, and high school grades. Research with these data has consistently demonstrated the importance of teachers to student achievement and has shown that certain teacher credentials, including experience, National Board Certification (NBC), and preparation are associated with teacher effectiveness. In recent years, North Carolina has extended its testing regime into early grades (K-2) with a suite of reading assessments, the mCLASS Reading 3D, designed to measure students’ foundational reading skills (e.g. phonemic awareness), reading fluency, and comprehension. Given the state’s current policy focus on reading proficiency and the relationships between early grades academic performance and later schooling outcomes, it is imperative that we know which teacher characteristics predict teacher performance in early grades. Therefore, in this policy brief, we assess the extent to which policy relevant teacher credentials predict the performance— measured by teacher value-added estimates and evaluation ratings—of early grades reading teachers. We hope this research propels further work on student achievement and teacher performance in early grades and can inform state and district policy and staffing decisions.
Background
In 2012 the North Carolina General Assembly created Read to Achieve as part of the Excellent Public Schools Act. Read to Achieve is an intensive focus on early grades reading with two main components: (1) formative and on-going assessments of students’ reading fluency and comprehension to aid instruction and target reading interventions and (2) eliminating social promotion by requiring 3rd grade students to be proficient on the EOG reading exam or an alternate assessment. To meet Read to Achieve’s requirement of a formative reading assessment,the State Board of Education adopted mCLASS Reading 3D in 2012, provided trainings on the instrument in 2012-13, and rolled out the assessment, statewide, in 2013-14. mClass Reading 3D is a diagnostic reading tool that includes DIBELS, a series of short assessments to measure foundational reading skills, and Text Reading and Comprehension (TRC), an assessment to measure reading fluency and comprehension. Early grades students complete the mCLASS Reading 3D with their classroom teacher or another member of the school staff at the beginning, middle, and end of the school year.1 Beginning in 2014- 15, teachers receive a mCLASS Reading 3D value-added estimate (EVAAS score) based on students’ gains on the TRC portion of the exam.2
In this policy brief we focus on the mCLASS Reading 3D value-added scores of 1st and 2nd grade teachers in the 2014-15 school year. Additionally, we examine these teachers’ evaluation ratings on Standard 1 (Leadership) and Standard 4 (Facilitating Student Learning) of the North Carolina Educator Evaluation System (NCEES).
In analyses, we estimate the associations between these measures of teacher performance and the following policy- relevant credentials: teacher experience, graduate degree status, NBC, licensure exam scores, and having a license in reading. In select models, focused on early-career teachers, we also assess associations with teacher preparation routes. We consider these credentials given their connections to teacher pay in North Carolina and their associations with teacher value-added in previous research. To provide a fuller context for our teacher credential results, we compare value-added and evaluation rating outcomes for 1st and 2nd grade teachers with those of upper elementary grades (3-5) teachers. This allows us to assess how relationships between teacher performance and credentials may vary across performance measures—mCLASS Reading 3D, EOG reading exams, NCEES—and grade levels.
Table 1 provides descriptive characteristics on the elementary grades teachers in our sample, with a particular focus on teachers with a license in reading. Overall, 13 percent of 1st and 2nd grade teachers and 11 percent of 3rd-5th grade teachers have a reading license. Across elementary grades, teachers with a reading license are more likely to have been prepared at a UNC system institution, have more years of experience, and are more likely to have a graduate degree or NBC. Teachers with and without reading licenses work in schools with comparable performance composites and percentages of economically- disadvantaged and racial/ethnic minority students.
Do credentials predict the value-added of early grades teachers in reading?
To estimate the associations between teacher value-added and teacher credentials, we use teachers’ EVAAS scores from the mCLASS Reading 3D exam for teachers of grades 1-2 and the EOG reading exam for teachers of grades 3-5. We standardize these EVAAS scores, within grade, such that we can interpret the reported coefficients as a percent of a standard deviation in teacher effectiveness (effect size). Our primary models compare across teachers, statewide; additional specifications compare teachers to other teachers working in the same schools. All models include our focal teacher credentials and controls for classroom (e.g. average days absent, percent economically-disadvantaged students) and school characteristics.
Figure 1 displays credentials results for teachers in grades 1-2 and 3-5. In early grades (1-2), NBC status, higher licensure exam scores (standardized), and having a reading license all predict significantly higher EVAAS estimates for the mCLASS Reading 3D exam. For example, 1st and 2nd grade teachers who are NBC have value-added estimates nearly one-tenth of a standard deviation higher than their non-NBC peers. Likewise, 1st and 2nd grade teachers with a reading license have value-added estimates one-tenth of a standard deviation higher than teachers without a reading license. These credential estimates decrease in magnitude but remain statistically significant when comparing teachers within schools.
Figure 1 indicates that there is not a strong or consistent relationship between teacher experience and teacher value-added for the mCLASS Reading 3D exam. Conversely, teacher experience is the only credential that significantly predicts teacher effectiveness on the EOG reading exam in grades 3-5. Teachers with NBC, a graduate degree, higher licensure exam scores, or a reading license perform comparably to their peers without these credentials in upper elementary grades. Lastly, in models (not presented) limited to teachers with less than five years of experience, there are no statistically significant differences in effectiveness for 1st and 2nd grade teachers based on preparation type. In this sample of early-career 1st and 2nd grade teachers, however, the association between value-added and having a reading license is statistically significant and nearly twice as large as in the full teacher sample.
Do credentials predict the evaluation ratings of early grades teachers?
Since nearly all teachers receive evaluation ratings on Standard 1 (Leadership) and Standard 4 (Facilitating Student Learning) of the NCEES, we assess the associations between these two measures and our focal teacher credentials.3 The outcome variable for these models is ordinal—ratings of developing (2), proficient (3), accomplished (4), or distinguished (5)— so we estimate ordered logistic regressions controlling for the same teacher credential, classroom, and school characteristics as in our EVAAS analyses. Our primary specification is a multi-level model with teachers nested within schools.
Table 2 displays odds ratios for earning higher evaluation ratings for teachers in grades 1-2 and 3-5. Unlike the value-added analyses, these teacher credential results are largely similar across grade levels. In elementary grades, teachers with more experience, NBC, graduate degrees (except for 1st and 2nd grade teachers on Standard 4), higher licensure exam scores, and reading licenses are more likely to earn higher evaluation ratings.
To better convey the magnitude of these differences, Figure 2 presents predicted probabilities for 1st and 2nd grade teachers rating at developing, proficient, accomplished, or distinguished on Standard 4 of the NCEES. Figure 2 shows that approximately 22 percent of first year teachers and 55 percent of teachers with 10 years of experience are predicted to earn evaluation ratings above proficient (accomplished or distinguished). Sixty-two percent of teachers with a reading license are predicted to earn ratings above proficient compared to 58 percent of teachers without a reading license. Finally, in models (not presented) limited to teachers with less than five years of experience, results indicate that UNC system prepared teachers in grades 3-5 earn significantly higher evaluation ratings than out-of-state prepared and alternative entry teachers. In this sample of early- career teachers, those with a reading license earn significantly higher evaluation ratings (particularly in 1st and 2nd grade).
Discussion
In this brief we assessed the extent to which policy relevant teacher credentials predict the performance of early grades teachers on the mCLASS Reading 3D exam and the NCEES. This work is particularly important given the grade-level coverage of traditional testing regimes and North Carolina’s recent focus on early grades reading proficiency. A limitation of this work is that teacher value-added estimates from the mCLASS Reading 3D are currently available in 2014-15 only; continued research should seek to replicate and extend these analyses with more data.
For 1st and 2nd grade teachers, we find that NBC, licensure exam scores, and having a reading license predict significantly higher value-added scores on the mCLASS Reading 3D exam. These credentials also predict significantly higher evaluation ratings on Standard 1 (Leadership) and Standard 4 (Facilitating Student Learning) of the NCEES. This positive NBC result is consistent with prior research on teacher credentials and value-added; it is unclear from these analyses whether NBC teachers were already more effective prior to certification or whether the certification process particularly benefits early grades reading instruction.4 Perhaps most interestingly, 1st and 2nd grade teachers with a reading license were more effective than their peers without a reading license. Given our data limitations (one year of mCLASS Reading 3D scores), we cannot test whether this result is attributable to teacher selection—more effective teachers decide to earn a reading license—or whether the process of earning a reading license benefits early grades reading instruction. Towards this end, however, we note that teachers earn a reading license in North Carolina by completing an undergraduate (add-on) program or graduate degree. These programs typically require 15-18 or 30-36 credit hours, respectively, and focus on topics such as early literacy (e.g. phonemic awareness and phonics), instructional strategies for reading comprehension, reading assessment and remediation, and children’s literature.
Finally, while teacher experience predicts significantly higher evaluation ratings, there is not a strong relationship between teacher experience and effectiveness for the mCLASS Reading 3D exam. This value-added result is contrary to previous research on teacher credentials and to our findings for the reading EOG exam in grades 3-5. There may be multiple factors behind this lack of returns to experience, however, we hypothesize that it is attributable to recent developments in reading instruction. Essentially, inexperienced teachers are more likely to have been exposed to relatively new and research-validated methods for teaching early grades reading during their teacher preparation. Taken together, these analyses should encourage further research to better understand the knowledge and skills underlying early grades teacher quality. Findings from this and future work can inform state and district policy and staffing decisions.
For more research on this topic:
Clotfelter, C.T., Ladd, H.F., & Vigdor, J.L. (2007). Teacher credentials and student achievement: Longitudinal analysis with student fixed effects. Economics of Education Review, 26(6), 673-682.
Croninger, R.G., Rice, J.K., Rathbun, A., & Nishio, M. (2007). Teacher qualifications and early learning: Effects of certification, degree, and experience on first-grade student achievement. Economics of Education Review, 26(3), 312-324.
Joshi, R.M., Binks, E., Hougen, M., Dahlgren, M.E., Ocker-Dean, E., & Smith, D.L. (2009). Why elementary teachers might be inadequately prepared to teach reading. Journal of Learning Disabilities, 42(5), 392-402.
Palardy, G.J. & Rumberger, R.W. (2008). Teacher effectiveness in first grade: The importance of background qualifications, attitudes, and instructional practices for student learning. Educational Evaluation and Policy Analysis, 30(2), 111-140.
(December 2016)
EPIC is an interdisciplinary team that conducts rigorous research and evaluation to inform education policy and practice. We produce evidence to guide data-driven decision-making using qualitative and quantitative methodologies tailored to the target audience. By serving multiple stakeholders, including policy-makers, administrators in districts and institutions of higher education, and program implementers we strengthen the growing body of research on what works and in which context. Our work is ultimately driven by a vision of high quality and equitable education experiences for all students, and particularly students in North Carolina.
http://publicpolicy.unc.edu/epic-home/
