Johns Hopkins Institute for Education Policy
June 3, 2016
Many states include students’ academic achievement, as measured by standardized tests, in their teacher-evaluation systems. IQ and standardized tests measure students’ cognitive skills, defined as the ability to perform higher mental processes of reasoning, remembering, understanding, and problem solving (Bernstein et al., 2007). Research shows that cognitive ability is highly correlated with school success and significantly contributes to future earnings (Cawley, Heckman, & Vytlacil, 2001; Green & Riddell 2003). At the same time, test scores and IQ are not predictive of success later life success. For example, adolescent test scores explain 17% of the variation in adult earnings, while IQ alone accounts for 7% (Bound, Brown, & Mathiowetz, 2011; Heckman & Kautz, 2012; Kautz et al., 2014). Educators and researchers have long suggested that certain other, non-academic, skills are also important factors in student success (Bandura & Schunk, 1981; Ames & Archer, 1988; Zimmerman, 1990). The problem has been defining and measuring these so-called non-cognitive skills.
What do we know about non-cognitive skills? How do they relate to cognitive skills? How are they measured? Can they be learned, and if so, how? These questions are critical and timely: the Every Student Succeeds Act (ESSA), signed into law in December 2015, requires states to include at least one non-academic measure in their accountability systems and provides a federal funding stream to enable new models to be tested in the field.
What are non-cognitive skills? Why are they important?
Non-cognitive skills may be broadly defined as personality traits or “patterns of thought, feelings, and behavior” (Borghans et al., 2008), but the specific set of skills referenced depends upon the field of study. For example, psychologists classify non-cognitive skills in terms of the “Big Five” categories: openness to experience, conscientiousness, extraversion, agreeableness, and neuroticism (Bernstein et al., 2007). Educators tend to focus on non-cognitive skills that are directly related to academic success. For example, the University of Chicago’s Consortium on Chicago School Research concluded that the non-cognitive skills most strongly associated with academic performance are: academic behaviors (e.g. going to class and participating), academic perseverance (e.g. grit and self-discipline), academic mindsets (e.g. feeling a sense of belonging within an academic community and believing that ability and competence can grow with effort), learning strategies (e.g. metacognitive strategies and goal-setting), and social skills (e.g. interpersonal skills and cooperation) (Farrington et al., 2012). Thus, there are a number of different skills, both affective (such as “growth mindset”) and also behavioral (such as regular school attendance), that fall within the broad category known as non-cognitive skills.
Cognitive and non-cognitive skills develop differently. Non-cognitive skills develop across a person’s lifetime and do not peak until late adulthood. For example, under normal circumstances, conscientiousness expands from childhood until one’s fifties or sixties. This is in contrast to IQ, which peaks in late adolescence and then declines (Borghans et al., 2008). The educational implications of these developmental differences are clear: while the malleability of non-cognitive skills makes remediation possible and promising, their importance makes early-childhood cultivation the ideal (Kautz et al., 2014).
Non-cognitive skills reinforce cognitive skills. They can be measured independently, but they are also interdependent. Students with stronger non-cognitive skills also demonstrate higher academic achievement throughout the schooling process (Gabrieli, Ansel, & Krachman, 2015). For example, IQ test scores are determined by a person’s inborn intelligence and also by her ability to focus and self-motivate (Borghans et al., 2008). Students’ academic behaviors, including regular school attendance, completing homework, and participating in class, are strongly related to measures of academic achievement, such as grades (Farrington et al., 2012). Thus, well-developed non-cognitive skills promote the development of cognitive skills.
More broadly, non-cognitive skills influence school performance and reach into adulthood – for better or worse. Studies across the fields of education, economics, and psychology indicate that non-cognitive skills predict a variety of adult outcomes, including academic achievement, employment, financial stability, criminal behavior, and health (Gabrieli, Ansel & Krachman, 2015). Researchers have also found that non-cognitive skills are more predictive of long-term outcomes than are test scores (Chetty et al. 2011; Heckman & Rubinstein 2001; Lindquist & Vestman 2011; Mueller & Plug, 2006).
How can schools develop students’ non-cognitive skills?
1.Teachers influence students’ non-cognitive skills…..
Teachers influence students’ academic skills (Rivkin, Hanushek, & Kain, 2005); they also affect students’ non-cognitive skills (Gershenson, 2016; Jackson 2012; Jennings and DiPrete 2010; Koedel 2008; Ruzek et al. 2014).
For example, Jackson (2012) found that teachers have important effects on non-cognitive skills – as measured by absences, suspensions, grades, and on-time grade progressions – that differ from their effects on cognitive skills. While the precise mechanism of how teachers affect non-cognitive skills is unknown, Jackson found that this effect is especially strong for English teachers.
Additional research corroborates this finding. For example, Ruzek et al. (2014) found that teachers influence their students’ motivation, as measured by mastery and performance achievement goals. Gershenson (2016) also found that teachers have important effects on student absences.
2. …..but teachers vary in their ability to do so effectively
Several studies show that teachers vary in their ability to enhance students’ non-cognitive skills. For instance, several economists studied Ecuadoran kindergarteners who had been randomly assigned to teachers. The researchers administered a number of assessments, including tests of children’s executive function, inhibitory control, working memory, capacity to pay attention, and cognitive flexibility. They found that a one standard deviation increase in classroom quality – essentially an estimate of a teacher’s effectiveness within one classroom – was associated with a 0.07 standard deviation growth in students’ executive function scores. They also found that a one standard deviation increase in classroom quality led to a 0.11 increase in language and math test scores. This suggests that teachers vary in their ability to increase student’s non-cognitive skills, although the effect sizes are smaller for non-cognitive than for cognitive skills (Araujo et al., 2016).
Education researchers Blazar and Kraft (2015) similarly observed that upper-elementary teachers have large effects on their students’ non-cognitive skills as measured by students’ self-reported behavior in class, self-efficacy in math, and happiness in class. In contrast to Araujo et al., Blazar and Kraft found teachers’ effects on these skills is similar in magnitude to teachers’ effect on cognitive skills as measured by math test scores.
Such results hint at a possible cause of such disparate findings: researchers are investigating different things. Blazar and Kraft found that teacher effects sizes are largest for their students’ happiness in class; they found smaller effect sizes for students’ self-efficacy and classroom behavior. Auraujo et al., in contrast, are estimating teacher effects on students’ executive function. The age of the students is another potential source for differences in effect size: teachers’ influence may vary according to their students’ grade levels. This highlights, again, the difficulty of interpreting research on non-cognitive skills: it all depends upon what, precisely, is being measured.
3. Developing students’ cognitive and non-cognitive skills requires teachers to use different capacities
There seems to be a weak correlation between a teacher’s ability to increase cognitive skills and to increase non-cognitive skills, which suggests that teachers need recourse to different techniques (Jackson, 2012; Blazar & Kraft, 2015; Gershenson, 2016).
Blazar and Kraft (2015) found that the correlation between a teacher’s effect on students’ math achievement and the same teacher’s effect on students’ reported feelings of self-efficacy in math is only 0.19. This weak relationship means that teachers who are effective in improving non-cognitive skills are not necessarily as effective in improving cognitive skills.
Other research corroborates this finding. Jackson (2012) found merely a weak correlation between a teacher’s ability to increase test scores and to improve students’ non-cognitive skills. Gershenson (2016) similarly found that the capacities teachers need to boost students’ attendance are different than those needed to increase students’ test scores, although the mechanisms are unclear.
What do we know about the precise teaching capacities that seem to be associated with improvements in students’ non-cognitive skills? While research in this area is limited, Blazar and Kraft (2015) found that the teachers whose students have the largest non-cognitive gains are demonstrably good at providing emotional support to their students and at organizing the classroom. The authors also found a negative relationship between the mathematical errors in a teacher’s lessons and his students’ happiness and levels of self-efficacy in math class.
These findings have important implications for how teachers are evaluated. A teacher who is skilled at developing students’ grit and self-determination, but poor at increasing their test scores, might be rated as ineffective, thus undervaluing her contribution to student learning. Which is more important? If these skills are not mutually exclusive, how can a school system reinforce the importance of both?
4. The long-term effects of non-cognitive skills are not fully understood
Research also sheds light on the limitations of our understanding about measuring, promoting, and discouraging particular non-cognitive traits. Economists Papageorge, Ronda, and Zu (2016) found that externalizing behavior increases student misbehavior and reduces educational attainment for school children – a negative outcome. However, externalizing behavior has positive effects when the same children are in the workforce: students who displayed more externalizing behavior as children had higher earnings as adults. Policymakers therefore face significant challenges in determining which non-cognitive skills should be developed or discouraged in schools, how to measure them, and how to use the data within accountability systems.
The research above confirms the importance of non-cognitive skills for a variety of life outcomes, including academic attainment, career success, and social wellbeing. It further confirms that schools play an important role in the development of non-cognitive skills: teachers influence their students’ non-cognitive skills, for good or for ill.
It might seem to follow that districts and states should incentivize teachers to cultivate their students’ non-cognitive skills. Our limited understanding of non-cognitive skills, however, means that they are constrained in going about this work.
First, it is unclear which non-cognitive skills are the most important to develop in students, and how best to measure them. The research above employs a variety of access points that range from absences and school suspensions to student survey responses about their happiness during class. There is no scholarly consensus on what and how to measure.
Some measures of non-cognitive skills appear relatively easy for school districts to use rigorously. For example, there is strong evidence that a student’s exhibiting negative behaviors such as high suspension, absence, and failure rates, is highly predictive of high school drop-out. These data points serve as effective early warning indicators; Balfanz et al. (2007) found that missing more than 20% of school, failing math or English, and receiving an out-of-school suspension in sixth grade, predicted roughly 60% of the students who dropped out of high school. Research in Chicago Public Schools found that 9th graders who were “on-track”, i.e. had accumulated enough credits to be promoted to 10th grade, were four times more likely to graduate from high school than those who had not (Allensworth & Easton, 2005, 2007). School districts already collect these data, making their use in accountability systems relatively straightforward.
Other non-cognitive skills – such as self-control, self-regulation, persistence, and academic confidence – are not as easily measured. While it is generally agreed that these sensibilities are important, some believe that these skills can be accurately and meaningfully measured (Kautz et al., 2014), while others do not (Duckworth & Yeager, 2015). A primary concern is that some scholars hold that the only feasible way of measuring these skills is by eliciting students’ ideas or beliefs through self-reported questionnaires (Kautz et al., 2014). Other scholars express concern about the validity of self-reported measures and reference bias. Furthermore, there is little research about how student responses would be affected if they were used for evaluative purposes or within a high-stakes environment. Despite these concerns, some school districts, such as California’s CORE districts (nine school districts that serve more than one million students), are beginning to implement such measures. They will publicly report information from four non-cognitive measures, which will comprise 8% of schools’ performance rating for the 2015-2016 school year (West, 2016).
Once researchers have agreed upon which non-cognitive skills are most important to develop in schools and the best way to measure them, the next pressing question becomes how schools should support them. While research is fairly clear that teachers have major impacts on students’ cognitive learning (Clotfelter, Ladd, & Vigdor, 2010) and on non-cognitive skills, these two outcomes of interest seem to require different teaching skills.
Researchers are attempting to find effective ways for teachers to develop their students’ non-cognitive skills. Between 2002 and 2013, the Institute of Education Science funded more than 245 projects that focused on social-behavioral competencies (Yamaguchi & Hall, 2016). This research included instructional interventions, professional development programs, educational technologies, and assessments. The initial findings are promising, although there is currently no consensus about which strategies are, in the end, the most effective (Yamaguchi & Hall, 2016; Farrington et al., 2012).
In sum, research has established that non-cognitive skills are important for student outcomes; that teachers play an important role in improving these skills within their students; and that the requisite educator capacities are different than those needed for improving students’ academic achievement. However, which non-cognitive skills should be targeted, how best to measure them, and how to prepare teachers to nurture them, remain open questions. As states consider which non-academic factors to include in their accountability systems under ESSA, they have reason to hesitate before jumping on the non-cognitive bandwagon at this point in time.
Allensworth, E. M., & Easton, J. Q. (2005). The on-track indicator as a predictor of high school graduation. Chicago: Consortium on Chicago School Research.
Allensworth, E. M., & Easton, J. Q. (2007). What matters for staying on track and graduating in Chicago Public High Schools. Chicago, IL: Consortium on Chicago school research.
Ames, C., and Archer, J. (1988) Achievement goals in the classroom: Students’ learning strategies and motivation processes. Journal of Educational Psychology, 80(3), 260-267
Araujo, M. C., Carneiro, P. M., Cruz-Aguayo, Y., & Schady, N. (2016). Teacher Quality and Learning Outcomes in Kindergarten. Quarterly Journal of Economics. Forthcoming.
Balfanz, R., Herzog, L., & Mac Iver, D. J. (2007). Preventing student disengagement and keeping students on the graduation path in urban middle-grades schools: Early identification and effective interventions. Educational Psychologist, 42(4), 223-235.
Bandura, A., and Schunk, D.H. (1981) Cultivating competence, self-efficacy, and intrinsic interest through proximal self-motivation. Journal of Personality and Social Psychology, 41, 586-598.
Bernstein, D., L. Penner, A. Clarke-Sterwart and E. Roy (2007) Psychology, 8th edition, Wadsworth Publishing
Blazar, D., & Kraft, M. A. (2015). Teacher and Teaching Effects on Students’ Academic Behaviors and Mindsets (No. c1c4216bd08f408fb6bdf16a3f536b40). Mathematica Policy Research.
Borghans, Lex, A. Duckworth, J. Heckman, and B.ter Weel. (2008). “The Economics and Psychology of Personality Traits.” Journal of Human Resources, vol. 43, no. 4, 972–1059.
Cawley, J., J., Heckman& E. Vytlacil (2001). Three observations on wages and measured cognitive ability. Labour Economics, 8(4), 419-442.
Chetty, R., J.N. Friedman, N. Hilger, E. Saez, D. Schanzenbach, and D. Yagan. (2011). How Does Your Kindergarten Classroom Affect Your Earnings? Evidence from Project STAR. Quarterly Journal of Economics, vol. 126, no. 4, pp. 1593–1660
Clotfelter, C. T., Ladd, H. F., and Vigdor, J. L. (2010). Teacher credentials and student achievement in high school: A cross-subject analysis with student fixed effects. Journal of Human Resources, 45(3), 655-681.
Duckworth, A. L., & Yeager, D. S. (2015). Measurement matters assessing personal qualities other than cognitive ability for educational purposes. Educational Researcher, 44(4), 237-251.
Farrington, C.A., Roderick, M., Allensworth, E., Nagaoka, J., Keyes, T.S., Johnson, D.W., & Beechum, N.O. (2012). Teaching adolescents to become learners. The role of non-cognitive factors in shaping school performance: A critical literature review. Chicago: University of Chicago Consortium on Chicago School Research
Gabrieli, C., Ansel, D., and Krachman, S. (2015). Ready to Be Counted: The Research Case for Education Policy Action on Non-Cognitive Skills. Retrieved from: https://goo.gl/7e8Kkg
Garcia, E. (2014). The Need To Address Non-cognitive Skills in the Education Policy Agenda. EPI Briefing Paper #386. Economic Policy Institute. Retrieved from: http://goo.gl/i3kFV2
Gershenson, S. (2015). Linking teacher quality, student attendance, and student achievement. Education Finance and Policy.
Green, D. A., & Riddell, W. C. (2003). Literacy and earnings: an investigation of the interaction of cognitive and unobserved skills in earnings generation. Labour Economics, 10(2), 165-184.
Heckman, James J., and Yona Rubinstein. (2001). “The Importance of Non-cognitive Skills: Lessons from the GED Testing Program.” The American Economic Review, vol. 91, no. 2, 145–149.
Jackson, C. K. (2012). Non-cognitive ability, test scores, and teacher quality: Evidence from 9th grade teachers in North Carolina (No. w18624). National Bureau of Economic Research.
Jennings, J.L., and T.A. DiPrete. (2010) “Teacher Effects on Social and Behavioral Skills in Early Elementary School.” Sociology of Education, vol. 83, no. 2, pp.135–159.
Kautz, T., Heckman, J. J., Diris, R., Ter Weel, B., & Borghans, L. (2014).Fostering and measuring skills: Improving cognitive and non-cognitive skills to promote lifetime success (No. w20749). National Bureau of Economic Research.
Koedel, C. (2008). “Teacher Quality and Dropout Outcomes in a Large, Urban School District.” Journal of Urban Economics, vol. 64, no. 3, pp. 560–572.
Lindqvist, E., and R. Vestman. 2011. “The Labor Market Returns to Cognitive and Non-cognitive Ability: Evidence from the Swedish Enlistment.” American Economic Journal: Applied Economics, vol. 3, no. 1, 101–128.
Mueller, G., and E. Plug. (2006) “Estimating the Effect of Personality on Male and Female Earnings.” Industrial & Labor Relations Review, vol. 60, no. 1, pp. 3–22.
Olson, S. (2012). From Neurons to Neighborhoods: An Update: Workshop Summary. Washington, D.C.: National Academies Press.
Papageorge, N. W., Ronda, V., & Zheng, Y. (2016). The Economic Value of Breaking Bad: Misbehavior, Schooling and the Labor Market. Draft (April 8, 2016).
Rivkin, Steven G, Eric A Hanushek, and John F Kain. (2005). “Teachers, schools, and academic achievement.” Econometrica 73 (2):417–458.
Ruzek, E.A., T. Domina, A.M. Conley, G.J. Duncan, and S.A. Karabenick. (2014) “Using Value-Added Models to Measure Teacher Effects on Students’ Motivation and Achievement.” The Journal of Early Adolescence, vol. 35, nos. 5–6, pp. 852–882.
West, Martin. (2016) Should Non-Cognitive Skills Be Included in School Accountability Systems? Preliminary Evidence from California’s CORE Districts. Brookings: Evidence Speaks Series. Retrieved at: http://www.brookings.edu/research/reports/2016/03/17-non-cognitive-skills-school-accountability-california-core-west
Yamaguchi, R., and Hall, A. (2016). A Compendium of Social-Behavioral Research Funded by NCER and NCSER: 2002-2013 (NCER 2016-2002) Washington, DC: National Center for Education Research, Institute of Education Sciences, U.S. Department of Education. Available at: http://ies.ed.gov/.
Zimmerman, B.J. (1990) Self-regulated learning and academic achievement: An overview. Educational Psychologist, 25(1), 3-17.