Abstract
Does school climate ameliorate or exacerbate the impact of neighborhood violent crime on test scores? Using administrative data from the New York City Department of Education and the New York City Police Department, we find that exposure to violence in the residential neighborhood and an unsafe climate at school lead to substantial test score losses in English language arts (ELA). Middle school students exposed to neighborhood violent crime before the ELA exam who attend schools perceived to be less safe or to have a weak sense of community score 0.06 and 0.03 standard deviations lower, respectively. We find the largest negative effects for boys and Hispanic students in the least safe schools, and no effect of neighborhood crime for students attending schools with better climates.
Despite the well-documented drop in violent crime in American cities, violence is part of daily life for many children. A growing body of research shows that exposure to neighborhood violence negatively affects academic performance, particularly among children living in high crime neighborhoods (Burdick-Will et al. 2011; Sharkey 2010; Sharkey et al. 2014; Schwartz et al. 2016). Whereas police, government officials, and civic organizations seek to reduce crime, schools can play a role in mitigating the negative effect of exposure to violence on students. Schools vary along many dimensions, including academic quality, student body characteristics, and school climate and environment. In some schools, disorder and conflict may contribute to feelings of fear and vulnerability among students; in other schools, students feel safe and supported. As reported in an article about New York City schools in the New York Times, “[school name] is more than just a place to learn algebra and history. A public middle school, it is seen by many families as a safe zone in a crime-plagued neighborhood” (Hu 2014). School climate, including how safe, orderly, and welcoming a school is perceived to be, may affect how youth are able to cope with traumatic events at home or in the residential community. Factors outside of school influence student success, yet little is known about whether school climate moderates the effects of these external events. In this article, we focus on the relationship between school climate and neighborhood crime and answer the question of whether school climate ameliorates or exacerbates the impact of neighborhood crime on academic performance.
To answer this question, we combine detailed administrative and survey data on neighborhood violent crime, student achievement, and student perceptions of school climate. Administrative data are key to our analysis. First, student-level data from the New York City Department of Education allow us to track test scores for the universe of public middle school students in New York City over time and observe their demographic characteristics and residential addresses. Second, incident-level crime data from the New York City Police Department (NYPD), which we geocode to individual street segments throughout the city, provide us with a daily measure of violence occurring on the blockfaces where those students reside. Finally, we use responses to an annual survey that the New York City Department of Education administers to all middle and high school students to construct measures of school climate that we link with students’ school records. This linkage provides us a unique look at how the impact of violence varies depending on school climate.
Our empirical approach capitalizes on the exogenous exposure of students to violent events in their neighborhood relative to the timing of standardized exams to estimate the causal acute effect of exposure to neighborhood violence on student outcomes (see Sharkey 2010; Sharkey et al. 2014). Within this framework, we examine how the acute effect of neighborhood crime varies with school climate, measured in different ways. We use factor analysis to combine middle school (grades six to eight) student responses to the New York City Learning Environment Survey and create three school-level scales that capture key constructs of school climate: school safety, disorder, and sense of community. We divide schools into quartiles for each of these dimensions, and estimate how the acute effect of neighborhood violence on English language arts (ELA) and mathematics test scores differs across schools with different climates.
To summarize our findings: students suffer declines in standardized test scores following exposure to a violent crime if they attend schools perceived as unsafe or having a weak sense of community. Specifically, middle school students exposed to violent crime before the test who attend schools that are less safe or have a weak sense of community score 0.06 and 0.03 standard deviations lower in the ELA exam, respectively. Students attending the schools perceived as being the safest, the least disorderly, or having the strongest sense of community suffer no visible reduction in test performance when exposed to violent crime, suggesting that schools with stronger climates might insulate students from the negative effects of neighborhood violence.
LITERATURE REVIEW
Living in a disadvantaged and dangerous neighborhood affects the lives of young people along multiple dimensions, including their health, education, and employment. A growing body of research highlights the effect of exposure to neighborhood violence on the academic attainment and achievement of students (Burdick-Will et al. 2011; Harding 2009; Rendón 2014; Sharkey 2010). In New York City, exposure to a violent assault or homicide in the week before a standardized exam decreases achievement in ELA relative to students who are exposed in the week after the exam (Sharkey et al. 2014).
School climate may also affect the academic performance of students (Thapa et al. 2013). Exposure to violence at school reduces attendance, decreases test scores, increases misbehavior, and reduces the likelihood of high school graduation and college attendance (Bowen and Bowen 1999; Burdick-Will 2013; Grogger 1997). Being the victim of an attack at school is associated with increased student misbehavior and declines in grades (Patton, Woolley, and Hong 2012). Even witnessing violence at school has consequences for student conduct, attitudes about school, and attendance (Janosz et al. 2008). School violence does not have to be extreme to have negative effects on students. Exposure to disorder in school, such as bullying, is negatively related to achievement and is also associated with more serious school violence (Arseneault, Bowes, and Shakoor 2006; Chen 2007; Juvonen, Wang, and Espinoza 2011). Schools may also be able to insulate students from violent neighborhoods by becoming a safe haven from the surrounding violence and disorder (Patton, Woolley, and Hong 2012).
The evidence suggests that four primary dimensions of school climate are likely to influence student performance: school-based violence and disorder, school safety, school discipline, and sense of community within the school. Specifically, in New York City, feeling unsafe at school decreases the academic achievement of middle school students, and the largest effects are found in schools with the most school-based violence (Lacoe 2016).
School disciplinary policy and student perceptions of the fairness of school discipline may also affect achievement. At the school level, the suspension rate is correlated with the share of students who pass competency exams (Raush and Skiba 2004). Youth who are suspended struggle to make academic progress over time and are more likely to drop out of high school than their peers who are not suspended (Arcia 2006; Lacoe and Steinberg 2018). Research has yielded mixed evidence of the efficacy of school security measures, such as metal detectors or police in schools. Some studies find these measures improve perceptions of school climate (Bhatt and Davis 2016). Others find decreases in perceived safety among students and increased involvement with the juvenile justice system (Theriot 2009). Therefore, school disciplinary policies that take zero tolerance approaches emphasizing out-of-school suspensions, or school security measures, may also affect student achievement if they make students feel less safe. Finally, the degree to which students feel connected to their school and a sense of belonging at school can affect their academic achievement. For instance, characteristics of the school environment influence students’ level of engagement and participation in school, which in turn may affect their academic achievement (Wang and Holcombe 2010).
These dimensions of school climate may affect youth differently depending on their racial and ethnic background or gender. Johanna Lacoe finds that African American and Hispanic middle school students are more likely to report feeling unsafe in the classroom and on school grounds than white and Asian peers who attend the same schools (2015). Patrick Sharkey and his colleagues find that neighborhood violence has the most pronounced effect on the achievement of black students, with little effect on Hispanic students, despite similar rates of exposure to neighborhood violence between the two groups (2014). Studies also suggest that boys and girls respond differently to school climate and neighborhood violence as well (Harding 2009).
In sum, the literature shows that community violence can be detrimental to students’ academic success. Further, research also suggests that the climate within a school (including safety, disorder and support levels) can shape students’ academic performance. This article bridges these two literatures to investigate whether and to what extent school climate moderates the effect of neighborhood crime on middle school students’ test scores.
THEORY
Several theorists have put forth models and frameworks to describe how multiple environments affect youth outcomes (Bronfenbrenner 2004; Eccles and Roeser 2011; Kirk 2009). In particular, Jacquelynne Eccles and Robert Roeser describe schools as prime developmental contexts for youth during adolescence. Schools are organizations with customs, norms, and rules that influence student interactions, learning, and development on a daily basis (Eccles and Roeser 2011). Schools, however, are not located in a vacuum, but instead are intimately connected to the surrounding neighborhoods, which for most middle school students are where they live. “By attending to the social structure of community conflict, whatever its spatial form, schools can understand, and possibly anticipate, the development of violent confrontations and possibly intervene to redirect the conflict to some other outcome” (Mateu-Gelabert and Lune 2003, 366). School climate may dictate how successfully schools manage youth responses to violence, distinguishing some schools as safe havens.
Building on this literature, our theoretical model in figure 1 describes how schools may play a moderating role in the relationship between exposure to community violence and educational outcomes. Schools may moderate this relationship if school factors change the magnitude or direction of the impact of violent crime on test scores (that is, insulate students from the full effect of exposure, or exacerbate the response to violence).
Conceptual Model of the Role of School Climate
The primary school climate factors that may affect the relationship between neighborhood violent crime and academic outcomes are school-based violence and disorder, perceptions of safety, discipline, and school supports. The direction of the relationship, however, is unknown. Exposure to violence everywhere (at school and at home) may desensitize students and lessen the effects of neighborhood violence on outcomes. That is, for example, the effects of exposure to neighborhood crime on children attending schools that are perceived as unsafe or disorderly may be smaller or nonexistent. Alternatively, exposure to violence or disorder at school may compound the effect of violence in students’ home neighborhoods, causing them to perform poorly on exams, so that we would observe the largest test score losses after exposure to violent crime for children who feel unsafe at school. In contrast, if the school represents a safe haven from a violent home neighborhood, the effect of exposure to violence may be smaller.
Other aspects of the environment can also shape effects of neighborhood violence. If the disciplinary environment is strong and effective, it may support students who feel at risk in their home neighborhood. Likewise, a supportive, inclusive, and friendly school environment may insulate students from the negative effects of exposure to violence in their neighborhood. Conversely, if students feel little sense of community at the school or view the disciplinary environment as unfair or biased, then their academic performance may be more affected by violence they observe or experience outside of the school.
DATA AND MEASURES
We exploit three detailed sources of administrative data. First, we use point-specific crime data from the NYPD on daily violent crime—homicide and aggravated assault—occurrences in New York City from 2004 to 2010.1 The data contain the spatial coordinates of the crimes that we geocode to the blockface, a street segment between the two closest cross streets (figure A1), using ArcGIS.
Second, we use longitudinal student administrative records from the city’s Department of Education. This dataset contains a wide range of student demographic characteristics including student race-ethnicity, gender, participation in special education, and receipt of free or reduced-price meals. It includes test scores on the ELA and mathematics standardized tests, as well as the school students attend and their residential address in each academic year (as of October). Our measure of exposure to neighborhood violence captures the number of violent crimes that occur on students’ residential blockface each year they remain enrolled in New York City public schools.2 Students who live on the same blockface are considered to be exposed to the same crimes. Note that we do not know if the child actually witnessed the crime. However, because the blockface is such a small geographic unit and homicides and violent assaults are serious offenses, it is likely that students will have either direct or indirect knowledge of the crime.
Third, to obtain measures of school climate we use student answers to the Learning Environment Survey collected by the New York City Department of Education. The survey is administered annually to students, parents and teachers in grades six to twelve during the spring semester (between mid-February and mid-March). In this article, we focus on all middle school students’ responses to the survey. The survey started in 2007, the first year of our panel.
Our analytic sample contains 16,146 students in 533 schools in grades six to eight from academic years 2006–2007 to 2009–2010. We restrict the sample to students living in high poverty census tracts who are exposed to violent crime within one week—seven days—of the ELA test.3 We focus on high poverty census tracts because it allows us to exclude sections of the city that have high crime rates but that are relatively wealthy, such as midtown Manhattan.4 Further, as prior work shows (Sharkey et al. 2014), exposure to violence has larger effects on these students, and thus understanding the role of school climate for these more vulnerable students seems especially important. We also exclude students in charter schools, in ungraded special education, and those exposed to crime both before and after the test.
Constructing School Climate Measures
To construct school climate measures we take both a theory- and data-driven approach similar to that of Matthew Kraft, William Marinell, and Darrick Yee (2016). First, we review questions in the Learning Environment Survey and select those that capture the four dimensions of school climate identified in the literature as important determinants of student outcomes: safety, disorder, sense of community, and disciplinary environment. We identify seventeen survey questions in these domains. For example, we use questions about feelings of safety in classrooms, in hallways and locker rooms, and on school grounds outside the school building. We also select questions related to bullying, fighting, substance use, gangs, perceptions of disciplinary fairness, conflict resolution, and the presence of safety agents. Finally, we also use questions about whether students feel welcome at school, treat each other with respect, or just look out for themselves. All responses consist of a scale from one to four.5 We code all responses so that an answer of one in the survey would be the best outcome, an answer of two would be the second best outcome, an answer of three would be the third best outcome, and an answer of four would be the worst outcome.
We use exploratory principal components factor analysis to identify whether student responses capture one overall measure of school climate or map into distinct climate dimensions. We rotate the factor loadings using oblique rotation because it assumes correlation among the factors instead of treating them as exogenous, and we expect the different climate measures to be correlated with one another.6 For example, a school that students perceive as highly disorderly is also likely to be perceived as unsafe. Rotating factor loadings maximizes the loadings for each factor, facilitating interpretability. After rotation, we find that the responses to the survey questions map onto three factors that capture three dimensions of school climate: safety, sense of community, and disorder and conflict.7 Contrary to expectations, there is no separate dimension for the disciplinary environment. The first factor (safety) explains 28 percent of the item variance, the second factor (sense of community) approximately 23 percent, and the third factor (disorder) 22 percent. Table 1 presents the relevant survey questions in each factor with the corresponding factor loadings. Consistent with the literature, we show those with factor loadings of 0.4 or greater.8
Rotated Factor Loadings, School Climate Measures
To create school safety, disorder, and sense of community scales we compute factor scores for each student answering the surveys.9 To obtain school-level measures we follow Kraft, Marinell, and Yee and average those scores for each school across years to obtain time invariant school climate scales or indices (2016).10 The resulting scales are centered on zero. Higher values indicate weaker climates and lower values indicate stronger climates. Panel A of figure 2 plots the distribution of these scales across all the 593 schools in our data (including those not in high poverty census tracts). Overall, the disorder scale shows less variation: most values are concentrated around zero and below (less disorder) and no schools are perceived as having either very high levels or very low levels of disorder. Variation is greater in the sense of community measure, and the distribution is more skewed. Most schools are perceived to have a lower than average sense of community (indicated by positive values on the scale), but more schools are perceived as having very strong sense of community than as having a very weak sense. We observe a similar pattern for the school safety scale.
Distribution of School Climate Measures Across Schools
The school climate in the schools the students in our sample attend might differ from the full sample of schools, but we find they look fairly similar. Panel B of figure 2 shows the distribution of school climate measures for the 533 schools in the analytical sample and demonstrates that these distributions do not look substantially different from the full sample.11
EMPIRICAL STRATEGY
Our empirical strategy compares students exposed to violent crime within a one-week window around standardized testing dates. Specifically, we compare the test performance of students exposed to violent crime in the week before a standardized test with that of otherwise similar students exposed in the week after. The identifying assumption is that the occurrence of a violent crime on a student’s residential blockface one week before or after the test is conditionally random within this window. This strategy yields causal estimates of the acute effect of violent crime on test scores.
The empirical strategy relies on the assumption that students exposed to violent crime within a week of the ELA or math exam are very similar to each other. Students exposed to violent crime one week before and after the tests are quite similar demographically (see table A2). Hispanics make up the majority of students in all four samples (more than 50 percent), and black students represent approximately 40 percent. The samples are all evenly distributed between male and female students, and students receiving free or reduced-price lunch are overrepresented, as are students whose language at home is not English. To further test the assumption that students exposed to neighborhood crime before and after the test are similar, we estimate a regression model predicting exposure in the week before the ELA and math tests as a function of individual demographic characteristics. These models also include grade, year, and borough fixed effects. Results from a joint-F test on the demographic controls confirm that our sample is balanced, supporting the appropriateness of the identification assumption (table A2).12
We begin by estimating a baseline specification as shown in equation (1)
In this model, Test represents student i’s test score on the ELA or math exam, measured as a z-score standardized for each grade citywide with a mean of zero and a standard deviation of one. Crime takes a value of one if a student was exposed to a homicide or aggravated assault on their block in the week before the ELA test, and is zero if they were exposed in the week after the test. The coefficient of interest is β, and it can be interpreted as a causal estimate of the acute effect of exposure to violent crime. The model also includes a vector (X′) of student demographic controls: gender, race-ethnicity, eligibility for free or reduced-price lunch, special education, limited English proficiency, foreign born, home language not English, and over age for grade. Grade fixed effects are γg , year fixed effects are at , and eit is the usual error term. We follow this baseline specification by adding student i’s test scores lagged one year, thus controlling for a student’s prior performance.
To estimate whether school climate moderates the acute effect of neighborhood crime, we first divide schools into quartiles based on their scores on the three climate perception measures: safety, disorder, and sense of community. Schools in the first quartile are those with stronger climates (perceived as safe) and those in the fourth quartile are those with weaker climates (perceived as unsafe). Then, for each climate measure, we estimate our baseline model stratified by climate quartile. In this way, we compare the test performance of students exposed to violent crime on their block in the week before the test with those exposed in the week after attending schools that have similar climates. Specifically, we estimate as follows:
We extend our baseline specification by adding a set of interactions between the crime exposure indicator and each of the four climate quartiles. In this model, β yields estimates of the acute effect for each climate quartile by comparing students exposed before the test with those exposed after attending schools in the same quartile. For example, we compare the test performance of a student exposed to violent crime in the week before the ELA test with a similar student exposed in the week after, who both attend schools perceived as being the least safe (quartile four). If schools that are perceived as safer, or less disorderly, or with a stronger sense of community act as safe havens for students living in violent neighborhoods, we may see no difference in test performance between children exposed to homicides and violent assaults in the week before the test or after. Conversely, if schools with weaker climates (safety, disorder, and sense of community) exacerbate the effect of living in violent neighborhoods, we may see a decrease in performance after exposure to violent crime.
Schools with different climates may vary on a number of other characteristics that can influence both school climate and student performance. We examine the robustness of our main results through a series of tests. First, we add school-level, time-varying spending data, teacher-pupil ratio, and reported incidents of school violence to control for other school characteristics that might be correlated with student achievement or contribute to school climate. Second, we also estimate models with school fixed effects to control for time-invariant school characteristics that might be correlated both with perceptions of school climate and student achievement. Third, we conduct a falsification test using exposure to property crimes as our main crime exposure indicator. If students are, indeed, affected by neighborhood violent crime then we should see no effect of exposure to less serious property crimes on test scores. Finally, we test the robustness of our results using the full sample of students instead of the high poverty sample.
RESULTS
We begin by examining the demographic composition of schools with stronger and weaker climates for the three climate measures: safety, disorder, and sense of community. The most striking differences across quartiles concern the racial-ethnic composition of the students (figure 3, panel A). Black students are overrepresented in schools with weak climates (quartile four) across the three climate scales. Indeed, more than 50 percent of black students attend schools in quartile four across the three climate dimensions. In contrast, schools in quartile one are more than 60 percent Hispanic. We observe a similar pattern in schools with more mixed climates (quartiles two and three). Students who are white or Asian are the smallest group in the sample, and are also more likely to attend schools that are safer, less disorderly, and more community-oriented. As for gender, differences across quartiles are relatively small but girls are more likely to attend schools in the first quartile. This is a high poverty sample, thus more than 90 percent of students are eligible for free or reduced-price lunch, however, the percentage of poor students is slightly higher in quartile four schools than in the other three quartiles. Students whose home language is not English are less likely to attend schools with the weakest climates. In contrast, students who are over age for grade are overrepresented in quartile four schools across the three dimensions. Across all climate measures, schools are fairly similar in the percent of students with limited English proficiency, the share of foreign born, and those in special education.13
Student Characteristics, High Poverty Sample
Panel B of figure 3 shows differences for school spending categories: counseling, drug prevention programs, attendance and outreach, and school safety.14 Climate quartiles show no large differences for these school resources. In general, spending in counseling and attendance-outreach is lower in quartile one schools than in the other quartiles. As for other school resources (not shown), per pupil spending on classroom instruction is slightly higher in quartile three and four schools; pupil-teacher ratios and spending on school leadership are fairly similar across quartiles. Perhaps not surprisingly, school violence increases as we move from quartile one to quartile four schools across all dimensions.15
Does the Acute Effect of Violent Crime on Test Scores Vary with the School Climate?
In the regression results that follow we show models with demographic controls, and with lagged test scores. In most cases these results are very similar to each other and we show all of them for completeness. In the discussion, however, we focus on those that account for a student’s prior performance as the preferred specification. Results from our baseline regression (table 2) show no overall average acute effect of exposure to violent crime on ELA test scores for middle school students (grades six and seven). That is, middle school students exposed to violent crime in the week before the ELA test perform no differently on average than comparable students exposed after.16 This average effect, however, masks significant variation across schools as figure 4 shows, suggesting that school-level factors might play a role in ameliorating or exacerbating the acute effect. Indeed, when we explore whether results vary across school climate quartiles, we find that the average estimate conceals significant heterogeneity by school climate.
Regression Results: Exposure to Neighborhood Violent Crime and Test Scores, One-Week Window, High Poverty Sample
School-Specific Random Slopes, Violent Crime
Table 3 shows that exposure to violence affects students in schools with the weakest climates (quartile four). Students attending schools deemed the least safe (quartile four) score 0.06 standard deviations lower as a result of exposure to violent crime one week before testing (column 2). Similarly, column 6 shows students in schools that have the weakest sense of community (quartile four) score 0.03 standard deviations lower after exposure to violent crime in the week before the test (significant at the 10 percent level). We do not find strong evidence of an acute effect for students attending more disorderly schools. In models not accounting for prior performance (column 3), students exposed to violent crime prior to the ELA test attending schools perceived as the most disorderly (quartile four) score 0.08 standard deviations lower (significant at the 10 percent level). Once we control for prior performance, this coefficient drops to –0.04 and is no longer statistically significant.17
Regression Results: ELA, Exposure to Neighborhood Violent Crime, and School Climate Quartile
Results in this section indicate that safety and sense of community are the most critical elements of school climate in moderating the effects of violence. Our findings suggest that schools have the potential to insulate students from the negative effects of exposure to neighborhood violence on academic performance, shown by the absence of any negative effect on test scores for students attending schools with strong (quartile one) or mixed climates (quartiles two and three).18
Subgroup Analyses
Of course, school climate may not matter equally for all students; some students may be more sensitive to the climate of their school. In this section, we explore whether the moderating effect of school climate varies by gender and race-ethnicity. Gender analyses are motivated by the fact that boys and girls may use different coping mechanisms to deal with traumatic events, and may respond differently within these varied school climates (Rasmussen, Aber, and Bhana 2004). As for race-ethnicity, research has shown significant racial differences in sensitivity to violence. In particular, Sharkey and his colleagues find that the school performance of black students is especially sensitive to violent environments, and that exposure to violence has little effect on the academic performance of Hispanic students (2014). It is possible that differences in the school environments experienced by black and Hispanic students (and the concentration of black students in schools with weaker climates) explain these differences. In the analyses that follow, we focus on schools with weak climates (quartile four).19
Our results suggest that the negative effect of exposure to neighborhood crime is concentrated among boys attending weak climate schools. As panel A of table 4 shows, boys exposed to violent crime in the week before the ELA exam score approximately 0.10 standard deviations lower when they attend the least safe schools. They score 0.08 standard deviations lower (significant at the 10 percent level) in the most disorderly schools, and 0.04 standard deviations lower in schools with weaker sense of community (also significant at the 10 percent level). There is no observed effect for girls, but these coefficients are not statistically different than those for boys.
Regression Results: ELA, Quartile 4 Schools by Subgroup
Next, we explore differences by race and ethnicity. Due to sample size limitations, these models include only Hispanic and black students.20 Black students exposed to violent crime before the ELA test attending schools perceived to be the least safe, score 0.05 standard deviations lower on the ELA exam (table 4, panel B). The largest effect, however, is for Hispanic students in the least safe schools. These students score 0.09 standard deviations lower. This finding suggests that stronger school climates might offer some protective effect for Hispanic students. We observe no effect of exposure to violence on test scores among black and Hispanic students who attend more disorderly schools or schools with the weakest sense of community. Note that the samples in the disorder measure get very small, so we may be underpowered to detect an effect. Coefficients for black and Hispanic students are not statistically different from each other and, in general, results in this table are only significant at the 10 percent level.21
Robustness and Falsification Tests
The results so far show that school climate may play an important role in moderating the effects of neighborhood violence on student outcomes. However, there may be school-level confounders that bias these results. School climate measures may reflect differences in other factors that are correlated with student perceptions and achievement. To address this issue, we add several additional sources of administrative data to construct time-varying school-level controls. First, we combine our student-level data with the school-based expenditure reports. These data provide detailed information regarding school spending by budget item as well as pupil-teacher ratios. We select spending on classroom instruction, leadership, and relevant instructional support spending categories (counseling services, drug prevention programs, attendance-outreach, and school safety). Second, we add the rate of reported school violent incidents from the violent and disruptive incident reports data. As an additional test, we reestimate our models adding school fixed effects to control for time-invariant characteristics of schools that may be correlated both with school climate and student achievement.
In these models, our results remain largely unchanged (table A3). Students in the least safe schools exposed to violent crime in the week before the ELA test score 0.056 standard deviations lower than those exposed after. As for the other measures, results are not statistically significant, but point estimates are similar as our main specifications (–0.035 for disorder and –0.021 for sense of community). Taken together, these results support our finding that school climate matters for children exposed to violent crime, but we cannot completely rule out that other unobserved school-level factors might still be at play.
If violent crimes are more salient and the key source of stress for students and not simply capturing other things happening in the neighborhood, we should see little change in school performance after exposure to property crimes (for results of this falsification test, see table A4). Indeed, we find no evidence that exposure to property crime affects test scores, or that this effect varies with the climate of the school.
The primary results are estimated on a high poverty sample. To test the robustness of our findings we estimate the school climate specifications on the full sample of students. Overall, our conclusions are unchanged (table A5). Exposure to violent crime before the ELA test lowers test scores of students in schools deemed least safe by 0.04 standard deviations. This coefficient is smaller but within the confidence interval of the estimate for the high poverty sample. We also find that children exposed to neighborhood crime who attend schools that have a weak sense of community score 0.03 standard deviations lower in ELA (all significant at the 10 percent level).
Results by gender and race-ethnicity are also robust to using the full sample of students. Schools that are perceived as unsafe and having a weaker sense of community seem to exacerbate the negative effect of neighborhood crime on boys. Point estimates are smaller (–0.06 and –0.04, respectively) and significant at the 10 percent level only. As for race-ethnicity, results on the full sample are also consistent with the high poverty sample (see table B7 of the online appendix).22
Mathematics
So far we have only reported results for the ELA test because research shows the largest effects of community violence on reading and no effects on math (Sharkey et al. 2014; Schwartz et al. 2016). We also estimate our baseline model with math test scores as the outcome. The baseline specification—without stratifying the sample by school climate—shows no significant impact of crime exposure on math performance (table 2). Stratifying the sample by school climate quartile for each of the climate measures also yields no significant differences in test performance between students exposed to violent crime before the math test and students exposed after. That is, school climate does not moderate the effect of neighborhood violence on students’ math test scores, providing further evidence that community violence tends to affect performance in reading but not math (table 5).
Regression Results: MATH, Exposure to Neighborhood Violent Crime, and School Climate Quartile
DISCUSSION
This article investigates the role of school climate on the relationship between exposure to neighborhood violence and academic achievement for middle school students (grades six through eight) in New York City public schools. To do so, we leverage several sources of administrative data that provide advantages for this kind of analysis. Most notably, by using data on the entire city public school system we are able to generate more precise estimates than most of the literature using survey data with much smaller samples. As a result, we are able to focus our attention on very specific windows of time around public school assessments, and to make comparisons among students living in individual blockfaces within the city. By combining multiple administrative datasets, including student records, incident records from the NYPD, and school climate surveys, we are able to make progress in understanding the mechanisms by which violence in the residential environment affects students’ performance in school. Merging together multiple datasets that cover the entire city and all of its public schools allows for an analysis that would not be possible with any single source of data.
The results from our analysis provide a more nuanced picture of the impact of violence than shown by previous research. Overall, we find no significant acute effect of exposure to violence for the sample of sixth- to eighth-grade students in high poverty neighborhoods. This finding, however, masks the substantial variation in effects found in schools with different levels of disorder, safety, and sense of community. Schools with strong climates (across all dimensions—safety, disorder, and sense of community) and those with mixed climates (quartiles two and three) may insulate students from the negative effects of exposure to neighborhood violence. It follows that students experiencing decreases in ELA test scores after exposure to neighborhood violence are concentrated in schools with the weakest climates, particularly those perceived as the least safe. Specifically, students exposed to community violence before the test attending the least safe schools score 0.06 standard deviations lower, which amounts to 40 percent of the test score gap between poor and nonpoor students in our sample.23 For these students, attending a school with a weak climate further increases their academic disadvantage.
The analyses by race-ethnicity and gender uncover that the effect of exposure to violence is particularly salient for boys and Hispanic students in schools deemed the least safe. This last finding is interesting in light of previous research that found no effect of neighborhood crime on the test performance of Hispanic students (Sharkey et al. 2014). Indeed, it seems that although the majority of Hispanic students attend schools with strong climates, those in schools with weak climates see large declines in achievement following exposure to violence. Thus, the null average effect from previous studies obscured the finding that students in schools with weak climates may be particularly affected by neighborhood violent crime.
The magnitudes of the effects are significant. For example, a 0.10 standard deviation decrease in test scores for boys represents a 33 percent decline relative to the mean for boys in the sample. To put these numbers in context, the magnitude of this effect is comparable to the positive gains from school-level interventions such as class sizes (Chingos 2013).
We find no effects on math. This finding, while still puzzling, is not surprising. Previous work in New York City shows effects of neighborhood violence are concentrated on ELA scores (Sharkey et al. 2014; Schwartz et al. 2016). Differing psychological and cognitive processes may be involved in learning reading and math concepts, and these may be differentially affected by acute stress resulting from neighborhood violence. For example, Sharkey and his colleagues find that exposure to homicides is linked with lower attention and impulse control, which is especially important for reading instruction (Sharkey et al. 2012; Liew et al. 2008). Further, the development of language skills is more dependent on home factors, whereas math tends to be more influenced by school-level mechanisms (Bryk and Raudenbusch 1988).
We note a few key limitations in the empirical work. First, the estimation strategy provides strong identification of the acute effect of exposure to violence but does not provide evidence of whether or not this is a testing effect of it has long-term consequences for learning. Further, this article does not speak to the effects of repeated or cumulative exposure to neighborhood violence or how schools might respond. Understanding these longer-term effects may illuminate potential interventions aimed at children experiencing chronic exposure. That said, the acute effects are important, in and of themselves, due in part to the reliance on standardized tests for decisions about retention, high school admissions, or program participation for middle school students. Second, although we find no effect of exposure to violence on test scores for students at schools with strong or mixed climates, this does not imply that exposure to violence has no effect on these students. Exposure to violence may manifest in the lives of children in other ways that are not captured by test scores in the short run. Further work should investigate the effect on other outcomes, such as absenteeism, behavioral problems, or disciplinary referrals, to gain a broader perspective on how neighborhood violence affects students. Examining such outcomes would also provide insight into the mechanisms underlying the decreases in test scores for students in schools with weak climates.
Finally, more work could be done to unpack what is captured by our measures of school climate. For example, we are unable to say anything in this article about teacher quality or teacher experiences in these schools, and how their views and actions might shape school climate. As other articles in this issue suggest, the school environment is complex and factors such as teacher’s views on issues like diversity and cultural competencies (Penner et al. 2019) or school organizations such as PTAs (Murray et al. 2019) might be important determinants of school climate. Further, although schools with weaker climates have more reported incidents of violence, school climate is more than a reflection of violence in the school or in the neighborhood. Our climate constructs capture overall perceptions of school climate, but we cannot fully measure what particular factors contribute to a stronger (or weaker) school climate.
In sum, although schools are unable to control the experiences students have beyond their walls, the climate within each school can play a role in helping students cope with external forces. This article provides suggestive evidence that many schools are safe havens for young people who live in dangerous neighborhoods, insulating them from the acute effect of exposure to violence on achievement. More research is needed to understand how schools successfully foster strong climates along multiple dimensions, to identify strategies to improving school climate, and to measure how changes in school climate affect other student outcomes.
Appendices
Blockface Geography
Response Rates, School Climate Questions
Student Characteristics by Exposure to Violent Crime, One-Week Window, High Poverty Sample
Robustness Test: ELA, School Climate Specification with School-Level Controls
Falsification Test: ELA, Exposure to Property Crime and School Climate Quartile
Robustness Test: ELA, Exposure to Neighborhood Violence and School Climate Quartile, Full Sample
FOOTNOTES
↵1. The data also have information about robberies (excluded from our measure), property crime including burglary, larceny, motor vehicle theft, and arson. The data include other less serious crimes such as drug sales or use, weapons, simple assault, prostitution, gambling, graffiti, trespassing, disturbing the peace, and moving vehicle violations.
↵2. Our annual address data allow us to track students’ exposure to violent crime even when they move. Approximately 85 percent of students never move. These numbers are similar if we use the high poverty sample or the full sample of students. Of those who move, about 13 percent move only once, and 1.2 percent move more than once during the sample period. Results reported in the article are not sensitive to using a sample of nonmovers (table B8).
↵3. High poverty census tracts are those with a child poverty rate above the citywide median in 2000. We focus on this sample because most of the analyses that follow examine effects on ELA. Note that we also conduct analyses using math test scores as the outcome. These analyses use a sample of students exposed within seven days of the mathematics test. This sample contains 16,676 student-year observations in 535 schools for students living in high poverty census tracts. Testing dates vary by year and grade. The ELA test was administered between January and early February from 2007 to 2009, and in April in 2010. The math test was administered later in the spring (between March and May depending on the year).
↵4. Students in high poverty census tracts make up 67 percent of the overall sample of students in grades six through eight and thus are representative of a significant portion of the public school population in the city. This number is roughly 89 percent when we restrict the sample to students exposed within one week of the ELA test. By restricting the sample in this way we reduce the potential for results to be overly influenced by anomalous sections of New York City, such as midtown Manhattan, which is a very wealthy area but also contains a high crime rate because of its density of commercial and tourist activity and very high daytime population. That said, results are largely unchanged when estimating our models on the full sample of students.
↵5. Some questions include responses such as: all of the time, most of the time, some of the time, never. Other questions include responses: strongly agree, agree, disagree, and strongly disagree.
↵6. We created climate scales using exogenous rotation as well. These measures are highly correlated with the ones used in this article. Exogenous rotation is preferred when measures are used as predictors in the same model to avoid multicollinearity (Kraft, Marinell, and Yee 2016).
↵7. These three factors have eigenvalues greater than one (Kaiser-Guttman stopping criterion).
↵8. To further check the robustness of these measures, we calculated Cronbach’s alpha for the three constructs separately, including all relevant questions we identified for each construct. Overall, we find that the three school climate measures are highly reliable, having a Cronbach’s alpha reliability value of 0.7 or greater.
↵9. Weighted sums of standardized versions of the questions, with the factor loadings used as weights.
↵10. Despite some variation over time in school climate measures, perceptions of school climate do not appear to change substantially over time. When we divide the three scales into quartiles, the majority of schools always stay in the same quartile (32 to 37 percent, depending on the measure) or experience small changes (move up or down one quartile, 35 to 38 percent of schools). Less than 2 percent of schools move three or more quartiles from one year to the next (for example, move from an unsafe (Q4) school to a safe (Q1) school). Thus, a minority of schools experience large changes in perceptions of school climate. Further, the direction of the changes in perceptions of school climate over time is not always consistent. In some schools, perceptions of climate improve one year, but decline in the next. It is unclear if these movements reflect meaningful variations in the school environment students face each year. Exploiting this annual variation to estimate our effects may just leave measurement error as the source of variation. Using time invariant measures minimizes this problem while still allowing us to extract meaningful conclusions about the relationship between the school environment and neighborhood crime. Not all students answer the surveys. Table A1 shows the percentage of students who answered all seventeen questions in our measures. Response rates are low on the first year of the survey but significantly increase after that (reaching about 80 percent).
↵11. For additional details on these distributions see table B1 in the online appendix, available at https://www.rsfjournal.org/content/5/2/141/tab-supplemental.
↵12. Regression results of the balance test are available in table B2 of the online appendix.
↵13. This information is available in tabular form in the online appendix (panel A, tables B3–B5).
↵14. These are selected budget items under the instructional support spending category.
↵15. Detailed information in tabular form is available in the online appendix (panels B and C, tables B2–B4).
↵16. In prior work, the acute effect on ELA test scores is driven by students in elementary school grades (for more details, see Sharkey et al. 2014).
↵17. Note that this quartile has the least number of observations (1,013) relative to quartile four for the other scales (2,550 for safety and 6,016 for sense of community).
↵18. We also estimate the probability that a student would pass the ELA test. We find that students in the least safe schools are 5 percentage points less likely to pass the test, and that those in the more disorderly schools are 3 percentage points less likely to pass. Students in schools with a weaker sense of community are also 5 percentage points less likely to pass the ELA test, but this effect is significant at the 10 percent level (see online appendix table B6).
↵19. Results reported for the weak climate schools in each category only because there are no effects of attendance at a strong or mixed climate school on test scores (tables available from authors on request).
↵20. When we stratify the sample by school climate quartiles and race-ethnicity we are left with a very small number of observations for white and Asian students: forty for students who are white in quartile four schools in the safety measure, and forty-five for students who are Asian. In the most disorderly schools (quartile four), observations number 105 for students who are white and 134 for students who are Asian. Numbers are even smaller in the schools with a weak sense of community (quartile four): nineteen and twenty-seven for whites and Asians, respectively.
↵21. We find no statistically significant differences between students exposed to crime before the ELA exam attending schools in quartiles one to three and those exposed after. Tables available from authors.
↵22. We also test the sensitivity of results reported in the paper to opening the window of exposure. We estimated all baseline and subgroup models on the high poverty sample using a two-week and a one-month window of exposure. These results are also consistent with findings for the one-week window, albeit smaller in magnitude. Specifically, students exposed to violent crime two weeks before the ELA test attending the least safe schools score 0.04 standard deviations lower and those exposed in the month before the test score 0.024 standard deviations lower and 0.022 standard deviations lower in schools with weaker sense of community. Tables available from authors.
↵23. The estimated test score gap between poor and nonpoor students in the full sample is 0.15 standard deviations.
- © 2019 Russell Sage Foundation. Laurito, Agustina, Johanna Lacoe, Amy Ellen Schwartz, Patrick Sharkey, and Ingrid Gould Ellen. 2019. “School Climate and the Impact of Neighborhood Crime on Test Scores.” RSF: The Russell Sage Foundation Journal of the Social Sciences 5(2): 141–66. DOI: 10.7758/RSF.2019.5.2.08. We gratefully acknowledge the support of the W. T. Grant Foundation. We thank participants at the Russell Sage Foundation conference on Using Administrative Data for Science and Policy for their comments. We also thank the RSF editors of this issue and three anonymous reviewers for their feedback. Direct correspondence to: Agustina Laurito at malaurit{at}uic.edu, 400 S. Peoria St., Room 2113 AEH (MC 278), Chicago, IL 60607; Johanna Lacoe at jlacoe{at}mathematica-mpr.com, 505 14th St., Suite 800, Oakland, CA 94612–1475; Amy Ellen Schwartz at amyschwartz{at}syr.edu, 426 Eggers Hall, Syracuse, NY 13244; Patrick Sharkey at patrick.sharkey{at}nyu.edu, 295 Lafayette St., Fourth Floor, New York, NY 10012; and Ingrid Gould Ellen at ingrid.ellen{at}nyu.edu, 295 Lafayette St., Second Floor, New York, NY 10012.
Open Access Policy: RSF: The Russell Sage Foundation Journal of the Social Sciences is an open access journal. This article is published under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.
REFERENCES
In this issue
Jump to section
Related Articles
Cited By...
- No citing articles found.