How a School Phone Ban Test Became the Trial of the Smartphone

Thirty-three states and a pouch

Florida went first, with a statewide law signed by Gov. Ron DeSantis, a Republican, in May 2023. On June 6 of that year, the social psychologist Jonathan Haidt published "The Case for Phone-Free Schools" on After Babel. The newsletter belongs to a lab at New York University's Stern School of Business that campaigns for phone-free schools. He wrote that lockable pouches, "such as those made by Yondr," and phone lockers were "the policies most likely to produce substantial educational, social, and mental-health benefits."

"Much of that momentum grows from years of efforts by the psychologist Jonathan Haidt, who has pushed for schools to ban phones," Greg Toppo wrote in The 74 on May 4. The national study's authors wrote that the pouch "is an example of the kind of stringent, physically binding restriction that proponents of limits on phones have advocated for (Haidt, 2023)."

The authors did not sound triumphant. E. Jason Baron, a Duke professor and a co-author, described the result to Angelo of Fortune in plain terms: "If you're a school and you really care about getting kids off their phones, this seems to work very well. If you really care about test scores, then what our study can tell you is that, up to three years after you're not seeing much."

A second American study pointed the same way, with slightly more favorable results. David N. Figlio, an economist at the University of Rochester, and Umut Özek, a senior economist at the RAND Corporation, studied one of the country's 10 largest school districts, in Florida. Its leaders had gone beyond the state law to a "bell-to-bell" ban covering the whole school day. Test scores rose modestly in the second year. In a May 25 article for The Hechinger Report, Jill Barshay put the Florida gain at "the equivalent of moving a student from the 50th percentile, dead in the middle, to the 51st percentile." Suspensions rose in the first year. Figlio and Özek reported that in the schools where phone use had been heaviest, "the rate of in-school suspensions for Black students increases by 30 percent, while rates for white and Hispanic students remain steady."

Toppo reported that students, on the whole, oppose the bans. In January, Alvin Buyinza of Word In Black, a news collaborative of Black-owned publishers, reported Pew Research Center figures: 46 percent of white teenagers supported school phone bans, against 33 percent of Black teenagers and 36 percent of Hispanic teenagers.

The same numbers, two verdicts

The first verdict came from the skeptics. The technology-policy blog Techdirt has a long-standing editorial line against technology bans. On June 2, it read the national study as proof of what it had said before: "The new study just confirms what the earlier research already showed. Blanket bans make for good press releases. They don't make for better students." The post's wider lesson was that "blanket bans of technology are almost never the answer." It granted the problem: "Obviously, if kids are just sitting in class all day staring at their phones instead of paying attention to the teacher, that's a problem." It also said that "for some schools a full ban could absolutely make sense."

On September 17, Carmen Lim and Zach Rausch of the NYU Stern lab answered in a review on After Babel, "Are Phone-Free Schools Working?" "The effects on test scores were particularly promising (contrary to recent headlines), especially in cases where phone use actually declined in schools," they wrote. The review concluded that removing phones from the school day is "a promising first step in addressing declining student performance."

Christopher J. Ferguson, a researcher, has long argued that the alarm over smartphones is a moral panic. On October 1, he widened the frame. In a post on his newsletter, Secrets of Grimoire Manor, he and Will Dobud of Charles Sturt University argued that teenage mental health is improving even as screen use stays high. "The social media panic is following a similar script," they wrote.

The researchers who ran the American studies sit between the two camps, and they are not neutral either. Thomas Dee, a Stanford education economist who co-led the national study, told Ailsa Chang of NPR's All Things Considered on May 4 that "one of the concerns I have about this study is that it might encourage people to walk away from phone bans as a compelling reform." He went on: "And I think that would be a major mistake." David N. Figlio, the Rochester economist who co-wrote the Florida study, kept apart two questions that the loud camps had run together. "Cellphones still could be having a large effect on the diminishment of student achievement, even if cellphone bans are not turning this around by a tremendous amount," he told Jill Barshay of The Hechinger Report.

Each camp sounds right on its own terms. The near-zero average is real, and so is a small gain the study found in high-school math. Neither fact answers the question each camp used it to answer.

One small number, made big and made small

To read the study's most argued-over number, you need two terms. Researchers often report an effect as a fraction of a standard deviation, a measure of how spread out scores are. A result is "statistically significant" when it is larger than chance alone would easily produce. Significance says an effect is probably there, not that it is big enough to matter.

The national study found that in high schools, the pouches raised math achievement "by roughly 0.024 student-level standard deviations. For context, this corresponds to roughly a 0.9 percentile point increase in the test score distribution." Middle schools did slightly worse. In math, the gap between the two kinds of school was statistically significant. Across all subjects, it was not.

Thomas Dee, the Stanford economist who co-led the national study, rounded the gain away. He told Ailsa Chang of NPR that "we saw no appreciable effect on student test scores at either the middle school or the high school level." That is fair to the averages. It also sets aside the high-school math gain, which the paper he co-wrote reports as statistically significant. It sets aside the small middle-school decline as well.

Carmen Lim and Zach Rausch, in their After Babel review, rounded the same number up. "This 'modest' benefit (0.024 student-level standard deviations) on high school students' math performance is equivalent to roughly one to two months of schooling," their review said. Averages can hide real effects. Asking where an effect appears is part of what Hardly Thinking calls B1: The Effect-Size Reflex. The tool asks of any finding how big it is, for whom, and compared to what. The review also shows its arithmetic in a footnote. But the unit was a choice. By the footnote's main figure, a year of high-school math growth is about 0.155 standard deviations. Counted in months of that growth, almost any effect looks large.

In an October 10 post on his newsletter, Christopher J. Ferguson did the reverse. His subject was a different study, published in the journal Science, of a phone ban in college classes. "A measly d = .073 which, even with 16000 students fueling passive power, was only barely significant (p = .021)," he wrote. "Keep in mind, folks like John Hattie recommend a d = .40 for indication of successful educational policies and this is nowhere close." ("d" is an effect measured in standard deviations; "passive power" means a huge sample makes tiny effects easy to detect.) His reason is that "effects this small are very often false positive due to various sources of methodological noise." He meets the obvious reply directly: "any effect sprinkled over a large population can be valuable. No, that's just not true. We don't even have confidence such an effect is real!"

That is B1 asked honestly: in huge samples, tiny effects can reach statistical significance and still be flukes of method. The post also argues about cost by analogy with medicine. In medicine, it says, "dangerous medications were promoted" on small significant effects. The weak step is the comparison. A benchmark for educational policies in general is one answer to "compared to what?" But a phone rule costs little per student and carries none of a drug's risks. The medical analogy does not settle whether a small, real effect from a cheap rule is worth having. A cheap rule of this kind is unlikely ever to pass a bar of 0.40.

Neither camp's translation of the number is false. The persuasion is in the unit. Whichever one felt more natural to you is a fair clue to which camp you were already in.

A forecast in writing

Jonathan Haidt's 2023 essay did something rare for a public alarm: it wrote down a forecast. "Would it improve educational outcomes? I believe that the answer to all of these questions is yes," he wrote. He then asked for schools to be assigned at random to adopt pouches or lockers earlier or later. "That way we could gain high-quality experimental evidence as to whether phone-free schools really confer the benefits I have predicted based on the research." That is C4: The Prediction Ledger in its honest form. The essay made a dated forecast, labeled it as one, and asked for the evidence that would check it.

The same paragraph urged schools not to wait. Schools "can resolve to move up to lockable pouches or phone lockers, and many schools could implement these changes by September," he wrote. Acting first can be defended when a rule is cheap and the worry serious. It is also the habit of prescribing the cure before anyone has measured it.

The grading is where the ledger slips. The September review by Lim and Rausch takes up test scores first, and it treats them generously. Its first section argues that the national study's evidence on test scores is better than the headlines claimed. Its second turns to two other 2026 studies, from Norway and Brazil, and both are favorable. It never sets the 2023 "yes" beside a pouch study whose average effect on test scores was close to zero. Its third section, "Extending the Conversation Beyond Test Scores," moves to goals measured less well or not at all: "reducing distractions, increasing face-to-face interaction, decreasing bullying and social drama". The 2023 essay did name social life as a goal, and three years is early. But the review, from the lab behind the forecast, judged the evidence by its best subgroup and two other studies, not by what the forecast said. The review edges toward the weaponized form of C4, selective grading. That form judges opponents on outcomes and allies on intentions.

The campaign has also shown what it looks like to grade itself. On March 17, After Babel removed a guest post that contained inaccuracies and thanked "attentive readers" for their feedback. "In this case, our editorial process didn't meet that standard, and the post was published without sufficient vetting," Haidt and Rausch wrote in a public note. It is the plainest public correction by either camp in this fight.

Every panic in the ledger

The critics keep a ledger too, and theirs is a ledger of history. Christopher J. Ferguson and Will Dobud's October 1 post set it out: "Comic books, pinball machines, school radio, rock music, and video games each produced similar public narratives, congressional hearings, and calls for prohibitive bans on the new technology. In every case, claims of impending doom to kids did not pan out as evidence grew." The same post said: "Of course, some teenagers are really struggling, and the tech companies have an obligation to respond to credible risks."

The list is real history, and its first entry is well documented. A Law Library of Congress blog post describes what happened in 1954. The psychiatrist Fredric Wertham argued before a Senate subcommittee that comic books were, in the post's words, "a significant contributing factor to the rise in juvenile delinquency." Most other experts disagreed. But "in the minds of many Americans the connection between comic books and youth crime was self-evident," and that October the industry adopted a code restricting what comics could show. Wertham's papers were later opened to researchers. The University of Illinois reported that Carol Tilley, a professor there, found that he had "revised children's ages, distorted their quotes, omitted other causal factors."

But "in every case" is a ledger with one column, built from alarms that ended well. It leaves out warnings about popular habits that were waved away and proved right. Tobacco is a substance, not a medium, so it fits the list loosely. But it shows what the list cannot. According to a summary by the National Library of Medicine, the 1964 Surgeon General's report "estimated that average smokers had a nine- to ten-fold risk of developing lung cancer compared to non-smokers."

Hardly Thinking calls this pattern G7: Moral Panic and the Folk Devil. The October 1 post shows its dismissal side. The post points to youth outcomes that improved during the smartphone era, such as violence and teen pregnancy, and turns the other side's logic back on it: "If we're tempted to assume causation from correlations, we can't have it both ways, condemning screens for some patterns while ignoring other improving patterns." That is fair as far as it goes. But the post's own opening reads a time trend as a verdict: "Increasingly, evidence from the Centers for Disease Control (CDC) says 'no.'" A few paragraphs on, it asserts a motive without evidence: "schools would love to shift the blame to cellphones and screens." Used this way, the panic label closes the question instead of disciplining it.

The skeptics have something on the credit side as well. In August 2024, Ferguson asked about the bans: "Do they increase detentions and suspensions? Could they actually have a negative impact on student well-being?" Both American studies found a first-year rise in discipline, and the national study found a first-year dip in well-being. He had asked the right questions in advance. The same post made a forecast: "At the best, smartphone bans will simply be a waste of time." So far it has fared less well, given the national study's suggestive evidence that well-being later rose.

The Florida study, retold

David N. Figlio and Umut Özek, the Florida researchers, summed up their working paper with a hedge: "Overall, our findings reveal that cellphone bans could improve student outcomes." The paper's abstract is already firmer: "we find significant improvements in student test scores in the second year of the ban after that initial adjustment period." Their May 12 article in Education Next, a journal of education research and opinion, said: "Overall, our findings reveal that cellphone bans improve student outcomes." On the way to a general audience, the sentence lost its "could." One district's result came to read as a rule about bans everywhere.

The first test-score figure Education Next's readers meet is a before-and-after change: "By the end of the second year, scores are up by about 3.5 percentiles compared to scores from May 2023, the end of the pre-ban year." That change includes everything else that happened in two years. The study's own comparison, between schools with heavy and light phone use before the ban, comes later: "a cellphone ban in high-use schools increases test scores by 1.1 percentiles overall." Across all exams, the working paper reports 0.6 of a percentile.

Leading with the larger number makes the Florida result sound bigger than the study's own comparison shows. The same article reports the cost plainly, including the 30 percent rise in in-school suspensions for Black students in high-use schools. The After Babel review discusses the discipline findings without saying who bore them.

The silver bullet

On May 4, Dana Goldstein's report for The New York Times, reprinted by The Boston Globe, opened: "Banning cellphones was supposed to improve many of the problems ailing American education, including distraction, bullying, declining test scores, and absenteeism." Two paragraphs later it said whose hopes these were: "the goals that educators and parents hoped for." Claire Cameron's May 4 article for Scientific American ran under the subtitle "Banning cell phones in schools has been touted as a silver bullet for poor test scores and low student well-being and attendance".

Here the charitable reading of the press is mostly right. Someone did make the promise: Jonathan Haidt's 2023 essay forecast "substantial educational, social, and mental-health benefits" from pouches. Against those words, a report that bans did not raise scores is fair. One phrase does not hold. Nobody in this fight is quoted calling bans "a silver bullet." The strongest forecast on record came with a request for a randomized experiment. Most of the coverage of the May studies described what the bans' advocates had promised in terms the advocates themselves would accept. One subtitle did not.

Emily Oster, a Brown University economist and the founder of ParentData, wrote in a May 7 opinion essay for The New York Times that the bans "have been trumpeted as game changers," and she quoted one headline: "How the Phone Ban Saved High School." She also argued for wider measures. "Test scores are easy to measure, but a lot of the discussion around phone bans focuses on the experiences of students, how they interact with one another and whether the classroom feels engaging to both students and teachers," she wrote. "We should be measuring those outcomes systematically." Her verdict on the bans cut both ways: "Phone bans may be helpful in some ways, but they aren't a cure-all, and that shouldn't be the bar for success."

The national study did look past test scores. Besides discipline and well-being, it reports "little evidence of effects on school attendance and no clear evidence of improvements in self-reported classroom attention or perceived online bullying." Fortune's headline counted one result: "Schools are spending tens of millions banning phones from classrooms, but test scores aren't improving." Techdirt's verdict, "They don't make for better students," rested on the same single measure. The same measures cut against the bans' advocates. Much of their case rests on social and classroom benefits. The study could measure two of them, attention and online bullying, and neither showed a clear improvement. The benefits it could not measure can be claimed without being checked. Hardly Thinking calls the question underneath E2: The Operationalization Demand: what, specifically, would count as success, measured how? Carmen Lim and Zach Rausch's After Babel review asks it at length, listing measures for future studies from noise levels to library book checkouts. Asked before the results, the question keeps a policy from being judged by whatever was easiest to count. Answered only afterward, with a measure picked to fit the results, it lets each camp choose the measure that flatters it.

The Pattern Underneath: moral panic and the folk devil

G7: Moral Panic and the Folk Devil describes a pattern in which a social problem is pinned on a group or practice, the "folk devil." The loudness of the alarm then comes loose from the scale of the harm. In this fight the folk devil is a device. Whether the device deserves the blame is the question the national study could not answer.

G7 fails in two directions. At one pole, the alarm runs ahead of the evidence, with certainty about the harm and the cure before either is measured. That pole appeared in the 2023 "yes" paired with "by September," and in a review that graded the forecast against its best subgroup. At the other pole, the dismissal treats every worry as one more panic because earlier worries were overblown. It uses the label to stop looking. That pole appeared in "In every case," in a "measly" effect held to a single bar, and in a motive assigned to schools without evidence. The grammar's own page adds a newer twist: "every scare tends to arrive with its debunking attached." In this fight, the scare and its debunking arrived in the same months and read the same evidence.

The two loud camps share a habit. Each read a study of one school rule, over at most three years, as a verdict on the bans. Each then took that verdict as support for its wider view of phones.

The findings fall roughly evenly between the two loud camps. Each translated a small number into the unit that suited it, and each graded a ledger generously on its own side. One habit turned up on one side only: the motive assigned without evidence came from the dismissal side. The researchers and the press were not neutral referees: one researcher rounded a finding down, two dropped a hedge, and one subtitle inflated a promise. Each pole feels like good sense from inside: the alarm sounds like prudence to a worried parent, and the dismissal sounds like experience to anyone who has watched earlier scares fade.

The tools, used and abused

The tools on HT are questions a reader can run on any argument, including their own. Each has a fair use and a weaponized one.

B1: The Effect-Size Reflex asks how big an effect is, for whom, and compared to what. Used fairly, it asked whether an average hid a high-school gain, and whether a huge sample made a tiny effect look solid. Used as a weapon, it picks the unit that flatters the conclusion, as months of schooling did, or sets a bar that a cheap rule is unlikely ever to clear, as the 0.40 benchmark did.

C4: The Prediction Ledger says to write a forecast down, date it and grade it later, as Jonathan Haidt's 2023 essay and Christopher J. Ferguson's 2024 questions make possible. Used as a weapon, the ledger is graded selectively: a forecast against its best subgroup, and history against only the alarms that failed.

E2: The Operationalization Demand asks what, specifically, would count as success, and how it would be measured. Used fairly, it is Emily Oster's "We should be measuring those outcomes systematically," said of students' experiences at school, and Carmen Lim and Zach Rausch's list of measures for the next studies. Used as a weapon, the measure is chosen after the results arrive, to fit them: test scores for the bans' critics, and benefits not yet measured for their advocates.

The ledger is the tool to turn on yourself. Before the next study arrives, write down what result would change your mind about phones in school, when you would expect to see it, and how big it would have to be to matter, compared with what. If you can name such a result only for the side you oppose, you are keeping a ledger with one column. Running a tool on your own side first is a habit HT calls A4: The Mirror Check.

Reader review

Every HT article ends by asking readers to check it against its own rules. The question is not what you think of phones in school but whether this article met the standards it applies to others.

1. Find the passage where this article was hardest on the side you agree with, and the one where it was softest on the side you oppose. Were they judged by the same standard?
2. The article says the national study cannot settle whether smartphones harmed teenagers. Find any sentence where it leans one way on that question anyway.
3. The article runs B1, the Effect-Size Reflex, on others' choice of units and words. Find a place where it makes a finding sound bigger or smaller than its source did.
4. The article grades Jonathan Haidt's 2023 forecast and Christopher J. Ferguson's 2024 questions with C4, the Prediction Ledger. Did it grade both by their own words, or hold one of them to a harder test?
5. Run E2, the Operationalization Demand, on the article's own opening claim that the bans "did what they were built to do." What, specifically, would count as doing it, and does the article show that it was measured?
6. The article credits the campaign's public correction and the critics' 2024 questions. Did one side's credit read as praise and the other's as a concession?
7. Choose one quotation and check it against the linked source. Is its context fairly represented?
8. Students get less attention here than the two loud camps, especially the Black students who bore the rise in Florida suspensions. So do the lawmakers who passed the bans. What might the article have found if it had held them to the same standard?