A reexamination of a previous study of error-rate in capital cases suggests that the system is not “broken.”

A few months ago, the front page of the New York Times announced the results of a major investigation of appeals in capital cases.1  “The most far-reaching study of the death penalty in the United States,” the article began, “has found that two out of three convictions were overturned on appeal, mostly because of serious errors by incompetent defense lawyers or overzealous police officers and prosecutors who withheld evidence.” The study itself, authored by Professor James S. Liebman of Columbia University Law School and colleagues, stated that the “overall error-rate in our capital punishment system was 68%,” pronounced the system “broken” and promised more research into the causes.2  In fact, because of a significant omission in the Liebman study, the newspaper accounts were seriously mistaken: the conviction reversal rate in death penalty cases is closer to 27 percent than 68 percent. What accounts for the discrepancy?

The Liebman study did not distinguish between reversals of convictions and reversals of death sentences. The difference is not just semantics. When a guilty verdict is reversed, the defendant must be released or given a full retrial, a process that is costly to the criminal justice system, burdensome to the witnesses, and painful to the family of the murder victim. By contrast, when the sentence alone is reversed, the guilty verdict stands. The capital defendant may be given a new penalty hearing leading to another death sentence, or he may be kept in prison, usually for life.

This article will show that, once the distinction between guilt and sentence is taken into account, only about 27 percent of capital convictions—not two-thirds—are set aside. Furthermore, according to Liebman’s own data, the overwhelming majority of these defendants are eventually reconvicted and sentenced to imprisonment or death.3

Any assessment of the capital justice system must take into account the relative number of sentence and conviction reversals. Were it true that two-thirds of all convictions were overturned, we might agree that the system is indeed collapsing. Such an outcome would suggest that the trial courts are failing or perhaps that the appellate judges are (as has been said in other contexts) irrationally exuberant. It would mean that the system is extremely inefficient, incapable of dependably determining guilt without a decade or more of appeals. Indeed, if two-thirds of all guilty verdicts are being reversed, then hundreds of people are languishing on death row for years without a reliable determination of their culpability.

If, on the other hand, most of the reversals in death penalty cases affect the sentence alone, then the charge of systemic failure is considerably overblown. When only sentences are overturned, it cannot be concluded that trial courts are incapable of determining guilt or that the system is in need of major reconstruction. It cannot be said that prisoners are incarcerated without proper trial, or that repeated trials are necessary. The soundest conclusion (unsurprisingly) is that judges and jurors, like many reasonable people, disagree about the appropriateness of death sentences, and (more problematically) that the system reflects and encourages this disagreement.

We are not sanguine about the capital processing system. Justice takes too long, costs too much and, given the ratio of actual executions to death sentences, is too often honored only in the breach.4  And even if, as we show below, most of the reversals are of the penalty alone, there are, nonetheless, too many reversals in capital cases. Later, we explore possible explanations for this. But for now, we must question the implication that the system has failed due to its inability to conduct reliable capital trials.
 

A broken system?

Once Liebman and his colleagues combined the state post-conviction review decisions with the cases resolved on direct appeal, they found that the rate of reversal for serious error increased to approximately 47 percent. Finally, when they added federal habeas corpus decisions to the data, the rate rose to approximately 68 percent. Thus, in the end, the authors concluded that approximately two out of three death penalty decisions reviewed over the 23 years of the study period were reversed either on direct appeal, post-conviction review, or federal habeas corpus review.

Recognizing that after reversal some of the capital convictions overturned may have been reinstated upon retrial, and some of the reversed death sentences may have been reimposed after new sentencing hearings, the investigators also sought to determine the ultimate outcome of the successfully appealed cases. This proved difficult since trial courts (where retrials and repeat sentencing hearings take place) do not publish their decisions. However, Liebman and his colleagues were able to investigate the ultimate determinations in 301 reversals at the state post-conviction review stage, finding that after retrial or resentencing, 93 percent were found guilty of murder and 7 percent were acquitted of murder. The authors also found that 82 percent of the reconvictions/resentencings ended with sentences less than death.

Beyond identifying the overall rates, the study also found that reversal rates of death penalty cases on direct appeal and post conviction review varied significantly from state-to-state, ranging from 13 percent in Virginia to 77 percent in Wyoming. States such as North Carolina (65 percent), Florida (58 percent) and Mississippi (69 percent), located in what the authors referred to as the nation’s “death belt,” had some of the highest levels of “serious error” on direct appeal and post-conviction review. When federal habeas corpus review was added in, reversal rates of capital judgments rendered in these states went even higher.

Based on their findings, Liebman and his colleagues concluded that the death penalty system is “collapsing under the weight of its own mistakes.” They added that while their study “describes the extent of the problem,” future research will examine its causes and recommend the appropriate policy responses.

Like many in the national press, some may have concluded that the Liebman study provided evidence that over two-thirds of death penalty convictions were overturned, clearly an unsettling finding. Having read hundreds of capital appeals ourselves, however, that conclusion simply did not square with our impressions. We strongly suspected that many of the appeals resulted in overturned sentences, but not convictions. If this were true, then by failing to differentiate between conviction and sentence when computing the reversal rates, the Liebman study would provide an inaccurate picture of the death penalty system and an unreliable guide for policy makers. To test our impressions, we looked more closely at the reversals of death verdicts by state high courts to determine the respective proportions of conviction and sentence reversals.
 

A reexamination

We included in our data all decisions by the selected state high courts reversing either a capital conviction or capital sentence, whether on direct appeal or post-conviction review.8  Using Westlaw, we identified 837 death penalty reversals from the 26 states. We then coded each case for conviction reversal or sentence reversal. The results are set out in Table 1.

Overall, of the 837 death penalty reversals in our data, 328 (39 percent) were conviction reversals and 509 (61 percent) were sentence reversals only. Thus, most of the state high court reversals at the direct appeal and post-conviction review stages addressed sentencing errors, not errors relating to the guilt or innocence of the capital defendant.

In interpreting these results, one should keep in mind that our data included reversals only, while the Liebman study included both affirm-ances and reversals. Specifically, Liebman and his colleagues found that approximately 5 out of every 10 death penalty decisions were reversed at either the direct appeal or post-conviction review stages. Assuming these findings hold true for the 1990-1999 period that we investigated, then of these 5 reversals, 2 were conviction reversals and 3 were sentence reversals only.

As is true with overall reversal rates, there is significant variation in sentence and conviction reversal rates among the states. For example, among states with more than 10 death penalty reversals, four—Florida, Indiana, North Carolina and Tennessee—had sentence reversal scores in excess of 70 percent. Two of these states, Florida and North Carolina, were identified by Liebman and colleagues as the most error-prone when considering overall reversal rates. However, much of this error appears to be occurring at the sentencing phase.

Conversely, reversals in some states were predominately conviction reversals; Alabama, Georgia and Kentucky, for example, had more than 60 percent conviction reversals. This variation across states suggests that different types of reforms may be needed in different states depending on the stage of the capital proceeding at which most of the errors are occurring. These results would appear to be especially important to state policy makers who may be considering changes to their death penalty processes.

There is a third stage of review—federal habeas corpus—that we did not fully investigate. However, we have little reason to believe that the distribution of reversals between sentence and conviction reversals at this stage differs significantly from our results. To confirm this, we examined habeas corpus reversals by the Ninth Circuit Court of Appeals, the largest federal circuit and considered to be one of the more liberal courts in the federal system. Employing another Westlaw search, and limiting our review to the five Ninth Circuit states included in our study,9  we found that during the 10 years of our study period, there were 29 capital cases reversed by the court, of which 8 (27.6 percent) were conviction reversals and 21 (72.4 percent) were sentence reversals. This result was consistent with our findings on direct appeal and post-conviction review. Thus, considering all stages of review, it appears that while a significant percentage of capital decisions are being reversed, most deal with sentencing issues, not questions of guilt or innocence.

Taking all three types of review into consideration, Liebman and colleagues determined that the capital case reversal rate (conviction or sentence) was 68 percent. Given our finding that 39 percent of reversals are conviction reversals, and assuming that the overall 68 percent reversal rate holds for our period, then, when all three types of review are considered, the overall capital conviction reversal rate is approximately 27 percent.10

It should be remembered that these conviction reversals cannot be equated with findings of innocence, as there may be additional trial-level proceedings that reinstate verdicts or sentences. Liebman’s investigation of outcomes, albeit based on a small sample, found that reinstatement occurred in 93 percent of all capital reversals. If the Liebman outcome findings for post-conviction review hold true here, then 9 out of 10 defendants whose convictions were reversed on direct appeal or post-conviction review are again found guilty of murder, although most do not receive death sentences.
 

Why all the reversals?

The capital penalty hearing, by and large unknown in ordinary trials, offers unique opportunities for reversals. In the typical noncapital trial, sentence is imposed by a judge at a post-trial hearing conducted without either jury or witnesses. Consequently, appeals courts have relatively few opportunities to overturn noncapital sentences, as sentencing is not subject to the usual trial errors. By contrast, the capital sentencing hearing is very much a part of the trial. Capital trials are bifurcated—divided in two—by virtual mandate of the Supreme Court.12  If the accused is found guilty of capital murder, the same jury sits for the sentencing phase, at which the prosecution must prove that the crime was sufficiently aggravated to warrant the ultimate penalty and the defense may offer evidence of mitigation. This penalty proceeding—with its examination and cross-examination of witnesses, closing arguments by lawyers, instructions from the bench, and jury deliberation followed by a life-or-death verdict—has been aptly characterized as a mini-trial. Like a trial, it offers ample occasion for findings of reversible error.13

Consider ineffective assistance of counsel claims, the most frequently relied upon ground for death penalty appeals.14  We notice many reversals for failure of defense counsel to sufficiently investigate potentially mitigating evidence, such as favorable character witnesses, or possible psychological problems.15  Such evidence may be more relevant to sentence than guilt, and therefore more likely to lead to an overturned penalty than a conviction reversal.16  Furthermore, the law governing ineffective counsel is more hospitable to claims aimed at the penalty phase of the trial.17  Thus, it is the sentencing portion of a death case that provides the most fertile ground for ineffective counsel claims—the most potent of all appellate arguments.

In addition, overturning a capital sentencing trial is a relatively “low-cost” reversal. If an appeals court overturns a conviction, as opposed to the sentence, the defendant must be tried again or set free. As we observed earlier, retrials are expensive for the criminal justice system, difficult for the witnesses, and, in capital cases especially, agonizing for the families of the victims. Moreover, if the passage of time clouds memories or removes witnesses from our midst, reconviction may be difficult, notwithstanding defendant’s guilt in fact.

By contrast, if a capital sentencing hearing alone is reversed, there is no need for a costly and risky full retrial; at worst, the capital defendant—now a convicted murderer—remains in prison for life. Thus, the very existence of the capital penalty trial stimulates more reversals; as our study amply documents, more than 60 percent of all capital reversals were for sentence phase errors.

While most reversals are of the sentence only, the fact remains that a significant number of capital convictions are overturned. Why? The standard explanation for the numerous reversals in death penalty cases focuses on the alleged poor performance of the trial actors—the prosecutors, defense attorneys and trial judges.18  The argument runs as follows. The prosecutors are under considerable public pressure to seek the death penalty in high-publicity murders, consequently, they decline to accept guilty pleas, take weak cases to trial, and are tempted to engage in misconduct, such as witholding exculpatory evidence, to insure against loss. For their part, defense lawyers are often too inexperienced to handle the complexities of capital cases, or, if in private practice, have a financial incentive to devote less than the time needed to properly conduct a capital defense. Thus, defense attorneys are prone to sins of both omission and commission, often rising to constitutionally ineffective assistance of counsel. Finally, it is argued, the trial judges have insufficient experience conducting capital trials, thus accounting for their relative unfamiliarity with the special pitfalls of death penalty cases.

We are not in a position to confirm or refute the standard explanation, although some of it strikes us as counterintuitive.19  We are convinced, however, that there are other factors peculiar to death penalty cases that must be taken into account when seeking to explain the high conviction reversal rate.

Death penalty cases offer far more opportunities and incentives for reversal than other criminal trials. First of all, the number of post-conviction review possibilities in capital cases is extraordinary. Unlike noncapital cases, every death penalty conviction must be reviewed by the state’s highest court. Since most noncapital criminal cases are not appealed at all, or are rather summarily reviewed by overworked intermediate appellate tribunals, there is greater opportunity for finding error in a death case. In addition, the federal constitutional rulings of the state supreme court can be challenged through a petition to the United States Supreme Court. Supreme Court review (which is discretionary) is usually followed by a state post-conviction motion, which starts with the trial court judge but may, if unsuccessful, work its way up the state court ladder (and, if constitutional questions are raised, advance to the U.S. Supreme Court once again).

Once state remedies are exhausted, the death row inmate may petition a single-judge United States district court for the federal writ of habeas corpus. If unsuccessful, he may appeal to one of the United States courts of appeals, and then (yet again) to the U.S. Supreme Court. Although Congress has placed limits on federal habeas corpus review, the effectiveness of this reform is not yet clear.20 Finally, it should be understood that success in any one of these proceedings could result in a retrial or another sentencing hearing, which, should a death sentence be the outcome, would start the entire post-conviction process over again.

Second, death row inmates probably have more incentive to file appeals than any other prisoner. The death incentive is obvious, but its impact should not be underestimated. Most prisoners who are not on death row are incarcerated for relatively brief periods and are released before they have the opportunity to lodge many appeals.21  For example, federal habeas corpus review (which is commonly sought in death cases) requires exhaustion of state remedies and therefore, as a practical matter, is unavailable for short-term inmates.22 Death row inmates are in for life, and even unsuccessful appeals are advantageous, as they delay the ultimate sentence. It is unlikely that any other prisoner considers additional time behind bars a victory.

Third, the quality and intensity of review in capital cases generate more opportunities for reversal. As is apparent from the length and detail of the court opinions, capital appeals are much more exhaustive than noncapital. Sometimes, appellate lawyers, undoubtedly motivated by strong opposition to the death penalty, meticulously scrutinize the record and scrupulously assert every conceivable claim. Moreover, in contrast to noncapital cases, state courts oftentimes waive procedural rules against “unpreserved” claims, and even raise points on their own initiative. Finally, close questions of law provide an open invitation to appellate judges—who may also be motivated by distaste for the death penalty—to find error.23

Fourth, because a life is at stake, appellate judges may have a very low threshold for capital trial error. Indeed, the Supreme Court has made clear that the margin for error should be smaller in capital cases.24  Consequently, acceptable harmless error in noncapital trials may be treated as reversible error in a death case.
In short, we believe that the number of opportunities to appeal, the motivation of the capital appellant to seek review, the scrupulousness of the appellate actors in uncovering error, the low threshold for findings of reversible error, and the incentives to overturn the unique capital sentencing hearing, have at least as much bearing on the number of successful appeals in death penalty cases as the shortcomings in the trials themselves.
 

A different picture

First of all, taking ultimate dispositions into account, the “failure rate” for death sentences is 56 percent, not 68 percent. More importantly, once we see that 95 percent of all murder convictions are ultimately upheld, but that in half of these the death sentences are reduced to imprisonment, three inferences may be drawn. First, capital trial courts do an acceptable job of determining guilt. Second, the principal tension within the system is over sentence, not guilt. And third, the sentencing conflict is between trial judges and jurors who support the death sentences and appeals courts, which are far less supportive.

The Liebman report assumes that appellate outcomes are accurate and implies that the outsized reversal rate is proof that the trial courts are failing. We are not convinced. How can it be that the same trial actors who perform so acceptably in the guilt phase of the capital trial fall into such error in the penalty phase? We think that it is just as plausible to conclude that there are problems with the capital appeals process as with the capital trial process. The capital punishment system has created unique opportunities and incentives for appeals courts to undo death sentences while leaving convictions intact. Given the controversy surrounding the death penalty, it should perhaps surprise no one that such a system has evolved or that appeals courts have come to play this role. But that does not mean that we have to be satisfied with it.

The capital punishment process has institutionalized—at great expense to the entire criminal justice system—disputes over death sentences. As we see it, the proper public policy question is not “Why are so many capital cases reversed?,” but rather, “Do we need a capital appeals process that expends so many scarce criminal justice resources to determine sentences?”
 

 1.  Butterfield, Death Sentences Being Overturned In 2 Of 3 Appeals, N.Y. Times, June 12, 2000, at A1.
 2.  Liebman,  Fagan and  West, “A Broken System: Error Rates in Capital Cases, 1973-1995” sec. II (June 12, 2000).
 3. Although Liebman and his colleagues were only able to investigate a small sampling of 301 cases, they found that 75 percent of the decisions reversed ultimately resulted in a conviction of murder and a prison sentence, 18 percent were given the death penalty again, and 7 percent were acquitted of murder.
 4. The Liebman study found that the average time between sentence and execution was 9 years, rising to 10.6 years during the period 1989-95.  Justice Department figures for 1998 indicate a 10.8 year delay. U.S. Dept of Justice, Bureau of Justice Statistics Bulletin, Capital Punishment 1998 12 (1999). Regarding expense, the Liebman study reports that capital cases cost from 2.5 to 5 times more than life-sentenced cases. Despite all of the time and money, between 1973 and 1995, only 313 of the 5,760 death-sentenced defendants were executed, a rate of 5.4 percent.
 5. Previous studies were limited to one state or a few years. E.g.,  Greenberg, Against the American System of Capital Punishment, 99 Harv. L. Rev. 1670 (1986) (covering cases decided 1982-5);  Paternoster, Capital Punishment in America  (1991) (discussing studies of Florida, Georgia and South Carolina);  Uelmen, Review of Death Penalty Judgments by the Supreme Courts of California: A Tale of Two Courts, 23 Loy. L. A. L. Rev. 237 (1989) (covering California, 1977-88).
 6. The states are Alabama, Arizona, Arkansas, California, Florida, Georgia, Idaho, Illinois, Indiana, Kentucky, Louisiana, Maryland, Mississippi, Missouri, Montana, Nebraska, Nevada, North Carolina, Oklahoma, Pennsylvania, South Carolina, Tennessee, Texas, Utah, Virginia, and Wyoming.
 7. Following the invalidation of the death penalty statute in Furman v. Georgia, 408 U.S. 238 (1972), capital punishment laws were rewritten in 35 states. The approval of the revised statute in Gregg v. Georgia, 428 U.S. 153 (1976) marks the beginning of the modern era for the death penalty. The Liebman study reported that appellate decisions rendered during the period 1973 to 1982 were “extremely volatile.”
 8. The reversal of a capital conviction will, of course, also be a reversal of the sentence imposed for that conviction.  However, we refer to such a reversal as a “conviction reversal.”  A “sentence reversal” is a reversal of the death sentence only, with an affirmance of or no decision on the capital conviction. We counted as reversals appellate decisions affirming a lower court’s ruling on post-conviction review if that ruling reversed the conviction or the sentence. We did not include appellate decisions granting a defendant’s request for a lower court hearing on his claim, as such an appellate ruling, although favorable to the defense, overturns neither the sentence nor the conviction.
 9. Arizona, California, Idaho, Montana and Nevada.
 10. If 68 of 100 capital decisions are reversed after direct, post-conviction, and federal habeas corpus review, and 39 percent of these are conviction reversals, then convictions in 26.52 (39 percent of 68) of 100 capital decisions are reversed. We use the 68 percent overall reversal rate Liebman found for decisions during 1973-1995 because our data do not allow us to identify a rate for our 1990-1999 study period.  It may be that the reversal rate was lower or higher during this period, although there is no reason to believe this difference would be significant.  While the Liebman study noted that direct appeal and federal habeas reversal rates dropped in the early 1990s, they increased in 1995.  In addition, state post-conviction review error rates appear to have risen through 1990-1995. Of course, a finding that the actual overall reversal rate for our period was higher or lower than 68 percent would not affect our results on the relative distribution of sentence and conviction reversals.
 11. The Liebman report estimates the error rate in noncapital criminal cases that have gone through state direct appeal and state and federal post-conviction review to be less than 15 percent.
 12. See Gregg v. Georgia, 428 U.S. 153 (1976) (approving Georgia’s death penalty statute, which provided for bifurcated trial).
 13. See Carter, Harmless Error in the Penalty Phase of a Capital Case: A Doctrine Misunderstood and Misapplied, 28 Ga. L. Rev. 125, 148 (1993) (the nature of a sentencing proceeding in a capital case makes it difficult for an appellate court to find that error was harmless).
 14. Liebman found that ineffective assistance of counsel was the leading ground for reversals on post-conviction review.
 15. The Supreme Court entertained such arguments in Burger v. Kemp, 483 U.S. 776 (1987) (finding that counsel’s lack of investigative thoroughness was “supported by reasonable professional judgment”).
 16. The jury must weigh favorable character or psychological evidence against aggravating evidence when determining the sentence.
 17. Under the Supreme Court’s test for ineffective assistance, the appellant must prove that counsel’s deficient performance was “prejudicial,” i.e., that there is a reasonable probability that the outcome would have been different without the defense error. Strickland v. Washington, 466 U.S. 668 (1984). This is a difficult test to meet in noncapital cases and in the guilt phase of capital cases where strong evidence of guilt would preclude proof of prejudice. In capital sentencing trials, by contrast, it is reasonable to conclude that the failure to introduce character or psychological evidence was prejudicial because mitigating evidence of this nature must be given great weight in determining whether or not to impose a death sentence. In sum, these investigative error claims are more likely to succeed with capital sentencing trials.
 18. Based on a preliminary review of selected reversals on post-conviction review, the Liebman study found that most of the reversals were due to “egregiously incompetent defense lawyering,” prosecutorial suppression of evidence, and erroneous instructions by the trial judge.
19. With their greater length, more elaborate procedures, more experienced personnel, greater media scrutiny and, as all of the actors are well aware, greater stakes, it is not likely that death penalty trials on the whole are administered less cautiously than their noncapital counterparts. See e.g.,  Gross, The Risks of Death: Why Erroneous Convictions Are Common in Capital Cases, 44 Buff. L. Rev. 469, 496 (stating that capital defendants “may be better represented than other criminal defendants”). Moreover, the premise of the prosecutor-error argument—that prosecutors are pressured into seeking the death penalty—seems especially doubtful. We do not understand why a prosecutor would want to take a weak case into a high-publicity trial. In fact, there is considerable empirical evidence of the willingness of district attorneys to plead out murder cases. The Baldus study of Georgia capital prosecutions indicated that a significant proportion of Georgia’s potential capital cases resulted in pleas and life sentences. See Baldus et al., Equal Justice and the Death Penalty 106
(1990) (“[I]n 59 percent of the death eligible cases, a life sentence was imposed by default when the prosecutor unilaterally waived the penalty trial.”).
20. The Antiterrorism and Effective Death Penalty Act of 1996, Pub. L. 104-132, 110 Stat. 1214 (1996). AEDPA imposes time limits for filing habeas petitions, bars, with a few narrow exceptions, multiple habeas petitions, and mandates deference by the federal habeas judges to state court rulings.
 21. Offenders released from state prisons in 1996 served an average two and one-half years. U.S. Dept of Justice, Bureau of Justice Statistics Special Report, Truth in Sentencing in State Prisons 1 (1999).
22. 28 U.S.C. § 2254(b) requires that state prisoners file direct appeals and state post-conviction motions before seeking federal habeas corpus. See U.S. Dept. of Justice, Bureau of Justice Statistics, Federal Habeas Corpus Review 12 (1995) (finding that it takes nearly five years before a federal habeas petition is filed).
23. Judicial behavior studies in the social science literature have found that votes of state supreme court justices in death penalty cases can, in part, be explained by the justices’ ideological predispositions. See e.g.,  Brace and  Hall, Studying Courts Comparatively: A View from the American States, 48 Pol. Res. Q. 5 (1995); Emmert and Traut, The California Supreme Court and the Death Penalty, 22 Am. Pol. Q. 41 (1994).
 24. See California v. Ramos, 463 U.S. 992, 998-99 (1983) (“The Court . . . has recognized that the qualitative difference of death from all other punishments requires a correspondingly greater degree of scrutiny of the capital sentencing determination”).