Statistical Analysis in Commercial Litigation

Like life, the world of litigation gets ever more complex.The Hadley v Baxendale days of one party contracting directly with another to provide a simple service are, in many regards, long gone. As business arrangements have grown more complex, determining and proving (or disputing) the value of damages has become equally challenging.The legal standard for damages, however, is largely unchanged. Courts require plaintiffs to prove their losses with reasonable certainty.This is easy in the case of a plaintiff who purchased “Widget A” for $1.00 each, but was billed for “Widget B” at $1.10 each.Whether it's one widget or 100 million, the math is simple; but in the real world, people don't litigate when the facts are that simple.

In commercial litigation, it is sometimes difficult to evaluate the losses associated with individual transactions.In other cases, it is possible to determine individual losses with precision, but the volume of transactions may be indeterminate or else so large that computing damages with precision is impracticable.In cases like these, courts have recognized that statistical analysis can satisfy the reasonable certainty standard ¹. Statistical sampling relies on the principle that a properly selected sample will reflect the characteristics of the population within a specified margin of error to a specified confidence level. The factfinder can extrapolate the results of the detailed analysis of the sample to the population and be assured, within the known uncertainties of statistical analysis, of awarding damages that are not speculative.

As Benjamin Disraeli allegedly observed, “there are three types of lies — lies, damn lies, and statistics.”The key to presenting or rebutting damages on the basis of statistical analysis lies in understanding the strengths and weaknesses of the process, when it is needed and when it is not because courts will exclude testimony based on bad statistics.² We were recently involved in a case involving an alleged breach of contract between a medical billing service company (Plaintiff) and a hospital (Defendant) that illustrates many of these issues.

Defendant was a 200-bed regional hospital located in a relatively economically depressed community. Many of its patients were beneficiaries of Medicaid or other government health programs.Recognizing that part of its economic difficulties stemmed from its inefficient internal billing department, the hospital contracted with Plaintiff to evaluate Defendant's billing procedures and, ultimately, to provide billing and collection services.Plaintiff's fee for these services was a percentage of the hospital's collections.A couple of years into the arrangement, the hospital fell behind in its payments to Plaintiff, and Plaintiff notified Defendant that unless it cured the deficiency, Plaintiff intended to terminate in accordance with the provisions of their agreement.In response, the hospital unilaterally terminated the contract without the notice required in the agreement.Not surprisingly, litigation ensued, including Plaintiff's claim for unpaid fees and Defendant's counter-claim that Plaintiff had negligently performed its responsibilities and that the unpaid fees were more than offset by the damages the hospital had sustained due to Plaintiff's improper or inadequate billing of patient services.

In support of its claim that the billing company failed to bill patient encounters properly, the hospital assembled a list of almost 8,000 open accounts receivable that, the hospital alleged, Plaintiff had not properly billed and collected.The values for these open accounts ranged from $3 to $118,000 and totaled around $7.8 million.

To support its counter-claim, Defendant retained a healthcare Billing Consultant to review individual files to determine how much should have been collected.Presumably concerned about the cost of the file review, the hospital retained another consultant, the Damages Expert, to select a sample and extrapolate the Billing Consultant's assessment of those sample files to the rest of the population.While the Damages Expert dressed up his report with complicated formulas, there were a number of fundamental problems with his analysis that cast doubt on his conclusions and which illustrate the types of issues that can occur in statistical analyses.

Was the Population Properly Defined?

To test whether or not Plaintiff had properly billed the hospital's patient encounters, the Damages Expert selected a sample of 200 transactions for the Billing Consultant's review.The sample, however, came not from the actual population of patient encounters that Plaintiff had billed for the hospital over the course of the relationship, but rather, from the list of open accounts that Defendant's staff had assembled for purposes of the litigation.

As a result, the sample was immediately flawed in several respects.First, it was based on inadmissible hearsay.While the Federal Rules of Evidence, and virtually all state court rules of evidence, permit experts to consider otherwise inadmissible materials in performing their analyses, that exception is only to the extent professionals in the field routinely consider those inadmissible materials in the course of their non-litigation engagements.In this case the hospital's listing of allegedly bad transactions would not qualify for the exception because it was created solely for the litigation, with underlying assumptions inherent in the selection process. More importantly, by limiting the population to the hospital's list of alleged “bad” transactions, the Damages Expert introduced bias into the analysis.The question before the court on Defendant's counter-claim was whether or not Plaintiff had materially breached the contract to bill and collect for the hospital's patient encounters.Even if the hospital's experts properly assessed the 8,000 customer interactions on the list that the hospital staff had assembled, would that have constituted a breach of an agreement that involved billing for tens, if not hundreds, of thousands of patient encounters?

Finally, the listing used to define a population of transactions that Plaintiff “failed” to bill and collect properly included hundreds of patient encounters that had occurred in the days and months leading up to the hospital's unilateral termination of the agreement. These accounts were outstanding as of the contract's termination date not because of any failure to properly bill them, but rather because the payors had not processed them yet. Additionally, the hospital failed to fully pursue collection of some of these accounts after it terminated the contract, thus raising the issue of Defendant's failure to mitigate its damages.

Is the Sample Representative of the Population?

Perhaps the single most fundamental requirement for computing damages based on a statistical sample is that the sample reflect the characteristics of the population.In this case, the Damages Expert never actually established the characteristics of the population to evaluate whether a sample selected from the population would be representative. He re-sorted the list of open accounts by the dollar amount outstanding, and arbitrarily divided the list into five “strata” so that each stratum would include approximately $1,700,000. From each stratum, the Damages Expert selected varying number of entries as part of his sample. For example, he included all 62 entries in the first stratum, 21 of the 414 items in the third stratum and 74 of the 6,090 items in the fifth stratum.

Among the factors he should have considered, but didn't, were:

• Age of alleged receivables – Even when everything goes smoothly, healthcare bills can take months to be paid. Between the hospital's multiple departmental inputs and the insurance companies' verification of coverage, coordination of benefits and repricing services, and requests for additional documentation, the process can takes weeks or months with no negligence by any participant in the process. The list of open accounts that the Damages Expert used to draw his sample included hundreds of accounts that were not delinquent, but rather simply had not yet worked their way through the process to be paid. In fact, many of these open accounts had not been processed specifically because the hospital had terminated the agreement and suspended the Plaintiff's access to the necessary computer systems to process the bills.
  
  
• Identity of the payor – The hospital served an economically depressed community.Most of the hospital's patients (more than 77%) qualified for government programs including Medicaid.Medicaid alone accounted for 24% of the hospital's patients.The Damages Expert did nothing to assess whether his sample matched the hospital's payor mix, and as a result, around 43% of the items in the sample involved Medicaid patients. Accordingly, Medicaid patients were substantially over-represented in the sample.

Do All Items Have a Known Probability of Selection?

Statistical sampling, also known as probability sampling, requires that every item in the population have a known chance of selection.The actual selection of sample items is left to chance. Stratification of the population can sometimes be useful.For example, if the hospital had implemented different procedures for patient interactions with fees over specified thresholds, then stratifying the population based on those thresholds would be an appropriate means for assuring that the sample included items subject to those different procedures.Other reasonable bases for stratifying the population might be the type of third-party payor or the type of procedure (inpatient vs. out-patient; emergency room visit vs non-emergency room procedures, etc.)

In this case, however, the Damages Expert's stratification was merely an arbitrary attempt to “top-load” the sample.Thus, while the 62 items in the first stratum represented 20% of the value of the population, they represented 85% of the value of the sample. Further, each item in the first stratum had a 100% chance of selection.The items in the second stratum had only an 8.7% chance of selection, and items in the fourth stratum had only a 2.4% chance of selection.

While this might appear to represent a “known chance of selection,” that chance only becomes known after the process is complete, and otherwise similar items end up with radically different chances of selection depending on which side of the arbitrary dividing lines they happen to be.For example, the last item in the first stratum had an alleged value of $6,567 and the first item in the second stratum had an alleged value of $6,548, a difference of only $19.The Damages Expert provided no basis, other than alleged value, for distinguishing between these two items.As a result, the item with an alleged value of $6,567 had a 100% chance of selection while the item with an alleged value of $6,548 had only an 8.7% chance of selection. If one of the first 60 items in the Damages Expert's re-sorted listing been a few thousand dollars higher or lower, the chance of selection for these two items would have been radically different.

Was Sampling Even Needed?

At the end of the day, the most fundamental question was left unanswered – was sampling needed to prove damages on the hospital's counterclaim?The list of open accounts that the Damages Expert used for sampling purposes contained fewer than 8,000 items. Ultimately, hundreds of these items, or about 25% of the alleged outstanding value, should have come off of the list as they worked their way through third-party payors' processing systems and were paid.The only transactions that were even relevant to the hospital's claim for damages were those that were ultimately written off.

The hospital employee who prepared the list testified that he reviewed hundreds of transactions in a “few days” as he was preparing the list.This would suggest that the Billing Consultant and her staff could have reviewed all of the actually unpaid transactions on the list in a matter of a couple of weeks.There was no evidence on the record for why sampling, with its inherent uncertainties, was even necessary.