Forensic Statistics: Taking the Mishandling and Misuse of Statistics to Court
R.A. Weitzman
Water Ratepayers Association of the Monterey Peninsula (president), Fairfield Place, Carmel, CA 93923, USA
Submission:March 09, 2023;Published:March 20, 2023
*Corresponding author:RA Weitzman, Water Ratepayers Association of the Monterey Peninsula (president), Fairfield Place, Carmel, CA 93923, USA
How to cite this article:RA Weitzman. Forensic Statistics: Taking the Mishandling and Misuse of Statistics to Court. J Forensic Sci & Criminal Inves. 2023; 17(1): 555955 DOI:10.19080/JFSCI.2023.17.555955.
Abstract
After briefly recounting a recently published story about the forensic use of statistics to uncover the mishandling and misuse of statistics in testing the accuracy of a model developed to predict the environmental impact of a desalination project proposed by a government-regulated private utility, this article shows how to challenge those statistical misdeeds in court and then concludes by identifying other, currently high-profile government-administered projects or policies supported by possibly, perhaps even likely, legally actionable mishandled or misused statistics as examples to illustrate the widespread applicability and potential consequentiality of forensic statistics.
Keywords: Forensic Statistics; Predictive Models; Misuse of Statistics; Fraud
Introduction
A 2021 article in the American Statistical Association Journal Chance [1] ended with the following sentences: “This story is only an example. Statistics may have widespread use as a forensic tool, even in the legal world itself, where an amateur sleuth like me might turn out to be an actual one.” The following four sections will briefly recount that story and, carrying it a step further into the forensic realm, will show how to take the mishandling and misuse of statistics reported there to court. The concluding section will provide examples of other potentially consequential applications of forensic statistics.
Mishandling and Misuse of Statistics to Support a Project
Specifically, the story reported in that 2021 Chance article is about the forensic use of statistics to identify the mishandling and misuse of statistics in the Environmental Impact Report (“EIR”) on a proposed California desalination project (“the project”) in which hydrogeologists used a statistical model to predict possible creation or exacerbation of seawater intrusion into an aquifer supplying source water for the project. Because the owner of the water utility was a private company, the state Public Utilities Commission (“the Commission”) was responsible for assuring the accuracy of the model’s predictions.
Mishandling
The initial project modeler concealed the inability of the model to provide accurate estimates of water elevation in a seawater-intruded source-water aquifer by including that aquifer together with two others which did not supply source water in the dataset used to determine model accuracy. Although all three aquifers together yielded an acceptable measure of model accuracy, the source- water aquifer-when later independently evaluated alone-did not. Also excluded from the three-aquifer study was a freshwater aquifer shown later to provide about one-half to three-quarters of the project’s source water. When this information was reported to the Commission, it responded by replacing the initial modeler with another to investigate and possibly resolve those problems.
Misuse
How did the second modeler meet this challenge? The model used by both modelers divided the measurement of the water level in an aquifer at a specific time and location into an estimated and an error component. The estimated component was a sum of unique contributions to aquifer water level by different possible influences on it that could vary in extent over time and location, like the rate at which a pump extracted water from the aquifer. In that sum, the influences were weighted statistically to minimize the variation of the error components of the measurements over time and location. All of that so far is good, but here is where the trouble comes. What both modelers did wrong was to attempt to reduce the error variation even further by altering possibly inaccurate measurements of some of the influences without reweighting the model based on the altered information.
The result of that alteration was to create an impermissible predictive relationship between estimates and errors (errors, by definition, being unpredictable) that exaggerated estimates of the rise or fall of aquifer water level over time. That exaggeration, which reflected a negative correlation between estimates and errors, showed up in a consistent underestimation of the decline of aquifer water levels over time that was so troubling that it led the second modeler to reapply the model to changes in water levels rather than to the water levels themselves. Although that reapplication had no foundation in fact and was made solely because time was running out for completion of the modeling work, the Commission accepted the work and certified both the EIR and the project.
Possible Fraud in the Project’s EIR Modeling
According to California law (Civil Code Sections 1709, 1710, 1572, and 1573), fraud may take, among others, either of the following two forms:
1. Concealment Fraud occurs when there is a fiduciary or other relationship between the parties where there is a duty of full disclosure. The concealing person, with an intention to deceive, does not disclose important facts that the concealing person knows but the victim does not and could not know. Further, the victim reasonably relied on and was harmed by the concealment.”
2. Constructive Fraud (negligent misrepresentation) occurs when the perpetrator misrepresents to the victim that an important falsehood is true. But the perpetrator may have honestly believed that the false representation is true. Yet, the perpetrator had no reasonable grounds for believing the representation was true when he or she made it; and he or she intends that victim rely on the representation. The victim must reasonably rely on and be harmed by the false representation”
From these definitions, a judge or a jury might reasonably conclude that the first modeler committed concealment fraud on the Commission when in their contracted relationship the modeler concealed the inability of the model to provide accurate estimates of water elevation in the sea-water- intruded aquifer by including that aquifer with two nonsource- water aquifers in the dataset it used to determine model accuracy.
The concealment fraud perpetrated by the first modeler did not end there, According to the EIR, as noted earlier, the project could draw up to two-thirds of its source water from a freshwater aquifer, only the remainder being drawn from the saltwater-intruded aquifer, but the dataset used to measure the model’s accuracy excluded that freshwater aquifer whose water levels-as shown later by the second modeler-the model could hardly predict at all.
The second modeler avoided concealment fraud-by confirming that the measure of model accuracy for the seawater-intruded aquifer was outside the acceptable range and that the correlation of errors with estimates was far from zero-but perpetrated constructive fraud by providing different interpretations of those numbers to help support the project.
For the measure of model accuracy, the second modeler arbitrarily raised the threshold of acceptability to accommodate the model and for the non-zero error-estimate correlation erroneously attributed it to model bias due to other non-zero correlations with error, including the one noted earlier in which estimates declined more rapidly than water levels.
The article in Chance had four independent peer reviewers representing the American Statistical Association to check its validity. So little or no doubt exists that the two modelers had mishandled and misused statistics in both the evaluation of their model and its application without evaluation to data different from the data the model had been developed to estimate. Although the article fell short of identifying this mishandling and misuse as fraud, a California judge or jury informed of the legal definitions of fraud presented earlier in this section might very likely do so. Identification of a crime and even a criminal, of course, is not enough to go to court.
Needed as well are a legally chargeable defendant and a plaintiff having legal standing. Contract law in California requires that the Commission protect ratepayers from fraud. According to Public Contract Code Section 100(d), “it is the intent of the Legislature for public agencies “ to eliminates favoritism, fraud, and corruption in the awarding of public contracts.” Failing to do that, the Commission as the public agency that contracted with the EIR consultants to do their work certainly qualifies as a chargeable defendant in this case. The ratepayers being the ultimate injured party, the local water management district, whose board is elected by residents constituting a substantial majority of the ratepayers, would clearly have the required standing. Whether legal action takes place depends on whether the Commission continues its support of the project now that it has become aware of the fraud.
Conclusion
Contracted public work, of course, is not the only possible subject of statistical forensic analysis. Governmental support of efforts to control global warming based on predictions by models has already become a subject of such analysis [2]. Even actions by governments ostensibly to control the recent pandemic may be actionable. Post-mortem accounts of the pandemic suggest that its economic and educational upheaval might have been avoided by governmental release of simple statistics like the proportions of people within different age groups who tested positive for the virus but avoided hospitalization. Acting on that information-that the only vulnerable people, aside from the elderly, were almost solely those who also suffered serious health problems like obesity and diabetes-the world could have avoided the mandates that kept adults home from work and children from school, with all the resulting chaos. Concealing those statistics or challenging their validity when exposed very likely constitutes prosecutable fraud by administrative branches of government.
References
- Weitzman RA (2021) From divining rods to statistics: a forensic analysis of the misuse of statistics in the estimation of environmental impact. Chance 34(4): 18-20.
- Weitzman RA (2022) Tuning a model in climatology and calibrating one in hydrogeology: an informative comparison. American Journal of Theoretical and Applied Statistics 11(3): 83-88.