Witnesses using bogus statistics

I explained below how court psychologists use bogus statistics to bluff the court. Now a statistics professors offers a list of ways to separate lies from statistics:

1. “Focus on how robust a finding is, meaning that different ways of looking at the evidence point to the same conclusion.”

2. Don’t confuse statistical with practical significance.

3. “Be wary of scholars using high-powered statistical techniques as a bludgeon to silence critics who are not specialists.”

4. “Don’t fall into the trap of thinking about an empirical finding as ‘right’ or ‘wrong.’ At best, data provide an imperfect guide.”

5. “Don’t mistake correlation for causation.”

6. “Always ask ‘so what?’”

7. Make your data and analysis public.

Sure enough, the Jodi Arias psycho experts made all 7 mistakes.

Some of the best questions came from the jury:

10:16 p.m. ET: Juror question: Why did you write in autopsy report that the membrane that protects the outside of the brain was intact but the brain was damaged?

Dr. Horn said that was a typo, and it’s just part of their standard from.

Wow, how could he make this mistake? And how did all the lawyers miss this?

I guess the psychologist Geffner caught this. The significance is that the prosecution claims that Arias shot Alexander after the stabbings and throat-slashing. Arias claims that she shot first. If the prosecutor is right, then she is surely still lying to cover up her guilt, but I was unconvinced that she shot last.

The closing arguments are now being given in the Arias trial. The evidence for her guilt is overwhelming, and the only question is how much the jury was softened and confused by psychologist testimony. The validity and relevance of that testimony is questionable. The core scientific backbone is the DSM-IV diagnostic criteria, and the use of multiple-choice questionnaires to help make diagnoses. Now I read this:

In a potentially seismic move, the National Institute of Mental Health – the world’s biggest mental health research funder, has announced only two weeks before the launch of the DSM-5 diagnostic manual that it will be “re-orienting its research away from DSM categories”.

Update: The defense attorney Nurmi is making its closing statement, and saying that the case is "not about the sexual orientation of the witnesses". He is referring to Alyce LaViolette, the domestic violence expert. Her expertise was not based on any facts or research, but her personal experiences. And her experience, apparently, is that she has been unable to have a successful relationship with a man. Yes, the case is not about the witnesses, but common sense ought to tell you than her domestic violence opinions were bogus.

Nurmi's main argument so far is that if Arias had carefully planned the murder, then she would not have made the mistakes that she did. Also, if she drove 1000 miles to kill Travis, then why did she bother to have sexual relations with him before killing him? Why would she take pictures? Nurmi says that doesn't make any sense. It makes sense to me if she wanted vengeance, and she was going to delete the pictures. My guess is that she went there to try to talk Travis into taking her to Cancun instead of another woman, and was enraged when he refused.

Update: here is more on DSM-5 criticism:

The DSM is a manual for diagnosing and classifying mental disorders and is widely used by doctors and researchers. As Thomas Insel writes in his post, the goal of the manual is to “provide a common language for describing psychopathology.” The problem, however, is a “lack of validity.” He writes:

Unlike our definitions of ischemic heart disease, lymphoma, or AIDS, the DSM diagnoses are based on a consensus about clusters of clinical symptoms, not any objective laboratory measure. In the rest of medicine, this would be equivalent to creating diagnostic systems based on the nature of chest pain or the quality of fever.

The alternative, Insel writes, is a new NIMH project called Research Domain Criteria (RDoC), which will incorporate genetics, imaging, cognitive science, and other information into diagnoses. Mental health researchers are taking on more molecular- and cellular-focused tools , such as those based on DNA analysis or brain scans, in the hopes of improving diagnoses and treatment (see “The Hunt Is On for Biological Signs of Post-Traumatic Stress Disorder” and “Why Obama’s Brain-Mapping Project Matters”).

The hunt is for this:

“The goal is to develop quantitative biomarkers, such as from a blood test, urine analysis, or fMRI, that can be used to determine objectively if someone has PTSD,” says Len Polizzotto, Draper’s vice president in charge of the program. The markers could also help reveal whether treatment is working, he says.

That would make psychiatry a lot more scientific, but it is just not possible today.

Since I became a science writer 30 years ago, I have heard countless claims about breakthroughs in our understanding and treatment of mental illness. And yet as the NIMH decision on the DSM indicates, the science of mental illness is still appallingly primitive.

Update: Here is another good article on how psychologist could become more scientific. And here is another psychology fraud about a study that "suggested that dancing is a signal for sexual selection in humans". Plausible conclusion, but the data was suspect. Here are more statistical tips.