Last week the APA released its report on ex-gay therapy, to a somewhat muted response from the charlatans of the ex-gay political machine.  Oh yes…we’re so very happy that the APA acknowledges that a patient’s religious needs must be taken into account, they said, politely skimming over the overwhelming evidence that trying to force gay people into straight jackets harms them deeply.  You had to expect they wouldn’t leave it at that.

Now comes the "final" release of the Jones and Yarhouse "study" of ex-gay "therapy"…touted in that well known scientific peer reviewed publication, the Baptist Press…

Study: Ex-gay ministry has 53 percent success rate

Sure it does.  You read through the brief article for a while and, of course, you see little nuggets like this one pop out at you:

Jones expressed frustration that the APA task force didn’t take their 2007 study seriously.

"They selectively apply rigorous scientific standards," he said…

Yes.  Of course.  It’s all a consperacy of the scientists to further the militant homosexual agenda.  Oh…have I meantioned that Exodus paid Jones and Yarhouse for their labors?  Naturally that didn’t affect their scientific rigorousity I’m sure.

Or…not…

While Jones and Yarhouse’s study appears to be very well designed, it quickly falls apart on execution. The sample size was disappointingly small, too small for an effective retrospective study. They told a reporter from Christianity Today that they had hoped to recruit some three hundred participants, but they found “many Exodus ministries mysteriously uncooperative.” They only wound up with 98 at the beginning of the study (72 men and 26 women), a population they describe as “respectably large.” Yet it is half the size of Spitzer’s 2003 study.

Jones and Yarhouse wanted to limit their study’s participants to those who were in their first year of ex-gay ministry. But when they found that they were having trouble getting enough people to participate (they only found 57 subject who met this criteria), they expanded their study to include 41 subjects who had been involved in ex-gay ministries for between one to three years. The participants who had been in ex-gay ministries for less than a year are referred to as “Phase 1″ subpopulation, and the 41 who were added to increase the sample size were labeled the “Phase 2″ subpopulation.

This poses two critically important problems. First, we just saw Jones and Yarhouse explain that the whole reason they did a prospective study was to reduce the faulty memories of “change experiences that happened in their pasts” — errors which can occur when asking people to go back as far as three years to assess their beginning points on the Kinsey and Shively-DeCecco scales. This was the very problem that Jones and Yarhouse hoped to avoid in designing a prospective longitudinal study, but in the end nearly half of their results ended up being based on retrospective responses.

-Jim Burroway, Box Turtle Bulletin,  September 17th, 2007 – A Preliminary Review of Jones and Yarhouse’s "Ex-Gay? A Longitudinal Study"

[Emphasis mine] So basically their data was corrupted by the same half-assed sloppiness of the Spitzer study.  Oh but wait…it gets better.  Again from Burroway…

Whenever a longitudinal study is being conducted over a period of several years, there are always dropouts along the way. This is common and to be expected. That makes it all the more important to begin the study with a large population. Unfortunately, this one wasn’t terribly large to begin with; it started out at less than half the size of Spitzer’s 2003 study. Jones and Yarhouse report that:

Over time, our sample eroded from 98 subjects at our initial Time 1 assessment to 85 at Time 2 and 73 at Time 3, which is a Time 1 to Time 3 retention rate of 74.5%. This retention rate compares favorable to that of the best “gold standard” longitudinal studies. For example, the widely respected and amply funded National Longitudinal Study of Adolescent Health (or Add Health study reported a retention rate from Time 1 to Time 3 of 73% for their enormous sample.

The Add Health Study Jones and Yarhouse cite began with 20,745 in 1996, ending with 15,170 during Wave 3 in 2001-2002. But this retention rate of 73% was spread over some 5-6 years, not the three to four years of Jones and Yarhouse’s study.

What’s more, the Add Health study undertook a rigorous investigation of their dropouts (PDF: 228KB/17 pages) and concluded that the dropouts affected their results by less than 1 percent. Jones and Yarhouse didn’t assess the impact of their dropouts, but they did say this:

We know from direct conversation that a few subjects decided to accept gay identity and did not believe that we would honestly report data on their experience. On the other hand, we know from direct conversations that we lost other subjects who believed themselves healed of all homosexual inclinations and who withdrew from the study because continued participation reminded them of the very negative experiences they had had as homosexuals. Generally speaking, as is typical, we lost subjects for unknown reasons.

Remember, Jones and Yarhouse described those “experiencing difficulty with change would be likely to get frustrated or discouraged early on and drop out of the change process.” And so assessing the dropouts becomes critically important, because unlike the Add Health study, the very reason for dropping out of this study may have direct bearing on both questions the study was designed to address: Do people change, and are they harmed by the process? With as much as a quarter of the initial population dropping out potentially for reasons directly related to the study’s questions, this missing analysis represents a likely critical failure, one which could potentially invalidate the study’s conclusions.

[Emphasis mine] Harm…what harm?  We didn’t speak to anyone who was harmed…

But look a tad more closely at what Jones and Yarhouse "know"…

On the other hand, we know from direct conversations that we lost other subjects who believed themselves healed of all homosexual inclinations and who withdrew from the study because continued participation reminded them of the very negative experiences they had had as homosexuals.

Healed.  Healed.  They believed themselves healed.  Not cured.  Not changed.  But…healed.  This is the language of religion, not science.  And now you know where Jones and Yarhouse were coming from, and why they were good with allowing data into their study that could only weaken it from a scientific point of view. 

It didn’t matter.  They needed bodies to get a big enough sample size that they could plausibly go on with it and give the kook pews something they could wave around and claim that scientists were conspiring against them on behalf of the godless homosexual menace.  They would have known going into it, that the APA would regard their study as flawed because they engineered the flaws into it themselves.  Anyone who was serious about it would have gone back to their funding and told them they couldn’t do it without more first year subjects (a lot more), and more participation from the drop-outs.  But they kept on with it anyway.  Because knowing whether or not ex-gay therapy works wasn’t the point.  Knowing whether or not it harms the very people it purports to help wasn’t the point.  Having something to wave back at the APA was the point.  That promise that they would report the results whether or not they embarrassed Exodus was as empty as the promise that "change is possible".   Neither one had a money back guarantee.

[Update…]  Yarhouse is identified Here, as an evangelical psychologist and graduate of Regent University.  Regent is Pat Robertson’s baby.  This man is as likely to be objective about ex-gay therapy as he is to be a flying pig.   Jones is of Wheaton College, which is described by The Princeton Review’s Best 351 Colleges thusly: "If the integration of faith and learning is what you want out of a college, Wheaton is arguably the best school in the nation with a Christ-based worldview."   Well this team really looks like a couple of objective researchers to me…

[Update again…]  Timothy Kincaid at Box Turtle Bulletin goes another round with this "study"…finds it not too much different from the previous round…

In short, the Jones and Yarhouse study was funded and fully supported by Exodus and conducted by two researchers who were avid supporters of ex-gay ministries. They wanted to study 300 participants, but after more than a year, they could only find 57 willing to participate. They then changed the rules for acceptance in order to increase the total to 98. After following this sample for 4 years, 25 dropped out. Of the remainder, only 11 reported “satisfactory, if not uncomplicated, heterosexual adjustment.” Another 17 decided that a lifetime of celibacy was good enough.

Good enough for the Baptist Press!