A new study on conversion therapy, out of TCD’s School of Nursing and Midwifery, is causing a bit of a stir this week, with a number of media pieces highlighting various claims from the report, and Minister Roderick O’Gorman speaking forcefully of the need to ban conversion therapy.
Particular attention has been paid to a claim, made by one respondent, that they were subjected to electro-shock treatment (ECT) when they were 12 years old.
However, a review of the report’s methodology highlights some rather unusual aspects of the report, including the involvement of several groups currently lobbying for the banning of conversion therapy in the design of the survey.
The report was commissioned by the Department of Children, Equality, Disability, Integration and Youth, and the authors of the report thank the Department “for providing guidance and support throughout.” It is worth noting the Government, prior to commissioning this study, had already committed to banning conversion therapy. In fact, when the research was commissioned Minister Roderic O’Gorman said that the purpose of the research was to “ensure that the legislation [banning conversion therapy] is based on reliable evidence.”
The first thing I have to highlight is something which the report itself states clearly, on page 71, but which none of the media reports I’ve seen have bothered to note – “the survey used a non-probability sample and is therefore not statistically representative of the wider LGBT+ community.” This is to say that the report actually tells us nothing about the prevalence of conversion therapy practices in Ireland, nor the type of person it occurs to, nor the common forms which conversion therapy takes in Ireland.
The decision to use a non-representative sample is particularly odd given a) the government funding of the research, and b) the statement that establishing: “who is subjected to conversion practices,” “if there are longer term consequences of such practices for the individual,” and “if there are any support needs for people who have been subjected to conversion therapy” are the objectives of the study.
Now I may be a simple hack, and not a distinguished group of TCD academics, but it would seem obvious to me that you can’t establish much of anything about a population if you’re using a non-representative sample, and I can’t quite figure out why you would pick the objectives of your study and then implement a design that makes it literally impossible to achieve those objectives.
Anyway, the sample size of the survey were indeed rather small. The whole sample was 278 people, with only 70 of those saying they had been offered conversion therapy and only 38 saying they had experienced conversion therapy. I’m unsure, having read the report, if those 38 people are a subset of the 70 who were offered conversion therapy, or an entirely separate group. I presume the former.
If you limit the survey respondents to those who experienced conversion therapy within the last 25 years, that sample size drops from 38 to 21 – 24 people. I’d like to give a special nod to Gay Community News here, as they somehow misread the study as having shown “278 people who were subjected to ‘conversion therapy’ practices in Ireland.” Wrong on the numbers, and wrong again as not all of the people who said they experienced conversion therapy said they experienced it in Ireland.
It is worth saying that the claim that a respondent underwent ECT when he was 12 is noted just once in the report, in a footnote on page 49, and absolutely no details are provided. Given that some of the participants said they underwent conversion therapy 30-40 years ago, with more saying they would prefer not to say when they had undergone conversion therapy, that incident, assuming it occurred as described, could have happened 40 years ago. I have no earthly idea why more, as in any, information on this particular incident isn’t included in the report.
It should also be noted that the study says that interviews were only conducted with people who had experienced conversion therapy within the last 25 years, and that they choose a limit of 25 years because they wanted to focus on modern conversion therapy instead of what may have occurred to people when conversion therapy was “a standard medical practice.” That seems to me to be a very reasonable concern and a fair cut-off.
What I don’t understand is why this wasn’t applied to survey responses – the survey picked up data from people who had undergone conversion therapy as far back as 30 – 40 years ago. Again, only 21 -24 people stated they had experienced conversion therapy in the last 25 years.
The Government didn’t just fund this study, they appear to have been deeply involved in its design and dissemination. A “research advisory group…provided guidance on the survey design, the interview guide, dissemination of the survey and participant recruitment for the interviews.” It’s unclear exactly how much influence that group had but, given that it contained representatives from the Department on it, and given that the Department commissioned this report, it would seem fair to assume that it had a fair amount of influence on this research.
The group also contained representatives from LGBT Ireland; Gay Project [Cork]; the Transgender Equality Network Ireland (TENI); and “the psychotherapy services.” So the advisory contained: the people paying for the research; an advocacy group, an advocacy group, another advocacy group, and one which has had its own share of bad press recently; and at least one actual professional.
I would note that LGBT Ireland responded to the publication of this report with a tweet which read “The new Government report on conversion practices confirms what we already knew,” which would certainly be my concern.
The new Government report on conversion practices confirms what we already knew – that conversion practices are dangerous, harmful and should be banned.
The Government must act on this report and legislate to ban conversion practices in Ireland as a matter of urgency.
— LGBT Ireland (@LGBT_ie) February 17, 2023
It should go without saying that allowing not one, but several, advocacy groups to influence academic work has historically been considered rather a bad move. However, the fun doesn’t stop there.
One of the individuals thanked in the report is a “Mr Alan Edge from LGBT Ireland” for his “invaluable assistance to the research team.” Edge is a campaign officer for LGBT Ireland, as well as an Independent Cllr on South Dublin County Council. In his role at LGBT Ireland he oversees the Ban Conversion Therapy Campaign. The report notes that this campaign “created a steering committee which also supported the recruitment of participants and promoted the research.”
The report doesn’t state how many participants were sourced directly from this campaign group.
This report, which again was designed and commissioned to inform Government policy, seems to have willingly invited in an individual who is deeply involved in both the activism and politics of banning conversion therapy. And this person was let control the recruitment of an untold number of the participants in the study. Serious researchers tend to put considerable effort into protecting their results from being biased by either external or internal factors and so I’ve got to say I’m actually pretty impressed with the sheer audacity of just giving an activist, at least partial, control over your research.
Oh, and as the report doesn’t name the people on the research advisory group, but does note that that group contained representatives from LGBT Ireland, Edge may have sat on that group as well.
The report does not contain a full version of the survey questions, and the survey itself appears to have been taken offline, so I can’t comment on that side of things. The report does state that all of the respondents to the survey were anonymous, which is just fantastic given what we’ve discussed up to this point.
None of this should be taken to suggest that Edge acted in any way inappropriately, but involving an activist in this fashion is inexcusable. There are ways of doing things in order to mitigate the possibility of bias tainting your results, and involving every advocacy group and activist on God’s green earth in your research is not in line with those mitigations. And then, on top of that, you put in fully anonymous responses to your survey, which opens the door to activist manipulation on an unquantifiable scale.
What I wonder is what exactly the Department of Education was doing all through this process. They had to have known about the design of the study, which means they should have known all of the issues with it. Given that, and that the Department paid for this work, one would have to assume that the Department actively supported the study design and the involvement of the advocacy and activist groups; they may even have been the people who suggested the inclusion of these groups. I, for one, would be rather interested in knowing exactly why the Department supported the research being conducted in this fashion, rather than in a more rigorous one, and if it was the Department, rather than the TCD academics, who suggested this design.
Loading ...