The International Association of Athletics Federations (IAAF) has been forced to submit a revised version of a 2017 scientific Study referenced in support of its Differences of Sex Development (DSD) Regulations, after independent scientists found flaws in the data. Last week, the IAAF released the data used in the original 2017 study (PDF below) to Roger Pielke Jr., Ross Tucker and Erik Boye. As well as finding that the Study had included the times of Russian athletes disqualified for doping, the scientists found three errors in the data used:

• Duplicated athletes – more than one time is included for an individual;
• Duplicated times – the same time is repeated once or more for an individual athlete;
• Phantom times – no athlete could be found with the reported time.

The errors, shown in the table on the right, are explained in Roger Pielke Jr.’s blog post, which outlines the work carried out by the
independent scientists. As shown in the table on the left, they found that the revised study involved 220 data points being eliminated from the original 2017 Study, representing between 17% and 33% of the total data used. As such, they have called for the 2017 Study – published by the British Journal of Sports Medicine (BJSM) – to be retracted (see PDF below).

As The Sports Integrity Initiative has previously reported, the 2017 Study used data collected at the 2011 Daegu IAAF World Championships and the Moscow 2013 World Championships, where it has been alleged that significant doping took place. The Tübingen Study (PDF below) found that between 29% and 34% of the 1,800 athletes taking part in the 2011 Daegu Worlds had doped during the previous year.

Papa Massata Diack, son of former IAAF President Lamine Diack, has admitted that the IAAF held back on announcing doping positives ahead of the Moscow 2013 Worlds, which could mean that athletes who had doped competed. As such, whether the free testosterone (fT) levels of female athletes at either event can be relied on as being endogenous (naturally produced) is subject to debate. 

IAAF response

The revised version of the Study admits that the data set is flawed, but its introduction does not appear to acknowledge the inaccuracies in the data highlighted by Pielke Jr., above. ‘The main criticisms expressed about the data used and our statistical analysis were (1) our ‘concentration’ on free testosterone (fT) rather than total testosterone (T)3 (presumably because the T results were only presented in the Internet version of the paper), (2) the fact that 17.3% of the athletes were sampled at both World Championships (Daegu and Moscow),3 (3) the fact that no correlation analysis was performed other than comparison of fT or T tertiles,3 (4) the absence of statistical comparison between a group with high T levels and a group with normal T levels4 and (5) the lack of adjustment for multiple comparisons (suggesting that the significant differences observed in our study could have happened by chance)’, it reads. Duplicated athletes, duplicated times and phantom times are not mentioned in the introduction to the revised Study.

‘The new [Study] is not peer reviewed, nor are its data publicly available for replication’, writes Pielke Jr. ‘By not being candid about their data errors in [the 2017 Study], Bermon and colleagues have added confusion on top of confusion. This is not how science is supposed to work.’

The IAAF argues that despite the data having been modified, the conclusions of the 2017 Study remain the same. ‘An updated paper using a modified data, set following recent feedback on some of the original data, was accepted and published just last week by BJSM’, wrote an IAAF spokesperson in an email. ‘The revised data shows there is no significant impact on the final conclusions which remain the same. This research is a very specific study into the degree of performance enhancement in elite female athletics where over 90% of those athletes have testosterone levels in the normal female range (0.12nmol/L to 1.79nmol/L) and the degree of performance enhancement is clear in some disciplines (400m to 1 mile). This specific study forms just a small part of 15 years of research which has led to the new regulations, designed to address those athletes that have naturally occurring testosterone levels in the male range of 7.7nmol/L to 29.4nmol/L and should be seen in this context.’

This response is puzzling in a couple of ways. Firstly, it appears to admit that the testosterone levels of 10% of all female athletes registered with the IAAF falls outside of what it considers to be the ‘normal female range’. Secondly, it states that the DSD Regulations are designed to regulate athletes that have naturally occurring testosterone levels within what it considers the ‘male range’ (7.7 nmol/L to 29.4 nmol/L). That is incorrect. Given the importance of the DSD Regulations to an apparently large percentage of female elite athletes, it is important that such descriptions of who the IAAF is regulating are correct. 

The DSD Regulations (PDF below) are designed to regulate athletes with one of seven listed differences in sex development (DSDs) competing in international events run between 400m and one mile in the IAAF female category, if their testosterone levels are above 5 nmol/L and have an ‘androgenising effect’ (i.e. if that testosterone is taken up by their androgen receptors and boosts their physiology). In other words, the DSD Regulations are applicable to females whose natural testosterone levels fall significantly below what the IAAF considers to be the ‘male range’.

The IAAF’s Explanatory Notes to the DSD Regulations (PDF below) explain that women with Polycystic Ovary Syndrome (POS) could have circulating testosterone up to 4.8 nmol/L. ‘Therefore, the only female athletes competing with levels above 5 nmol/L would be intersex/DSD athletes, doped athletes, and athletes with adrenal or ovarian tumours’, it argues.

In the Explanatory Notes, the IAAF justifies that 5 nmol/L limit as necessary because it argues that below 5 nmol/L, ‘there is limited evidence of any material testosterone dose-response’, and because ‘an increase in circulating testosterone from normal female range up to between 5 and 10 nmol/L delivers a clear performance advantage’.

Potential implications

Testosterone is an androgenic hormone that is naturally produced by the human physiology. The reason it has been so successful as a doping agent is understood to be because the human physiology does not distinguish between testosterone that is naturally produced (endogenous) by an individual’s physiology, and testosterone that is added to their physiology (exogenous). 

It is understood that this makes testosterone especially effective as a doping agent in female sport. Females generally have lower baseline levels of testosterone, so females are able to take less than males would require to equate a significant performance advantage.

The 2017 Study is only referenced in support of the DSD Regulations. The main research cited to back the DSD Regulations is a 2018 Study, ‘Circulating Testosterone as the Hormonal Basis of Sex Differences in Athletic Performance’, which is due to be published by Endocrine Reviews. The authors of the study are Professor David Handelsman of the University of Sydney, Professor Angelica Hirschberg of Karolinska Institutet and Dr. Stéphane Bermon, who authored the 2017 Study.

The IAAF has admitted that the 2018 Study involves giving athletes exogenous testosterone. Giving athletes exogenous testosterone is doping. It will provide a performance advantage, as an athlete’s physiology has something extra that was not naturally produced by their physiology. Additional exogenous testosterone is not what the DSD Regulations seek to regulate.

When you read about an athlete being sanctioned for ‘testosterone’, it means that they have added exogenous testosterone to their physiology – the same method apparently used by the 2018 Study in support of the IAAF’s DSD Regulations. Whether this can be equated as being the same as any advantage that endogenously produced testosterone of above 5 nmol/L might provide a female athlete with a DSD in one of the Restricted Events outlined by the IAAF is a question the Court of Arbitration for Sport (CAS) may have to consider, perhaps in the Caster Semenya case.

The IAAF argues that other studies support its conclusions and – to be fair – increasing an athlete’s endogenous testosterone and measuring the effects would be almost impossible. None of these studies have been seen by The Sports Integrity Initiative. We therefore do not know if comparisons as to whether the level of advantage allegedly enjoyed by athletes regulated under the DSD Regulations is greater than other natural physiological advantages, such as lung capacity, living at altitude, or height (in IAAF events such as high jump and pole vault).

It remains to be seen if the BJSM will accept the call for the 2017 Study to be retracted. Dr. Stéphane Bermon, the IAAF’s Chief Medical Officer, was involved in both studies. It also remains to be seen whether Endocrine Reviews will also reconsider its publication of the 2018 Study, which provides the main support for the DSD Regulations.

Touché

Earlier this year, The Sports Integrity Initiative retracted, revised and rewrote an article following valid complaints about inaccuracies from the IAAF. The IAAF asked to respond to the rewritten article, which it did, largely by attacking the withdrawn article. 

The conclusions of both articles was the same. This was that the IAAF has not sufficiently demonstrated that the degree of advantage held by athletes regulated under the DSD Regulations is so much greater than other natural physiological advantages that it is necessary to exclude them from international female competition in its Restricted Events. The studies it cites may achieve this, but nobody has yet seen them.

Concerning the 2017 Study, the IAAF’s argument appears to be that as the conclusions remain the same, the the data inaccuracies are not important and the Study does not need to be retracted, revised, or rewritten. Whether the BJSM agrees remains to be seen. 

Whether this will affect the planned implementation of the DSD Regulations in October is unknown. The 2018 Study used to underpin the Regulations involves one of the same authors as the 2017 Study, and is understood that both studies were commissioned by the IAAF. Developments are likely to be keenly watched by the CAS, where Caster Semenya is challenging the DSD Regulations.