I found this article interesting, and while its not specific to the aging research (Geroscience) field, it seems relevant as I’ve noticed increasing numbers of the paper I see in the aging field are coming out of China. I also wonder about the quality of the research papers in the geroscience field coming out of Russia, given that the entire country seems to be very corrupt.
China’s fake science industry: how ‘paper mills’ threaten progress
The country has become a prolific producer of academic research but fraudulent studies risk serious real-world consequences
…
“To survive in Chinese academia, we have many KPIs [key performance indicators] to hit. So when we publish, we focus on quantity over quality,” says a physics lecturer from a prominent Beijing university. “When prospective employers look at our CVs, it is much easier for them to judge the quantity of our output over the quality of the research,” he adds. The world’s scientific publishers are becoming increasingly alarmed by the scale of fraud. An investigation last year by their joint Committee on Publication Ethics (Cope) concluded: “The submission of suspected fake research papers . . . is growing and threatens to overwhelm the editorial processes of a significant number of journals.”
“The worst impact is on sincere Chinese researchers,” says Bimler. “There is enough junk coming from China that researchers privately admit that they don’t read papers if they’re from a Chinese source . . . Scientists don’t have time to determine what is junk and what isn’t.”
Fake Publications in Biomedical Science: Red-flagging Method Indicates Mass Production
Background: Integrity of academic publishing is increasingly undermined by fake
science publications massively produced by commercial “editing services” (so-called “paper
mills”). They use AI-supported, automated production techniques at scale and sell fake
publications to students, scientists, and physicians under pressure to advance their careers.
Because the scale of fake publications in biomedicine is unknown, we developed a simple
method to red-flag them and estimate their number.
Methods: To identify indicators able to red-flagged fake publications (RFPs), we sent
questionnaires to authors. Based on author responses, three indicators were identified:
“author´s private email”, “international co-author” and “hospital affiliation”. These were used
to analyze 15,120 PubMed®-listed publications regarding date, journal, impact factor, and
country of author and validated in a sample of 400 known fakes and 400 matched presumed
non-fakes using classification (tallying) rules to red-flag potential fakes. For a subsample of
80 papers we used an additional indicator related to the percentage of RFP citations.
Results: The classification rules using two (three) indicators had sensitivities of 86%
(90%) and false alarm rates of 44% (37%). From 2010 to 2020 the RFP rate increased from
16% to 28%.
Given the 1.3 million biomedical Scimago-listed publications in 2020, we
estimate the scope of >300,000 RFPs annually. Countries with the highest RFP proportion are Russia, Turkey, China, Egypt, and India (39%-48%), with China, in absolute terms, as the largest contributor of all RFPs (55%
Sounds like there is lots of evidence (aside from this paper) that China and Russia have significant issues with “fake” papers, but data in this paper is not great.
I don’t want to go into the politics but I highly doubt the Ukraine produces the most trustworthy scientific literature in the world even though the Ukraine score very high on the corruption index
This is the reason why I currently chose quercetin over dihydroquercetin despite there is someone in my country and in Russia claim that the latter is more potent. And I haven’t bought sunflower disk peptide yet despite some articles discussing them on some official WeChat accounts. The rule by man situation made the science and medicine less reliable here (e.g. one mediocre psychiatrist here can become Chief Psychiatrist if he/she is very good at the politics of the office, although not all the Chief Psychiatrist are mediocre, one one mediocre Chief Psychiatrist can already ruin the lives of many patients), so I usually have to rely on the peer views of foreigners to make sure I won’t buy something useless.
The broader issue of what science you can rely on is quite complex. If we simply take the “sirtuins”. There is a form of conventional wisdom that they are pro longevity. However, they are HDACs and there is evidence that HDAC inhibitors are pro longevity.
If you follow Charles Brenners arguments on twitter where he links to various papers, I think he is right in that sirtuins are not miracle longevity genes.
David Sinclair does a good job of publicly promoting this thesis, but I don’t think he is right.
Yes we also get conflicts of interest and faked papers (although not as many faked papers as are claimed). However, in the end science has always been quite like this and that science has moved on to a limited extent funeral by funeral.
I think we are already doing this I personally cross check with other studies if metastudies are not available on certain studies. If results are contradicting I also research who was behind study and what are his interests and look for confirmation in other studies. Besides other things I do I was consulting a supplement company and did see from inside how studies are planned and executed and how interpretation of results is sometimes really a bit strained in favor of the objective of the research. And there is promotion of the results that can help spread like fire the misinformation. Sometimes it is really hard to get to the truth as there are so much we know at the moment and people from the beginning of time handle this ambiguity and uncertainty with difficulty. We search for oracles, people who can explain and predict the future. But in many cases this oracles are as lost as we are.
Right. This is where I got the 50% of Russian papers are fake info. According to Stuart Ritchie, it’s the highest of any country at just over 48% (close enough to 50%). Unless I misunderstood his tweet infographic.
Probably the simplest measurement of “fake” papers will end up being “cited” papers, although I don’t have the time nor access to the data to do this. On the door of my lab from grad school we had a chart taped up from an article which if I remember correctly was from the journal Science or Nature and listed the major academic/scientific disciplines and the average paper in that field’s number of times it was cited. The thought was the more times subsequent researchers cited the papers in their own paper, the more useful the cited reeearch likely was. The chart listed them in order of the citations: physics papers had 23 on average (I’m making up numbers: it was 28 years ago, but roughly correct), chemistry had 16, biochem had 13, geology had 8, and then dwindled to below 1 for psychology, and far below 1 for art history (meaning out of 100 of the average art history papers, one was cited once by subsequent withers — doesn’t paint a pretty picture of future worth). I assume if you did this by country of research organization you might see similarly stark results.
From above post in this thread:
Good advice. I don’t have the time to figure out what is fake so I generally ignore Chinese and Russian sources of anything. North Korean too, but they don’t seem to be prolific in the longevity space.
The only Chineee paper which does cause me some consternation is a study of 11k people taking NAC who appear to have a larger incidence of accelerated knee osteoarthritis (referenced in another thread here, I believe in the GlyNAC or “useful supplements” thread). I have been doing heavy weightlifting for the past six month and making excellent progress (which real results both in strength, body composition, and I believe increased hormonal activity) but I can “feel” my knees. I certainly don’t want to damage them, but it makes sense that if I’m lifting heavy my muscles will be sore from the effort. So it’s stuck in my mind more than I care to admit.
I personally cross check with other studies if metastudies are not available on certain studies. If results are contradicting I also research who was behind study and what are his interests and look for confirmation in other studies. Besides other things I do I was consulting a supplement company and did see from inside how studies are planned and executed and how interpretation of results is sometimes really a bit strained in favor of the objective of the research. And there is promotion of the results that can help spread like fire the misinformation. Sometimes it is really hard to get to the truth as there are so much we know at the moment and people from the beginning of time handle this ambiguity and uncertainty with difficulty. We search for oracles, people who can explain and predict the future. But in many cases this oracles are as lost as we are.