Print Digg

Article

Pondering Science Labor

Friday, August 31, 2018


To raise new questions, to regard old questions from a new angle, requires creative imagination, and marks real advances in science.  —Albert Einstein   


Science advances slowly on a foundation of trusted discoveries and intense labor. Being able to replicate scientific findings is crucial for scientific progress.


However, according to geophysicist, Marcia McNutt, PhD, former editor of Science magazine, current president of the National Academies of Sciences, and the chair of the National Research Council, "There are a number of reasons why peer-reviewed preclinical studies may not be reproducible." 


McNutt suggests that "the system under investigation may be more complex than previously thought, so that the experimenter is not actually controlling all independent variables. Authors may not have divulged all of the details of a complicated experiment, making it irreproducible by another lab. It is also expected that through random chance, a certain number of studies will produce false positives. 


"If researchers are not alert to this possibility and have not set appropriately stringent significance tests for their results, the outcome is a study with irreproducible results. Although there is always the possibility that an occasional study is fraudulent, the number of preclinical studies that cannot be reproduced is inconsistent with the idea that all irreproducibility results from misconduct in research."


Social science replication


A new article published this week under the science and public section of Science News addressed replication and how social science studies are done.  


The researchers point out that what started out a few years ago as a crisis of confidence in scientific results has now evolved into an opportunity for improvement. Both researchers and journal editors are exposing how studies get done and are encouraging independent redos of published reports. 


The research team in question, led by Caltech economist Colin Camerer, examined 21 social science papers published in two major scientific journals, Nature and Science, between 2010 and 2015. Five replication teams directed by coauthors of the new study successfully reproduced effects reported for 13 of those investigations. The researchers reported these results online this week in Nature Human Behaviour.


The new study reinforces the need for most scientific disciplines to view any single study with caution. Fortunately this is a lesson that most nutritional supplement product designers, nutrition science writers, and nutrition journal gatekeepers learned years ago. 


The authors of this new social science study point out a troubling aspect of unsuccessful experimental redos actually included four or five times as many study participants as originally studied, with the statistical strength to detect actual effects becoming weaker than reported for the initial investigations. In other words, the best replications, which exceeded initial studies in their ability to detect actual effects, are frequently only partially successful. 


On the other hand, scientific journals in many disciplines tend not to publish studies that disconfirm previous findings, leaving initial findings unchallenged, particularly in the social sciences. 


The good news: The new report suggests that such practices are changing. In the past few years, more and more journals are requiring peer review before submitting research papers for publication review. 


This is considered an obvious way to bolster peer reviews, filter out weak studies, and help scientists of all ages submit better research articles for publication.


Ellen Troyer, with Spencer Thornton, MD, David Amess and the Biosyntrx staff




 

PEARL

For our readers who might not know, the National Academies of Sciences and their committees have been advising our nation on issues of science and medicine for more than 150 years with written reports submitted to Congress that should affect policy and decision making. Here is the PDF link to their 2017 report. It addresses:

 

  • Education and social issues 
  • Health and safety
  • Natural resources 
  • Science, technology, and research

The Biosyntrx founders, staff, and scientific advisors hope our readers have a thoughtful and fun 2018 Labor Day weekend.