p[sychlogo


ORI RESPONSE
Based on the 2013 disclosure and coverage of Adam Savine’s data fraud case, we felt it important to post further details on our side of the story, and its relationship to the CCP Lab and research. The information is covered in the form of a FAQ for interested collaborators, colleagues, and the public-at-large.
Links to Reports on the Case


 


FAQ's


 

What is the connection between Adam Savine and the CCP Lab?
Adam was a graduate student in the lab from 2007 – 2012, where he worked under the supervision of Dr. Todd Braver. During his time as a graduate student, Adam also collaborated with other faculty (Mark McDaniel, Roddy Roediger) and students in the department. Adam left the lab in 2012, prior to suspicions of his data fraud, in August 2012. He was planning to return for his Ph.D. dissertation defense at the end of August, but when suspicions of the data fraud arose and the Research Integrity investigation began, the defense was called off. Adam currently has no affiliation with the lab, and we have had no contact with him since August 2012.


What is Adam Savine’s current status in academia?
To our knowledge, Adam no longer has any academic affiliation. He was prepared to begin a post-doctoral fellowship at University of Michigan, working with John Jonides, but to our knowledge that position was also terminated as soon as Dr. Jonides was made aware of the investigation. We are not currently in contact with Adam, and do not have any direct knowledge of his current whereabouts.


What events led to Adam’s data fraud being discovered?
Adam had turned in a final draft of his dissertation in July 2012, and was planning to give an oral defense of it at the end of August. He left St. Louis during that period. During the same period, concerns regarding Adam’s ethical integrity arose. This led to increased suspicions regarding his dissertation findings. Adam was requested to make all data, analyses and results files available to Dr. Braver. Moreover, he was asked to provide a step-by-step “walk-through” description of each analysis and figure. Adam eventually did make the files available. During review of these files, Dr. Braver noticed some surprising discrepancies and directly confronted Adam about them. Specifically, Adam was instructed that he would not be allowed to hold his defense until he could explain the data to Dr. Braver’s satisfaction, and attest to its accuracy and validity. Eventually, Adam acknowledged the discrepancies and agreed to cancel the defense. Dr. Braver immediately reported Adam to the WUSTL office of Academic Integrity, thus beginning the investigation. When the investigation began, Adam’s computer and the associated data files were impounded by the Research Integrity committee. We were explicitly instructed to not examine or interfere with any data connected to Adam during the course of the investigation. During this investigation, which was initiated based only on the unpublished dissertation data, Adam was also questioned about other research that he had been involved with. Adam admitted to data fraud on 2 papers that were published with members of the CCP lab (plus a third paper involving Dr. McDaniel’s lab), including Dr. Braver as senior author and principal investigator. These papers involved datasets that were independent from his dissertation research (i.e., the research that initiated the investigation).

When did Dr. Braver and other members of the CCP lab first learn of the data fraud on the published papers?
Our knowledge of the data fraud regarding the published papers was gained the same day it was revealed publicly (March 4, 2013) on the HHS ORI website. We learned about it after receiving email from colleagues. There was no forewarning to us that the investigation had been concluded, what had been learned, or the data and papers involved. Currently, our knowledge regarding the specifics of the data fraud is limited to what is published on the HHS ORI website in its final case summary.

What is the current and future status of the two papers listed by ORI as containing data fraud?
We will comply with guidance from ORI and WUSTL, with plans to retract these papers very soon. They are no longer available from our lab website. We must emphasize that we have not yet had the opportunity to thoroughly inspect and re-analyze the data from those papers, but we definitely plan to do this. In this way we hope to determine exactly which findings are valid, and which are not. Once this is known, we plan to report the true findings back to the community.

Why are some of the papers containing data fraud cited in subsequent papers from the CCP lab in recent months?
We received official word that these papers contained data fraud, and were not informed of any allegations about them before March 4, 2013. Moreover, since the investigation began, the data has been sequestered and we were instructed not to examine anything ourselves.

Are the subsequent papers that cite Savine’s work on the papers in question also subject to retraction or increased skepticism?
Some of our recent papers have cited the prior Savine papers as theoretical, conceptual, and/or experimental precedents for follow-up studies, before we knew anything about data fraud. Nevertheless, this subsequent work has involved distinct experimental preparations and methodologies, and so should stand on its own, as independent verifications of the basic ideas. In particular, we feel that the subsequent papers should be considered on their own merit, which we believe is standard practice in science. Indeed, we believe that this represents one of the most important and positive aspects of scientific research – that it is objective, should be replicable in principle, and is self-correcting. Thus, we have no plans to amend or retract any of the subsequent papers that cited the previous work.

What about future citations to the work in question?
Since learning about the investigation outcome, we plan to stop citing the research in the two papers in question, at least until we are able to discover exactly which findings from those papers are accurate, and which are not. We recommend that other researchers also do the same.

Are there problems with any other papers that have Savine’s name on them?
At this point, we do not have any reason to believe there is data fraud or questions with any papers other than the ones specifically identified by the ORI investigation. Once the data in question is released to us, we will attempt to more thoroughly assure ourselves of this assertion.

What is your obligation to the NIH, who funded the research in the papers and dissertation in question?
Dr. Braver was the Principal Investigator (PI) on the NIH grants that funded the research in question. However, the work that Adam was involved with constituted only a small portion of that research funding. At this point, we are not aware of any formal obligation to the NIH based on the ORI findings. However, we take very seriously our role and obligation as stewards of these research funds. Consequently, we will make every effort to discern and discover exactly what are the valid and accurate findings in the research that Adam reported. Once we have thoroughly re-analyzed the data we will report the true findings back to the community (whether in a publication or technical report).

Isn’t it reasonable to have suspicion and distrust of other work that comes out of the CCP lab?
We need to emphasize that Adam acted alone and of his own accord with regard to data fraud. As a lab, we hold ourselves to very high scientific and ethical standards, which are the primary reason that Adam was reported for investigation, as soon as evidence of data fraud was discovered. We expect that the scientific community will hold our work up to the same level of scrutiny that is given to any theoretical or experimental claim. However, we hope and trust that our colleagues will not engage in “guilt-by-association” thinking, nor unfairly pre-judge the rest of the work that has come, or will come, out of this lab.

Why wasn’t the data fraud contained in the published papers detected earlier? Isn’t it Dr. Braver’s responsibility as supervisor to prevent this from happening?
These are very difficult and complicated questions to answer. As past history has shown, and hopefully everyone involved with scientific research appreciates, if a person is determined, it is certainly possible to tamper with data and findings, while escaping easy detection by others involved. Moreover, there has to be a trust relationship between mentor and students, with senior students being given more responsibility and autonomy, so that they can learn how to function as independent investigators. This is critical for them to be able to move onto the next stage of their careers, in which independence is expected. Dr. Braver definitely supervised and checked on Adam’s work to the best of his ability, while balancing it with a degree of autonomy given to Adam in accordance with his senior status. Without exhaustively double-checking all aspects of Adam’s work (as was begun with his dissertation once suspicions arose), it would not have been possible to catch this data fraud any earlier. It is also important to emphasize that as part of Adam’s graduate training, both in formal courses, and in direct one-on-one discussions with Dr. Braver, explicit guidelines were provided regarding what would be considered academic integrity violations, data fraud, and other related infractions. Adam was well aware of the negative consequences of engaging in such behavior. Thus, there could not have been any ambiguity in his mind that what he was doing was unethical, unacceptable, and never condoned by Dr. Braver or the academic community. He bears the full responsibility for his actions. Nevertheless, it should go without saying that Dr. Braver and the CCP lab are extremely shocked, upset, and disappointed by Adam’s actions and their repercussions for us. We are looking for ways that we can learn useful lessons from this unfortunate affair, and to do our best to ensure that nothing like this ever happens in the lab again.

Are there any changes you are going to make to lab policy as a result of this affair, or do you have any advice for other labs?
One way in which these events have influenced our thinking is of the need to standardize, document, and report all aspects of data analysis and processing, so as to increase their transparency and reproducibility by others. We are aware of the increasing attention given to strategies for conducting fully reproducible science (e.g., Donoho, 2010), and are giving serious consideration to how we might implement some of these strategies as a lab (e.g., IPython Notebook). We believe that this is an important development for the fields of Psychology and Neuroscience as a whole. Increased transparency and reproducibility in science will not ensure that data fraud is eliminated, but it may act as a deterrent to potential perpetrators. We encourage other labs to think about these issues as well, and welcome any suggestions with regard to approaches that may be particularly effective, or easy to implement as lab policy.


Links to Reports on the Case


http://retractionwatch.wordpress.com/2013/03/05/wash-u-psych-researcher-cited-in-ori-probe-faces-multiple-retractions/#more-12841

http://www.stltoday.com/lifestyles/health-med-fit/health/health-matters/washington-university-student-accused-of-faking-research/article_c366b458-7849-5613-81b6-478c228cb4e4.html

http://www.stltoday.com/lifestyles/health-med-fit/washington-u-student-s-mentor-talks-about-discredited-research/article_e2275d60-1ead-5906-851a-59c7a4daf6e5.html

http://retractionwatch.wordpress.com/2013/03/07/wash-u-psychologist-sheds-light-on-inquiry-against-former-psychology-grad-student

Retraction Note to: Local and global effects of motivation on cognitive control. Cogn Affect Behav Neurosci. 2013 Jun 22. [.pdf file]


 

[CCP Home Page]