PDF] Reproducibility via Crowdsourced Reverse Engineering: A

The significant success of reverse-engineering the important accomplishments of DeepMind's Alpha Zero exemplifies the leverage that can be achieved by a concerted effort to reproduce results. The reproducibility of scientific findings are an important hallmark of quality and integrity in research. The scientific method requires hypotheses to be subjected to the most crucial tests, and for the results to be consistent across independent trials. Therefore, a publication is expected to provide sufficient information for an objective evaluation of its methods and claims. This is particularly true for research supported by public funds, where transparency of findings are a form of return on public investment. Unfortunately, many publications fall short of this mark for various reasons, including unavoidable ones such as intellectual property protection and national security of the entity creating those findings. This is a particularly important and documented problem in medical research, and in machine learning. Fortunately for those seeking to overcome these difficulties, the internet makes it easier to share experiments, and allows for crowd-sourced reverse engineering. A case study of this capability in neural networks research is presented in this paper. The significant success of reverse-engineering the important accomplishments of DeepMind's Alpha Zero exemplifies the leverage that can be achieved by a concerted effort to reproduce results.

Toxicogenomics: A 2020 Vision: Trends in Pharmacological Sciences

Advances in systems biology modeling: 10 years of crowdsourcing DREAM challenges - ScienceDirect

Developing a framework for digital objects in the Big Data to Knowledge (BD2K) commons: Report from the Commons Framework Pilots workshop - ScienceDirect

Crowdsourcing in Cognitive and Systems Neuroscience - Brian P. Johnson, Eran Dayan, Nitzan Censor, Leonardo G. Cohen, 2022

The Missing Pieces of Artificial Intelligence in Medicine: Trends in Pharmacological Sciences

Can the crowd judge truthfulness? A longitudinal study on recent misinformation about COVID-19

Growing a Workflow Language with GNU Guix - TIB AV-Portal

PDF) The Reproducibility of Statistical Results in Psychological Research: An Investigation Using Unpublished Raw Data

Challenges: Crowdsourced solutions

Combinatorial optimization of mRNA structure, stability, and translation for RNA-based therapeutics

PDF) European Handbook of Crowdsourced Geographic Information

The future(s) of open science - Philip Mirowski, 2018

PDF) Crowdsourced Reverse Engineering: Experiences in Applying Crowdsourcing to Concept Assignment

Crowdsourced RNA design discovers diverse, reversible, efficient, self-contained molecular switches

Faceting the post-disaster built heritage reconstruction process within the digital twin framework for Notre-Dame de Paris