4a. Exhaustive algorithms: Data dredging, Data fishing, Data snooping, p-hacking

The Data Scientist is responsible for separating correlations that are the results of chance or deliberate data-mining driven searches vs. well established hypothesis-driven correlated information. Where exhaustive methods have been used to locate anomalies etc these results should be clearly declared as such, and not represented as a consequence of specific hypothesis-driven analyses, without further statistical tests.

Leave a Reply

Your email address will not be published. Required fields are marked *