Open Citizen ScienceOpen Mind

How we test

The same rules as our Indus script project: a good-looking result must not be able to come from trying until something works.

Five rules

  1. Seal it before you look. Each measure, its preprocessing, its threshold and its pass mark are written down and SHA-256 hashed before we see any outcome.
  2. Test on people it never saw. Data is split by subject, and data from the same lab counts as a near-copy when we test transfer.
  3. Plant an answer first. A pipeline must recover an injected effect of known size, and must not call spectrum-matched noise 'conscious', before its output on real data counts.
  4. Let someone else try to break it. Every positive result goes to a red team that did not produce it.
  5. Publish it either way. Failed routes are written up with the size of effect the test could have caught.

Traps we watch for