Blind sampling (revision 3)

Blind sampling is the selection of material for testing under a procedure controlled by neither the party being tested nor the party commissioning the test. In a regulated setting it is what allows a result on a few units to support a statement about a whole lot: the units were drawn according to a written sampling plan, their identity was recorded, and the laboratory did not know whose material it was.^[1][2] The sampling plan is part of the quality system rather than of the analysis, which is why an accredited laboratory can produce an impeccable result on an unrepresentative sample.

Almost no independent testing of research peptides is blind in this sense. The ordinary sequence is that a purchaser buys material, chooses a vial, and sends it to a testing service. Every step of that is under the purchaser's control, and the supplier knows in advance that some proportion of its output will be tested but not which.^[3]

Sampling is the step that connects a measurement to a population. A result is only as good as the answer to the question "of what is this a sample?", and three separate failures can break that connection.

1. Selection by the tested party. If the supplier chooses which vial is tested, the result describes the best vial the supplier could find. No amount of analytical rigour downstream repairs this.
2. Selection by the commissioning party. If the purchaser chooses, the result describes the vial the purchaser chose — which in practice means the one from the order that looked wrong, or the one from the line they care about. Both are informative about that vial and neither is representative.
3. Selection by outcome. If results are submitted to a public tally only when the submitter finds them interesting, the tally is shaped by what motivated submission rather than by what was found. This is selection on the outcome and it is the mechanism that makes community aggregates weak evidence.^[3]