Notes from the practice

A language model can now turn a clean questionnaire into a working survey in minutes. It will run, and it will look right on every screen anyone happens to click through. What it will not do is notice that a filter contradicts the screener, that the last quota cell can never fill, or that a piped brand name is wrong on one path in twelve. Those were always the expensive errors, and generated code produces them with complete confidence.

Faster drafting makes verification more important, not less. The question is no longer who can write the script, but who stands behind it. Modern tooling is used wherever it makes the work faster; every build is still specified, tested against a written plan and independently audited before a respondent sees it.

A soft launch shows you what a few dozen respondents did. It cannot show you the paths they did not take: the rare screener combination, the final quota cell, the respondent who backs up and changes an answer. Those paths are where errors live. Soft launch confirms an instrument that has already been verified; it should never be the first time anyone looks at the data.

A twelve-page questionnaire with a rotating concept test and nested quotas carries more risk than a forty-page attitudinal battery. Cost and risk follow logical complexity, not length. The questionnaire is reviewed first so that the quotation reflects what the study actually demands, and neither party is surprised later.

The person who wrote the logic is the person least able to see what it fails to do; they test what they intended, not what they built. So every build is audited by an external firm that did not write it. And because that firm is paid for every defect it finds, rather than a flat fee for signing off, its only incentive is to look harder. Every defect it finds is one your respondents never see.