“you need to get more scientific with something that's going to be in production and that could be as simple as you know just running that prompt like 30 times um and then pasting it into Google doc right um and and then just reviewing like okay like how often does it do bad things”

Run the prompt 100 times in async, then review the failures

Running a prompt once and shipping it is the prompt-engineering equivalent of medieval bloodletting — a lucky hit mistaken for a working system. Before any prompt goes into production, run it 30-100 times asynchronously, dump outputs into a doc, and cluster the failures. Bulk review surfaces real failure modes instead of the one good output that tricked you.