“the first couple times we tried it it was like amazing the problem is over time there's just enough like hallucinations and things that were wrong that I ended up not being able to trust it so I basically had to go and recheck the work every single time which more or less reduced the value of it back to zero”

LLM workflows need a reliability filter before they create leverage

Most AI-augmented internal tools feel magical for the first few runs, then hallucinate enough that you re-check every output — eroding all the leverage. Real value comes from the small subset of tasks that survive a reliability filter and graduate into trusted automation. Run new LLM workflows in shadow mode for weeks; promote only the ones where you stop double-checking.