Six hard-won lessons from testing AI systems daily

Jaskaran Singh spent five years building consumer apps before shifting to AI testing, where his job is to push models until they fail. Writing for Dev.to, he shares six lessons learned the hard way — not from textbooks, but from real moments of confusion and failure. The piece covers how AI breaks text into tokens (each with a cost against a model's budget), how context windows act like a size-limited sticky note that silently erases older information, and how the temperature setting controls how adventurous or predictable a model's word choices are. Singh illustrates each concept with a concrete personal example, including a monitoring tool he built for Canada's immigration website that started misbehaving once its context window overflowed.