Google Engineer Says AI Search Models Still Act Like Black Boxes

Google’s newest search layers are giving professionals a harder problem than simple ranking shifts: the system is becoming more powerful even as it becomes less legible. Nikola Todorovic, a Google engineer, said AI models can behave like a black box, where even the people building them do not always see exactly what is happening underneath. That is the central operational shift for search teams. The old habit of treating Search like a fixed set of deterministic levers is breaking down, because AI now sits on top of retrieval, ranking, and quality systems that still matter, but no longer tell the whole story.

Google’s own history shows how gradual that change has been. The company said its first machine learning system in Search was built to catch spelling problems, then expanded into understanding synonyms and query-document context. Todorovic pointed to SafeSearch as an early proving ground because it could be isolated from the main ranking flow, making it easier to test machine learning without destabilizing Search. Google’s documentation says SafeSearch relies on automated systems that use machine learning and signals from text, images, videos, and links, and that safety systems still work even when SafeSearch is off. In 2022, Google said those safety algorithms improve hundreds of millions of searches globally every day.

That layered approach now defines AI Overviews. Google says the feature uses a customized Gemini model working alongside existing quality and ranking systems and the Google Knowledge Graph. It is designed to surface information backed by top web results and links to supporting pages, especially for more complex questions that may have previously taken multiple searches. Google also says AI Overviews are a core Search feature that cannot be turned off, though users can switch to the Web filter for text-only links. The company warns that AI Overviews can and will make mistakes, a reminder that visibility now depends on more than one output layer.

The reach is already enormous. By May 2025, Google said AI Overviews had more than 1.5 billion monthly users across 200 countries and territories, and that in its biggest markets, including the United States and India, the feature drove more than a 10% increase in usage for queries where it appeared. Google also said people who use AI Overviews tend to search more and report higher satisfaction. For search professionals, the lesson is blunt: stop expecting a clean cause-and-effect map from a single query to a single ranking position. AI Mode makes that even more fluid, using query fan-out to break a question into subtopics and issue multiple queries at once. In that environment, structured data, source clarity, and strong signals across the web matter more, not less, because the model may be opaque even when the content it lifts from is not.