Is Grep All You Need? How Agent Harnesses Reshape Agentic Search

Researchers conducted an empirical study comparing grep-based and vector retrieval methods in LLM agent systems, finding that grep generally outperforms vector retrieval for agentic search tasks on the LongMemEval dataset. The study also revealed that performance depends significantly on the agent architecture and tool-calling paradigm used, independent of the underlying retrieval strategy, and that grep retrieval remains more robust when searches must navigate through irrelevant contextual information.