Mitchell Hermon

Available for research internships in 2027, on site or remote. PhD expected 2028.

Security-Fidelity Tradeoffs: The Hidden Cost of Prompt Injection Defense ICML 2026 Spotlight

First author, with collaborators at Amazon. I designed SecFid, a benchmark that tells a model processing injected text as data apart from one silently dropping it, and released its data and evaluation code.

Unroll is a free VS Code extension for reading agent traces and eval runs. It shows each run as a conversation and links every step to the JSON record it came from.

SecFid has the benchmark data and evaluation code from Security-Fidelity Tradeoffs (ICML 2026).

Python, PyTorch, Hugging Face Transformers, and vLLM for research, and TypeScript and React for tools like Unroll. I build evaluation harnesses that run across open-weight and API models, and I have trained models with DPO.

Software engineering intern at Amazon in 2022. I built a Java workflow on AWS Lambda, DynamoDB, and SNS for a labor-planning tool, with estimated annual savings of $10 million.

Sloan Scholar, SURGE Fellow, and twice on the list of Teachers Ranked as Excellent by Their Students.

I identify real problems that undermine trust in AI systems, build evaluations to find when and why they occur, and use those findings to design targeted mitigations. Most of my current work is on secure and reliable agents.

SecFid separates a model handling an injected instruction as data from a model silently dropping it. Standard prompt-injection benchmarks count both as secure.

That’s Deprecated! studies what code models do when a prompt contradicts what they learned, and whether that conflict is visible in their activations. I led the dataset construction and evaluation.

Fidelity-aware preference training uses SecFid’s scoring as a training signal, teaching a defended model to keep injected text as data instead of dropping it.

Activation steering shifts whether a code model follows the prompt or its prior knowledge. I led this part of the That’s Deprecated! project.

Unroll is a free VS Code extension for reading agent traces and eval runs. It shows each run as a conversation and links every step to the JSON record it came from.

SecFid has the benchmark data and evaluation code from Security-Fidelity Tradeoffs (ICML 2026).

I’m open to collaborating on LLM evaluation and agent safety.

I identify real problems that undermine trust in AI systems, build evaluations to find when and why they occur, and use those findings to design targeted mitigations.

For my first-author ICML 2026 Spotlight paper, I built SecFid, which showed that prompt-injection defenses can look secure while dropping content the user needed. In an initial experiment, training on SecFid’s signal recovered most of that content. In That’s Deprecated!, I led the dataset, evaluation, and mitigation for code models that fall back on outdated APIs they learned in training.

Most of my current work is on secure and reliable agents. I also maintain Unroll, an open-source VS Code extension for reading agent traces.

Teaching assistant for Data Science Discovery (five semesters, including this fall) and Data Visualization (two summers). President of the ACM chapter at the University of Iowa.

I love coffee, and there’s usually a decaf pour-over in my hand, unless a paper deadline calls for the real thing. I run, sometimes with my corgi, Agnes, though on those “runs” she mostly pulls and I follow. I also boulder, and I enjoy DIY projects on my house, which is over 100 years old.

I studied computer science, statistics, and mathematics at the University of Iowa, and was president of the ACM chapter there.