Strands Decider: Why Not an Encoder?

Marc Brooker, an engineer at AWS, compares different model architectures for the strands-decider task to determine whether encoder-based models outperform causal language models. While encoder models like T5Gemma achieved lower latency (~35ms vs 65ms), they showed comparable or slightly worse accuracy and calibration than the causal decoder-based v19 model despite conventional wisdom suggesting bidirectional encoders should perform better for classification tasks. The findings suggest that for this particular use case, the performance trade-offs don't strongly favor encoders over the modified LLM approach.