| |
Show HN: Automatic deduction of FlashAttention kernel from ~25 lines of SQL
A developer demonstrates that the FlashAttention kernel—a memory-efficient attention mechanism used in transformers—can be automatically derived from approximately 25 lines of SQL through the einfold query optimizer, which rewrites SQL plans into optimized array algorithms without requiring explicit attention-specific rules. The approach works across multiple SQL engines and can compute both the forward pass and gradients of causal attention by treating it as a softmax-weighted average of a join operation.
Read Full Article →
← More Tech news