Recurrent Depth in Transformers: Balancing Compute and Memory Efficiency
An analysis of recurrent depth and Sparse MoE as a way to trade memory efficiency for gradient stability in transformer architectures.
Research
Papers that actually matter
42 articles in this section.
An analysis of recurrent depth and Sparse MoE as a way to trade memory efficiency for gradient stability in transformer architectures.
A new study explores using multi-pass verification to recover accuracy lost in 2-bit and 3-bit quantized models, though critics argue it’s a workaround.
An OpenAI model has disproven a geometry conjecture, highlighting the shift from human intuition to high-speed automated counterexample searching in mathematics.
Two AI assistants are accelerating drug retargeting by filtering medical literature, though physical lab validation remains the primary bottleneck in drug discovery.
Exploring how agentic reasoning models move beyond static RAG to automate the discovery of niche, long-tail facts with high precision.
AI-generated research papers and fraudulent paper mills are creating a synthetic feedback loop that threatens the validity of academic prestige metrics.
The IntentGrasp benchmark reveals a critical void in LLM architecture: the inability to distinguish between literal instructions and actual human intent.
An analysis of Anthropic’s ‘Teaching Claude Why’ research and the trade-offs between constitutional reasoning and raw model performance.
Exploring how LLM agents can infer restricted data from partial evidence, creating a new vector for privilege escalation in enterprise RAG pipelines.
Google DeepMind’s AlphaEvolve uses Gemini to autonomously evolve code, raising questions about the trade-off between performance and maintainability.