Harshil Vejendla

This paper proposes Curriculum Chaos Forecasting (CCF), a training paradigm that organizes training data based on dynamical systems theory. CCF significantly enhances performance on unseen, real-world benchmarks by progressively introducing more chaotic dynamics.

This paper presents Drift-Adapter, a lightweight, learnable transformation layer designed to bridge embedding spaces between model versions. It enables continued use of existing ANN indexes, effectively deferring full re-computation and reducing recompute costs.

Introduces Wave-PDE Nets, a neural architecture that simulates the second-order wave equation. Each layer propagates its hidden state as a continuous field, offering an alternative to traditional attention mechanisms.

Introduces SliceMoE, an architecture that routes contiguous slices of embedding to Mixture-of-Experts (MoE) layers, aiming to address capacity bottlenecks and load-balancing issues in transformer scaling.

Addresses the efficiency issues of existing Bayesian methods for uncertainty estimation in LLMs by proposing a distillation approach that avoids multiple sampling iterations during inference.

Explores a memory-efficient approach for large language model inference using hybrid one-bit caches to manage key-value pairs, addressing memory-bound problems in long-context inference.

Presents LATTA, a test-time adaptation method designed to improve the robustness and stability of pretrained models against distribution shifts, overcoming limitations of existing methods like Tent.