Infinite-Parameter LLMs: Generating and Adapting Weights from Live Data

Researchers propose "Infinite-Parameter LLMs," a new architecture that generates model weights dynamically from live interaction data rather than storing fixed parameters, inspired by Mixture-of-Experts designs. Using a compact hypernetwork and Bayesian updates, the approach allows models to learn from user-supplied information during runtime while maintaining a constant memory footprint, effectively creating infinite adaptable parameters. This method promises advantages over traditional in-context learning by embedding knowledge into weights, freeing context windows, and enabling better generalization across conversation turns.