New LLM optimization technique slashes memory costs up to 75%

Universal Transformer Memory uses neural networks to determine which tokens in the LLM’s context window are useful or redundant.