So many cool ideas are being developed by the community. Here are some of my favorites.

• Batch inference optimization: FlexGen, llama.cpp
• Faster decoder with techniques such as Medusa, LookaheadDecoding
• Model merging: mergekit
• Constrained sampling: outlines, guidance, SGLang
• Seemingly niche tools that solve one problem really well, such as einops and safetensors.