Bigger Context Windows Don't Make Better AI
The arms race to 1M context windows assumes more context is better. Research from Chroma proved it makes things worse - and labs have been expanding context windows ever since.
The arms race to 1M context windows assumes more context is better. Research from Chroma proved it makes things worse - and labs have been expanding context windows ever since.