Ring Attention? This morning I read a fascinating article that is set to completely change how we interact with the likes of ChatGPT.
Soon, you’ll be able to put millions of words into context windows of AI models, researchers say.
Let’s start with a little background…
As of today – the middle of October in the year 2023 (that already feels like by the time someone reads this, there will be so many more advancements, it will feel like I am explaining how a typewriter works and isn’t it amazing!) – anyway, as of today, the likes of ChatGPT can take in and process at most a few thousand words at a time.
Imagine if rather than really long and complex prompts, one could instead feed ChatGPT the entire book of “The Great Gatsby” by Fitzgerald, AND the entire book of “1984” by Orwell, asked ChatGPT to combine the two and spit out an entirely new novel about greed, love, and ‘The American Dream’?
A Google researcher and a UC Berkeley professor have come up with the process to allow AI to do just that!
What is Ring Attention?
It is called, “Ring Attention”, and is an innovative technique that allows AI models to process and analyse significantly more data simultaneously than was previously thought possible.
Problem Solved by Ring Attention:
Traditional AI models, especially those used in natural language processing, face memory constraints. They were limited in the number of words or data points they could analyze at once. Ring Attention essentially removes those limits.
Changing AI & ChatGPT Input Capacities:
With “Ring Attention”, AI models like ChatGPT can potentially process and generate much longer content in a single instance.
Imagine…
Imagine being able to input articles and data from hundreds of different studies on a particular topic into ChatGPT and ask it to analyze the information from a particular perspective…
Now, imagine your life only 321 days ago… (less than a year)
Why 321 days? Because that’s the day before ChatGPT was released to the general public.
~ Bella
Ms Bella St John – Founder of Leaders XL