Presentation
How Gradient Extended Llama 3’s Context Length to 1M on Crusoe
DescriptionGradient AI fine-tuned Meta’s Llama3 models on long-context data to develop derivatives capable of processing up to 1 million tokens worth of text (again, ~700k words). Powered by Crusoe’s climate-aligned cloud, Gradient was able to release these to the developer community only a few weeks after Meta’s initial launch of Llama3. These models are available on HuggingFace, the most popular model repository, and to date have been downloaded over 100k times.
By open-sourcing these models, Crusoe and Gradient enabled developers to continue to innovate on long-context models. With over 100k users, the community can identify weaknesses, highlight strong suits, and create new, unique user experiences. Such releases also challenge closed-source companies to further invest in their offerings and spur democratization of key technologies.
By open-sourcing these models, Crusoe and Gradient enabled developers to continue to innovate on long-context models. With over 100k users, the community can identify weaknesses, highlight strong suits, and create new, unique user experiences. Such releases also challenge closed-source companies to further invest in their offerings and spur democratization of key technologies.
Presenters
Event Type
Industry Session
TimeWednesday, 31 July 20245:00pm - 5:30pm MDT
LocationRoom 607
Session TimeWednesday, 31 July 20249:00am - 5:30pm MDT
LocationRoom 607
New Technologies
Generative AI Day
Exhibitor
Full Conference
Full Conference Supporter
Experience
Exhibits Only
Exhibitor Full Conference
Exhibitor Experience
Wednesday
