Photo credit: www.technologyreview.com
“No longer do I have to drive a symbol of racism, greed and ignorance! Life is suddenly so much better!”
—Actor Bette Midler shares her relief after selling her Tesla, as reported by Insider.
The big story
Large language models are capable of impressive feats, but the reasons behind their abilities remain unclear.
In recent research, Yuri Burda and Harri Edwards from OpenAI sought to determine how to enable a large language model to perform basic arithmetic tasks. Their initial efforts were met with obstacles, as the models could only recall sums they had encountered rather than applying their knowledge to new problems.
During their experiments, Burda and Edwards unintentionally allowed some runs to continue for extended periods. This repetitive exposure to example sums eventually enabled the models to learn how to add two numbers, albeit at a much slower pace than anticipated.
Interestingly, the researchers noted instances where the models would struggle with a task and suddenly exhibit a clear understanding, an occurrence they described as “grokking.” This phenomenon, along with others, poses significant questions within the AI research community. The behavior of larger models, particularly those in the realm of large language models, often contradicts theoretical expectations derived from traditional mathematics.
This situation underscores a remarkable aspect of deep learning—the fundamental driver behind the ongoing AI revolution: despite its apparent success, the precise mechanics of how and why it operates remain elusive. For a deeper exploration of this topic, read the full story.
—Will Douglas Heaven
We can still have nice things
A refuge for comfort, enjoyment, and a bit of fun to lighten your day. Have some suggestions? Reach out or send your ideas my way.
+ Discover the impact of Wham! as they introduced Western pop music to China four decades ago.
+ It turns out that sharks can make sounds after all!
+ Microsoft is ringing in its 50th anniversary, celebrating 50 fascinating and unusual inventions along the way.
+ Curious about what a prototaxite is? You’re not alone!
Source
www.technologyreview.com