inflection point — The generative AI revolution has begunhow did we get here? A new class of incredibly powerful AI models has made recent breakthroughs possible.
HaomiaoHuang – Jan 30, 2023 12:00 pm UTC Enlarge / This image was partially AI-generated with the prompt “a pair of robot hands holding pencils drawing a pair of human hands, oil painting, colorful,” inspired by the classic M.C. Escher drawing. Watching AI mangle drawing hands helps us feel superior to the machines… for now. AurichAurich Lawson | Stable Diffusion reader comments 38 with 0 posters participating Share this story Share on Facebook Share on Twitter Share on Reddit
Progress in AI systems often feels cyclical. Every few years, computers can suddenly do something theyve never been able to do before. Behold! the AI true believers proclaim, the age of artificial general intelligence is at hand! Nonsense! the skeptics say. Remember self-driving cars?
The truth usually lies somewhere in between.
Were in another cycle, this time with generative AI. Media headlines are dominated by news about AI art, but theres also unprecedented progress in many widely disparate fields. Everything from videos to biology, programming, writing, translation, and more is seeing AI progress at the same incredible pace. Why is all this happening now?
Further ReadingThe basics of modern AIhow does it work and will it destroy society this year?You may be familiar with the latest happenings in the world of AI. Youve seen the prize-winning artwork, heard the interviews between dead people, and read about the protein-folding breakthroughs. But these new AI systems arent just producing cool demos in research labs. Theyre quickly being turned into practical tools and real commercial products that anyone can use.
Theres a reason all of this has come at once. The breakthroughs are all underpinned by a new class of AI models that are more flexible and powerful than anything that has come before. Because they were first used for language tasks like answering questions and writing essays, theyre often known as large language models (LLMs). OpenAIs GPT3, Googles BERT, and so on are all LLMs. Advertisement
But these models are extremely flexible and adaptable. The same mathematical structures have been so useful in computer vision, biology, and more that some researchers have taken to calling them “foundation models” to better articulate their role in modern AI.
Where did these foundation models came from, and how have they broken out beyond language to drive so much of what we see in AI today? The foundation of foundation models
Theres a holy trinity in machine learning: models, data, and compute. Models are algorithms that take inputs and produce outputs. Data refers to the examples the algorithms are trained on. To learn something, there must be enough data with enough richness that the algorithms can produce useful output. Models must be flexible enough to capture the complexity in the data. And finally, there has to be enough computing power to run the algorithms.
The first modern AI revolution took place with deep learning in 2012, when solving computer vision problems with convolutional neural networks (CNNs) took off. CNNs are similar in structure to the brain’s visual cortex. Theyve been around since the 1990s but werent yet practical due to their intense computing power requirements.
In 2006, though, Nvidia released CUDA, a programming language that allowed for the use of GPUs as general-purpose supercomputers. In 2009, Stanford AI researchers introduced Imagenet, a collection of labeled images used to train computer vision algorithms. In 2012, AlexNet combined CNNs trained on GPUs with Imagenet data to create the best visual classifier the world had ever seen. Deep learning and AI exploded from there.
CNNs, the ImageNet data set, and GPUs were a magic combination that unlocked tremendous progress in computer vision. 2012 set off a boom of excitement around deep learning and spawned whole industries, like those involved in autonomous driving. But we quickly learned there were limits to that generation of deep learning. CNNs were great for vision, but other areas didnt have their model breakthrough. One huge gap was in natural language processing (NLP)i.e., getting computers to understand and work with normal human language rather than code. Advertisement
The problem of understanding and working with language is fundamentally different from that of working with images. Processing language requires working with sequences of words, where order matters. A cat is a cat no matter where it is in an image, but theres a big difference between this reader is learning about AI and AI is learning about this reader.
Until recently, researchers relied on models like recurrent neural networks (RNNs) and long short-term memory (LSTM) to process and analyze data in time. These models were effective at recognizing short sequences, like spoken words from short phrases, but they struggled to handle longer sentences and paragraphs. The memory of these models was just not sophisticated enough to capture the complexity and richness of ideas and concepts that arise when sentences are combined into paragraphs and essays. They were great for simple Siri- and Alexa-style voice assistants but not for much else.
Getting the right training data was another challenge. ImageNet was a collection of one hundred thousand labeled images that required significant human effort to generate, mostly by grad students and Amazon Mechanical Turk workers. And ImageNet was actually inspired by and modeled on an older project called WordNet, which tried to create a labeled data set for English vocabulary. While there is no shortage of text on the Internet, creating a meaningful data set to teach a computer to work with human language beyond individual words is incredibly time-consuming. And the labels you create for one application on the same data might not apply to another task. Page: 1 2 3 4 5 6 7 8 Next → reader comments 38 with 0 posters participating Share this story Share on Facebook Share on Twitter Share on Reddit Advertisement Channel Ars Technica ← Previous story Next story → Related Stories Today on Ars
The growing popularity and reach of the Premier League globally is leaving rival European football competitions struggling to compete.
Not only to find an audience, but to find outlets to even show the matches.
So German football had to think differently – going to where Gen Z is engaging with football through content creators.
And that’s why tonight, Harry Kane’s Bayern Munich will begin their defence of the Bundesliga title live to 1.4 million subscribers on the That’s Football channel on YouTube.
Image: Harry Kane in Bundesliga action last season. Pic: Reuters
It’s run by Mark Goldbridge, known for passionate but often provocative, punchy commentary about players on streams going viral.
His brand was built by being filmed reacting to watching Manchester United matches.
“People need to appreciate that we have a certain content style, and that’s very, very popular,” Goldbridge told Sky News.
“That is an area that needs to be catered [to] and that’s why, without the rights, we’ve had such big, big audiences.”
Goldbridge revealed he isn’t paying to show his 20 Friday night matches this season – reinforcing how the Bundesliga struggled to find a buyer in Britain.
Sky Sports previously had a four-year rights deal to exclusively show those German matches here, but will now only show the prestige Saturday evening slot live.
Image: Bundesliga teams Eintracht Frankfurt and RB Leipzig during their match in April. Pic: Reuters
European leagues are finding it increasingly difficult in this market to sell their rights because domestic football is so dominant and appealing.
The focus of football budgets is on domestic games for Sky as well as Discovery-owned TNT Sports, which also focuses its European football coverage on men’s continental competitions, including the Champions League.
More Premier League matches will be shown live than ever before – with at least 215 on Sky, the parent company of Sky News, and others on TNT.
Sky Sports also has live men’s rights to the English Football League and Scottish matches, as well as sharing the Women’s Super League with the BBC.
The Bundesliga is also making the games broadcast by Goldbridge’s channel available to the BBC to stream online. They will further be on The Overlap, a YouTube channel part-owned by Gary Neville.
Image: Behind the scenes of covering a Premier League game
‘A progressive step’
Bundesliga International CEO Peer Naubert said: “Our approach is as diverse as our supporters: by combining established broadcasters with digital platforms and content creators, we are taking a progressive step in how top-level football can be experienced.
“This multi-layered strategy allows us to connect with more audiences across the UK and Ireland, giving every supporter the chance to engage … in the way that suits them best.”
While the former England and Manchester United player is a star pundit on Sky, he could also be seen as a rival to the Comcast-owned broadcaster by attracting fans to newer outlets of his channel.
Goldbridge doesn’t see himself as a rival yet to long-established broadcasters.
“We’re not looking to replace what you can find on Sky or the BBC or anything like that,” he said. “This is a community that will be live with us, watching the Bundesliga, learning about it.
“And if I get a pronunciation wrong, or I don’t know about a player, then I’ve got my community there to back me up. I don’t profess to know everything.”
Image: Kane celebrates the Bundesliga title with his Bayern Munich teammates. Pic: Reuters
‘This is the future’
But he can be relatable to audiences, with more than two million subscribing to his The United Stand channel, earning him millions of pounds over the last decade.
“We’ve been there growing in the background and I think certain media outlets have ignored that, maybe hoping it would go away,” he said.
“I certainly think synergy and collaboration need to happen more because there are things in the mainstream that I don’t like and there will be people out there that really don’t like the way we watch football, but a lot of people do.
“And it’s about offering that choice to people and there are different ways people listen to football on the radio, people watch it with a commentator, some people turn the audio off completely, some people watch things like this (watch-a-long).
“And I think that is the future, to offer more choice.”
The Ministry of Defence (MoD) has announced it will buy £118m worth of air defence missile systems for the British Army.
But will this new purchase protect an increasingly vulnerable UK from attack, and why now?
For more than 50 years, the British Army relied on the Rapier air defence missile system to protect deployed forces.
In 2021, that system was replaced by Sky Sabre.
Image: Soldiers demonstrating the Sky Sabre air defence missile system. Pic: MoD
The new system is mobile, ground-based, and designed to counter various aerial threats, including fighter aircraft, attack helicopters, drones, and guided munitions.
It’s known for its speed, accuracy, and ability to integrate with other military assets, including those of the Royal Navy and Royal Air Force (and NATO).
What is the Land Ceptor missile, and why do we need more of them?
Sky Sabre includes radar, command, and control capability and – most importantly – a missile to intercept incoming threats.
The Land Ceptor missile weighs around 100kg, has a 10kg warhead, and can intercept threats out to around 15 miles (25km), making it around three times more effective than the Rapier system it replaced.
Image: The Land Ceptor missile during test-firing in Sweden in 2018. File pic: MoD
When the MoD made the decision to replace the Rapier system, the global threat environment was very different to that experienced today.
Since the end of the Cold War, the UK has been involved in expeditionary warfare – wars of choice – and generally against less capable adversaries.
So, although the Land Ceptor missile is very capable, defence planning assumptions (DPAs) were that they would not need to be used in a serious way, commensurate with the threat.
However, as the Russian invasion of Ukraine has demonstrated (as has the series of Iranian attacks on Israel), significantly larger stockpiles are required against a more capable enemy.
Image: Sky Sabre has a surveillance radar. Pic: MoD
Is the UK vulnerable to missile attack?
In short, yes. Although the Land Ceptor missile does provide an excellent point-defence capability, it is not an effective counter to ballistic or hypersonic missiles – the Sea Viper mounted on Royal Navy Type 45 Destroyers using the Aster 30 missile has that capability.
In the Cold War, the UK had Bloodhound missiles deployed around the UK to provide a missile defence capability, but as the perceived risks to the UK abated following the collapse of the Soviet Union, UK missile defence fell down the priorities for the MoD.
Although the radar based at RAF Fylingdales forms part of the Ballistic Missile Early Warning System (BMEWS), and can detect incoming threats, the UK no longer has an effective interceptor to protect critical national infrastructure.
Instead, the UK relies on the layered defences of European allies to act as a deterrence against attack.
In the near term, this timely order for Land Ceptor missiles doubles the British Army’s tactical capability.
However, as the conflicts in Ukraine and the Middle East have demonstrated, ballistic (and increasingly hypersonic) missiles are being produced in increasing quantity – and quality.
Without significant (and rapid) investment, this critical gap in national military capability leaves the UK vulnerable to attack.
A newly-discovered dinosaur with an “eye-catching sail” along its back and tail is to be named after record-breaking yachtswoman Dame Ellen MacArthur.
Istiorachis macaruthurae was identified and named by Jeremy Lockwood, a PhD student at the University of Portsmouth and the Natural History Museum.
Istiorachis means “sail spine” and macaruthurae is taken from the surname of Dame Ellen, who became famous for setting a record for the fastest solo non-stop round-the-world voyage in 2005.
Dame Ellen is from the Isle of Wight, where the creature’s fossils were found.
Image: Jeremy Lockwood with the spinal column of the dinosaur. Pic: University of Portsmouth/PA
Image: Lockwood said the creature had particularly long neural spines. Pic: University of Portsmouth/PA
Before Dr Lockwood analysed them, the fossils, which date back 125 million years, were thought to be from one of the two known iguanodontian dinosaur species from the island.
“But this one had particularly long neural spines, which was very unusual,” he said.
Writing in the scientific journal Papers in Palaeontology, Dr Lockwood said his study showed the dino would have probably had a pronounced sail-like structure along its back.
The exact purpose of such features “has long been debated, with theories ranging from body heat regulation to fat storage”.
In this case, researchers think it was most likely to be for “visual signalling, possibly as part of a sexual display”.
Image: Yachtswoman Dame Ellen MacArthur in 2014. File pic: PA
For the study, the researchers compared the fossilised bones with a database of similar dinosaur backbones which allowed them to see how these sail-like formations had evolved.
Dr Lockwood said his team showed Istiorachis’s spines “weren’t just tall, they were more exaggerated than is usual in Iguanodon-like dinosaurs, which is exactly the kind of trait you’d expect to evolve through sexual selection”.
Professor Susannah Maidment, of the Natural History Museum, said: “Jeremy’s careful study of fossils that have been in museum collections for several years has brought to life the iguandontian dinosaurs of the Isle of Wight.
“His work highlights the importance of collections like those at [Isle Of Wight museum] Dinosaur Isle, where fossil specimens are preserved in perpetuity and can be studied and revised in the light of new data and new ideas about evolution.
“Over the past five years, Jeremy has single-handedly quadrupled the known diversity of the smaller iguanodontians on the Isle of Wight, and Istiorachis demonstrates we still have much to learn about Early Cretaceous ecosystems in the UK.”