Connect with us

Published

on

inflection point — The generative AI revolution has begunhow did we get here? A new class of incredibly powerful AI models has made recent breakthroughs possible.

HaomiaoHuang – Jan 30, 2023 12:00 pm UTC Enlarge / This image was partially AI-generated with the prompt “a pair of robot hands holding pencils drawing a pair of human hands, oil painting, colorful,” inspired by the classic M.C. Escher drawing. Watching AI mangle drawing hands helps us feel superior to the machines… for now. AurichAurich Lawson | Stable Diffusion reader comments 38 with 0 posters participating Share this story Share on Facebook Share on Twitter Share on Reddit

Progress in AI systems often feels cyclical. Every few years, computers can suddenly do something theyve never been able to do before. Behold! the AI true believers proclaim, the age of artificial general intelligence is at hand! Nonsense! the skeptics say. Remember self-driving cars?

The truth usually lies somewhere in between.

Were in another cycle, this time with generative AI. Media headlines are dominated by news about AI art, but theres also unprecedented progress in many widely disparate fields. Everything from videos to biology, programming, writing, translation, and more is seeing AI progress at the same incredible pace. Why is all this happening now?

Further ReadingThe basics of modern AIhow does it work and will it destroy society this year?You may be familiar with the latest happenings in the world of AI. Youve seen the prize-winning artwork, heard the interviews between dead people, and read about the protein-folding breakthroughs. But these new AI systems arent just producing cool demos in research labs. Theyre quickly being turned into practical tools and real commercial products that anyone can use.

Theres a reason all of this has come at once. The breakthroughs are all underpinned by a new class of AI models that are more flexible and powerful than anything that has come before. Because they were first used for language tasks like answering questions and writing essays, theyre often known as large language models (LLMs). OpenAIs GPT3, Googles BERT, and so on are all LLMs. Advertisement

But these models are extremely flexible and adaptable. The same mathematical structures have been so useful in computer vision, biology, and more that some researchers have taken to calling them “foundation models” to better articulate their role in modern AI.

Where did these foundation models came from, and how have they broken out beyond language to drive so much of what we see in AI today? The foundation of foundation models

Theres a holy trinity in machine learning: models, data, and compute. Models are algorithms that take inputs and produce outputs. Data refers to the examples the algorithms are trained on. To learn something, there must be enough data with enough richness that the algorithms can produce useful output. Models must be flexible enough to capture the complexity in the data. And finally, there has to be enough computing power to run the algorithms.

The first modern AI revolution took place with deep learning in 2012, when solving computer vision problems with convolutional neural networks (CNNs) took off. CNNs are similar in structure to the brain’s visual cortex. Theyve been around since the 1990s but werent yet practical due to their intense computing power requirements.

In 2006, though, Nvidia released CUDA, a programming language that allowed for the use of GPUs as general-purpose supercomputers. In 2009, Stanford AI researchers introduced Imagenet, a collection of labeled images used to train computer vision algorithms. In 2012, AlexNet combined CNNs trained on GPUs with Imagenet data to create the best visual classifier the world had ever seen. Deep learning and AI exploded from there.

CNNs, the ImageNet data set, and GPUs were a magic combination that unlocked tremendous progress in computer vision. 2012 set off a boom of excitement around deep learning and spawned whole industries, like those involved in autonomous driving. But we quickly learned there were limits to that generation of deep learning. CNNs were great for vision, but other areas didnt have their model breakthrough. One huge gap was in natural language processing (NLP)i.e., getting computers to understand and work with normal human language rather than code. Advertisement

The problem of understanding and working with language is fundamentally different from that of working with images. Processing language requires working with sequences of words, where order matters. A cat is a cat no matter where it is in an image, but theres a big difference between this reader is learning about AI and AI is learning about this reader.

Until recently, researchers relied on models like recurrent neural networks (RNNs) and long short-term memory (LSTM) to process and analyze data in time. These models were effective at recognizing short sequences, like spoken words from short phrases, but they struggled to handle longer sentences and paragraphs. The memory of these models was just not sophisticated enough to capture the complexity and richness of ideas and concepts that arise when sentences are combined into paragraphs and essays. They were great for simple Siri- and Alexa-style voice assistants but not for much else.

Getting the right training data was another challenge. ImageNet was a collection of one hundred thousand labeled images that required significant human effort to generate, mostly by grad students and Amazon Mechanical Turk workers. And ImageNet was actually inspired by and modeled on an older project called WordNet, which tried to create a labeled data set for English vocabulary. While there is no shortage of text on the Internet, creating a meaningful data set to teach a computer to work with human language beyond individual words is incredibly time-consuming. And the labels you create for one application on the same data might not apply to another task. Page: 1 2 3 4 5 6 7 8 Next → reader comments 38 with 0 posters participating Share this story Share on Facebook Share on Twitter Share on Reddit Advertisement Channel Ars Technica ← Previous story Next story → Related Stories Today on Ars

Continue Reading

Business

Rachel Reeves facing ‘pressure’, but ‘people should give her time’, says Wes Streeting

Published

on

By

Rachel Reeves facing 'pressure', but 'people should give her time', says Wes Streeting

The health secretary has said that the cabinet is aware of the “pressure” on Chancellor Rachel Reeves amid volatile markets and a challenging broader economic picture – but appealed for the public to “give her time”.

Wes Streeting argued that the public “underestimates” the “amount of heavy-lifting” Ms Reeves has had to do and will have to continue to do, as he declared “total confidence” in her leadership in a staunch defence of her handling of the economy.

Separately, international development minister Anneliese Dodds, who attends cabinet, told Sky News that Ms Reeves has been “very clear about the long-term plan for our country” and she herself is “confident in that long-term plan”.

The comments from the two key ministers come after the past week saw a drop in the pound and an increase in government borrowing costs, which has fuelled speculation of more spending cuts or tax rises.

Streeting has ‘total confidence in chancellor’s leadership’

Speaking at the Jewish Labour Movement’s annual conference in north London, the health secretary acknowledged the fierce competition among all government departments for any available public funding from the Treasury, and told party members that all ministers “have to make choices and trade-offs” in where funding goes.

Mr Streeting went on to say that the chancellor and her deputy, Darren Jones, have “the hardest job of all because they have to make those choices across every bit of government spending, and they have to think about what’s in the interests of our overall economy and how we get businesses growing”.

Please use Chrome browser for a more accessible video player

Chancellor’s ‘pragmatic’ approach to China

He said: “I think people continue to underestimate both the amount of heavy lifting she has had to do in her first six months, and the amount of heavy lifting she will have to do in her next six months.

“And the cabinet doesn’t underestimate that – we understand the choices she has to make, the pressure she is under.”

As a result, cabinet ministers all “have a responsibility” to both “make tough choices and drive reform and value for money” within their departments, and also be “drivers of economic growth”.

“Nothing in the last six months has shaken my conviction that economic growth is the number one priority,” he said.

Follow Sky News on WhatsApp
Follow Sky News on WhatsApp

Keep up with all the latest news from the UK and around the world by following Sky News

Tap here

Continuing his defence of the chancellor’s handling of the economy so far, Mr Streeting said she is “trying to break us out of what has been the status quo and the economic orthodoxy of more than a decade”.

“People need to give her time, and they need to not forget that, without [Sir Keir Starmer’s] leadership, certainly we wouldn’t have won the last general election.

“Without Rachel’s leadership, we wouldn’t have won the last general election either. She built Labour’s economic credibility out of the ashes they were left in after the Corbyn leadership. And she has built that trust, built up that plan, and now she’s following through.”

He declared that he has “total confidence in the leadership that Rachel’s providing, and the leadership that the cabinet is following and driving with her, because all of us have to deliver economic growth for our country”.

Minister ‘confident in chancellor’s long-term plan’

Speaking in a separate session at the conference, Ms Dodds noted “speculation” about the fiscal headroom (the amount of money the chancellor will have available to spend), but said: “We have to focus on actually the evidence.

“And when we look at the evidence, we can see that the UK government has a chancellor who is very clear about the long-term plan for our country. She’s been delivering on it.”

Read more:
Even if Siddiq resigns, the damage may already be done to Starmer
Grim economic news raises stakes for embattled chancellor

Ms Dodds, who also attends cabinet, pointed to a “new fiscal system”, the chancellor’s new Industrial Strategy Council, as well as “record levels of investment under Rachel Reeves’s leadership”.

“I think it’s really important for us to focus on those fundamentals, on what has been achieved in a very short space of time. And I’m confident in that long-term plan that Rachel has been setting out.

“And we can already see the benefit of that, frankly, in terms of the UK’s reputation when it comes to public finances, but economic management more generally. Certainly that’s what I’ve heard internationally and keep hearing just now.”

Chancellor accused of having ‘fled to China’

Chancellor Rachel Reeves with Chinese vice premier He Lifeng  in Beijing. Pic: Reuters
Image:
Chancellor Rachel Reeves with Chinese vice premier He Lifeng in Beijing. Pic: Reuters

The pair were speaking as the chancellor holds meetings in China in a bid to drum up investment for the UK economy, having ignored calls to cancel the long-planned trip because of economic turmoil at home.

Opposition parties have accused the chancellor of having “fled to China” rather than explain how she will fix the UK’s flatlining economy, and former prime minister Boris Johnson said Ms Reeves had “been rumbled” and said she should “make her way to HR and collect her P45 – or stay in China”.

Speaking during her trip, Ms Reeves said she would not alter her economic plans, with the October budget designed to return the UK to economic stability, and reiterated that “growth is the number one mission of this government”.

She said that “action” will be taken to meet the fiscal rules. That action is reported to include deeper spending cuts than the 5% efficiency savings already expected to be announced later this year, while cuts to the welfare bill are also said to be under consideration.

Continue Reading

Politics

Rachel Reeves facing ‘pressure’, but ‘people should give her time’, says Wes Streeting

Published

on

By

Rachel Reeves facing 'pressure', but 'people should give her time', says Wes Streeting

The health secretary has said that the cabinet is aware of the “pressure” on Chancellor Rachel Reeves amid volatile markets and a challenging broader economic picture – but appealed for the public to “give her time”.

Wes Streeting argued that the public “underestimates” the “amount of heavy-lifting” Ms Reeves has had to do and will have to continue to do, as he declared “total confidence” in her leadership in a staunch defence of her handling of the economy.

Separately, international development minister Anneliese Dodds, who attends cabinet, told Sky News that Ms Reeves has been “very clear about the long-term plan for our country” and she herself is “confident in that long-term plan”.

The comments from the two key ministers come after the past week saw a drop in the pound and an increase in government borrowing costs, which has fuelled speculation of more spending cuts or tax rises.

Streeting has ‘total confidence in chancellor’s leadership’

Speaking at the Jewish Labour Movement’s annual conference in north London, the health secretary acknowledged the fierce competition among all government departments for any available public funding from the Treasury, and told party members that all ministers “have to make choices and trade-offs” in where funding goes.

Mr Streeting went on to say that the chancellor and her deputy, Darren Jones, have “the hardest job of all because they have to make those choices across every bit of government spending, and they have to think about what’s in the interests of our overall economy and how we get businesses growing”.

Please use Chrome browser for a more accessible video player

Chancellor’s ‘pragmatic’ approach to China

He said: “I think people continue to underestimate both the amount of heavy lifting she has had to do in her first six months, and the amount of heavy lifting she will have to do in her next six months.

“And the cabinet doesn’t underestimate that – we understand the choices she has to make, the pressure she is under.”

As a result, cabinet ministers all “have a responsibility” to both “make tough choices and drive reform and value for money” within their departments, and also be “drivers of economic growth”.

“Nothing in the last six months has shaken my conviction that economic growth is the number one priority,” he said.

Follow Sky News on WhatsApp
Follow Sky News on WhatsApp

Keep up with all the latest news from the UK and around the world by following Sky News

Tap here

Continuing his defence of the chancellor’s handling of the economy so far, Mr Streeting said she is “trying to break us out of what has been the status quo and the economic orthodoxy of more than a decade”.

“People need to give her time, and they need to not forget that, without [Sir Keir Starmer’s] leadership, certainly we wouldn’t have won the last general election.

“Without Rachel’s leadership, we wouldn’t have won the last general election either. She built Labour’s economic credibility out of the ashes they were left in after the Corbyn leadership. And she has built that trust, built up that plan, and now she’s following through.”

He declared that he has “total confidence in the leadership that Rachel’s providing, and the leadership that the cabinet is following and driving with her, because all of us have to deliver economic growth for our country”.

Minister ‘confident in chancellor’s long-term plan’

Speaking in a separate session at the conference, Ms Dodds noted “speculation” about the fiscal headroom (the amount of money the chancellor will have available to spend), but said: “We have to focus on actually the evidence.

“And when we look at the evidence, we can see that the UK government has a chancellor who is very clear about the long-term plan for our country. She’s been delivering on it.”

Read more:
Even if Siddiq resigns, the damage may already be done to Starmer
Grim economic news raises stakes for embattled chancellor

Ms Dodds, who also attends cabinet, pointed to a “new fiscal system”, the chancellor’s new Industrial Strategy Council, as well as “record levels of investment under Rachel Reeves’s leadership”.

“I think it’s really important for us to focus on those fundamentals, on what has been achieved in a very short space of time. And I’m confident in that long-term plan that Rachel has been setting out.

“And we can already see the benefit of that, frankly, in terms of the UK’s reputation when it comes to public finances, but economic management more generally. Certainly that’s what I’ve heard internationally and keep hearing just now.”

Chancellor accused of having ‘fled to China’

Chancellor Rachel Reeves with Chinese vice premier He Lifeng  in Beijing. Pic: Reuters
Image:
Chancellor Rachel Reeves with Chinese vice premier He Lifeng in Beijing. Pic: Reuters

The pair were speaking as the chancellor holds meetings in China in a bid to drum up investment for the UK economy, having ignored calls to cancel the long-planned trip because of economic turmoil at home.

Opposition parties have accused the chancellor of having “fled to China” rather than explain how she will fix the UK’s flatlining economy, and former prime minister Boris Johnson said Ms Reeves had “been rumbled” and said she should “make her way to HR and collect her P45 – or stay in China”.

Speaking during her trip, Ms Reeves said she would not alter her economic plans, with the October budget designed to return the UK to economic stability, and reiterated that “growth is the number one mission of this government”.

She said that “action” will be taken to meet the fiscal rules. That action is reported to include deeper spending cuts than the 5% efficiency savings already expected to be announced later this year, while cuts to the welfare bill are also said to be under consideration.

Continue Reading

Politics

Tulip Siddiq could lose job if found she broke rules amid housing allegations, cabinet member suggests

Published

on

By

Tulip Siddiq could lose job if found she broke rules amid housing allegations, cabinet member suggests

Anti-corruption minister Tulip Siddiq could lose her job if the investigation into her properties finds she broke government rules, a cabinet member has suggested.

Science Secretary Peter Kyle was asked about Ms Siddiq on Sunday Morning with Trevor Phillips.

She has referred herself to the prime minister’s independent adviser on ministers’ interests, Sir Laurie Magnus, following reports she lived in properties in London linked to allies of her aunt, Sheikh Hasina, the deposed prime minister of Bangladesh.

There have also been questions about trips she took to Russia alongside her aunt.

Ms Siddiq insists she has “done nothing wrong”.

As economic secretary to the Treasury, Ms Siddiq oversees anti-corruption efforts in the financial sector as part of her brief.

Mr Kyle told Sky News: “With Tulip, she’s referred herself straight away to this.

“There is a process under way and we know full well it will be a functional process, and the outcomes of it will be stuck to by the prime minister and this government, a complete contrast to what we’ve had in the past.”

He gave this answer after Trevor pointed out Labour would have been calling for a sacking if the roles were reversed and the Tories were in power.

Read more:
Siddiq refers herself to ethics watchdog
Minister caught up in anti-corruption probe

Please use Chrome browser for a more accessible video player

‘Tulip Siddiq will lose job if she broke rules’

Mr Kyle contrasted his party’s stance with the Conservative one – saying he called for an investigation into allegations of bullying from Priti Patel, and she “had to be dragged to that inquiry”.

He added that he let the inquiry pan out.

“The results came out, she was found guilty, and no action happened,” Mr Kyle said.

His response came after Conservative leader Kemi Badenoch called for Ms Siddiq to be sacked yesterday.

Shadow chancellor Mel Stride repeated the calls today to Sky News.

He said: “What is not right is that the prime minister is not moving her out of that position and getting her to step down

“Because she is the anti-corruption minister, she has serious charges laid against her now, or serious accusations around corruption, and it’s going to be really impossible for her to do that job under current circumstances.

“So she should step down, and the prime minister needs to get a grip of that.”

👉Listen to Politics At Jack And Sam’s on your podcast app👈

The Sunday Times this week interviewed the current leader of Bangladesh, Muhammad Yunus, who called for Ms Siddiq to be investigated.

He said the properties should be handed back to his government if they were acquired through “plain robbery”.

Continue Reading

Trending