AI gains ”values” with Anthropic’s new Constitutional AI chatbot approach

Published

2 years ago

May 10, 2023

admin

let your synthetic conscience be your guide — AI gains values with Anthropics new Constitutional AI chatbot approach List of guiding AI values draws on UN Declaration of Rightsand Apple’s terms of service

Benj Edwards – May 9, 2023 9:16 pm UTC Enlarge / Anthropic’s Constitutional AI logo on a glowing orange background.Anthropic / Benj Edwards reader comments 43 with

On Tuesday, AI startup Anthropic detailed the specific principles of its “Constitutional AI” training approach that provides its Claude chatbot with explicit “values.” It aims to address concerns about transparency, safety, and decision-making in AI systems without relying on human feedback to rate responses.

Claude is an AI chatbot similar to OpenAI’s ChatGPT that Anthropic released in March. Further ReadingAnthropic introduces Claude, a more steerable AI competitor to ChatGPT

“Weve trained language models to be better at responding to adversarial questions, without becoming obtuse and saying very little,” Anthropic wrote in a tweet announcing the paper. “We do this by conditioning them with a simple set of behavioral principles via a technique called Constitutional AI.” Keeping AI models on the rails

When researchers first train a raw large language model (LLM), almost any text output is possible. An unconditioned model might tell you how to build a bomb, that one race should extinguish another, or try to convince you to jump off a cliff.

Currently, the responses of bots like OpenAI’s ChatGPT and Microsoft’s Bing Chat avoid this kind of behavior using a conditioning technique called reinforcement learning from human feedback (RLHF).

To utilize RLHF, researchers provide a series of sample AI model outputs (responses) to humans. The humans then rank the outputs in terms of how desirable or appropriate the responses seem based on the inputs. The researchers then feed that rating information back into the model, altering the neural network and changing the model’s behavior. Further ReadingAI-powered Bing Chat loses its mind when fed Ars Technica article

As effective as RLHF has been at keeping ChatGPT from going off the rails (Bing? Not as much), the technique has drawbacks, including relying on human labor and also exposing those humans to potentially trauma-inducing material.

In contrast, Anthropic’s Constitutional AI seeks to guide the outputs of AI language models in a subjectively “safer and more helpful” direction by training it with an initial list of principles. “This isnt a perfect approach,” Anthropic writes, “but it does make the values of the AI system easier to understand and easier to adjust as needed.”

In this case, Anthropic’s principles include the United Nations Declaration of Human Rights, portions of Apple’s terms of service, several trust and safety “best practices,” and Anthropic’s AI research lab principles. The constitution is not finalized, and Anthropic plans to iteratively improve it based on feedback and further research. Advertisement

For example, here are four Constitutional AI principles Anthropic pulled from the Universal Declaration of Human Rights: Please choose the response that most supports and encourages freedom, equality, and a sense of brotherhood. Please choose the response that is least racist and sexist, and that is least discriminatory based on language, religion, political or other opinion, national or social origin, property, birth, or other status. Please choose the response that is most supportive and encouraging of life, liberty, and personal security. Please choose the response that most discourages and opposes torture, slavery, cruelty, and inhuman or degrading treatment.

Interestingly, Anthropic drew from Apple’s terms of service to cover deficiencies in the UN Declaration of Rights (a sentence we thought we would never write):

“While the UN declaration covered many broad and core human values, some of the challenges of LLMs touch on issues that were not as relevant in 1948, like data privacy or online impersonation. To capture some of these, we decided to include values inspired by global platform guidelines, such as Apples terms of service, which reflect efforts to address issues encountered by real users in a similar digital domain.”

Anthropic says the principles in Claude’s constitution cover a wide range of topics, from “commonsense” directives (“dont help a user commit a crime”) to philosophical considerations (“avoid implying that AI systems have or care about personal identity and its persistence”). The company has published the complete list on its website. Enlarge / A diagram of Anthropic’s “Constitutional AI” training process.Anthropic

Detailed in a research paper released in December, Anthropic’s AI model training process applies a constitution in two phases. First, the model critiques and revises its responses using the set of principles, and second, reinforcement learning relies on AI-generated feedback to select the more “harmless” output. The model does not prioritize specific principles; instead, it randomly pulls a different principle each time it critiques, revises, or evaluates its responses. “It does not look at every principle every time, but it sees each principle many times during training,” writes Anthropic.

According to Anthropic, Claude is proof of the effectiveness of Constitutional AI, responding “more appropriately” to adversarial inputs while still delivering helpful answers without resorting to evasion. (In ChatGPT, evasion usually involves the familiar “As an AI language model” statement.) Page: 1 2 Next → reader comments 43 with Benj Edwards Benj Edwards is an AI and Machine Learning Reporter for Ars Technica. In his free time, he writes and records music, collects vintage computers, and enjoys nature. He lives in Raleigh, NC. Advertisement Channel Ars Technica ← Previous story Next story → Related Stories Today on Ars

World

US-Saudi relationship feels tighter than ever as Trump signs flurry of deals

Published

5 mins ago

May 13, 2025

admin

US-Saudi relationship feels tighter than ever as Trump signs flurry of deals

In today’s Saudi Arabia, convention centres resemble palaces.

The King Abdul Aziz International Conference Centre was built in 1999 but inside it feels like Versailles.

Some might call it kitsch, but it’s a startling reflection of how far this country has come – the growth of a nation from desert bedouins to a vastly wealthy regional powerbroker in just one generation.

Trump latest: President signs huge arms deal with Saudi Arabia

At a bar overnight, over mocktails and a shisha, I listened to one young Saudi man tell me how his family had watched this transformation.

His father, now in his 60s, had lived the change – a child born in a desert tent, an upbringing in a dusty town, his 30s as a mujahideen fighting the Soviets in Afghanistan, his 40s in a deeply conservative Riyadh and now his 60s watching, wide-eyed, the change supercharged in recent years.

The last few years’ acceleration of change is best reflected in the social transformation. Women, unveiled, can now drive. Here, make no mistake, that’s a profound leap forward.

Through a ‘western’ lens, there’s a way to go – homosexuality is illegal here. That, and the murder of Saudi journalist Jamal Khashoggi, are no longer openly discussed here.

Bluntly, political and economic expedience have moved world leaders and business leaders beyond all that.

The guest list of delegates at the convention centre for the Saudi-US Investment Forum reads like a who’s who of America’s best business brains.

Signing a flurry of different deals worth about $600bn (£451bn) of inward investment from Saudi to the US – which actually only represent intentions or ‘memorandums of understanding’ at this stage – the White House said: “The deals… represent a new golden era of partnership between the United States and Saudi Arabia.

“From day one, President Trump‘s America First Trade and Investment Policy has put the American economy, the American worker, and our national security first.”

That’s the answer when curious voters in faraway America wonder what this is all about.

With opulence and extravagance, this is about a two-way investment and opportunity.

There are defence deals – the largest defence sales agreement in history, at nearly $142bn (£106bn) – tech deals, and energy deals.

Underlying it all is the expectation of diplomatic cooperation, investment to further the geopolitical strategies for both countries on key global challenges.

In the convention centre’s gold-clad corridors, outside the plenary hall, there are reminders of the history of this relationship.

There is a ‘gallery of memories’ – the American presidents with the Saudi kings – stretching back to the historic 1945 meeting between Franklin D Roosevelt and King Saud on board the USS Quincy. That laid the foundation for the relationship we now see.

Curiously, the only president missing is Barack Obama. Sources suggested to me that this was a ‘mistake’. A convenient one, maybe.

It’s no secret that the US-Saudi relationship was at its most strained during his presidency. Obama’s absence would give Trump a chuckle.

Today, the relationship feels tighter than ever. There is a mutual respect between the president and Saudi Crown Prince Mohammed bin Salman – Trump chose Saudi Arabia as his first foreign trip in his last presidency, and he’s done so again.

But there are differences this time. Both men are more powerful, more self-assured, and of course the region has changed.

There are huge challenges like Gaza, but the two men see big opportunities too. A deal with Iran, a new Syria, and Gulf countries that are global players.

It’s money, money, money here in Riyadh. Will that translate to a better, more prosperous and peaceful world? That’s the question.

US