Internet firm Cloudflare will start blocking artificial intelligence crawlers from accessing content without website owners’ permission or compensation by default, in a move that could significantly impact AI developers’ ability to train their models.

Starting Tuesday, every new web domain that signs up to Cloudflare will be asked if they want to allow AI crawlers, effectively giving them the ability to prevent bots from scraping data from their websites.

Cloudflare is what’s called a content delivery network, or CDN. It helps businesses deliver online content and applications faster by caching the data closer to end-users. They play a significant role in making sure people can access web content seamlessly every day.

Roughly 16% of global internet traffic goes directly through Cloudflare’s CDN, the firm estimated in a 2023 report.

“AI crawlers have been scraping content without limits. Our goal is to put the power back in the hands of creators, while still helping AI companies innovate,” said Matthew Prince, co-founder and CEO of Cloudflare, in a statement Tuesday.

“This is about safeguarding the future of a free and vibrant Internet with a new model that works for everyone,” he added.

AI crawlers are automated bots designed to extract large quantities of data from websites, databases and other sources of information to train large language models from the likes of OpenAI and Google.

Whereas the internet previously rewarded creators by directing users to original websites, according to Cloudflare, today AI crawlers are breaking that model by collecting text, articles and images to generate responses to queries in a way that users don’t need to visit the original source.

This, the company adds, is depriving publishers of vital traffic and, in turn, revenue from online advertising.

Tuesday’s move builds on a tool Cloudflare launched in September last year that gave publishers the ability to block AI crawlers with a single click. Now, the company is going a step further by making this the default for all websites it provides services for.

OpenAI says it declined to participate when Cloudflare previewed its plan to block AI crawlers by default on the grounds that the content delivery network is adding a middleman to the system.

The Microsoft-backed AI lab stressed its role as a pioneer of using robots.txt, a set of code that prevents automated scraping of web data, and said its crawlers respect publisher preferences.

“AI crawlers are typically seen as more invasive and selective when it comes to the data they consumer. They have been accused of overwhelming websites and significantly impacting user experience,” Matthew Holman, a partner at U.K. law firm Cripps, told CNBC.

“If effective, the development would hinder AI chatbots’ ability to harvest data for training and search purposes,” he added. “This is likely to lead to a short term impact on AI model training and could, over the long term, affect the viability of models.”

WATCH: AI engineers are in high demand — but what is the job really like?

XAI, the artificial intelligence startup run by Elon Musk, raised a combined $10 billion in debt and equity, Morgan Stanley said.

Half of that sum was clinched through secured notes and term loans, while a separate $5 billion was secured through strategic equity investment, the bank said on Monday.

The funding gives xAI more firepower to build out infrastructure and develop its Grok AI chatbot as it looks to compete with bitter rival OpenAI, as well as with a swathe of other players including Amazon-backed Anthropic.

In May, Musk told CNBC that xAI has already installed 200,000 graphics processing units (GPUs) at its Colossus facility in Memphis, Tennessee. Colossus is xAI’s supercomputer that trains the firm’s AI. Musk at the time said that his company will continue buying chips from semiconductor giants Nvidia and AMD and that xAI is planning a 1-million-GPU facility outside of Memphis.

Addressing the latest funds raised by the company, Morgan Stanley that “the proceeds will support xAI’s continued development of cutting-edge AI solutions, including one of the world’s largest data center and its flagship Grok platform.”

xAI continues to release updates to Grok and unveiled the Grok 3 AI model in February. Musk has sought to boost the use of Grok by integrating the AI model with the X social media platform, formerly known as Twitter. In March, xAI acquired X in a deal that valued the site at $33 billion and the AI firm at $80 billion. It’s unclear if the new equity raise has changed that valuation.

xAI was not immediately available for comment.

Last year, xAI raised $6 billion at a valuation of $50 billion, CNBC reported.

Morgan Stanley said the latest debt offering was “oversubscribed and included prominent global debt investors.”

Competition among American AI startups is intensifying, with companies raising huge amounts of funding to buy chips and build infrastructure.

OpenAI in March closed a $40 billion financing round that valued the ChatGPT developer at $300 billion. Its big investors include Microsoft and Japan’s SoftBank.

Anthropic, the developer of the Claude chatbot, closed a funding round in March that valued the firm at $61.5 billion. The company then received a five-year $2.5 billion revolving credit line in May.

Musk has called Grok a “maximally truth-seeking” AI that is also “anti-woke,” in a bid to set it apart from its rivals. But this has not come without its fair share of controversy. Earlier this year, Grok responded to user queries with unrelated comments about the controversial topic of “white genocide” and South Africa.

Musk has also clashed with fellow AI leaders, including OpenAI’s Sam Altman. Most famously, Musk claimed that OpenAI, which he co-founded, has deviated from its original mission of developing AI to benefit humanity as a nonprofit and is instead focused on commercial success. In February, Musk alongside a group of investors, put in a bid of $97.4 billion to buy control of OpenAI. Altman swiftly rejected the offer.

— CNBC’s Lora Kolodny and Jonathan Vanian contributed to this report.

Huawei has open-sourced two of its artificial intelligence models — a move tech experts say will help the U.S.-blacklisted firm continue to build its AI ecosystem and expand overseas. 

The Chinese tech giant announced on Monday the open-sourcing of the AI models under its Pangu series, as well as some of its model reasoning technology.

The moves are in line with other Chinese AI players that continue to push an open-source development strategy. Baidu also open-sourced its large language model series Ernie on Monday. 

Tech experts told CNBC that Huawei’s latest announcements not only highlight how it is solidifying itself as an open-source LLM player, but also how it is strengthening its position across the entire AI value chain as it works to overcome U.S.-led AI chip export restrictions.

In recent years, the company has transformed from a competent private sector telecommunications firm into a “muscular technology juggernaut straddling the entire AI hardware and software stack,” said Paul Triolo, partner and senior vice president for China at advisory firm DGA-Albright Stonebridge Group.

In its announcement Monday, Huawei called the open-source moves another key measure for Huawei’s “Ascend ecosystem strategy” that would help speed up the adoption of AI across “thousands of industries.”

The Ascend ecosystem refers to AI products built around the company’s Ascend AI chip series, which are widely considered to be China’s leading competitor to products from American chip giant Nvidia. Nvidia is restricted from selling its advanced products to China. 

Ray Wang, principal analyst at Constellation Research, said the chip-to-model strategy is similar to that of Google, a company that is also developing AI chips and AI models like its open-source Gemma models.

Huawei’s announcement on Monday could also help with its international ambitions. Huawei, along with players like Zhipu AI, has been slowly making inroads into new overseas markets.

In its announcement Monday, Huawei invited developers, corporate partners and researchers around the world to download and use its new open-source products in order to gather feedback and improve them.

“Huawei’s open-source strategy will resonate well in developing countries where enterprises are more price-sensitive as is the case with [Huawei’s] other products,” Einstein said. 

As part of its global strategy, the company has also been looking to bring its latest AI data center solutions to new countries.