We presented survey respondents with four groups of written copy about different subjects: two groups of five sentences, and two groups of five paragraphs. The participants had to select which ones they thought were written by AI.
Ai-generated contentX%= % of respondents who correctly thought this was written by AI.
Which of the following sentences about Google Ads do you think are AI-generated?
Google Ads are an effective way for businesses to reach their target audience through paid advertisements.61.9%
Google Ads are great for small businesses who want to advertise their business on search engines like Google, Yahoo!, Bing, etc..51%
Google Ads targeting allows you to show your ads to the people most likely to be interested in your products.49%
Google Ads gives businesses of all sizes the ability to reach a targeted audience.45.6%
The flexibility of Google Ads means you’re in control of what you spend, per month, per day and per ad.34%
Which of the following sentences about link building do you think are AI-generated?
Link building is the practice of creating links between websites in order to improve their rankings on search engines.61.6%
Link building remains one of the most important and effective digital marketing tactics.50.7%
SEO experts consider backlinks to be the most effective tactic for driving improved search rankings.50.7%
A website without links will be ranked lower than one with lots of links.49.3%
Targeting links from relevant, niche websites is a great way to drive traffic and indicate value to search engines.41.8%
Which of the following paragraphs about blog writing best practice do you think are AI-generated?
When writing a blog article, make sure you include an introduction paragraph that outlines the topic you're discussing. This gives the reader a sense of where you're coming from and why you're talking about something.51.7%
When writing a blog post it’s essential to know your audience. Otherwise, you’re just guessing what they need and want. Great content is usually based on data, gained via industry and keyword analysis.50.3%
Ensure you use headers that clearly outline the topics you’re covering. This helps readers who are scanning for specific information, but it also helps search engine crawlers understand what your piece is about, helping your post rank.43.5%
The number one rule of writing is to write for yourself first and then worry about whether others will read it. If you don't like something, rewrite it. It's okay to take feedback, but if you're getting negative criticism, try not to let it affect you.42.2
If you want to write a great article, you must first know your audience. Who are they? What do they care about? Why should anyone read this article? The answers to those questions will help you craft content that gets people excited about reading it.37.4
Which of the following paragraphs about keyword usage in blogs do you think are AI-generated?
The best way to find out what keywords people are searching for is by using Google Analytics. This free tool will show you which pages are getting traffic from search engines, social media sites, email marketing campaigns, etc.57.3%
Keyword stuffing is where a writer loads a piece of content with an unnaturally high frequency of the same keyword. The intention behind this is to trick search engines into ranking a piece higher in search engine results pages, though it no longer works.49%
Many SEOs recommend writing naturally about a topic, using the target keyword a couple of times, and including semantic variations of it. To prevent accidental keyword stuffing, check your keyword density when proofreading.46.2%
Keyword density (sometimes called keyword frequency) refers to how often a specific keyword is used in your copy. Keyword density is often reported as a percentage - and though the debate continues, a ‘good’ percentage is suggested to be 1-2%.38.5%
If you want to rank well organically, it’s important to write naturally. This means writing like you would speak – using simple language and clear sentences. It’s not just about having keywords sprinkled throughout your text.37.1%
Contrary to user belief (as highlighted in a previous question), with the proper effort put into the strategy, the prompt design, and the model's adaptation, AI-generated content can be, in some cases, indistinguishable from human-generated content. The merit of the content strategy and editing will become increasingly more important. In the meantime, prompt design will continue to become a skill that's needed by people using such tools that will help improve the quality of long-form machine-generated content on the user side, while the development of more advanced models will assist with this on the technology side.
Understanding the shortcomings of the models used in content generation will also be pivotal in overcoming their limitations and making the content less distinguishable from out-of-the-box machine-generated content, and therefore also making it more competitive from a search standpoint.
It feels almost cruel to completely undermine the assertions made during the previous questions, but the contrast between how the majority of respondents perceive AI content quality, and how they performed in this mini-test, is fascinating.
It has to be said, we're looking at short form content here, and I suspect the results would be different if we were talking about long form content, but nevertheless these numbers show AI is getting much better at writing passages of copy that are as good as a human writer.
If you look at the total set of data here, 53% of the responses correctly identified whether the content was generated by AI or written by human.
On the surface that might seem like a victory for human writers, but that gap is narrow. That might seem particularly surprising considering the advantage the human writers were given in this test: Respondents knew that some text was written by AI and where specifically considering this point and the topic is quite a specialist one. Most critically though, the audience is one that is already knowledgeable on the topic.
My own tests suggest that the topical knowledge of the reader is the biggest factor in the results (even more so than the knowledge of the human writers the AI is being compared against).
Despite these advantages, the audience was only marginally better at sorting human from AI content than if it had been done by flipping a coin.
It seems like lots of people can't tell the difference, though I think with the longer the copy, the more AI writing stands out. When you get to paragraphs and pages, the issues increase significantly.