Making AI-generated Articles Sound More Like a Human

High quality AI generated content can’t be done in a single prompt. Adaptity is a system chaining together multiple prompts, each doing different tasks before bringing everything together to write the article. The team has taken some inspiration for this system from my writing system post, so that’s a great place to begin if you want to see how these types of applications work.

Every good human writer starts with research, and Adaptify is no different: they plug into search engines to find relevant information about the topic, and insert that into the article writing prompt. Keyword research is automatically done, and the results inform another series of prompts for generating and selecting title ideas, as well as defining the outline of the content. Then each section is written in turn, before another series of prompts identify and fix any issues with the final article.

As an advisor to Adaptify and co-author of Prompt Engineering for Generative AI (published June 2024 through O’Reilly), I help the team out with solving specific problems as they continue to improve the quality of their AI-generated articles. In this post I’m going to talk about how to make AI-generated content sound more human, and less like the soulless output you get as standard from ChatGPT. You’ll see how a prompt engineer works day-to-day, and take away prompting strategies you can use to make your own AI-generated content more human.

Adaptify is a complex system, so as a prompt engineer you’d typically want to isolate one of the steps and try and improve it along one dimension through A/B testing different prompting strategies. In this case we’re interested in making the output of GPT-4o sound more ‘human’ and less like AI.

This is a growing concern in the world of AI, because while AI-generated content can be extremely helpful and is within Google’s guidelines when used appropriately, you don’t want it to sound too much like AI. People are starting to notice ChatGPT overuses specific words like ‘delve’ and may react negatively to these markers.

It’s important to state that we want our AI to sound more human not to trick anybody into reading AI content (we believe you should disclose AI usage), but because people prefer reading more humanized content. GPT-4o has been fine-tuned by OpenAI to be a helpful chatbot, not a world-class ghostwriter, and though it can be great at this task when steered in the right direction. If we can make our AI-generated content sound more human, we’ll make our articles easier to read, more engaging, and better aligned with the preferences of our readers.

Adaptify uses the framework LangChain and has their observability tool LangSmith turned on, making it easier for me as a prompt engineer to understand what it is they’re doing without having to dig through their code. I found the part of the chain that deals with writing intros, and took a look at a recent run of that chain to see what prompt they were sending to OpenAI.

I can see both the prompt and the response (in green), and stripping out any customer-specific information I can get at the underlying structure of the prompt template:

``

``

``

``

``

``

``

``

``

``

``

``

``

``

``

To get the right writing style, Adaptify is employing the roleplay strategy, asking the AI to write in the style of Rand Fishkin from SparkToro. Fishkin is a legend in the SEO world as a founder of Moz, a popular SEO tool, and has an informative writing style that is great for informational content where you need to explain complex topics to non-experts. They also include a handful of instructions to keep the content short and right to the point, as standard GPT-4 content tends to be too wordy.

Let’s the the difference in results by deleting that section of the prompt and comparing what output we get for the introduction section, keeping all other context and instructions in the prompt the same:

You can see visually that Adaptify’s prompt generates an intro section that is half as long. The GPT-4o response included the word ‘delve’! It also has a few other obvious AI markers like talking about a ‘comprehensive guide’, and making a claim about studies on the return on investment of bathroom remodeling without providing links backing up that claim. I hope you’ll agree the Adaptify response is a better answer, more direct to the point, and more engaging to read.

One of the great joys of prompt engineering is that the work is never done: you can always improve. Role Playing prompts that mention someone by name are undeniably effective, but they can become unreliable over time as the model updates on that author’s latest writing (already there would be a difference between Fishkin’s writing style at SparkToro, and his previous work at Moz). I prefer the more flexible approach of unbundling the elements of a famous creator’s style and then modifying them to make something more unique. The other problem is a lack of personal touch, including personal anecdotes and a more conversational tone to engage the reader. I know this is written by an AI that can’t have personal anecdotes, but it can provide anecdotes the target audience may identify with.

First, let’s get a style guide or list of bullet points that encapsulate the writing style of Rand Fishkin, by prompting ChatGPT to understand what it knows about him:

Briefly describe the tone and writing style of Rand Fishkin from SparkToro in bullet points. Do not mention his name. Someone who had never heard of him could replicate his writing style from these bullet points.

- Conversational and approachable tone

- Engaging and relatable, often using personal anecdotes

- Informative and insightful with a focus on practical advice

- Transparent and honest, openly sharing both successes and failures

- Data-driven and analytical, emphasizing evidence-based conclusions

- Occasionally humorous, using light-hearted language to maintain interest

- Clear and concise, avoiding unnecessary jargon

- Encourages critical thinking and questioning assumptions

- Empathetic and supportive, understanding reader challenges and providing encouragement

When we were emulating Fishkin we didn’t have full control over what aspects of his writing style we were emulating. Now we have an unbundled list of bullet points we can more easily modify some of the points to arrive at something more original. For example, we wanted the writing to be simple and clear, but ChatGPT only mentions clear. It also talks about being data-driven, which is a good fit for SEO but perhaps not for bathroom remodeling. I also found it wasn’t adding enough relatable examples, so I added a line for that too. Here’s the modified list I added to the prompt:

Now let’s see how that impacts the output we get:

Whether you personally like the new style or not, it definitely sounds a lot less like the standard ChatGPT response. It uses relevant examples like the ‘baking a cake’, and personal anecdotes that establish expertise like ‘...on a single 15 Amp circuit’. It even used the word spiffy. If you don’t like elements of this writing style, you can just go in and edit the bullet point responsible, or add a new one into the mix. Roleplay is a great tactic for getting good results from an LLM, but I find I get far more creative and original results with more flexibility using this unbundling approach. I cover this and hundreds more examples in Prompt Engineering for Generative AI, and if you’re applying AI at scale, like the team at Adaptify is, you’ll find it useful.