Does AI content creation work?

The interest in AI content creation tools like ChatGPT and Jasper has continued to grow as more people consider how these tools can fit into their workflow. So, we’ve resurfaced this experiment we ran in 2021 that explored the maturity of AI-written content at the time. How does it compare to today?

Originally published 08/30/2021

Clockwork leverages AI in many of our client initiatives, and we wanted to explore the maturity of AI content creation. Could tools like Jasper, an AI copywriting tool and the testbed of current and future AI systems used by Facebook, generate usable content? How much training or guidance does a tool for AI content creation require? What level of fidelity is possible? In other words, could we give Jasper a topic and some keywords and get a search engine optimized blog article on a specialized subject?

Here’s what our experiment looked like:

Jasper provides a portfolio of starter templates for formats including email, website, social media, and SEO. We tested out nearly all of them.

The long-form assistant includes a “blog-post workflow” to guide the inputs and provide choice-points for the user. We started by dropping in our content idea and keywords. We were curious if Jasper could create an article about evaluating your tech stack. We gave it a few sentences and questions to help it “think” through the focus.

After we submitted our topic, Jasper gave us the option of writing our own title or giving us some options.

Jasper promises content that is optimized for engagement and search engines. All of the titles provided had a click-inviting format. The tool will keep providing title options for as many times as you want to “generate more ideas.”

We selected “11 Steps to Modernize Your Infrastructure.” The tool will then generate introduction paragraphs based on the title. These were the first three created.

After choosing the first option provided, we moved to blog post generation.

AI generated blog post

This blog post is about modernizing your infrastructure. The first question you should ask yourself is: what are the most important components in my technology stack? I’ll discuss how to think about this, and some of the steps involved in evaluating your current stack, deciding what to upgrade and replace, and ensuring a scalable platform.

You should also consider what type of technology stack you want to have, which will determine the components that are important. For example, if your goal is to be highly scalable and distributed then a microservices architecture using containers might make sense, but this may not work as well for an organization with legacy software written in COBOL or assembly language.

The first step is to perform an audit of your technology stack. This involves identifying what components exist and what type of organization you are: embedded, legacy software manufacturer or IT service provider. The next steps depend on the outcome of this information gathering process. If you’re running a highly scalable distributed system then migrating from monolithic systems to microservices using container technology may make sense. If you’re an embedded system developer or manufacturer, then upgrading to a modern software development methodology and hardware might be the next step.

In any case, it’s important to reevaluate your stack regularly since technologies change fast and can become obsolete quickly. You should have regular discussions with stakeholders on how they’re using technology, and consider new technologies that might be beneficial to them. This will help ensure a modern platform for your organization’s infrastructure.

Identify the components that are most important in your technology stack For example: if you want high scalability and distribution use microservices with containers – Perform an audit of what type of organization you have (embedded, legacy software manufacturer, or IT service provider) This will determine the next steps you should take.
If you’re a highly scalable distributed system then migrate from monolithic systems to microservices using container technology; if you’re an embedded system developer and/or manufacturer upgrade your current stack with modern development methodology and hardware.
Reevaluate your tech stack regularly as technologies change quickly and can become obsolete.
Have regular discussions with stakeholders on how they’re using technology, and consider new technologies that might be beneficial to them. This will help ensure a modern platform for your organization’s infrastructure.

So, does AI content creation work?

What we found repeatedly with experiments in long-form content is that Jasper generated passable first draft editions. Those first drafts usually included awkward phrasing or strange details (like the reference to COBOL and assembly language). With over a dozen attempts on half a dozen topics, we were unable to get to a refined final long-form draft. But, the AI-generated content was inevitably useful.

We were impressed with the short-form content generation — from titles to taglines to posts, the tool provided options that were immediately usable. Jasper provided great case study titles, CTAs (calls to action), and social media posts.

The experience reminded us of the Turing test. Developed by Alan Turing and originally called the imitation game, the test evaluates an AI’s ability to mimic human interaction. The longer the content, the faster we saw the AI behind the curtain.

Curious how AI can fit into your tech stack? Talk to us.