GPT, OpenAI's AI-based language model, has revolutionized content production. It automatically generates text and content that is indistinguishable from human content. Johannes Sommer takes you on the journey of AI-powered content production.

What is hidden behind the acronym GPT?

GPT is on everyone's lips, but what is meant by the 3 letters?
G = Generative language model that generates natural language text. It mimics human speech and automatically generates text that is indistinguishable from content written by real people.
P = Pretrained, means that the language model has been trained on an extensive set of global text data.
T = Transformer is the name of the neural network architecture used to train the model.

OpenAI - The driving force behind AI innovation

The American company OpenAI conceived the language model. After some development and Microsoft's current involvement, commercialization is a stated goal and corporate purpose. Assistance functions are being integrated into Microsoft products. Many other players, including Google, Meta, SAP and Adobe are investing heavily in the integration of AI technologies into their own product landscape.

The fascinating puzzle of the language model and its training

The base of the language model was trained on all content available on the English-language Internet and beyond.

It is important to emphasize that a language model uses probabilities to analyze the context of an input and generate the most likely next word based on that.

The model learns from a variety of content on the Internet and adjusts its probabilities. Changing the context changes the probabilities of the next words. ChatGPT does not work by retrieving knowledge, but by generating strings of words based on probabilities.

Example: “I ate the pizza, while it was still ?”. Based on the probability of the next word, the language model continues writing this sentence. In this example, the probability that the next word is “hot” is 80.45%.

When the context of the input changes, the probability of the next word changes. For the input, “My oven broke, so I ate the pizza while it was still ?” the next most likely word is no longer “hot” but “cold”.

ChatGPT calculates which word will be next. This is how word chains are built. This next word or character is called a token.

Tokens & Prompts: key components behind GPT's language model

Tokens are the generated output, while the input is called a prompt. A prompt can be a question. The entire conversation is considered as input in ChatGPT. Questions and answers are analyzed in context, because the complete course of the conversation is considered when using ChatGPT.

The length of the input and output plays a role in the calculation of the tokens. Because with longer inputs more tokens can be generated, which in turn increases the complexity of the calculation.

The other side of the coin

Despite the impressive abilities, there are weaknesses, some of which will be described. For example, calculating, counting and spelling do not work well. This is because - and this cannot be repeated often enough - GPT is not a knowledge model, but a language model. An example illustrates this:

How long does it take 24 musicians to play a piece of music if it takes 12 musicians 10 minutes? This question was asked to both GPT-3 and GPT-4.  

GPT-3 replied: 5 minutes.

If you point out the error to GPT-4, an apology and explanation of why this is not logical follows: And of course the correct solution ensues. This example illustrates how enormous the leap in development from GPT-3 to GPT-4 was.  

When humans become the problem for AI

Because humans train AI, unintentional bias in AI systems is a challenge. It is a problem that cannot be completely eliminated.

What is going to drastically change?

AI has found its way into many areas of work. Texts are created AI-supported, counter-arguments or summaries are written, images and photos are generated based on a text briefing, codes are written, videos or multimedia books are created in real time. And every week, a multitude of new companies and startups are forming to develop new applications. Accordingly, we face a veritable flood of applications that also impact our business processes.

Creative approaches to successful content production

Below are just a few ideas of areas where AI can help you with content creation.

• Content summary, teasers, generated rewrites
• Personalized newsletters
• Individualized agency messages  
• Augmented writing
• Content discovery and research
• SEO ghostwriting
• Video book production for podcasts and audio books
• Illustrations for content
• Generated captions

Examples from Retresco's practical experience

• Immobilienscout creates automated exposés.
• Retresco produces 124,000 soccer match reports a week for the German Football Association (DFB). The referees transmit the data to Fußball.de, based on which the texts are created.
• Media Markt and Saturn create automated product descriptions that are managed by a small editorial team.

As the examples show, customer data forms the basis for customized, automatically generated texts. Previously selected attributes are used by the AI to describe the products or write the texts. Even though a human is still involved in the process to check and confirm the model, this can produce content on a large scale.

Data protection with OpenAI?

Open AI has recently done a lot to become compliant with the GDPR. For example, business customers can exclude their data from being used for training language models.

Summary

Roy Amara, former President of the Institute for the Future said: “We tend to overestimate the impact of a technology in the short term. Of course, there are one or two teething problems that we smile at. But we underestimate the impact in the long run.”