Familiarise yourself with the capabilities and limitations of large language models

Bastian says: “Get familiarised with large language models.

This is one of the foundational elements of AI. Marketers need to understand how they work and what capabilities they have, but also what limitations they have, so they can become more efficient and utilise them in the best way possible.”

How would you summarise the way that large language models work?

“Essentially, it’s a huge amount of data that has been put into a specific type of database, and with that data, you can fulfil certain types of tasks. For example, ChatGPT is an interface for a large language model provided by OpenAI.

It’s important to understand that these models don’t write anything. They have training data and, based on that training data, they predict the likelihood of what’s supposed to happen next. This is how they generate text.

If you give the model a task through prompting (formulating something via the interface), there is no creative magic happening. Based on the data that has been ingested in the training process, the model simply creates a new output.”

What should and shouldn’t LLMs be used for?

“There’s a whole lot of new tech around AI, including on the generative AI side, and it’s moving extremely fast. The biggest issue is that a lot of the stuff that’s currently being shown off is predominantly in a beta state. Most of it is not really production-ready.

People are fascinated by the topic (myself included) and there are a lot of gains to be found already. On the other hand, we need to be cautious. Things are changing fast and what is currently there has limitations.

Say you task the model with creating a piece of text around the Audi A3. The Audi A3 has different features in different countries. In Germany, it might have different dimensions, or it might come with a different default package than it does in the UK or France. If you just task the model with producing a piece of content around that, then you might end up with mixed facts. The dimensions might be wrong, or the default pricing might be wrong, for example.

It’s extremely important to fact-check, double-check, and triple-check the output. It can be used for research, and it can be used for drafting things but it’s by no means meant to produce content without double-checking. That is one of the biggest misconceptions with ChatGPT, Jasper, or any other model right now.”

If your prompts are higher quality, do you need to fact-check less?

“You can increase the quality, but errors will still happen. The model is predicting things and predictions can be wrong.

Using the Google search generative experience (the AI-based snippet on top of the search results), I was inputting queries about tech companies. I asked, ‘How many unicorns are there in France?’, and it said, ‘27’. When I gave it a slightly different prompt for the same question it said, ‘36’. This is what we call a hallucination.

The issue is that the model doesn’t know. There is no exact number and there is no precise data source that says exactly how many unicorn companies are in France. Therefore, it’s trying to ‘guesstimate’ what the right answer could be. That is very dangerous. That was a soft example, but a lot of important research relies on information being accurate, especially from big companies like Google. You have to be mindful and cautious about blindly trusting that info.

On the other hand, they’re fantastic for speeding up certain processes and tasks that you’re doing in your day-to-day. If you are doing research, you could now use a large language model where before you would have needed to go to 10/20/30 different sources. You still have to check it, but you would have needed to check your 20 sources as well. In that regard, you’re becoming more efficient.

We have a huge client that does corporate training within their organisation, and they previously had to do recordings in person, in a studio. It’s expensive, it takes time, the person needs to be there, etc. Now, they’re doing it with an AI avatar. They recorded the speaker once and recorded their voice, so they have voice samples from previous trainings. Now they only need to produce scripts and the AI avatar is going to be in the video.

It’s even cooler because they’re a multilingual organisation. With AI, they can scale that because they just use a different avatar or voiceover and they have it in French, Spanish, etc. Some people are not as fluent in English as others, and this helps to ensure that their experience is included and onboarded. There are already massive gains to be found in terms of efficiency and even satisfaction.”

Can using a tool like Jasper be more effective than using ChatGPT natively?

“It depends on the use case.

It’s also worth noting that there are different large language models as well – like Neuroflash which is popular in the German region – and they all have different training and different types of feedback mechanisms. Some are better suited to certain types of tasks than others. Med-PaLM is a large language model specifically for pharmaceutical and medical information. It is much more accurate in those areas, but it has only been trained on that kind of content. It couldn’t give you an answer about the best chess strategy, for example.

Having specific models for specific tasks does make sense but we’re currently using large language models for a lot of things they are not meant to be used for. In the future, those models may start to rely on third-party tools through APIs or a different type of architecture so that they can then pull data from other sources when they’re not confident enough to properly answer a question.

There are a ton of smaller tools right now, which is mainly because they are trying to solve certain specific tasks. As an SEO, this is great, especially if you’re on the freelance or consulting side. You can use these tools without having to spend tons of time and/or money on implementation, which is a massive efficiency gain. However, there’s a high chance that 80% of those special interest tools will die out in the next 18/24 months and be absorbed by the bigger models.”

Are there any specific SEO tasks that you would definitely use LLMs for at the moment?

“It can already really help with things like meta descriptions and page titles when it comes to working at scale. I would still have someone handcrafting the meta description in a more manual way for the homepage or the meta description template for a certain specific type of content.

However, if you’re talking about thousands or millions of pages, then the next best thing after templating it would be to use an LLM. You can dump in a URL (or even scrape that through a ChatGPT plugin), and then it’s going to pull the data and create meta descriptions within certain limitations. For that kind of content generation, and suggestions at scale, it’s great.

It can also produce more lightweight content. An article about a certain island in Greece might be relatively easy to do compared to a very specific guideline for XYZ because that’s not based on common knowledge. It’s also great for classification. If you need a new hierarchy for internal linking, you can map out and classify pages through an LLM. It’s much more efficient and it can ingest much more data than you would ever be able to do manually.

It is going to get crazier in the next few years, from a variety of standpoints. Google has the SGE snippet on top, and that’s obviously going to change search results quite significantly. Not only is something pushed to the top, but that may have an impact on metrics, and we need to consider how to ensure our clients or brands appear in the snippets.

There are also going to be huge advancements on the tooling side. Running your own LLM is not rocket science anymore. Google’s Vertex and some others have created relatively simple, ready-made solutions already. With a bit more time, a lot of the tool providers will move towards that direction.”

How are LLMs likely to change and how can SEOs prepare for those changes?

“Sadly, there’s no one answer to that. Most LLMs are transformer-based right now. That’s the architecture and the underlying foundational technology. They are great for what we’re currently using them for, but they also have limitations.

For example, to input more knowledge into an LLM, you need more training data. That means the model will grow in size, which also means that you need more hardware. If you query a transformer-based LLM currently, from a tech standpoint, you need to go through the entire model. That obviously doesn’t scale endlessly. It scales to the extent that you can throw more and more hardware at the problem (you’ve probably seen the insane numbers that NVIDIA released recently and the stock market’s reaction to it), however, it can’t scale endlessly.

Therefore, we will probably start seeing different approaches, and some projects are currently researching different directions that could be taken. From an implementation standpoint, as an organisation, you need to build some middleware.

You might currently be in the position to use an LLM for your SEO, which is fantastic. You can take all your knowledge and your content, put it in an LLM and then run certain types of models on your own. However, the LLM model itself might change. If you build a middleware API layout then, moving forward, it will be much easier to adapt to a new model. You can just move over the data and then instantly benefit from the work that you have already done.

We know that the models might change. I can’t say how this might look in practice (I’m coming from a practitioner’s standpoint, not a researcher’s) but this would be a logical approach to take. We’re seeing a lot of large organisations do this because they want to reap the benefits right now. There are fantastic gains to be had. You can make customer service more efficient and share knowledge internally more efficiently if you can collect all your knowledge from different sources and put it into these types of models.

Don’t just sit on it and wait two years. Taking a hybrid approach, and being prepared to shift over, is a much better way to do it.”

How should SEOs use LLMs to enhance their knowledge and stay up-to-date?

“The big difference is prompting and how you actually interact with the LLMs. This is going to change too because we’re moving towards a more multimodal world, where we will not only have the ability to prompt using text but also combine different types of inputs. BARD now allows you to upload a photo and ask, ‘What’s in this image?’, and it will give you a description of the photo. It’s like a combination of Lens and BARD. We’re also seeing multimodality with things like gestures.

You need to understand how you can prompt the model to ensure that you get what you want out of it, keeping in mind that there are certain limitations such as hallucination. You might use a plugin to bring in external data or do certain calculations. Even if you’re not using it in practice, I would strongly recommend at least having a play.

Also, don’t forget about the code interpreter in ChatGPT Plus. As SEOs, we do a ton of analysis, looking at numbers and trying to figure out what’s happening. From an agency perspective, if you look at a new website that you’re not familiar with and you’re seeing ups and downs and dips, you can overlay that with different types of data sources. You can ask a model like this to generate ideas about what might have happened. That is a really cool way to use the LLM as a research assistant before you do it yourself to double-check. Again, it’s about the efficiency that you gain from utilising it.

Experimenting and having a play will help you to understand what is happening, even though it’s hard to say that you understand how AI really works. Even engineers don’t understand it completely, in a way, because the machines are reiterating and learning by themselves. However, you can understand how to utilise it, play with it, and figure it out.

That has always been part of SEO. When I started out, we learnt by tinkering and playing, and this has never really changed. If you’re not naturally curious about things, then you will have a very hard time. That is going to be even more true in the future.”

If an SEO is struggling for time, what should they stop doing right now so they can spend more time doing what you suggest in 2024?

“Stop worrying so much about all the possible ranking factors. People spend so much time trying to figure out isolated, tiny things. Changing a title here or a meta description there is not going to make a huge difference. It might make a difference for that page specifically but, in the grand scheme of things, it’s minor.

Move away from the small nitty-gritty stuff. Don’t buy cheap links and don’t buy cheap content. That’s a massive waste of time and money. Instead, zoom out. Try to understand what Google actually wants to rank, because that’s a big question for the future.

This is going to be a very hard realisation for some but, if you look at the current Google SGE implementation, it’s becoming much harder to acquire certain types of organic traffic. In the past, a lot of people relied on churning out generic content, but that’s essentially common knowledge. There’s no reason for Google to rank an article that is just a rephrasing of something that has already been put out there 100,000 times. They will just do that themselves and keep the traffic.

Moving forward, you need to figure out how to become more of an expert and much more well-versed in a certain topic. Just saying, ‘Here’s how to do XYZ’, is not going to cut it in 2024.”

Bastian Grimm is CEO at Peak Ace, and you can find him over at PeakAce.Agency.