By Andrea VolpiniAndrea Volpini

2 years ago

What is SEO automation? Here is what you can do with the help of artificial intelligence in 2023 to improve the SEO of your website.

SEO automation is the process of using software to optimize a website’s performance programmatically. This article focuses on what you can do with the help of artificial intelligence to improve the SEO of your website. 

Let’s first remove the elephant in the room: SEO is not a solved problem (yet), and while we, as toolmakers, struggle to alleviate the work of web editors on one side while facilitating the job of search engines on the other, SEO automation is still a continually evolving field, and yes, a consistent amount of tasks can be fully automated, but no, the entire SEO workflow is still way too complicated to be entirely automated. There is more to this: Google is a giant AI, and adding AI to our workflow can help us interact at a deeper level with Google’s giant brain. We see this a lot with structured data; the more we publish structured information about our content, the more Google can improve its results and connect with our audience.

  1. An introduction to automatic SEO
  2. Will Artificial Intelligence Solve SEO?
  3. Automating SEO Image Resolution
  4. Automating product description creation
  5. GPT-3 for e-commerce
  6. How Does SEO Automation Work? 

An introduction to automatic SEO 

When it comes to search engine optimization, we are typically overwhelmed by the amount of manual work that we need to do to ensure that our website ranks well in search engines. So, let’s have a closer look at the workflow to see where SEO automation can be a good fit.

  1. Technical SEO: Analysis of the website’s technical factors that impact its rankings, focusing on website speed, UX (Web Vitals), mobile response, and structured data.
    • Automation: Here, automation kicks in well already with the various SEO suites like MOZ, SEMRUSH, and WooRank, website crawling software like ScreamingFrog, Sitebulb, etc., and a growing community of SEO professionals (myself included) using Python and JavaScript that are continually sharing their insights and code. If you are on the geeky side and use Python, my favorite library is advertools by @eliasdabbas ? .
  2. On-Page SEO: Title Tag, Meta Descriptions, and Headings.
    • Automation: Here is where AI/deep learning brings value. We can train language models specifically designed for any content optimization task (i.e., creating meta descriptions or, as shown here by @hamletBatista, title tag optimization). We can also use natural language processing (like we do with WordLift) to improve our pages’ structured data markup ?. 
  3. Off-page SEO: Here, the typical task would be creating and improving backlinks.
    • Automation: Ahrefs backlink checker is probably among the top solutions available for this task. Alternatively, you can write your Python or Javascript script to help you claim old links using the Wayback machine (here is the Python Package that you want to use).
  4. On-site search SEO: the Knowledge Graph is the key to your on-site search optimization.
    • Automation: Here we can create and train a custom Knowledge Graph that makes your on-site search smarter. So, when a user enters a query, the results will be more consistent and respond to the user’s search needs. Also, through Knowledge Graph, you will be able to build landing page-like results pages that include FAQs and related content. In this way, the user will have relevant content, and their search experience will be more satisfying. By answering users’ top search queries and including information relevant to your audience, these pages can be indexed on Google, also increasing organic traffic to your website.
  5. SEO strategy: Traffic pattern analysis, A/B testing, and future predictions.
    • Automation: here also we can use machine learning for time series forecasting. A good starting point is this blog post by @JR Oaks. We can use machine learning models to predict future trends and highlight the topics for which a website is most likely to succeed. Here we would typically see a good fit with Facebook’s library Prophet or Google’s Causal Impact analysis.

Will Artificial Intelligence Solve SEO?

AI effectively can help us across the entire SEO optimization workflow. Some areas are, though, based on my personal experience, more rewarding than others. Still, again – there is no one size fits all and, depending on the characteristics of your website, the success recipe might be different. Here is what I see most rewarding across various verticals.

  • Automating Structured Data Markup
  • Finding new untapped content ideas with the help of AI
  • Automating Content Creation
    • Creating SEO-Driven Article Outlines
    • Crafting good page titles for SEO
    • Improving an existing title by providing a target keyword
    • Generating meta descriptions that work
    • Creating FAQ content on scale
    • Data To Text in German
  • Automating SEO Image Resolution
    • AI-powered Image Upscaler
  • Automating Product Description Creation
    • GPT-3 for e-commerce
    • How to create product description with GPT-3

Automating Structured Data Markup

Structured data is one of these areas in SEO where automation realistically delivers a scalable and measurable impact on your website’s traffic. Google is also focusing more and more on structured data to drive new features on its result pages. Thanks to this, it is getting simpler to drive additional organic traffic and calculate the investment return.

Here is how we can calculate the ROI of structured data.

Here is a concrete example of a website where, by improving the quality of structured data markup (on scale, meaning by updating thousands of blog posts), we could trigger Google’s Top stories, to create a new flow of traffic for a news publisher. 

Finding new untapped content ideas with the help of AI 

There are 3.5 billion searches done every day on Google, and finding the right opportunity is a daunting task that can be alleviated with natural language processing and automation. You can read Hamlet Batista’s blog post on how to classify search intents using Deep Learning or try out Streamsuggest by @DataChaz to get an idea. 

Here at WordLift, we have developed our tool for intent discovery that helps our clients gather ideas using Google’s suggestions. The tool ranks queries by combining search volume, keyword competitiveness, and if you are already using WordLift, your knowledge graph. This comes in handy as it helps you understand if you are already covering that specific topic with your existing content or not. Having existing content on a given topic might help you create a more engaging experience for your readers.

Here is a preview of our new ideas generator – write me to learn more

Automating Content Creation 

Here is where I expect to see the broadest adoption of AI by marketers and content writers worldwide. With a rapidly growing community of enthusiasts, it is evident that AI will be a vital part of content generation. New tools are coming up to make life easier for content writers, and here are a few examples to help you understand how AI can improve your publishing workflow. 

Creating SEO-Driven Article Outlines

We can train autoregressive language models such as GPT-3 that use deep learning to produce human-like text. Creating a full article is possible, but the results might not be what you would expect. Here is an excellent overview by Ben Dickson that demystifies AI in the context of content writing and helps us understand its limitations.  

There is still so much that we can do to help writing be more playful and cost-effective. One of the areas where we’re currently experimenting is content outlining. Writing useful outlines helps us structure our thoughts, dictates our articles’ flow, and is crucial in SEO (a good structure will help readers and search engines understand what you are trying to say). Here is an example of what you can actually do in this area. 

I provide a topic such as “SEO automation” and I get the following outline proposals:

  • What is automation in SEO?
  • How it is used?
  • How it is different from other commonly used SEO techniques?  

You still have to write the best content piece on the Internet to rank, but using a similar approach can help you structure ideas faster.

Crafting good page titles for SEO

Creating a great title for SEO boils down to: 

  • helping you rank for a query (or search intent);
  • entice the user to click through to your page from the search results.

It’s a magical skill that marketers acquire with time and experience. And yes, this is the right task for SEO automation as we can infuse the machine with learning samples by looking at the best titles on our website. Here is one more example of what you can do with this app. Let’s try it out. Here I am adding two topics: SEO automation and AI (quite obviously).

The result is valuable, and most importantly, the model is stochastic, so if we try the same combination of topics multiple times each time, the model generates a new title.

Improving an existing title by providing a target keyword

You can optimize the titles of your existing content by providing a target keyword in order to rank on Google and search engines based on it. For example, suppose I take “SEO automation” as my target keyword for this article and I want to optimize my current content title. Here is the result.

Generating meta descriptions that work

Also, we can unleash deep learning and craft the right snippet for our pages or at least provide the editor with a first draft to start with for meta description. Here is an example of an abstractive summary for this blog post.  

Creating FAQ content on scale 

The creation of FAQ content can be partially automated by analyzing popular questions from Google and Bing and providing a first draft response using deep learning techniques. Here is the answer that I can generate for “Is SEO important in 2021?”

Data To Text in German

By entering a list of attributes, you can generate content in German. For example, in this case, I’m talking about The Technical University of Berlin, and I’ve included a number of attributes that relate to it and this is the result.

DISCLAIMER: Access to the model has been recently opened to anyone via an easy-to-use API and now any SEO can find new creative ways to apply AI to a large number of useful content marketing tasks. Remember to grab your key from OpenAI.

AI Text Generation for SEO: learn how to train your model on a generic data-to-text generation task. 

Automating SEO Image Resolution

Images that accompany the content, whether news articles, recipes, or products, are a strategic element in SEO that is often overlooked.

In multiple formats (1:1, 4:3 and 16:9), large images are needed by Google to present content in carousels, tabs (rich results across multiple devices) and Google Discover. This is done using structured data and following some essential recommendations:

  • Make sure you have at least one image for each piece of content.
  • Make sure the images can be crawled and indexed by Google (sounds obvious but it’s not).
  • Ensure the images represent the tagged content (you don’t want to submit a picture of roast pork for a vegetarian recipe ?).
  • Use a supported file format (here’s a list of Google Images supported file formats).
  • Provide multiple high-resolution images that have a minimum amount of pixels in total (when multiplying one size with the other) of:
    • 50,000 pixels for Products and Recipes
    • 80,000 pixels for News Articles
  • Add the same image in the structured data in the following proportions: 16×9, 4×3, and 1×1.

AI-powered Image Upscaler

With Super-Resolution for Images, you can enlarge in and enhance images from your website using a state-of-the-art deep learning model.

WordLift automatically creates the required version of each image in the structure data markup in the proportions 16×9, 4×3 and 1×1. The only requirement is that the image is on the smaller side by at least 1,200 pixels.

Since this isn’t always possible, I came up with a workflow and show you how it works here.

Use this Colab

1. Setting up the environment

The code is fairly straightforward so I will explain how to use it and a few important details. You can simply run the steps 1 and 2 and start processing your images.

Prior to doing that you might want to choose if to compress the produced images and what level of compression to apply. Since we’re working with PNG and JPG formats we will use the optimize=True argument of PIL to decrease the weight of images after their upscaling. This option is configurable as you might have already in place on your website an extension, a CDN or a plugin that automatically compresses any uploaded image. 

You can choose to disable (the default is set to True) the compression or change the compression level using the form inside the first code block of the Colab (1. Preparing the Environment). 

2. Loading the files

You can upload the files that you would like to optimize by either:

A folder on your local computer

  • A list of comma separated image URLs
  • In both cases you are able to load multiple files and the procedure will keep the original file name so that you can simply push them back on the web server via SFTP.
  • When providing a list of URLs the script will first download all the images in a folder, previously created and called input.

Once all images have been downloaded you can run the run_super_res() function on the following block. The function will first download the model from TF Hub and then will start increasing the resolution of all the images x4. The resulting images will be stored (and compressed if the option for compression has been kept to True) in a folder called output.

Once completed you can zip all the produced files contained in the output folder by executing the following code block. You can also change the name of the zipped file and eventually remove the output folder (in case you want to run it again).

To discover the results achieved in our first test of this AI-powered Image Upscaler, we recommend reading our article on how to use the Super-Resolution technique to enlarge and enhance images from your website to improve structured data markup by using AI and machine learning.

Automating product description creation

AI writing technology has made great strides, especially in recent years, dramatically reducing the time it takes to create good content. But human supervision, refinements, and validations remain crucial to delivering relevant content. The human in the loop is necessary and corresponds to Google’s position on machine learning-generated content, as mentioned by Gary Illyes and reported in our web story on machine-generated content in SEO.Right now our stance on machine-generated content is that if it’s without human supervision, then we don’t want it in search. If someone reviews it before putting it up for the public then it’s fine.

GPT-3 for e-commerce

GPT-3 stands for Generative Pre-trained Transformer. It is an auto-regressive language model that uses deep learning to produce human-like text. OpenAI, an AI research and deployment company, unveiled this technology with 175 billion language parameters. It is the third-generation language prediction model in the GPT-n series and the successor to GPT-2, created by Microsoft-funded OpenAI.

You can use GPT-3 for many uses, including creating product descriptions for your e-commerce store.

How to create product description with GPT-3

GPT-3 can predict which words are most likely to be used next, given an initial suggestion. This allows GPT-3 to produce good sentences and write human-like paragraphs. However, this is not an out-of-the-box solution to perfectly drafting product descriptions for an online store.When it comes to customizing GPT-3’s output, fine-tuning is the way to go. There is no need to train it from scratch. Fine-tuning allows you to customize GPT-3 to fit specific needs. You can read more about customizing GPT-3 (fine-tuning) and learn more about how customization improves accuracy over immediate design (learning a few strokes). Fine-tuning GPT-3 means providing relevant examples to the pre-trained model. These examples are the ideal descriptions that simultaneously describe the product, characterize the brand, and set the desired tone of voice. Only then could companies begin to see real value when using AI-powered applications to generate product descriptions. Examples of the power of GPT-3 for e-commerce product descriptions are in this article, where we show you two different cases: 

  • Using the pre-trained model of GPT-3 without fine-tuning itFine-tuning the pre-trained model with relevant data
  • GPT-3 for product description: learn how to train your model to produce human-like text with machine learning. 

    How Does SEO Automation Work? 

    Here is how you can proceed when approaching SEO automation. It is always about finding the right data, identifying the strategy, and running A/B tests to prove your hypothesis before going live on thousands of web pages. 

    It is also essential to distinguish between:

    • Deterministic output – where I know what to expect and
    • Stochastic output – where the machine might generate a different variation every time, and we will need to keep a final human validation step.   

    I believe that the future of SEO automation and the contribution of machine/deep learning to digital marketing is now. SEOs have been automating their tasks for a while now, but SEO automation tools using AI are just starting to take off and significantly improve traffic and revenues.

    The image we used in this blog post is a series of fantastical objects generated by OpenAI’s DALL·E new model by combining two unrelated ideas (clock and mango).

    Are you ready for the next SEO?
    Try WordLift today!