Don’t worry, this isn’t another post about how “everything is changing”. In fact it’s about a problem that digital marketers have been facing for a long time. Forecasts.
It’s also about how ‘fan’ charts (like the one below) can help. You can’t normally make them in Google Sheets but we’ve put together a no-code-needed solution for you to do just that!
Forecasts - the impossibility of predicting the future
Forecasting is a big source of tension in businesses.
On the one hand, in most cases it’s a bad idea for a company to make decisions totally at random. Not many people would suggest changing your marketing spend every month based on a dice roll.
On the other hand, it can take months or years to roll out big business changes. So, to know what a reasonable decision is, we have to know what’s going to happen a long way in the future—we need to forecast.
I know that some marketing specialists will tense up reading that sentence, because it’s impossible to know what will happen in the future, right?
Ignoring global pandemics, recessions, and machine learning revolutions, there's all kinds of changes that can mean the next year for our business will completely change.
Even if we know the next 12 months will be exactly the same as the last, it’s pretty difficult to know exactly what worked last year, and by exactly how much. We end up having to get into that messy realm of attribution. We'll cover that in a later post.
So, business leaders need to make a decision based on something, whether that be a forecast or evidence to prove their decision is justified. Meanwhile, most specialists tend to be uncomfortable with giving a forecast the whole company will work towards because the expertise they use to produce the figure tells them just how flawed that figure can be.
What’s the solution? Refuse to make a prediction? Go back to rolling the dice? Pick a number out of the air?
Not quite.
Reminding ourselves that no one needs it to be exact
If you sit down with a business leader and ask them whether they are expecting a perfect prediction to base their decisions on (down to the dollar/pound), they would probably look at you as if you’re mad. They know these things can’t be exact, but they also know that an informed guess is still probably better than that dice roll.
Of course, it can be easy to forget that we don’t need an exact number when we’re talking about forecasts. If an agency is talking to you about £100K growth, that’s the number you’re going to discuss. You won’t keep saying “plus or minus 10%”.
The real fear that specialists have is that everyone agrees “this forecast isn’t exact” at the time, but at some point (maybe when the CEO looks at your projections, maybe when it’s time to review results) someone forgets that it’s a best guess and starts holding people to account over one number.
We’re not going to talk about avoiding blame culture here (perhaps a good future topic for Paddy Moogan’s ‘The New Leader’). Instead, we’re going to talk about a visual way to remind everyone, every time they look at a forecast, that our prediction falls within a range. Introducing fan charts.
I’m a big fan of fan charts
This is a fan chart which my colleague, Dave, made in Google Sheets. Don't worry about the specific data it’s showing, let’s just think about what it communicates.
When I read this chart, one of the things I think is “we’ve got a pretty clear idea of revenue until about half way through the year, and then it could fall in a range”.
Of course the other thing I think is “Wahey 📈📈📈”, but fan charts work just as well if it’s 📉📉📉.
You might recognise fan charts from tools like Causal Impact or Facebook Prophet, they are both great forecasting and statistical analysis tools we use at Aira.
Example from Casual Impact
Example from Facebook Prophet
To use those predictive models the way we do at Aira, you usually need to be able to code (or you need access to a team like Aira’s Innovations team). But you don’t need code for the visualisation, which I think is a really overlooked part of their value.
Just having the quick, visual cue that the real answer probably falls somewhere within a range can speed up communication massively.
How do I make a fan chart in Google Sheets?
Luckily enough, Dave Westby and I put together a template sheet which you can use right here.
Just paste your data into this table. You don’t have to work in months, it can be in days, weeks, years, whatever. You can add as many rows as you want below the table and they will appear in the chart.
The ‘Revenue (Low)’ column is what you think the worst-case scenario is, ‘Revenue (high)’ is best case. You can rename the columns of course.
If you’re using the fan chart to show a combination of past data and a future prediction, you can use the ‘Past data’ column to mark a row as historical, and that’ll create the orange line in the chart.
If you don’t know what your upper/lower bounds are but you think it could be plus or minus 20%, there’s also a tab where you can just put the number you want to chart, and then write your % margin of error and it will automatically calculate a fan chart for you.
Go forth and use fan charts!
As I mentioned before - there are some really powerful tools like Causal Impact and Facebook Prophet which produce fan charts as a kind of side effect of statistical analysis. The mathematical work that those libraries do is very valuable, but I think the fan chart visualisations are very much overlooked.
Thanks to tools like those, fan charts are an increasingly common way to express uncertainty so that business leaders can understand it and still make decisions.
Even if we only use them to reduce that friction, to get to the important conversations about how we’ll try to achieve the forecast, that’s a huge benefit.
So in summary:
- Forecasting is hard, and it’s basically impossible to get exact numbers
- We still need to base decisions on something, so our best options are to be as accurate as possible, and communicate our limitations
- Fan charts are a huge help in communicating limitations of a forecast
- Use our sample sheet to make your own!
The latest edition of Aira’s innovation update is here! Fortunately, it’s really not hard to come up with things to talk about.
Again, we’ve got a quick summary of some of the stuff we’ve been reading recently, some updates of what we’ve been doing at Aira, and this time some details on some of the ways our Consultants have been using new tools.
What we’ve been reading
To be honest, we’re finding it hard to cut down our reading list for these updates, but here’s a selection of interesting pieces on some different topics:
- Recast did a great quick-read on how companies of different sizes/stages of growth should think about attribution. Honestly, for a Mixed Media Modelling tool company, it’s pretty impressive that Recast dedicates so much time to telling people when they might not need MMM.
- YouGov released a whitepaper about how consumers are responding to the cost-of-living crisis. They cover 13 countries but with a particular focus on the UK and US. Some interesting stats in there but, as with anything survey-based, it’s worth bearing in mind the data can be skewed by who happened to respond.
- Google have created a Machine Learning model that can take an item of clothing and show how it would look on different body types. While this is cool, we imagine that most clothing companies will want more control over the way customers see their clothes than simply relying on Google correctly guessing what designers changed between sizes.
- McKinsey think that AI is going to add 4.4 trillion to the global economy, but only as long as we make sure people are supported as they adapt roles/change jobs (this is not a short read)
What we’ve been doing
- We shared an internal tool we’ve been using to reformat tables to be presentable (this kind of unexciting work is exactly the kind of thing we want to take off consultants’ plates)
- We hosted our first HubSpot User Group in June, with a range of talks on AI. External speaker Claudia Higgins from Conductor spoke about the state of AI in SEO and was followed up by Aira’s very own David Westby who looked at AI, Google and Accountability with a keen focus on the changes being implemented by the search engine and the potential ramifications for businesses. Matthew Purchase, also from Aira, delivered a talk on a brief history of AI and HubSpot, looking at the existing AI features within the platform, the recent acquisitions ,and speculated on where this could go.
- We announced the return of MKGO for this September, with free tickets currently available here. The event will be held at Milton Keynes Gallery, and we've got some great speakers lined up, including Azeem Ahmad, Mehul Garg and our very own Katherine Monkcom, so what are you waiting for? Grab your free ticket before they run out!
What we’ve been thinking - Everyone is innovating
At Aira we believe pretty strongly that innovation shouldn’t just come from the ‘Innovation’ team - they should come from the people working directly on client problems. People who are getting daily exposure to what our clients need. It’s the same reason our Consultants are the ones who wrote up their perspective on how Google’s adoption of AI is likely to change the search landscape.
In this update we wanted to share some stories about how our Consultants are using tools like ChatGPT to supercharge their day-to-day work.
Building two image compression tools using ChatGPT - Mathew Stuckey
I recently did some work for a client which required compressing a large amount of their image catalogue in a bid to improve load times.
After becoming frustrated with the constant back and forth due to the file limits imposed by web-based tools, I decided to look into creating my own image compression tool via ChatGPT which I could run through codepen.io (check it out here!).
Initially, I’d simply asked if it could “code me an image compression tool”, which obviously it told me it could do easily, and then gave me a code script that didn’t function in the slightest.
I spent a lot of time chatting with GPT, simply telling it that the tool didn’t work listening to its apologies and then retrying with the next line of code it gave me. Eventually I gave up.
But the next day, after hitting another file limit on a free tool, I revisited the idea. After some reading up on image compression and GPT prompting, I eventually came up with the following prompt;
“You are a high-end developer with substantial knowledge of coding and image compression techniques.
I would like you to give me the individual HTML, CSS and Javascript code for an image compression tool. The tool should be able to handle both JPEG and WEBP files.
It should allow for multiple files to be uploaded at once up to a limit of 20.
The tool will then compress those files using lossy compression for the webp and jpeg at a rate of 0.3.
The tool should have a progress bar so that the user can see the status of the compression job.
The output should be a list, so that the user can redownload the individual files as they need.
The tool should be styled in a simple, modern way and be easy to navigate for users.”
This gave me a MUCH better looking output which, after copying into Codepen, still didn’t work at all.
But it was far closer.
One of the benefits of using a tool like Codepen is it allowed me to have a back and forth with GPT by just presenting it with any errors that were shown in the console box, it was then able to assess the error and adjust the code accordingly.
In fact, after copy and pasting the single error that was thrown up this time around, GPT was able to debug its own code and promptly provided me with an amended version of the Javascript.
Once this code was swapped over, success! I now had a fully functional compression tool that would allow me to compress client images without any intrusive ads, usage limits or requirements to upgrade to a pro version.
The success of this made me look into other tools I could also create using the power of GPT. So, I also put together a similar WEBP conversion tool which can take multiple image files and convert them all into WEBP format, with the output being an automatically downloaded .zip file of the new images.
Where tools like Convertio only allow for two conversions at a time, I can now transfer an entire image catalogue in one go and convert a client's images to a faster format in one click. This saves a huge amount of time and effort, meaning I have more time to focus on work elsewhere.
Categorising content using ChatGPT - Kathryn Monkcom
Like my colleagues, I’ve been spending some time investigating ways in which ChatGPT can help us do better work, faster. Of course, we don’t always just assume these tools are the best ones for the job and ,in this case, GPT isn’t ready to fly solo, but there’s a couple of things we can do now.
When it comes to inbound strategy, planning concepts for gated content can be one of the more time-consuming tasks. It often comes down to:
- Research what’s already out there
- Come up with some new ideas and choose one
- Test how it performs
- Repeat
A good way to narrow down your options is to try to find examples of high traffic content that not many people have covered. And the best way to do that is to scrape as many websites as possible in your niche, categorise the content and see what’s getting a lot of traffic, but doesn’t seem to be covered in much detail.
I wanted to see if GPT could help us speed up the categorisation step. I found that I had to do a lot of manual cleanup on the output, so I don’t think it’s quite ready to do everything yet. I ended up using GPT as a first pass, followed by some trusty RegEx and then my own manual checking.
Here’s my process to give you a head start so you can come to your own conclusions - or come up with something better!
- I used Ahrefs to scrape my client’s site, and their competitors, getting monthly traffic and a title for each page
- I used GPT for Sheets™ and Docs™ to group the pages into one of: Checklist, Guide, Resource, Ebook, Template, Other. You could also use it to group by topic or theme, i.e. ‘Marketing Automation’ or ‘Strategy’.
- I then used sheet regex formulas over the same page titles to put together flexible categories based on the same terms. This was a great way to sense-check/overrule GPT categories.
- For each group I counted up the traffic, and the total number of pages, to find categories where there was lots of traffic but not that many pages.
By looking at traffic numbers and page count I could quickly see that Checklists were one of the best performing but least used formats. My client had been focusing all their time on guides, which performed well but had a lot of competition. This was great insight that helped us plan our next few months of highly effective content.
As I mentioned, GPT didn’t do a great job of categorising the pages alone. It seemed to rely a lot on exact word match. For now - RegEx is more reliable but I did get some help from our GPT-based “make me a formula” tool to speed that up, so there was still some value to be had there!
The next thing we’re going to try is scraping the sites further to get page content and check them for related words or phrases. With that extra information, GPT may well take the lead!
Prompt engineering to help with SEO tasks - Matthew Kohli
I am a well-known ChatGPT-enthusiast at Aira and I’ve spent quite a lot of time looking for ways to use the tool to improve our work. For example, our copy briefing process.
Aira’s copy briefing process goes something like this;
- SEO Strategist does keyword research, competitor analysis, and client interviews to understand the industry
- SEO Strategist turns that research into a detailed brief (with recommended structure and titles)
- An expert copywriter (normally expert in both copywriting and the client industry) writes the content.
In my experience, GPT can’t do steps 1 or 3. It gets facts wrong, doesn’t have the depth of a good copywriter, and doesn’t care about business impact (not to mention all the industry context it is missing). However, it is pretty great at rewriting or restructuring information, so it can speed up step 2 by a lot.
Here’s the process I use:
First, complete your research (kind of goes without saying) then download “LinkReader” and “Webflow” ChatGPT plugins. Finally, use this prompt structure:
“Please help me create a comprehensive article heading structure for my client X (client URL). The primary keyword and topic for this article is "X"
The client does X
Their main target audience is X
Their TOV is X, see the following example for reference:
Example text, example text, example text etc
Their main competitors are:
Competitor 1
Competitor 2
Competitor 3
Competitor 4
Competitor 5
Their USP vs the competition is X
Supporting keywords to target are:
Keyword 1
Keyword 2
Keyword 3
We want to internally link to the following articles. They do not need their own headers, just provide suggestions for where it would be relevant to link to them:
Internal product/service page URL - this is a relevant product/service that we want to encourage users to engage with
Internal blog URL - this is a supporting article
Internal blog URL - this is a supporting article
Internal blog URL - this is a supporting article
The top 5 ranking pages for the primary keyword X are:
URL 1
URL 2
URL 3
URL 4
URL 5
We want to create something that is comprehensive and engaging to outperform the current articles on page 1. Give me a heading structure in a table format please, with two columns. In the first column on the left I need the heading level (h1,h2,h3) and the heading title together, then in the right column I would like a brief explainer for the writer.
Separate from the heading structure table, please provide an engaging meta description in less than 155 characters, thank you.”
I’ve included an example of using this prompt for Aira, and the results, at the end of this post.
Of course, the final step is to review the results, make sure they line up with your expectations, and to coordinate with your copywriter.
Hope this saves you some time!
Example prompt for “Internal Linking For SEO: Comprehensive Guide” for Aira:
Please help me create a comprehensive article heading structure for my client Aira Digital (https://aira.net/). The primary keyword and topic for this article is "internal linking" and the article title should be “Internal Linking For SEO: Comprehensive Guide”.
The client is a Digital Performance Marketing Agency that specialises in SEO, PPC, Inbound Marketing, Content Marketing, and Account-based Marketing
Their main target audience is companies looking for digital marketing support for their business
Their TOV is friendly and professional, see the following example for reference:
“Anyone can promise you traffic. Aira grows your businesses with multi-channel digital marketing that's measured in sales and revenue.”
Their main competitors are:
https://www.impressiondigital.com/
https://withcandour.co.uk/
https://www.brainlabsdigital.com/
https://www.seoworks.co.uk/
https://www.bluearray.co.uk/
Their USP vs the competition is:: “The best people. Because you need specialists who can communicate as well as they implement. A plan is only as good as the people who execute it. And a poor experience cancels out a great result. So we invest in people who get that. Everyone on your account will know your name, take ownership without ego, and come to you proactively with ideas, updates, and opportunities.”
Supporting keywords to target are:
- Internal linking for seo
- Internal linking best practices
- seo internal linking
- internal link strategy
Provide suggestions for where we should include the internal links under the relevant headings, we want to make it as clear as possible for our copywriters. We want to internally link to the following articles:
- https://aira.net/seo/ - this is a relevant product/service that we want to encourage users to engage with
- https://aira.net/blog/sudden-drop-in-website-traffic/ - this is a supporting article
- Ihttps://aira.net/blog/what-factors-and-attributes-are-most-important-to-consider-in-your-link-building-strategy/ - this is a supporting article
- https://aira.net/blog/how-ai-is-set-to-dramatically-change-performance-marketing/ - this is a supporting article
The top 5 ranking pages for the primary keyword X are:
- https://yoast.com/internal-linking-for-seo-why-and-how/
- https://moz.com/learn/seo/internal-link
- https://backlinko.com/hub/seo/internal-links
- https://www.semrush.com/blog/internal-links-guide-to-building-strategy-that-works/
- https://ahrefs.com/blog/internal-links-for-seo/
We want to create something that is comprehensive and engaging to outperform the current articles on page 1. Give me a heading structure in a table format please, with two columns. In the first column on the left I need the heading level (h1,h2,h3) and the heading title together, then in the right column I would like a brief explainer for the writer.
Separate from the heading structure table, please provide an engaging meta description in less than 155 characters, thank you.
Here’s the output:
Meta Description: "Boost your SEO with our comprehensive guide on internal linking. Learn the basics, best practices, and strategies to improve your site's performance."
Heading level and title | Brief explainer for the writer |
H1: Internal Linking For SEO: Comprehensive Guide | This is the main title of the article. It should introduce the topic and grab the reader's attention. |
H2: Understanding the Basics of Internal Linking | Explain what internal linking is, why it's important, and how it impacts SEO. |
H3: What is Internal Linking? | Define internal linking in a simple, easy-to-understand way. |
H3: The Importance of Internal Linking for SEO | Discuss the benefits of internal linking for SEO, including site navigation, spreading link equity, and improving page authority. |
H2: Internal Linking Best Practices | Provide a detailed guide on how to effectively implement internal linking strategies. |
H3: Using Anchor Text Effectively | Discuss the importance of anchor text in internal linking and provide tips on how to choose the right anchor text. |
H3: Balancing Link Value | Explain how to balance the distribution of link value across the website. |
H3: Creating a User-Friendly Link Structure | Discuss how to create a link structure that enhances user experience and boosts SEO. |
H2: Developing an Effective Internal Link Strategy | Provide a step-by-step guide on how to develop an effective internal link strategy. |
H3: Conducting an Internal Link Audit | Discuss how to conduct an internal link audit and why it's important. |
H3: Identifying Opportunities for Internal Links | Provide tips on how to identify opportunities for internal links within the website. |
H3: Implementing the Internal Link Strategy | Discuss how to implement the internal link strategy, including adding new links and updating existing ones. |
H2: Case Study: Successful Internal Linking in Action | Provide a case study or example of a successful internal linking strategy. This could be a client success story or a well-known example in the industry. |
H2: Common Mistakes to Avoid in Internal Linking | Discuss common mistakes in internal linking and provide tips on how to avoid them. |
H2: Conclusion: The Power of Internal Linking for SEO | Wrap up the article by reinforcing the importance of internal linking for SEO and encouraging the reader to implement the strategies discussed. |
If you have any questions about the topics we've covered in this month's Pulse, don't hesitate to get in touch.
Welcome to the second Marketing Innovation Pulse newsletter. I'm Dave, Senior Data & Insights Consultant at Aira, and part of the innovations team here at Aira.
The main focus of this newsletter is on forecasting. More specifically, SEO forecasting.
With marketing budgets increasingly scrutinised, it’s never been more important to be able to make predictions about what value you’re able to bring (especially as an agency). We’ve spent a lot of time thinking about this… and working on a solution.
This newsletter will give you a bit of an insight into that thinking, as well as what else we’ve been up to.
What we’ve been reading
Unsurprisingly, there’s still a fairly heavy 'Machine Learning' lean this month, but thankfully many of the big tech companies are making moves in one way or another, which means a bit more variety in our reading list!
- We've been reading articles about Perplexity.ai, an interesting ChatGPT alternative, which is meant to be more concerned with factual accuracy and includes embedded links to sources as standard.
- A short thread from Barry Adams about newspapers and zero search volume keywords.
- The announcement about Google Product Studio which could, among other things, help advertisers grab your products out of existing images and put them on a totally different background.
- Canada is preparing to pass a law that makes Facebook pay news sites to distribute their content. In response, Facebook is preparing to stop showing news sites in Canada (starting with a test on 5 June). We’re not sure quite how this will pan out but it’s interesting to watch all the same!
- A fascinating, in-depth article looking at accountability in artificial intelligence. This is likely to be a topic we’ll be hearing and discussing a lot about moving forward.
What we've been doing
- Aira was welcomed as part of the HubSpot Partner Advisory Council.
- We announced an internal tool that generates Google Sheets formulas based on a description of what you want to do.
- After Google I/O, Paddy shared his thoughts on the latest developments in search, and our search teams shared their thoughts about how AI is likely to change search.
- We tried out 'Baby AGI', which claims to take complicated instructions, build a plan and complete the plan based on making multiple OpenAI API calls. Honestly? We found the results underwhelming.
- We also tried out 'Dalai' a Large Language Model (like GPT) that you can run on your laptop. Interesting, but so incredibly slow that it’s not really useful for much other than research, particularly given how cheap GPT is.
What we've been thinking - the key challenges in building forecasts, and how we approach SEO forecasting
With marketing budgets receiving greater scrutiny within a tougher economic landscape (certainly in the UK), businesses are increasingly focused on how efficient they can be with their marketing spending.
When speaking to different organisations, we’ve noticed an increased emphasis on this question:
“If I were to invest x, what would my return be?”
Generally, SEO professionals can be reluctant to forecast precise numbers (and consequently be accountable for hitting those numbers), as there are many external factors that can impact traffic.
These factors include core algorithm updates from Google, the introduction of new SERP features (especially with the AI changes that Google is planning) and other external factors that can all impact the traffic, conversions and ultimately, revenue.
At Aira, though, we understand that forecasts are crucial to making business decisions, so we’ve put a lot of thought into how best to estimate sessions, conversions and revenue.
In this post, we’ll explain some of the main challenges of forecasting and the strategies we use to handle them.
There are three crucial elements of a forecast
- Business relevance
- Adaptability
- Acceptable degree of accuracy (and inaccuracy communicated clearly)
Business relevance
Each business is unique, and as a result, each website is going to be distinct as well, with varying seasonal demand, conversion rates, average order values and timelags between people visiting the site and the business making money.
If we were to compare an ecommerce website selling shoes with a B2B SaaS company specialising in bespoke ERP software, the levels of traffic, conversions and seasonality will differ wildly.
For the ecommerce website, 90%+ of the website pages are likely to be transactionally focussed (i.e category pages or product pages) and the time between someone visiting the website and then converting is likely to be relatively small, with a substantially lower average order value than the SaaS company.
On the other hand, for a B2B SaaS company, the site is likely to have far more of a mix of informational content, alongside some more commercially focused content. Both of these types of content will have different conversion rates and time lags from an initial visit to the site to the business making money.
As a result, the forecast needs to be able to cater to these different businesses and the way in which they drive traffic and make money.
Adaptability
Forecasts need to be adaptable to the different scenarios.
In an ideal world, a forecast would just require a click and away we go, but in reality, there needs to be a lot more thought that goes on in the background.
Let’s say, for example, that you’re forecasting conversions for an ecommerce site and you’re initially going to focus on improving CRO. You’d want to run the forecast with conversion rates set at the current level, but also run the same tests with an improved conversion rate and compare the two.
The key is asking the right questions and being able to cater to the forecast accordingly.
When it comes to forecasting organic sessions, the key levers we can pull are predominantly centred on improving existing content, producing new content and improving the technical state of the site, and it’s useful to be able to map out different scenarios.
Some of the questions we may ask are:
- If we were to prioritise technical health first, what impact is that going to have on existing content?
- Would we be better focused initially on building new content or optimising existing content (i.e where is there greater opportunity)?
- How will different budgets allow us to do different levels of work - and what consequent impact will that have on the bottom line?
Acceptable degree of accuracy (and inaccuracy communicated clearly)
Forecasts are seldom going to be bang on the money, but a forecast should at least be in the right ballpark.
This greater accuracy comes from:
a) Using historical website data and pattern-based forecasting.
b) Making the forecast bespoke for the specific business and the type of website they have, their conversion rates, lags, average order values, and the types of customers they are looking to engage and take action on their site.
An important element of the forecast result is also communicating the acceptable degree of accuracy and inaccuracy, and transparently displaying a margin of error based on upper and lower limits.
Challenges of forecasting
A forecast is not a guarantee. It’s a prediction based on combining business intelligence and historical data in order to make estimations about the future.
Using historical data is central in order to understand seasonal trends and the overall trend of traffic, and be able to map these into the future. Historical data is also crucial in terms of understanding conversion rates, average order values, time to go from first session to conversion, etc (otherwise we’d just be picking them out of thin air).
There are two main types of forecasting methods that can be used which have their own relative benefits and drawbacks:
- Pattern-based forecasting
- Opportunity-based forecasting
Pure pattern-based approach
Pattern-based forecasting involves analysing past trends, patterns and relationships in the data to project future results. Examples of this include Facebook’s Prophet and Richard Fergie’s Forecast Forge.
Example output using Facebook’s Prophet.
Benefits:
- The forecasts take in the context of the current site and as a result, understand seasonal variations throughout the year and more effectively map them into the future.
- The forecasts are grounded in the context of what has been achieved and are therefore able to understand and build forecasts that follow the prior trends - whether that be up or down (or stable).
- If the current projections are down, then the forecast will continue to forecast down. The benefit of this is that the forecast is grounded in reality and won’t always assume an improvement, which is more likely with an opportunity-based forecast. This means that the forecasts are likely to be more representative of the reality of site traffic.
Challenges:
- Pattern-based forecasts rely on historical data, so if historical data doesn’t exist the model is limited. The general advice is that you’d want at least two years in order for the model to really understand seasonality.
- The model can’t predict what it hasn’t seen before. If we’re forecasting organic sessions based on significant technical improvements and an avalanche of new content, the model, in itself, would not be able to factor that in.
- The model can be overly optimistic. If a new site has launched and has seen a continuous uplift in traffic in the first year, for example, the model will assume that this will continue regardless of the Total Overall Market.
- The model assumes that you’re not going to make any drastic changes which, if you’re an SEO agency, doesn’t really fit in with the type of work you’d like or in most cases need to do.
- The model doesn’t involve looking at what customers actually want and doesn’t lend itself to any action - instead, it is more focused on predictions based purely on if the status quo continued.
Pure ‘Opportunity’-based approach
Opportunity-based forecasting focuses on making future predictions based on how much of a total potential market we'd be able to own. At Aira, we have the Keyword Navigator which allows us to see the maximum number of additional sessions we could acquire from new or existing pages, and how many pages it would take to build/optimise.
Example output from looking purely at incremental traffic.
Benefits:
- In contrast to the pattern-based approach, opportunity-based forecasts don’t assume that we’ll keep growing forever. There is an understanding built in that there is a total addressable market, and it’s more a question of how much of that market we could own.
- It’s grounded in the reality of the market we’re operating in, whether there’s a huge opportunity for growth or whether there are more marginal gains.
- The model is centred on the customers and the specific metrics they are looking for. That means that forecasts can be more impacted by specific actions (as opposed to overall trends, which is more common with pattern-based forecasts).
Challenges:
- Unlike pattern-based forecasts, this approach doesn’t take into account historical trends. For example, if website traffic has been falling year on year, this model will not factor that in and will assume incremental growth year on year.
- Seasonality is often ignored, which again, means that many opportunity forecasts just show incremental growth instead of being able to understand that in ‘low months’ when your products are less popular you might actually expect a drop in traffic.
- Opportunity-based forecasts have a tendency to be too conservative compared to pattern-based forecasts where there is no upper ceiling. This is due to the fact there is a defined upper limit that your forecast couldn’t surpass.
- Using opportunity forecasts is useful for saying ‘this is what is in reach if you do everything possible this year,’ but it takes more time to look at forecasting monthly projections based on a limited amount of activity per month.
- Another issue with this approach is that lags are not often factored in and as a result, the assumption is that as soon as you start working, you instantly see an uptick in sessions and conversions. In reality, there is a time delay in doing the initial work, getting it implemented on-site and then seeing the benefits.
Aira’s solution to forecasting organic performance
At Aira, we’ve developed a solution that allows us to develop forecasts that rely on combining the overall potential traffic from new and existing pages with historical projections from previous years.
This solution can be broken down into:
- Stage 1 - Calculating the total organic traffic opportunity, for us using the Keyword Navigator. You can also see an explanation of some of the reasoning we use in my blog post here.
- Stage 2 - Breaking down the Total Opportunity into tasks and calculating how many of those tasks we could get done over time based on a specific budget.
- Stage 3 - Adding lags in order to cater for time for work to be delivered, implemented on-site and start ranking.
- Stage 4 - Using seasonality to upweight/downweight impact over time.
- Stage 5 - Making everything formulas and variables in order to allow flexibility and map different scenarios.
- Stage 6 - Presenting the forecasts using confidence intervals.
We’ll cover the most important stages in the sections below.
Taking overall opportunity and breaking down to individual tasks
Total potential traffic numbers are a brilliant feature of our Keyword Navigator Tool, as we’re able to see:
- The amount of traffic we could attain from building new pages and optimising existing pages.
- How many pages it would require to build/optimise in order to hit that overall traffic number.
Essentially, we estimate where we could rank based on current search results and we calculate how much more traffic (and revenue) opportunity we have, per keyword, using Search Console clickthrough curve, current site rankings and on-site conversion rate.
However, it would be a pretty wild assumption to assume this is the total addressable market and we’ll get everything in a year unless you do the maths.
Breaking down the potential opportunity into specific tasks allows us to ground these traffic estimations into actual work, which further allows us to estimate how much additional traffic we could look to drive month on month.
This also allows consultants to keep accountability and make considerations for other tasks such as quarterly reviews, technical audits, and weekly meetings where the team is not going to work on tasks that will directly increase traffic (though are still important).
This also allows us to communicate with clients about the rate that we’ll be moving and consequently what we would be able to do with a greater budget, or if we were to prioritise some tasks over others.
To break the traffic estimate down into tasks, we look at things like:
- Are we already ranking for this search? Are we already ranking pretty well? If so, is it a matter of simple optimisations?
- Are we ranking for highly related keywords? Check out this post for how we use SERP similarity to cluster keywords.
- What’s the technical health of the website?
Factoring in lags
We’d all love to publish a new page, and ‘BAM!’ immediately get traffic and conversions coming to our site straight off the bat. In reality, though, there are a number of time lags that are going to impact how long it takes to start benefiting from new content.
The most obvious of these is the time it takes to actually start doing the content-related work, such as finalising the keyword research, writing the briefs, writing the copy, reviewing the copy and finally uploading the copy to the site.
Another consideration is the time it takes for content to be crawled, indexed and rank which can vary significantly between different websites depending on the technical state of their site and the type of site they have (i.e news publisher or jobs site is generally going to have a quicker indexation time compared to a small B2B site).
So, in all of our calculations, we figure out when we’d expect the work to be done and then work out when we’d expect that work to have an impact based on what we understand about lags for that business.
We also factor in the average time between initial sessions becoming conversions which again significantly differ between different types of websites (i.e. ecommerce compared to B2B SaaS). These all help us to allow more realistic predictions of how additional traffic and conversions are likely to occur.
Seasonality
One method for understanding seasonality is simply using last year's traffic, but by using machine learning we create a basic trend line for the coming year, handling seasonality and growth, and smoothing out outliers.
If we just took that as our forecast for the year ahead, we’d run into the issues we mentioned above about pure pattern-based approaches. So instead, we figure out the day-by-day trends and use them to upweight/downweight our impact estimates. If weekly and monthly seasonality shows that a day is typically 50% lower than average, we take our impact estimates for that day (based on Step 2) and drop it by 50%. The same approach works for a steady upward/downward trend over time. This offers all the benefits of pattern-based forecasting but is grounded in the reality of the actual opportunity a business has!
One reason this is important is that prioritising work earlier may well pay dividends if the site is in good shape by the time a high month rolls around. This may be through prioritising content-focused or technical work earlier, or even pulling budget forward in order to capitalise on busier periods.
In the example below we can see that Month 8 is when this search starts to drive a lot of the traffic to the site, therefore prioritising work earlier on will mean that once the busy season rolls around, our new content has already been indexed and is more likely to drive sessions and, ultimately, conversions and revenue.
Making everything formulas and variables
As discussed earlier, having a forecasting model which is flexible and adaptable is really important in order to map out different scenarios. Our forecasting model is based on formulas that allow us to do exactly this.
This allows us to answer questions such as:
- What if we improved the conversion rate?
- What if we frontloaded our budget?
- How would our forecast change if the client’s in-house team took control of the content copy production? Or how would it look if we split this 50/50?
- What is the difference in sessions/conversions with different levels of budget?
- What if we focused on technical SEO initially at the cost of content production?
- What would the model look like if we adjusted the confidence intervals?
What we mean here is, there are probably 10 different things a consultant needs to update manually when changing our forecast. Everything else is totally formula based. There’s no manual copy-pasting of data from one tab to another. It was pretty hard to wrap our heads around some of the array formulas we needed to use, though we were able to call upon Aira’s AI Google Sheet formula helper, ‘make me a formula’, at times for inspiration.
Presenting the forecast with confidence intervals
When presenting the results of the forecast, it’s important to clearly communicate:
- The number we’re sharing isn’t the only possible outcome
- What a best case scenario looks like
- What a worst case scenario looks like
Making the upper and lower bounds on the line graph below was a challenge in itself, but suffice to say it’s all created entirely in Google Sheets. We’ll share how we did it in another post!
To wrap it up
The way that we approach SEO forecasts is a blend of historical data and estimates of how much incremental traffic we can bring based on how much of a total addressable market we are able to claim.
There are plenty of elements that play a role in forecasts and are a combination of using the existing data, adding in business intelligence and making decisions, and mapping out different scenarios about the schemes of work.
Do you ever struggle to keep up with the pace of change in the digital marketing and tech space? That's no surprise, because it's rapid - but we've got you covered.
I'm Robin, Associate Director of Strategy and Innovation at Aira, and I've been wanting to share our thoughts on industry developments, new tech and what we've been working on.
We’ve got some interesting reading to share, a tool we’ve been using at Aira, and thoughts on some of the attribution updates in GA4.
What we’ve been reading
There’s a lot of interesting stuff flying around at the moment. As you might expect a lot of it is related to Machine Learning/GPT-type stuff. This is not going to be all-ML all the time, but this time around we’ll just relax into it and share a bunch of things on that topic.
- 'You Are Not a Parrot' - great article from Intelligencer about the difference between humans and tools like Chat GPT
- Some people are bringing together lots of different LLMs (tools like Chat GPT) to perform more complex tasks. They are messy, cobbled together, and pretty imperfect, but Thomas Smith thinks that in that way they are kind of like the human brain
- Some people think that we should stop training AI systems now, so they signed an open letter calling for a halt to all AI training
- The OpenAI CEO confirming they are not working on GPT-5 and responding to that letter
What we’ve been doing
- You may have seen an announcement from Paddy about Aira’s new approach to content distribution. It might seem like a big shift from the outside, particularly for people who have always known Aira’s Digital PR prowess, but it’s actually a continuation of something we’ve been doing internally for quite some time.
- We shared a tool we’ve been using to generate Title Tags and Meta Descriptions to get past the initial “blank page” stage. That Twitter thread also includes some of Aira’s policies on the role of ML tools in our work.
- Our Paid Media team bagged the UK Paid Media award for best use of Facebook and Instagram ads, and Holly Pierce, one of our Paid Media Consultants won a rising star award.
What we’ve been thinking - GA4 changes and attribution
On 6th April, Ginny Marvin (Google Ads liaison) announced that first-click, linear, position-based and time decay attribution models are being removed from Google Ads and GA4.
Unsurprisingly, this set off a fresh wave of consternation from marketers about having to rely on GA4 come July (seriously, if you haven’t got it set up by now, you need to get cracking).
It’s worth bearing in mind a couple things about this.
GA4 comes with auto BigQuery exports
You would have had to pay hundreds of thousands of pounds for this with UA. If you want, you can use that granular data to recreate your favourite attribution models for comparison.
I strongly recommend switching on the BigQuery exports if you haven’t already, if only because it’ll give you ongoing granular data access stretching back more than the 14 month limit that GA4 imposes. Even if you don’t think you’ll personally be digging into historic data that much, if you’re thinking of doing any impact modelling or forecasting in the future using tools like Prophet or Causal Impact, you’ll want 24 months worth of data at least.
As Google’s Ads Liaison points out, you can also use explorations to investigate user journeys
This isn’t quite the same as having a model baked in from the start that you can easily flick to and see how different channels are performing. You’ll have to dig to get your answers and, to be honest, I think that’s part of the point.
Google can’t have their free analytics tool eagerly disagreeing with their own ads
As we’re all keenly aware, attribution is almost never a straightforward conversation. For something so ‘mathsy’ there are a lot of opinions that fly around - and it’s only getting harder with cookie changes. On the one hand, cutting down support for models probably makes life easier for Google engineers/processing costs. On the other hand, it almost definitely reduces the scenarios where PPC teams are having their impact questioned by other teams or senior stakeholders who have been flicking between the models available in GA.
I don’t think it’s a coincidence that these models are being removed from Google Ads and GA4 at the same time. I also don’t think we need to be wearing too big a tinfoil hat to remark on the fact that the three models that are sticking around (Last Click, Ads-Preferred Last Click, and Data Driven) are either the easiest to measure, the most likely to make ads look good, or the hardest to pick apart.
So, maybe you’re thinking: ‘What should I do about attribution now?’
It may well not be the first time you’ve thought this, what with the aforementioned cookie stuff, Safari’s ever-tightening grip on tracking, growing ad blocker usage, GDPR etc. etc.
There’s also companies like Uber saying they turned off 80% of their ads and saw no impact, articles suggesting that attribution-based analytics is a scam, and a growing buzz from big tech companies like Facebook about how we need to stop “attributing” and start “modelling”.
As you can imagine, these questions and answers are very important to us at Aira. We’re not interested in how to make analytics numbers look good - our focus is business impact. So doing a bunch of activity that just takes credit for conversions that would have happened anyway is not the one.
The problem with user-tracking approaches
If you’re using tools like GA or ad pixels; click tracking, cookies, and fingerprinting all rely on following people every step of the way.
They assume that every conversion is due to something they have recorded (or some combination of things). Because they are so myopically focused on individual user journeys they are naturally bad at spotting broader patterns. They are less likely to pick up when conversions would happen anyway. So, they can split the numbers up for you and give you an idea of what’s working, but they can’t really give you an answer to the question “what did this get me that I wouldn’t have got anyway?”
The problem with maths-based approaches
Tools like Causal Impact, Robyn, and Uber Orbit look at historic data to try to gauge what the actual impact of activity is. Essentially - they try to understand the patterns of your business so well that they can say “when you changed x, shortly afterwards sales were higher than we’d expect, so we think x is responsible”. As a result - they need lots of data because of how well they need to understand those patterns to spot when something is unusual.
That means it can take a while to see what the true results of activity are. It’s pretty time-intensive to pull the data and it’s almost impossible to use these tools to inform granular decision making (like whether a headline is a good one to use).
The trick is to use both user-tracking and maths-based approaches, but for different things
Understandably, no one wants to run the risk of investing heavily in something that actually doesn’t have any impact, but at the same time we can’t spend all our time trying in vain to measure the value of small pieces of activity which we’re pretty sure are a good idea anyway.
Often the right answer with small-scale tactical stuff is relatively easy to spot. If we optimise a page and we start ranking better for our target keywords, that’s probably a good thing. If we start getting more traffic and that traffic is converting, that’s probably a good thing.
It’s usually the larger-scale decisions that are harder to validate. Should we be spending on brand advertising? Is top-of-funnel activity bringing us revenue further down the line?
We probably won’t change our attitude to top-of-funnel each week, but we could well be changing our headlines that fast. So, standard user-tracking probably gives us most of what we need for the small scale optimisations. Maybe we don’t have easy access to first-click models in GA any more, but we can probably use things like user signals to get an idea of if the traffic we’re bringing onto the site seems engaged. At the same time - we can try to measure our bigger investments, and strategies with some of the more maths-based approaches. When it comes to making decisions about bigger investments - we’re more likely to have long-term data, and more likely to have the time to do this kind of analysis. Aside from anything else, we’d be talking about a quarterly commitment rather than a daily one!
As ever, the last piece of the puzzle is for us all to accept that whatever measurement we choose it will almost always be wrong. While we can try our best to make it as accurate as possible, the goal is not to understand the ultimate ROI of every single small thing we do. We can put sensible measurements in place to avoid making silly decisions, but we’re always going to be doing a whole bunch of things at the same time anyway. If our whole plan relies on us just finding the one perfect change that will bring us business success, we’re already in trouble!
Thanks for reading
I hope you enjoyed this combined reading-list-and-ramble! We’re going to try to do more of these in future so if you particularly liked (or loathed) any parts of this please do let us know!