[MozCon 2021] Taking Charge of Your Indexability

@areej_abuali
Taking Charge of Your
Indexability
How to Optimise and Prioritise
Your Technical Work
Areej AbuAli
SEO Consultant
@areej_abuali

@areej_abuali
HELLO!
I’m here to talk to you about
something that took me a while to
wrap my head around...
2

@areej_abuali
But spending the last 18 months
working on a 40M+ indexed page
website forced me to finally get it...
3

“ We are IN CONTROL of how
Google crawls and indexes
our website
4

“ We are IN CONTROL of how Google
crawls and indexes our website
(but it can feel scary!)
5

@areej_abuali
Because if we don’t do it properly,
then we won’t benefit from many*
things we’re doing right
6
*All the awesome content and off page work

@areej_abuali
So if we’re recommending SEO fixes
(small or large), let’s please
drop the 100-page audit
7

@areej_abuali
And focus on prioritising recommendations
that will have an impact
8

@areej_abuali
We work for a
car aggregator
website

@areej_abuali
Our users can...
▸ Search for cars near their location
11

@areej_abuali
▸ Filter the search results by make,
model, year, type, transmission,
engine size, colour, price range (and
more!)
12
Our users can...

@areej_abuali
▸ Filter the search results by make, model, year, type,
transmission, engine size, colour, price range (and
more!)
▸ View car listings and contact sellers
13
Our users can...

@areej_abuali
more!)
▸ View car history information
14
Our users can...

@areej_abuali
more!)
▸ Read the latest car news
15
Our users can...

@areej_abuali
16
Our users can...

@areej_abuali
Most webmasters tend to set up
websites in one of two ways...
17

@areej_abuali
1. Index everything to capture as much
ranking opportunity as possible
18
They either attempt to...

@areej_abuali
2. No-index a huge chunk to avoid
suffering from index bloat
19

@areej_abuali
Index bloat is when a site has an
excessive number of low-value
pages indexed
20

@areej_abuali
21
We might end up indexing:
▸ Every possible geo-location
▸ Every possible filter combination
▸ Every possible car listing page
▸ Every possible car history information page
If we attempt to index everything...

@areej_abuali
Why is this a problem?
22

@areej_abuali
23
Will Google bother crawling…?

@areej_abuali
24
Will Google bother indexing…?

@areej_abuali
25
Do you even want to rank for…?

@areej_abuali
The answer to all three questions is
most probably: No
26

@areej_abuali
27
1. Google might not crawl all our pages
2. Google might not index all our pages
3. Our valuable pages might not be crawled
and/or indexed
4. Our valuable pages might not rank
If we attempt to index everything…

@areej_abuali
I’m being generous by using the word
“might”
28

@areej_abuali
29

@areej_abuali
30
We’d miss out on ranking potential for:
If we no-index a huge chunk...

@areej_abuali
31

@areej_abuali
So, how and where do we
32

@areej_abuali
Before we make any decision on how
to control our indexability, we need
to know the answer to
one question
33

@areej_abuali
What’s the high level KPI that we’re
being measured against?
34

@areej_abuali
1. Organic rank
35
It probably isn’t...

@areej_abuali
1. Organic rank
2. Organic sessions
36

@areej_abuali
1. Organic rank
2. Organic sessions
3. Some third party tool’s visibility chart...
37

@areej_abuali
38

@areej_abuali
It’s most likely
organic leads
or revenue

@areej_abuali
An organic lead on
our car aggregator site
happens when a user
contacts a seller for a
potential car

@areej_abuali
Indexed Pages
41
Organic Leads
So, why don’t we
tie these together?

@areej_abuali
more!)
▸ Read the latest car news
42
Let’s go back to this list...

@areej_abuali
I tend to understand things better when
they’re in table format, so...
43

@areej_abuali
44
Core Templates Description
Car search pages (CSP)
Result pages when a user
searches for cars by location
(includes all
search filters)
Car listing pages (CLP)
Listing pages when a user
chooses a car to learn more about
it
Car history pages (CHP)
Information pages that include car
valuation and history
Other
All other pages (homepage, blog,
about, etc…)

@areej_abuali
We now need to figure out which
templates provide the most value
45

@areej_abuali
1. # of indexed pages (and % from total)
2. # of organic sessions (and % from total)
3. # of organic leads (and % from total)
47
Sessions & Leads date range --> past 12
months
What data do we need per template?

@areej_abuali
48
Template Indexed
Pages
Organic
Session
s
Organic
Leads
CSP 35% 55% 75%
Other 10% 20% 15%
CLP 10% 15% 9%
CHP 45% 10% 1%

@areej_abuali
49
Template Indexed
Pages
Organic
Session
s
Organic
Leads
CSP 35% 55% 75%
Other 10% 20% 15%
CLP 10% 15% 9%
CHP 45% 10% 1%

@areej_abuali
Wouldn’t we be better off spreading our
link equity across more
valuable pages?
50

@areej_abuali
Wouldn’t we be better off having Google
crawl our more valuable pages?
51

@areej_abuali
And most importantly:
Is it worth doubling up our number
of indexed pages for only
1% of organic leads?
52

@areej_abuali
Websites do not
(and most probably should not)
need to have every single page indexed
53

@areej_abuali
Instead, focus on indexing pages that
have the potential to provide
good search results
54

@areej_abuali
Yes, this feels like a fairly straight
forward example...but every
aggregator site tends to have one of these
55

@areej_abuali
56
Template Indexed
Pages
Organic
Session
s
Organic
Leads
CSP 35% 55% 75%
There’s also so much more that can be
dissected on a template by template basis

@areej_abuali
▸ Make
▸ Model
▸ Year
▸ Type
57
▸ Transmission
▸ Engine size
▸ Colour
▸ Price range
Our indexed filters

@areej_abuali
▸ Make (10)
▸ Model (4)
▸ Year (10)
▸ Type (4)
58
▸ Transmission (2)
▸ Engine size (5)
▸ Colour (10)
▸ Price range (5)
Our indexed filters

@areej_abuali
That’s 800,000 potential indexed pages
purely based on our search result filters
59

@areej_abuali
60
Template Indexed
Pages
Organic
Session
s
Organic
Leads
CSP 35% 55% 75%
So it’s fundamental that we break this
down to the next level

@areej_abuali
61
Template Indexed
Pages
Organic
Sessions
Organic
Leads
CSP/Make
CSP/Model
CSP/Year
CSP/Type

“ The more we slice and dice our
data, the more we’ll be able to
make informed decisions.
62

@areej_abuali
A typical meeting in the life
of an SEO...
64

@areej_abuali
Let’s say hello to a senior stakeholder
named Mark
65

@areej_abuali
“Car history information pages are
critical for our consumers. They can
look up any car registration
number and get all the information
they need.”
66
Mark

@areej_abuali
“Yes, but they are useless for Google.
They’re thin pages that provide low
organic value and double up
our indexed pages.”
67
SEO:

@areej_abuali
“Nevermind Google, what does this
mean for LEADS?”
68
Mark:

@areej_abuali
“Well, I’m glad you asked because I
spent the past 2 weeks doing this piece
of analysis where I used BigQuery to
group our page templates, calculate our
organic leads per template, and…. ”
69
SEO:

@areej_abuali
70
“....”
Mark:

@areej_abuali
“1% of our total organic leads. ”
71
SEO:

@areej_abuali
Dream Life Scenario:
“Is that it? De-index these pages
immediately!”
72
Mark:

@areej_abuali
Real Life Scenario:
“Work with Team A to build a strategy
that can convince Team B to convince
Team C to consider convincing Team D
to potentially de-index the pages.”
73
Mark:

@areej_abuali
74
“....”
SEO:

@areej_abuali
Joking aside, I promise you there is a
purpose to this story
75

“ As SEOs, we need to speak the
SAME language as senior
stakeholders to get sign off on
technical fixes.
76

@areej_abuali
78
Template Indexed
Pages
Organic
Session
s
Organic
Leads
CSP 35% 55% 75%
Other 10% 20% 15%
CLP 10% 15% 9%
CHP 45% 10% 1%

@areej_abuali
79
Template Indexed
Pages
Organic
Session
s
Organic
Leads
CHP 45% 10% 1%

@areej_abuali
I know what you’re thinking:
10% of total organic sessions is
a scary thing to simply get rid of...
80

@areej_abuali
I’m very risk-averse but in this specific case:
What good is 10% traffic that doesn’t convert?
81

@areej_abuali
Every aggregator/classified type website
tends to have something similar to a
car history page
82

@areej_abuali
As Mark told us:
These pages are useful for our users
because they want to look up info via
a car registration number
83

@areej_abuali
So to be on the safe side, let’s do a few
checks on our car history pages (CHPs)
84

@areej_abuali
Do they take up lots of crawl budget?
85
Yes, we analysed our logs and in an
average week, 50% of Googlebot’s
crawl requests are spent on CHPs.

@areej_abuali
Do they rank for important terms?
86
No, we analysed our ranking data
and CHPs barely rank for any
keywords we care about.

@areej_abuali
Do they have unique content?
87
Yes, but it’s thin, only useful for
users on a personalised basis
and doesn’t convert.

@areej_abuali
Do they have high quality backlinks?
88
No, they barely have backlinks
pointing at them and none that
are high quality.

@areej_abuali
▸ 45% of total indexed pages
▸ 50% of weekly crawl requests
▸ 10% of total organic sessions
▸ Thin content
▸ Minimal ranking
▸ Minimal backlinks
▸ 1% of total organic leads
89
So, to summarise:

@areej_abuali
Now, do we feel more comfortable
de-indexing them?
90

@areej_abuali
91
Things to remember
Make sure your sitemaps are setup
to automatically remove any
noindex pages

@areej_abuali
This is an example of something that might
seem easy in principle but can be
more complex due to legacy code
92

@areej_abuali
93
Things to remember
Our CHPs won’t de-index overnight,
it could take a few months (or more) for
Google to flush them out of the index

@areej_abuali
95
Things to remember
Blocking CHPs from robots.txt
should be on the roadmap but give it
1-3 months to flush out of the index

@areej_abuali
As an industry, let’s make a pact to drop the
100-page audit documents
97

@areej_abuali
98
Prioritisation is KEY

@areej_abuali
99
I like to use T-shirt Sizing
because it’s simple

@areej_abuali
100
How much of an SEO impact will
this recommendation likely have on
our overall organic leads?
SEO Impact

@areej_abuali
101
How much Technical Effort will this
recommendation take to
implement?
Tech Effort

@areej_abuali
103
For every identified issue:
Prioritisation Matrix

@areej_abuali
105
If you want to learn
more about my full
framework:
bit.ly/areej-matrix

@areej_abuali
So, what’s the first thing that tends to
happen once an SEO recommendation
gets implemented?
107

@areej_abuali
Real Life Scenario:
“Are organic leads up yet?”
108
*Yup, still
Mark!
Senior Stakeholder*:

@areej_abuali
Yes, our senior stakeholder will be expecting
results, and they’ll be expecting it
fast
109

@areej_abuali
▸ Communicate before
▸ Communicate during
▸ Communicate after
110
Manage expectations

@areej_abuali
Under-promise
&
Over-deliver
111

@areej_abuali
De-indexing CHPs is simply a
stepping stone*
for fixing technical foundation
112
*It won’t magically increase our share of
voice

@areej_abuali
Communicating updates
is one thing...
113

@areej_abuali
Communicating updates that are backed
up with data is a game changer
114

@areej_abuali
Indexed Pages
115
Organic Leads
Let’s go back here...

@areej_abuali
▸ Update these metrics on a weekly basis
▸ Show WoW, MoM and YoY change
▸ Make our dashboards accessible & open
116
Keep everyone informed

@areej_abuali
Stakeholders feel better knowing
the information is there
(even if they don’t bother checking it)
117

@areej_abuali
And it’ll serve as a benchmark and
can be used as case studies for
future sign off
118

@areej_abuali
I’ve personally never worked on a car
aggregator website (yet)
This was just an example
120

@areej_abuali
But this methodology can apply to all
forms of aggregator sites:
fashion, jobs, property...
121

@areej_abuali
When I first pitched this talk, my aim was
to go into detail about the
technical SEO requirements
for aggregator sites
122

@areej_abuali
But the actual technical recommendation
is the easy part...
123

@areej_abuali
It’s the backing up our recommendation
with data, the stakeholder management,
the meetings for the sake of meetings…
(those are the hard parts)
124

@areej_abuali
The first time I worked on an aggregator-
type site, I panicked
125

I felt completely out of my depth
and I didn’t know where or how to start

@areej_abuali
But they’re now the type of websites that
I enjoy working on the most...
127

@areej_abuali
Because over time, I’ve come to realise that
they all have similar problems that
require similar solutions
128

@areej_abuali
And once we fix the problems that actually
matter, the impact feels very rewarding
129

@areej_abuali
Yes, rewarding in terms of
SEO impact
130

@areej_abuali
But also in terms of achieving higher level
of trust with stakeholders...
131

@areej_abuali
Which makes the process
(and our life!)
easier the next time round
132

@areej_abuali
The past 132 slides were based on an
example that simply touched
*one* metric
133

@areej_abuali
I tied number of indexed pages
(a technical SEO metric)
to organic leads
(the KPI that matters to the business)
134

@areej_abuali
This same concept can be translated
across far more SEO metrics
135

@areej_abuali
As SEOs, it’s fundamental that we
don’t work in silos
136

@areej_abuali
We can achieve far more if we
measure our wins
using metrics that our business cares about
137

@areej_abuali
And please remember...
138

“ It’s okay to feel overwhelmed
working on something new =)
139

@areej_abuali
bit.ly/areej-mozcon
140

@areej_abuali
141
THANKS!
Questions?
You can find me on:
▸ areejabuali.com
▸ @areej_abuali

[MozCon 2021] Taking Charge of Your Indexability

More Related Content

What's hot

What's hot (20)

Similar to [MozCon 2021] Taking Charge of Your Indexability

Similar to [MozCon 2021] Taking Charge of Your Indexability (20)

Recently uploaded

Recently uploaded (20)

[MozCon 2021] Taking Charge of Your Indexability

Editor's Notes