Simple, but not easy, we aimed to eliminate low value content, at scale, through Content Pruning and log file analyses.
1. Identified the Pages
Our first step was to crawl the website for a full list of URLs. We used those URLs alongside our client's analytics data to pull in relevant data points to complete step 2.
2. Determined Individual Page Values
To do so, we set a threshold on performance to help evaluate pages, at scale, by section.
The metrics used to determine the SEO value of a page were:
- Organic traffic
- Organic Impressions
- Referring Domains / Backlinks
- Keyword Rankings
3. Defined Criteria for "Minimal SEO Value" Pages
Pages that were at or below the following metric baselines were deemed minimal SEO value pages that we would take on to step 4.
- 50 organic sessions
- 50 organic impressions
- 5 referring domains
- 14 ranking keywords (position 1-100)
We identified 14K URLs that fit the criteria.
4. Evaluated 14K URLs
Not all URLs are created equally. Our evaluation of the minimal SEO value pages produced recommendations for how to handle each page:
- 301 redirect
- Noindex
- 404
- Block via robots.txt
5. Implementation
After that, we began working with the client to implement and QA our team's pruning recommendation. As of the time of this posting, we have a 90% completion rate on handling those 14K pages.
6. Calculated Wasted Crawl Budget Impact
Our team used log file data to observe and pull benchmarks on wasted crawl budget from 14K minimal value content before and after the pruning began.
Within 5 months into pruning implementation: +8% lift in the number of hits to high value site sections.
7. Future Proofing
Last thing on our list was to work with our client to update internal processes on checks and balances to prevent additional low value content from being created in the future
