Almost all modern websites are developed with users and browser rendering capabilities in mind. Browsers generally handle even complex tasks without issue.
But when it comes to organic search, all search engines rank web pages as HTML documents. And search bots, unlike modern browsers, have limited capabilities. If you don’t pay attention to how search bots view your pages, you could lose out in the race for top organic search rankings.
We’ll explain further how to check your website’s rendering and fix any errors that hinder ranking.
What is page rendering in SEO?
Rendering is a web development term. It refers to the process of converting a web document’s code into the interactive web page you ultimately see in your browser. Rendering is essentially the process of executing all the rules defined in the HTML code, JavaScript, and CSS styles.
What happens in the browser when displaying a document:
- The browser receives resources from the server (HTML code, CSS, JS, images).
Based on the received HTML code, the application of CSS rules and the execution of JS, the DOM (Document Object Model) is formed. - Styles are loaded and recognized, and the CSSOM (CSS Object Model) is formed.
- A render tree—a set of render objects—is formed based on the DOM and CSSOM. The render tree replicates the DOM structure, but it excludes invisible elements (for example, elements with the display:none; style). Each line of text is represented in the render tree as a separate renderer. Each render object contains a corresponding DOM object (or block of text) and the style calculated for that object. Simply put, the render tree describes the visual representation of the DOM.
- For each render tree element, its position on the page is calculated—layout occurs. Browsers use the flow method, which in most cases requires a single pass to position all elements (for tables, more passes are required).
- Finally, all of the above is drawn in the browser – painting.
Mechanics
Rendering, or the process of displaying a page in Google, begins after crawling but before indexing. The mechanics are quite simple: the crawler requests the page and all the resources needed to render it.
The data is then passed to Rendertron, a headless Chrome page rendering solution built with Puppeteer. There, all the data is processed.
There are two important factors that affect rendering:
1. Google tries to cache all resources and ignores HTTP caching rules. This is because most webmasters create them thoughtlessly, forcing users to repeatedly download the same file, even though it hasn’t changed.
2. Google may not receive all the additional resources, so it’s worth reducing the number of requests required to render the page properly.
When it comes to JavaScript processing, Google clearly states that performance is important to them, so:
- limit the consumption of processor resources;
- may interrupt script execution;
- and they say that excessive CPU consumption negatively impacts indexing.
How to check how search engine robots render your pages
The first method is Google Search Console
This tool helps webmasters get more organic traffic and provides insight into how search engines perceive page content.
You can view the crawled page for any website address that is in the Google index:
After clicking, a window opens on the right side of the screen displaying the bot’s rendered HTML code for the page. This is how the search robot processed the document.
Here you can copy the HTML code of the rendered page and paste it into any code editor. You can then review the code for correctness, checking the presence of all important semantic elements, its position in the DOM tree, microdata, internal links, and more.
You can save the document as HTML and view it visually in any browser. But it’s safer to read and check the code, as modern browsers are very smart and can often automatically correct serious layout errors and close tags.
The second method is if the site is not yours and there is no access to GSC
This method is necessary if we want to compare the rendering of our pages and those of our competitors, to check whether they are returning improved pages with text content to the bot and more functional versions to the user.
Rich Snippets Testing Tool
The tool sends requests to the server just like native Googlebots. The response you receive will be the same as the response the search crawler receives.
A major advantage of this tool is the ability to select the search bot type between desktop and mobile. This is useful when websites use the Dynamic Serving mobile optimization method. This means that the server returns different HTML code depending on the device making the request. Comparing the code of a single page across different agents can reveal, for example, that a desktop server receives pages with text, while a mobile server receives pages without. We’ve also encountered cases where the microdata returned by the bots differed.
After the test is complete, click View Tested Page. This will bring up a window with the page’s rendered HTML code, familiar from the Google Search Console interface.
Mobile-Friendly Test
This is the name of a tool for testing page responsiveness. Unlike the previous tool, this one doesn’t allow you to select the bot that will make the server requests.
Click View Tested Page again and you will be taken to the same window as in the previous tool.
What should you pay attention to when checking the code?
Semantically important zones in content:
- page title in the title tag;
- page descriptions in meta tags;
- correctness of links in link tags;
- schema.org micro-markup elements.
Visually invisible but important layout elements:
- document headings in h tags;
- text descriptions;
- the number of elements presented on the page (for example, product cards in listings);
- image addresses in the src attributes of the img tag;
- Correctness of anchors and addresses in links.
Semantic elements of layout:
- article, section, nav, table tags and their contents are present;
- These tags are correctly placed in the HTML version of the document that the bot indexed.
Comparison of rendered code from different tools
Can we be sure that the page is rendered by the search bot exactly as it appears in the tools? If we’re talking about Google Search Console, it’s more likely to be. However, if we’re talking about other tools, including Rich Snippets Testing Tools and Mobile-Friendly Test, they may not always provide accurate information.
The point is that rendering is a dynamic process. The result we get is the result of a specific response to a specific request at a specific moment in time. Will it match what a search crawler would get when requesting the same resource at a different time? Possibly, but not guaranteed.
As an example, we compared the code stored in Google Search Console with the rendered code of the same page from:
- GSC Live Test;
- Rich Snippets Testing Tool;
- Mobile-Friendly Test;
- Rendered Page Screaming Frog.
GSC vs. GSC Live Test
The code retrieved from GSC Live Test has a difference, such as the addition of a slash at the end of the closing tags. However, the HTML stored in the console doesn’t contain such slashes. Livetest also didn’t detect the GoogleAds script, which is present in the GSC version of the indexed page. The quotation marks are also different. In the indexed version, they look like this: ‘, while in the LiveTest version, they look like this: \”.
GSC vs Rich Snippets Testing Tool
There are a couple of minor differences in the GSC code. For example, the presence of the googleads script, which is absent from the Rich Snippets code, as well as from the Livetest code. However, no more significant differences were found.
GSC vs. Mobile-Friendly Test
The Mobile-Friendly Test summarizes all the differences between the two previous tests. It doesn’t include a script, and slashes are added before closing tags, and quotation marks are replaced.
GSC vs Screaming Frog Rendered Page
Screaming Frog’s page rendering capabilities are superior to Google’s—differences are visible in entire blocks. For example, for Googlebot, this block was display:none and empty, while in Screaming Frog’s rendered version, it was display:block with internal blocks containing content.
What to do if all the tools don’t work?
In reality, this is unlikely. But if such an emergency does occur, we always have our primary working tool at our disposal: the web browser. To check which HTML document the browser received, select the page file in the dev panel’s network tab. Then, in the Preview tab, you’ll see visual differences between the received page and the page after all additional scripts have been executed.
But the Response tab is more interesting and of primary importance to us. Here you can see and parse the HTML document received by the browser.
The arrow in the screenshot indicates the Beautifier button, which will make the code easier to read. If the site doesn’t have a separate Server Side Rendering engine for the bot, it’s important to ensure that all content blocks and semantic elements are visible in this document. Then, regardless of the search engine robots’ rendering resources, we can be sure that the bot has received everything we need.
As a result
In practice, we often encounter situations where in-house specialists can’t understand why a website isn’t growing. The problem is that search engines simply don’t see most of the content on these resources, leading to significant losses in the race for top search results. For this reason, we recommend using GSC to check your website’s page rendering and Rich Snippets Testing Tools to check your competitors’ pages.
For our clients, we simply don’t tolerate improper rendering on their website. For example, when we started working with FlyArystan, their website contained all language versions on a single URL and didn’t render them separately. We fixed this before the website’s release. We also recommend everyone check this early in the launch process.
Source: Promodo.ua
