Organic search marketing has drastically changed over the years, first with a focus on keywords and then a swift transition into conversion optimization. While the user is always the most important element, it is still critical to consider the measures we take for search engines to crawl our site. Before the user even makes it to the website the search engines must have an easy entry into the site.
Prior to even thinking about search engines crawling your site, consider how well your website is communicating with your server. Check data from Google Analytics by going to Behavior>Site Speed>Page Timings and review metrics such as Avg. Redirection Time, Domain Lookup, Avg. Server Connection Time and Avg. Server Response Time.
Consider the Google Search Console as the true starting point for search engines.Go to the Google Search Console>Crawl>Robots.txt and assess how Google understands your robots.txt file. Be sure to run your most important pages to see if you are making any fundamental mistakes. An XML sitemap for pages, images and video should also be submitted to the Google Search Console and Bing Webmasters Tools.
For years, Page Load Time has made a strong impact on SEO. The CSS and Javascript coding should be kept in externally referenced files for call-up. People often forget that they have pages with 404 errors that redirect to another URL. Although a redirect may not be the worst thing, it takes time away from the search engine crawl.
You could unknowingly possess duplicate content on your website. SiteLiner is a great resource for sifting out duplicate content on multiple pages of a domain or content duplicated across sub-domains.