Screaming Frog Log File Analyser

About

The Screaming Frog SEO Log File Analyser lets you upload your log files, verify search engine and AI bots, identify crawled URLs, and analyse bot behaviour. It is available for macOS, as well as Windows and Linux. You can drag and drop log files directly onto the interface, and the tool processes the data so you can analyse and filter it. It supports Apache and W3C Extended log formats, which cover Apache, IIS, NGINX, and Amazon Elastic Load Balancing.

The app shows which URLs Googlebot and other search bots crawl, when, and how often. It reports crawl frequency, the most and least crawled URLs and directories, and broken links, client and server errors, and temporary and permanent redirects. It also highlights large or slow pages through average bytes and response time, and can match imported URL lists against log data to find uncrawled or orphan pages. You can combine and compare any data with a 'URLs' column, such as crawls, directives, or external link data.

The free version analyses 1,000 log events in one project. A paid licence, priced at £99 per year, removes the log event limit and project limit and includes technical support. Licences last one year, after which renewal is required.