SiteAnalyzer - free SEO spider and audit tool

[chaser]

Dear colleagues!

I present to you the SiteAnalyzer software, which is intended for technical analysis and site audit (server response codes, presence of "broken" links, links to external resources, check for availability of title, keywords, description, h1-h6 headers, check for duplicate pages, calculate PageRank, etc). This is a Free analogue of the program Screaming Frog SEO Spider and similar.
The analysis is performed by scanning all pages of the site (crawling the site by links), after which, based on parsed data, you can audit the parameters of interest.

Screens

Key features

• Scanning of all pages of the site, as well as images, scripts and documents
• Getting server response codes for each page of the site (200, 301, 302, 404, 500, 503, etc.)
• Determining the presence and content of Title, Keywords, Description, H1-H6
• Find and display "duplicate" pages, meta tags and headers
• Determining the presence of the attribute rel="canonical" for each page of the site
• Following the directives of the file "robots.txt", the meta tag "robots", or X-Robots-Tag
• Accounting "noindex" and "nofollow" when crawling the pages of the site
• Reference analysis: the definition of internal and external links for any page of the site
• Calculation of internal PageRank for each page of the site
• Site Structure Visualization on the graph
• Determining the number of redirects from the page
• Sitemap "sitemap.xml" generation (with the possibility of splitting into several files)
• Filtering data by any parameter (flexible configuration of filters of any complexity)
• Export reports to CSV and Excel (full report in Excel format)

Differences from analogues

• Low demands on computer resources, low consumption of RAM
• Scanning websites of any volumes due to the low requirements of computer resources
• Portable format (works without installation on a PC or directly from removable media)

I will be glad to any comments and wishes on the work and development of the program.

Web: site-analyzer.pro

PS. The program is distributed for Free.

 

[chaser]

New version SiteAnalyzer 2.0.2

What's new:

• increased crawling speed (growth of about 10%), increased stability of the scanner
• the speed of drawing the nodes of the graph is increased by an order of 10 times (now the graph can easily display up to 100,000 nodes)
• optimized time for the final saving of the project to the database (relevant for large sites)
• optimized display of duplicate headers H1-H6 (duplicate pages removed)
• added the ability to turn off alerts about the release of a new version when the program starts
• added a column indicating the number of incoming links to a page on the PageRank tab
• added ability to rotate the graph and change the scale by an arbitrary value
• fixed not always correct definition of non-canonical pages (Wrong Canonical)
• «Access violation» error occurred when deleting a large number of URLs
• URL disappearance bug fixed when pausing the project / URL list crawl process
• fixed a bug where the Delete button did not work in the Quick filtering field
• minor usability errors fixed, minor bugs fixed

More info: site-analyzer.pro/news/version-2-0-2-speed/

 

[Colin]

Went to try it out but you have to make an account to get a free trail for 14 days.

 

[chaser]

Went to try it out but you have to make an account to get a free trail for 14 days.

You are confusing something, we have no trials, all for free - site-analyzer.pro/buy/

 

[chaser]

New version SiteAnalyzer 2.1

What's new:

• added the ability to export every external link, 404 error, and image in a single report together with all the pages that contain them
• added the ability to manually rescan the URLs that return a response code of 0 (Read Timeout error) in one click from the context menu
• added the ability to hide and display certain columns and tabs in the main data block
• added the "Read Timeout" and "Blocked by Robots.txt" status updates for the URLs that return a status code of 0
• added a display mode for the graph visualization that sets the size of the nodes based on their PageRank
• added an extra button to calculate PageRank on the corresponding tab
• added the ability to rearrange and save the order of columns on tabs
• the active tab is now saved when switching between projects
• added the ability to parse the URLs protected with a .htpasswd file
• added the ability to export the data from the Dashboard tab to Excel
• fixed the AV error caused by switching to the "SERP Snippet" tab with no active project selected
• fixed the incorrect accounting of the inbound and outbound links and their mapping in different blocks of the program
• fixed the incorrect mapping of the percent indicators in the "Level" node
• fixed the incorrect mapping of the "Content-Type" node data in the project scan statistics
• fixed the unstable data filtering for an exact match
• fixed the problem of continuing the paused site scans
• fixed the incorrect HTTP/S sign when exporting reports to Excel
• fixed the incorrect accounting of canonical URLs during the website scanning
• fixed the incorrect accounting of Allow and Disallow statistics for robots.txt
• fixed the incorrect accounting of images ALT and TITLE attributes
• fixed the unstable PageRank calculation
• corrected the typo "Alernate" to "Alternate" on the Info tab

More info: site-analyzer.pro/news/version-2-1-export/

 

[Colin]

You are confusing something, we have no trials, all for free - site-analyzer.pro/buy/

Sorry, I did get it wrong. I ended up going to a different site.

 

[chaser]

We have released a new version of SiteAnalyzer, we'd love to hear feedback and suggestions for the new version.

New version SiteAnalyzer 2.2

What's new:

• we added a custom filters module to search for content on websites during crawling
• we added internal links chart of a website
• when dragging a node of a graph, its child elements are dragged together with it
• when clicking a node of a graph, inbound and outbound links have different colors
• the visualization graph legend is interactive now (when clicking the elements in the legend, the corresponding nodes are highlighted on the graph)
• X-Robots Tag option added to the website crawling settings
• optimized parsing for H1-H6 headers that utilize classes
• software hang in the final stage of large projects crawling is eliminated
• fixed incorrect statistics representation of duplicate meta descriptions
• fixed incorrect statistics representation of 404 pages
• blocked URLs in Robots.txt now return status code 600
• Response Time parameter is calculated more accurately now
• fixed incorrect Sitemap.xml generation
• redirects are displayed more accurate now
• sorting by URLs is more accurate now

More info: site-analyzer.pro/news/version-2-2-custom-search/

 

[chaser]

We have released a new version of SiteAnalyzer: we added the Page Load Performance graph, added a new Links node on the Custom Filters tab, new Content tab and the incorrect display of H1-H6 subheadings has been fixed and more.

New version SiteAnalyzer 2.3

What's new:

• added the Page Load Performance graph, it can be used to analyze the load speed of a website
• added the Content tab, which displays the content distribution stats per symbols and words on each page
• additionally, the links chart is interactive now, it is connected with the Low Number of Backlinks report in the Custom Filters section
• custom Filters tab features a new Links node
• added an Image Sitemap generator
• fixed the bug when scanning of large websites with many redirects caused a program crash
• fixed the bugged Start button in the empty main window. Now it correctly reacts to changes in the URL field
• fixed the bug that caused the incorrect H1-H6 headers display

More info: site-analyzer.pro/news/version-2-3-page-speed/

 

[bracknelson]

Looks nice will give a try, then only I can make any conclusion

 

[pavman123]

To be honest I don't see the point of using that when we have SEM rush and Ahrefs which do the exactly the same thing right?

 

[chaser]

To be honest I don't see the point of using that when we have SEM rush and Ahrefs which do the exactly the same thing right?

SiteAnalyzer makes site audits completely free, so it will be useful for those who do not have paid Ahrefs accounts for analyzing competitors

 

[chaser]

New version SiteAnalyzer 2.4

What's new:

• added data export on Custom Search and Custom Filters tabs
• added data filtering on the Custom Search and Custom Filters tabs
• in Custom Filters section (3xx redirects) we added export of target canonical URLs
• restored ability to scan sites hosted on local servers (localhost)
• fixed incorrect display of actual information about duplicate site pages
• fixed incorrect display of Alt and Title for images
• fixed incorrect accounting of external links, when the URL of the investigated domain is included in the URL of the external link
• fixed bug of displaying incorrect information when switching between projects
• fixed bug of incorrect display of infinite redirects
• fixed bug of freezing progress bar during data export
• fixed bug of freezing the process of opening a project at 90% completion
• fixed not always correct accounting of Robots.txt rules
• fixed not always correct data accounting in the module "Internal Links Chart"
• fixed typo "Content T1ype" in section "Crawl Stats"
• accelerated pause of the scanning process when importing a URL from a file
• added display of links to images from external domains on the "Images" tab

More info: site-analyzer.pro/version-history/

 

[chaser]

SiteAnalyzer 2.5 Released!

In version 2.5 we have added data scraping features, content uniqueness checker, and Google PageSpeed score checker. Plenty of different bugs have been fixed.

What's new:

• added data scraping based on XPath, CSS, XQuery, RegEx
• added a website content uniqueness checker
• added Google PageSpeed score checker
• added the group sites by folder feature
• added the list of projects filter by name
• fixed incorrect processing of URL exceptions
• fixed incorrect processing of website’s crawl depth
• the display of redirects for URLs imported from a file has been restored
• the ability to rearrange and save the order of columns on tabs has been restored
• the processing of non-canonical pages has been restored, fixed an issue with empty meta tags
• the display of anchor links on the Info tab has been restored
• increased the import speed for a large number of URLs from the clipboard
• fixed an issue with incorrect parsing of title and description
• the display of alt and title for images has been restored
• fixed an issue when a freeze occurred when switching to the "External Links" tab while scanning a project
• fixed a program freeze that occurred when switching between projects and updating the nodes of the "Crawl Stats" tab
• fixed incorrect definition of nesting level for URLs with parameters
• fixed data sorting by HTML-hash field in the main table
• the processing of Cyrillic domains has been optimized
• the interface of the program settings has been updated
• the logo design has been updated

More info: site-analyzer.pro/news/version-2-5-scraping/

 

[chaser]

New version SiteAnalyzer 2.6

In version 2.6 we added supports custom HTTP headers and virtual Robots.txt files, the uniqueness check tab now features the list of URLs with the most similar content, fixed more than 30 bugs...

What's new:

• added the ability to use custom HTTP headers when accessing a server
• added the ability to use a virtual robots.txt file instead of the real one (located at the root of the site)
• the uniqueness check tab now features the list of URLs with the most similar content
• added the ability to cancel the proxy list test at any moment after its launch
• rescan of the project’s custom URLs can now be multithreaded according to the settings
• added the SERPRiver service to the Yandex XML settings tab, which provides information about page indexing on Yandex
• fixed the Custom Search feature, used for content search on websites
• added the date of the last scan to the list of projects
• added the ability to move several projects to folders using drag-and-drop or context menu
• added extra buttons to check Google PageSpeed and content uniqueness on the corresponding tabs
• added the ability to open a robots.txt file of a specific site in a web browser
• added the ability to open the specific URL at Web.Archive.org
• added the "Source" column in the "Images" tab
• optimized the work of the exception list for website scan, added the RegEx support
• optimized and enhanced the rule setting for robots.txt files
• fixed an issue that occurred when a robots.txt file had incorrect rules
• fixed incorrect subdomain inclusion when the "Include subdomain" option is active
• fixed incorrect encoding of the HTML code in the data extraction test form
• fixed incorrect sorting of the "TOP domains" tab, as well as other filters of the "Custom Filters" tab
• fixed a bug that occurred when adding unlisted URLs to the filter of projects
• fixed incorrect encoding for sites that use the Windows-1251 encoding
• fixed incorrect data filtering when switching between regular tabs and Custom Filters
• fixed a bug that occurred when scanning a large number of sites in the list of projects
• fixed the detailed decryption of data received from Google PageSpeed
• fixed error statistics display for Title, Description and H1 headings
• the redundant context menu has been hidden in the Custom filters section
• optimized adding a large number of URLs to the list of projects
• optimized the process of URLs removal in projects
• fixed incorrect detection of URL nesting level

More info: site-analyzer.pro/news/version-2-6/

 

[chaser]

We have released a new version of SiteAnalyzer, we'd love to hear feedback and suggestions for the new version.

New version SiteAnalyzer 2.7

In version 2.7 we added personal accounts, White Label reports and a graph node search. H1-H6 duplicates were replaced with H1 duplicates, fixed incorrect accounting of Canonical tags.

What's new:

• personal accounts have been added to the website along with the ability to use authorization keys to remove all the demo version restrictions
• the White Label section has been added to the app’s settings, which can be used to create branded reports
• added nodes search by a URL or a Title header in the visualization graph
• fixed an error that occurred when scanning sites that contain non-Latin characters
• the filtering of H1-H6 Duplicates in the "SEO Statistics" panel has been replaced with H1 Duplicates
• fixed incorrect detection of the HTTP / HTTPS protocol in the "SEO Statistics" panel
• fixed an error when parsing and scraping data led to incorrectly displayed encoding
• an option to "Export data to Excel" has been added to the main menu of the app and the project list menu
• fixed incorrect handling of JavaScript code when finding 301 redirects
• fixed incorrect handling of Canonical tags

More info: https://site-analyzer.pro/news/version-2-7-white-label/

 

[DavidBee]

Malwarebytes is blocking access to the site

 

[botmariner]

To be honest I don't see the point of using that when we have SEM rush and Ahrefs which do the exactly the same thing right?

Good for you.
I also have both, and many others.

What value this comment adds to this thread?

For one thing, there many people just starting out, probably without the budget to purchase a solid crawler.
It would be such a good treat to have it back then myself.

And don't let the Free fool you.
That's a beast with the weight of a feather

Since I spotted it, I'm running 2 some times 3 SAP instances at the same time while I'm working on other tasks. On the same machine (nothing exotic).

Give it a go first...

 

[chaser]

Malwarebytes is blocking access to the site

Yes, I already wrote to thr Malwarebytes technical support, they promised to fix it.
Apparently, as usual, it took longer than they planned.

 

[DavidBee]

Thanks - working now

 

[DavidBee]

Yes, I already wrote to thr Malwarebytes technical support, they promised to fix it.
Apparently, as usual, it took longer than they planned.

Thanks - working ok now