October 4th, 2024

SiteOne Crawler – big tool with no users

SiteOne Crawler is a website analysis tool for developers and SEO specialists, generating audit reports for up to 1,000 URLs, allowing website cloning, and facilitating online report sharing.

Read original articleLink Icon
SiteOne Crawler – big tool with no users

SiteOne Crawler is a versatile website analysis tool designed for developers, QA engineers, SEO specialists, and website owners. It offers features such as a desktop application and command-line interface for easy installation and use. The tool can generate audit reports for up to 1,000 URLs, analyzing various aspects of a website including SEO, performance, accessibility, and security. Users can clone websites for offline use, which is beneficial for archival purposes or disaster recovery plans. The application also includes a feature for uploading HTML audit reports to a secure URL for easy sharing. SiteOne Crawler aims to be user-friendly, catering to both advanced users and those with less technical expertise. It is designed to identify imperfections in websites, assist in CI/CD pipelines, and provide recommendations for optimization. The developers encourage user feedback for future enhancements and aim to expand the tool's functionality based on community needs.

- SiteOne Crawler is suitable for developers, QA engineers, and SEO specialists.

- It generates detailed audit reports for website analysis, limited to 1,000 URLs.

- Users can clone websites for offline access, aiding in archival and disaster recovery.

- The tool features an online report upload option for easy sharing.

- Continuous development is planned based on user feedback and community suggestions.

Link Icon 3 comments
By @janreges - 6 months
Hi andrei-akopian. I am the main author of SiteOne Crawler and thank you for this post!

For about 20 years, I have been leading the development and infrastructure of the Czech webdev company SiteOne, so I wouldn't say that I am not a professional ;) I have around 50,000 hours of practice. But I definitely know a lot of better programmers than me and many of them are at SiteOne :)

However, after the extremely premature birth of my son, I had to change my role at SiteOne, but all the more I wanted to help my colleagues with a useful tool that we generally lacked and at the same time I wanted to give myself joy in difficult times.

I want to implement a number of other useful improvements into the crawler (some of them are also described in the documentation and roadmap) and work on its promoting in parallel. The more people will use it, the more it will help to optimize the website, the more it will help developers and testers with various needs, the more joy it will give me :)

By the way, the GUI application https://github.com/janreges/siteone-crawler-gui is just a visual wrapper over the command-line version of SiteOne Crawler and integrates it into itself.

By @dmje - 6 months
This looks really great and I'll definitely be giving it a go. Like others I really don't understand why it hasn't got more traction (if, when I try it, it does what it says on the tin!). I suspect the comments about the homepage aesthetic probably couple with the fact the dev is a dev and not a marketing person!
By @andrei-akopian - 6 months
For some odd reason, the tool didn't get any replies on Reddit, HN, AlternativeTo.net or GitHub, despite looking fairly sophisticated. The dev is also non-professional.

Maybe I am bad at digging, but I couldn't find anything about siteone.io. The entire situation is strange.

Older discussion with one reply: https://news.ycombinator.com/item?id=41292651