Show HN: I made tool that let's you see everything about any website

Checking two websites/domains I'm responsible for, this information is really confusing or just plain wrong. The "DNS Records" card for MX is not the IP addresses of the actual MX records (nor am I sure why it would be -- why wouldn't the MX records be shown here?). "DNS Server" is the addresses of the webservers, not the DNS servers for the domain from whois or from the SOA record. It can show certificate information, but not the cipher suites? Traceroute fails because traceroute isn't available/isn't in path (the error shown is "/bin/sh: line 1: traceroute: command not found"). Firewall seems to be looking specifically for a web-application-firewall, but "firewall" is a somewhat generic term that includes a number of different technologies. Email configuration is wrong, probably because a website is not the same as a domain -- I don't have SPF or DKIM records for the www subdomain, because that's not where we send email from. The "Redirects" card says it followed one redirect, but there is no redirect on the address I provided.

Does this come down to trying to stuff a bunch of stuff for domains into a presentation and information gathering method for websites?

For cases where it can not be determined, it would be best to say "can not be determined" rather than "No", because the last thing anyone needs is some PHB giving people grief because, for example, the WAF in use doesn't expose itself to this detector.

This service is scraping data from somewhere else, it reports us on Amazon and we migrated to gcp a year ago.

I’m a bit confused on the “Threats” section, entering my member management startup https://embolt.app shows malware detected with a timestamp dating back to 2018 (we launched this year).

I checked out another startup I know of (https://highlight.io) and it listed the same results.

Maybe I’m misinterpreting what this section means?

For some reason the Quality check was always failing with an error 403, even though I had followed the link to create a Google API key and passed it as an env var to the Docker container.

Ended up cloning the project to see by myself what URL it uses... turns out that the Google API was returning a JSON document with instructions to enable the PageSpeed Insights API! I'd never used Google Cloud before, so I had been a bit clueless until that point :-)

My suggestion is that the "Show Error" button showed the actual output of the API calls, because otherwise this very useful JSON from Google was being lost in translation.

Now that I checked the code it's clear that there are actually 2 things to enable that are accessed with the API key:

* PageSpeed Insights API: https://console.cloud.google.com/apis/library/pagespeedonlin...

* Safe Browsing API: https://console.cloud.google.com/apis/api/safebrowsing.googl...

So I'd suggest adding this info to either or both of the README and the app itself.

Otherwise, a very very cool project! I've been checking several of my sites for the last hour.

>https://web-check.xyz/check/http%3A%2F%2F127.0.0.1

>City: undefined, undefined, undefined

Heh

The docker version[1] worked better for me to test out. The free website version does not have all the features (like Chromium) enabled which is why some of the report data is missing or incorrect.

Looks like a super promising project! Thanks for building and sharing.

[1] https://hub.docker.com/r/lissy93/web-check

From the first of 3 previous submissions to HN: https://news.ycombinator.com/item?id=36839603

What's the difference between the link in this post (https://v1.web-check.xyz/) and on your Github (https://web-check.xyz/)?

Beautiful! Thanks for open sourcing this!

I've been working on a project [1] that probably wants to become a live crawler like this, but it's currently batch based. I'm focused on RSS feeds and microformats [2]. Can you share any details on what kind of performance / operational costs you're seeing while you're on the HN front page? The fly.toml looks like $5/month could suffice?

[1] https://alexsci.com/rss-blogroll-network/

[2] https://microformats.org/wiki/Main_Page

A correction to the post's title: https://youryoure.com/?apostrophe Should have been lets. Those are two different words with different meanings!

Great site btw

Looks nice, some feedback though:

It shows my dnssec as not present even though https://dnssec-analyzer.verisignlabs.com/ which it links to shows all green for my test site.

The DNS records panel seems a bit broken, it shows my SPF record as the NS ("NS v=spf1 mx -all").

The Server Records panel has a "ports" entry, but that only shows the first open port (for me 22).

When showing Response Time its pretty critical to show where you requested it from. Since you're showing the "location" of the server you could even subtract/show what part of the response time is due to distance latency (or ping the server and use the RTT).

It'd be useful to show things like what protocol is used (http, h2, h3), what cipher was used, etc.

Global Ranking chart should perhaps be inverted? Currently it goes down the more popular the site becomes.

TLS Security Issues & TLS Cipher Suites just send undefined to the tls-observatory site (https://tls-observatory.services.mozilla.com/api/v1/results?...).

HSTS without subdomains shows as "No", there should probably be different levels for "none", "without subdomains", "without preload", "with preload" "in the preload list".

So broken that it’s probably just a tool to collect URLs

I have been using this and I have got to say this is one of the best open source projects at least for me as I need to look up URLs reputation and this is highly helpful in how everything is organized as cards. One screen to get all the helpful information you need. I'm looking forward to the API version and if I could use this as a replacement for VT. I did notice one thing sometimes when you lookup a URL you don't get back any response and when you check network activity tab on a browser you see the requests are getting rejected

Every section has little (i) icons and all of them are useless.

For my site it shows under "Site Features" a "root authority". Okay that's new to me, let's see what that means. The full explanation is: "Checks which core features are present on a site." That's like answering "water" when someone asks "what's water?"

The use cases section of the info is similarly useless and additionally hyperbolic in most instances, such as: "DNSSEC information provides insight into an organization's level of cybersecurity maturity and potential vulnerabilities". If DNSSEC for one domain can tell me about the overall security maturity of an organisation as well as reveal potential vulnerabilities, please enlighten me because that'd be very useful for redteaming assignments

The thing detects January 1st 2008 as the page's content type, which makes no sense (checked with curl, that's indeed incorrect)

Server location is undefined at the top of the page (first impression; the section with the map) but later in the server info section it guesses a random city in the right country

It reports page energy consumption in KWg. Kelvin×Watt×grams, is this a typo for kWh? One kWh is about as much energy as 50 smartphone batteries can hold, as if a page (as measured by its size in bytes) would ever use that amount of energy. You can download many 4k movies on one smartphone charge (also when considering the power consumption of routers), surely that's not the unit being used to judge html weight?

The raw json results, where I was hoping fields might have clearer (technical) labels than the page, remains blank when trying to open it

Overall, I'm not sure what the intended use of this site is. It presents random pieces of information with misleading contextualisation and no technical explanation, some of which show incorrect values and many of which don't work (failing to load or showing error values like undefined). Maybe tackle it in sections, rethinking what the actual goal is here and, once you've identified one, writing that goal into the "use cases" section and implementing it, finally writing in the "what is this" section what it is the site is checking for, then repeat for the next useful piece of information you can come up with, etc.

I have a issue with the website background - on a high refresh rate display with 240Hz, the background animation is incredibly fast and its super distracting.

Seems like there may be some issue with the crawl rules. What is it looking for that would to the error "t.robots is not defined" ?

How do I see how a site is handled in DNS?

For example https://www.whatsmydns.net/#A/www.bispebjerghospital.dk shows that the address is only resolvable from some locations.

I contacted the hostmaster and they admitted they have blocking in the DNS server.

Would be nice to see this also on this site.

If the scheme is not lowercase it seems to erroneously detect malware and provides a zip file url for some malware which does not exist on the page. Seems like a bug !

example URL "with" malware: Https://cnn.com example URL without malware: https://cnn.com

> everything about any website

you're missing subdomains & certs, a very crucial part of investigations imo

Seems like the hostname section detected a different site entirely to the one I input (some site that shared the same IP long ago?), and the mail section failed to detect my (valid, according to gmail) DKIM records entirely...

The UI is slick and presents a lot of info in an easy to parse way, but something is going on with Sentry and FancyBackground.tsx that's causing my laptop fans to spin up while idling on the page.

Neat, bonus points for colorful log messages in console. One thing though: any ideas what is causing horizontal scrollbar to appear in Firefox? I observe this issue on several websites, but never figured out the issue.

It doesn't work very well. I put in my own web address, which is definitely behind cloudfront, and it said it's unprotected, as well as a bunch of other vulns it doesn't have.

The AAAA record listing seems to only display the A record value(s).

Would be nice to be able to compare results between dates.

Maybe I'm misunderstanding but I think there's been a mistake with the "Bad URLs Count", it shows a date instead of what I'd expect (a number)

The tech stack check seems to fail every time. Would love to see with the tech stack details included. Nice and fast, otherwise!

Today I learned : https://securitytxt.org/

It says my domain "grepular.com" doesn't have dnssec. It does. It also says I don't use DKIM or DMARC. I do.

Amazing! Reminds me that I need to learn a bunch of stuff I know nothing about.

This is really neat, kudos!

typing example.com should be fine, I tried www.example.com which also didn't work, it had to be https:/ /www.example.com (I didn't try https:/ /example.com )

Host is reporting some domain I've never heard of.

In the tech-stack it gives me "Chromium not found"...

Hey that was a pleasantly great experience.

I don't have anything to add. Nicely done.

Thanks!

"Energy Usage for Load" is specified in "KWg". What does that mean? Is it a typo for "kWh"?

I enjoyed the UI, cool aesthetic.

man this is beautiful and fast. good job!

Very cool tool!

Very cool

Let’s add a function which lists up all sub domain.