13ft – A site similar to 12ft.io but is self hosted

Hello everyone, it's the author here. I initially created 13ft as a proof of concept, simply to test whether the idea would work. I never anticipated it would gain this much traction or become as popular as it has. I'm thrilled that so many of you have found it useful, and I'm truly grateful for all the support.

Regarding the limitations of this approach, I'm fully aware that it isn't perfect, and it was never intended to be. It was just a quick experiment to see if the concept was feasible—and it seems that, at least sometimes, it is. Thank you all for the continued support.

Running a server just to set the user agent header to the googlebot one for some requests feels a bit heavyweight.

But perhaps it’s necessary, as it seems Firefox no longer has an about:config option to override the user agent…am I missing it somewhere?

Edit: The about:config option general.useragent.override can be created and will be used for all requests (I just tested). I was confused because that config key doesn’t exist in a fresh install of Firefox. The user agent header string from this repo is: "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

It seems to me that google should not allow a site to serve different content to their bot than they serve to their users. If the content is unavailable to me, it should not be in the search results.

It obviously doesn't seem that way to Google, or to the sites providing the content.

They are doing what works for them without ethical constraints (Google definitely, many content providers, eg NYT). Is it fair game to do what works for you (eg. 13ft)?!

I don't think this will work reliably as other commenters pointed out. A better solution could be to pass the URL through an archiver, such as archive.today:

https://archive.is/20240719082825/https://www.nytimes.com/20...

Missed opportunity to call it 2ft, as in standing on one's own.

has 12ft.io even been working anymore? I feel like the only reliable way now is archive.is

Nice effort, but after one successful NYT session, it fails and treats the access as though it were an end user. But don't take my word for it : try it. One access, succeeds. Two or more ... fails.

The reason is the staff at the NYT appear to be very well versed in the technical tricks people use to gain access.

This is awesome! People who use Kagi can also set up a regex redirect to automatically use this for problematic sites.

I continue my search for a pay wall remover that will work with The Information. I'm honestly impressed that I've never been able to read an Information article in full.

Why not just use uBlock Origin for the aspect of cleaning up the popups / ads and such?

I found this when looking for fun self hosted apps. It's pretty bare bones but does seem to work well with articles I've found so far.

12ft.io doesn't really work anymore.

If you're on iOS + Safari I recommend the "open in internet archive" shortcut, which is actually able to bypass most paywalls.

https://www.reddit.com/r/shortcuts/comments/12fbk8m/ive_crea...

I’m more inclined to use archive(.org|.ph). But this is a decent workaround when archive is unavailable.

Side note: paywalls are annoying but most publications are often available for free via public library.

For example, NYT is free via my public library. PL offers 3-day subs. A few other decent publications are available as well. Availability of publications is YMMV as well.

This could be used as a proxy to web interfaces on the same local network couldn't it?

There are probably much better and more secure options, but this might be an interesting temporary kludge.

> Port 5000 is in use by another program. Either identify and stop that program, or start the server with a different port.

An instruction on how to specify port would be nice.

I am not familiar with 12ft.io, I wanted to try it out, but I get "Internal Server Error" when trying to visit a website.

Bypass Paywalls Clean has moved here btw, https://github.com/bpc-clone?tab=repositories

I'm surprised this works at all.

What sysadmin is so naive to rely on the Googlebot useragent??

Doesn't everyone fetch the official list of Googlebot IPs and then add those to a whitelist?

Does it help when pretending to the google bot to be running on an IP from inside the Google Cloud?

Just a caution you are probably headed for this result if hosted on GitHub (or GitLab)

https://github.com/bpc-clone/bypass-paywalls-clean-filters

https://news.ycombinator.com/item?id=40007670

https://news.ycombinator.com/item?id=40015961

It's always seemed easier to me to use FF + Ublock + Bypass Paywalls. Never fails.

>> This is a simple self hosted server that has a simple but powerful interface to block ads, paywalls, and other nonsense. Specially for sites like medium, new york times which have paid articles that you normally cannot read. Now I do want you to support the creators you benefit from but if you just wanna see one single article and move on with your day then this might be helpful

Personally I'm not a fan of this attitude. I've read and digested the arguments for it, but, for me, it runs close to "theft".

For example, read the sentence again, but in the context of a restaurant. Sure I wanna support the creators, but what if I just want a single meal and then get on with my day?

Businesses, including news web sites, need to monetize their content. There are a variety of ways they do that.

You are free to consume their content or not. You either accept their monetization method as desirable or you do not.

The "I just want to read one article" argument doesn't fly. If the article is so compelling, then follow their rules for accessing it.

Yes, some sites behave badly. So stop visiting them. There is lots of free content on the web that is well presented and lacks corporate malfeasance. Read some of that instead.

I get that I'm gonna get downvoted to oblivion with this post. HN readers are in love with ad blockers and paywall bypasses. But just because you can do something, just because you think it should be "free, no ads", does not make it right.

Creators create. They get to choose how the world sees their creation. Support it, don't support it, that's up to you. Deciding to just take it anyway, on your terms (however righteous you feel you are) is not ok.

Now if someone could just package this into a browser extension it would be great!

The next step being 11ft 8 inches.[1]

[1] http://11foot8.com/

I'll gladly pay for journalist content, but not when a single article is going to be $15/mo and hard to cancel.

Is there some way to support journalism across publications?

It once was Google's requirement that you'd serve the same content to the Google crawler as to any other user. No surprise that Google is full of shit these days.

Counterpoint - if you like the content enough to go through this - just pay for it. Monetary support of journalism or content you like is a great way to encourage more of it.

From my experience, pihole is very easy to setup for this use case: https://pi-hole.net/