Polyfill supply chain attack hits 100K+ sites

Game theory at work? Someone needs to maintain legacy code for free that hosts thousands of sites and gets nothing but trouble (pride?) in return. Meanwhile the forces of the world present riches and power in return to turn to the dark side (or maybe just letting your domain lapse and doing something else).

If security means every maintainer of every OSS package you use has to be scrupulous, tireless, and not screw up for life, not sure what to say when this kind of thing happens other than "isn't that the only possible outcome given the system and incentives on a long enough timeline?"

Kind of like the "why is my favorite company monetizing now and using dark patterns?" Well, on an infinite timeline did you think service would remain high quality, free, well supported, and run by tireless, unselfish, unambitious benevolent dictators for the rest of your life? Or was it a foregone question that was only a matter of "when" not "if"?

It's amazing to me that anyone who tried to go to a website, then was redirected to an online sports betting site instead of the site they wanted to go to, would be like "hmm, better do some sports gambling instead, and hey this looks like just the website for me". This sort of thing must work on some percentage of people, but it's disappointing how much of a rube you'd have to be to fall for it.

Important context given by the author of polyfill:

> If your website uses http://polyfill.io, remove it IMMEDIATELY.

I created the polyfill service project but I have never owned the domain name and I have had no influence over its sale. (1)

Although I wonder how the GitHub account ownership was transferred.

(1) https://x.com/triblondon/status/1761852117579427975

> this domain was caught injecting malware on mobile devices via any site that embeds cdn.polyfill.io

I've said it before, and I'll say it again: https://httptoolkit.com/blog/public-cdn-risks/

You can reduce issues like this using subresource intergrity (SRI) but there are still tradeoffs (around privacy & reliability - see article above) and there is a better solution: self-host your dependencies behind a CDN service you control (just bunny/cloudflare/akamai/whatever is fine and cheap).

In a tiny prototyping project, a public CDN is convenient to get started fast, sure, but if you're deploying major websites I would really strong recommend not using public CDNs, never ever ever ever (the World Economic Forum website is affected here, for example! Absolutely ridiculous).

It seems like Cloudflare predicted this back in Feb.

https://blog.cloudflare.com/polyfill-io-now-available-on-cdn...

Washington Post home page external content:

    app.launchdarkly.com
    cdn.brandmetrics.com
    chrt.fm
    clientstream.launchdarkly.com
    events.launchdarkly.com
    fastlane.rubiconproject.com
    fonts.gstatic.com
    g.3gl.net
    grid.bidswitch.net
    hbopenbid.pubmatic.com
    htlb.casalemedia.com
    ib.adnxs.com
    metrics.zeustechnology.com
    pixel.adsafeprotected.com
    podcast.washpostpodcasts.com
    podtrac.com
    redirect.washpostpodcasts.com
    rtb.openx.net
    scripts.webcontentassessor.com
    s.go-mpulse.net
    tlx.3lift.com
    wapo.zeustechnology.com
    www.google.com
    www.gstatic.com

Fox News home page external content:

    3p-geo.yahoo.com
    acdn.adnxs.com
    ads.pubmatic.com
    amp.akamaized.net
    api.foxweather.com
    bat.bing.com
    bidder.criteo.com
    c2shb.pubgw.yahoo.com
    cdn.segment.com
    cdn.taboola.com
    configs.knotch.com
    contributor.google.com
    dpm.demdex.net
    eb2.3lift.com
    eus.rubiconproject.com
    fastlane.rubiconproject.com
    foxnewsplayer-a.akamaihd.net
    frontdoor.knotch.it
    fundingchoicesmessages.google.com
    global.ketchcdn.com
    grid.bidswitch.net
    hbopenbid.pubmatic.com
    htlb.casalemedia.com
    ib.adnxs.com
    js.appboycdn.com
    js-sec.indexww.com
    link.h-cdn.com
    pagead2.googlesyndication.com
    perr.h-cdn.com
    pix.pub
    player.h-cdn.com
    prod.fennec.atp.fox
    prod.idgraph.dt.fox
    prod.pyxis.atp.fox
    rtb.openx.net
    secure-us.imrworldwide.com
    static.chartbeat.com
    static.criteo.net
    s.yimg.com
    sync.springserve.com
    tlx.3lift.com
    u.openx.net
    webcontentassessor.global.ssl.fastly.net
    www.foxbusiness.com
    www.googletagmanager.com
    www.knotch-cdn.com
    zagent20.h-cdn.com

So there's your target list for attacking voters.

I think JS (well, ES6) has a ton of positive qualities, and I think it's a great fit for many of its current applications. However, this is a pretty good example of what bothers me about the way many people use it. I see a lot of folks, in the name of pragmatism, adopt a ton of existing libraries and services so they don't have to think about more complex parts of the problem they're solving. Great! No need to reinvent the wheel. But, people also seem to falsely equate popularity with stability-- if there's a ton of people using it, SURELY someone has vetted it, and is keeping a close eye on it, no? Well, maybe no? It just seems that the bar for what people consider 'infrastructure' is simply too low. While I don't think that, alone, is SO different from other popular interpreted languages, the light weight of JS environments means you need to incorporate a LOT of that stuff to get functionality that might otherwise be part of a standard library or ubiquitous stalwart framework, which dramatically increases the exposure to events like this.

I think a lot of people conflate criticism of JS with criticism of the way it's been used for the past number of years, putting a nuanced topic into a black-and-white "for or against" sort of discussion. I've done a number of projects heavily using JS-- vanilla, with modern frameworks, server-side and in the browser-- and aside from some fundamental annoyances with it's approach to a few things, it's been a great tool. There's nothing fundamental about JS itself that makes applications vulnerable to this like a buffer overflow would, but the way it is used right now seems to make it a lot easier for inexperienced, under-resourced, or even just distracted developers to open up big holes using generally accepted techniques.

But I've never been a full-time front-end-web or node dev, so maybe I'm mistaken? Compared to the server-side stuff I've worked with, there's definitely a concerning wild west vibe moving into modern JS environments. I think there was a casual "well, it's all in client-side userspace" attitude about the way it was used before which effectively shifted most of the security concerns to the browser. We should probably push a little harder to shake that. Think about how much JS your bank uses in their website? I'll bet they didn't write all of their own interaction libraries.

I'm surprised there is no mention of subresource integrity in the article. It's a low effort, high quality mitigation for almost any JS packages hosted by a CDN.

EDIT: Oh, it's because they are selling something. I don't know anything about their offerings, but SRI is made for this and is extremely effective.

I had this conversation countless times with developers: are you really ok if someone hijacks the CDN for the code you're including? They almost always seem to be fine with it, simply because everyone else is doing it like this. At the same time they put up with countless 2FAs in the most mundane places.

The follow up of "you know that the random packages you're including could have malware" is even more hopeless.

I don’t understand why JS devs treat dependencies the way they do. If you are writing scripts that run on other people’s computers I feel like the dependencies used should be vetted for security constantly, and should be used minimally.

Meanwhile most libraries seem to have 80 trillion dependencies written by random github accounts called “xihonghua” or something with no other projects on their account.

I checked, and here are the top domains that are still using Polyfill.io as of today: https://pastila.nl/?00008b47/8a0d821be418cdd5003a2d620d76589...

However, theguardian.com is using it from its own domain, which is safe. But most of the other 5000 websites don't.

Always host your dependencies yourself, it's easy to do & even in the absence of a supply chain attack it helps to protect your users' privacy.

Gotta love the code excerpt verifying the device type that checks for "Mac68K" and "MacPPC" strings. Your retro Macs are safe!

Software supply chains feel like one of the Internet's last remaining high-trust spaces, and I don't think that's going to last long. A tidal wave of this is coming. I'm kind of surprised it's taken this long given how unbelievably soft this underbelly is.

Don't forget that the author of core-js, a library not a service like polyfill but with almost the same goal, is tired of open source as well https://github.com/zloirock/core-js/blob/master/docs/2023-02...

I added a ublock rule on mobile to exclude this domain

||polyfill.io^

Any other practical steps that mobile users can take?

But Microsoft Azure for GitHub ScanningPoint 2024 is SOC2 compliant. How could this happen?

Clientside mitigation: install noscript.

https://addons.mozilla.org/en-US/firefox/addon/noscript/

You can’t expect to remain secure on the modern web while running arbitrary javascript from anyone and everyone.

I really wish we didn't have to trust the websites we go to. This exploit is a good example of the problem. I go to a website and something in the content of the website forces you to go to an entirely different place. I didn't click on a link, it can just do this based on any criteria it wants to apply.

I can’t believe the Financial Times didn’t secure the domain for the project. They backed it for a long time then dropped support of it.

I wonder if the polyfills themselves are compromised, because you can build your poly fill bundles via npm packages that are published by JakeChampion

I am always shocked at the amount of sites that still use remotely hosted dependencies.

Sure, in a true supply chain attack, you wouldn’t be able to trust npm or github or whatever, but atleast you wouldn’t be compromised immediately.

And even outside of security concerns, why would you ever allow someone else to deploy code to your site without testing it first?

I’ll go ahead and make an assumption that the Chinese government was involved. Countries badly need to figure out a way to punish bad actors in cybersecurity realms. It seems that this type of attack, along with many others, are quickly ramping up. If there isn’t competent policy in this area, it could become very dangerous.

MathJax [1] still recommends this way of using it:

  <script src="https://polyfill.io/v3/polyfill.min.js?features=es6"></script>
  <script id="MathJax-script" async src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>

Therefore, if you ever used MathJax, by possibly copying the above and forgetting, make sure to patch it out.

[1] https://www.mathjax.org/#gettingstarted

EDIT: To clarify, patch out just the polyfill (the first line in the snippet). You can of course keep using MathJax, and the second line alone should be enough. (Though still better host a copy by yourself, just in case).

One of these days we're going to learn our lesson and just write our own damned code.

In times of widespread attacks, it would be better to list out actions that affected parties can take. Here is what I found:

- remove it fully (as per original author). It is no more needed - use alternate cdns (from fastly or cloudflare)

Also as a good practice, use SRI (though it wouldn’t have helped in this attack)

I posted a note here: https://cpn.jjude.com/@jjude/statuses/01J195H28FZWJTN7EKT9JW...

Please add any actions that devs and non-devs can take to mitigate this attack.

The phrase "supply chain attack" makes it sound like it's some big, hard to avoid problem. But almost always, it's just developer negligence:

1. Developer allows some organization to inject arbitrary code in the developer's system

2. Organization injects malicious code

3. Developer acts all surprised and calls it an "attack"

Maybe don't trust 3rd parties so much? There's technical means to avoid it.

Calling this situation a supply chain attack is like saying you were victim of a "ethanol consumption attack" when you get drunk from drinking too many beers.

This has been expected since 4 months ago. I wish the past posts (https://news.ycombinator.com/item?id=39517757, https://news.ycombinator.com/item?id=39523523) got more traction and people implemented mitigations on a larger scale before exploitation started.

So what did the malware actually do?

If you don't use version pinning, you are rolling the dice every single build.

So who was the previous owner that sold us all out?

This was obviously written by a JS beginner (ex: basic mistake on the "missing" function, the author thinks that `this` refers to the function?, same about the `undefined` implicit global assignment..)

Since this is effecting end users of websites directly. So instead of waiting for developer to fix it, this could help you stay protected until then

https://medium.com/@wrongsahil/protecting-yourself-from-poly...

> "If you own a website, loading a script implies an incredible relationship of trust with that third party," he Xeeted at the time.

Are people actually calling Tweets "Xeets" now?

When you are selling an open source project, what outcome do you expect? People who are interested in the project non-financially will demonstrate their value in other ways (PRs, reviews, docs, etc) leading to the more common succession of maintainers without exchanging money. I don't think it's reasonable for software authors, who take the route of giving their projects to buyers rather than top contributors, to act surprised when the consequences roll in.

Is this save once more because it moved back to Cloudflare?

    ;; QUESTION SECTION:
    ;cdn.polyfill.io.  IN A
    
    ;; ANSWER SECTION:
    cdn.polyfill.io. 553 IN CNAME cdn.polyfill.io.cdn.cloudflare.net.
    cdn.polyfill.io.cdn.cloudflare.net. 253 IN A 172.67.209.56
    cdn.polyfill.io.cdn.cloudflare.net. 253 IN A 104.21.23.55

Or is Cloudflare warning about this and hosting the attack site?

Is anyone taking the steps to inform affected site owners?

Isn't there some hash in the script tag for these kinds of stuff? Maybe that should be mandatory or something? This broke half the internet anyway.

> However, in February this year, a Chinese company bought the domain and the Github account.

Github accounts of open source software are now for sale?

start projects with

    Content-Security-Policy: default-src 'self';

then add narrow, individually justified exceptions.

> Sansec security forensics team, which on Tuesday claimed Funnull, a Chinese CDN operator that bought the polyfill.io domain and its associated GitHub account in February, has since been using the service in a supply chain attack.

Quite the CDN, eh? :/

They are denying everything on their Twitter @Polyfill_Global lol

https://x.com/Polyfill_Global

I tend to avoid [other people's] dependencies like the plague. Not just for security, but also for performance and Quality. I think I have a grand total of two (2), in all my repos, and they are ones that I can reengineer, if I absolutely need to (and I have looked them over, and basically am OK with them, in their current contexts).

But I use a lot of dependencies; it's just that I've written most of them.

What has been annoying AF, is the inevitable sneer, when I mention that I like to avoid dependencies.

They usually mumble something like "Yeah, but DRY...", or "That's a SOLVED problem!" etc. I don't usually hang around to hear it.

Maybe this will lead to everyone building their own stuffs. A pretty good outcome for SWEs, isn't it?

I rarely use a CDN for that reason; I prefer to have the libraries hosted with the site.

Is there an easy way to go about scanning my domains to see if I'm impacted?

I don't think most people understand that although '100k+ sites are infected', the actual malware code runs on the visitors (our) machines, the servers themselves are fine ! I read the news yesterday and was 'yeah, that's too bad', but only later did it dawn on me that I'm the one that's potentially going to have to deal with the consequences.

This needs to be mitigated client side, not rely on the good will of the administrators.

So glad I removed polyfill as soon as all that nonsense went down a few months ago.

Sadly this is how the Web works.

We need a much more decentralized alternative, that lets static files be served based on content hashes. For now browser extensions are the only way. It’s sad but the Web doesn’t protect clients from servers. Only servers from clients.

Sigstore is doing a lot of interesting work in the code supply chain space. I have my fingers crossed that they find a way to replace the current application code signing racket along the way.

The fact two Fastly employees are involved does immense damage for its brand.

Had it not been for the greed of Fastly's employee, this whole thing could've been avoided.

Name and shame the sports betting site

Is there any evidence that whoever is currently behind polyfill.io is a "Chinese company," as most reports claim?

The company known as "Funnull" appears to be based in the Philippines. The phone number associated with their WhatsApp and WeChat accounts has country code +63. Also they appear to be based at 30th Street, 14th Floor, Net Cube Center, E-Square Zone, Metro Manila, Taguig, Philippines at least if the information on this page [1], purportedly from the founder of the company that was acquired and renamed Funnull, is to be trusted (Click "View Source," then run it through Google Translate)

Claude translation:

===

> Announcement

> I am the former owner of the original Philippine company Anjie CDN. After my incident, the company was managed by my family. Due to their isolation and lack of support, they were persuaded by unscrupulous individuals to sell the company. Subsequently, the company was acquired and renamed as Fangneng CDN, which later developed into the ACB Group.

> This is precisely what I need to clarify: Fangneng CDN company and the ACB Group have no connection with me or my family. Recently, many companies have contacted my family and threatened them, believing that Fangneng CDN company has stolen important information such as member data and financial transactions through client domain names using infiltration and mirroring techniques, and has stolen customer programs through server rental services. This matter is unrelated to me and my family. Please contact Fangneng CDN company to resolve these issues.

> I reiterate: As my family has long since closed Anjie CDN company, any events that occurred afterwards are unrelated to me and my family!

> Note: Due to my personal issues, this statement is being released on my behalf by my family.

> Fangneng CDN's actual office location: 30th Street 14TH Floor, Net Cube Center, E-Square Zone, Metro Manila, Taguig, Philippines.

> Due to the release of this statement, the Anjie domain name has been reactivated by Fangneng CDN company. Currently, Anjie and Fangneng are one company, so I once again declare that any conflicts and disputes arising from Anjie CDN and Fangneng CDN companies are not related to me or my family in any way!

> First publication date of the announcement: May 18, 2022

> Announcement update date: May 18, 2024

[1] Original URL: https://cc.bingj.com/cache.aspx?q=%E5%AE%89%E6%8D%B7%E8%BF%9...

Archive.today version: https://archive.is/uDNoV