Anyone Can Access Deleted and Private Repository Data on GitHub

I reported this on their HackerOne many years ago (2018 it seems) and they said it was working as intended. Conclusion: don't use private forks. Copy the repository instead.

Here is their full response from back then:

> Thanks for the submission! We have reviewed your report and validated your findings. After internally assessing the finding we have determined it is a known low risk issue. We may make this functionality more strict in the future, but don't have anything to announce now. As a result, this is not eligible for reward under the Bug Bounty program.

> GitHub stores the parent repository along with forks in a "repository network". It is a known behavior that objects from one network member are readable via other network members. Blobs and commits are stored together, while refs are stored separately for each fork. This shared storage model is what allows for pull requests between members of the same network. When a repository's visibility changes (Eg. public->private) we remove it from the network to prevent private commits/blobs from being readable via another network member.

Users should never be expected to know these gotchas for a feature called "private", documented or not. It's disappointing to see GitHub calling it a feature instead of a bug, to me it just shows a complete lack of care about security. Privacy features should _always_ have a strict, safe default.

In the meantime I'll be calling "private" repos "unlisted", seems more appropriate

The biggest gotcha here is probably that if you start of with a private repo and a private fork, making the repo public also makes the fork "public".

GitHub may very well say that this is working as intended, but if it truly is then you should be forced to make both the repo and fork public at the same time.

Essentially "Making repo R public will make the following forks public as well 'My Fork', 'Super secret fork', 'Fork that I deleted because it contained the password to my neighbours wifi :P'.

OK. I'm not sure if the last one would actually be public, but I wouldn't be surprised if that was "Working as intended(TM)" - GitHub SecOps

Surprised at the comments minimizing this.

I've used github for a long time, would not have expected these results, and was unnerved by them.

I'd recommend reading the article yourself. It does a good job explaining the vulnerabilities.

IMO, the real vulnerability here is the way the Github Events archive exposes the SHA1 hashes of the vulnerable repositories. It would be easy to trawl the entire network to access these deleted/private repositories, but only because they have a list of them.

Similar (but less concerning) is the ability to use short SHA1 hashes. You'd have to either be targeting a particular repository (for example, one for which a malicious actor can expect users to follow the tutorial and commit API keys or other private data) or be targeting a particular individual with a public repository who you suspect might have linked private repositories. It's not free to guess something like "07f01e", but not hard either.

If these links still worked exactly the same, but (1) you had to guess 07f01e8337c1073d2c45bb12d688170fcd44c637 and (2) there was no events API with which to look up that value, this would be much, much less impactful.

This is not new. Many people have noticed this before, e.g. https://hikari.noyu.me/blog/2020-05-05-github-private-repos-...

Hubber here (same username on github.com). We in GitHub's OSPO have been working on an open source GitHub App to address the use case where organizations want to keep a private mirror of an upstream public fork so they can review code and remove IP/secrets/keys that get committed and squash history before any of those changes are made public. Getting a beta release this week, in fact - check it out, I'm curious what yall think about the approach

https://github.com/github-community-projects/private-mirrors

In fact, there is a process to request complete removal of data, but it involves sending an email that will be reviewed by github staff: https://docs.github.com/en/site-policy/content-removal-polic...

On the other hand, once an API key or password has been published somewhere, you should rotate it anyway.

Can this be used to host illegal content? I.e.: fork a popular repo, commit a pirated book to the fork, delete the fork, use the original repo to access the pirated book?

What would github do after receiving a DMCA request in that case?

There's quite a long list of "open core" companies whose models are, start from a private repository (i.e. company is in stealth), make a private fork that will include for-profit code with enterprise features, make the original repository public so that the core will be open-source.

That GitHub is telling these companies, and bear in mind that these companies are paying customers of GitHub, yeah we don't care that your private proprietary code can be hacked off GitHub by anybody, is incredibly disturbing. Is there really not enough pressure from paying customers to fix this? Is Microsoft just too big to care?

In response to the end of the article "it’s important to note that some of these issues exist on other version control system products." I actually have experience helping someone with an issue on BitBucket with PII data that you can't rotate.

Once we eliminated the references in the tree and all forks (they were all private thankfully), we reached out to BitBucket support, and they were able to garbage collect those commits, and purge them to the point where even knowing the git hashes they were not locatable directly.

> The implication here is that any code committed to a public repository may be accessible forever

That's exactly how you should treat anything made available to the public (and there's no need for the subsequent qualifier that appears in the article—"as long as there is at least one fork of that repository").

Most of this report is just noise. GitHub repos are public. Public stuff can be shared. Public stuff shared previously and then deleted is "still available", but it was shared previously and not really subject to security analysis.

The one thing they seem to be able to show is that commits in private branches show up in the parent repository if you know the SHAs. And that seems like a real vulnerability. But AFAICT it also requires that you know the commit IDs, which is not something you can get via brute forcing the API. You'd have to combine this with a secondary hole (like the ability to generate a git log, or exploiting a tool that lists its commit via ID in its own metadata, etc...).

Not nothing, but not "anyone can access private data on GitHub" as advertised.

I maintain a pretty popular template for SaaS websites. Every few weeks someone would send a PR with all their private fork data, then quickly try to delete it.

Making it a "template" repo mostly fixed the issue. That creates a copy instead of a fork. However it still happens from time to time.

I think the first two points are a result of private data (commit/fork/issue) being able to refer to public data without making the reference public.

Say a private commit depends on a public commit C. Suppose in the public repo, the branch containing C gets deleted and C is no longer reachable from the root. From the public repo's point-of-view, C can be garbage-collected, but GitHub must keep it alive, otherwise the deletion will break the private commit.

It would be "a spooky action at a distance" from the private repo's POV. Since the data was at a time public, the private repo could have just backed up everything. In fact, if that's the case, everyone should always backup everything. GitHub retaining the commit achieves the same effect.

The public repo's owner can't prevent this breakage even if they want to, because there's no way to know the existence of this dependency.

The security issue discussed in the post is a different scenario, where the public repo's owner wants to break the dependency (making the commit no longer accessible). That would put too much of a risk for anyone to depend on any public code.

My mental model is that all commits ever submitted to GitHub will live forever and if it's public at one time, then it will always be publicly accessible via its commit hash.

Really the only semi-interesting part of this is "if you make a private repo public, data from other private forks might be discoverable", but even that seems pretty minor, and the best practice for taking private repos public is to copy the data into a new repo anyway.

Earlier thread: https://news.ycombinator.com/item?id=39481933

I'm not so sure about the "forever" part as git gc is a thing, and at least in 2013 they ran it regularly: https://stackoverflow.com/a/56020315

No idea about nowadays though. There is this blog post:

https://github.blog/engineering/scaling-gits-garbage-collect...

> We have used this idea at GitHub with great success, and now treat garbage collection as a hands-off process from start to finish.

Clearly a POLA violation (principle of least astonishment)

Unrelated, but another interesting one is any non-admin contributors being able to add (and I believe update) secrets in a private repo for use in GH actions. It can’t be done via the UI, but can be done via the API or VSCode extension.

When I looked into it a while back, apparently it is intended behavior, which just seems odd.

>This is such an enormous attack vector for all organizations that use GitHub that we’re introducing a new term: Cross Fork Object Reference (CFOR)

Have we stopped naming vulnerabilities cute and fuzzy names and started inventing class names instead? Does this have a logo? Has this issue been identified anywhere else?

Does any variant of this apply to DMCA’d repos in the repo network?

For example if the root repo is DMCA’d, or, if repo B forks repo A, then B adds some stuff that causes B to get DMCA’d. Can A still access B?

>Commit hashes can be brute forced through GitHub’s UI, particularly because the git protocol permits the use of short SHA-1 values when referencing a commit. A short SHA-1 value is the minimum number of characters required to avoid a collision with another commit hash, with an absolute minimum of 4. The keyspace of all 4 character SHA-1 values is 65,536 (16^4). Brute forcing all possible values can be achieved relatively easily.

>But what’s more interesting; GitHub exposes a public events API endpoint. You can also query for commit hashes in the events archive which is managed by a 3rd party, and saves all GitHub events for the past decade outside of GitHub, even after the repos get deleted.

Oof

ISTM there’s a straightforward mitigation or two available to GitHub:

1. If a URL would be in the “[t]his commit does not belong to any branch of this repository, and may belong to a fork outside of the repository” and that URL uses a shortened commit hash, return 404 instead. Assuming no information leakage via timing, this would make semi-brute-force probing via short hashes much harder.

GitHub is clearly already doing the hard work for this.

2. A commit that was never public should not become public unless it is referenced in a public repository.

This would require storing more state.

How is this more of a vulnerability than the existence of sites like archive.org is? Isn't it just a fact of the Internet that once you make something public, you can't fully take it back later?

The title makes it seem more severe than it is. This only applies to GH forks of public repositories (or repositories that become public). Forks mirror the upstream repo's visibility.

I don’t use GitHub for anything serious, rather my own Gitea. However:

> Any commits made to your private fork after you make the “upstream” repository public are not viewable.

Does that mean a private repo that has never been or will be public isn’t accessible? That scenario wasn’t mentioned.

even better you can actually commit to other forks if they creates pull request to you.

(there is checkbox allowing that when you are opening PR that I bet almost noone noticed)

I reported that years ago and all they changed it that they extended documentation about this "feature"

my main issue was that you cannot easily revoke this access because target repo can always reopen PR and regain write access.

but they basically "stated works as intended"

The only valid one is the last (3rd) one:

Accessing commits on a private fork when it's upstream is made public

The other 2 are just common sense... push something to a public repo and it's public forever. Everyone knows once somethings on the internet it's already too late to make it secret again.

Key takeaways for me:

1) Never store secrets in any repo ever! As soon as you discover that its happened, rotate the key/credential/secret asap!!

2) Enterprises that rely on forking so that devs can colab are fucked! Protecting IP by way of private repos is now essentially broken on GH!

3) what the actual fuck github!!??

Truffle is practically famous for clickbait like this. They have a YouTube channel full of it. Their behavior in the security industry steered us far away from them as a vendor.

So does that mean that forked repos don't do garbage collection of unreferenced commits?

If I force push and orphan a commit, I expect that will get garbage collected and be gone forever.

Or if I commit a file I shouldn't have and rewrite my repo history and push up a whole new history, is the old history still hanging out forever?

If true, then it seems that there is no way to delete any commits at all from any repo that has any forks?

How much help is turning off the "Allow forking" option https://docs.github.com/en/repositories/managing-your-reposi... ?

Doesn't this kind of make sense? We are not dealing with personal property. We are dealing with term licensed software.

Github is a software distributing network. Like the app store, or Steam. They grant you access to licensed content, which you self license, and then they facilitate access for you. Based on the honor system. But some things can just be assumed to be true for the sake of simplicity and liability.

For example, If I make a repo public and then take it private the hashes that were obtained while it was open are still open. If I make a repo that's closed and open it, the whole thing is open.

If you fork a public repo and make private commits on it to a software distributor like Github, that is probably just going to end in a violation of the license. In this scenario, Github is saving you from yourself.

I found some obscure instances where user expectation doesn't match reality on GitHub before, and nobody there cares.

If anyone's wondering: Organizations that require SAML are included in your organizations even when you don't have a SAML session when signing in elsewhere via OAuth. Unlike generalized per-organization app authorizations, where GitHub can actually hide organization membership. Only way to find out if a user has a SAML session is for the consuming app to request the membership with your token, and interpret 403 as "no SAML session". As far as I know only Tailscale implemented this. This really sucks for apps like SonarCloud where someone can now view work code from their so cleanly separated personal and professional use GitHub account.

On the positive side this takes care of all those companies forking open source software and not contributing back

To fork private, I always just make a new repo and push to it. Looks like that behaves correctly here.

I actually think this is a good thing and should simply be made more clear. The reason is the following from the article:

> I submitted a P1 vulnerability to a major tech company showing they accidentally committed a private key ... They immediately deleted the repository,

That is a ridiculous response to a compromised key. The repository should not have been "deleted", the key should have been revoked.

Imagine if you lost a bag with 100 keys to your house. Upon realising you desperately try to search for the bag only to find it's been opened and the keys spread around. You comb through the grass and forests nearby collecting keys and hoping you find them all.

Or you just change the locks and forget about it.

If you upload something, anything, to a computer system you do not own you need to consider it no longer secret. It's as simple as that. Don't like it? Don't do it.

I detest things like delete buttons in messaging apps and, even worse, email recall in Outhouse-style email apps. They just give people a false sense of security. I've been accidentally sent someone's password several times on Teams. Yeah you deleted the message, but my memory is very good and, trust me, I still know your password.

If there's a security problem here it's in people believing you can delete stuff from someone else's system, or that that systems make it look like you can. The solution is the same though: education. Don't blame GitHub. Don't force them to "fix" this. That will only make it worse because there are still a million other places people will upload stuff and also won't actually delete stuff.

I reported a similar and even more damaging I my opinion (https://hackerone.com/reports/2240374) and they also dismissed as by design.

Turns out I found out you could even invite external collaborators into your fork and totally bypass enforced SSO.

Even if you block forking into your main repo, the existing forks remains active and still can pull from upstream.

It feels like if you need proper security, you have to go with enterprise

This behaviour is also important for ergonomic submodules. The .gitmodules file lists the upstream repo as the origin. So, if you're modifying an upstream project in a submodule and push changes to a fork, it's important that the SHA that git tracks is still reachable through the upstream link.

Ultimately I don't think it's feasible to break this behaviour and the most we can hope for is a big red warning when something counterintuitive happens.

I cannot access the commit https://github.com/trufflesecurity/trufflehog/commit/7bc0b shown in one of the pictures in the article (right before "Where do you get these hash values") despite this repo is even public.

What gives?

People should realize that once you upload something, it will be out there, forever. I assume this happens to everything.

Trusting some company will actually delete your stuff is kind of naive in my opinion.

The example of people forking and putting an API key in the repo, I would never let my people do this. Once you push, it will be "out there".

There is a reason that anyone who cares about forks being private forever, (even if you delete it) should never use or trust a third party. I never use Github. I run my own git server and everyone else should to in my opinion. Github has always been a huge security problem.

But that's just me...

I wonder if copyleft projects can use this to find license violations and force the altered code into the open.

A serious security issue indeed, if someone knows the hash.

How I manage this is that every time I want to open-source a previously private feature, I take the changeset diff and apply that to the files in the public repository. Same features, but plausibly different hash.

Are Radicle private repos safe in this respect? https://radicle.xyz/guides/protocol#private-repositories

So... this is only an issue with forking, right? And, forking is not the same thing as branching... right? I'm just trying to make sure I understand this since I do branching all the time, but have never forked anything.

So the moment something is published on the Internet publicly, there's a chance it will be saved and you will not be able to get it deleted.

That, unfortunately, sounds like the result of publishing something on the Internet. Not GitHubs fault.

People are so preoccupied with putting the code on GitHub. It’s like it doesn’t exist before it’s on GitHub.

If you’re not gonna share it then it hardly matters. Use a backup drive.

Git is distributed. You don’t have to put your dotfiles on GitHub. Local is enough.

Should GitHub be liable for any damages caused by this issue, like some think Crowdsec should be for what happened last week?

Morally seems even worse, Crowdsec did it by accident, GitHub knows about it for years now.

The few times I made private copy public I made a brand new git repo, copied the working copy over, and published that as public. I'd never include past private git history when making something public.

Great website design that loads fine without scripts but then runs something that requires features found only in newer browsers and then deletes the entire content when that fails. Why?

commits done to private repose being public (point 2&3) is always a non minor security vulnerability IMHO

it doesn't matter if it's behaving as intended or how there are forks

also point 1 implies that github likely doesn't properly GCes there git which could have all kinds of problematic implications beyond the point 1 wrt. purging accidental leaked secrets or PI....

all in all it just shows github might not take privacy security serious ... which is kinda hilarious given that private repo using customers tend to be the paying customers

My mother can't. And she doesn't mind.

Question: but for deleted you just mean commit to delete it? Because if you remove the commit from the repo, it should disappear.

>forks copyleft project >tries to make it private, violating license >fork made public

Sounds like a win for foss

I always acted as if there were no such thing as private data on github. Maybe even the internet as a whole.

A “delete” means it should be gone forever from the service it was removed from.

“Private” means it should only be available to specific involved parties only.

If you implement any other behavior to these concepts you are implementing anti patterns.

We need to be precise and consistent in the wording of the functions we are providing in order to ensure we easily can understand what is going on, without having to interpret documentation to be able to fully understand what is going on.

So if I read the article correctly if I never fork or otherwise contribute from my private repo I’m good?

This is why for private and business projects, we don't use GitHub, we use Amazon CodeCommit.

I wonder how all of the companies using "private" repos on GitHub feel about this.

1 more reason to use GitLab <3

This walks like a dark pattern and quacks like a dark pattern. People's entire livelihoods are at stake and they don't care. Most likely because plausible deniability and obscure TOS rights of how and when the code is used is more valuable to them than the reputation hit. It is hard to imagine this is very hard to fix.

Known "issue" since day 1. It's a feature.

Wow, that's crazy. I tried a 6 digit hash and got a 404, then I tried another 6 digit hash and got "This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository."

Insane

I learned about it years ago when I accidentally pushed secrets to the repo. When after rebasing and force pushing to the branch I was still able to access that commit, we decided to stop using GitHub.

I'm surprised that nobody suggested self hosting a GitLab or Gitea instance. [1]

[1] https://ipv6.rs/cloudseeder

Microsoft: It's the EUs fault!

Also Microsoft: It's a feature!

all your private photos on gdrive have publically accessable urls too. most ppl dont know all their private photos are exposed to the world.

I won't be surprised if "right to be forgotten"/GDPR abusers will spam github and force them to act on it, eventually.

----

This is clearly documented and can be explained even to non-technical managers.

From my POV calling that vulnerability is trying to build a hype.

I think that having quote from here on visibility changing settings page would be even more clear: https://docs.github.com/en/pull-requests/collaborating-with-...

They have the yellow banner to detect when you likely access a hash like this. Why do they allow those commit hashes to be accessed through the short commit hashes?

Commit hashes are essentially capabilities, you should be able to access any data that you have a capability for. But allowing access via a 16bit prefix is just idiotic, and equivalent to accepting just the first two bytes of a 256bit cryptographic signature...

Just rebase/squash everything.

Cool, another way to access youtube-dl next time it gets deleted from GitHub.

I’m glad I don’t use forks

Tangential to the article but interested in seeing how Microsoft will fare compared to Tesla

Thank you for relaying this. I'll be moving off Github this weekend.

> 1) Fork the repo. 2) Hard-code an API key into an example file. 3) <Do Work> 4) Delete the fork.

... yeah if <Do Work> is push your keys to GitHub.

Holy shit. What a joke of a company.

and people have been yelling at me for refusing to ever use github since 2013 lolo

and people yell at me for not using github ever

Wow. This is wild!

Come on, this is not surprising.

"Private repositories" were never private as I said before. [0]

[0] https://news.ycombinator.com/item?id=23057769

Data that you place with an entity that is a large organization with many commercial and government ties - must be assumed to be accessible to some of those parties.

And if that entity has a complex system of storage and retrieval of data by and for many users, that changes frequently, without public scrutiny - it should be assumed that data breaches are likely to occur.

So I don't see it as very problematic that GitHub's private repositories, or deleted repositories, are only kind-sorta-sometimes private and deleted.

And it's silly that the article refers to one creating an "internal version" of a repository - on GitHub....

Still, interesting to know about the network-of-repositories concept.

This isn't a bug IMO.

If you know the hash of some data, then you either already have the data yourself, or you learned the hash from someone who had the data.

If you already have the data, there is no vulnerability - since you cannot learn anything you don't already have.

If you got the hash from someone, you could likewise have gotten the data from them.

People do need to be aware that 'some random hex string' in fact is the irrevocable key to all the data behind that hash - but that's kinda inherent to gits design. Just like I don't tell everyone here on HN my login password - the password itself isn't sensitive, but both of us know it accesses other things that are.

If github itself was leaking the hash of deleted data, or my plaintext password, then that would be a vulnerability.