Bytecode Breakdown: Unraveling Factorio's Lua Security Flaws

Unexpected!

Since lua interprets bytecode, it can check the arguments to the bytecode are meaningful. Point to memory lua allocated, things like that.

Turns out it doesn't do that. Feed it bytecode with invalid arguments passed to the instructions and it executes it anyway. The rest of the compromise follows.

Further, instead of fixing their interpreter, the game plan is to statically analyse bytecode. Which turns out to only work in simple cases.

For a sandbox friendly interpreted language this is pretty disappointing. I wonder if they'd take patches to stop the interpreter trusting the input - presumably performance regression is the fear there, which seems dubious when luajit is the fast option anyway.

I wish this was more defined or documented somehow. You're kind of left on your own to figure out whether a language is reasonably guaranteed to be safe or not.

Some example scenarios:

- Code is static and is executed directly by a user, the default case languages care typically care about. Including Lua.

- Code is dynamically fetched and executed through some update process, hopefully only through official channels. Here you can get away by making the process secure, but who knows.

- Code can be added by the user through plugins, this can be made easier through stores with the click of a button. You can review plugins, but this is hardly done. Here you need to consider if the code should be sandboxed or the user should be careful.

- A multiplayer game where a server can be extended with custom code via plugins, but not the clients. Here you need to consider that the users/gamers who are hosting servers are eager to try many different plugins. The plugin community (gamers) can also be a lot more dangerous.

- A multiplayer game where the server can execute arbitrary code on clients, just like a browser. Here you need to be very careful about sandboxing, especially on clients as gamers will just join random servers without thinking about the security implications.

The last point being Factorio's case. I'm not necessarily disagreeing that it's the developers job to evaluate this, but sometimes it's not obvious that for example the load function in Lua can run arbitrary bytecode which is unsafe.

To be honest, I wasn't aware that Lua's bytecode is unsafe, but I am aware that LuaJIT's bytecode is unsafe. But as far as I know this fact is just stated randomly in the mailing list and github issues as an obvious fact.

There is another thing about servers being able to crash clients (just run some infinite loop on them), but this much harder and maybe pointless to avoid.

Factorio has a really good dev team behind it so I trust that they're doing their best to fix these issues. Though, gamedev in general seems to be more of a creative endeavor which puts things like code practices and security on the back burner. I wonder how many zero day exploits are lurking in game clients / servers.

In general, verifying programs is extremely hard, not just because of rice's theorem but because it's so easy to miss a spot, especially for non-trivial bytecode languages like lua's. wasm has no concepts of for loops for example.

It's strange that after upstream has given up on the problem as it was too hard, factorio devs have chosen to try to fix the verifier/write their own (not sure which of the two they did).

Minetest's loadstring function forbids bytecode entirely: https://github.com/minetest/minetest/blob/9a1501ae89ffe79c38...

I wonder why factorio mods need the ability to execute raw lua bytecode. If they don't have it, there would be no need for a verifier.

It's quite dangerous in the first place to execute lua code downloaded over the network. JS execution environments have gone through decades of cycles of discoveries of exploits and fixes. Lua gets those as well but on a smaller scale, and with less staffing to improve security. The main protection is I guess that there is fewer people running malicious game servers.

Total newb question but why do games use lua, as opposed to (for instance) embedded JavaScript with some defined interface into the game (e.g. APIs to adjust game state).

Seems to me that this would benefit from the much more severe hardening work that's gone into isolating browser environments (a hard and VERY well tested and funded target), as well as the massive work that's gone into performance optimizing dynamic types.

Plus if the mod needs UI there's the canvas and potentially even React etc if a Dom-like model is available.

As a non-security dev, I'll drop the obligatory "wow this is incredibly impressive!" I can't believe how clear and logically you'd have to think to track down these intricate failure cases. Definitely not my strong suit! I'm much more of an "ideas guy" ;)

Content-wise: Wow... We are totally, completely, utterly screwed once people start putting together ensembles of AI SWEs equipped with 10,000 blog posts like this one on finding weird memory exploits. Ultimately I think we're gonna need a whole new paradigm for security, or at least some new element in the stack. It's my potentially naive opinion that all the modern talk about "trusted" clients and DB roles and all that is trying to patch holes in swiss cheese; hopefully, we can find a new stack of LLM-maintained swiss cheese to add on instead!

So... this demonstrates an exploit that relies on a feature that is advertised as exploitable: loading byte code. What am I missing?

We are seriously lucky such capable people are on the good side.

IMO, Lua bytecode should never be usable anywhere outside of embedded systems that don't have enough resources to run the Lua source code parser. Besides security vulnerabilities, the only other thing it seems to be useful for is closed-source programs.

Unless I missed it (I admit I skimmed towards the end) The author does not discuss at all the actual remediation that was taken. I would love to hear more about that.

Never enable in your Lua apps:

a. Bytecode

b. Debug, Io, Os libraries

...

Also, be very careful with sandboxing. It can be deceptively easy to break out of.

Everyone eventually learns this lesson, see: Roblox (removed bytecode almost a decade ago after, if I recall correctly, an exploit exfiltrated their server tokens), Company of Heroes (bytecode bug leading to RCE)

Luau should come with safe defaults, from what I've been told.

I’m confused about one thing.

It doesn’t follow to me, that since all clients are running their own simulation, Lua scripts must run on every client too.

If a client runs a Lua script, why can’t we just run it on their machine and propagate any game state changes (if the script adds an inserted, for example,) as if the player made those changes themselves?

Is it impractical to employ firecracker vm like separation to isolate untrusted code, severely reducing the impact of any bugs?

Browsers split their various components across multiple processes to provide isolation. VMs would provide even more isolation.

I literally just downloaded a factorial demo to my work laptop. Is this something I need to be concerned about if I don’t play online?

bytecode is also architecture dependent

At this point, I have serious doubts whether bytecode and JIT systems, whether it be Lua in Factorio or JavaScript in Chrome, can ever be verifiably secure. I think we would all be better off if, like Apple’s Lockdown mode, we can disable anything JIT on a high stakes system.

I don’t blame Factorio though - this (anonymous?) researcher is 100x developer material.