July 31st, 2024

Rustgo: Calling Rust from Go with near-zero overhead

An experiment explores calling Rust code from Go to enhance performance in critical applications. It proposes linking Rust directly into Go, avoiding cgo overhead, and details calling conventions for efficient integration.

Read original articleLink Icon
Rustgo: Calling Rust from Go with near-zero overhead

The article discusses an experiment to call Rust code from Go with minimal overhead, aiming to replace assembly code in performance-critical applications. The author highlights Rust's advantages, such as being more readable than assembly while still allowing for optimizations. The challenge lies in Go's Foreign Function Interface (cgo), which, while enabling calls to C functions, incurs significant performance costs that are unsuitable for small, frequently called functions.

The proposed solution involves linking Rust code directly into Go programs, treating it similarly to assembly. The author outlines the process of compiling Rust into a static library and linking it with Go using the system linker. This method avoids the overhead associated with cgo, allowing for more efficient function calls.

To successfully call Rust functions from Go, the author explains the need to understand calling conventions for both languages. The Go calling convention is mostly undocumented, but the author provides insights into how arguments and return values are managed during function calls. The article concludes with a practical example of invoking a Rust function from Go, demonstrating the potential for high-performance inter-language calls without the complexity and overhead of cgo. The experiment illustrates the feasibility of integrating Rust into Go applications, leveraging Rust's performance benefits while maintaining Go's usability.

Link Icon 11 comments
By @nickcw - 3 months
That was a great read. All that linker wrangling is sure to break on the next version of go/rust/linker isn't it?

I wonder if it would have been easier to disassemble the rust binary and turn it into Go assembly and use that.

That would need a fairly complicated program to process the binary back into assembler. Maybe getting the rust compiler to output assembly and processing into Go assembly would be the way.

Using Go assembly would save fighting with the linker, be more likely to survive upgrades and it would be cross platform (well at least on platforms with the same CPU arch).

By @hardwaregeek - 3 months
lol I remember looking at this when doing our major port from Go to Rust. I noped out once I saw the raw assembly portion. Great for a side project but probably not the move for a cross platform binary run by thousands of people. Still a very cool post and the literature for Go/Rust interop is def lacking
By @yutijke - 3 months
https://github.com/petermattis/fastcgo, which is now 7 years old seems to do something similar without the need to about obscure CGO FFI configuration. It also seems to be more generally applicable for any language with C interop.

There had been an issue for having something similar in the language itself - https://github.com/golang/go/issues/42469, but the Golang compiler team rejected it. If you have followed similar discussions around this with the Golang compiler team, you will notice a pattern of interaction that strongly indicates that they are very much opposed to ever accepting this into the compiler.

By @zxilly - 3 months
There are many years since the article being written. I'm wondering if there is a better solution in 2024.
By @gnabgib - 3 months
(2017) Discussions:

2017 (282 points, 68 comments) https://news.ycombinator.com/item?id=15017519

2019 (107 points, 37 comments) https://news.ycombinator.com/item?id=20600178

By @odanalysis - 3 months
Does this mean you could now use gokrazy to run rust apps on an sbc with better startup speeds than linux??
By @sitkack - 3 months
This is totally sick! I can't wait to go through these repro steps using new versions.
By @ethegwo - 3 months
it makes me think about https://news.ycombinator.com/item?id=41117749 it is the mirror of this project: effient wal to call Go from Rust
By @Havoc - 3 months
What is the benefit of this?

I was under the impression that go isn't that far off from rust on most speed metrics being both compiled. So this adds complexity for what gain?