Writing GUI applications on the Raspberry Pi without a desktop environment

AvaloniaUI can also utilize framebuffer to run on Raspberry PI, see https://docs.avaloniaui.net/docs/guides/platforms/rpi/runnin...

I've been trying to setup a pipeline of pass through hdmi from an HDMI input to an HDMI output with an OrangePI5 Plus. I could talk for a long time (now) about the issues with vendor supplied kernels and unsupported hardware. I was completely naive until I had the hardware in hand, having not done any embedded work.

Right now, the best of breed thought that I have is to run Weston, have a Qt application that is full screen, and to use DMA buffers so I can do some zero copy processing. Rockchip has their own MPP and RGA libraries that are tied into the Mali GPU, and I'm not smart enough to understand the current level of driver/userspace support to not leverage these libraries.

Rockchip and the ARM ecosystem is such a mess.

If anyone has any pointers, experience, approaches, code, etc, I would love to see it.

LVGL is pretty much the IOT industry standard for 32-bit architectures like Renesas RX, Arm Cortex-M, ARC, TI MSP, Atmel...

https://lvgl.io/

PyGame is also great for scripting quick UIs for the fb. You don't want to write big apps in it but for ease of setup and use it's sweet.

One problem with Raspberry Pi displays is that not all of them provide vsync signal in SPI mode. That leads to high CPU usage (due to very high frame rate) and its generally inefficient. Choose your display carefully.

I've successfully utilized Qt5 (probably would work with 6 too) for framebuffer/OpenGL.

QML is nice, animations were much smoother than I expected.

The way I did it was to run Weston without any icons or anything and using systemd to start my app and also having systemd reopen my app if it closes. Worked well enough.

We have a bunch of examples of GUIs that run inside containers (with different backends, frameworks, without compositor etc): https://github.com/toradex/vscode-torizon-templates.

Don’t forget about the Kivy framework. It’s a python framework that renders openGL accelerated UIs, also headlessly on the raspberry pi if needed. https://kivy.org

Years ago I played with TekUI on a Allwinner A10 board and it worked fine also displaying GUIs on the framebuffer without any underlying X environment.

http://tekui.neoscientists.org/

Only caveats: it's quite old and doesn't seem maintained anymore, although it still compiles fine, and it's Lua only, but being written in C it shouldn't be too hard to port it to other languages.

conceivably interesting to you if you want to do this: my library yeso https://gitlab.com/kragen/bubbleos/blob/master/yeso supports drawing on the framebuffer or in an x window; for c programs you choose which at link time, but the luajit and python bindings dynamically one shared library or the other according to whether they're running under x-windows or not

yeso is a very small and simple library, so you have to do more things from scratch than with libraries with more comprehensive functionality, but being able to test your app in a window before running it on the framebuffer could be useful

yeso's input handling on the linux framebuffer is not as complete as its x-windows handling, but it's good enough that yeso programs like ./tetris-fb (with wasd) and ./mand.lua do work on the framebuffer console. the terminal emulator admu-shell-fb mostly works, but it has the problem that things like control-z or control-c suspend or kill the terminal emulator instead of what you're running in it :)

(i haven't actually tried yeso on the pi, but if it doesn't work there i'll fix it)

I've really tried a few times to go down this rabbit hole and get something working and this thread has like 7 projects I've never come across before.

Yet another alternative for drawing directly on the framebuffer is DirectFB 2: https://directfb2.github.io/

It supports Vulkan, OpenGL, Cairo and other technologies.

I've been doing some R&D for building a Home Assistant dashboard with a nice touchscreen display I bought to go on the wall at home, with the ultimate goal of expanding it into many home-rolled displays.

Having slimmed down the RPi Lite OS as much as I can, running Wayfire with a Chromium kiosk is just too much for the Pi 3B+ I'm using once I add a streaming camera to the dashboard, and it can't cope. My goal is to have a responsive touch-screen display for Home Assistant using something in the form-factor of a Pi Zero 2W, so that I can put the SBC _inside_ the display and build a wooden picture frame to house it all, so it doesn't stick out like a sore thumb.

I'm not sure what kind of API HA has for the frontend, but my first thought, was to build a native application with a Go backend (I write Go for my day job) and use something like Wails[0] for the frontend and completely cut out the heavy-weight Chromium browser.

I have Pi4's and Pi5's but I really want to use the littlest amount of compute (and power) I can to achieve this, even it means writing the UI myself. I've tried looking for a lighter-weight browser that I can simply run the HA dashboard in to no avail.

[0]https://wails.io/

MPV can play videos direct to the FB, with hardware acceleration too

I like this write up. It is timely too.