Entering text in the terminal is complicated

This is nice. As always, Julia's articles are always a win.

Here's some stuff that's missing:

Within shell scripts, you can use `stty` to change a lot of stuff about the terminal, including how it deals with inputs. You can rewire all of these defaults and behaviors.

Here's an experiment I did a while ago using sh and stty: https://gist.github.com/alganet/63f1dbc97b8fd35f7bb14ec30f79...

It is able to capture and understand most keyboard combinations and even mouse gestures (hold/drag/drop) from within the shell in any VT100-compatible terminal (mintty, xterm, Terminal.app, vscode, so many others).

Runnable demo:

    bash -c "$(curl -L  https://git.io/fjToH)"

Run it and press some keys or move the mouse. Use Ctrl+W to exit. It supports zsh and ksh too, but not dash or other shells lacking `read -rn1`.

---

Here's another funny thing:

    `vi | cat -v`

If you pipe an interactive program to `cat -v`, you can see which VT100 escapes that program is using. I learned a lot from how vi does it.

Over 20 years ago now I wrote a state machine around readline that meant you could use it as a multi-line editor. Not "multi-line" as in a single line can wrap, but a true editor that lets you move up and down, but is windowless.

Here's a video:

    https://github.com/colmmacc/jot/raw/master/jot-demo.mp4

and the CVS repository for the Unix terminal IM client it is part of is at:

    https://c-hey.redbrick.dcu.ie/src/c-hey_cvs_latest/

It tracks and redraws your cursor when you move up and down between lines, and also pays attention to SIGWINCH to redraw things when the terminal size changes. I've never had the time to rewrite it in Rust, but I'd like to and then release it as a small library.

It's always surprised me that nobody else has done this.

> some things this post left out

+ wide characters

+ different keyboard modes causing the same key press to be presented by different ANSI escape sequences

+ different TTY states (eg local echo)

+ different OSs having subtly different syscalls for changing TTY states

+ differing support for terminal emulation (most these days roughly emulate xterm, but even there, none aim for 100% be compatibility)

+ a lack of a consensus on how to check for features provided by a terminal (some use ANSI escape sequences and wait for an ansi sequence to be returned from the term (that’s how the old VTs worked), some check $TERM (that’s how the first wave of terminal emulation was detected), some terms expose themselves as xterm and ignore the VT feature sequences, but set other env vars. it’s honestly a bigger mess than the user-agent string.

…and this is assuming a POSIX system. Things get doubly interesting when you throw Windows into the mix

In bash, if you set $EDITOR to your favorite text editor, you can send the current line to $EDITOR with ctrl-x ctrl-e. After editing the command you can save and exit for the command to execute.

> First, there’s “the baseline” – what happens if a program just accepts text by calling fgets() or whatever and doing absolutely nothing else to provide a nicer experience.

[...] there are actually a few features that you get for free just from your terminal, without the program needing to do anything special at all.

The things you get for free are: [...]

- backspace

- Ctrl+W, to delete the previous word

- Ctrl+U, to delete the whole line

Linux noob here, so this may be a stupid question, but how does that work under the hood?

The documentation for fgets() says [1]:

> Reads at most count - 1 characters from the given file stream and stores them in the character array pointed to by str. Parsing stops if a newline character is found (in which case str will contain that newline character) or if end-of-file occurs.

Does that mean that, by default, fgets() blocks until the user enters a newline and, before they do, it lets them edit the line buffer using the backspace, CTRL+W and CTRL+U shortcuts?

But there seems to be no guarantee that a program is using fgets() to read an entire line - it could also set count to 2 and read individual characters. Those could not be "unread" anymore when a backspace occurs. Is there some magic that lets fgets() buffer characters internally in such a situation, or would the backspace/line editing functionality just be broken then?

[1] https://en.cppreference.com/w/c/io/fgets

> some programs (the dash shell, cat, nc, git commit --interactive, etc) don’t support using arrow keys at all.

The dash shell does in fact support an editing mode if compiled with libedit. Debian derivatives do not do so (likely for concerns of space). They very much should, as people who start with bash will have much to unlearn.

The POSIX standard also specifies "set -o vi" as an optional extension. Every shell claiming POSIX-compatibility must support set -o vi if the shell implements this editing mode (so ignore inputrc if it pleases you).

"[set -o] vi: Allow shell command line editing using the built-in vi editor. Enabling vi mode shall disable any other command line editing mode provided as an implementation extension."

https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V...

I have continually contented that the terminal is the single thing eternally condemning Linux to <5% market penetration.

It's not just entering text. The entire experience is complicated. It's a Concorde jet cockpit[1] (with no labels too), when 95% of the population just wants to fly their drone around[2].

[1]https://qph.cf2.quoracdn.net/main-qimg-2566f4c91b894e4169d77... [2]https://media.thedroningcompany.com/images/tincy/WQZpC56vqMp...

Likely an unpopular take, but I switched to the "Natural text editing" preset in iTerm2 to get editing shortcuts that match the traditional macOS ones. It has the advantage of just remapping to the normal control sequences inside terminal apps, so you basically get this functionality everywhere without needed to change it in multiple places/worry about readline support/etc. It isn't perfect (I have rarely needed to enter sequences manually that then are remapped), but I find I have used way more command editing since its part of my muscle memory.

https://pliszko.com/blog/post/2021-10-31-natural-text-editin...

the top three default readline keybindings that would improve people's lives if they knew about them:

- control-w, which julia mentions in the post, to delete the last word

- control-o: when you're recalling a line from history, edited or no, runs the line and recalls the following line from history. so you can run a sequence of five commands from history by navigating to the first one and hitting control-o five times

- control-r: search backwards in time through your history as you type a search string. control-r again jumps to the previous hit, control-s goes back forward in time. hitting enter or control-o executes it

So when I use Windows Terminal and I press Ctrl-C, it always does the right thing: If I’m selecting text I want to copy it, if I’m not selecting text I want to kill the running process.

On Linux, every Terminal app I’ve used stubbornly refuses to copy when I press Ctrl-C, demanding I press Ctrl-Shift-C. When I paste, if I forget my manners and use Ctrl-V instead of Ctrl-Shift-V, I am punished by getting a weird character when I start typing again. And I am constantly pressing Ctrl-V because of muscle memory since no other app works this way.

Is there any terminal app for Linux that does things the Windows Terminal way and won’t slap me on the wrist for Improper Teletype Usage?

Whenever someone brings up this subject, I'm reminded of this classic article from Linus which has appeared many times on HN: http://www.linusakesson.net/programming/tty/index.php

warp and and some powershell version were the only “terminals” i used that seemed fixing the input line, but also did not quite feel right. the worst parts of terminals for me are that input line does not behave like a normal text field with mouse click, select and everything a user would expect in another context, that history mostly scrolls on a line basis instead of smooth pixel scrolling like any other document (don’t get me started on pagers, i still don't manage to disable man pages opening in some archaic mode that takes 3 tries to even exit from git), commands that default to tuis or clearing the screen. i tried to avoid unix as long as possible because of this and the absurd random / hostile folder names + layout but just because *nixes won does that mean we have to keep these archaic absurdities alive? why does even something where innovation would be simple like the vscode terminal feel like that and not like the chrome console with a normal text input line with graphic autocomplete and mostly immutable command response history with collapse/expandable sections? even unix commands that would not work in this mode could just run in something like an inline widget or new tab where they could read input lines by character or whatever weird behavior they expect.

I plan to make one day an alternative to terminals. My idea is not fully fleshed out, but I would like to try to forgo TTYs altogether. Use only pipes, file descriptors and job control.

The closest thing I saw to what I have in mind is this: https://github.com/letoram/cat9/ but more in a way of interface.

Nice to see this writeup. Entering text into the terminal, and copying output out, is a huge barrier to entry for beginners. Windows terminals in particular feel downright antagonistic to use.

> took me maybe 15 years of using the terminal every single day to get used to using Ctrl+A to go to the beginning of the line (or Ctrl+E for the end).

And the curse of bad defaults strikes again (daily).

After the realization that this is a weird default you don't train yourself for 15 years, but change it to your comfortable/common keybinds!

Also worth mention the Plan 9 "fixed prompt" where any command is at the top, with it's output underneath, there are few variants and in few cases they are comfy.

Another neat trick is to use ctrl-x ctrl-e, this gets the command in a vim buffer (default editor for me). Edit the command and then :wq to reflect the changes

For me, Ubuntu's terminal is perfect.

Give me arrows, tab, Ctrl+(R/C/D/W) etc and its a great.

But I agree with the point that different systems have their own implementation which could lead to frustration.

I am working on set of servers only accessible through PAM and those admins don't allow anything. It gets so frustrating.

I feel a lot of the frustration also comes from shifting to unfamiliar environment. Like I work on those restricted servers and am cursing the admins, but when I switch to Ubuntu's terminal I am thanking the gods for liberation.

Periodic reminder that many emacs-like keybindings exist in almost every text field in macOS: Ctrl A, E, P, N, K, Y, T, and probably others I'm forgetting (start of line, end of line, previous line, next line, kill (delete everything to the end of the line and put it in a special pasteboard), yank (paste from that special pasteboard) and transpose characters).

Edit: more complete list here: https://jblevins.org/log/kbd

Edit 2: can't believe I forgot Ctrl-D, which is forward delete.

"The other day I asked what folks on Mastodon find confusing about working in the terminal, and one thing that stood out to me was "editing a command you already typed in".

This really resonated with me: even though entering some text and editing it is a very "basic" task, it took me maybe 15 years of using the terminal every single day to get used to using Ctrl+A to go to the beginning of the line (or Ctrl+E for the end)."

Only took me a year or so to get used to using "v" in vi mode and fc. I am not a "developer" but I prefer textmode command line to GUI. I do not use X11/Wayland/etc.

I learned sh on NetBSD which I still think is one of the best shells available. It's fast. vi mode is the default. People may disagree but I think for editing single a line of text, i.e., a "command", vi mode offers more precision. For example, being able to jump directly to a particular column number.

https://web.archive.org/web/20240528201424if_/https://pubs.o...

   #list previous commands
   fc -l 
   #edit 5th command on the list
   fc 5 

IMO, as a dumb computer user who is not a "developer", this is not complicated. I think terminal emulators are complicated, though. I do not use them.

I believe fc came to POSIX from ksh.

For me, it is not that UNIX is objectively good. It is that the available alternatives are still comparatively bad.

I think the lowest hanging fruit would be doing something (anything) to improve discovery of commands. I have great grokking abilities but terrible memorization skills, and I end up forgetting how to do what I want to do and maybe even have done before. Browsing man pages for every command is my bane. It's also why I dislike GUI with unlabelled icons, I never remember what each one does.

> It took me maybe 15 years of using the terminal every single day to get used to using Ctrl+A to go to the beginning of the line (or Ctrl+E for the end).

That looks like some cognitive problem or ADHD. (Obviously, completely unrelated to intellect.)

> some programs (cat, nc, git commit --, etc) don’t support using arrow keys at all: if you press arrow keys, you’ll just see ^[[D^[[D^[[C^[[C^

I've long argued that the readline-like functionality should be in the kernel. The POSIX line discipline should be replaced or augmented with full blown editing with history recall.

Imagine rlwrap, but always there, all the time, in the TTY driver.

There are downsides, because history wants to be contextual and persistent. The kernel knows what process is making the read() call on the TTY, though, so that could be somehow arranged. Certain new security issues come up also.

Wirth showed us decades ago how to do UIs using text in a modern way yet we remain firmly tied to digital analogues of teletype terminals.

https://en.wikipedia.org/wiki/Oberon_(operating_system)

> it took me maybe 15 years of using the terminal every single day to get used to using Ctrl+A to go to the beginning of the line (or Ctrl+E for the end)

Ctrl+LeftArrow and Ctrl+RightArrow also work with every readline-compatible shell and they are much more intuitive, especially if you are not used to terminal text editors.

> it’s very inconsistent between programs

Readline or emulations thereof are pretty common and the de-facto standard. But of course programs can do their own thing if they want to - it's not like GUI text eidtors all have consistent shortcuts.

cat / nc / etc. are not editors at all and therefore don't provide those shortcuts. They take an input stream and forward it.

The vi vs emacs thing permeates a lot of tools. I find it useful to map all tools to be consistent with one or the other. Personally I use vi. Unfortunately setting vi mode for readline has a few caveats, but I was able to work around all of them with the settings in:

https://www.pixelbeat.org/settings/.inputrc

I switched my Mac terminal to Warp.dev simply for the default mac text input. I can ⌥ and ⌘ left and right to select words or until the end of the line ⌘A to select all. It comes with so many other bells and whistles that I've mostly ignored for this simple convenience. I don't use emacs (or vim) keybindings in any other context and I'm not sure why the terminal forces you to.

I've never seen dash dump escape sequences instead of acting on arrow keys. Probably, it's due to a missing or wrong TERM value?

Nowadays, programs should be hard-coded to accept ANSI sequences and ignore TERM (unless they need to make some fine-grained distinction like do we have 256 color xterm compatibility).

Those programs will never have that problem of not understanding arrow keys.

A meta-question for those of us very comfortable composing complex sequences on the command line: when I’m trying to help someone with something, how can I avoid becoming frustrated by their lack of facility - or worse, their lack of interest in obtaining facility, without being an ass and taking over the keyboard? Grumble.

I use M-x shell and variants in Emacs for almost all of my command line work. No problems with editing, as it is always Emacs on top. For true ncurses apps there are some limitations inside vterm or ansi term but even there, I can still use elisp and fix these problems if needed.

> the readline keybindings come from Emacs

> Because I’m a vim user, It took me a very long time to understand where these keybindings come from

libreadline supports a basic vi mode. In bash, `set -o vi` lets you use vim-style editing. It is a lifesaver.

I just press and hold backspace and try again from scratch

Or you can just write a bash script file to do shortcut executions of complicated terminal commands...

See also the same blog's previous article, "What happens when you press a key in your terminal?"

https://jvns.ca/blog/2022/07/20/pseudoterminals/

(It's a great article, and I'm fairly surprised the post doesn't directly link to it)

Rlwrap is a great utility. rlwrap telnet ... makes life so much easier.

Mode ∞: shell mode in Emacs.

I thought this was going to cover RS232 implementations. Oh well, next time :-)

> license reasons, if the program’s license is not GPL-compatible – readline is GPL-licensed, not LGPL

The older I get, the less patient I become with finding out some user experience sucks because of GPL / non-GPL knife fights.

It's been thirty-nine years now. I think the GPL was useful at its origin, but now the benefits of open-source are proven out, the world is deeply interconnected via the Internet, and it's a hindrance to have some code burden other code with requirements. I, for one, don't expect to write any further GPL code in my lifetime. I'd rather code be maximally unencumbered from interoperation.

It took the author 15 years to learn basic readline commands?

I have great respect for Julia Evans and love her contributions that get shown here.

But...

> "I’ve always thought that vi mode seems really cool, but for some reason even though I’m a vim user I didn’t really like using it when I tried it."

Really, that seems weird to me.

I use zsh (btw) and the problem she describes is a non-issue. And I am not a pro. (Sorry, Julia).