What's New in POSIX 2024
POSIX 2024 updates include error handling for newlines in filenames, C17 compliance, support for NUL separators in commands, and a focus on resource management and interoperability for improved script reliability.
Read original articlePOSIX 2024 introduces significant updates, particularly in the XCU section, which focuses on the shell command language and standard utilities. One major change addresses the handling of filenames in shell scripts, specifically the issue of newlines in filenames. The new specification encourages utilities to error out when creating or processing filenames that contain newlines, aiming to improve script reliability and interoperability. Additionally, the specification now supports the use of NUL characters as separators in commands like `find` and `xargs`, although this approach has limitations. Another key update is the requirement for C17 compliance, moving away from the previous C89 requirement, which allows developers to utilize more modern C features. This change is expected to facilitate the adoption of contemporary programming practices across various platforms. Furthermore, the specification emphasizes the importance of limits and cooperation within operating systems, enhancing user control over resource management. Overall, POSIX 2024 marks a significant step towards modernizing the standard and improving usability for developers.
- POSIX 2024 introduces error handling for filenames containing newlines in shell scripts.
- The specification now requires C17 compliance, moving away from the outdated C89 standard.
- New support for NUL character separators in commands like `find` and `xargs` is included.
- The updates aim to enhance script reliability and interoperability across systems.
- Emphasis on limits and cooperation within operating systems is reinforced in the new specification.
Related
POSIX 2004 Changes
The POSIX 2024 standard is out with enhancements. Sortix OS integrates most new features, excluding some complex or optional functions like posix_close(). Implementation status varies per function for compatibility.
What's new in C++26 (part 1)
C++26 is developing features like specifying reasons for deleted functions, unnamed placeholder variables, structured binding in control statements, and user-generated messages in static_assert, enhancing code clarity and usability.
Techniques I Use to Create a Great User Experience for Shell Scripts
The article outlines six techniques to enhance user experience in shell scripts, focusing on error handling, colorful output, progress reporting, platform adaptation, and timestamped file outputs for better usability.
OpenBSD now enforcing no invalid NUL characters in shell scripts
OpenBSD's ksh now prohibits invalid NUL characters in scripts to unify shell behavior and prevent inconsistencies. Users must be on OpenBSD-current to utilize this update for improved reliability.
The Heart of Unix (2018)
Unix remains a powerful programming environment, allowing users to extend functionality through shell scripts. However, it has outdated components and modern Linux distributions complicate user interactions, suggesting a need for evolution.
- Many commenters express excitement about the stricter error handling for newlines in filenames, viewing it as a positive step towards better script reliability.
- There are concerns about the requirement for C17 compliance, with some questioning the practicality and implications of not having a concrete standard.
- Several users highlight the challenges of internationalization in shell scripts, noting the added complexity it brings.
- Some commenters appreciate the updates as a way to improve existing code, contrasting it with past practices that enforced stricter compliance.
- There is a general curiosity about how these updates will be adopted across different systems and their impact on legacy POSIX systems.
Oh my god. This makes me so happy. This is the most lovely think I've read in the world of computing since the unix gods decided that newlines were to be a single character.
The philosophy underlying the sentence "Wouldn’t it be nice if the naive scripts were just correct now?" is incredibly positive. We are surrounded by arrogant jerks who break old code by aggressively enforcing stricter compliance of some stupid rules. But here come these posix heros who do the exact opposite: make old code correct! There is hope in mankind after all.
- find(1p) now supports -print0
- xargs(1p) now supports the -0 argument
- newlines in filenames now should throw errors in many utilities
- a complier implementing the c17 standard is now required
- ulimit is expanded
- renice can use relative values
- a timeout utility has been added
- make adds support for $^ $+ ::= :::= != ?= +=
- logger is improved
- gettext is adopted
- readlink and realpath are adopted
- rm now supports -d to remove empty directories and -v for verbose
- various improvements to printf, sed, test
Finally. Now let's do the rest: https://dwheeler.com/essays/fixing-unix-linux-filenames.html
Filenames should be boring printable normalized UTF-8. I have never, not once, seen a good reason that a filename should be able to contain random binary gobbledygook
SRC != ls *.c
is fine in a makefile as far as POSIX is concerned, because:> Applications shall select target names from the set of characters consisting solely of slashes, hyphens, periods, underscores, digits, and alphabetics from the portable character set
… makes PowerShell start to look damn good.
The way I feel posix, and other descriptive standards work best is when they describe what every one is already doing. This is opposed to prescriptive standards which try focus on how the "correct" way to do somthing, prescriptive standards tend to be over engineered and may or may not actually work.
see also: descriptive and prescriptive dictionaries. http://www.englishplus.com/news/news1100.htm
For shell scripts part of a large system, that's probably fine. For small scripts, that's not very practical. You are not only adding a compilation step, you're also requiring distribution of multiple files. That's a pain.
It just kind of kills the convenience of a simple shell script. I would probably end up writing a makefile to manage all of this and at that point I am only a hop skip and jump away from using a compiled language instead of shell.
But restricting this at syscall level is definite insanity, among with excuses.
Is this really good?
If you can't rely on anything concrete being guaranteed, and it is open to interpretation what "modern" or "major toolchains" are, why have a standard?
(Listed in the Sortix article linked in OP.)
Pathnames can neither contain NUL nor '/'.
Re: `find -print0` / `xargs -0`:
> Previous POSIX releases have considered -print0 before, but never ended up adopting it because using a null terminator meant that any utility that would need to process that output would need to have a new option to parse that type of output.
What nonsense. Just add the `-0` or similar options as needed.
> More precisely, this approach does not resolve our original problem. xargs(1p) can’t sort, and therefore we still have to handle that logic separately, unless sort(1p) also grows this support, even after read(1p). This problem continues with every other type of use-case. Importantly, it breaks the interoperability that POSIX was made to uphold.
More nonsense.
> A bunch of C functions3 are now encouraged to report EILSEQ if the last component of a pathname to a file they are to create contains a newline (put differently, they’re to error out instead of creating a filename that contains a newline).
Ok, that's tolerable. Ditto utilities (notice here they were able to make a list of utilities).
I wish it would have been c99. What does c17 add exactly, more C++-esque complexity or not? Why was it not c99 (or perhaps even c11) over c17? Genuine questions.
Related
POSIX 2004 Changes
The POSIX 2024 standard is out with enhancements. Sortix OS integrates most new features, excluding some complex or optional functions like posix_close(). Implementation status varies per function for compatibility.
What's new in C++26 (part 1)
C++26 is developing features like specifying reasons for deleted functions, unnamed placeholder variables, structured binding in control statements, and user-generated messages in static_assert, enhancing code clarity and usability.
Techniques I Use to Create a Great User Experience for Shell Scripts
The article outlines six techniques to enhance user experience in shell scripts, focusing on error handling, colorful output, progress reporting, platform adaptation, and timestamped file outputs for better usability.
OpenBSD now enforcing no invalid NUL characters in shell scripts
OpenBSD's ksh now prohibits invalid NUL characters in scripts to unify shell behavior and prevent inconsistencies. Users must be on OpenBSD-current to utilize this update for improved reliability.
The Heart of Unix (2018)
Unix remains a powerful programming environment, allowing users to extend functionality through shell scripts. However, it has outdated components and modern Linux distributions complicate user interactions, suggesting a need for evolution.