Python type hints may not be not for me in practice
Chris Siebenmann expresses skepticism about Python's type hints for small tasks, finding them cumbersome. He prefers adding them post-stabilization for documentation, envisioning a system allowing distinct types without upfront details.
Read original article
Chris Siebenmann reflects on his experience with Python's optional type hints, expressing skepticism about their practicality for small programming tasks. He initially considered using type hints but ultimately decided against it, finding that they require him to define concrete types for basic data structures like dictionaries and lists, which can be cumbersome. He notes that while he often recognizes distinct types, he prefers not to specify their exact forms, as the typing documentation does not provide a straightforward way to create distinct types without losing flexibility. Siebenmann acknowledges that adding type hints after code stabilization might offer some benefits, such as documentation and error prevention when revisiting the code later. However, he believes that the common advice to write type hints during development may not apply to his short programs, where adding them afterward is manageable. He concludes that while practicing type hints could make them easier to use, his infrequent programming means he would need to relearn them each time. His ideal scenario would involve creating distinct types for function arguments and returns, allowing for type checking without needing to define every detail upfront.
- Chris Siebenmann is skeptical about the practicality of Python type hints for small programs.
- He finds type hints cumbersome as they require defining concrete types for basic data structures.
- Adding type hints after code stabilization may provide documentation and error prevention benefits.
- Siebenmann believes that the advice to write type hints during development may not suit his programming style.
- He envisions an ideal type hinting system that allows for distinct types without upfront detail specification.
Related
The human typewriter, or why optimizing for typing is short-sighted
The article highlights the drawbacks of optimizing code for typing speed, advocating for readability and clarity over efficiency, as obfuscated code increases mental strain and confusion among developers.
Software development topics I've changed my mind on after 6 years in industry
The author reflects on their software development views, emphasizing typed languages for diverse teams, the importance of architecture, context-dependent best practices, and questioning the necessity of many project managers.
38 commentsThey say "...so I can't create a bunch of different names for eg typing.Any and then expect type checkers to complain if I mix them."
`MyType = NewType('MyType', Any)`
is how you do this.
At the end, they suggest a workflow: "I think my ideal type hint situation would be if I could create distinct but otherwise unconstrained types for things like function arguments and function returns, have mypy or other typing tools complain when I mixed them, and then later go back to fill in the concrete implementation details of each type hint"
That's just doing the above, but then changing the `NewType('MyType', Any)` to something like `NewType('MyType', list[dict[str, int]])` later when you want to fill in the concrete implementation.
I think most developers who revisit their projects 6+ months later would disagree with the second part of this statement.
My typical flow for "quick scripts" is:
on first pass I'll add basic type hints (typing ":str" after a func param takes .2 seconds)
for more complex data structures (think a json response from an api), dict (or typing.Dict) work fine
if you want a Python project to be maintainable, type hints are a requirement imho.
From the GitHub page:
RightTyper is a Python tool that generates types for your function arguments and return values. RightTyper lets your code run at nearly full speed with almost no memory overhead. As a result, you won't experience slow downs in your code or large memory consumption while using it, allowing you to integrate it with your standard tests and development process. By virtue of its design, and in a significant departure from previous approaches, RightTyper only captures the most commonly used types, letting a type checker like mypy detect possibly incorrect type mismatches in your code.
We also use Python in some places, including the shitty Python type-system (and some cool hackery to make SQLAlchemy feel very typed and work nicely with Pydantic).
I don't know how other IDEs behave, but VScode + the Python extensions try to infer the missing hints and you end up with beauties such as `str | None | Any | Unknown`, which of course are completely meaningless.
Even worse, the IDE marks as an error some code that is perfectly correct, because it somehow doesn't match those nonsensical hints. And so it gives you the worst of both worlds: a lot of false positives that you quickly learn to ignore, dooming the few actual type errors to irrelevance, because you'll ignore them anyways until they blow up at runtime, just as it'd happen without typehints.
I'm missing a lot simple functions with explicit argument names and docstrings with arguments types and descriptions clearly but discreetly documented.
It was one big strength of Python to have so simple and clean code without too much boilerplate.
Also, I have the feeling that static typing extremist are trying to push the idea that type hinting allows to ensure to not mix types as it would be bad. But from my point of view the polymorphic and typing mixing aspect is a strong force of Python.
Like having dictionaries that are able to hold whatever you want is so incredible when you compare to trying to do the equivalent in Java for example.
One part where I find type hint to be wonderful still is for things like pydantic and dataclasses!
- The LLMs can really help with typing tricky situations. If your editor can't already tell you what to use, asking an LLM usually can give me the answer.
- Type annotations for code that might change is a lifesaver, because when I change it later on I now get a bunch of conflicts where I've used it using the old way.
- Feel free to add annotations where it makes sense and is easy, and if something doesn't make sense or it is too hard to figure out the right type, you can skip it and still gain the benefits of using it elsewhere.
- Annotations don't "force you to think about types", you already are thinking about types. They let you think a bit less about types I would argue, because they're documented in function calls and returns. "Can I read() from input_file, or do I need to open()read()?" "input_file:Path" makes it better documented, without encoding the object type in the name.
I'm coming up on 30 years of using Python, and I never really missed typing, but honestly I write basically all of my new code with annotations because of the IDE benefits I get from it. I love that the Python implementation allows me to get the benefits without forcing it on me. In my distant past I very much loved coding in C, but was quite happy with Python's lack of strict typing. This feels like a good middle-ground.
If you are at work doing professional programming then typing helps avoid bugs and makes programming more of a reliable and robust process.
But doing your own thing, doing little utilities, banging out quick stuff, do exactly what makes you happy.
Programming should be fun not a chore and if types make it a chore for you then drop them.
Personally I find myself more comfortable and productive using types. Stating your types has a similar benefit to journaling, in my view. It's a forcing function for clarifying your ideas about the problem domain. Some perceive this as overhead, I perceive this as front loading. If my ideas are murky, I will run into trouble sooner or later. The later it is, the more painful it will be.
I think it largely comes down to different habits of working and thinking. I don't think one way is superior to another inherently (though types are important for collaboration), but that different people work in different ways.
We have pyre enforcement at work, the problem is, that it has been gradually turned on over time, so some stuff is pyre compliant (or just strategically ignored) and some stuff isnt, so when you open some old code to do something, you have a million errors to deal with.
That would be fine if types were enforceable. in runtime type hinting does shit all.
I would dearly love a "strict" mode where duck typing is turned off and variable are statically typed. However I suspect that will never happen, even though it'd speed up a load of stuff if done correctly (type inference happens a lot)
I suspect to use type hints properly, I'd need to think a bit more C-like and create dataclasses as types to make things more readable, rather than using Dict[str,int] or what ever.
It sounds to me like you're describing the NewType pattern which is just slightly farther down the page you linked in the article.
It's perfectly feasible to write maintainable, well-designed code in a dynamic language. I've worked with some extremely robust and ergonomic Clojure codebases before, for example. However, in Clojure, the language pushes you into its own "pit of success".
Personally, I never feel that with Python.
Code like this [0] could simply be 3 functions. Instead it's 3 classes, plus a base class `AstNode`, just so the author can appease the type checker by writing `body: List[AstNode]` instead of the dynamically-typed `body = []`.
[0] https://gist.github.com/sportsracer/16a1e294966cfba83ba61e6a...
Typing hints are also a moving target: they have changed, sometimes significantly, on every minor Python release since they came to be.
The `Optional` type came and went (being replaced by the new Union syntax.) Type classes are usually born in the `typing` module, then some of them get moved to `collections`, `abc`, or `collections.abc`. Some types have even been moved several times. `TypeAlias` came and went (replaced by `type`). `List` became `list`. `Tuple` became `tuple`. Forward declarations require a workaround by using string literals. But in Python 3.14, they no longer do, and the workaround will become deprecated.
I'm an evangelist for static type checking, and I never write a Python function signature without complete typing annotations. I also think that typing annotations in Python are evolving in a good way. However, for long-lived Python scripts, I highly recommend that developers are aware of, and factor in, the effort necessary to keep up with the evolution of static typing across Python versions. That effort spent on migrating is going to come on top of the mental load that typing hints already add to your plate.
For x in tests, type annotations, and documentation*:
If you write your x first then you have to decide what your API is. This is great if you want to think about your API. Sometimes though you just want to get down to it and play around with a new idea. Either way is fine.
As soon as you start sharing code or patching production code or patching someone else’s production code, one must insist on seeing some kind of x. Having x around the outside of a system — rather than requiring x be added throughout the entire system — is often good enough.
*The useful, architecture kind.
Cleaner/safer function args and return types is a common motivation for dataclass. has benefits over many/complex args besides typing too.
switching to the typed variants whether typecript, python type-hints, mypy etc will force you to do the dance to make the compiler happy instead of working on code.
which is why for me - JSDoc is really good - it uses types for documentation and ends there.
A) The type system is fairly bad, as type systems go (forgivable) B) the type checkers are, for large codebases, excruciatingly slow, to the point that the tests are faster!!
The second is not forgivable.
Hyperbolically: You have to be able to edit code at the speed of thought - whatever it takes - or else programming languages cease to be a more useful tool than just thinking.
If you type slower than you think, or can’t do the type-hint-based textual translation as quickly as you think, then… yeah - it’s not good for you.
The advice I’d wanna hear for myself is: just get better. But the advice I’d give to my coworkers is: have explicit domains where you’re able to do whatever is most efficient and effective, and then when you hand off data to the next subsystem - obey a contract. A schema. Be that type hints or a .proto file or a database schema or an API. Doesn’t matter.
1. design a memory layout for faster execution
2. press dot and get suggestions in IDE
other usage of types brings more problem than it solves.
Yes, thats the point.
In the way we resolve these issues in natural language, we can resolve bugs in python, that is, “do you mean integer ’3’ or string’3’” instead of insisting we define everything always forever.
To me, people who use type hinting are just letting me know they have written code that doesn’t check in line.
Related
The human typewriter, or why optimizing for typing is short-sighted
The article highlights the drawbacks of optimizing code for typing speed, advocating for readability and clarity over efficiency, as obfuscated code increases mental strain and confusion among developers.
Software development topics I've changed my mind on after 6 years in industry
The author reflects on their software development views, emphasizing typed languages for diverse teams, the importance of architecture, context-dependent best practices, and questioning the necessity of many project managers.