What Is Theoretical Computer Science?

This is a great article and I especially liked the notion:

>Theoretical physics is highly mathematical, but it aims to explain and predict the real world. Theories that fail at this “explain/predict” task would ultimately be discarded. Analogously, I’d argue that the role of TCS is to explain/predict real-life computing.

as well as the emphasis on the difference between TCS in Europe and the US. I remember from the University of Crete that the professors all spent serious time in the labs coding and testing. Topics like Human-Computer Interaction, Operating Systems Research and lots of Hardware (VLSI etc) were core parts of the theoretical Computer Science research areas. This is why no UoC graduate could graduate without knowledge both in Algorithms and PL theory, for instance, AND circuit design (my experience is from 2002-2007).

I strongly believe that this breadth of concepts is essential to Computer Science, and the narrower emphasis of many US departments (not all) harms both the intellectual foundations and practical employment prospects of the graduate. [I will not debate this point online; I'll be happy to engage in hours long discussion in person]

> Thinking of theoretical computer science as a branch of mathematics is harmful to the discipline.

Maybe the issue is how he thinks about mathematics... He quotes Von Neuman saying:

> We should remember the warning of John von Neuman,e one of the greatest mathematicians and computer scientists of the 20th century, regarding the danger of mathematics driven solely by internal esthetics: “There is a grave danger that the subject will develop along the line of least resistance.”

But for example, there is an article in Quanta [0] about a recent proof in Number Theory (you cannot get more "mathematical" than that), and the guy who proved it said:

> “Mathematics is not just about proving theorems — it’s about a way to interact with reality, maybe.”

Which is in line with Von Neuman's quote, and with the spirit of what the author is saying.

So maybe a more accurate subtitle would be:

"Thinking of theoretical computer science as a mathematical formal system is harmful to the discipline."

[0] https://www.quantamagazine.org/big-advance-on-simple-soundin...

I met the author informally once but didn't know "who" he was, a famous academic scientist and professor etc. He read my paper and caught a math typo. :)

Regarding the article, I feel like there's some context missing, is this part of some ongoing debate about TCS? The piece abruptly ends.

One that comes to mind is the recent breakthroughs in AI which have caught theorists flat-footed. Sanjeev Arora and others openly admit we don't have a good theoretical understanding of the empirical successes of deep learning, how they are able to do the things they do with emergent phenomena, scaling effects, etc.

Scott Aaronson has also suggested that theoretical CS should be seen as analogous to theoretical physics.

On the other hand, I don't know if the argument could be abused to dismiss things like P vs NP as having no realistic scientific value. That would be the flip side of Vardi's argument. I do vaguely recall Aaronson in one of his blog posts or pdf essays* giving a more subtle argument about why P vs NP is central to computer science anyways, and I think that would be a slightly different position than either Vardi's or his reading of Widgerson's here.

*See the first several subsections about common Objections in pp 6-7 in: https://www.scottaaronson.com/papers/pnp.pdf

I am a bit confused.

The argument seems to be that 1. a theoretical computer scientist is not a general mathematician and wouldn't be hired to a tenured math position ("sociological argument") and 2. treating it as a branch of math is "harmful" to theoretical computer science.

There are hints about what OP believes TCS is which isn't math, but I wonder why not make that an explicit argument — that would make it much easier to reason about and argue either for or against.

Without that, neither 1 nor 2 make a convincing case for anything: 1. a Linear Algebra expert might not get a position in a Statistics department either, and 2. it'd be useful to show some "harm" before you claim anything being "harmful". CS is also lucrative enough that it pays to have a dedicated faculty just for CS too (as in, it attracts students, contracts with external parties for a university, etc).

>“There is a grave danger that the subject will develop along the line of least resistance.”

What does von Neumann mean here? Why is it bad that it will develop along the line of least resistance? Does von Neumann advice that working on "harder" problems is more beneficial for TCS? Could not one argue that we should be solving away low hanging fruits first?

I am not sure if I am understanding von Neumann's quote nor this article properly. I would love to hear some simpler explanation (I am a new BSc. CS graduate).

I quite like the sociological definition for several reasons. Rather than trying to pin down precise criteria, you simple can ask people "Are you a mathematician? Are you a theoretical computer scientist?" And once someone has gone through that filter, everything that follows is opinion and also a historical snapshot of what people felt the field contained at the time. It provides future theoreticians and students a way to orient themselves on a map which is only partially drawn.

The definition of "theoretical physics" might have rapidly changed between 1900, 1920, 1940, and 1950--but certainly people who called themselves theoreticians remained mostly unchanged. Analyzing how everyone's definitions were changing gives a wealth of information about when and where breakthroughs were happening. 1919 and 1945 come to mind as such examples of when a theoretical field changed as a result of experiments [1][2].

Back to computing: Dijkstra told the story of attempting to put "Programmer" as his profession on his marriage certificate in 1957, and was rejected [3]. Clearly there are both pros and cons of using the sociological definition of a field. We all know programming existed before 1957, but the perception of it as a profession was so foreign that it wasn't allowed on an official document. It would've been impossible, apparently, where he lived to ponder about "What is programming?" if no one could BE a programmer. For that reason, we should probably be flexible and always be willing to discuss different definitions for every field so that we gain the benefits from multiple lines of reasoning.

[1] https://en.wikipedia.org/wiki/Eddington_experiment

[2] https://en.wikipedia.org/wiki/Trinity_(nuclear_test)

[3] https://amturing.acm.org/award_winners/dijkstra_1053701.cfm

Ah, Professor Vardi, a fascinating case study in our department. His devotion to the 'science' in computer science is truly something to behold. It's not every day you see someone try to reconcile Turing machines with the second law of thermodynamics ...

Dr. Vardi's Second Law of Thermodynamics for boolean SAT and SMT (Satisfiability Modulo Theory) solvers is truly a marvel of interdisciplinary ambition. In his framework, computational entropy is said to increase with each transition of the Turing machine, as if bits themselves somehow carry thermodynamic weight. He posits that any algorithm—no matter how deterministic—gradually loses "information purity" as it executes, much like how heat dissipates in a closed system. His real stroke of genius lies in the idea that halting problems are not just undecidable, but thermodynamically unstable. According to Dr. Vardi, attempts to force a Turing machine into solving such problems inevitably lead to an "entropy singularity," where the machine's configuration becomes so probabilistically diffuse that it approaches the heat death of computation. This, he claims, is why brute-force methods become inefficient: they aren’t just computationally expensive, they are thermodynamically costly as well. Of course, there are skeptics who suggest that his theory might just be an elaborate metaphor stretched to breaking point—after all, it’s unclear if bits decay in quite the same way as particles in a particle accelerator.

For nature we have many models, physics, chemistry, biology, ..., depending on our needs. None of them are more wrong, but they operate at different scales and are useful in different contexts.

My gripe with theoretical computer science was that if felt like a Newtonian physics level model of digital processes, while an equivalent of biology level models would be needed for/suited to most "real-life computing".

> Theoretical physics is highly mathematical, but it aims to explain and predict the real world. Theories that fail at this “explain/predict” task would ultimately be discarded. Analogously, I’d argue that the role of TCS is to explain/predict real-life computing.

Incidentally, real-life computing depends quite a lot on the laws of the physics of the universe we live in. This has been known quite clearly since the theoretical discovery of quantum computing, with the resultant split of classical complexity theory and quantum complexity theory.

Morever, physics also determines what sort of devices you can make and with what performance. The real-life speed of solving NP-complete problems depends on the absolute and relative performance of transistors, CPUs, memories etc.

Hence, I submit that TCS has the same relationship to Physics as does Mechanical Engineering to Physics.

I’m just a biochemist/engineer who is passionate about other sciences, but IMHO his understanding of mathematics is what is counter-productive. It’s a bizarrely anti-intellectual take from someone who is clearly an intellectual.

Link to the excellent book "Mathematics and Computation" by Avi Wigderson mentioned in the article (PDF): https://www.math.ias.edu/files/Book-online-Aug0619.pdf

Source: https://www.math.ias.edu/avi/book

He says he affiliates a bit with physics. That is what I studied when I was young. Yes, physics attempts to concern itself with the real world. For instance, nobody in their right mind would have anything to do with quantum mechanics if it wasn't how the real world operated. In that sense it seems to me that computer science is much closer to mathematics. The computer is an artificial system constructed to be relatively easy to reason about.

> NP-completeness theory, however, does not explain or predict the unreasonable effectiveness of SAT solvers.

I don't think that's a fair characterisation. Clearly, the SAT subset that SAT solvers solve efficiently is in P, so in that sense complexity theory "explains" it and even "predicts" it: clearly, there are subsets of SAT in P; some simple ones, such as 2SAT, can be easily described.

What complexity theory is yet to do is:

1. Either prove that the subset of SAT that can be solved efficiently covers all of SAT (in which case either P=NP or NP can be solved with such a low exponent that makes it tractable in practice) or identify and succinctly describe the subset of SAT that modern solvers solve efficiently, and it is in that sense that it doesn't yet "predict" it. But there are many open questions (including P vs NP) in complexity theory; that doesn't mean that the discipline is flawed. Many problems studied in complexity theory originate from practical questions, such as cryptography, and complexity theory does successfully and usefully explain and predict such real-world "phenomena".

2. Explain why that subset of SAT is prevalent in practice (making solvers "effective"). I'm not sure this is the job for complexity theory or any theoretical discipline for that matter (as opposed to empirical ones); after all theoretical computer science is meant to be theoretical. But there are mathematical disciplines (e.g. random graph theory) that do describe and even explain and predict mathematical structures that are more likely to arise in "nature".

> U.S. TCS has a quite narrower scope than European TCS, which I find unfortunate.

TCS is generally considered to have two primary subdisciplines: Theory of Computation (ToC, sometimes also referred to as "Theory A"), which is concerned with complexity and algorithms, and Theory of Programming (ToP, sometimes also referred to as "Theory B"), which is concerned with programming languages and type systems (and may be considered a sub-discipline of formal logic). Indeed, US ToC is more prominent in the US while ToP is more prominent in Europe, but I don't think some of the other CS sub-disciplines Moshe mentions (such as databases) are commonly considered theoretical computer science.

I don't have a position on the utility of describing theoretical computer science as a branch of mathematics or not doing so, but the very name "theoretical computer science" acknowledges that there are other sub-disciplines in computer science that are empirical rather than theoretical (such as studying the mistakes programmers empirically make, or the common causes for cascading failures, or studying side-channel attacks etc. etc.).

I don't think that those who consider TCS to be a branch of mathematics also consider all of CS to be a branch of mathematics or claim that theoretical computer science should aim to answer all interesting questions in computing, just as theoretical physics acknowledges the necessity of experimental physics and doesn't claim to cover everything physicists do.

As in theoretical physics and mathematics, results in theoretical computer science do sometimes explain and predict things we observe in the real world and sometimes not yet. But in all these cases, the theoretical subdisciplines aren't wholly subservient to the empirical ones or vice-versa.

As an example, the question of computability (decidability) isn't directly practical (as many decidable problems arent practically tractable), but the more practical question of tractability/complexity directly evolved from the study of computability.

> Theoretical physics is highly mathematical, but it aims to explain and predict the real world. Theories that fail at this “explain/predict” task would ultimately be discarded.

Still waiting for theoretical physics to discard string theory, as it fails in predicting anything.

I’m late to the party but I’ll drop the truth at the bottom: computer science in its own right is just modern philosophy of mind. It’s the specification of the conceptual limits of cognition.

I love his “sociological sense”, and agree that at some level all these words are just words in language games that are defined with great diversity across the world’s academics, much less engineers and laypeople. But I also agree that there’s something worth defending in “computer science in its own right”, as I said above; that is, in turn, the core task of philosophy of science.

Of course I would expect him to push back on this description of the field because philosophy has lost all its street cred, but that would only encourage me to redouble my efforts. If it’s not empirical study of actual results, and it’s not mathematical study of intuitively-based theorems, what else is left?

My master's degree in computer science technically says, "Theoretical computer science" as the specialization. I was a mathematics undergrad, and generally enjoy doing proofs, albeit I'm not the best at them.

With that said, I regard TCS as the study of discretized computation at scale. Or, more colloquially, what is the fastest way to do X, N times? This could be sorting, searching, indexing, drawing, tracking, balancing, storing, accessing, saving, reading, writing, whatever. The distinguishing characteristic is complexity analysis in time and space.

As a counterexample, I think cryptographic methods such as AES and RSA are much more pure math / number theory than TCS.

I like thinking of CS theory as "math, with more hand-waving". Or, I can't remember where I read it, but something about CS being the mathematics of asymptotes.

This: https://recursion.is

I don’t know. It seems like a total mess to me and everyone deeply immersed in it a broken person. Horrible science. Horrible. Look how broken the society is when everyone gets a computer. Awful. Just awful. But can’t stop what’s coming, but I also don’t have to pretend CS is coherent or interesting or even a science (it’s not).

Interesting topic! Completely disagree with his take though.

> The centrality of computing stems from the fact that it is a technology that has been changing the world for the past 80 years, ever since the British used early computing to change the tide of war in World War II.

I take issue with the idea hinted at here. Just like Algebra and other branches of mathematics were invented to deal with daily down-to-earth issues like accounting or farming, you'd be hard pressed to find a consensus that mathematics "aims to explain the real world".

The historical origin is very clear, but the train left the "real world" station pretty fast, "unreasonable effectiveness" notwithstanding. Am I to understand that because Enigma was broken using a physical machine, the field is bound to study physical reality? To me this feels as uncomfortable as to refer to astronomy as "telescope studies".

> I believe that thinking of TCS as a branch of mathematics is harmful to the discipline. [...] > Theories that fail at this “explain/predict” task would ultimately be discarded. Analogously, I’d argue that the role of TCS is to explain/predict real-life computing

Yeah, if you hired me to design harmful approaches, not in a year I would have come up with something as harmful as this.

I have often argued that computer scientists should be more interested in FPGAs and ASICs. Does the notion of Turing complete apply to a stack of LUTs? Look up tables are of course a nice mathematical generalisation. And how does that relate to prolog?

it's when I'm a computer scientist, theoretically