Guide to Machine Learning with Geometric, Topological, and Algebraic Structures

The paper's references have some good ones for getting more acquainted with these subjects; this one being a nice dense one to start with:

- Geometric Deep Learning Grids, Groups, Graphs, Geodesics, and Gauges: https://geometricdeeplearning.com/

Great selection of works, but I am missing a lot of references from topology in ML, with the article only assuming a very cursory perspective in terms of 'topology captures connectivity and/or continuity.'

Some works from my colleagues and me go a little bit deeper (no pun intended), for instance:

- Neural Persistence Dynamics: https://arxiv.org/abs/2405.15732

- Simplicial Representation Learning with Neural $k$-Forms: https://openreview.net/forum?id=Djw0XhjHZb

- A general review on topology in machine learning: https://www.frontiersin.org/journals/artificial-intelligence...

There are more things in topology and machine learning, Horatio, than are dreamt of in your article ;-)

One common theme I see in the paper(e.g. in protein folding) is:

"Identify what properties are important (geometry, algebra, topo) and which one is an useful prior and then "use" the guide to select an initial struct. This is probably harder than it sounds(unlike bayesian priors which are more forgiving for one to select, but quite like them in that they both require special assumptions)."

I wonder: could one use it to bring together certain multimodal data and a proposed network for a task? Like could one bring in sensor, map topology, urban topology, pictures which have certain properties and that help me use this guide to make a statement like : "Street data could be embedded with Sensor data to do ABC kind of inference using XYZ NNetwork structure because this paper suggests that is a reasonable thing to do"?

I am 100% convinced that these kind of approaches will be what delivers ML research from the current resource-hungry and ungeneralizable status quo. Low-dimensional Euclidean geometry is special. Higher-dimensional Euclidean spaces are less special. Most real-life data is high-dimensional, not at all smooth, and possessing a structure you cannot call Euclidean with a straight face. Look at what works with tabular data (which is probably most of what practitioners work with in the wild). It's gradient boosted trees, not neural networks.

There is a fundamental mismatch between the data we usually work with and the spaces we shove it into. Tools from algebraic topology and geometry are old hat in physics. If anything, they should be even more useful in ML.

Is geometric, topological, and algebraic ML/data analysis actually used in the industry? It is certainly beautiful math. However, during grad school I met a few pure math PhD students who were saying that after finishing their PhD they will just go into industry to do topological data analysis (this was about 10 years ago and ML wasn't yet as hyped up). However, I have never heard of anybody actually having success on that plan.

some people on this thread are asking about jobs. The bigger picture here is that previously intractable problems are going to be solved with a new combination of math, data and compute.. there are lots of commercial cases that will change dramatically. How can individual people or small groups benefit from serious problem solving, economically?

Note that the GPU hardware is setup for Euclidean matrix operations. Even if you had a deep structure learner, it won't necessarily help you if you have to go back to emulating it on Euclidean hardware.