MeshAnything – Converts 3D representations into efficient 3D meshes

Nice looking results, hopefully not too cherry-picked. Every 3D model generation paper posted on HN has people complaining that the meshes are bad, so this kind of research is welcome and necessary for generated 3D assets to be used in actual games.

Weird custom non-commercial license unfortunately. Notes from the GitHub readme:

> It takes about 7GB and 30s to generate a mesh on an A6000 GPU

> trained on meshes with fewer than 800 faces and cannot generate meshes with more than 800 faces

Certainly a lot of scope for this kind of thing .. people who do lidar scans or photogrammetry of buildings tend to end up with very large meshes or very large point clouds.. which means they need souped up PCs and expensive software to wrangle into some usable CAD format.

Its an area where things can be improved a lot imho - I did some work a while back fitting flat planes to pointclouds, and ended up with mesh model anything from 40x to 100x smaller data than the ptcloud dataset. see quato.xyz for samples where you can compare the cloud, the mesh produced.. and view the 3D model in recent browsers.

My approach had some similarity to gaussian splats... but using only planar regions .. great for buildings made of flat slabs, less so for smooth curves and foliage.

Applying their MeshAnything algo to fine meshes from photogrammetry scans of buildings would be of great benefit - probably getting those meshes down to a size where they can be shared as 3D webgl/threejs pages.

Even deciding on triangle points to efficiently tesselate / cover a planar region with holes etc, is basically a knapsack problem, which heuristics, monte-carlo and ML can improve upon.

Definitely the best result for low polygon creation I've seen. Great job!

Still triangles rather than polygons, but we are getting closer.

The end goal should be:

1) Polygons, mostly 4 sided, rather than triangles.

2) Edge smoothness/creases to separate hard coders from soft corners. (Which when combined with polygons enables SubD support: https://graphics.pixar.com/opensubdiv/docs/subdivision_surfa...)

3) UV for textures that are aligned with the natural flow of textures on those components.

4) Repeating textures (although sometimes not) that work with the UVs and combine to create PBR textures. (Getting closer all the time: https://gvecchio.com/stablematerials/)

After the above works, I think people should move on to inferring proper CAD models from an image. Basically infer all the constraints and the various construction steps.

I am all in for any development in this domain. Just to spread some sense of scale, We recently processed (manually) the point cloud scan of one of the (<1% of whole complex) working Oil Refinery. The total volume of point cloud was 450GByte. Our previous project of slightly larger scope was 2.1TByte.

So the scale shown in this paper feels like toys! Not undermining the effort at all. We need to start somewhere anyway.

For the same reason, I feel puzzled looking at Industrial scenes in Video Games. They are like 3 order of magnitude simplified compared to a real plant.

Calling AI-generated meshes "Artist-created" just because it aims to look similar as human-made ones is misleading.

I like how the Social Impact paragraph notes reduced labor costs, yay! Not e.g., reduced need for artists, so you're all out of a job.

MeshAnything generates meshes with hundreds of times fewer faces, significantly improving storage, rendering, and simulation efficiencies, while achieving precision comparable to previous methods.

The converted meshes are not efficient. They are also full of n-gons so you need to retopo no matter what...

Very good, hope they realise that you need tessellation for shading. Some of those models look a bit too optimised.

The topology is decent but no artist is creating meshes like this. The name feels mismatched. I’ve seen some better topology generation papers at siggraph last year which addressed quads better, though I’d need to dig through my archive to find it.

The triangle topologies in this paper made don’t follow the logical loops that an artist would work as. Generally it’s rare an artist would work directly in triangles, versus quads. But that aside, you’d place the loops in more logical places along the surface.

The face and toilet really stand out to me as examples of meshes that look really off.

Anyway, I think this is a good attempt at a reasonable topology generation, but the tag line is a miss.

Words have meanings, you can’t call AI generated meshes, “Artist created Meshes” not matter how good you think your results are.

Beside good topology is dependent on the use case, it’s very different if you are doing animation, a 3D print, a game or just a render.

Hm. I tried the online demo,

https://huggingface.co/spaces/Yiwen-ntu/MeshAnything

on the provided sample "hat". I tried with and without checking "Preprocess with marching cubes" and "Random Sample". Both outputs had holes in the output mesh where the original did not.

Am I doing this wrong, or is the algorithm buggy?

You can run it yourself here: https://huggingface.co/spaces/rerun/InstantMesh

Calling these meshes "Artist-Created Meshes" is disgusting. I know researchers in this field want the word "artist" to follow the same fate as "computer" thanks to their work, but it's too soon to say the least. Can we get AI researchers? I bet RLHF can make their writing more humble than the current ones.

Sentiments aside, that's an impressive approach.

Looks interesting, I do have few complicated models will test it out and see.

Why do people keep making their own special licenses?

https://github.com/buaacyw/MeshAnything/blob/main/LICENSE.tx...

Hugh Hoppe is rolling in his grave.

Stunning!

the service needs to be better, please improve and errors in the configuration of the website