Gaussian Splat

[ir]

Gaussian Splat is a very promising technology - great to have it with VRED 2027 and congratulations to the development team! 

I have tried different approaches and tools with limited success. Most times it breaks down when importing the 3DGS PLY file into VRED with the attached error (which is not really helpful - a readable error message would be good to have).

@pascal: Would you please share a stable process here from capture to rendering in VRED, step by step, incl. the tools used? 

Tx! Ingolf 

[seiferp]

What kind of file is this or where is it from? Try one of these https://superspl.at/?features=downloadable&time=all or sth. from https://developer.xgrids.com/#/download?page=sampledata or https://marble.worldlabs.ai/ I´m using Postshot to create Splats from video or images captured. I also have the PortalCam from XGRIDS which comes with an application called LCC Studio that exports LCC or PLY files.

[ir]

Tx Pascal!  I'll look in to the suggested. 

So far I have tested lidar, photo and video capture with SplatKing (works fine)  => processing with RealityScan(works fine) > VRED Import(mostly breaks down) 

[seiferp]

I don´t think Reality Scan can export Gaussians. Instead it exports point clouds as PLY and the camera alignment data as a JSON file which than can be reconstructed to a Gaussian using Postshot. I have not tried LichtfeldStudio yet, but it seams to be the free alternative to Postshot and might process your RealityKing captures as well.

[michael5KCGS]

Hi! I'm Michael, developer of SplatKing. Just wanted to say thank you for using the app! 

[ir]

Congratulations Michael - fantastic app! 

Having all the bits and pieces, it would now be great if one of you extremely smart professionals could point us users a practicable and stable workflow from capture to rendering in VRED without us having to understand all the details ...   

[seiferp]

It’s actually not very difficult, and the process of generating Gaussians is independent of VRED. You can simply record a video, load it into Postshot, adjust the settings as needed (the default settings usually work quite well), remove any unnecessary parts, and export the final Gaussian as a PLY file.

The more interesting aspect is achieving the best possible quality for the Gaussian. Depending on the quality requirements, a phone may no longer be sufficient, and you might need to use dedicated devices such as the XGRIDS Portal camera. This also depends on whether your goal is to capture individual assets or entire environments. Fortunately, there are many helpful videos and tutorials available on the subject.

[seiferp]

@ir I assume you’ll be attending AIF in Darmstadt this year as well. It could be a great opportunity to discuss the topic in more detail there. XGRIDS will also be represented if the topic of environment splats is of interest to you.

[ir]

Hello Pascal, sure, I'll be at the AIF and more than happy to discuss with the experts.

In the meantime I have discovered an option to export a FBX file from RealityScan for VRED - that is really sweet! 

The Gaussian Splat opens up fantastic opportunities for test driving vehicles in VRED. 

[seiferp]

Colt Chen from XGRIDS will be at AIF and his setup https://www.linkedin.com/posts/activity-7425892455176208384-drUa?utm_source=share&utm_medium=member_... might be from interest for you to discuss.

[michael_nikelsky]

The issue is that PLY is probably the worst specified file format out there as it can just contain about everything in every way without following any specification. So the PLY file you get out of Reality Scan is probably not a PLY file containing gaussian splat data in the common format but either some sort of point cloud or even just normal triangles.