3DGS

GaMeS: Mesh-based adapting and modification of Gaussian Splatting

GaMeS: Mesh-based adapting and modification of Gaussian Splatting

Abstract Gaussian Splatting (GS) is a novel, state-of-the-art technique for rendering points in a 3D scene by approximating their contribution to image pixels through Gaussian distributions, warranting fast training and real-time rendering. The main drawback of GS is the absence of a well-defined approach for its conditioning due to the necessity of conditioning several hundred thousand Gaussian components. To solve this, we introduce the Gaussian Mesh Splatting (GaMeS) model, which allows modification of Gaussian components in a similar way as meshes. We parameterize each Gaussian component by the vertices of the mesh face. Furthermore, our model needs mesh initialization on input or estimated mesh during training. We also define Gaussian splats solely based on their location on the mesh, allowing for automatic adjustments in position, scale, and rotation during animation. As a result, we obtain a real-time rendering of editable GS.

Read More
QuantumGS: Quantum Encoding Framework for Gaussian Splatting

QuantumGS: Quantum Encoding Framework for Gaussian Splatting

Abstract Recent advances in neural rendering, particularly 3D Gaussian Splatting (3DGS), have enabled real-time rendering of complex scenes. However, standard 3DGS relies on spherical harmonics, which often struggle to accurately capture high-frequency view-dependent effects such as sharp reflections and transparency. While hybrid approaches like Viewing Direction Gaussian Splatting (VDGS) mitigate this limitation using classical Multi-Layer Perceptrons (MLPs), they remain limited by the expressivity of classical networks in low-parameter regimes. In this paper, we introduce QuantumGS, a novel hybrid framework that integrates Variational Quantum Circuits (VQC) into the Gaussian Splatting pipeline. We propose a unique encoding strategy that maps the viewing direction directly onto the Bloch sphere, leveraging the natural geometry of qubits to represent 3D directional data. By replacing classical color-modulating networks with quantum circuits generated via a hypernetwork or conditioning mechanism, we achieve higher expressivity and better generalization.

Read More
MeshSplats: Mesh-Based Rendering with Gaussian Splatting Initialization

MeshSplats: Mesh-Based Rendering with Gaussian Splatting Initialization

Abstract Gaussian Splatting (GS) is an emerging, pivotal technique in 3D computer graphics. Most GS-based algorithms exclude the use of classical methods such as ray tracing, which offer numerous inherent advantages in rendering. For example, ray tracing can handle incoherent rays for advanced lighting effects, including shadows and reflections. To address this issue, we introduce MeshSplats, a method which converts GS to a mesh-like format. Following the completion of training, MeshSplats transforms Gaussian elements into mesh faces, enabling rendering using ray tracing methods with all their associated benefits. Our model can be used immediately after transformation, yielding a mesh without additional training, with only a slight reduction in reconstruction quality. We can enhance the quality by applying a dedicated optimization algorithm that operates on mesh faces rather than Gaussian components. Importantly, MeshSplats acts as a wrapper, converting pre-trained GS models into a ray-traceable format. The efficacy of our method is substantiated by experimental results, underscoring its extensive applications in computer graphics and image processing.

Read More

VR Escape Room Game Powered by GS-Verse 3DGS Engine

VR Escape Room Game Powered by GS-Verse 3DGS Engine An exciting new project has emerged in the space of real-time 3D rendering and interactive VR gaming: Escape Room (GS-Verse). Built on top of the GS-Verse Gaussian Splatting engine, this VR escape room demonstrates the potential of incorporating 3D Gaussian Splatting (3DGS) representations inside interactive gaming setups.

Read More
LumiMotion: Improving Gaussian Relighting with Scene Dynamics

LumiMotion: Improving Gaussian Relighting with Scene Dynamics

Abstract In 3D reconstruction, the problem of inverse rendering, namely recovering the illumination of the scene and the material properties, is fundamental. Existing Gaussian Splatting-based methods primarily target static scenes and often assume simplified or moderate lighting to avoid entan- gling shadows with surface appearance. This limits their ability to accurately separate lighting effects from mate- rial properties, particularly in real-world conditions. We address this limitation by leveraging dynamic elements– regions of the scene that undergo motion–as a supervisory signal for inverse rendering. Motion reveals the same sur- faces under varying lighting conditions, providing stronger cues for disentangling material and illumination. This the- sis is supported by our experimental results which show we improve LPIPS by 23% for albedo estimation and by 15% for scene relighting relative to next-best baseline. To this end, we introduce LumiMotion, the first Gaussian-based approach that leverages dynamics for inverse rendering and operates in arbitrary dynamic scenes. Our method learns a dynamic 2D Gaussian Splatting representation that em- ploys a set of novel constraints which encourage the dy- namic regions of the scene to deform, while keeping static regions stable. As we demonstrate, this separation is crucial for correct optimization of the albedo. Finally, we release a new synthetic benchmark comprising five scenes under four lighting conditions, each in both static and dynamic variants, for the first time enabling systematic evaluation of inverse rendering methods in dynamic environments and challenging lighting.

Read More
VeGaS: Video Gaussian Splatting

VeGaS: Video Gaussian Splatting

Abstract Modern video representations often prioritize reconstruction quality and compression, but they can be difficult to edit in a precise and controllable way. VeGaS (Video Gaussian Splatting) tackles this gap by adapting Gaussian Splatting ideas to 2D videos while explicitly modeling nonlinear motion and appearance changes across time.

Read More
NegGS: Negative Gaussian Splatting

NegGS: Negative Gaussian Splatting

Abstract One of the key strengths of 3D rendering lies in its ability to simulate intricate scenes with high fidelity. Among the recent advancements, Gaussian Splatting has gained prominence for its fast training and inference capabilities.

Read More