Better camera alignment to 3DGS

[hbb1]

Motivation

There is feedback (issue1, issue2) that 2DGS doesn't works for non-ideal pinhole camera, while 3DGS works by modifying the projmat. For better alignment, I change the code accordingly. I keep minimal changes to the original code and validate it through some experiments.

Method

1. 3DGS uses ndc projection matrix while 2DGS uses intrinsic matrix. For better alignment, I change the intrinsic to a projection matrix and simplify the projection process. Now it should work for non-ideal pinhole cameras by modifying the projmat.

2. I also implemented and tested trans_precomp so people can use pytorch for pre-compute transformation matrix.
   Disadvantage: not well aligned to cov3D_precomp, where only scaling and rotations are involved.
   Advantage: trans_precomp involves the camera outside the render, so it natively supports gradient flow to the camera using Pytorch. This would be useful for projects that require camera pose optimization.

Experiments

test on the data nerf-synthetic chair, without regularizations, using an RTX 3090.

EXP	PSNR	Time
previous	35.29	12 min
new	35.35	12 min
new (pre_comp)	35.35	21 min

Conclusion

• The projection alignment is good. It produces identical results to the previous, and the code is more readable.
• The trans_precomp does not support normal rendering currently. One possible minimal change is to rewrite the trans_precomp to a3x4 matrix and do so to transMat. But this requires refactoring the geometry buffer for clarity. Leave to future works.

Future works

Improve trans_precomp to support normal rendering.