Singular Matrix

[LennyAharon]

since we are simply trying to add very small epsilon and our variance is very large we end up dealing with a singular matrix issue in some cases. The solution will be the following:

under stats.py add regulation term to make sure matrix is invertible

    B = np.zeros((v.shape[0], W.shape[1], W.shape[1]))
    n_latent_dim = W.shape[1]
    for i in range(v.shape[0]):
        # Add regularization term to ensure matrix is invertible
        matrix_to_invert = W.T @ np.diag(1.0 / (v[i] + epsilon)) @ W + epsilon * np.eye(n_latent_dim)
        B[i] = np.linalg.inv(matrix_to_invert)

Also add this fix to the last part

    M = {view_idx: np.zeros((v.shape[0], 1)) for view_idx in range(num_views)}
    for i in range(v.shape[0]):
        for view_idx in range(num_views):
            idxs = slice(2 * view_idx, 2 * (view_idx + 1))
            # Add regularization to ensure Q matrix is invertible
            Q_reg = Q[view_idx][i] + epsilon * np.eye(2)
            M[view_idx][i] = diff[i, idxs].T @ np.linalg.inv(Q_reg) @ diff[i, idxs]

[themattinthehatt]

@LennyAharon sorry for the slowness, getting to this now. Can you say more about how you're getting this error? You say it has to do with having large variances; however, the test below, which has extremely large variances, passes with the current code.

def test_compute_mahalanobis_singular_matrix_large_variance():
    """Test compute_mahalanobis behavior with a variance matrix that would be singular."""

    # Create input data
    N = 5  # Number of samples
    C = 2  # Number of views (each view has 2D coordinates)

    np.random.seed(42)
    x = np.random.randn(N, 2 * C)  # Random observations

    # Variance with all zeros (this will cause a singular matrix)
    v = 1e20 * np.ones((N, 2 * C))

    # 1. Check that function issues a warning with epsilon=0 (singular matrix)
    # with pytest.warns(RuntimeWarning):
    compute_mahalanobis(x, v, epsilon=0, v_quantile_threshold=None)

    # 2. Check that function succeeds with nonzero epsilon
    # with warnings.catch_warnings():
    #     warnings.simplefilter("error")  # Treat warnings as errors to ensure none are raised
    result = compute_mahalanobis(x, v, epsilon=1e-6, v_quantile_threshold=None)

    # Check that outputs are correctly shaped
    assert 'mahalanobis' in result
    assert 'posterior_variance' in result
    assert 'reconstructed' in result

    for view_idx in range(C):
        assert result['mahalanobis'][view_idx].shape == (N, 1)
        assert result['posterior_variance'][view_idx].shape == (N, 2, 2)

    assert result['reconstructed'].shape == (N, 2 * C)

Can you give me some values of W and v that are failing for you?