Covmat is not a positive definite, symmetric square matrix

[ark93-cosmo]

Hello,
When running an MCMC for the Dark Matter-Dark Radiation interaction model implemented in CLASS (with the NADM formalism), every now and then I get an error when resuming the run:

Traceback (most recent call last):
    updated_info, sampler = run(info)
                            ^^^^^^^^^
  File "site-packages/cobaya/run.py", line 132, in run
    sampler = sampler_class(updated_info["sampler"][sampler_name],
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "site-packages/cobaya/sampler.py", line 279, in __init__
    self.initialize()
  File "site-packages/cobaya/samplers/mcmc/mcmc.py", line 192, in initialize
    self.set_proposer_initial_covmat(load=True)
  File "site-packages/cobaya/samplers/mcmc/mcmc.py", line 362, in set_proposer_initial_covmat
    self.proposer.set_covariance(
  File "site-packages/cobaya/samplers/mcmc/proposal.py", line 216, in set_covariance
    raise LoggedError(self.log, "The given covmat is not a positive-definite, "
cobaya.log.LoggedError: The given covmat is not a positive-definite, symmetric square matrix.

However, when I load the .covmat file and check np.allclose(covmat.T, covmat) and np.all(np.linalg.eigvals(covmat) > 0), I find that it's positive-definite and symmetric. One way to solve the problem that seems to work for me is to modify the number of significant figures appearing in the .covmat file (e.g. 9.1234e-4 instead of 9.1234447895e-4), although I don't think this is the way it should work.
Am I missing something here or is this just a numerical instability? Is there a more permanent way to solve this issue?
For completeness, here's an example yaml file (this problem happened with different data combinations):

theory:
  classy:
    extra_args:
      output: tCl,pCl,lCl,mPk
      non linear: hmcode
      lensing: 'yes'
      l_max_scalars: 6000
      compute damping scale: 'yes'
      N_ncdm: 1
      m_ncdm: 0.06
      N_ur: 2.0308
      deg_ncdm: 1
      P_k_max_h/Mpc: 100.0
      accurate_lensing: 1
      delta_l_max: 1800
      hyper_flat_approximation_nu: 7000.0
      hyper_sampling_flat: 32.0
      l_linstep: 20
      l_logstep: 1.025
      l_max_g: 40
      l_max_ncdm: 30
      l_max_pol_g: 60
      l_max_ur: 35
      l_switch_limber: 30.0
      perturbations_sampling_stepsize: 0.05
      radiation_streaming_approximation: 2
      radiation_streaming_trigger_tau_over_tau_k: 240.0
      start_large_k_at_tau_h_over_tau_k: 0.05
      start_small_k_at_tau_c_over_tau_h: 0.0004
      start_sources_at_tau_c_over_tau_h: 0.006
      tight_coupling_trigger_tau_c_over_tau_h: 0.005
      tight_coupling_trigger_tau_c_over_tau_k: 0.008
      tol_ncdm_synchronous: 1.0e-06
      transfer_neglect_delta_k_S_e: 0.17
      transfer_neglect_delta_k_S_t0: 0.17
      transfer_neglect_delta_k_S_t1: 0.05
      transfer_neglect_delta_k_S_t2: 0.17
      ur_fluid_approximation: 2
      ur_fluid_trigger_tau_over_tau_k: 130.0
      m_idm: 1000000000.0
      stat_f_idr: 0.875
      f_idm: 1
      recombination: HyRec
      idm_dr_tight_coupling_trigger_tau_c_over_tau_k: 0.008
      idm_dr_tight_coupling_trigger_tau_c_over_tau_h: 0.005
      hmcode_version: '2020'
    ignore_obsolete: true
    path: /path/to/code/class_public/
likelihood:
  S1920_lite:
    external: &id001 !!python/name:candl.interface.CandlCobayaLikelihood ''
    data_set_file: /path/to/spt3g_candl_library/lib/python3.12/site-packages/spt3g_candl_library/data/1920_TTTEEE_v41/lite/TTTEEE_1920_lite.yaml
    clear_internal_priors: true
    clear_specific_priors: []
    feedback: true
    wrapper: null
    additional_args: {}
  clipy_highl_ttteee:
    external: *id001
    data_set_file: /path/to/cobaya/data/planck_2018/baseline/plc_3.0/hi_l/plik/plik_rd12_HM_v22b_TTTEEE.clik
    clear_internal_priors: false
    clear_specific_priors: []
    feedback: true
    wrapper: clipy
    additional_args:
      all_priors: true
  clipy_lowl_tt:
    external: *id001
    data_set_file: /path/to/cobaya/data/planck_2018/baseline/plc_3.0/low_l/commander/commander_dx12_v3_2_29.clik
    clear_internal_priors: false
    clear_specific_priors: []
    feedback: true
    wrapper: clipy
    additional_args: {}
  planckpr4lensing.PlanckPR4Lensing: {}
  bao.desi_dr2:
    path: /path/to/site-packages/cobaya_packages/data
  muse3glike.cobaya.spt3g_2yr_delensed_ee_optimal_pp_muse:
    components:
    - ϕϕ
params:
  omega_b:
    latex: \Omega_\mathrm{b} h^2
    prior:
      max: 0.1
      min: 0.005
    proposal: 0.0001059155
    ref:
      dist: norm
      loc: 0.022227812
      scale: 5.295775e-05
    renames:
    - ombh2
  omega_cdm:
    latex: \Omega_\mathrm{c} h^2
    prior:
      max: 0.99
      min: 0.001
    proposal: 0.0014695144
    ref:
      dist: norm
      loc: 0.12173822
      scale: 0.0007347572
    renames:
    - omch2
  theta_s_1e2:
    prior:
      max: 10
      min: 0.5
    proposal: 0.0003161778
    drop: true
    ref:
      dist: norm
      loc: 1.0416924
      scale: 0.0001580889
    latex: 100\theta_\mathrm{s}
    renames:
    - theta_MC_100
  logA:
    drop: true
    latex: \log(10^{10} A_\mathrm{s})
    prior:
      max: 3.91
      min: 1.61
    proposal: 0.0082951305
    ref:
      dist: norm
      loc: 3.05777
      scale: 0.00414756525
  n_s:
    latex: n_\mathrm{s}
    prior:
      max: 1.2
      min: 0.8
    proposal: 0.006680365
    ref:
      dist: norm
      loc: 0.94739986
      scale: 0.0033401825
    renames:
    - ns
  tau_reio:
    latex: \tau_\mathrm{reio}
    prior:
      dist: norm
      loc: 0.051
      scale: 0.006
    proposal: 0.00295852905
    ref:
      dist: norm
      loc: 0.050880188
      scale: 0.001479264525
    renames:
    - tau
  100*theta_s:
    latex: 100\theta_*
    derived: false
    value: 'lambda theta_s_1e2: theta_s_1e2'
  H0:
    latex: H_0
    derived: true
  omegamh2:
    derived: 'lambda Omega_m, H0: Omega_m*(H0/100)**2'
    latex: \Omega_\mathrm{m} h^2
  A:
    derived: 'lambda A_s: 1e9*A_s'
    latex: 10^9 A_\mathrm{s}
  A_s:
    derived: true
    latex: A_\mathrm{s}
    renames:
    - As
    value: 'lambda logA: 1e-10*np.exp(logA)'
  Omega_Lambda:
    derived: true
    latex: \Omega_\Lambda
    renames:
    - omegal
  Omega_m:
    derived: true
    latex: \Omega_\mathrm{m}
    renames:
    - omegam
  YHe:
    derived: true
    latex: Y_\mathrm{P}
    renames:
    - yheused
    - yhe
  age:
    derived: true
    latex: '{\rm{Age}}/\mathrm{Gyr}'
  clamp:
    derived: 'lambda A_s, tau_reio: 1e9*A_s*np.exp(-2*tau_reio)'
    latex: 10^9 A_\mathrm{s} e^{-2\tau}
  rs_drag:
    derived: true
    latex: r_\mathrm{drag}
    renames:
    - rdrag
  s8h5:
    derived: 'lambda sigma8, H0: sigma8*(H0*1e-2)**(-0.5)'
    latex: \sigma_8/h^{0.5}
  s8omegamp25:
    derived: 'lambda sigma8, Omega_m: sigma8*Omega_m**0.25'
    latex: \sigma_8 \Omega_\mathrm{m}^{0.25}
  s8omegamp5:
    derived: 'lambda sigma8, Omega_m: sigma8*Omega_m**0.5'
    latex: \sigma_8 \Omega_\mathrm{m}^{0.5}
  sigma8:
    derived: true
    latex: \sigma_8
  S_8:
    derived: 'lambda s8omegamp5: s8omegamp5/np.sqrt(0.3)'
    latex: $S_8$
  Gamma_0_nadm:
    prior:
      min: 1.0e-30
      max: 5.0e-07
    ref:
      dist: norm
      loc: 4.09e-09
      scale: 5.0e-10
    proposal: 1.0e-09
    latex: $\Gamma_\mathrm{0,nadm}$
  N_idr:
    prior:
      min: 1.0e-30
      max: 2
    ref:
      dist: norm
      loc: 0.005449842711
      scale: 0.01
    proposal: 0.02
    latex: $\Delta N_{\mathrm{eff}}$
  Ecal:
    prior:
      max: 1.2
      min: 0.8
    ref: 1.0076405454047297
    latex: E_{\rm cal}
  Tcal:
    prior:
      dist: norm
      loc: 1.0
      scale: 0.0036
    ref:
      dist: norm
      loc: 1.0001004216209306
      scale: 5.0e-05
    latex: T_{\rm cal}
    proposal: 5.0e-05
  A_planck:
    prior:
      min: 0.9855708264
      max: 1.0161611736
    ref:
      dist: norm
      loc: 1.000866
      scale: 0.0021850248
    proposal: 0.0021850248
    latex: y_\mathrm{cal}
  calib_100T:
    prior:
      min: 0.99538027462
      max: 1.00394952538
    ref:
      dist: norm
      loc: 0.9996649
      scale: 0.00061208934
    proposal: 0.00061208934
    latex: c_{100}
  calib_217T:
    prior:
      min: 0.9938432472100001
      max: 1.00257727279
    ref:
      dist: norm
      loc: 0.99821026
      scale: 0.00062385897
    proposal: 0.00062385897
    latex: c_{217}
  A_cib_217:
    prior:
      min: 0.8805411000000021
      max: 93.2510389
    ref:
      dist: norm
      loc: 47.06579
      scale: 6.5978927
    proposal: 6.5978927
    latex: A_cib_217
  xi_sz_cib:
    prior:
      min: 0
      max: 2.5271466499999997
    ref:
      dist: norm
      loc: 0.52615675
      scale: 0.2858557
    proposal: 0.2858557
    latex: xi_sz_cib
  A_sz:
    prior:
      min: 0
      max: 19.0911765
    ref:
      dist: norm
      loc: 5.3853046
      scale: 1.9579817
    proposal: 1.9579817
    latex: A_sz
  ksz_norm:
    prior:
      min: 0
      max: 20.539624
    ref:
      dist: norm
      loc: 3.4545182
      scale: 2.4407294
    proposal: 2.4407294
    latex: ksz_norm
  gal545_A_100:
    prior:
      min: -3.9757127000000008
      max: 21.7868409
    ref:
      dist: norm
      loc: 8.9055641
      scale: 1.8401824
    proposal: 1.8401824
    latex: A^\mathrm{dustTT}_{100}
  gal545_A_143:
    prior:
      min: -1.5442960999999986
      max: 23.3159981
    ref:
      dist: norm
      loc: 10.885851
      scale: 1.7757353
    proposal: 1.7757353
    latex: A^\mathrm{dustTT}_{143}
  gal545_A_143_217:
    prior:
      min: -4.459335499999998
      max: 41.6334315
    ref:
      dist: norm
      loc: 18.587048
      scale: 3.2923405
    proposal: 3.2923405
    latex: A^\mathrm{dustTT}_{\mathrm{143}\times\mathrm{217}}
  gal545_A_217:
    prior:
      min: 42.7048497
      max: 144.6782723
    ref:
      dist: norm
      loc: 93.691561
      scale: 7.2838159
    proposal: 7.2838159
    latex: A^\mathrm{dustTT}_{217}
  ps_A_100_100:
    prior:
      min: 0
      max: 453.039358
    ref:
      dist: norm
      loc: 259.9458
      scale: 27.584794
    proposal: 27.584794
    latex: ps_A_100_100
  ps_A_143_143:
    prior:
      min: 0
      max: 101.06062399999999
    ref:
      dist: norm
      loc: 46.630941
      scale: 7.775669
    proposal: 7.775669
    latex: ps_A_143_143
  ps_A_143_217:
    prior:
      min: 0
      max: 107.02083859999999
    ref:
      dist: norm
      loc: 42.611383
      scale: 9.2013508
    proposal: 9.2013508
    latex: ps_A_143_217
  ps_A_217_217:
    prior:
      min: 0
      max: 185.629737
    ref:
      dist: norm
      loc: 115.05083
      scale: 10.082701
    proposal: 10.082701
    latex: ps_A_217_217
  galf_TE_A_100:
    prior:
      min: -0.15512786699999997
      max: 0.383949987
    ref:
      dist: norm
      loc: 0.11441106
      scale: 0.038505561
    proposal: 0.038505561
    latex: A^\mathrm{dustTE}_{100}
  galf_TE_A_100_143:
    prior:
      min: -0.070685849
      max: 0.340456169
    ref:
      dist: norm
      loc: 0.13488516
      scale: 0.029367287
    proposal: 0.029367287
    latex: A^\mathrm{dustTE}_{\mathrm{100}\times\mathrm{143}}
  galf_TE_A_100_217:
    prior:
      min: -0.11078190500000001
      max: 1.072500125
    ref:
      dist: norm
      loc: 0.48085911
      scale: 0.084520145
    proposal: 0.084520145
    latex: A^\mathrm{dustTE}_{\mathrm{100}\times\mathrm{217}}
  galf_TE_A_143:
    prior:
      min: -0.15561869100000003
      max: 0.604969571
    ref:
      dist: norm
      loc: 0.22467544
      scale: 0.054327733
    proposal: 0.054327733
    latex: A^\mathrm{dustTE}_{143}
  galf_TE_A_143_217:
    prior:
      min: 0.10728795099999999
      max: 1.223596809
    ref:
      dist: norm
      loc: 0.66544238
      scale: 0.079736347
    proposal: 0.079736347
    latex: A^\mathrm{dustTE}_{\mathrm{143}\times\mathrm{217}}
  galf_TE_A_217:
    prior:
      min: 0.20879164000000006
      max: 3.9526629599999996
    ref:
      dist: norm
      loc: 2.0807273
      scale: 0.26741938
    proposal: 0.26741938
    latex: A^\mathrm{dustTE}_{217}
sampler:
  mcmc:
    Rminus1_cl_stop: 0.2
    Rminus1_stop: 0.02
    learn_proposal: true
    learn_proposal_Rminus1_max: 1000
    learn_proposal_Rminus1_max_early: 1000
    max_tries: 10000
    drag: true
    oversample_power: 0.4
    proposal_scale: 0.9
    output_every: 1
    covmat: /path/to/CLASS_DMDR_N0_NADM_S1920lite_Plk_ACTDR6lite_new.covmat
output: CLASS
timing: true

[cmbant]

Are you checking the temporary output covmat/can you attach an example? I thought the CovmatSampler.dump_covmat was writing at quite high default precision.

[ark93-cosmo]

DMDR_SPB_covmat.txt
Attached is the .covmat file when this error occurred. What do you mean exactly by temporary output covmat? Basically, I was resuming the run after the job hit the walltime limit, and then this error occurred.

[cmbant]

When you resume it uses .covmat file produced along with the current chain outputs. When you start it should read your /path/to/CLASS_DMDR_N0_NADM_S1920lite_Plk_ACTDR6lite_new.covmat.
If your chain is converging very slowly, maybe the output covmat does not yet have enough samples to be invertible (i.e. symptom of chain not moving/converging in reasonable time), but the example you attach certainly seems to be plenty of significant figures (but still has a near-zero negative eigenvalue).

[ark93-cosmo]

Since I'm resuming the run, COBAYA used the file attached previously. The MCMC ran for ~ 21hrs, with an acceptance rate of ~ 50% and R-1 is now ~ 1.5. Should I interpret this as a slow convergence?
What I find weird is that when I do the same checks made in proposal.py line 214 on this file, I'm getting True. These are the eignevalues of this matrix:

[2.25020909e+03, 1.95890902e+03, 5.70643737e+02, 1.64960281e+02,
       4.95367084e+01, 1.40279678e+01, 5.05326123e+00, 2.12215293e+00,
       1.21523416e+00, 8.30337791e-01, 2.55322549e-01, 7.95496516e-02,
       6.98534552e-02, 2.65840046e-03, 2.38556684e-03, 1.25454683e-03,
       1.34082670e-03, 5.17637971e-04, 2.64071076e-04, 1.27870413e-04,
       1.57092141e-05, 9.04486851e-06, 4.54283151e-06, 1.31082203e-06,
       3.81132781e-07, 1.25199124e-07, 1.16975402e-07, 4.37115299e-08,
       7.85051629e-09, 3.99955787e-16, 4.77433338e-06]

[cmbant]

On my machine smallest eig is -4.46384937e-15, so clearly hitting machine precision and basically singular.
Maybe you should sample a scaled version of Gamma_0_nadm that is O(1).

[ark93-cosmo]

Indeed I should be doing that, but also the fact that changing the number of sig figs solves this issue hints at some numerical instability. Maybe it has to do with the numpy version users have.
It's not a big issue, but I thought people should be aware about it.
Thanks Antony!