Decrease test memory usage

[loganharbour]

Motivation

#32424 enables us to track memory usage and limit memory usage per test. We should set bounds that are reasonable for memory usage for each job.

Design

The following tests should be decreased if possible:

framework test

• fvkernels/mms/advective-outflow.kt/KTLimitedUpwind
• fvkernels/mms/advective-outflow.kt/KTLimitedVanLeer
• mfem/complex.MFEMComplexWaveguide
• mfem/kernels.MFEMDarcy
• mfem/lor.MFEMCurlCurlLOR
• transfers/coord_transform/both-transformed/pp_interpolation.both-transformed

framework heavy

• fvkernels/fv_adapt.steady-mms

python test

• MooseDocs/test/extensions.all/appsyntax [slots=2]

electromagnetics heavy

• auxkernels/azimuthal_Faradays_law.scalar_azim_magnetic_time_deriv_csv
• auxkernels/azimuthal_Faradays_law.vector_azim_magnetic_time_deriv_csv

navier_stokes

• finite_element/ins/cg-dg-hybrid/channel-flow.hybrid-channel [navier_stokes-opt]

porous_flow heavy

• examples/coal_mining.coarse

solid_mechanics

• ics/volume_weighted_weibull.test_finer

stochastic_tools

• multiapps/nested_multiapp.cmd_line/normal
• multiapps/nested_multiapp.parameter_transfer/normal
• samplers/ParallelActiveLearning.*

subchannel heavy

• examples/MultiApp.check

xfem

• mesh_cut_2D_fracture.incline_crack

Impact

Enforces strict resource usage which leads to more reliable testing. Also forces us to write smaller tests.

Jobs to be updated

• Conda mac, Test modules: https://civet.inl.gov/job/3627990/

[10.39s] [1058MB]       OK navier_stokes/test:finite_element/ins/cg-dg-hybrid/channel-flow.hybrid-channel [navier_stokes-opt]
[7.297s] [ 892MB]       OK xfem/test:mesh_cut_2D_fracture.incline_crack [xfem-opt]

• Conda moose mac (Test navier_stokes, Test xfem): https://civet.inl.gov/job/3628052/

[12.02s] [1089MB]       OK finite_element/ins/cg-dg-hybrid/channel-flow.hybrid-channel

• Conda moose mac, Test xfem: https://civet.inl.gov/job/3628052/

[6.922s] [ 941MB]       OK mesh_cut_2D_fracture.incline_crack

• Clang, Test moose test, Test modules: https://civet.inl.gov/job/3627987/

[4.978s] [ 513MB]       OK transfers/coord_transform/both-transformed/pp_interpolation.both-transformed

• Clang, Test modules: https://civet.inl.gov/job/3627987/

[13.82s] [ 941MB]       OK navier_stokes/test:finite_element/ins/cg-dg-hybrid/channel-flow.hybrid-channel [navier_stokes-opt]

• Clang min, Test modules: https://civet.inl.gov/job/3627988

[5.334s] [ 535MB]       OK transfers/coord_transform/both-transformed/pp_interpolation.both-transformed
[9.655s] [ 513MB]       OK mfem/lor.MFEMCurlCurlLOR [pedantic check]

• Conda moose linux, Test navier_stokes: https://civet.inl.gov/job/3627991/

[13.36s] [ 963MB]       OK finite_element/ins/cg-dg-hybrid/channel-flow.hybrid-channel

• Conda moose linux, Test stochastic_tools: https://civet.inl.gov/job/3627991/

[9.488s] [ 964MB]       OK multiapps/nested_multiapp.parameter_transfer/normal
[9.604s] [ 955MB]       OK multiapps/nested_multiapp.cmd_line/normal

• Conda moose linux, Test solid_mechanics: https://civet.inl.gov/job/3627991/

[1.722s] [ 487MB]       OK ics/volume_weighted_weibull.test_finer

• Conda moose linux, Text xfem: https://civet.inl.gov/job/3627991/

[14.15s] [ 756MB]       OK mesh_cut_2D_fracture.incline_crack

• Conda rocky, Test moose test: https://civet.inl.gov/job/3627992/

[7.144s] [ 473MB]       OK transfers/coord_transform/both-transformed/pp_interpolation.both-transformed

• Conda rocky, Test moose test: https://civet.inl.gov/job/3627992/

[21.08s] [ 961MB]       OK navier_stokes/test:finite_element/ins/cg-dg-hybrid/channel-flow.hybrid-channel [navier_stokes-opt]

• Conda rocky, Test modules: https://civet.inl.gov/job/3627992/

[21.08s] [ 961MB]       OK navier_stokes/test:finite_element/ins/cg-dg-hybrid/channel-flow.hybrid-channel [navier_stokes-opt]

• Debug, Test moose test: https://civet.inl.gov/job/3631147/

[16.21s] [ 703MB]       OK mfem/kernels.MFEMDarcy
[30.62s] [ 612MB]       OK transfers/coord_transform/both-transformed/pp_interpolation.both-transformed
[7.048s] [ 500MB]       OK mfem/complex.MFEMComplexWaveguide
[113.0s] [ 467MB]       OK mfem/lor.MFEMGradDivLOR [pedantic check]

• GCC min, Test moose test: https://civet.inl.gov/job/3628004/

[5.167s] [ 595MB]       OK mfem/kernels.MFEMDarcy
[9.620s] [ 552MB]       OK mfem/lor.MFEMCurlCurlLOR [pedantic check]

• GCC min, Test modules: https://civet.inl.gov/job/3628004/

[15.71s] [ 948MB]       OK navier_stokes/test:finite_element/ins/cg-dg-hybrid/channel-flow.hybrid-channel [navier_stokes-opt]

• 32bit dof IDs, Test moose test: https://civet.inl.gov/job/3628065/

[3.183s] [ 585MB]       OK mfem/kernels.MFEMDarcy
[2.947s] [ 538MB]       OK transfers/coord_transform/both-transformed/pp_interpolation.both-transformed

• 32bit dof IDs, Test modules: https://civet.inl.gov/job/3628065/

[6.143s] [ 775MB]       OK navier_stokes/test:finite_element/ins/cg-dg-hybrid/channel-flow.hybrid-channel [navier_stokes-opt]

• Disable HDF5, Test moose test: https://civet.inl.gov/job/3627997/

[5.097s] [ 665MB]       OK mfem/kernels.MFEMDarcy
[4.559s] [ 597MB]       OK transfers/coord_transform/both-transformed/pp_interpolation.both-transformed
[8.755s] [ 587MB]       OK mfem/lor.MFEMCurlCurlLOR [pedantic check]

• Disable HDF5, Test modules: https://civet.inl.gov/job/3627997/

[13.64s] [1010MB]       OK navier_stokes/test:finite_element/ins/cg-dg-hybrid/channel-flow.hybrid-channel [navier_stokes-opt]
[6.473s] [ 727MB]       OK stochastic_tools/test:multiapps/nested_multiapp.parameter_transfer/normal [stochastic_tools-opt]
[6.666s] [ 723MB]       OK stochastic_tools/test:multiapps/nested_multiapp.cmd_line/normal [stochastic_tools-opt]
[13.51s] [ 722MB]       OK xfem/test:mesh_cut_2D_fracture.incline_crack [xfem-opt]

• Framework 1 https://civet.inl.gov/job/3628001/

[10.34s] [ 641MB]    ERROR mfem/lor.MFEMCurlCurlLOR FAILED (KILLED: OVER MEMORY) [max_cpus=1]
[5.459s] [ 476MB]       OK mfem/kernels.MFEMDarcy

• Framework 2: https://civet.inl.gov/job/3628002/

[9.499s] [ 653MB]    ERROR mfem/lor.MFEMCurlCurlLOR FAILED (KILLED: OVER MEMORY) [max_threads=1]

• Framework increased sparsity, Test moose test: https://civet.inl.gov/job/3628003/

[37.07s] [ 649MB]       OK transfers/coord_transform/both-transformed/pp_interpolation.both-transformed

• Heavy, Heavy test moose: https://civet.inl.gov/job/3628006/

[39.13s] [ 828MB]       OK fvkernels/fv_adapt.steady-mms [min_cpus=16,max_parallel=16]

• Heavy, Heavy test modules: https://civet.inl.gov/job/3628006/

[96.94s] [2096MB]       OK electromagnetics/test:auxkernels/azimuthal_Faradays_law.vector_azim_magnetic_time_deriv_csv [implicit max_parallel=1,implicit max_threads=1]
[98.58s] [2090MB]       OK electromagnetics/test:auxkernels/azimuthal_Faradays_law.scalar_azim_magnetic_time_deriv_csv [implicit max_parallel=1,implicit max_threads=1]
[84.02s] [1325MB]       OK porous_flow/examples/coal_mining.coarse
[72.73s] [1029MB]       OK subchannel/examples/MultiApp.check

• Modules, Test modules combined: https://civet.inl.gov/job/3628012/

[15.88s] [ 958MB]       OK navier_stokes/test:finite_element/ins/cg-dg-hybrid/channel-flow.hybrid-channel
[11.97s] [ 946MB]       OK stochastic_tools/test:multiapps/nested_multiapp.parameter_transfer/normal
[10.71s] [ 934MB]       OK stochastic_tools/test:multiapps/nested_multiapp.cmd_line/normal
[18.89s] [ 675MB]       OK xfem/test:mesh_cut_2D_fracture.incline_crack

• Modules, Test navier_stokes: https://civet.inl.gov/job/3628012/

[18.79s] [ 962MB]       OK test:finite_element/ins/cg-dg-hybrid/channel-flow.hybrid-channel

• Modules, Test solid_mechanics: https://civet.inl.gov/job/3628012/

[1.813s] [ 470MB]       OK test:ics/volume_weighted_weibull.test_finer

• Modules, Test xfem: https://civet.inl.gov/job/3628012/

[32.05s] [ 709MB]       OK test:mesh_cut_2D_fracture.incline_crack [FINISHED]

• Modules Standalone, Test navier_stokes and clean: https://civet.inl.gov/job/3628051/

[8.894s] [ 931MB]       OK test:finite_element/ins/cg-dg-hybrid/channel-flow.hybrid-channel

• Modules Standalone, Test solid_mechanics and clean: https://civet.inl.gov/job/3628051/

[0.812s] [ 470MB]       OK test:ics/volume_weighted_weibull.test_finer

• Modules Standalone, Test xfem and clean: https://civet.inl.gov/job/3628051/

[6.955s] [ 700MB]       OK test:mesh_cut_2D_fracture.incline_crack

• Modules debug, Test modules (debug): https://civet.inl.gov/job/3628013/

[43.77s] [1058MB]       OK stochastic_tools/test:multiapps/nested_multiapp.cmd_line/normal
[42.92s] [1058MB]       OK stochastic_tools/test:multiapps/nested_multiapp.parameter_transfer/normal

• Modules parallel, Test navier_stokes *: https://civet.inl.gov/job/3628014/

[50.87s] [ 432MB]       OK test:finite_element/ins/hdg/ip/lid-driven.lid-mms [max_parallel=8,FINISHED,min_cpus=8]

• Modules parallel, Test xfem: https://civet.inl.gov/job/3628014/

[16.59s] [ 521MB]       OK test:mesh_cut_2D_fracture.incline_crack

• Modules recover, Test navier_stokes recover: https://civet.inl.gov/job/3628015/

[18.31s] [ 957MB]       OK test:finite_element/ins/cg-dg-hybrid/channel-flow.hybrid-channel_part1 PART1

• Modules restep, Test navier_stokes restep: https://civet.inl.gov/job/3628016/

[17.94s] [ 959MB]       OK test:finite_element/ins/cg-dg-hybrid/channel-flow.hybrid-channel

• Modules restep, Test solid_mechanics restep: https://civet.inl.gov/job/3628016/

[1.779s] [ 465MB]       OK test:ics/volume_weighted_weibull.test_finer

• Modules restep, Test xfem restep: https://civet.inl.gov/job/3628016/

[28.63s] [ 752MB]       OK test:mesh_cut_2D_fracture.incline_crack [FINISHED]

• No optional, Test moose test: https://civet.inl.gov/job/3628018/

[4.906s] [ 445MB]       OK transfers/coord_transform/both-transformed/pp_interpolation.both-transformed

• No optional, Test modules: https://civet.inl.gov/job/3628018/

[19.40s] [ 941MB]       OK navier_stokes/test:finite_element/ins/cg-dg-hybrid/channel-flow.hybrid-channel [navier_stokes-opt]
[17.08s] [ 658MB]       OK xfem/test:mesh_cut_2D_fracture.incline_crack [xfem-opt]

• Parallel sweep evens: https://civet.inl.gov/job/3628057/

[3.490s] [ 537MB]       OK mfem/complex.MFEMComplexWaveguide [max_cpus=1]
[7.787s] [ 520MB]       OK mfem/lor.MFEMCurlCurlLOR [max_cpus=1,pedantic check]
[4.798s] [ 395MB]       OK mfem/lor.MFEMGradDivLOR [max_cpus=1,pedantic check]

• Parallel sweep odds: https://civet.inl.gov/job/3628053/

[8.958s] [ 612MB]       OK mfem/lor.MFEMCurlCurlLOR [pedantic check,max_cpus=1]
[5.265s] [ 493MB]       OK mfem/lor.MFEMGradDivLOR [pedantic check,max_cpus=1]
[3.528s] [ 467MB]       OK mfem/complex.MFEMComplexWaveguide [max_cpus=1]

• Python, test python: https://civet.inl.gov/job/3631171/

[46.87s] [ 953MB]       OK MooseDocs/test/extensions.all/appsyntax [slots=2]

• Test, Test moose test and store: https://civet.inl.gov/job/3628021/

[5.551s] [ 664MB]       OK mfem/kernels.MFEMDarcy
[9.641s] [ 581MB]       OK mfem/lor.MFEMCurlCurlLOR
[5.240s] [ 548MB]       OK transfers/coord_transform/both-transformed/pp_interpolation.both-transformed
[3.980s] [ 487MB]       OK mfem/complex.MFEMComplexWaveguide
[9.938s] [ 451MB]       OK fvkernels/mms/advective-outflow.kt/KTLimitedVanLeer
[5.651s] [ 447MB]       OK mfem/lor.MFEMGradDivLOR

• Test, Test modules and store: https://civet.inl.gov/job/3628021/

[17.89s] [1016MB]       OK navier_stokes/test:finite_element/ins/cg-dg-hybrid/channel-flow.hybrid-channel [navier_stokes-opt]
[20.07s] [ 733MB]       OK xfem/test:mesh_cut_2D_fracture.incline_crack [xfem-opt]
[9.486s] [ 723MB]       OK stochastic_tools/test:multiapps/nested_multiapp.cmd_line/normal [stochastic_tools-opt]
[7.811s] [ 679MB]       OK stochastic_tools/test:multiapps/nested_multiapp.parameter_transfer/normal [stochastic_tools-opt]

• Valgrind: https://civet.inl.gov/job/3628048/

[246.9s] [3686MB]       OK mfem/kernels.MFEMDarcy
[285.5s] [3501MB]       OK transfers/coord_transform/both-transformed/pp_interpolation.both-transformed
[197.6s] [2469MB]       OK mfem/complex.MFEMComplexWaveguide
[125.0s] [1977MB]       OK transfers/multiapp_variable_value_sample_transfer.array_sample_test/centroid
[136.3s] [1878MB]       OK multiapps/quadrature_point_multiapp.test
[316.0s] [1815MB]       OK mfem/lor.MFEMHeatConductionLOR [pedantic check]

• Valgrind heavy: https://civet.inl.gov/job/3628049/

[512.5s] [2770MB]       OK mfem/lor.MFEMCurlCurlLOR [pedantic check]
[321.4s] [2488MB]       OK mfem/lor.MFEMGradDivLOR [pedantic check]

• Valgrind solid_mechanics: https://civet.inl.gov/job/3628033/

[68.21s] [1885MB]       OK test:ics/volume_weighted_weibull.test_finer

• libMesh disable deprecated, Test moose test: https://civet.inl.gov/job/3628010/

[5.590s] [ 662MB]       OK mfem/kernels.MFEMDarcy
[10.40s] [ 574MB]       OK mfem/lor.MFEMCurlCurlLOR [pedantic check]
[5.313s] [ 550MB]       OK transfers/coord_transform/both-transformed/pp_interpolation.both-transformed
[13.17s] [ 497MB]       OK fvkernels/mms/advective-outflow.kt/KTLimitedUpwind
[4.211s] [ 489MB]       OK mfem/complex.MFEMComplexWaveguide
[5.944s] [ 452MB]       OK mfem/lor.MFEMGradDivLOR [pedantic check]

• libMesh disable deprecated, Test modules: https://civet.inl.gov/job/3628010/

[15.02s] [1013MB]       OK navier_stokes/test:finite_element/ins/cg-dg-hybrid/channel-flow.hybrid-channel [navier_stokes-opt]
[7.165s] [ 728MB]       OK stochastic_tools/test:multiapps/nested_multiapp.parameter_transfer/normal [stochastic_tools-opt]
[16.17s] [ 722MB]       OK xfem/test:mesh_cut_2D_fracture.incline_crack [xfem-opt]

Framework recipes to check

• CUDA GPU
• Distributed mesh sweep evens
• Distributed mesh sweep odds

App recipes to check

• Apptainer moose-openmpi
• Apptainer openmpi controlled apps
• Apptainer openmpi internal apps
• Apptainer openmpi public apps
• Controlled apps
• Internal apps