Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ __pycache__/
*.hdf5
*.keras
*.csv
!results/*.csv
*.pdf
*.yaml
!config/config_hydra.yaml
Expand Down
8 changes: 5 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ For each dataset sub-folder, there exists a single `json` file containing the in

## Docker

This repository supports the usage of docker. In order to create the docker image using the [dockerfile](dockerfile), simply run the following command (assuming you have docker installed and nvidia cuda container as well):
This repository supports the usage of docker. In order to create the docker image using the [dockerfile](https://github.com/MSD-IRIMAS/DeepRehabPile/blob/main/dockerfile), simply run the following command (assuming you have docker installed and nvidia cuda container as well):
```bash
docker build --build-arg USER_ID=$(id -u) --build-arg GROUP_ID=$(id -g) -t deep-rehab-pile-image .
```
Expand Down Expand Up @@ -80,7 +80,7 @@ matplotlib==3.9.0

## Running the code on a single experiment

For each experiment, our code runs multiple initialization (default 5) of a model on a single fold of a single dataset, reports the evaluation metrics on each initialization, as well as the ensemble performance of all initializations. The results reported in our article are of the ensemble performance.
For each experiment, our code runs multiple initialization (default 5) of a model on a single fold of a single dataset, reports the evaluation metrics on each initialization, as well as the ensemble performance of all initialization. The results reported in our article are of the ensemble performance.

If you wish to run a single experiment on a single dataset, on a single fold of this dataset, using a single model then first you have to execute your docker container to open a terminal inside if you're not inside the container:
```bash
Expand All @@ -90,7 +90,7 @@ Then you can run the following command for example top run LITEMV on the IRDS_cl
```bash
python3 main.py task=classification dataset_name=IRDS_clf_bn_EFL fold_number=0 estimator=LITEMV
```
The code uses [hydra](https://hydra.cc/docs/intro/) for the parameter configuration, simply see the [hydra configuration file](config/config_hydra.yaml) for a detailed view on the parameters of our experiments.
The code uses [hydra](https://hydra.cc/docs/intro/) for the parameter configuration, simply see the [hydra configuration file](https://github.com/MSD-IRIMAS/DeepRehabPile/blob/main/config/config_hydra.yaml) for a detailed view on the parameters of our experiments.

## Running the whole benchmark

Expand All @@ -107,6 +107,8 @@ nohup ./run_regression_experiments.sh &

## Results

All the results are available in ``csv`` format in the [```results/``` folder](https://github.com/MSD-IRIMAS/DeepRehabPile/blob/main/results/)

<img id="img-overview"
src="https://raw.githubusercontent.com/MSD-IRIMAS/DeepRehabPile/main/static/num-params-plot.png"
class="interpolation-image"
Expand Down
220 changes: 220 additions & 0 deletions results/accuracy_ensemble.csv

Large diffs are not rendered by default.

220 changes: 220 additions & 0 deletions results/balanced_accuracy_ensemble.csv

Large diffs are not rendered by default.

61 changes: 61 additions & 0 deletions results/flops.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
dataset,LITEMV,ConvTran,MotionGRU,FCN,H-Inception,DisjointCNN,STGCN,ConvLSTM,VanTran
IRDS_clf_bn_EFL,14335190,17401276,21373602,40776716,64577930,111219212,752591132,679721996,820161676
IRDS_clf_bn_EFR,17972280,21886456,26715042,50969996,80720720,139019084,940738832,898642316,1043723836
IRDS_clf_bn_SAL,19589498,23998020,29207714,55726860,88254022,151992204,1028541092,1000805132,1150588300
IRDS_clf_bn_SAR,18155574,22187388,27071138,51649548,81796906,140872332,953282012,913237004,1058891404
IRDS_clf_bn_SFE,22747760,27950696,33836962,64561036,102244440,176085580,1191602432,1190536076,1353331516
IRDS_clf_bn_SFL,26568144,32871208,39534498,75433868,119463416,205738700,1392293312,1424051084,1610500924
IRDS_clf_bn_SFR,25320074,31326948,37754018,72036108,114082486,196472076,1329577412,1351077644,1529229964
IRDS_clf_bn_STL,15769114,19188868,23510178,44854028,71035046,122339084,827850212,767290124,908697484
IRDS_clf_bn_STR,15769114,19188868,23510178,44854028,71035046,122339084,827850212,767290124,908697484
KERAAL_clf_bn_CTK,67572098,89140828,98714274,188690444,297731282,516730508,1361853884,3931618316,4754540428
KERAAL_clf_bn_ELK,72280024,95949896,105509282,201679756,318226810,552301260,1455604672,4216111244,5174126396
KERAAL_clf_bn_RTK,64634570,84885004,94422690,180486668,284786738,494264588,1302642860,3751938572,4495663372
KERAAL_clf_mc_CTK,67572238,89140904,98714406,188690968,297731806,516731032,1361854216,3931618840,4754541464
KERAAL_clf_mc_ELK,72280164,95949972,105509414,201680280,318227334,552301784,1455605004,4216111768,5174127432
KERAAL_clf_mc_RTK,64634710,84885080,94422822,180487192,284787262,494265112,1302643192,3751939096,4495664408
KIMORE_clf_bn_LA,127332988,214522256,246484642,461497996,761012445,1200443404,6108061532,7556007692,17840186236
KIMORE_clf_bn_LT,140107960,243112852,271302306,507965708,837637844,1321313420,6723080108,8331204620,20595602188
KIMORE_clf_bn_PR,148750516,263021160,287960738,539156364,889071331,1402445580,7135900796,8851542284,22543549756
KIMORE_clf_bn_Sq,97838236,153583280,189370018,354558604,584669061,922276108,4692676316,5771992844,12165549436
KIMORE_clf_bn_TR,142782620,249146032,276401826,517513868,853382789,1346149772,6849453788,8490491660,21183516028
KINECAL_clf_bn_3WFV,61430428,80609120,91524514,174648460,276586572,476324044,3223597592,3554875532,4346578300
KINECAL_clf_bn_GGFV,70685854,94226828,105412258,201150988,318557826,548603788,3712781612,4124068364,5196236044
KINECAL_clf_bn_QSEC,141362958,209480940,210816674,402298380,637108882,1097187852,7425562892,8444096012,13277490700
KINECAL_clf_bn_QSEO,141658492,209905952,211172770,402977932,638185068,1099041100,7438106072,8458690700,13309682044
SPHERE_clf_bn_WUS,31780406,43419444,71281826,132279564,222202114,336231564,1463516372,1787768588,3464541196
UCDHE_clf_bn_MP,9869996,15207328,50044066,89957516,161275479,209038348,561434380,504677900,2463623036
UCDHE_clf_bn_Rowing,9869996,15207328,50044066,89957516,161275479,209038348,561434380,504677900,2463623036
UCDHE_clf_mc_MP,9870136,15207404,50044198,89958040,161276003,209038872,561434712,504678424,2463624072
UCDHE_clf_mc_Rowing,9870206,15207442,50044264,89958302,161276265,209039134,561434878,504678686,2463624590
UIPRMD_clf_bn_DS,17204876,21129568,28291746,53554316,86257109,143280524,866911580,870956300,1134849916
UIPRMD_clf_bn_HS,14664668,17899984,24101538,45620876,73479281,122056460,738480284,716265740,953091964
UIPRMD_clf_bn_IL,16358140,20049200,26895010,50909836,81997833,136205836,824101148,819392780,1073737084
UIPRMD_clf_bn_SASLR,13393284,16298152,22006434,41654156,67090367,111444620,674264636,638920460,863991100
UIPRMD_clf_bn_SL,18051612,22213776,29688482,56198796,90516385,150355212,909722012,922519820,1196489596
UIPRMD_clf_bn_SSA,15673728,19241444,25847458,48926476,78803376,130899724,791993324,780720140,1028248204
UIPRMD_clf_bn_SSE,14240020,17365080,23403170,44298636,71349643,118519308,717075068,690483980,923259964
UIPRMD_clf_bn_SSIER,15673728,19241444,25847458,48926476,78803376,130899724,791993324,780720140,1028248204
UIPRMD_clf_bn_SSS,13980256,17097988,23053986,43637516,70284824,116750732,706372460,677593100,908393356
UIPRMD_clf_bn_STS,18636024,23029452,30736034,58182156,93710842,155660940,941829836,961192460,1243065100
EHE_reg_BWL,25853713,30570297,18995803,39296513,51837065,127250945,534205721,1094873345,564354177
EHE_reg_BWR,20933521,24714841,15379035,31812097,41964577,103016193,432452281,788803841,452380033
EHE_reg_HUD,46288705,54886413,33914971,70169729,92561078,227218817,953938661,2357410049,1048523313
EHE_reg_WB,52914769,63049865,38888027,80460801,106135749,260541313,1093849641,2778255617,1217881601
EHE_reg_WF,49223345,58593473,36175451,74847489,98731383,242365441,1017534561,2548703489,1125010433
EHE_reg_WH,30299361,35706421,22160475,45845377,60475492,148456065,623239981,1362684161,664060273
KIMORE_reg_LA,127332913,214522213,246484571,461497729,761012178,1200443137,6108061361,7556007425,17840185713
KIMORE_reg_LT,140107885,243112809,271302235,507965441,837637577,1321313153,6723079937,8331204353,20595601665
KIMORE_reg_PR,148750441,263021117,287960667,539156097,889071064,1402445313,7135900625,8851542017,22543549233
KIMORE_reg_Sq,97838161,153583237,189369947,354558337,584668794,922275841,4692676145,5771992577,12165548913
KIMORE_reg_TR,142782545,249145989,276401755,517513601,853382522,1346149505,6849453617,8490491393,21183515505
UIPRMD_reg_DS,17204801,21129525,28291675,53554049,86256842,143280257,866911409,870956033,1134849393
UIPRMD_reg_HS,14664593,17899941,24101467,45620609,73479014,122056193,738480113,716265473,953091441
UIPRMD_reg_IL,16358065,20049157,26894939,50909569,81997566,136205569,824100977,819392513,1073736561
UIPRMD_reg_SASLR,13393209,16298109,22006363,41653889,67090100,111444353,674264465,638920193,863990577
UIPRMD_reg_SL,18051537,22213733,29688411,56198529,90516118,150354945,909721841,922519553,1196489073
UIPRMD_reg_SSA,15673653,19241401,25847387,48926209,78803109,130899457,791993153,780719873,1028247681
UIPRMD_reg_SSE,14239945,17365037,23403099,44298369,71349376,118519041,717074897,690483713,923259441
UIPRMD_reg_SSIER,15673653,19241401,25847387,48926209,78803109,130899457,791993153,780719873,1028247681
UIPRMD_reg_SSS,13980181,17097945,23053915,43637249,70284557,116750465,706372289,677592833,908392833
UIPRMD_reg_STS,18635949,23029409,30735963,58181889,93710575,155660673,941829665,961192193,1243064577
Loading
Loading