Adding ROPES, RACE tasks

[zphang]

Ropes

• RoBERTa-base: EM=45.7, F1=53.8
• RoBERTa-large: EM=55.9, F1=62.0

RACE

Results forthcoming.

Other Notes

• RACE isn't publicly downloadable
• Ropes requires tweaking the SQuAD featurization itself because of its special format (the answer span can be within the question itself)

[codecov]

Codecov Report

Merging #1234 (80bd7b8) into master (e5fbea4) will decrease coverage by 0.55%.
The diff coverage is 29.41%.

@@            Coverage Diff             @@
##           master    #1234      +/-   ##
==========================================
- Coverage   56.43%   55.88%   -0.56%     
==========================================
  Files         147      149       +2     
  Lines       10598    10819     +221     
==========================================
+ Hits         5981     6046      +65     
- Misses       4617     4773     +156     

Impacted Files	Coverage Δ
jiant/scripts/download_data/constants.py	100.00% <ø> (ø)
...pts/download_data/dl_datasets/hf_datasets_tasks.py	100.00% <ø> (ø)
jiant/tasks/evaluate/core.py	36.17% <ø> (ø)
...t/scripts/download_data/dl_datasets/files_tasks.py	7.54% <9.09%> (+0.04%)	⬆️
jiant/tasks/lib/ropes.py	20.39% <20.39%> (ø)
jiant/tasks/lib/race.py	55.35% <55.35%> (ø)
jiant/tasks/retrieval.py	100.00% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e5fbea4...80bd7b8. Read the comment docs.

[jeswan]

@zphang Do you have RACE results you can post here?

[HarshTrivedi]

@zphang @jeswan RACE is downloadable (at least now) via HF datasets. I've added it in a different branch.

[zphang]

Following up on Harsh's update, RACE is now downloadable, and the PR has been updated to reflect this.

Results (val):

• RoBERTa-base
  • RACE (middle): 68.2
  • RACE (high): 64.8
  • RACE (all): 66.4
• RoBERTa-large
  • RACE (middle): 85.7
  • RACE (high): 79.7
  • RACE (all): 79.7

Compare to published results, on the test set:

• RACE (middle): 86.5
• RACE (high): 81.3
• RACE (all): 83.2

Both Harsh and I encountered some training instability (requires multiple restarts).