Skip to content

Task tweaks #1149

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
zphang merged 5 commits into from
Oct 13, 2020
Merged

Task tweaks #1149

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
0145286
qasrl
zphang Oct 13, 2020
b555a34
set logic
zphang Oct 13, 2020
dc2cfd1
update readme
zphang Oct 13, 2020
7fbd3d9
ANLI documentation
zphang Oct 13, 2020
3c8e3cd
Merge branch 'master' into task_tweaks
jeswan Oct 13, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,8 +77,8 @@ downloader.download_data(["mrpc"], "/content/data")
# Set up the arguments for the Simple API
args = run.RunConfiguration(
run_name="simple",
exp_dir="/content/exp",
data_dir="/content/data",
exp_dir="/path/to/exp",
data_dir="/path/to/exp/tasks",
model_type="roberta-base",
tasks="mrpc",
train_batch_size=16,
Expand All @@ -91,15 +91,16 @@ run.run_simple(args)

Bash version:
```bash
BASE_PATH=/path/to/exp
python jiant/scripts/download_data/runscript.py \
download \
--tasks mrpc \
--output_path /content/data
--output_path /path/to/exp/tasks
python jiant/proj/simple/runscript.py \
run \
--run_name simple \
--exp_dir /content/data \
--data_dir /content/data \
--exp_dir /path/to/exp \
--data_dir /path/to/exp/tasks \
--model_type roberta-base \
--tasks mrpc \
--train_batch_size 16 \
Expand Down
2 changes: 1 addition & 1 deletion guides/tasks/supported_tasks.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@
| MultiRC | multirc | ✅ | ✅ | multirc | SuperGLUE |
| MRPC | mrpc | ✅ | ✅ | mrpc | GLUE |
| QAMR | qamr | ✅ | ✅ | qamr | |
| QA-SRL | qa-srl | ✅ | ✅ | qa-srl | |
| QA-SRL | qasrl | ✅ | ✅ | qasrl | |
| EP-NER | ner | ✅ | | ner | Edge-Probing |
| PAWS-X | `pawsx_{lang}` | ✅ | ✅ | pawsx | XTREME, multi-lang |
| WikiAnn | `panx_{lang}` | ✅ | ✅ | panx | XTREME, multi-lang |
Expand Down
4 changes: 3 additions & 1 deletion guides/tasks/task_specific.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,9 @@

### Adversarial NLI

[Adversarial NLI](https://arxiv.org/pdf/1910.14599.pdf) has 3 rounds of adversarial data creation. A1/A2/A3 are expanding supersets of the previous round.
[Adversarial NLI](https://arxiv.org/pdf/1910.14599.pdf) has 3 rounds of adversarial data creation. A1, A2 and A3 are different rounds of data creation. When downloading, you can use the task names `adversarial_nli_r1`, `adversarial_nli_r2`, `adversarial_nli_r3` to point the the different rounds.

When doing training on the full ANLI dataset, which is SNLI+MNLI+A1+A2+A3, perform training in a multi-task manner with proportional sampling, and be sure to set the `task_to_taskmodel_map` to have all tasks point to the same NLI head.


### Masked Language Modeling (MLM)
Expand Down
2 changes: 1 addition & 1 deletion jiant/scripts/download_data/runscript.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@

# DIRECT_DOWNLOAD_TASKS need to be directly downloaded because the nlp
# implementation differs from the original dataset format
NLP_DOWNLOADER_TASKS = GLUE_TASKS | SUPERGLUE_TASKS | OTHER_NLP_TASKS - DIRECT_DOWNLOAD_TASKS
NLP_DOWNLOADER_TASKS = (GLUE_TASKS | SUPERGLUE_TASKS | OTHER_NLP_TASKS) - DIRECT_DOWNLOAD_TASKS
SUPPORTED_TASKS = NLP_DOWNLOADER_TASKS | XTREME_TASKS | SQUAD_TASKS | DIRECT_DOWNLOAD_TASKS


Expand Down