Add CI for Autoparallel experiment llama3 on 4 GPUs

[xmfan]

Stacked PRs:

• [Autoparallel] Add local_map variant of DSv3 and 2D mesh AP #2129
• Rename auto_parallel experiment to autoparallel #2128
• ->Add CI for Autoparallel experiment llama3 on 4 GPUs #2105

---

Validated with python -m torchtitan.experiments.auto_parallel.tests.integration_tests artifacts-to-be-uploaded --ngpu 4

NGPU=4 LOG_RANK=0,1,2,3 CONFIG_FILE=./tests/integration_tests/base_config.toml TRAIN_FILE=torchtitan.train COMM_MODE= TORCHFT_LIGHTHOUSE=http://localhost:29510 PYTORCH_ALLOC_CONF=expandable_segments:True torchrun --nproc_per_node=4 --rdzv_backend c10d --rdzv_endpoint=localhost:0 --local-ranks-filter 0,1,2,3 --role rank --tee 3 -m torchtitan.train --job.config_file ./tests/integration_tests/base_config.toml --job.dump_folder artifacts-to-be-uploaded/llama3_autoparallel_fsdp_tp --model.name auto_parallel.llama3 --parallelism.data_parallel_shard_degree 2 --parallelism.tensor_parallel_degree 2 --job.custom_config_module=torchtitan.experiments.auto_parallel.job_config

[xmfan]

The pattern across the other workflows seems to still be naming this 8 GPU

[wwwjn]

IIUC it basically says we have at most 8 GPU used in each integration tests (and we only have 8 available)

[xmfan]

This doesn't work yet, I'm thinking of enabling it in a different PR

[tianyu-l]

sgtm, except that the test is not passing?

[xmfan]

Test will pass after meta-pytorch/autoparallel#274

[tianyu-l]

sgtm