Adding SkyPilot example for FlexGen

README.md

@@ -66,13 +66,20 @@ python3 -m flexgen.apps.helm_run --description mmlu:model=text,subject=abstract_
 ```
 Note that only a subset of HELM scenarios is tested. See more tested scenarios [here](flexgen/apps/helm_passed_30b.sh).
 
-### Run FlexGen on any cloud with SkyPilot
-FlexGen benchmark can be launched with [SkyPilot](http://skypilot.co), a tool for launching ML jobs on any cloud.
-You can use a single command below to automatically launch the benchmark on any cloud with SkyPilot, after you setup your cloud account locally (check how to setup SkyPilot [here](https://skypilot.readthedocs.io/en/latest/getting-started/installation.html)).
+### Run FlexGen on Any Cloud with SkyPilot
+FlexGen benchmark can be launched with [SkyPilot](https://github.com/skypilot-org/skypilot), a tool for launching ML jobs on any cloud.
+First, install SkyPilot and check you have some cloud credentials ([docs](https://skypilot.readthedocs.io/en/latest/getting-started/installation.html)):
+```bash
+pip install "skypilot[aws,gcp,azure,lambda]"  # pick your clouds
+sky check
 ```
-sky launch -c flexgen --detach-setup flexgen/apps/task.yaml
+You can now use a single command to automatically launch the benchmark on any cloud:
+```bash
+sky launch -c flexgen --detach-setup flexgen/apps/skypilot.yaml
 ```
-Note that you can replace the run section with any FlexGen command. You can log into the cluster running the job with `ssh flexgen` and terminate the cluster with `sky down flexgen`.
+You can then log into the cluster running the job with `ssh flexgen` for monitoring. Once the job has finished, you can terminate the cluster with `sky down flexgen` or pass in `--down` flag to the command above to have the cluster terminate itself automatically.
+
+To run any other FlexGen command, you can edit [`flexgen/apps/skypilot.yaml`](./flexgen/apps/skypilot.yaml) and replace the `run` section.
 
 ### Data Wrangling
 You can run the examples in this paper, ['Can Foundation Models Wrangle Your Data?'](https://arxiv.org/abs/2205.09911), by following the instructions [here](flexgen/apps/data_wrangle).

flexgen/apps/README.md

@@ -27,7 +27,16 @@ python3 helm_run.py --description mmlu:model=text,subject=abstract_algebra,data_
 ```
 
 ### Run on any cloud with SkyPilot
-Run FlexGen benchmark on any cloud with [SkyPilot](http://skypilot.co).
+FlexGen benchmark can be launched with [SkyPilot](https://github.com/skypilot-org/skypilot), a tool for launching ML jobs on any cloud.
+First, install SkyPilot and check you have some cloud credentials ([docs](https://skypilot.readthedocs.io/en/latest/getting-started/installation.html)):
+```bash
+pip install "skypilot[aws,gcp,azure,lambda]"  # pick your clouds
+sky check
 ```
-sky launch -c flexgen --detach-setup task.yaml
+You can now use a single command to automatically launch the benchmark on any cloud:
+```bash
+sky launch -c flexgen --detach-setup skypilot.yaml
 ```
+You can then log into the cluster running the job with `ssh flexgen` for monitoring. Once the job has finished, you can terminate the cluster with `sky down flexgen` or pass in `--down` flag to the command above to have the cluster terminate itself automatically.
+
+To run any other FlexGen command, you can edit [`skypilot.yaml`](skypilot.yaml) and replace the `run` section.

flexgen/apps/task.yaml → flexgen/apps/skypilot.yaml

@@ -1,9 +1,11 @@
-# benchmark.yaml
 # A SkyPilot job definition for benchmarking FlexGen.
+# References:
+#   https://skypilot.readthedocs.io/en/latest/getting-started/quickstart.html
+#   https://skypilot.readthedocs.io/en/latest/reference/yaml-spec.html
 
 # Specify the resources required for this job.
 resources:
-  accelerators: T4:1
+  accelerators: T4:1  # Can replace with other GPU type and count, see `sky show-gpus`.
   instance_type: n1-highmem-32 # On GCP with 1 T4 GPU and more than 200GB of RAM.
   # instance_type: g4dn.16xlarge # On AWS with 1 T4 GPU and more than 200GB of RAM.
   # Azure does not support T4 GPUs with more than 200GB of RAM.