Skip to content

[Resources] Add memory in resources#1746

Merged
Michaelvll merged 22 commits intomasterfrom
memory-filtering
Mar 15, 2023
Merged

[Resources] Add memory in resources#1746
Michaelvll merged 22 commits intomasterfrom
memory-filtering

Conversation

@Michaelvll
Copy link
Copy Markdown
Collaborator

@Michaelvll Michaelvll commented Mar 6, 2023

Fixes #1673, this simulates the behavior of cpus in resources, thanks to the great effort from @WoosukKwon in #1622

Added a memory filtering field for the resources:

  1. memory: 16 the memory should be 16GB
  2. memory: 16+ the memory should be more than 16GB
  3. memory: 4x the memory to core ratio should be larger than 4 times. (Not expose this API as it might be confusing)

Logics for choosing the CPU instance to use:

  1. Use the general-purpose CPU instance by default, when the --memory is not specified

  2. Use the cheapest CPU instance that satisfies the requirement, when the --memory is specified.

  3. sky launch -c normal

I 03-07 21:32:21 optimizer.py:712] Considered resources (1 node):
I 03-07 21:32:21 optimizer.py:760] ------------------------------------------------------------------------------------------------
I 03-07 21:32:21 optimizer.py:760]  CLOUD    INSTANCE           vCPUs   Mem(GB)   ACCELERATORS   REGION/ZONE   COST ($)   CHOSEN   
I 03-07 21:32:21 optimizer.py:760] ------------------------------------------------------------------------------------------------
I 03-07 21:32:21 optimizer.py:760]  AWS      m6i.2xlarge        8       32        -              us-east-2     0.38          ✔     
I 03-07 21:32:21 optimizer.py:760]  Azure    Standard_D8_v5     8       32        -              eastus        0.38                
I 03-07 21:32:21 optimizer.py:760]  GCP      n2-standard-8      8       32        -              us-central1   0.39                
I 03-07 21:32:21 optimizer.py:760]  Lambda   gpu_1x_a100_sxm4   30      200       A100:1         us-east-1     1.10                
I 03-07 21:32:21 optimizer.py:760] ------------------------------------------------------------------------------------------------
  1. sky launch -c cpus --cpus 32+: general purpose
I 03-07 21:33:55 optimizer.py:760] -----------------------------------------------------------------------------------------------
I 03-07 21:33:55 optimizer.py:760]  CLOUD    INSTANCE          vCPUs   Mem(GB)   ACCELERATORS   REGION/ZONE   COST ($)   CHOSEN   
I 03-07 21:33:55 optimizer.py:760] -----------------------------------------------------------------------------------------------
I 03-07 21:33:55 optimizer.py:760]  AWS      m6i.8xlarge       32      128       -              us-east-2     1.54          ✔     
I 03-07 21:33:55 optimizer.py:760]  Azure    Standard_D32_v5   32      128       -              eastus        1.54                
I 03-07 21:33:55 optimizer.py:760]  GCP      n2-standard-32    32      128       -              us-central1   1.55                
I 03-07 21:33:55 optimizer.py:760]  Lambda   gpu_4x_a6000      56      400       A6000:4        us-east-1     3.20                
I 03-07 21:33:55 optimizer.py:760] -----------------------------------------------------------------------------------------------
  1. sky launch -c compute-optimized --cpus 32+ --memory 32+
I 03-07 21:43:40 optimizer.py:760] ------------------------------------------------------------------------------------------------
I 03-07 21:43:40 optimizer.py:760]  CLOUD    INSTANCE           vCPUs   Mem(GB)   ACCELERATORS   REGION/ZONE   COST ($)   CHOSEN   
I 03-07 21:43:40 optimizer.py:760] ------------------------------------------------------------------------------------------------
I 03-07 21:43:40 optimizer.py:760]  GCP      n2-highcpu-32      32      32        -              us-central1   1.15          ✔     
I 03-07 21:43:40 optimizer.py:760]  Azure    Standard_F32s_v2   32      64        -              eastus        1.35                
I 03-07 21:43:40 optimizer.py:760]  AWS      c6i.8xlarge        32      64        -              us-east-2     1.36                
I 03-07 21:43:40 optimizer.py:760]  Lambda   gpu_4x_a6000       56      400       A6000:4        us-east-1     3.20                
I 03-07 21:43:40 optimizer.py:760] ------------------------------------------------------------------------------------------------
I 03-07 21:43:40 optimizer.py:760] 
  1. sky launch -c memory-optimized --memory 32+
I 03-07 21:44:13 optimizer.py:760] ------------------------------------------------------------------------------------------------
I 03-07 21:44:13 optimizer.py:760]  CLOUD    INSTANCE           vCPUs   Mem(GB)   ACCELERATORS   REGION/ZONE   COST ($)   CHOSEN   
I 03-07 21:44:13 optimizer.py:760] ------------------------------------------------------------------------------------------------
I 03-07 21:44:13 optimizer.py:760]  AWS      r6i.xlarge         4       32        -              us-east-2     0.25          ✔     
I 03-07 21:44:13 optimizer.py:760]  Azure    Standard_E4_v5     4       32        -              eastus        0.25                
I 03-07 21:44:13 optimizer.py:760]  GCP      n2-highmem-4       4       32        -              us-central1   0.26                
I 03-07 21:44:13 optimizer.py:760]  Lambda   gpu_1x_a100_sxm4   30      200       A100:1         us-east-1     1.10                
I 03-07 21:44:13 optimizer.py:760] ------------------------------------------------------------------------------------------------

Tested (run the relevant ones):

  • Any manual or new tests for this PR (please specify below)
    • pytest tests/test_optimizer_dryruns.py with added memory related unit tests.
  • All smoke tests: pytest tests/test_smoke.py
  • Relevant individual smoke tests: pytest tests/test_smoke.py::test_fill_in_the_name
  • Backward compatibility tests: bash tests/backward_comaptibility_tests.sh

@Michaelvll Michaelvll marked this pull request as ready for review March 8, 2023 06:29
@Michaelvll Michaelvll requested a review from WoosukKwon March 8, 2023 06:29
@Michaelvll Michaelvll changed the title [Backend] Add memory in resources [Optimizer] Add memory in resources Mar 8, 2023
@Michaelvll Michaelvll changed the title [Optimizer] Add memory in resources [Resources] Add memory in resources Mar 8, 2023
Copy link
Copy Markdown
Collaborator

@WoosukKwon WoosukKwon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job! Thanks for the PR. Left some minor comments.

@Michaelvll Michaelvll merged commit 704a9dd into master Mar 15, 2023
@Michaelvll Michaelvll deleted the memory-filtering branch March 15, 2023 05:04
juanmichelini pushed a commit to LCA-CI-builds-repair/skypilot that referenced this pull request Jul 10, 2025
* Add support for memory filtering

* Fix memory filtering

* rename to memory_gb

* format

* lint

* linting

* linting

* Rename memory-gb to memory and fix lambda

* reset ux_utils

* Fix instance family for Azure

* Do not expose the 'x' API of the memory to the users

* Add test for memory

* Add test for the instance types chosen

* format

* discard `x` for high level memory APIs

* float equal check

* address comments

* Add cpu types

* fix comment for TPU
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support auto selecting instance types via memory filter

2 participants