update to reflect change in offline branch

[longshuicy]

12/01/2025 update:

This PR now has the Dockerfile to build a special ray python 3.12 image; it's deployed currently on the google k8s engine.
To test:

uv run ray job submit \
  --address http://127.0.0.1:8265 \
  -- \
  python -m darts inference sentinel2-ray \
      --config-file /data/workflows_optimization/rts_ray_pipeline/ray_input/config_remote_ray.toml \
      --start-date 2024-07 \
      --end-date 2024-09 \
      --max-cloud-cover 100 \
      --max-snow-cover 100
Job submission server address: http://127.0.0.1:8265

Details on how to build new images and deployment, you could refer to https://docs.google.com/document/d/1xTs23P5_q-604AvJqK_5ZhSAtSwtPaol3po1tbE0p98/edit?tab=t.0

Right now this will run into GEE error, possibly missing the application credential json in the designated place.

 HttpError: <HttpError 403 when                   
                               requesting                                       
                               https://earthengine-highvolume.googlea           
                               pis.com/v1/projects/pdg-project-406720           
                               /algorithms?prettyPrint=false&alt=json           
                                returned "Caller does not have                  
                               required permission to use project               
                               pdg-project-406720. Grant the caller             
                               the                                              
                               roles/serviceusage.serviceUsageConsume           
                               r role, or a custom role with the                
                               serviceusage.services.use permission,            
                               by visiting     

---

To test:

On local machine, download the model/ geojson / and the needed arcticdem and tcvis then run

uv run darts inference sentinel2-ray --aoi-shapefile data/input/tiles_nwt_2010_2016_small.geojson \
--start-date 2024-07 \
--max-cloud-cover 100 \
--max-snow-cover 100 \
--end-date 2024-09 \
--verbose

My local config

[darts]
ee-project = "pdg-project-406720"

[darts.preprocess]
tpi-outer-radius = 100
tpi-inner-radius = 0

[darts.segmentation]
patch-size = 1024
overlap = 256
batch-size = 2
reflection = 32

[darts.paths]
output-data-dir = "data/output"
model-files=["data/input/s2-tcvis-final-large_2025-02-12.ckpt"]
arcticdem-dir = "data/input/arcticdem"
tcvis-dir = "data/input/tcvis"
overwrite = false # planet inference only
image-ids = ['20210724_194944_103c', '20210731_200455_92_2262'] # planet inference only, remove to process all images

[longshuicy]

This might be problematic and causing the backpressure.

[longshuicy]

I was running into this issue
File "/Users/cwang138/Documents/Projects/PDG/darts-nextgen/darts-utils/src/darts_utils/cuda.py", line 67, in move_to_host
if tile.cupy.is_cupy:
^^^^^^^^^
File "/Users/cwang138/Documents/Projects/PDG/darts-nextgen/.venv/lib/python3.12/site-packages/xarray/core/common.py", line 306, in getattr
raise AttributeError(
AttributeError: 'Dataset' object has no attribute 'cupy'. Did you mean: 'copy'?
hence I added different safeguard

Tobias Hölzer
Nov 13th at 4:50 PM
Ah, maybe try to add "import cupy_xarray " in the function before calling tile.cupy...
🆗
1

4:50
https://github.com/xarray-contrib/cupy-xarray
xarray-contrib/cupy-xarray
Interface for using cupy in xarray, providing convenience accessors.
Website
https://cupy-xarray.readthedocs.io/
Stars
85
Added by GitHub
4:51
Seems like ray doesn't import this properly 😅