Fix/openrouter rate limit and dmr #196
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…s8mGAn5UghuFemm5ydF3 Claude/add openrouter support 01 c fs8m g an5 ughu femm5yd f3
CONFIRMED: GPU (CUDA) is currently only used for Analysis, not Clustering RESEARCH FINDINGS: - GPU acceleration can provide 10-30x speedup for clustering tasks - KMeans: 10-50x faster, DBSCAN: 5-100x faster, PCA: 10-40x faster - Example: 5000 clustering runs that take 2-4 hours on CPU can complete in 5-15 minutes on GPU IMPLEMENTATION: Implemented GPU-accelerated clustering using RAPIDS cuML as an optional feature: New Features: - GPU-accelerated KMeans, DBSCAN, and PCA using RAPIDS cuML - Automatic fallback to CPU if GPU unavailable or encounters errors - New environment variable USE_GPU_CLUSTERING (default: false) - Maintains all existing clustering options and settings - Compatible with CUDA 12.2+ and NVIDIA GPUs Files Modified: 1. config.py: - Added USE_GPU_CLUSTERING configuration flag 2. tasks/clustering_gpu.py (NEW): - Created GPU clustering module with RAPIDS cuML implementations - GPU-accelerated classes: GPUKMeans, GPUDBSCAN, GPUPCA - CPU-only wrappers: GPUGaussianMixture, GPUSpectralClustering - Factory functions for model creation with GPU/CPU selection - Automatic GPU availability detection and graceful fallback 3. tasks/clustering_helper.py: - Imported GPU clustering module with fallback handling - Updated _perform_single_clustering_iteration to use GPU PCA when enabled - Modified _apply_clustering_model to support GPU clustering - Maintains full backward compatibility with CPU-only mode 4. Dockerfile: - Added cupy-cuda12x and cuml-cu12 installation for NVIDIA builds - Only installs GPU packages when BASE_IMAGE is nvidia/cuda - CPU builds remain unchanged and lightweight 5. deployment/.env.example: - Added USE_GPU_CLUSTERING configuration with documentation - Default: false (CPU only, backward compatible) 6. README.md: - Added "GPU Acceleration for Clustering" section - Documented performance improvements and usage instructions - Listed supported algorithms and compatibility requirements - Noted that GaussianMixture and SpectralClustering use CPU (no GPU version) Notes: - GPU clustering is OPTIONAL and disabled by default - CPU clustering remains the default for backward compatibility - All existing clustering parameters and settings are preserved - GaussianMixture and SpectralClustering always use CPU (no cuML implementation) - GPU usage: Analysis (ONNX inference) + Clustering (RAPIDS cuML, optional)
…Ju9JeTX1N4popdd3Kt6FoM
The bash -lc subshell prevented access to the BASE_IMAGE ARG, causing GPU packages (cupy, cuml) to never be installed. Changed to use 'set -ux;' pattern (matching base stage) which properly accesses Docker ARG variables in the current shell. This ensures cuML and cupy are installed when building with nvidia/cuda base images, enabling GPU-accelerated clustering.
The new docker image needed packages that where quite large, resulting in a very slow docker build
Feat/gpu clustering
Contributor
Author
|
Ow, this is on top of #195, sorry for the mess |
Owner
|
Is this still for clustering on GPU? can you add this change in the #195 so when finished I can download all in one and do only "one round of test" ? thanks. |
Contributor
Author
|
Included with #195 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
A fix for: