Add local image flag for fast context loading #2236
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Add a new experimental flag —x-localimage for use with —x-fastpush * When activated, the flag uses the .cog tmp directory as an rsync for the source code without weights * It separates the different build steps into different build contexts * On run/predict/train/serve it loads the weights in as volume mounts * This allows it to not reload the context each time a small file is changed for weights
Signed-off-by: Will Sackfield <[email protected]>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> Signed-off-by: Will Sackfield <[email protected]>
* Only if we use build fast
nevillelyh
reviewed
Apr 2, 2025
nevillelyh
approved these changes
Apr 2, 2025
NikhilSinha1
reviewed
Apr 2, 2025
| if imageName, err = image.BuildBase(cfg, projectDir, buildUseCudaBaseImage, DetermineUseCogBaseImage(cmd), buildProgressOutput); err != nil { | ||
| return err | ||
| } | ||
| if buildFast { |
Contributor
There was a problem hiding this comment.
nit: can you just throw this into the block above
if cfg.Build.Fast {
buildFast = true
imageName = config.DockerImageName(projectDir)
} else {
...
}
Contributor
Author
There was a problem hiding this comment.
Unfortunately I don't think we can, config fast build overwrites any CLI flag which is the first block, but in the 2nd block we need to know whether the fast build flag is in general on and overwrite the imageName
NikhilSinha1
reviewed
Apr 3, 2025
NikhilSinha1
reviewed
Apr 3, 2025
NikhilSinha1
reviewed
Apr 3, 2025
|
|
||
| // Rsync local src with our srcdir | ||
| if g.localImage { | ||
| err := g.rsyncSrc(srcDir, weights) |
Contributor
There was a problem hiding this comment.
Could just be me not understanding correctly but what we're doing here
- Symlinking everything in g.Dir to a subdirectory
- Hard link to all the symlinks?
I guess I'm a bit confused why we're doing the rsync first?
Contributor
Author
There was a problem hiding this comment.
Our process is:
- If a local image flag is used, created a src directory in
.cog - Create the necessary hard links and directories to mirror whats in the users current directory (sans weights and dockerignore directives), if it is symlink use the target path
- Tag that as a unique build context so it only changes the docker build when the user changes their code
* Synchronise directories between different golang modules
Contributor
Author
|
Going to merge this to build a new release |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
weights