Skip to content

GCS backend: concurrent uploads return 412 PreconditionFailed → daemon panics (v0.1.3-local) #398

@furu-kaggle

Description

@furu-kaggle

Self Checks

To make sure we get to you in time, please check the following :)

  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • "Please do not modify this template :) and fill in all the required fields."

Versions

  1. dify-plugin-daemon Version
    0.1.3-local
  2. dify-api Version
    v1.5.1 (Docker image langgenius/dify:1.5.1)

Describe the bug

With the Google Cloud Storage backend enabled, the daemon tries to upload the same plugin asset twice within ~300 ms.
The second write carries ifGenerationMatch=0 (added by storage.Conditions{DoesNotExist:true}), so GCS returns HTTP 412 Precondition Failed.
The 412 propagates unhandled and triggers an assertion error, which terminates the process.
Cloud Run restarts the container, but the new instance hits the same assertion again → infinite restart loop; the daemon never finishes initialisation.


To Reproduce

  1. Deploy one replica of dify-plugin-daemon:0.1.3-local with GCS storage.
   maxInstances: 1
   containerConcurrency: 1
  1. Cold-start the revision.
  2. Two storage.objects.create calls appear in Cloud Audit Log for the same key within ~1 s; the second ends in 412 PRECONDITION_FAILED.
  3. The pod exits on an assertion error; Cloud Run restarts it; the loop continues indefinitely.

Expected behavior

plugin-daemon should treat “object already exists” (HTTP 412) as an idempotent success, skip the duplicate upload, and continue initialisation instead of stopping on an assertion failure.


Screenshots / Logs

Image

Additional context

  • Failing code in dify-cloud-kit/oss/gcsblob/gcs.goSave():

    obj = obj.If(storage.Conditions{DoesNotExist:true})  // adds ifGenerationMatch=0
    if err := wc.Close(); err != nil {                   // 412 here
        return err                                       // bubbles up, triggers assertion
    }
  • Minimal fix suggestion – intercept 412 and treat as already-exists:

    if err := wc.Close(); err != nil {
        if gErr, ok := err.(*googleapi.Error); ok && gErr.Code == http.StatusPreconditionFailed {
            return nil // idempotent success
        }
        return err
    }

(Removing DoesNotExist:true would also work but drops the immutability guard.)

Happy to open a PR if this direction is acceptable. 🙂

::contentReference[oaicite:0]{index=0}

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions