[6.x] Enable background re-caching of static cache

[ryanmitchell]

This PR enables background re-caching of the static cache through a new static_caching.background_recache variable (or STATAMIC_BACKGROUND_RECACHE .env value).

When set to true the static cache will re-cache the page without invalidating the previous cache, so at every point a static file will exist and end users will not encounter slow loading pages.

Invalidation continues to occur as it currently does when content is deleted, or when this variable is set to false (the default).

[aerni]

Do I understand correctly, that this will serve the old cached file until the new cached file is ready? And then it gets rid of the old cached file? Or what exactly is the purpose of this? Sounds interesting for sure.

Also, from a configuration point of view, it might be easier to just have the config to be a boolean and then assign a random string automatically if the config is true.

[jasonvarga]

this will serve the old cached file until the new cached file is ready? And then it gets rid of the old cached file?

When the new cached file is ready, it'll just stomp over the top of the old one.

it might be easier to just have the config to be a boolean and then assign a random string automatically if the config is true.

We need to know the value ahead of time, so it can't just be a randomly changing string.

[ryanmitchell]

@aerni the idea of it being a fixed string was to provide some sort of security - that someone malicious couldn’t just constantly force your site to recache.

[aerni]

Personally, I'd be a little intimidated by the instruction to "add a unique string here that should not be easy to guess". Like, what is "easy to guess"? Couldn't we just use the APP_KEY as the unique string?

When the new cached file is ready, it'll just stomp over the top of the old one.

By "stomp over" I suppose you mean the old file will just not be served anymore? Won't this result in a lot of abandoned files pretty fast? Should there maybe be some sort of cleanup?

[ryanmitchell]

Maybe we could make it a Boolean config and the token could be a Crypt or Sha of the url. The app key is used in the Crypt so it wouldn’t be game-able.

[ryanmitchell]

By "stomp over" I suppose you mean the old file will just not be served anymore? Won't this result in a lot of abandoned files pretty fast? Should there maybe be some sort of cleanup?

it literally replaces the same file. So no clean up will be needed.

[aerni]

Maybe we could make it a Boolean config and the token could be a Crypt or Sha of the url. The app key is used in the Crypt so it wouldn’t be game-able.

Oh yeah, makes sense. It's early morning here 😄

[ryanmitchell]

@aerni here you go - its a boolean now. Feels simpler.

[aerni]

Love it, thanks! Curious as to why you decided to make this opt-in, though. Feels like everyone who uses static caching would want to enable this feature. Or is there any potential downside to it?

[ryanmitchell]

I make all my PRs opt in until Jason changes them to not be :)

[Krzemo]

this will serve the old cached file until the new cached file is ready? And then it gets rid of the old cached file?

When the new cached file is ready, it'll just stomp over the top of the old one.

An edge case idea:
In theory, isn't file access concurrency a concern here?
What if file is being read and locked and you try to overwrite it?
What if being written to and server tries to read it?
I realize this is purely theoretic and performance and reliability impact is probably negligible, but ....

[ryanmitchell]

@Krzemo Are those things not already a potential issue with the existing implementation? My understanding is modern filesystems will lock the file during writing, so any read operations will wait until the file is unlocked, so it shouldn't be an issue.

[Krzemo]

@ryanmitchell Im not worried about OS not doing its job.

Maybe asking different questions will help explain what I mean (I feel like I should run tests myself first - will do next week if still needed):

1. Does file get locked when HTML generation starts and freed when done or HTML is ready first and file gets locked for a brief moment only when it is being written? Can a file be locked because of first option for i.e. 1 minute+?
2. Does locking a file for writing make it unreadable? Can it end up in the same situation like with no background refresh where a user have to wait for file generation to finish not because there is no static file, but because file is locked for reading?

[briavers]

What's the status of this @jasonvarga ? We're having the same issues as #8410 and hope this solves it but it's been draging on for a while it seems

[duncanmcclean]

We haven't had time to look at it yet. We're currently focused on Statamic 6.

If you need these changes now, you could try applying this PR as a composer patch.

[jasonvarga]

Forgive the delay on this.

I like the no-config solution to the __recache query string value. But, wouldn't that only work for half measure?

If you're using full measure, you'd need to add something to your nginx to bypass the cached page (which is fine) but we need to make sure that every request that has __recache in it doesn't get through - only the one with the right query string. Which you explained here.

So maybe we have a config option for the token itself. Then you can put $theTokenYouKnow in your nginx.
You'd only need to set a token if you're using full measure.

Does that sound right? (You don't need to implement it. I just want a sanity check.)

Or... if picking a token is the issue, what if we do just use some consistent string and you get the value by running an artisan command?

$ php please static:recache-token
Your recache token is: 647109cf-2b08-45bd-ab25-4fca07e9eba2

Under the hood it's just something like Hash::make('recache') - it'd vary based on your APP_KEY.

[godismyjudge95]

I like the no-config solution to the __recache query string value. But, wouldn't that only work for half measure?

If you're using full measure, you'd need to add something to your nginx to bypass the cached page (which is fine) but we need to make sure that every request that has __recache in it doesn't get through - only the one with the right query string. Which you explained here.

Having just dealt with doing load balancing in nginx it is much easier/reliable to switch an nginx config based off a cookie. Maybe the cookie could get set via a backend call via JS (similar to nocache)? This would make it compatible with full cache too.

[ryanmitchell]

you're using full measure, you'd need to add something to your nginx to bypass the cached page (which is fine) but we need to make sure that every request that has __recache in it doesn't get through - only the one with the right query string. #9396 (comment).

So maybe we have a config option for the token itself. Then you can put $theTokenYouKnow in your nginx.
You'd only need to set a token if you're using full measure.

Does that sound right? (You don't need to implement it. I just want a sanity check.)

If youre using full measure with ignore_query_strings => false it should 'just work' - as the recache query string will bypass the static cache and the page will never be saved as HTML.

If you're using ignore_query_strings => true then it would hit the HTML, but I think it should be easy to update the server rules (eg https://statamic.dev/static-caching#nginx) to not hit the static pages when either the URL param is present or the X-Cache-Refresh header?

In both cases if the token is invalid it would serve the static HTML page if it exists (via the StaticCache middleware)

[jasonvarga]

Whether its a cookie, header, or query param, the server needs to check not only if it has the thing, but that it matches a value. Otherwise anyone could craft a request to invalidate the pages.

if the token is invalid it would serve the static HTML page if it exists

True, but if you enable full measure static caching, the point is to bypass PHP.

@godismyjudge95 I'm curious to hear more about the cookie thing. Why would reading a cookie be any better than a query param?

[godismyjudge95]

@godismyjudge95 I'm curious to hear more about the cookie thing. Why would reading a cookie be any better than a query param?

It could have been just me having a long day but I could not get NGINX in a reverse proxy/node balancer setup to consistently recognize a query param. It was getting lost or something along the way.

By contrast, NGINX makes cookies and their contents available via variables like $cookie_recache (for a cookie named recache). So you get to just use if ($cookie_recache) { to switch something or a map $cookie_user_authenticated $backend { in my case to change the upstream for the load balancer.

Again could just be skill issue on my part but I found the cookie method to be much easier to work with. 🤷‍♂️

[jasonvarga]

I'm not sure of the difficulty you had using a query parameter. Using a cookie sounds more complicated to me honestly.

This seems to work well and just requires adding a small block your nginx config.

if ($args ~* "live-preview=(.*)") {
    set $try_location @not_static;
}

+if ($arg___recache = "abc123def456") {
+    set $try_location @not_static;
+}

location / {
    try_files $uri $try_location;
}

You can get your token by running php please static:recache-token

$ php please static:recache-token

[INFO]  Your recache token is: abc123def456

That'll be a consistent hash based off your app key.

Or if you prefer, you can set the token explicitly in your config:

// config/statamic/static_caching.php
'recache_token' => 'my-explicit-token',

Since your nginx config is only allowing requests with that explicit key to bypass, visiting ?__recache=something-else won't have any effect.

[ryanmitchell]

[edit] GitHub sucks.... [/edit]

a normal thank you will do.
This is amazing.

[jasonvarga]

Thank you!

[jelleroorda]

Awesome! 🚀

Thanks @ryanmitchell and @jasonvarga!