"most recent backup" judgement is too tight

[GuitarBilly]

I think this is applicable for all npbackup versions on all OS

steps to reproduce

So I had npbackup on a cronjob */15 * * * *.
This I call the 'laptop' schedule, i.e. run every 15 mins and check the most recent backup is less than "one day / 24 hours". Perfect to have a highest chance to get a backup every day with unknown on and off times.

Now the result of this on an always-on machine is that the backup moves every day with 15 minutes:

The reason is that at the +24 hour mark the conclusion in the log is always that there is a recent backup and backup will be skipped.
So is there too much slack in my scheduler timing or is npbackup check for recent 24 hour backup too strict or both?

expected behaviour

i'd expect if more than say 23:59 time is elapsed, it is OK to have another backup. This will prevent a racing condition and a moving backup.

daily cron schedule

I can also replicate this with my current daily crontjob 01 2 * * * :

~/npbackup$ grep "2025-05-1"[345]" 02:0" npbackup-cli.log |grep syncthing
2025-05-13 02:00:03,112 :: INFO :: Most recent backup in repo syncthing is from 2025-05-12 02:00:04.156897+02:00
 >>> note that on 14th of May i changed the syncthing cronjob from 02:00 to 02:01 to prevent clash with another */15 cronjob. <<<
2025-05-14 02:01:03,177 :: INFO :: No recent backup found in repo syncthing. Newest is from 2025-05-12 02:00:04.156897+02:00
2025-05-14 02:01:03,177 :: INFO :: Running backup of ['/home/ubuntu/containers/syncthing'] to repo syncthing
2025-05-15 02:01:03,128 :: INFO :: Most recent backup in repo syncthing is from 2025-05-14 02:01:03.938439+02:00

From above log, the 13th May and 15th May they suffer from the too tight recent backup judgement.
When run, if the previous day backup job is only 1msec earlier then npbackup concludes that a new backup is not required and will skip the backup!

[deajan]

I'm aware of that situation for a long time now.

Actually, there's no way to prevent the "backup time shifting / moving" on laptops and not always on machines. The 24H delay is configurable (it's 1440 seconds by default, but you could put 1380 seconds or anything else to lower the judgment to 23:59), but it would still shift since backup runs will never take the same time, and last recent backup date is the end of a non fixed backup duration.

On always on machines, you're not supposed to use that every 15 minutes schedule, but schedule backups at fixed time, via GUI, or via --backup --force which will avoid checking for recent backups, or even set minimum_backup_age to 0 which would do the same.

If you have any ideas how to improve this, let me know.

[GuitarBilly]

@deajan there is only one way to improve: to always use the start time of the backup, not the finish time.

Actually I did a restic trial because I was not sure which time restic stores at completion of the snapshot. Different trom your statement, I find that restic does store the start time. Example:

Hence, for me changing that delay from 24:00 to 23:59 would work to allow some timing jitter in the cron jobs.

[GuitarBilly]

I have found the "minimum delay between two backups" in the GUI and corresponding "minimum_backup_age" in the configuration config file. I have set it to 1435 minutes now, so 23:55.

Will report mid next week how that goes.

[GuitarBilly]

here the update. All seems fine, I will keep the updated settings!

first backup job

Runs daily. After May 18 with above changes it runs every day, no skipped days as reported before (May 17/15/13).

second backup job

Runs every 15min on the 24/7 machine. After May 18 with above changes it runs at the same time, no 15 day advance every day anymore. I agree with your statement that a daily backup should be run once instead of every 15mins on a 24/7 machine, but this is just to test and prove npbackup fundamental behaviour.

EDIT : the remaining question is : will you consider changing the default value to allow scheduler/cron jitter?

[deajan]

Makes sense. Version 3.0.2 is due today.
Thanks for your report and feedback.