fix deadlock in debouncer

[mtak-]

Works around the issue described in #205. Tested on the fsevent backend only linux/macos/windows.

I tried to change locking as little as possible as it's somewhat hard to follow, and instead use a simple backoff algorithm to workaround the deadlock.

These tests fail on both branch origin/try-v4 and my fork.

test create_rename_overwrite_directory ... FAILED
test create_rename_overwrite_file ... FAILED

Would be nice to get a new version out with the fix as well!

[mtak-]

Many of the ci failures existed previously: https://travis-ci.org/passcod/notify/builds/579460084

[mtak-]

All of the failures are preexisting, the windows stable failure that appears new I'm guessing is a flaky test truncate_write_file? On the previous build, nightly failed with the same error.

@Dylan-DPC

[0xpr03]

I'll have to check this when I've got some time. Please ping me again otherwise in 3 days.

[0xpr03]

I think we want to warn users about the broken state of the debouncer in these cases, if we're not able to fix it sufficiently. This affects at least 4.X stable and 5.X. So breaking the debouncer isn't possible. Your PR also mentions macOS specifically. This issue persists at least also on linux, it may also be another issue with similar symptoms.
cc @Dylan-DPC

[mtak-]

The delay_zero test repros the issue pretty consistently on macos. If someone wants to test linux, just pasting that test into the current version should hopefully fail. Whereas running that test against this PR should hopefully succeed.

[mtak-]

I pushed up the delay_zero test without the deadlock fix workaround, to let travis test the platforms I don't have access to (windows/linux). The test failed on both platforms here. Altho the delay_zero test did pass one of the linux builds (it is a non-deterministic test).

Testing both platforms again with the deadlock fix shows the delay_zero test passing on all platforms here.

[0xpr03]

Thanks for all the work. I re-scheduled CI and will verify on linux again.

[0xpr03]

I can confirm that this resolves the deadlock problem under linux as well.
I can't say why truncate_write_file failed on windows for 1.26 but would merge this. cc @JohnTitor

[0xpr03]

truncate_write_file is broken already so I'd go ahead and merge this

[JohnTitor]

@0xpr03 Windows failure is caused by installation windows-sdk. b66d724 fixes this problem so you can cherry-pick this commit to try-v4 branch.