Using set() of FileSystemCache raises errors on Ubuntu 20.04

[pcbergbusch]

Using set() of FileSystemCache raises errors on Ubuntu 20.04

WARNING:root:Exception raised while handling cache file '/home/site/wwwroot/deal_sourcing/flask_session/5651fc15999c50354843e09982ff80ed'
Traceback (most recent call last):
File "/antenv/lib/python3.8/site-packages/cachelib/file.py", line 238, in set
self._run_safely(os.replace, tmp, filename)
File "/antenv/lib/python3.8/site-packages/cachelib/file.py", line 299, in _run_safely
output = fn(*args, **kwargs)
FileNotFoundError: [Errno 2] No such file or directory: '/home/site/wwwroot/deal_sourcing/flask_session/tmp8qwf4_ww.__wz_cache' -> '/home/site/wwwroot/deal_sourcing/flask_session/5651fc15999c50354843e09982ff80ed'

We are running a Flask/Dash app as an Azure web service on Ubunutu 20.04, which uses MSAL and AAD to authenticate.
The Flask app repeatedly tries to re-authenticate, does not allow the user to navigate the app as desired.
The above errors appear in the Azure Application Logs.

Environment:

• Python version: 3.8
• CacheLib version: 0.6.0

Can mitigate the problem by editing _run_safely as follows (see the # lines):

def _run_safely(self, fn: _t.Callable, *args: _t.Any, **kwargs: _t.Any) -> _t.Any:
    """On Windows os.replace, os.chmod and open can yield
    permission errors if executed by two different processes."""
    # if platform.system() == "Windows":
    if True:
        output = None
        wait_step = 0.001
        max_sleep_time = 10.0
        total_sleep_time = 0.0

        while total_sleep_time < max_sleep_time:
            try:
                output = fn(*args, **kwargs)
            # except PermissionError:
            except OSError:
                sleep(wait_step)
                total_sleep_time += wait_step
                wait_step *= 2
            else:
                break
    else:
        output = fn(*args, **kwargs)

    return output

[northernSage]

Hey @pcbergbusch, thanks for letting us know! Unfortunately I can't reproduce the described problem. When you say mitigate, do you mean that your application behaves as intended? Or does it still show undesired behavior? The patch with except OSError is catching quite a wide range of system-related errors, so I'm not sure if this is actually fixing the problem or just omitting it.

[pcbergbusch]

@northernSage - thanks for following up :)

By mitigate I mean that the problem is avoided, not fixed.

The code is failing when it tries to rename a token file. This is happening on my Ubuntu machine. I don't know why it fails, but I do know that it is not caught by a PermissionError.

So I changed the code to check any operating system (not just Windows) and the loop in run_safely continues as long as OSErrors are thrown. By doing this, the code runs successfully and my users do not experience authentication issues.

[john0isaac]

I'm able to replicate this only when connecting to Azure files which is a NAS.
That's why we were not able to replicate this before. A simple fix is to remove the conditional around windows platform for retrying performing operations on a file after the lock on the file is released like we have been doing for windows.

The raised error now from Azure Storage is PermissionError so there is no need to expand the exception handling with other errors.

Pushing a fix now.