Use monotonic time or modify the process.py script?

[ryan44guo]

Hi,
We meet a problem when starting.
When syncd starting and enter RUNNING state, the start.sh should exit almost immediately. But between the two events, the time changed by ntpd. so when the process.py run the code to exit the process, it find it quit too quickly, but its state is not STARTING (but RUNNING), so asserted.
In my opinion, in SONIC system, we should use monotonic time to control the interval-related process, maybe standarnd python library do not have it? although it seems not a difficult problem, we can install it if needed. I think it is not an isolation problem.
Of course, if we only deal with this question, we can modify the process.py, i.e, not assert the state to STARTING.
Hope to know the thought of the writer: What is your opinion about it?

[lguohan]

@jleveque , i remember we have address this issue by patching supervisord, can you confirm?

[jleveque]

@lguohan: Yes, this is a supervisor issue and should have been addressed by my patch from January: sonic-net/sonic-buildimage#1311.

@ryan44guo: Are you building SONiC from the latest source code?

[ryan44guo]

hi all,
I am running 201803 branch, so your patch have been applied.
I check your patches again, i think you have not resolved all questions about time.
In this problem, syncd have changed to RUNNING state, because it had been running more than 1 sec.
But when call "finish" after that, the time have been changed in this moment，so when this function get time again, it entered too_quickly branch, we know, this branch only valid for STARTING state, but now the state is RUNNING.
Our questions prove that these problems will occur in several situations, so i suggest that we should use monotonic time instead.
@jleveque @lguohan

[lguohan]

@ryan44guo , thanks for the feedback. you mentioned the monotonic time, how can we get it in python? Any idea?

[ryan44guo]

@lguohan , i am not familiar with python, but i think python should have some libs about monotonic time, although maybe it is not the default libs. there is an URL https://pypi.org/project/monotonic/ about it, maybe someone familiar with python can use it? Or, i have seen some codes on github, but i do not know which one is better.

[jleveque]

The maintainer of Supervisor wasn't happy with the way https://pypi.org/project/monotonic/ implements monotonic time for Python2, so he wasn't willing to use that library.

I found and fixed what I believe was the last remaining issue with clock rollback here: sonic-net/sonic-buildimage#2624

My changes were also merged into upstream Supervisor here: Supervisor/supervisor#1047

I am closing this issue for now. Please reopen if you see the issue again in the future.

[mnaberez]

I found and fixed what I believe was the last remaining issue with clock rollback here: sonic-net/sonic-buildimage#2624

My changes were also merged into upstream Supervisor here: Supervisor/supervisor#1047

Supervisor 3.4.0 (PyPI package; includes this patch)