Skip to content

Commit e6b8d51

Browse files
WeichenXu123kai-chi
authored andcommitted
[SPARK-28582][PYSPARK] Fix flaky test DaemonTests.do_termination_test which fail on Python 3.7
Fix flaky test DaemonTests.do_termination_test which fail on Python 3.7. I add a sleep after the test connection to daemon. Run test ``` python/run-tests --python-executables=python3.7 --testname "pyspark.tests.test_daemon DaemonTests" ``` **Before** Fail on test "test_termination_sigterm". And we can see daemon process do not exit. **After** Test passed Closes apache#25315 from WeichenXu123/fix_py37_daemon. Authored-by: WeichenXu <[email protected]> Signed-off-by: HyukjinKwon <[email protected]> (cherry picked from commit fbeee0c) Signed-off-by: HyukjinKwon <[email protected]>
1 parent 13fa689 commit e6b8d51

1 file changed

Lines changed: 5 additions & 2 deletions

File tree

python/pyspark/tests.py

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1916,7 +1916,7 @@ def test_malformed_RDD(self):
19161916

19171917
class DaemonTests(unittest.TestCase):
19181918
def connect(self, port):
1919-
from socket import socket, AF_INET, SOCK_STREAM
1919+
from socket import socket, AF_INET, SOCK_STREAM# request shutdown
19201920
sock = socket(AF_INET, SOCK_STREAM)
19211921
sock.connect(('127.0.0.1', port))
19221922
# send a split index of -1 to shutdown the worker
@@ -1939,9 +1939,12 @@ def do_termination_test(self, terminator):
19391939
# daemon should accept connections
19401940
self.assertTrue(self.connect(port))
19411941

1942+
# wait worker process spawned from daemon exit.
1943+
time.sleep(1)
1944+
19421945
# request shutdown
19431946
terminator(daemon)
1944-
time.sleep(1)
1947+
daemon.wait(5)
19451948

19461949
# daemon should no longer accept connections
19471950
try:

0 commit comments

Comments
 (0)