Skip to content

Commit b5882c6

Browse files
authored
[UnitTest]Enhance grep syntax to avoid random failed of test_dist_mnist_dgc_nccl (#32946)
* Enhance grep syntax to avoid random failed * Enhance grep syntax to avoid random failed
1 parent b8d493d commit b5882c6

File tree

1 file changed

+6
-3
lines changed

1 file changed

+6
-3
lines changed

python/paddle/fluid/tests/unittests/test_dist_mnist_dgc_nccl.py

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -25,12 +25,15 @@
2525

2626

2727
def count_of_sparse_all_reduce_calls(file_name):
28-
cmd = 'grep sparse_all_reduce_op_handle ' + file_name + ' | grep in_numel | wc -l'
28+
# NOTE(Aurelius84): The log file contains some binary contents that causes error
29+
# while `grep`. So we add `-a` to fix it.
30+
# -a, --text equivalent to --binary-files=text, make binaries equivalent to text.
31+
cmd = 'grep -a sparse_all_reduce_op_handle ' + file_name + ' | grep in_numel | wc -l'
2932
child = subprocess.Popen(cmd, stdout=subprocess.PIPE, shell=True)
3033
result = child.communicate()[0]
3134
print('test_info: result = ' + str(result))
3235

33-
# note. in python3, result is b'num', != 'num'
36+
# NOTE: in python3, result is b'num', != 'num'
3437
return int(result)
3538

3639

@@ -59,7 +62,7 @@ def tearDown(self):
5962
# only 1 layer use dgc now, run_step=5, rampup_begin_step=2, so 1 * (5 - 2) = 3
6063

6164
# temp close this test. In python3 CI, the log is right, but the result
62-
# has a problem, may be in multi process mode, log is not writed in time.
65+
# has a problem, may be in multi process mode, log is not written in time.
6366
# self.assertEqual(result, 3)
6467

6568

0 commit comments

Comments
 (0)