Commit a447cd8
[SPARK-17465][SPARK CORE] Inappropriate memory management in
## What changes were proposed in this pull request?
The expression like `if (memoryMap(taskAttemptId) == 0) memoryMap.remove(taskAttemptId)` in method `releaseUnrollMemoryForThisTask` and `releasePendingUnrollMemoryForThisTask` should be called after release memory operation, whatever `memoryToRelease` is > 0 or not.
If the memory of a task has been set to 0 when calling a `releaseUnrollMemoryForThisTask` or a `releasePendingUnrollMemoryForThisTask` method, the key in the memory map corresponding to that task will never be removed from the hash map.
See the details in [SPARK-17465](https://issues.apache.org/jira/browse/SPARK-17465).
Author: Xing SHI <shi-kou@indetail.co.jp>
Closes #15022 from saturday-shi/SPARK-17465.org.apache.spark.storage.MemoryStore may lead to memory leak1 parent bf3f6d2 commit a447cd8
2 files changed
Lines changed: 7 additions & 6 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
104 | 104 | | |
105 | 105 | | |
106 | 106 | | |
| 107 | + | |
107 | 108 | | |
108 | 109 | | |
109 | 110 | | |
| |||
Lines changed: 6 additions & 6 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
511 | 511 | | |
512 | 512 | | |
513 | 513 | | |
514 | | - | |
515 | | - | |
516 | | - | |
517 | 514 | | |
518 | 515 | | |
| 516 | + | |
| 517 | + | |
| 518 | + | |
519 | 519 | | |
520 | 520 | | |
521 | 521 | | |
| |||
530 | 530 | | |
531 | 531 | | |
532 | 532 | | |
533 | | - | |
534 | | - | |
535 | | - | |
536 | 533 | | |
537 | 534 | | |
| 535 | + | |
| 536 | + | |
| 537 | + | |
538 | 538 | | |
539 | 539 | | |
540 | 540 | | |
| |||
0 commit comments