Commit 7ac8b10
authored
downloader: docs on MMAP for data-files r/w and experiments with bufio (#10074)
Pros:
- it allows to not pre-alloc files:
#8688
- it allows to not "sig-bus" when no space left on disk (return
user-friendly error). see:
#8500 - but DB will be MMAP
anyway and may get "sig-bus"
FYI:
- seems no perf difference (but i tested only on cloud drives)
- erigon will anyway open it as mmap
Cons:
- i did implemented `fsync` for mmap (
anacrolix/torrent#755 ) - probably will need
implement it for bufio: anacrolix/torrent#937
- no zero-copy: more `alloc` memory will be holded by APP (PageCache
starvation). I see 2x mem usage (at `--torrent.download.slots=500` 20gb
vs 40gb)
- i see "10K threads exchaused" error earlier (on
`--torrent.download.slots=500`).
- what else?1 parent 382f881 commit 7ac8b10
File tree
2 files changed
+11
-0
lines changed- cmd/diag/downloader
- erigon-lib/downloader
2 files changed
+11
-0
lines changedFile renamed without changes.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2568 | 2568 | | |
2569 | 2569 | | |
2570 | 2570 | | |
| 2571 | + | |
| 2572 | + | |
| 2573 | + | |
| 2574 | + | |
| 2575 | + | |
| 2576 | + | |
| 2577 | + | |
2571 | 2578 | | |
| 2579 | + | |
| 2580 | + | |
| 2581 | + | |
| 2582 | + | |
2572 | 2583 | | |
2573 | 2584 | | |
2574 | 2585 | | |
| |||
0 commit comments