std: introduce GeneralPurposeAllocator #5998
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
`std.GeneralPurposeAllocator` is now available. It is a function that takes a configuration struct (with default field values) and returns an allocator. There is a detailed description of this allocator in the doc comments at the top of the new file. The main feature of this allocator is that it is *safe*. It prevents double-free, use-after-free, and detects leaks. Some deprecation compile errors are removed. The Allocator interface gains `old_align` as a new parameter to `resizeFn`. This is useful to quickly look up allocations. `std.heap.page_allocator` is improved to use mmap address hints to avoid obtaining the same virtual address pages when unmapping and mapping pages. The new general purpose allocator uses the page allocator as its backing allocator by default. `std.testing.allocator` is replaced with usage of this new allocator, which does leak checking, and so the LeakCheckAllocator is retired. stage1 is improved so that the `@typeInfo` of a pointer has a lazy value for the alignment of the child type, to avoid false dependency loops when dealing with pointers to async function frames. The `std.mem.Allocator` interface is refactored to be in its own file. `std.Mutex` now exposes the dummy mutex with `std.Mutex.Dummy`. This allocator is great for debug mode, however it needs some work to have better performance in release modes. The next step will be setting up a series of tests in ziglang/gotta-go-fast and then making improvements to the implementation.
* std.Mutex API is improved to not have init() deinit(). This API is designed to support static initialization and does not require any resource cleanup. This also happens to work around some kind of stage1 behavior that wasn't letting the new allocator mutex code get compiled. * the general purpose allocator now returns a bool from deinit() which tells if there were any leaks. This value is used by the test runner to fail the tests if there are any. * self-hosted compiler is updated to use the general purpose allocator when not linking against libc.
andrewrk
added
breaking
daurnimator
reviewed
Aug 8, 2020
|
|
||
| pub const Error = std.mem.Allocator.Error; | ||
|
|
||
| const small_bucket_count = math.log2(page_size); |
Contributor
There was a problem hiding this comment.
This should be configurable.
| free, | ||
| }; | ||
|
|
||
| fn up_to_nearest_power_of_2(comptime T: type, n: T) T { |
Contributor
There was a problem hiding this comment.
Use math.ceilPowerOfTwo
| // * stack_trace_addresses: [N]usize, // traces_per_slot for every allocation | ||
|
|
||
| const BucketHeader = struct { | ||
| prev: *BucketHeader, |
Contributor
There was a problem hiding this comment.
Shouldn't this be a std.LinkedList?
The high level Allocator interface API functions will now do a `@returnAddress()` so that stack traces captured by allocator implementations have a return address that does not include the Allocator overhead functions. This makes `4` a more reasonable default for how many stack frames to capture.
This makes `@returnAddress()` return 0 for WebAssembly (when not using the Emscripten OS) and avoids trying to capture stack traces for the general purpose allocator on that target.
even outside of unit tests
Sadly, trying to collect stack frames goes into an infinite loop on mips. This sets the default number of stack frames to collect to 0 on mips.
We don't pass no-omit-frame-pointer in release safe by default, so it also makes sense to not try to collect stack trace frames by default in release safe mode.
The tests are cleverly testing some alignment stuff, but were getting thwarted by Windows choosing to allocate 64K aligned pages.
Sahnvour
reviewed
Aug 8, 2020
| return power; | ||
| } | ||
|
|
||
| fn hash_addr(addr: usize) u32 { |
Contributor
There was a problem hiding this comment.
AutoHashMap should be better, especially now that it has a fastpath for easy to hash keys.
As pointed out by Sahnvour, AutoHashMap is both more convenient and will have better performance in this case.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
std.GeneralPurposeAllocatoris now available. It is a function thattakes a configuration struct (with default field values) and returns an
allocator. There is a detailed description of this allocator in the
doc comments at the top of the new file.
The main feature of this allocator is that it is safe. It
prevents double-free, use-after-free, and detects leaks.
Some deprecation compile errors are removed.
The Allocator interface gains
old_alignas a new parameter toresizeFn. This is useful to quickly look up allocations.std.heap.page_allocatoris improved to use mmap address hints to avoidobtaining the same virtual address pages when unmapping and mapping
pages. The new general purpose allocator uses the page allocator as its
backing allocator by default.
std.testing.allocatoris replaced with usage of this new allocator,which does leak checking, and so the LeakCheckAllocator is retired.
stage1 is improved so that the
@typeInfoof a pointer has a lazy valuefor the alignment of the child type, to avoid false dependency loops
when dealing with pointers to async function frames.
The
std.mem.Allocatorinterface is refactored to be in its own file.std.Mutexnow exposes the dummy mutex withstd.mutex.Dummy.This allocator is great for debug mode, however it needs some work to
have better performance in release modes. The next step will be setting
up a series of tests in ziglang/gotta-go-fast and then making
improvements to the implementation.