Skip to content

Avoid potential integer overflow in Mmap.mmap #41186

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
vtjnash merged 3 commits into from
Jul 19, 2021
Merged

Avoid potential integer overflow in Mmap.mmap #41186

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
3bcd671
Update Mmap.jl
kkretschmer Jun 11, 2021
1aef6dc
Mmap.mmap() added overflow check to the size calculation
kkretschmer Jun 15, 2021
256b6a0
Mmap.mmap: use proper `prod()` syntax in overflow error message
kkretschmer Jun 16, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion stdlib/Mmap/src/Mmap.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -191,7 +191,11 @@ function mmap(io::IO,
isopen(io) || throw(ArgumentError("$io must be open to mmap"))
isbitstype(T) || throw(ArgumentError("unable to mmap $T; must satisfy isbitstype(T) == true"))

len = prod(dims .|> big) * sizeof(T)
len = sizeof(T)
for l in dims
len, overflow = Base.Checked.mul_with_overflow(promote(len, l)...)
overflow && throw(ArgumentError("requested size prod$((sizeof(T), dims...)) too large, would overflow typeof(size(T)) == $(typeof(len))"))
Copy link
Contributor

@Seelengrab Seelengrab Jun 16, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This check could be moved outside of the loop, since we only care whether any overflow occured at all. Very minor though, since length(dims) can have a maximum of 64 elements before it is guaranteed to overflow for UInt64, so probably wouldn't change much runtime wise and should be fine as is.

I think you meant to write prod($((sizeof(T), dims...))) (notice the extra pair of parentheses to signify the method call).

Other than that, LGTM!

Copy link
Contributor Author

@kkretschmer kkretschmer Jun 16, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had the check outside the loop at first, but this is not guaranteed to work. If the overflow does not occur on the last iteration of the loop, the last one may succeed and overflow == false at the end of the loop. Therefore I think that the check has to be done after every call to mul_with_overflow or overflow has to be accumulated using logical or.

I will push a fixed error message. It will then look like this:
ArgumentError: requested size prod((1, 9223372036854771710, 2)) too large, would overflow typeof(size(T)) == Int64

Copy link
Contributor

@Seelengrab Seelengrab Jun 16, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking of unconditionally carrying the overflow with |= through the loop:

len = sizeof(T)
overflow = false
for l in dims
    len, of = Base.Checked.mul_with_overflow(...)
    overflow |= of
end
overflow && throw(...)

but as I said, I don't think it matters too much.

Yes, perfect - prod can't be splatted into after all, it takes collections.

end
len >= 0 || throw(ArgumentError("requested size must be ≥ 0, got $len"))
len == 0 && return Array{T}(undef, ntuple(x->0,Val(N)))
len < typemax(Int) - PAGESIZE || throw(ArgumentError("requested size must be < $(typemax(Int)-PAGESIZE), got $len"))
Expand Down
1 change: 1 addition & 0 deletions stdlib/Mmap/test/runtests.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,7 @@ c = mmap(s, Vector{UInt8}, (UInt16(11),))
finalize(c); c=nothing; GC.gc()
@test_throws ArgumentError mmap(s, Vector{UInt8}, (Int16(-11),))
@test_throws ArgumentError mmap(s, Vector{UInt8}, (typemax(UInt),))
@test_throws ArgumentError mmap(s, Matrix{UInt8}, (typemax(Int) - Mmap.PAGESIZE - 1, 2)) # overflow
close(s)
s = open(file, "r+")
@test isreadonly(s) == false
Expand Down