Hash of `Discriminant` now produces different results across CPU architectures

[korken89]

Hi, we just noticed that there has been changes to the inner of Discriminant, i.e. DiscriminantKind,
that causes a Hash of a discriminant to produce different results across CPU architectures.

E.g.

#[derive(Hash)]
enum SomeEnum {
    One,
    Two,
    Three,
}

When hashing this in the current stable one gets writes that look like this:

Hash for One:
[0, 0, 0, 0, 0, 0, 0, 0]

Hash for Two
[1, 0, 0, 0, 0, 0, 0, 0]

Hash for Tree
[2, 0, 0, 0, 0, 0, 0, 0]

And depending on the architecture in the current nightly one gets for armv7-unknown-linux-gnueabihf:

Hash for One:
[0, 0, 0, 0]

Hash for Two
[1, 0, 0, 0]

Hash for Tree
[2, 0, 0, 0,]

However for ex x86_64-unknown-linux-gnu it is still 8 bytes as before.

This is a problem as e.g. CRC hashes are no longer compatible between CPU architectures.
Not sure if this was intended, but it seems like a breaking change to me?

In our case we are sending structures between an embedded thumbv7em-none-eabihf and an
armv7-unknown-linux-gnueabihf system where CRCs are calculated based on #[derive(Hash)]
for said structures to be sent.

rustc --version --verbose:

rustc 1.46.0-nightly (5db778aff 2020-07-09)
binary: rustc
commit-hash: 5db778affee7c6600c8e7a177c48282dab3f6292
commit-date: 2020-07-09
host: x86_64-unknown-linux-gnu
release: 1.46.0-nightly
LLVM version: 10.0


rustc 1.44.1 (c7087fe00 2020-06-17)
binary: rustc
commit-hash: c7087fe00d2ba919df1d813c040a5d47e43b0fe7
commit-date: 2020-06-17
host: x86_64-unknown-linux-gnu
release: 1.44.1
LLVM version: 9.0

[kotauskas]

I don't think that the Hash trait was supposed to guarantee stable results over different architectures or crate/compiler versions: the only valid way of using it is for keys in HashMap/HashSet. There are crates specifically designed for consistent hashing with a specific algorithm, but those are not part of std and operate on bytes, and gathering those bytes from a value in memory is a Serde+Bincode thing.

[nbdd0121]

Discriminant has been changed from u64 to the actual integer type that represent the discriminant, to support #[repr(u128)] enum, so they hash differently.

[Mark-Simulacrum]

Tagging as a @rust-lang/libs regression -- I suspect that this is definitely outside the scope of "intentional coverage" of our stability policy, but we could probably change the Hash impl to always hash the ~same number of bytes. Regardless I don't think Rust guarantees the same value in the discriminant when compiling more than once (much less for different targets), so in that sense it's not really covered either.

[Amanieu]

Either way we should specifically mention the stability guarantees (or lack thereof) in the Hash documentation.

[SimonSapin]

However for ex x86_64-unknown-linux-gnu it is still 8 bytes as before.

I’m a bit confused because the issue description mentions two rustc versions. Do you get different results on different architectures with the same compiler version? Does std::mem::size_of::<SomeEnum>() also vary based on the architecture?

[nbdd0121]

I’m a bit confused because the issue description mentions two rustc versions. Do you get different results on different architectures with the same compiler version? Does std::mem::size_of::<SomeEnum>() also vary based on the architecture?

@SimonSapin No, SomeEnum will always have same size as i8 because i8 is sufficient to fit. However Discriminant<SomeEnum> will be of size isize.

rust/src/librustc_middle/ty/mod.rs

Lines 2179 to 2181 in e59b08e

	pub fn discr_type(&self) -> attr::IntType {
	self.int.unwrap_or(attr::SignedInt(ast::IntTy::Isize))
	}

[lcnr]

A quick summary of the problem:
mem::Discriminant::hash now calls isize::hash instead of u64::hash on all architectures.

This breaks code which relies on Hasher::write getting called with the same value on both 64 and 32 bit systems.
This can be fixed by manually implementing write_isize and write_usize for a Hasher if it should be stable between architectures.

(I think we still don't guarantee that enums with unspecified repr have the same discriminant when compiling for a different target)

Not really part of T-compiler, so I am removing I-prioritize for now, cc @rust-lang/lang

[SimonSapin]

The root of the issue seems to be that <SomeEnum as std::marker::DiscriminantKind>::Discriminant is isize for an enum without a repr attribute.

size_of::<SomeEnum> already is that of the smallest integer type that can fit all discriminant values. Couldn’t DiscriminantKind::Discriminant be that type as well, rather than isize?