Explain why this "Reed-Solomon' differs so much from storage ones

[burdges]

Although much faster for high shard counts, this crate provides rather different functionality from classical Reed-Solomon built upon syndrome decoding, etc. It's likely worth some explination.

[AndersTrier]

Hi Jeff

Thanks for the PR! You're right, that should be documented somewhere.

I think the explanation is a bit too verbose though.

How about just stating something like:

This crate does not detect or correct errors within a shard. I.e if data corruption is a likely scenario, you should implement a hash check for each shard, and don't feed the invalid shards to the decoder. Some suggestions for very fast error detection hashes are CRC32c (4 bytes), HighwayHash (8, 16 or 32 bytes) or xxHash (4, 8 or 16 bytes).

That's also more aligned with what Klaus documents in his implementation:

The encoder does not know which parts are invalid, so if data corruption is a likely scenario, you need to implement a hash check for each shard.
If a byte has changed in your set, and you don't know which it is, there is no way to reconstruct the data set.

[burdges]

As you like of course.. I sent some contact details by email, so maybe worth a brief off-line chat about langauge.

I typically prefer brevity like Klaus comment, but here I'm really addressing the concern raised in your PR to reed-solomon-16, by distancing this crate from matrix multiplication and error location, correction, and checking.

[AndersTrier]

here I'm really addressing the concern raised in your PR to reed-solomon-16

This PR malaire/reed-solomon-16#1? What concerns did I raise in that PR that you're addressing?

Note on hash now merged: #18
IMO that's sufficient. If someone is interested in the algorithm, there's a link to https://github.com/catid/leopard in the readme which references the relevant papers.

[FallingSnow]

I think the long version would go well in the wiki, if that's a feature you want to enable for this repo.

[sirandreww-starkware]

I think this should go in the rust docs in lib.rs. Very good explanation!