Skip to content

Commit 8fe22cb

Browse files
authored
Inline low level multiplication and reduction functions (#776)
Fixes a performance regression introduced in #667. Evidently, compiler relies a lot on knowing the slice sizes at compile time, so I'm inlining `schoolbook_multiplication()`, `schoolbook_squaring()`, and `montgomery_reduction_inner()`, so the compiler can optimize in case of `Uint`s. Benchmarks: - `wrapping ops/split_mul, U256xU256` - 26ns to 9ns - `Const Montgomery arithmetic/multiplication, U256*U256` - 41ns to 21ns - `Dynamic Montgomery arithmetic/multiplication, U256*U256` - 62ns to 44ns The effect is less pronounced for longer integers, but sill amounts to 5-10% speedup for U4096. On a higher level, this affects many `crypto-primes` benchmarks, e.g. doubles the speed of Lucas test for U128. Possible addition: I think `panic!` in these functions can be replaced with `debug_assert!`, but I don't insist on it.
1 parent d668d41 commit 8fe22cb

2 files changed

Lines changed: 3 additions & 0 deletions

File tree

src/modular/reduction.rs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ use crate::{Limb, Odd, Uint};
66
use {crate::BoxedUint, subtle::Choice};
77

88
/// Algorithm 14.32 in Handbook of Applied Cryptography <https://cacr.uwaterloo.ca/hac/about/chap14.pdf>
9+
#[inline(always)]
910
const fn montgomery_reduction_inner(
1011
upper: &mut [Limb],
1112
lower: &mut [Limb],

src/uint/mul.rs

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ pub(crate) mod karatsuba;
1717
/// schools.
1818
///
1919
/// The most efficient method for small numbers.
20+
#[inline(always)]
2021
const fn schoolbook_multiplication(lhs: &[Limb], rhs: &[Limb], lo: &mut [Limb], hi: &mut [Limb]) {
2122
if lhs.len() != lo.len() || rhs.len() != hi.len() {
2223
panic!("schoolbook multiplication length mismatch");
@@ -52,6 +53,7 @@ const fn schoolbook_multiplication(lhs: &[Limb], rhs: &[Limb], lo: &mut [Limb],
5253
/// Schoolbook method of squaring.
5354
///
5455
/// Like schoolbook multiplication, but only considering half of the multiplication grid.
56+
#[inline(always)]
5557
pub(crate) const fn schoolbook_squaring(limbs: &[Limb], lo: &mut [Limb], hi: &mut [Limb]) {
5658
// Translated from https://github.com/ucbrise/jedi-pairing/blob/c4bf151/include/core/bigint.hpp#L410
5759
//

0 commit comments

Comments
 (0)