Improve performance of determinant

[josdejong]

When running some simple benchmarks of mathjs:

node benchmark/matrix_operations.js

You see output like:

matrix operations mathjs A+A             x 85,375 ops/sec ±1.83% (89 runs sampled)
matrix operations mathjs A*A             x 2,843 ops/sec ±0.15% (96 runs sampled)
matrix operations mathjs A'              x 79,028 ops/sec ±1.73% (91 runs sampled)
matrix operations mathjs det(A)          x 105 ops/sec ±0.21% (75 runs sampled)
matrix operations sylvester A+A          x 69,318 ops/sec ±1.17% (92 runs sampled)
matrix operations sylvester A*A          x 5,659 ops/sec ±0.21% (96 runs sampled)
matrix operations sylvester A'           x 113,950 ops/sec ±1.79% (85 runs sampled)
matrix operations sylvester det(A)       x 9,778 ops/sec ±1.28% (88 runs sampled)
matrix operations numericjs A+A          x 426,161 ops/sec ±1.58% (81 runs sampled)
matrix operations numericjs A*A          x 33,622 ops/sec ±0.33% (95 runs sampled)
matrix operations numericjs A'           x 528,647 ops/sec ±1.30% (91 runs sampled)
matrix operations numericjs det(A)       x 81,438 ops/sec ±0.54% (89 runs sampled)
matrix operations ndarray A+A            x 342,996 ops/sec ±1.03% (92 runs sampled)
matrix operations ndarray A*A            x 25,012 ops/sec ±0.52% (91 runs sampled)
matrix operations ndarray A'             x 390,326 ops/sec ±1.68% (85 runs sampled)
matrix operations ndarray det(A)         x 400,209 ops/sec ±0.44% (95 runs sampled)

math.js is slower than other alternatives (up to 1 order of magnitude). This is probably caused by math.js dealing with mixed types rather than just numbers, and there is room for improvement there, but that's for an other topic.

There is one extreme outlier above: calculating the determinant det(A) which is really hundreds of times slower than other math libraries.

Who is interested in diving into this performance issue and try to improve the performance of the determinant?

[firepick1]

Determinants can result in big equations. Is the slowness due to: a) the creation of the equations or the b) the evaluation of the created equation?

If (b) then my Equations module might help, since it only evaluates sub-expressions once.

[josdejong]

There aren't really equations into play here, it's just numeric number crunching of a matrix (see det.js).

[firepick1]

Oh. I see. It's numeric, not symbolic. I had a hunch that a symbolic equation representing the determinant could be optimized. However, getting that symbolic equation would take some doing. 😶

[firepick1]

I agree that mathjs operator polymorphism (e.g., add() vs '+') may be the magnitude killer here. Its the old "interpreted vs. compiled" order of magnitude. If this hypothesis holds, then we might want to consider a sister package, "mathjs-double". Such a package would be tuned for performance but retain most of the mathjs API goodness. An alternative solution would be to go the C++ node extension route, which is a lot of work 😩

[firepick1]

(deleted comment. sorry wrong thread)

[josdejong]

For some Matrix operations, there is already an optimization in place that checks whether the Matrix does not contain mixed types (i.e. just numbers or just BigNumbers), and in that case, don't use the generic mixed type math.add, but the fast and simple implementation of math.add just for numbers. Maybe we could apply that here too.

But at this point I don't know if that's actually the cause, or that the current algorithm isn't efficient enough and we should look for a different algorithm. We need to figure that out first.

[firepick1]

As a thought exercise, if we expressed any particular determinant as an expanded equation (i.e., write out the det()) and compiled that expression to Javascript, it would be faster because we would have eliminated all the function calls and conditionals. Indeed, we know this works because other Javascript math packages are faster. An additional performance boost for matrices of size N>3 would be to memoize the sub-products that recur. Such memoization would precede the compilation into Javascript.

The above exercise leads one to consider an architectural approach to performance overall that would rely on the extensive use of compiled expression trees with fully expanded function nodes. Such an approach could yield blazingly fast evals of all compiled expressions. Some functions like sin() would still be atomic, but others like det() and derivative() would benefit. In other words, my hypothesis is that the symbolic processing of math expressions is the key to performance.

Symbolic processing of math expressions allows us to maintain polymorphism via compilation rather than having it burden each evaluation. In this manner, we would compile det([[a,b], [c,d]]) as a*d-b*c for doubles and compile it as subtract(multiply(a,d), multiply(b,c)) for all other number types.

[josdejong]

Yes, so we can optimize by doing analysis of the data types beforehand, and optimize evaluation for that.

Note that math.js already uses compiling expressions and typed functions dynamically, that's why the expression parser is very fast once compiled, which allows you to evaluate an expression against a scope very fast.

Also, for example the function det already handles special cases of 2x2, 3x3 matrices, so the function is already optimized in that regard.

I don't think we necessary need to compile dynamic JavaScript, but we need switches which check for mixed type vs. single type contents of matrices, and then use either the generic or type specific implementation of an operation. Something like:

// pseudo code for adding two matrices:
var A = math.matrix([...]);
var B = math.matrix([...]);
var operation = math.add;

// we need a new method which returns the different types of the values in the matrix 
// for example return ['number'] when single type, or ['number', 'BigNumber'] in case 
// of mixed types
var typesA = A.contentTypes(); 
var typesB = B.contentTypes(); 
if (typesA.length === 1 && typesB.length === 1 && typesA[0] === typesB[0]) {
  // both matrices are single type, and have the same type (for example 'number')
  // -> find the type specific implementation of operation math.add for numbers:
  var signature = typesA[0] + ',' + typesB[0]  // 'number,number'
  var addSingleType = operation.signatures[signature];
  if (addSingleType) {
    // evaluate with the fast, single type implementation of math.add
    return algorithm13(A, B, addSingleType);
  }
}

// evaluate with the generic, multi type implementation of math.add
// (algorithm13 is one of a set of helper functions which applies a 
// callback function on two matrices element wise)
return algorithm13(A, B, operation); 

[firepick1]

That would be faster, yes. 😀
I guess I'm just idly wondering what we might do with functions such as detExpr() or multiplyExpr() which would return the optimizable expression of the abstract mathematical function. For example, detExpr([[a,b][c,d]) would return 'ad-bc'. Although such functions are the insanely verbose anti-thesis of abstraction, they do permit fascinating optimization possibilites via the one-time computation of recurring sub-expressions. 🤔 (this is where you nod somberly and ignore me)

[ericman314]

I tried replacing the polymorphic arithmetic in det.js and multiply.js with the native operators:

100x100
det(number): 3176ms
det_native(number): 869ms

Virtually all of the speed up was due to refactoring multiply.js to use native types, since most of the work done in det.js is multiplying matrices. But this doesn't come close to the benchmarks of other matrix libraries, so I wondered whether there was an algorithm that doesn't require multiplying matrices at all.

It turns out you can use the LU decomposition to compute the determinant, which is much faster:

100x100
det(number): 3212ms
det_native(number): 860ms
det_lu(number): 54ms

But the LU decomposition can be numerically unstable so the computed determinant starts to differ significantly between the two methods once you get larger than about 50x50.

Implementing a native version of lup.js gives even better results. Here are the two with a 500x500 matrix
(At this point I stopped benchmarking det and det_native):

500x500
det_lu(number): 1628ms
det_lu_native(number): 515ms

[josdejong]

(this is where you nod somberly and ignore me)

ha ha, of course not. I like being able to think aloud and have a sort of a brainstorm, which can result in great ideas :)

@ericman314 thanks for doing these experiments. So optimizing for matrices with a single data type can yield an improvement in the order of a factor 3 or 4, and a much bigger gain could be replacing the algorithm with LU decomposition for example. But unstable results for larger matrices are an issue of course :(

[ericman314]

Anyone doing operations on matrices that big should 1) already understand the numerical issues involved, and 2) probably be using a dedicated and optimized linear algebra library. Maybe we should also be looking at benchmarks for calculating lots of small determinants as well as one big one?

…

On Aug 6, 2017 12:12 PM, "Jos de Jong" ***@***.***> wrote: (this is where you nod somberly and ignore me) ha ha, of course not. I like being able to think aloud and have a sort of a brainstorm, which can result in great ideas :) @ericman314 <https://github.com/ericman314> thanks for doing these experiments. So optimizing for matrices with a single data type can yield an improvement in the order of a factor 3 or 4, and a much bigger gain could be replacing the algorithm with LU decomposition for example. But unstable results for larger matrices are an issue of course :( — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#908 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGDTkd3KXOs3CjsH0ZcVi0AFnlWOy8wUks5sVgIOgaJpZM4OoxMg> .

[josdejong]

So you mean to say that it's acceptable to have a solution that becomes unstable for large matrices? That can be a pragmatic (and very fast) approach. I would love to somehow give a warning or error then. Or automatically fall back to a slow but precise and stable algorithm for large matrices. What do you think?

Maybe we should also be looking at benchmarks for calculating lots of small
determinants as well as one big one?

Yes, that will be best. Like calculating it for 2x2 or 3x3 can/should be extremely fast :) Feel free to extend the benchmark folder with new tests where needed.