It important to remember dense matrix multiplication (doing it by hand) is a O(N^3) operation, this is about approximations to multiplication that beat that already harsh complexity. There is a whole field that develops approximations to matrix multiplication of large matrices, I'm assuming this article is about using ML to find good approximations.

To the replies, the very act of evaluting a prediction from a NN is matrix multiplication (linear transform of a column vector is matrix multiplication). This doesn't replace matrix multiplication wholesale, lol. This is about multiplication in a specific case.

As long as their is enough consistency between approximating a multiply X*W and Z*Wt (Wt = W transpose), then it is possible it could be used in NN training.

Y = X*W is the forward propagation. If Z is an error or derivative of error, Z*Wt is the back propagation.

Its an interesting question as to how well that would work. Anything that speeds up matrix multiply in NN and deep learning in general would be a big deal.

But ML relies on GPUs with lots of parallelism, so e.g. O(N^3) becomes O(N^2) for N smaller than the number of cores (roughly speaking).

Lots of things are like this. I program my network using a network. I run a server (BMC) on my server. I start my engine with an engine (https://en.m.wikipedia.org/wiki/Lockheed_SR-71_Blackbird).

That's not ironic, it could be revolutionary for this algorithm and ML in general.