[MXNET-1350] Cholesky factorization: matrix inversion, determinant, logdeterminant

• Type: New Feature
• Status: To Do
• Priority: Minor
• Resolution: Unresolved
• Component/s: Apache MXNet Backend
• Assignee: Unassigned
• Reporter: Zach Boldyga
• Created: 10/Mar/19 19:53
• Updated: 10/Mar/19 20:00

Description

MXNet already includes a routine for Cholesky factorization, and allows for matrix inversion using this factorization.

The determinant and log determinant are straightforward to calculate using the Cholesky factorization, but MXNet does not provide a way to compute the determinant or log determinant. 

This issue was reported and upvoted by the community: https://github.com/apache/incubator-mxnet/issues/14360 . A reference was made to Torch, which includes routines for the inverse, determinant, and log determinant.

I'd like to implement the following changes:

1. Provide a single, clear API endpoint for calculating matrix inversion. e.g. mxnet.ndarray.linalg.inverse. This can be in addition to the existing endpoints, which may still be useful to some users. It will be simple to implement; a merger of the two existing ops (potrf, potri - used to calculate the Cholesky factorization and then calculate the inversion using that factorization). But currently it's confusing for users to discover that MXNet actually supports matrix inversion, and it's a two step process.
2. Add an API endpoint for calculating the matrix determinant. e.g. mxnet.ndarray.linalg.det. This would run Cholesky factorization internally, then use the resultant triangular matrix to simplify the determinant calculation.
3. Add an API endpoint for calculating the log determinant of the input matrix. e.g. mxnet.ndarray.linalg.logdet. Similar to #2, but it may be as simple as combining cholesky factorization and sumlogdiag internally. 
4. Perhaps in all cases it might make sense to provide a way to do the end-to-end calculation in one call (Cholesky factorization internally, then additional ops for inverse, det, etc.), and also an additional call for starting the operation given the Cholesky factorization? This would allow users to do the factorization once, then calculate the determinant, log det, and inverse without the additional overhead of repeating this expensive op (I believe it's O(N^3)?). But if the end-to-end option is also provided, the majority of users will just be able to quickly dive in and run the operation of their choosing without having to understand the importance of Cholesky factorization?

Overall, this is just piecing together of other existing linear algebra operations and clarifying the documentation so that users can access these operations without having to have a deeper understanding of linear algebra.

I can open a PR as soon as someone reviews this, just let me know if a different approach is preferred or if there are any specifics I should be careful about in regards to implementation. Thanks!