Uploaded image for project: 'Apache MADlib'
  1. Apache MADlib
  2. MADLIB-857

SVD computation for sparse native format in Madlib 1.6



    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None


      Hi all,
      i am trying to use madlib 1.6
      and in particular the module for SVD.
      i have a matrix/table in sparse format that means a collection of rows in format (i,j,value)
      and i want to compute the svd since i am trying to implement spectral clustering algorithm in Madlib.

      The problem is that i observe that it runs for ~2minutes and then it outputs
      this error:
      [Error Code: 0, SQL State: XX000] ERROR: spiexceptions.InternalError: spiexceptions.InternalError: Function "madlib.__svd_sparse_lanczos_sfunc(double precision[],integer,integer,double precision,double precision[],integer)": Internal error. Eigen assertion failed (index >= 0 && index < size()) in function Eigen::DenseCoeffsBase<Derived, 0>::CoeffReturnType Eigen::DenseCoeffsBase<Derived, 0>::operator[](Eigen::DenseCoeffsBase<Derived, 0>::Index) const [with Derived = Eigen::Map<const Eigen::Matrix<double, -1, 1>, 0, Eigen::Stride<0, 0> >; Eigen::DenseCoeffsBase<Derived, 0>::CoeffReturnType = double; Eigen::DenseCoeffsBase<Derived, 0>::Index = long int] at /root/madlib-1.6.0/build/third_party/src/EP_eigen/Eigen/src/Core/DenseCoeffsBase.h:156
      Where: Traceback (most recent call last):
      PL/Python function "svd_sparse_native", line 25, in <module>
      row_id, col_id, val_id, row_dim, col_dim, k, n_iterations, result_summary_table)
      PL/Python function "svd_sparse_native", line 370, in svd_sparse_native
      PL/Python function "svd_sparse_native", line 535, in _svd
      PL/Python function "svd_sparse_native"

      is this something important?

      what should i do the matrix that i am trying to do operations on is of size 76k*76k

      here is the command that i call:

      select madlib.svd_sparse_native('Laplacian','svd','i','j','value',76381,76381,100,200,'results');

      Any input it will be more than appreciated

      thanx in advance for your time

      kind regards




            riyer Rahul Iyer
            yannis_at yannis ats
            0 Vote for this issue
            1 Start watching this issue