Uploaded image for project: 'SystemDS'
  1. SystemDS
  2. SYSTEMDS-2020

Perftest bivariate statistics w/ codegen causes unnecessary sparse-dense conversions

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • SystemML 1.0.0
    • None
    • None

    Description

      [8,1] scale-categorical
      [3,1] scale-categorical
      [10,1] scale-categorical
      [6,1] scale-categorical
      [4,1] scale-categorical
      [1,1] scale-categorical
      [2,1] scale-categorical
      [7,1] scale-categorical
      [5,1] scale-categorical
      [9,1] scale-categorical
      [13,1] scale-scale
      [13,2] scale-categorical
      [14,1] scale-scale
      [11,1] scale-scale
      [14,2] scale-categorical
      [12,1] scale-scale
      [12,2] scale-categorical
      [11,2] scale-categorical
      [15,1] scale-scale
      [16,1] scale-scale
      [15,2] scale-categorical
      [16,2] scale-categorical
      [11,3] scale-categorical
      [12,3] scale-categorical
      [15,3] scale-categorical
      [12,4] scale-categorical
      [14,3] scale-categorical
      [16,3] scale-categorical
      [13,3] scale-categorical
      [15,4] scale-categorical
      [12,5] scale-categorical
      [16,4] scale-categorical
      [13,4] scale-categorical
      [15,5] scale-categorical
      [11,4] scale-categorical
      [12,6] scale-categorical
      [13,5] scale-categorical
      [16,5] scale-categorical
      [14,4] scale-categorical
      [15,6] scale-categorical
      [11,5] scale-categorical
      [12,7] scale-categorical
      [13,6] scale-categorical
      [14,5] scale-categorical
      [16,6] scale-categorical
      [12,8] scale-categorical
      [13,7] scale-categorical
      [15,7] scale-categorical
      [14,6] scale-categorical
      [11,6] scale-categorical
      [16,7] scale-categorical
      [12,9] scale-categorical
      [13,8] scale-categorical
      [14,7] scale-categorical
      [11,7] scale-categorical
      [16,8] scale-categorical
      [14,8] scale-categorical
      [13,9] scale-categorical
      [12,10] scale-categorical
      [11,8] scale-categorical
      [15,8] scale-categorical
      [16,9] scale-categorical
      [14,9] scale-categorical
      [13,10] scale-categorical
      [11,9] scale-categorical
      [15,9] scale-categorical
      [16,10] scale-categorical
      [14,10] scale-categorical
      [11,10] scale-categorical
      [15,10] scale-categorical
      [19,1] scale-scale
      [17,1] scale-scale
      [19,2] scale-categorical
      [17,2] scale-categorical
      [18,1] scale-scale
      [18,2] scale-categorical
      [19,3] scale-categorical
      [20,1] scale-scale
      [17,3] scale-categorical
      [20,2] scale-categorical
      [18,3] scale-categorical
      [19,4] scale-categorical
      [20,3] scale-categorical
      [17,4] scale-categorical
      [19,5] scale-categorical
      [18,4] scale-categorical
      [17,5] scale-categorical
      [19,6] scale-categorical
      [20,4] scale-categorical
      [18,5] scale-categorical
      [17,6] scale-categorical
      [19,7] scale-categorical
      [20,5] scale-categorical
      [18,6] scale-categorical
      [17,7] scale-categorical
      [19,8] scale-categorical
      [20,6] scale-categorical
      [18,7] scale-categorical
      [17,8] scale-categorical
      [19,9] scale-categorical
      [17,9] scale-categorical
      [18,8] scale-categorical
      [20,7] scale-categorical
      [19,10] scale-categorical
      [18,9] scale-categorical
      [20,8] scale-categorical
      [17,10] scale-categorical
      [20,9] scale-categorical
      [18,10] scale-categorical
      [20,10] scale-categorical
      [8,2] categorical-categorical
      [3,2] categorical-categorical
      [4,2] categorical-categorical
      [10,2] categorical-categorical
      [9,2] categorical-categorical
      [6,2] categorical-categorical
      [2,2] categorical-categorical
      [1,2] categorical-categorical
      [7,2] categorical-categorical
      [5,2] categorical-categorical
      17/11/17 16:08:05 WARN codegen.SpoofOperator: codegen.TMP43: Converted 566x640, nnz=9829 sideways input matrix from sparse to dense.
      17/11/17 16:08:05 WARN codegen.SpoofOperator: codegen.TMP22: Converted 727x640, nnz=9886 sideways input matrix from sparse to dense.
      17/11/17 16:08:05 WARN codegen.SpoofOperator: codegen.TMP57: Converted 1100x640, nnz=9934 sideways input matrix from sparse to dense.
      17/11/17 16:08:05 WARN codegen.SpoofOperator: codegen.TMP50: Converted 1100x640, nnz=9934 sideways input matrix from sparse to dense.
      17/11/17 16:08:05 WARN codegen.SpoofOperator: codegen.TMP79: Converted 1100x640, nnz=9922 sideways input matrix from sparse to dense.
      17/11/17 16:08:05 WARN codegen.SpoofOperator: codegen.TMP64: Converted 1100x640, nnz=9941 sideways input matrix from sparse to dense.
      17/11/17 16:08:05 WARN codegen.SpoofOperator: codegen.TMP36: Converted 1051x640, nnz=9924 sideways input matrix from sparse to dense.
      17/11/17 16:08:05 WARN codegen.SpoofOperator: codegen.TMP29: Converted 870x640, nnz=9916 sideways input matrix from sparse to dense.
      17/11/17 16:08:05 WARN codegen.SpoofOperator: codegen.TMP71: Converted 1100x640, nnz=9923 sideways input matrix from sparse to dense.
      17/11/17 16:08:05 WARN codegen.SpoofOperator: codegen.TMP85: Converted 1100x640, nnz=9938 sideways input matrix from sparse to dense.
      [1,2] ordinal-ordinal
      [9,3] categorical-categorical
      [2,2] ordinal-ordinal
      [8,3] categorical-categorical
      [10,3] categorical-categorical
      [5,2] ordinal-ordinal
      [6,3] categorical-categorical
      17/11/17 16:08:05 WARN codegen.SpoofOperator: codegen.TMP85: Converted 1100x1100, nnz=9961 sideways input matrix from sparse to dense.
      [7,3] categorical-categorical
      [4,2] ordinal-ordinal
      17/11/17 16:08:05 WARN codegen.SpoofOperator: codegen.TMP57: Converted 727x1100, nnz=9935 sideways input matrix from sparse to dense.
      17/11/17 16:08:05 WARN codegen.SpoofOperator: codegen.TMP57: Converted 566x1100, nnz=9918 sideways input matrix from sparse to dense.
      17/11/17 16:08:05 WARN codegen.SpoofOperator: codegen.TMP57: Converted 1100x1100, nnz=9964 sideways input matrix from sparse to dense.
      17/11/17 16:08:05 WARN codegen.SpoofOperator: codegen.TMP57: Converted 1100x1100, nnz=9958 sideways input matrix from sparse to dense.
      [3,2] ordinal-ordinal
      [9,4] categorical-categorical
      [8,4] categorical-categorical
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP57: Converted 566x429, nnz=9790 sideways input matrix from sparse to dense.
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP57: Converted 727x429, nnz=9849 sideways input matrix from sparse to dense.
      [6,4] categorical-categorical
      [10,4] categorical-categorical
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP57: Converted 1100x429, nnz=9900 sideways input matrix from sparse to dense.
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP57: Converted 1100x429, nnz=9904 sideways input matrix from sparse to dense.
      [7,4] categorical-categorical
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP85: Converted 1100x429, nnz=9915 sideways input matrix from sparse to dense.
      [9,5] categorical-categorical
      [8,5] categorical-categorical
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP57: Converted 727x635, nnz=9889 sideways input matrix from sparse to dense.
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP85: Converted 566x635, nnz=9848 sideways input matrix from sparse to dense.
      [10,5] categorical-categorical
      [6,5] categorical-categorical
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP57: Converted 1100x635, nnz=9922 sideways input matrix from sparse to dense.
      [7,5] categorical-categorical
      [1,3] categorical-categorical
      [2,3] categorical-categorical
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP57: Converted 1100x635, nnz=9930 sideways input matrix from sparse to dense.
      [5,3] categorical-categorical
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP85: Converted 1100x1100, nnz=9959 sideways input matrix from sparse to dense.
      [9,6] categorical-categorical
      [8,6] categorical-categorical
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP57: Converted 1100x1100, nnz=9963 sideways input matrix from sparse to dense.
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP57: Converted 1100x1100, nnz=9969 sideways input matrix from sparse to dense.
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP57: Converted 1100x635, nnz=9926 sideways input matrix from sparse to dense.
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP85: Converted 566x1100, nnz=9930 sideways input matrix from sparse to dense.
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP57: Converted 727x1100, nnz=9940 sideways input matrix from sparse to dense.
      [10,6] categorical-categorical
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP57: Converted 1100x1100, nnz=9958 sideways input matrix from sparse to dense.
      [8,7] categorical-categorical
      [7,6] categorical-categorical
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP85: Converted 566x1100, nnz=9915 sideways input matrix from sparse to dense.
      [6,6] categorical-categorical
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP57: Converted 1100x1100, nnz=9966 sideways input matrix from sparse to dense.
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP57: Converted 1100x1100, nnz=9960 sideways input matrix from sparse to dense.
      [9,7] categorical-categorical
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP57: Converted 727x1100, nnz=9928 sideways input matrix from sparse to dense.
      [3,3] categorical-categorical
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP85: Converted 870x1100, nnz=9957 sideways input matrix from sparse to dense.
      [8,8] categorical-categorical
      [4,3] categorical-categorical
      [9,8] categorical-categorical
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP57: Converted 566x1100, nnz=9915 sideways input matrix from sparse to dense.
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP85: Converted 1051x1100, nnz=9961 sideways input matrix from sparse to dense.
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP57: Converted 727x1100, nnz=9933 sideways input matrix from sparse to dense.
      [10,7] categorical-categorical
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP57: Converted 1100x1100, nnz=1100 sideways input matrix from sparse to dense.
      [6,7] categorical-categorical
      [3,3] ordinal-ordinal
      [7,7] categorical-categorical
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP85: Converted 1100x1100, nnz=9949 sideways input matrix from sparse to dense.
      [1,3] ordinal-ordinal
      [8,9] categorical-categorical
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP85: Converted 566x1100, nnz=9922 sideways input matrix from sparse to dense.
      [2,3] ordinal-ordinal
      [5,3] ordinal-ordinal
      [9,9] categorical-categorical
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP57: Converted 1100x1100, nnz=9964 sideways input matrix from sparse to dense.
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP85: Converted 727x1100, nnz=9940 sideways input matrix from sparse to dense.
      [8,10] categorical-categorical
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP85: Converted 566x444, nnz=9795 sideways input matrix from sparse to dense.
      [3,4] categorical-categorical
      [6,8] categorical-categorical
      [10,8] categorical-categorical
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP85: Converted 870x429, nnz=9883 sideways input matrix from sparse to dense.
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP85: Converted 1100x1100, nnz=9959 sideways input matrix from sparse to dense.
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP85: Converted 1100x1100, nnz=9954 sideways input matrix from sparse to dense.
      [9,10] categorical-categorical
      [4,3] ordinal-ordinal
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP85: Converted 727x444, nnz=9830 sideways input matrix from sparse to dense.
      [3,4] ordinal-ordinal
      [7,8] categorical-categorical
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP57: Converted 1100x1100, nnz=9960 sideways input matrix from sparse to dense.
      [6,9] categorical-categorical
      [10,9] categorical-categorical
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP85: Converted 1100x1100, nnz=9957 sideways input matrix from sparse to dense.
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP85: Converted 1100x1100, nnz=9963 sideways input matrix from sparse to dense.
      [7,9] categorical-categorical
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP85: Converted 1100x1100, nnz=9964 sideways input matrix from sparse to dense.
      [10,10] categorical-categorical
      [6,10] categorical-categorical
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP57: Converted 1100x444, nnz=9900 sideways input matrix from sparse to dense.
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP57: Converted 1100x444, nnz=9902 sideways input matrix from sparse to dense.
      [7,10] categorical-categorical
      17/11/17 16:08:06 WARN codegen.SpoofOperator: codegen.TMP85: Converted 1100x444, nnz=9885 sideways input matrix from sparse to dense.
      [3,5] categorical-categorical
      [1,4] categorical-categorical
      17/11/17 16:08:07 WARN codegen.SpoofOperator: codegen.TMP85: Converted 870x635, nnz=9922 sideways input matrix from sparse to dense.
      [5,4] categorical-categorical
      [4,4] categorical-categorical
      [2,4] categorical-categorical
      17/11/17 16:08:07 WARN codegen.SpoofOperator: codegen.TMP57: Converted 1100x429, nnz=9905 sideways input matrix from sparse to dense.
      17/11/17 16:08:07 WARN codegen.SpoofOperator: codegen.TMP57: Converted 1100x429, nnz=9888 sideways input matrix from sparse to dense.
      17/11/17 16:08:07 WARN codegen.SpoofOperator: codegen.TMP57: Converted 1051x429, nnz=9890 sideways input matrix from sparse to dense.
      17/11/17 16:08:07 WARN codegen.SpoofOperator: codegen.TMP57: Converted 1100x429, nnz=9881 sideways input matrix from sparse to dense.
      [5,4] ordinal-ordinal
      [1,4] ordinal-ordinal
      [2,4] ordinal-ordinal
      [4,4] ordinal-ordinal
      [3,5] ordinal-ordinal
      [5,5] categorical-categorical
      [1,5] categorical-categorical
      17/11/17 16:08:07 WARN codegen.SpoofOperator: codegen.TMP57: Converted 1100x635, nnz=9920 sideways input matrix from sparse to dense.
      17/11/17 16:08:07 WARN codegen.SpoofOperator: codegen.TMP85: Converted 1100x635, nnz=9933 sideways input matrix from sparse to dense.
      [2,5] categorical-categorical
      [3,6] categorical-categorical
      17/11/17 16:08:07 WARN codegen.SpoofOperator: codegen.TMP57: Converted 1100x635, nnz=9932 sideways input matrix from sparse to dense.
      [4,5] categorical-categorical
      17/11/17 16:08:07 WARN codegen.SpoofOperator: codegen.TMP57: Converted 870x1100, nnz=9953 sideways input matrix from sparse to dense.
      17/11/17 16:08:07 WARN codegen.SpoofOperator: codegen.TMP57: Converted 1051x635, nnz=9939 sideways input matrix from sparse to dense.
      [5,5] ordinal-ordinal
      [1,5] ordinal-ordinal
      [2,5] ordinal-ordinal
      [4,5] ordinal-ordinal
      [3,7] categorical-categorical
      17/11/17 16:08:07 WARN codegen.SpoofOperator: codegen.TMP57: Converted 870x1100, nnz=9950 sideways input matrix from sparse to dense.
      [4,6] categorical-categorical
      [5,6] categorical-categorical
      17/11/17 16:08:07 WARN codegen.SpoofOperator: codegen.TMP85: Converted 1100x1100, nnz=9943 sideways input matrix from sparse to dense.
      [2,6] categorical-categorical
      17/11/17 16:08:07 WARN codegen.SpoofOperator: codegen.TMP85: Converted 1100x1100, nnz=9960 sideways input matrix from sparse to dense.
      [1,6] categorical-categorical
      17/11/17 16:08:07 WARN codegen.SpoofOperator: codegen.TMP85: Converted 1051x1100, nnz=9955 sideways input matrix from sparse to dense.
      17/11/17 16:08:07 WARN codegen.SpoofOperator: codegen.TMP85: Converted 1100x1100, nnz=9962 sideways input matrix from sparse to dense.
      [3,8] categorical-categorical
      17/11/17 16:08:07 WARN codegen.SpoofOperator: codegen.TMP57: Converted 870x1100, nnz=9959 sideways input matrix from sparse to dense.
      [1,7] categorical-categorical
      17/11/17 16:08:07 WARN codegen.SpoofOperator: codegen.TMP57: Converted 1100x1100, nnz=9964 sideways input matrix from sparse to dense.
      [4,7] categorical-categorical
      [3,9] categorical-categorical
      [5,7] categorical-categorical
      17/11/17 16:08:07 WARN codegen.SpoofOperator: codegen.TMP85: Converted 1051x1100, nnz=9955 sideways input matrix from sparse to dense.
      17/11/17 16:08:07 WARN codegen.SpoofOperator: codegen.TMP57: Converted 1100x1100, nnz=9949 sideways input matrix from sparse to dense.
      [2,7] categorical-categorical
      17/11/17 16:08:07 WARN codegen.SpoofOperator: codegen.TMP85: Converted 870x1100, nnz=9940 sideways input matrix from sparse to dense.
      17/11/17 16:08:07 WARN codegen.SpoofOperator: codegen.TMP57: Converted 1100x1100, nnz=9962 sideways input matrix from sparse to dense.
      [1,8] categorical-categorical
      [3,10] categorical-categorical
      17/11/17 16:08:07 WARN codegen.SpoofOperator: codegen.TMP85: Converted 1100x1100, nnz=9952 sideways input matrix from sparse to dense.
      17/11/17 16:08:07 WARN codegen.SpoofOperator: codegen.TMP57: Converted 870x444, nnz=9867 sideways input matrix from sparse to dense.
      [4,8] categorical-categorical
      17/11/17 16:08:07 WARN codegen.SpoofOperator: codegen.TMP57: Converted 1051x1100, nnz=9952 sideways input matrix from sparse to dense.
      [2,8] categorical-categorical
      [5,8] categorical-categorical
      17/11/17 16:08:07 WARN codegen.SpoofOperator: codegen.TMP57: Converted 1100x1100, nnz=9952 sideways input matrix from sparse to dense.
      17/11/17 16:08:07 WARN codegen.SpoofOperator: codegen.TMP85: Converted 1100x1100, nnz=9958 sideways input matrix from sparse to dense.
      [1,9] categorical-categorical
      17/11/17 16:08:07 WARN codegen.SpoofOperator: codegen.TMP85: Converted 1100x1100, nnz=9959 sideways input matrix from sparse to dense.
      [4,9] categorical-categorical
      [5,9] categorical-categorical
      17/11/17 16:08:07 WARN codegen.SpoofOperator: codegen.TMP57: Converted 1051x1100, nnz=9962 sideways input matrix from sparse to dense.
      17/11/17 16:08:07 WARN codegen.SpoofOperator: codegen.TMP85: Converted 1100x1100, nnz=9961 sideways input matrix from sparse to dense.
      [2,9] categorical-categorical
      17/11/17 16:08:07 WARN codegen.SpoofOperator: codegen.TMP85: Converted 1100x1100, nnz=9961 sideways input matrix from sparse to dense.
      [1,10] categorical-categorical
      17/11/17 16:08:07 WARN codegen.SpoofOperator: codegen.TMP57: Converted 1100x444, nnz=9906 sideways input matrix from sparse to dense.
      [5,10] categorical-categorical
      [4,10] categorical-categorical
      17/11/17 16:08:07 WARN codegen.SpoofOperator: codegen.TMP85: Converted 1100x444, nnz=9895 sideways input matrix from sparse to dense.
      17/11/17 16:08:07 WARN codegen.SpoofOperator: codegen.TMP85: Converted 1051x444, nnz=9895 sideways input matrix from sparse to dense.
      [2,10] categorical-categorical
      17/11/17 16:08:07 WARN codegen.SpoofOperator: codegen.TMP57: Converted 1100x444, nnz=9887 sideways input matrix from sparse to dense.
      

      Attachments

        Issue Links

          Activity

            People

              mboehm7 Matthias Boehm
              mboehm7 Matthias Boehm
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: