Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-3987

NNLS generates incorrect result

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.1.0
    • 1.1.1, 1.2.0
    • MLlib
    • None

    Description

      Hi,

      Please see the example gram matrix and linear term:

      val P2 = new DoubleMatrix(20, 20, 333907.312770, -60814.043975, 207935.829941, -162881.367739, -43730.396770, 17511.428983, -243340.496449, -225245.957922, 104700.445881, 32430.845099, 336378.693135, -373497.970207, -41147.159621, 53928.060360, -293517.883778, 53105.278068, 0.000000, -85257.781696, 84913.970469, -10584.080103, -60814.043975, 13826.806664, -38032.612640, 33475.833875, 10791.916809, -1040.950810, 48106.552472, 45390.073380, -16310.282190, -2861.455903, -60790.833191, 73109.516544, 9826.614644, -8283.992464, 56991.742991, -6171.366034, 0.000000, 19152.382499, -13218.721710, 2793.734234, 207935.829941, -38032.612640, 129661.677608, -101682.098412, -27401.299347, 10787.713362, -151803.006149, -140563.601672, 65067.935324, 20031.263383, 209521.268600, -232958.054688, -25764.179034, 33507.951918, -183046.845592, 32884.782835, 0.000000, -53315.811196, 52770.762546, -6642.187643, -162881.367739, 33475.833875, -101682.098412, 85094.407608, 25422.850782, -5437.646141, 124197.166330, 116206.265909, -47093.484134, -11420.168521, -163429.436848, 189574.783900, 23447.172314, -24087.375367, 148311.355507, -20848.385466, 0.000000, 46835.814559, -38180.352878, 6415.873901, -43730.396770, 10791.916809, -27401.299347, 25422.850782, 8882.869799, 15.638084, 35933.473986, 34186.371325, -10745.330690, -974.314375, -43537.709621, 54371.010558, 7894.453004, -5408.929644, 42231.381747, -3192.010574, 0.000000, 15058.753110, -8704.757256, 2316.581535, 17511.428983, -1040.950810, 10787.713362, -5437.646141, 15.638084, 2794.949847, -9681.950987, -8258.171646, 7754.358930, 4193.359412, 18052.143842, -15456.096769, -253.356253, 4089.672804, -12524.380088, 5651.579348, 0.000000, -1513.302547, 6296.461898, 152.427321, -243340.496449, 48106.552472, -151803.006149, 124197.166330, 35933.473986, -9681.950987, 182931.600236, 170454.352953, -72361.174145, -19270.461728, -244518.179729, 279551.060579, 33340.452802, -37103.267653, 219025.288975, -33687.141423, 0.000000, 67347.950443, -58673.009647, 8957.800259, -225245.957922, 45390.073380, -140563.601672, 116206.265909, 34186.371325, -8258.171646, 170454.352953, 159322.942894, -66074.960534, -16839.743193, -226173.967766, 260421.044094, 31624.194003, -33839.612565, 203889.695169, -30034.828909, 0.000000, 63525.040745, -53572.741748, 8575.071847, 104700.445881, -16310.282190, 65067.935324, -47093.484134, -10745.330690, 7754.358930, -72361.174145, -66074.960534, 35869.598076, 13378.653317, 106033.647837, -111831.682883, -10455.465743, 18537.392481, -88370.612394, 20344.288488, 0.000000, -22935.482766, 29004.543704, -2409.461759, 32430.845099, -2861.455903, 20031.263383, -11420.168521, -974.314375, 4193.359412, -19270.461728, -16839.743193, 13378.653317, 6802.081898, 33256.395091, -30421.985199, -1296.785870, 7026.518692, -24443.378205, 9221.982599, 0.000000, -4088.076871, 10861.014242, -25.092938, 336378.693135, -60790.833191, 209521.268600, -163429.436848, -43537.709621, 18052.143842, -244518.179729, -226173.967766, 106033.647837, 33256.395091, 339200.268106, -375442.716811, -41027.594509, 54636.778527, -295133.248586, 54177.278365, 0.000000, -85237.666701, 85996.957056, -10503.209968, -373497.970207, 73109.516544, -232958.054688, 189574.783900, 54371.010558, -15456.096769, 279551.060579, 260421.044094, -111831.682883, -30421.985199, -375442.716811, 427793.208465, 50528.074431, -57375.986301, 335203.382015, -52676.385869, 0.000000, 102368.307670, -90679.792485, 13509.390393, -41147.159621, 9826.614644, -25764.179034, 23447.172314, 7894.453004, -253.356253, 33340.452802, 31624.194003, -10455.465743, -1296.785870, -41027.594509, 50528.074431, 7255.977434, -5281.636812, 39298.355527, -3440.450858, 0.000000, 13717.870243, -8471.405582, 2071.812204, 53928.060360, -8283.992464, 33507.951918, -24087.375367, -5408.929644, 4089.672804, -37103.267653, -33839.612565, 18537.392481, 7026.518692, 54636.778527, -57375.986301, -5281.636812, 9735.061160, -45360.674033, 10634.633559, 0.000000, -11652.364691, 15039.566630, -1202.539106, -293517.883778, 56991.742991, -183046.845592, 148311.355507, 42231.381747, -12524.380088, 219025.288975, 203889.695169, -88370.612394, -24443.378205, -295133.248586, 335203.382015, 39298.355527, -45360.674033, 262923.925938, -42012.606885, 0.000000, 79810.919951, -71657.856143, 10464.327491, 53105.278068, -6171.366034, 32884.782835, -20848.385466, -3192.010574, 5651.579348, -33687.141423, -30034.828909, 20344.288488, 9221.982599, 54177.278365, -52676.385869, -3440.450858, 10634.633559, -42012.606885, 13238.686902, 0.000000, -8739.845698, 16511.872845, -530.252003, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 118.430000, 0.000000, 0.000000, 0.000000, -85257.781696, 19152.382499, -53315.811196, 46835.814559, 15058.753110, -1513.302547, 67347.950443, 63525.040745, -22935.482766, -4088.076871, -85237.666701, 102368.307670, 13717.870243, -11652.364691, 79810.919951, -8739.845698, 0.000000, 26878.133950, -18588.407734, 3894.934299, 84913.970469, -13218.721710, 52770.762546, -38180.352878, -8704.757256, 6296.461898, -58673.009647, -53572.741748, 29004.543704, 10861.014242, 85996.957056, -90679.792485, -8471.405582, 15039.566630, -71657.856143, 16511.872845, 0.000000, -18588.407734, 23649.538538, -1951.083671, -10584.080103, 2793.734234, -6642.187643, 6415.873901, 2316.581535, 152.427321, 8957.800259, 8575.071847, -2409.461759, -25.092938, -10503.209968, 13509.390393, 2071.812204, -1202.539106, 10464.327491, -530.252003, 0.000000, 3894.934299, -1951.083671, 738.955915)
      val q2 = new DoubleMatrix(20, 1, 31755.057100, -13047.148129, 20191.244430, -25993.775800, -11963.550172, -4272.425977, -33569.856044, -33451.387021, 2320.764250, -5333.136834, 30633.764272, -49513.939049, -10351.230305, 872.276714, -37634.078430, -4628.338543, -0.000000, -18109.093788, 1856.725521, -3397.693211)

      NNLS result:

      0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0

      PR result: https://github.com/apache/spark/pull/2705

      QuadraticMinimizer result [0.130104; 0.126840; 0.072565; 0.154153; 0.144728; 0.129937; 0.121166; 0.161450; 0.199197; 0.187168; 0.159467; 0.144269; 0.117404; 0.109298; 0.086901; 0.221391; 0.000000; 0.174044; 0.162080; 0.045439]

      Octave result:
      octave:7> qp(x0, H, f, [], [], lb, ub)
      ans =
      0.13010
      0.12684
      0.07256
      0.15415
      0.14473
      0.12994
      0.12117
      0.16145
      0.19920
      0.18717
      0.15947
      0.14427
      0.11740
      0.10930
      0.08690
      0.22139
      0.00000
      0.17404
      0.16208
      0.04544

      Attachments

        Activity

          People

            coderxiang Shuo Xiang
            debasish83 Debasish Das
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: