Uploaded image for project: 'Apache Nemo'
  1. Apache Nemo
  2. NEMO-47

Use primitive types for AlternatingLeastSquare

    XMLWordPrintableJSON

    Details

      Description

      Change List<Integer> and List<Double> to int[] and float[].

      Using the primitive types gave me around 2X performance improvements for processing a Yahoo! Music dataset. The major benefit comes from creating a significantly fewer number of objects. Using floats instead of doubles also seem to improve overall performance.

      Note that Beam does not provide efficient coders for in[] and float[] by default, so we need to add custom coders for those types to avoid using the inefficient Java serializer.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                johnyangk John Yang
                Reporter:
                johnyangk John Yang
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: