Uploaded image for project: 'Apache Nemo'
  1. Apache Nemo
  2. NEMO-47

Use primitive types for AlternatingLeastSquare

    XMLWordPrintableJSON

Details

    Description

      Change List<Integer> and List<Double> to int[] and float[].

      Using the primitive types gave me around 2X performance improvements for processing a Yahoo! Music dataset. The major benefit comes from creating a significantly fewer number of objects. Using floats instead of doubles also seem to improve overall performance.

      Note that Beam does not provide efficient coders for in[] and float[] by default, so we need to add custom coders for those types to avoid using the inefficient Java serializer.

      Attachments

        Issue Links

          Activity

            People

              johnyangk John Yang
              johnyangk John Yang
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: