Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-19464

Remove support for Hadoop 2.5 and earlier

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.1.0
    • Fix Version/s: 2.2.0
    • Component/s: Spark Core, YARN
    • Labels:
    • Target Version/s:

      Description

      As discussed last year, and again on the mailing list this week, it's likely time to remove support for Hadoop 2.5 and earlier. Distros have been on 2.6 for over 2 years, and upstream is on to 2.7 / 3.x. It only affects Hadoop cluster users. 2.1.x continues to work on 2.2+.

      The benefit is mostly cleaning up the build matrix, reasoning about dependencies across Hadoop versions, and bits of code across the project that depend on reflection to deal with newer features. It makes it a little easier to use newer Hadoop APIs. De facto, I doubt that versions before 2.6 would be given serious support in the project already anyway.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                srowen Sean Owen
                Reporter:
                srowen Sean Owen
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: