Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12527

Upgrade Avro dependency to 1.7.7 or later

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotVotersStop watchingWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: 2.7.1
    • Fix Version/s: None
    • Component/s: build
    • Labels:
      None

      Description

      Hadoop has depended upon Avro 1.7.4 for a couple of years now (see HADOOP-9672), but Apache Spark depends upon what is currently the latest version of Avro (1.7.7).

      This can cause issues if Spark is configured to include the full Hadoop classpath, as the classpath would then contain both Avro 1.7.4 and 1.7.7, with the 1.7.4 classes possibly winning depending on ordering. Here is an example of this issue: http://stackoverflow.com/questions/33159254/avro-error-on-aws-emr/33403111#33403111

      Would it be possible to upgrade Hadoop's Avro dependency to 1.7.7 now?

        Attachments

        Issue Links

          Activity

          $i18n.getText('security.level.explanation', $currentSelection) Viewable by All Users
          Cancel

            People

            • Assignee:
              busbey Sean Busbey Assign to me
              Reporter:
              jonathak Jonathan Kelly

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment