Details

    • Type: Task Task
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Invalid
    • Affects Version/s: 0.23.0
    • Fix Version/s: 2.0.0-alpha
    • Component/s: None
    • Labels:
      None

      Description

      This is the MR part of removing deprecated RecordIO packages - parented by HADOOP-7781.

      Basically, we need to remove /hadoop-mapreduce-project/src/c++/librecordio and all associated build helpers around it.

      (For posterity, RecordIO has been replaced by Apache Avro http://avro.apache.org)

        Issue Links

          Activity

          Hide
          Harsh J added a comment -

          Ok I looked carefully again and there seems to be more:

          • TypedBytes uses and depends on the record package to provide byte buffers. Streaming uses this. It will have to be shifted to use something else in place. I'll try to get Klaas to comment on this.
          Show
          Harsh J added a comment - Ok I looked carefully again and there seems to be more: TypedBytes uses and depends on the record package to provide byte buffers. Streaming uses this. It will have to be shifted to use something else in place. I'll try to get Klaas to comment on this.
          Hide
          Klaas Bosteels added a comment -

          Not sure I agree with the removal of RecordIO, but that's a different discussion..

          The TypedBytes functionality in Streaming doesn't really have an actual hard decency on RecordIO, it just supports it. RecordIO records are automatically converted to typed bytes when they are taken as input so that streaming programs can easily read sequence files that contain them, but removing the RecordIO support will not break typed bytes based Streaming altogether.

          I guess we should add support for Avro (instead) though if that's going to be the new standard, but that could be a separate JIRA issue.

          Show
          Klaas Bosteels added a comment - Not sure I agree with the removal of RecordIO, but that's a different discussion.. The TypedBytes functionality in Streaming doesn't really have an actual hard decency on RecordIO, it just supports it. RecordIO records are automatically converted to typed bytes when they are taken as input so that streaming programs can easily read sequence files that contain them, but removing the RecordIO support will not break typed bytes based Streaming altogether. I guess we should add support for Avro (instead) though if that's going to be the new standard, but that could be a separate JIRA issue.
          Hide
          Harsh J added a comment -

          Per Owen on the parent JIRA, the deprecated recordio is still in use apparently and should not be removed.

          We can pursue this once we have an action plan later (if we do).

          Show
          Harsh J added a comment - Per Owen on the parent JIRA, the deprecated recordio is still in use apparently and should not be removed. We can pursue this once we have an action plan later (if we do).
          Hide
          Harsh J added a comment -

          Klaas,

          Thanks for commenting here! I noticed it too late, sorry.

          I'll poke around the typedbytes source when I get some time and see if I can help with an avro transition.

          Apparently, RecordIO is still in use by external users out there , and we're not removing it for now.

          Show
          Harsh J added a comment - Klaas, Thanks for commenting here! I noticed it too late, sorry. I'll poke around the typedbytes source when I get some time and see if I can help with an avro transition. Apparently, RecordIO is still in use by external users out there , and we're not removing it for now.

            People

            • Assignee:
              Harsh J
              Reporter:
              Harsh J
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development