Hadoop Common
  1. Hadoop Common
  2. HADOOP-6596

Should add version to the serialization of DelegationToken

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: security
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Now that we are adding the serialized form of delegation tokens into the http interfaces, we should include some version information.

      1. c-6596.patch
        1 kB
        Owen O'Malley
      2. c-6596.patch
        1 kB
        Owen O'Malley

        Issue Links

          Activity

          Hide
          Owen O'Malley added a comment -

          Here is a trivial patch to add a version byte, which should be sufficient. The other option is if someone wants to show me how to convert the delegation tokens to Avro. smile

          Show
          Owen O'Malley added a comment - Here is a trivial patch to add a version byte, which should be sufficient. The other option is if someone wants to show me how to convert the delegation tokens to Avro. smile
          Hide
          Owen O'Malley added a comment -

          I should actually, further comment that for this use case, it would be awkward to use Avro, precisely because we don't have a metadata channel. In this case, having the metadata embedded in the object's serialization is right.

          Show
          Owen O'Malley added a comment - I should actually, further comment that for this use case, it would be awkward to use Avro, precisely because we don't have a metadata channel. In this case, having the metadata embedded in the object's serialization is right.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12436866/c-6596.patch
          against trunk revision 915168.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          -1 patch. The patch command could not apply the patch.

          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/377/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12436866/c-6596.patch against trunk revision 915168. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/377/console This message is automatically generated.
          Hide
          Owen O'Malley added a comment -

          Forgot to include --no-prefix on the patch

          Show
          Owen O'Malley added a comment - Forgot to include --no-prefix on the patch
          Hide
          Jeff Hammerbacher added a comment -

          Hey Owen,

          Could you describe to me what you mean by "metadata channel"? The Avro file object container has room in the header for extensible metadata, as does the Avro RPC handshake request.

          Thanks,
          Jeff

          Show
          Jeff Hammerbacher added a comment - Hey Owen, Could you describe to me what you mean by "metadata channel"? The Avro file object container has room in the header for extensible metadata, as does the Avro RPC handshake request. Thanks, Jeff
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12436885/c-6596.patch
          against trunk revision 915168.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/378/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/378/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/378/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/378/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12436885/c-6596.patch against trunk revision 915168. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/378/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/378/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/378/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/378/console This message is automatically generated.
          Hide
          Owen O'Malley added a comment -

          Jeff, my point was that I am trying to put a serialized copy of the delegation token in a http request. Including a full-blown avro schema in a http request would be overkill. The context here is that I'm trying to pass along a delegation token in a http request. Since this will be used for hftp, enabling compatibility between versions is required and therefore I need versioning support. Going to thrift or protobufs seems like overkill for this, so I'm proposing adding a version byte that will let us change the serialization later.

          Show
          Owen O'Malley added a comment - Jeff, my point was that I am trying to put a serialized copy of the delegation token in a http request. Including a full-blown avro schema in a http request would be overkill. The context here is that I'm trying to pass along a delegation token in a http request. Since this will be used for hftp, enabling compatibility between versions is required and therefore I need versioning support. Going to thrift or protobufs seems like overkill for this, so I'm proposing adding a version byte that will let us change the serialization later.
          Hide
          Kan Zhang added a comment -

          Assuming we agree on the approach, +1 on the patch.

          Show
          Kan Zhang added a comment - Assuming we agree on the approach, +1 on the patch.
          Hide
          Owen O'Malley added a comment -

          Since this changes the serialization format of the type, there isn't a reasonable unit test.

          I just committed this.

          Show
          Owen O'Malley added a comment - Since this changes the serialization format of the type, there isn't a reasonable unit test. I just committed this.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk #261 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk/261/)
          . Add a version field to the AbstractDelegationTokenIdentifier's
          serialized value. (omalley)

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk #261 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk/261/ ) . Add a version field to the AbstractDelegationTokenIdentifier's serialized value. (omalley)
          Hide
          Jeff Hammerbacher added a comment -

          Hey Owen,

          Avro doesn't just perform serialization, it also defines an RPC protocol. There's currently versioned RPC over HTTP for Java in Avro 1.3. It seems like you're hand-coding a portion of the Avro RPC functionality--why not try out Avro here? My apologies if I don't understand your particular use case well enough to see the drawback.

          Thanks,
          Jeff

          Show
          Jeff Hammerbacher added a comment - Hey Owen, Avro doesn't just perform serialization, it also defines an RPC protocol. There's currently versioned RPC over HTTP for Java in Avro 1.3. It seems like you're hand-coding a portion of the Avro RPC functionality--why not try out Avro here? My apologies if I don't understand your particular use case well enough to see the drawback. Thanks, Jeff
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #193 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk-Commit/193/)
          . Add a version field to the AbstractDelegationTokenIdentifier's
          serialized value. (omalley)

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #193 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk-Commit/193/ ) . Add a version field to the AbstractDelegationTokenIdentifier's serialized value. (omalley)

            People

            • Assignee:
              Owen O'Malley
              Reporter:
              Owen O'Malley
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development