Uploaded image for project: 'Chukwa'
  1. Chukwa
  2. CHUKWA-736

SSL for agent rest api and rest adaptor

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.7.0
    • Fix Version/s: None
    • Component/s: Data Collection
    • Labels:
      None

      Description

      Users can optionally enable https for agent rest api and RestAdaptor.

      1. CHUKWA-736.patch
        20 kB
        shreyas subramanya
      2. CHUKWA-736.patch2
        20 kB
        shreyas subramanya
      3. CHUKWA-736.patch3
        21 kB
        shreyas subramanya

        Activity

        Hide
        eyang Eric Yang added a comment -

        Could we change REST url to /v2/ instead? I think ChukwaSslConstant can be renamed to ChukwaConstant to make it more scalable. Thanks

        Show
        eyang Eric Yang added a comment - Could we change REST url to /v2/ instead? I think ChukwaSslConstant can be renamed to ChukwaConstant to make it more scalable. Thanks
        Hide
        shreyass123 shreyas subramanya added a comment -

        Thanks Eric, I have made the modifications.

        Show
        shreyass123 shreyas subramanya added a comment - Thanks Eric, I have made the modifications.
        Hide
        eyang Eric Yang added a comment -

        Shreyas, it looks like the test case doesn't setup some environment properly. Here are the errors during test:

        -------------------------------------------------------------------------------
        Test set: org.apache.hadoop.chukwa.datacollection.agent.TestChukwaSsl
        -------------------------------------------------------------------------------
        Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.295 sec <<< FAILURE!
        testRestServer(org.apache.hadoop.chukwa.datacollection.agent.TestChukwaSsl) Time elapsed: 0.271 sec <<< ERROR!
        java.lang.NullPointerException
        at org.apache.hadoop.chukwa.datacollection.agent.TestChukwaSsl.testRestServer(TestChukwaSsl.java:40)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at junit.framework.TestCase.runTest(TestCase.java:168)
        at junit.framework.TestCase.runBare(TestCase.java:134)
        at junit.framework.TestResult$1.protect(TestResult.java:110)
        at junit.framework.TestResult.runProtected(TestResult.java:128)
        at junit.framework.TestResult.run(TestResult.java:113)
        at junit.framework.TestCase.run(TestCase.java:124)
        at junit.framework.TestSuite.runTest(TestSuite.java:243)
        at junit.framework.TestSuite.run(TestSuite.java:238)
        at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
        at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53)
        at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:123)
        at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:104)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:164)
        at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:110)
        at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:175)
        at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcessWhenForked(SurefireStarter.java:81)
        at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:68)

        And it's caused by:

        java.io.FileNotFoundException: /Users/eric/sandbox/chukwa/o/trunk/target/test/share/chukwa/lib/agent.dict (No such file or directory)
        at java.io.FileInputStream.open(Native Method)
        at java.io.FileInputStream.<init>(FileInputStream.java:138)
        at edu.berkeley.confspell.OptDictionary.read(OptDictionary.java:59)
        at edu.berkeley.confspell.OptDictionary.<init>(OptDictionary.java:16)
        at org.apache.hadoop.chukwa.util.ChukwaUtil.readConfiguration(ChukwaUtil.java:68)
        at org.apache.hadoop.chukwa.datacollection.agent.TestChukwaSsl.testRestServer(TestChukwaSsl.java:38)

        We probably want to do a resource copy from source's lib/agent.dict to target/test to mirror the CHUKWA_HOME during test phase.

        Show
        eyang Eric Yang added a comment - Shreyas, it looks like the test case doesn't setup some environment properly. Here are the errors during test: ------------------------------------------------------------------------------- Test set: org.apache.hadoop.chukwa.datacollection.agent.TestChukwaSsl ------------------------------------------------------------------------------- Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.295 sec <<< FAILURE! testRestServer(org.apache.hadoop.chukwa.datacollection.agent.TestChukwaSsl) Time elapsed: 0.271 sec <<< ERROR! java.lang.NullPointerException at org.apache.hadoop.chukwa.datacollection.agent.TestChukwaSsl.testRestServer(TestChukwaSsl.java:40) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83) at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:123) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:104) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:164) at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:110) at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:175) at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcessWhenForked(SurefireStarter.java:81) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:68) And it's caused by: java.io.FileNotFoundException: /Users/eric/sandbox/chukwa/o/trunk/target/test/share/chukwa/lib/agent.dict (No such file or directory) at java.io.FileInputStream.open(Native Method) at java.io.FileInputStream.<init>(FileInputStream.java:138) at edu.berkeley.confspell.OptDictionary.read(OptDictionary.java:59) at edu.berkeley.confspell.OptDictionary.<init>(OptDictionary.java:16) at org.apache.hadoop.chukwa.util.ChukwaUtil.readConfiguration(ChukwaUtil.java:68) at org.apache.hadoop.chukwa.datacollection.agent.TestChukwaSsl.testRestServer(TestChukwaSsl.java:38) We probably want to do a resource copy from source's lib/agent.dict to target/test to mirror the CHUKWA_HOME during test phase.
        Hide
        shreyass123 shreyas subramanya added a comment -

        Hi Eric, looks like there are 2 issues here

        • chukwa.store is a binary file and hence not included in patch (cause of NPE)
        • agent.dict not found (test case would pass even with this error)
          Let me try to generate a new store file during the testcase setup using keytool, so that this will work with any jvm.
        Show
        shreyass123 shreyas subramanya added a comment - Hi Eric, looks like there are 2 issues here chukwa.store is a binary file and hence not included in patch (cause of NPE) agent.dict not found (test case would pass even with this error) Let me try to generate a new store file during the testcase setup using keytool, so that this will work with any jvm.
        Hide
        shreyass123 shreyas subramanya added a comment -

        Fixed the test case.

        Show
        shreyass123 shreyas subramanya added a comment - Fixed the test case.
        Hide
        eyang Eric Yang added a comment -

        +1 Thanks Shreyas

        Show
        eyang Eric Yang added a comment - +1 Thanks Shreyas
        Hide
        shreyass123 shreyas subramanya added a comment -

        Thanks Eric, committed this.

        Show
        shreyass123 shreyas subramanya added a comment - Thanks Eric, committed this.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Chukwa-trunk #522 (See https://builds.apache.org/job/Chukwa-trunk/522/)
        CHUKWA-736. SSL support for chukwa (shreyas: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1662360)

        • /chukwa/trunk/CHANGES.txt
        • /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datacollection/adaptor/RestAdaptor.java
        • /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datacollection/agent/ChukwaAgent.java
        • /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datacollection/agent/ChukwaConstants.java
        • /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datacollection/agent/ChukwaRestServer.java
        • /chukwa/trunk/src/test/java/org/apache/hadoop/chukwa/datacollection/agent/TestChukwaSsl.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Chukwa-trunk #522 (See https://builds.apache.org/job/Chukwa-trunk/522/ ) CHUKWA-736 . SSL support for chukwa (shreyas: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1662360 ) /chukwa/trunk/CHANGES.txt /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datacollection/adaptor/RestAdaptor.java /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datacollection/agent/ChukwaAgent.java /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datacollection/agent/ChukwaConstants.java /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datacollection/agent/ChukwaRestServer.java /chukwa/trunk/src/test/java/org/apache/hadoop/chukwa/datacollection/agent/TestChukwaSsl.java

          People

          • Assignee:
            shreyass123 shreyas subramanya
            Reporter:
            shreyass123 shreyas subramanya
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development