Uploaded image for project: 'Apache Knox'
  1. Apache Knox
  2. KNOX-1602

JsonFilterReader should handle strings, numbers, booleans, and null at root

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.2.0
    • Component/s: None
    • Labels:
      None

      Description

      While looking at KNOX-1098, I generated an invalid query to Livy which resulted in the following response from Livy:

      HTTP/1.1 400 Bad Request
      Date: Wed, 14 Nov 2018 21:51:22 GMT
      Content-Type: application/json; charset=UTF-8
      Content-Length: 520
      Server: Jetty(9.2.16.v20160414)
      "Unrecognized field \"kind\" (class org.apache.livy.server.batch.CreateBatchRequest), not marked as ignorable (16 known properties: \"executorCores\", \"className\", \"conf\", \"driverMemory\", \"name\", \"driverCores\", \"pyFiles\", \"archives\", \"executorMemory\", \"files\", \"jars\", \"proxyUser\", \"numExecutors\", \"file\", \"args\", \"queue\"])\n at [Source: (org.eclipse.jetty.server.HttpInputOverHTTP); line: 1, column: 10] (through reference chain: org.apache.livy.server.batch.CreateBatchRequest[\"kind\"])"
      

      Knox failed to handle this response and instead threw the following exception:

      2018-11-14 16:49:12,489 ERROR knox.gateway (GatewayServlet.java:service(147)) - Gateway processing failed: javax.servlet.ServletException: org.apache.shiro.subject.ExecutionException: java.security.PrivilegedActionException: javax.servlet.ServletException: java.util.EmptyStackException
      javax.servlet.ServletException: org.apache.shiro.subject.ExecutionException: java.security.PrivilegedActionException: javax.servlet.ServletException: java.util.EmptyStackException
      at org.apache.shiro.web.servlet.AdviceFilter.cleanup(AdviceFilter.java:196)
      at org.apache.shiro.web.filter.authc.AuthenticatingFilter.cleanup(AuthenticatingFilter.java:155)
      at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:148)
      at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
      at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
      at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449)
      at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365)
      at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90)
      at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83)
      at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:387)
      at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362)
      at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
      at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:369)
      at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:269)
      at org.apache.knox.gateway.filter.ResponseCookieFilter.doFilter(ResponseCookieFilter.java:50)
      at org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:58)
      at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:369)
      at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:269)
      at org.apache.knox.gateway.filter.XForwardedHeaderFilter.doFilter(XForwardedHeaderFilter.java:30)
      at org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:58)
      at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:369)
      at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:269)
      at org.apache.knox.gateway.GatewayFilter.doFilter(GatewayFilter.java:168)
      at org.apache.knox.gateway.GatewayFilter.doFilter(GatewayFilter.java:91)
      at org.apache.knox.gateway.GatewayServlet.service(GatewayServlet.java:142)
      at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:859)
      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1623)
      at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:214)
      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
      at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
      at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
      at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
      at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588)
      at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
      at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)
      at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
      at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
      at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557)
      at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
      at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)
      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
      at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220)
      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
      at org.apache.knox.gateway.trace.TraceHandler.handle(TraceHandler.java:51)
      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
      at org.apache.knox.gateway.filter.CorrelationHandler.handle(CorrelationHandler.java:41)
      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
      at org.apache.knox.gateway.filter.PortMappingHelperHandler.handle(PortMappingHelperHandler.java:150)
      at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
      at org.eclipse.jetty.server.Server.handle(Server.java:502)
      at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
      at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
      at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
      at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
      at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:411)
      at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:305)
      at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:159)
      at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
      at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
      at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
      at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
      at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
      at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
      at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
      at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
      at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
      at java.lang.Thread.run(Thread.java:748)
      Caused by: org.apache.shiro.subject.ExecutionException: java.security.PrivilegedActionException: javax.servlet.ServletException: java.util.EmptyStackException
      at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:389)
      at org.apache.knox.gateway.filter.ShiroSubjectIdentityAdapter.doFilter(ShiroSubjectIdentityAdapter.java:72)
      at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:369)
      at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:269)
      at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:61)
      at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
      at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
      ... 67 more
      Caused by: java.security.PrivilegedActionException: javax.servlet.ServletException: java.util.EmptyStackException
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:422)
      at org.apache.knox.gateway.filter.ShiroSubjectIdentityAdapter$CallableChain.call(ShiroSubjectIdentityAdapter.java:143)
      at org.apache.knox.gateway.filter.ShiroSubjectIdentityAdapter$CallableChain.call(ShiroSubjectIdentityAdapter.java:75)
      at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90)
      at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83)
      at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:387)
      ... 73 more
      Caused by: javax.servlet.ServletException: java.util.EmptyStackException
      at org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:67)
      at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:369)
      at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:269)
      at org.apache.knox.gateway.identityasserter.common.filter.AbstractIdentityAssertionFilter.doFilterInternal(AbstractIdentityAssertionFilter.java:195)
      at org.apache.knox.gateway.identityasserter.common.filter.AbstractIdentityAssertionFilter.continueChainAsPrincipal(AbstractIdentityAssertionFilter.java:152)
      at org.apache.knox.gateway.identityasserter.common.filter.CommonIdentityAssertionFilter.doFilter(CommonIdentityAssertionFilter.java:93)
      at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:369)
      at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:269)
      at org.apache.knox.gateway.filter.rewrite.api.UrlRewriteServletFilter.doFilter(UrlRewriteServletFilter.java:57)
      at org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:58)
      at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:369)
      at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:269)
      at org.apache.knox.gateway.filter.ShiroSubjectIdentityAdapter$CallableChain$1.run(ShiroSubjectIdentityAdapter.java:91)
      at org.apache.knox.gateway.filter.ShiroSubjectIdentityAdapter$CallableChain$1.run(ShiroSubjectIdentityAdapter.java:88)
      ... 80 more
      Caused by: java.util.EmptyStackException
      at java.util.Stack.peek(Stack.java:102)
      at org.apache.knox.gateway.filter.rewrite.impl.json.JsonFilterReader.processValueString(JsonFilterReader.java:308)
      at org.apache.knox.gateway.filter.rewrite.impl.json.JsonFilterReader.processCurrentToken(JsonFilterReader.java:129)
      at org.apache.knox.gateway.filter.rewrite.impl.json.JsonFilterReader.read(JsonFilterReader.java:93)
      at org.apache.knox.gateway.filter.rewrite.impl.json.JsonUrlRewriteFilterReader.read(JsonUrlRewriteFilterReader.java:32)
      at org.apache.commons.io.input.ReaderInputStream.fillBuffer(ReaderInputStream.java:202)
      at org.apache.commons.io.input.ReaderInputStream.read(ReaderInputStream.java:246)
      at org.apache.commons.io.input.ReaderInputStream.read(ReaderInputStream.java:265)
      at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:2314)
      at org.apache.commons.io.IOUtils.copy(IOUtils.java:2270)
      at org.apache.knox.gateway.filter.rewrite.impl.UrlRewriteResponse.streamResponse(UrlRewriteResponse.java:193)
      at org.apache.knox.gateway.filter.GatewayResponseWrapper.streamResponse(GatewayResponseWrapper.java:58)
      at org.apache.knox.gateway.dispatch.AbstractGatewayDispatch.writeResponse(AbstractGatewayDispatch.java:61)
      at org.apache.knox.gateway.dispatch.DefaultDispatch.writeOutboundResponse(DefaultDispatch.java:181)
      at org.apache.knox.gateway.dispatch.DefaultDispatch.executeRequest(DefaultDispatch.java:115)
      at org.apache.knox.gateway.dispatch.DefaultDispatch.doPost(DefaultDispatch.java:302)
      at org.apache.knox.gateway.livy.LivyDispatch.doPost(LivyDispatch.java:49)
      at org.apache.knox.gateway.dispatch.GatewayDispatchFilter$PostAdapter.doMethod(GatewayDispatchFilter.java:179)
      at org.apache.knox.gateway.dispatch.GatewayDispatchFilter.doFilter(GatewayDispatchFilter.java:124)
      at org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:58)
      ... 93 more
      

      The root cause is that JsonFilterReader assumes that the Json will always be an object or an array. A simple unit test added to JsonFilterReaderTest throws the same EmptyStackException.

      @Test
      public void testString() throws IOException {
      String inputJson = "\"abc\"";
      StringReader inputReader = new StringReader( inputJson );
      JsonFilterReader filterReader = new TestJsonFilterReader( inputReader, null );
      String outputJson = new String( IOUtils.toCharArray( filterReader ) );
      System.out.println( "JSON=" + outputJson );
      
      JsonAssert.with( outputJson ).assertThat( "name<test-name>", is( "value:null<test-value>" ) );
      }

        Attachments

        1. KNOX-1602.patch
          5 kB
          Kevin Risden

          Activity

            People

            • Assignee:
              krisden Kevin Risden
              Reporter:
              krisden Kevin Risden
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: