Hadoop Common
  1. Hadoop Common
  2. HADOOP-6725

Evaluate HtmlUnit for unit and regression testing webpages

    Details

    • Type: Test Test
    • Status: Open
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: test
    • Labels:
      None

      Description

      HtmlUnit (http://htmlunit.sourceforge.net/) looks like it may be a good tool to help unit testing and evaluating our various webpages throughout the project. Currently this is done only occasionally in the code (usually falls to being a manual test during release cycles), and when it is done, usually the code to parse the webpage, etc. is re-written each time. The framework is Apache licensed, so including it won't be an issue. If it's found to be useful, new JIRAs for HDFS and MR should be opened.

      1. HADOOP-6725.patch
        0.6 kB
        Konstantin Boudnik

        Issue Links

          Activity

          Hide
          Konstantin Boudnik added a comment -

          Seems like what we need is JSPUnit for our front-end UI is auto-generated by JSP. JSPUnit runs on top of HtmlUnit framework though.

          Show
          Konstantin Boudnik added a comment - Seems like what we need is JSPUnit for our front-end UI is auto-generated by JSP. JSPUnit runs on top of HtmlUnit framework though.
          Hide
          Jakob Homan added a comment -

          That works too.

          Show
          Jakob Homan added a comment - That works too.
          Hide
          Steve Loughran added a comment -

          I am +1 for this and anything else which tests the webapps. HtmlUnit is the best in-java web app test framework I know of; I use it on my code. Right now we have some tests for a servlet or two, but nothing looks at the JSP/webapp content, so things like HADOOP-6461 aren't being picked up

          1. HtmlUnit is looking at moving to the ASF; support from the Hadoop team would be welcome
          2. JSF unit is separate, it's testing Java Server Faces. HtmlUnit will happily test JSP and pages with javascript in them
          3. For ease of testing, HTML elements need IDs. HADOOP-5388 covers this. The tags and the tests can go hand in hand.
          4. I can help with setting up the test framework, try and get Marc Guillemot to review the design too.

          I would really benefit from some HtmlUnit tests that could be run against a live cluster rather than just a miniDFS and miniMR cluster; the tests could be something standalone/redistributable that could be given the URL of a NN, DN, JT or TT and scan through all the expected pages. Having sequences to submit jobs and other actions then becomes further work.

          Show
          Steve Loughran added a comment - I am +1 for this and anything else which tests the webapps. HtmlUnit is the best in-java web app test framework I know of; I use it on my code. Right now we have some tests for a servlet or two, but nothing looks at the JSP/webapp content, so things like HADOOP-6461 aren't being picked up HtmlUnit is looking at moving to the ASF; support from the Hadoop team would be welcome JSF unit is separate, it's testing Java Server Faces. HtmlUnit will happily test JSP and pages with javascript in them For ease of testing, HTML elements need IDs. HADOOP-5388 covers this. The tags and the tests can go hand in hand. I can help with setting up the test framework, try and get Marc Guillemot to review the design too. I would really benefit from some HtmlUnit tests that could be run against a live cluster rather than just a miniDFS and miniMR cluster; the tests could be something standalone/redistributable that could be given the URL of a NN, DN, JT or TT and scan through all the expected pages. Having sequences to submit jobs and other actions then becomes further work.
          Hide
          Konstantin Boudnik added a comment -

          As soon as HADOOP-6332 is in the trunk (I really hope to complete the forward port work before the end of May) we can start using Herriot framework to perform such tests.

          Show
          Konstantin Boudnik added a comment - As soon as HADOOP-6332 is in the trunk (I really hope to complete the forward port work before the end of May) we can start using Herriot framework to perform such tests.
          Hide
          Konstantin Boudnik added a comment -

          I meant 'using HtmlUnit' framework.

          Show
          Konstantin Boudnik added a comment - I meant 'using HtmlUnit' framework.
          Hide
          Marc Guillemot added a comment -

          @Konstantin: this is not JSPUnit but JSFUnit. Do you use JSP or JSF?

          What kind of failures did you have in the past that should not occur again when tested (for instance with HtmlUnit)?

          Show
          Marc Guillemot added a comment - @Konstantin: this is not JSPUnit but JSFUnit. Do you use JSP or JSF? What kind of failures did you have in the past that should not occur again when tested (for instance with HtmlUnit)?
          Hide
          Steve Loughran added a comment -

          right now we have the situation in which pretty much none of the web UI is tested. API, yes, HTTP, no. There is a cute test that fetches some XML from a servlet and hands off the work to Xerces, but otherwise, nothing. This is unfortunate, as HADOOP-6461 shows -it is not clear that on SVN_TRUNK, the webapps are working.

          downstream, some people may be checking the web. My own deployment code takes a list of pages and runs through them, which is how I know about HADOOP-6461.

          What I would like to see test-wise is

          1. Something to run through all the standard HTML, JSP pages, artwork, check for their presence.
          2. Maybe: grab dependencies (stylesheets &c)
          3. Fetch the logs

          The more advanced stuff is to run through some real operations:

          1. create files via the client API, check you you can browse them
          2. submit a job on the command line, and track it via the web UI

          Security:

          1. Make unauthenticated calls and expect rejection.
          2. Have something log <script> text, fetch it from the logs, check the angle-brackets get filtered
          3. Check cookies are marked httponly once that gets adopted.
          4. anything else we can think of.
          Show
          Steve Loughran added a comment - right now we have the situation in which pretty much none of the web UI is tested. API, yes, HTTP, no. There is a cute test that fetches some XML from a servlet and hands off the work to Xerces, but otherwise, nothing. This is unfortunate, as HADOOP-6461 shows -it is not clear that on SVN_TRUNK, the webapps are working. downstream, some people may be checking the web. My own deployment code takes a list of pages and runs through them, which is how I know about HADOOP-6461 . What I would like to see test-wise is Something to run through all the standard HTML, JSP pages, artwork, check for their presence. Maybe: grab dependencies (stylesheets &c) Fetch the logs The more advanced stuff is to run through some real operations: create files via the client API, check you you can browse them submit a job on the command line, and track it via the web UI Security: Make unauthenticated calls and expect rejection. Have something log <script> text, fetch it from the logs, check the angle-brackets get filtered Check cookies are marked httponly once that gets adopted. anything else we can think of.
          Hide
          Konstantin Boudnik added a comment -

          Apparently, it worth mentioning that HtmlUnit requires xalan2.7. And that dependency can't be resolved with Ivy 2.0 (see IVY-1026)

          Show
          Konstantin Boudnik added a comment - Apparently, it worth mentioning that HtmlUnit requires xalan2.7. And that dependency can't be resolved with Ivy 2.0 (see IVY-1026 )
          Hide
          Konstantin Boudnik added a comment -

          To facilitate the evaluation. Please make sure that you are using Ivy at least 2.1.0-rc1

          Show
          Konstantin Boudnik added a comment - To facilitate the evaluation. Please make sure that you are using Ivy at least 2.1.0-rc1

            People

            • Assignee:
              Unassigned
              Reporter:
              Jakob Homan
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:

                Development