Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-5025

Fix flaky test failures in TestLoad.java

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.17.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Test cases testCommaSeparatedString2 and testGlobChars may fail if for some reason files (from any other sources) in /tmp have : (colon) in the filenames. This is because HDFS doesn't support colon since it has its own URI handling. Exception below.
      I propose we separate the working dir of these tests to use their own folder in /tmp.

      Failed to parse: java.net.URISyntaxException: Relative path in absolute URI: t:2sTest.txt
      at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:198)
      at org.apache.pig.test.TestLoad.checkLoadPath(TestLoad.java:317)
      at org.apache.pig.test.TestLoad.checkLoadPath(TestLoad.java:299)
      at org.apache.pig.test.TestLoad.testCommaSeparatedString2(TestLoad.java:189)
      Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: t:2sTest.txt
      at org.apache.hadoop.fs.Path.initialize(Path.java:206)
      at org.apache.hadoop.fs.Path.<init>(Path.java:172)
      at org.apache.hadoop.fs.Path.<init>(Path.java:94)
      at org.apache.hadoop.fs.Globber.doGlob(Globber.java:260)
      at org.apache.hadoop.fs.Globber.glob(Globber.java:151)
      at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:1637)
      at org.apache.pig.backend.hadoop.datastorage.HDataStorage.asCollection(HDataStorage.java:215)
      at org.apache.pig.backend.hadoop.datastorage.HDataStorage.asCollection(HDataStorage.java:41)
      at org.apache.pig.builtin.JsonMetadata.findMetaFile(JsonMetadata.java:119)
      at org.apache.pig.builtin.JsonMetadata.getSchema(JsonMetadata.java:191)
      at org.apache.pig.builtin.PigStorage.getSchema(PigStorage.java:518)
      at org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:175)
      at org.apache.pig.newplan.logical.relational.LOLoad.<init>(LOLoad.java:89)
      at org.apache.pig.parser.LogicalPlanBuilder.buildLoadOp(LogicalPlanBuilder.java:866)
      at org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3568)
      at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1625)
      at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1102)
      at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:560)
      at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421)
      at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:188)
      Caused by: java.net.URISyntaxException: Relative path in absolute URI: t:2sTest.txt
      at java.net.URI.checkPath(URI.java:1823)
      at java.net.URI.<init>(URI.java:745)
      at org.apache.hadoop.fs.Path.initialize(Path.java:203)

        Attachments

        1. PIG-5025.1.patch
          8 kB
          Ádám Szita
        2. PIG-5025.2.patch
          8 kB
          Ádám Szita
        3. PIG-5025.3.patch
          8 kB
          Ádám Szita
        4. PIG-5025.patch
          8 kB
          Ádám Szita

          Activity

            People

            • Assignee:
              szita Ádám Szita
              Reporter:
              szita Ádám Szita
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: