Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-13721

HPL/SQL COPY FROM FTP Statement: lack of DIR option leads to NPE

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.3.0
    • Component/s: hpl/sql
    • Labels:
      None

      Description

      The docs (http://www.hplsql.org/copy-from-ftp) suggest DIR is optional. When I left it out in:

      copy from ftp hdp250.example.com user 'vagrant' pwd 'vagrant'  files 'sampledata.csv' to /tmp overwrite
      

      I got:

      Ln:2 Connected to ftp: hdp250.example.com (29 ms)
      Ln:2 Retrieving directory listing
        Listing the current working FTP directory
      Ln:2 Files to copy: 45 bytes, 1 file, 0 subdirectories scanned (27 ms)
      Exception in thread "main" java.lang.NullPointerException
      	at org.apache.hive.hplsql.Ftp.getTargetFileName(Ftp.java:342)
      	at org.apache.hive.hplsql.Ftp.run(Ftp.java:149)
      	at org.apache.hive.hplsql.Ftp.copyFiles(Ftp.java:121)
      	at org.apache.hive.hplsql.Ftp.run(Ftp.java:91)
      	at org.apache.hive.hplsql.Exec.visitCopy_from_ftp_stmt(Exec.java:1292)
      	at org.apache.hive.hplsql.Exec.visitCopy_from_ftp_stmt(Exec.java:52)
      	at org.apache.hive.hplsql.HplsqlParser$Copy_from_ftp_stmtContext.accept(HplsqlParser.java:11956)
      	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:70)
      	at org.apache.hive.hplsql.Exec.visitStmt(Exec.java:994)
      	at org.apache.hive.hplsql.Exec.visitStmt(Exec.java:52)
      	at org.apache.hive.hplsql.HplsqlParser$StmtContext.accept(HplsqlParser.java:1012)
      	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:70)
      	at org.apache.hive.hplsql.HplsqlBaseVisitor.visitBlock(HplsqlBaseVisitor.java:28)
      	at org.apache.hive.hplsql.HplsqlParser$BlockContext.accept(HplsqlParser.java:446)
      	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:70)
      	at org.apache.hive.hplsql.Exec.visitProgram(Exec.java:901)
      	at org.apache.hive.hplsql.Exec.visitProgram(Exec.java:52)
      	at org.apache.hive.hplsql.HplsqlParser$ProgramContext.accept(HplsqlParser.java:389)
      	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visit(AbstractParseTreeVisitor.java:42)
      	at org.apache.hive.hplsql.Exec.run(Exec.java:760)
      	at org.apache.hive.hplsql.Exec.run(Exec.java:736)
      	at org.apache.hive.hplsql.Hplsql.main(Hplsql.java:23)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
      	at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
      

      Traceback leads to:

        /**
         * Get the target file relative path and name
         */
        String getTargetFileName(String file) {
          int len = dir.length();
          return targetDir + file.substring(len);
        }
      

      in Ftp.java

      When I added DIR '/' this worked.

        Attachments

          Activity

            People

            • Assignee:
              dmtolpeko Dmitry Tolpeko
              Reporter:
              cartershanklin Carter Shanklin
            • Votes:
              1 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: