[DRILL-4511] refresh over empty folder results in error, we need a better error message

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.7.0
• Fix Version/s: 1.11.0
• Component/s: Storage - Parquet
• Labels: ready-to-commit
• Assignee: Roman Kulyk
• Reporter: Khurram Faraaz
• Reviewer: Chun Chang
• Created: 15/Mar/16 11:01
• Updated: 18/Oct/17 22:05
• Resolved: 21/Jul/17 13:37

Description

refresh table metadata <empty-folder> results in error on sqlline prompt. We need a better message on sqlline prompt, something like, "The folder is empty, and there is no metadata to refresh"

[root@centos-01 ~]# hadoop fs -ls /tmp
...
drwxr-xr-x - root root 0 2015-07-21 21:16 /tmp/complex
...

0: jdbc:drill:schema=dfs.tmp> refresh table metadata complex;
+--------+--------------+
|   ok   |   summary    |
+--------+--------------+
| false  | Error: null  |
+--------+--------------+
1 row selected (0.131 seconds)

Stack trace from drillbit.log

2016-03-15 10:50:23,747 [2918170f-84ee-a6fb-38eb-266108bd64f3:foreman] INFO  o.a.drill.exec.work.foreman.Foreman - Query text for query id 2918170f-84ee-a6fb-38eb-266108bd64f3: refresh table metadata complex
2016-03-15 10:50:23,790 [2918170f-84ee-a6fb-38eb-266108bd64f3:foreman] ERROR o.a.d.e.p.s.h.RefreshMetadataHandler - Failed to update metadata for table 'complex'
java.lang.NullPointerException: null
        at org.apache.drill.exec.store.dfs.FileSelection.minusDirectories(FileSelection.java:153) ~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
        at org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.create(WorkspaceSchemaFactory.java:594) ~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
        at org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.create(WorkspaceSchemaFactory.java:370) ~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
        at org.apache.drill.exec.planner.sql.ExpandingConcurrentMap.getNewEntry(ExpandingConcurrentMap.java:96) ~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
        at org.apache.drill.exec.planner.sql.ExpandingConcurrentMap.get(ExpandingConcurrentMap.java:90) ~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
        at org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.getTable(WorkspaceSchemaFactory.java:524) ~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
        at org.apache.calcite.jdbc.SimpleCalciteSchema.getTable(SimpleCalciteSchema.java:82) ~[calcite-core-1.4.0-drill-r10.jar:1.4.0-drill-r10]
        at org.apache.calcite.jdbc.CalciteAbstractSchema$SchemaPlusImpl.getTable(CalciteAbstractSchema.java:177) ~[calcite-core-1.4.0-drill-r10.jar:1.4.0-drill-r10]
        at org.apache.drill.exec.planner.sql.handlers.RefreshMetadataHandler.getPlan(RefreshMetadataHandler.java:73) ~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
        at org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:94) [drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
        at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:927) [drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
        at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:251) [drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_45]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_45]
        at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
2016-03-15 10:50:23,801 [2918170f-84ee-a6fb-38eb-266108bd64f3:frag:0:0] INFO  o.a.d.e.w.fragment.FragmentExecutor - 2918170f-84ee-a6fb-38eb-266108bd64f3:0:0: State change requested AWAITING_ALLOCATION --> RUNNING
2016-03-15 10:50:23,801 [2918170f-84ee-a6fb-38eb-266108bd64f3:frag:0:0] INFO  o.a.d.e.w.f.FragmentStatusReporter - 2918170f-84ee-a6fb-38eb-266108bd64f3:0:0: State to report: RUNNING
2016-03-15 10:50:23,808 [2918170f-84ee-a6fb-38eb-266108bd64f3:frag:0:0] INFO  o.a.d.e.w.fragment.FragmentExecutor - 2918170f-84ee-a6fb-38eb-266108bd64f3:0:0: State change requested RUNNING --> FINISHED
2016-03-15 10:50:23,808 [2918170f-84ee-a6fb-38eb-266108bd64f3:frag:0:0] INFO  o.a.d.e.w.f.FragmentStatusReporter - 2918170f-84ee-a6fb-38eb-266108bd64f3:0:0: State to report: FINISHED

Issue Links

• links to: GitHub Pull Request #869