Details
-
Bug
-
Status: Open
-
Minor
-
Resolution: Unresolved
-
1.16.0, 1.17.0
-
None
-
None
Description
I can query delimited files in ADLS Gen 2 using wasb blob storage plugin, I can show files and see parquet files but cannot read them or write them.
I can use a DFS plugin to read and write parquet locally but not with Gen 2. ADLS Gen 1 works fine for reading and writing.
I have tried two version of Drill and also the recommended jar files as well as older versions and still no luck.
Does Drill support data lake gen 2 with parquet files.
This query creates the following error.
| CREATE TABLE az.tmp.sampleparquet AS (SELECT * FROM az.`/Conformed/DimGeo.psv`) |
(java.lang.RuntimeException) java.lang.NoSuchMethodError: com.microsoft.azure.storage.blob.CloudBlob.startCopyFromBlob(Ljava/net/URI;Lcom/microsoft/azure/storage/AccessCondition;Lcom/microsoft/azure/storage/AccessCondition;Lcom/microsoft/azure/storage/blob/BlobRequestOptions;Lcom/microsoft/azure/storage/OperationContext;)Ljava/lang/String;
org.apache.drill.common.DeferredException.addThrowable():101
org.apache.drill.exec.work.fragment.FragmentExecutor.fail():475
org.apache.drill.exec.work.fragment.FragmentExecutor.run():317
org.apache.drill.common.SelfCleaningRunnable.run():38
this query create the following error.
select * from az.`region.parquet`
SYSTEM ERROR: StorageException: The requested operation is not allowed in the current state of the entity.
Please, refer to logs for more information.
(org.apache.drill.exec.work.foreman.ForemanException) Unexpected exception during fragment initialization: Error while applying rule DrillScanRule, args [rel#44:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[az, region.parquet])]
org.apache.drill.exec.work.foreman.Foreman.run():302
java.util.concurrent.ThreadPoolExecutor.runWorker():1149
java.util.concurrent.ThreadPoolExecutor$Worker.run():624
java.lang.Thread.run():748