Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-3466

MongoDB issue with Drill 1.1.0 which was working in 1.0.0

    Details

    • Type: Bug
    • Status: In Progress
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.1.0
    • Fix Version/s: Future
    • Component/s: Storage - MongoDB
    • Labels:
      None
    • Environment:

      Windows, Apache Drill 1.1.0

      Description

      I was working with Drill 1.0.0 and MongoDB 3.0.4 and trying different queries, when I found that some kind of queries are well supported in Drill 1.1.0. That's when I switched to latest released Drill 1.1.0. But to my surprise the simplest query on MongoDB stopped working.

      For example MongoDB had following data:
      > db.events.find();
      { "_id" : ObjectId("559b4eaf014b4d18f8049d2a"), "deviceId" : 9876, "eventId" : "eventA", "eventInfo" :

      { "eventStatus" : true, "at" : ISODate("2015-07-07T03:59:43.488Z") }

      }
      { "_id" : ObjectId("559b4eb0014b4d18f8049d2b"), "deviceId" : 9876, "eventId" : "eventA", "eventInfo" :

      { "eventStatus" : false, "at" : ISODate("2015-07-07T03:59:44.655Z") }

      }
      { "_id" : ObjectId("559b4eb1014b4d18f8049d2c"), "deviceId" : 9876, "eventId" : "eventB", "eventInfo" :

      { "eventStatus" : true, "at" : ISODate("2015-07-07T03:59:45.658Z") }

      }
      { "_id" : ObjectId("559b4eb2014b4d18f8049d2d"), "deviceId" : 9876, "eventId" : "eventB", "eventInfo" :

      { "eventStatus" : false, "at" : ISODate("2015-07-07T03:59:46.664Z") }

      }
      { "_id" : ObjectId("559b4eb3014b4d18f8049d2e"), "deviceId" : 9876, "eventId" : "eventA", "eventInfo" :

      { "eventStatus" : true, "at" : ISODate("2015-07-07T03:59:47.670Z") }

      }
      { "_id" : ObjectId("559b4eb4014b4d18f8049d2f"), "deviceId" : 9876, "eventId" : "eventA", "eventInfo" :

      { "eventStatus" : false, "at" : ISODate("2015-07-07T03:59:48.676Z") }

      }

      Running with apache drill 1.1.0
      ==============================
      D:\Programs\apache_softwares\apache-drill-1.1.0\bin>sqlline.bat -u "jdbc:drill:zk=local"
      DRILL_ARGS - " -u jdbc:drill:zk=local"
      HADOOP_HOME not detected...
      HBASE_HOME not detected...
      Calculating Drill classpath...
      Jul 07, 2015 9:31:06 AM org.glassfish.jersey.server.ApplicationHandler initialize
      INFO: Initiating Jersey application, version Jersey: 2.8 2014-04-29 01:25:26...
      apache drill 1.0.0
      "a drill in the hand is better than two in the bush"
      0: jdbc:drill:zk=local> show databases;
      ---------------------

      SCHEMA_NAME

      ---------------------

      INFORMATION_SCHEMA
      cp.default
      dfs.default
      dfs.root
      dfs.tmp
      mongo.local
      mongo.testDB
      sys

      ---------------------
      8 rows selected (0.225 seconds)
      0: jdbc:drill:zk=local> select * from mongo.testDB.events;
      Error: SYSTEM ERROR: JsonParseException: Failure while reading ExtendedJSON typed value. Expected a VALUE_STRING but received a token of type VALUE_NUMBER_INT
      at [Source: org.apache.drill.exec.vector.complex.fn.SeekableBAIS@14c0bfbe; line: 1, column: 116]

      Fragment 0:0

      [Error Id: ab808e76-228f-4dc5-bf48-acb106e7675d on BGHM32007.ad.aricent.com:31010] (state=,code=0)
      0: jdbc:drill:zk=local> !quit
      Closing: org.apache.drill.jdbc.impl.DrillConnectionImpl

      switching back to Drill 1.0.0
      ========================
      D:\Programs\apache_softwares\apache-drill-1.1.0\bin>cd ..

      D:\Programs\apache_softwares\apache-drill-1.1.0>cd ..

      D:\Programs\apache_softwares>cd apache-drill-1.0.0

      D:\Programs\apache_softwares\apache-drill-1.0.0>cd bin

      D:\Programs\apache_softwares\apache-drill-1.0.0\bin>dir
      Volume in drive D is Data Disk
      Volume Serial Number is D4B5-4BF9

      Directory of D:\Programs\apache_softwares\apache-drill-1.0.0\bin

      07/06/2015 06:00 PM <DIR> .
      07/06/2015 06:00 PM <DIR> ..
      05/16/2015 08:05 AM 978 drill-conf
      05/16/2015 08:05 AM 6,342 drill-config.sh
      05/16/2015 08:05 AM 964 drill-embedded
      05/16/2015 08:05 AM 988 drill-localhost
      05/16/2015 08:05 AM 6,581 drillbit.sh
      05/16/2015 08:05 AM 1,078 dumpcat
      05/16/2015 08:05 AM 104 hadoop-excludes.txt
      05/16/2015 08:05 AM 1,114 runbit
      05/16/2015 08:05 AM 2,288 sqlline
      05/16/2015 08:05 AM 5,914 sqlline.bat
      05/16/2015 08:05 AM 1,139 submit_plan
      11 File(s) 27,490 bytes
      2 Dir(s) 139,456,872,448 bytes free

      D:\Programs\apache_softwares\apache-drill-1.0.0\bin>sqlline.bat -u "jdbc:drill:zk=local"
      DRILL_ARGS - " -u jdbc:drill:zk=local"
      HADOOP_HOME not detected...
      HBASE_HOME not detected...
      Calculating Drill classpath...
      Error setting configuration: isolation: java.lang.NullPointerException
      Jul 07, 2015 9:33:43 AM org.glassfish.jersey.server.ApplicationHandler initialize
      INFO: Initiating Jersey application, version Jersey: 2.8 2014-04-29 01:25:26...
      apache drill 1.0.0
      "the only truly happy people are children, the creative minority and drill users"
      0: jdbc:drill:zk=local> show databases;
      ---------------------

      SCHEMA_NAME

      ---------------------

      INFORMATION_SCHEMA
      cp.default
      dfs.default
      dfs.root
      dfs.tmp
      mongo.local
      mongo.testDB
      sys

      ---------------------
      8 rows selected (2.538 seconds)
      0: jdbc:drill:zk=local> select * from mongo.testDB.events;
      ----------------------------------------------------------------------------------------------------------------------------------------------------------------

      deviceId eventId  

      ----------------------------------------------------------------------------------------------------------------------------------------------------------------

      9876.0 eventA {"eventStatus":true,"at":{"dayOfYear":188,"dayOfMonth":7,"dayOfWeek":2,"era":1,"year":2015,"millisOfDay":34183488,"weekOfWeekyear":28,"yearOfCentu
      9876.0 eventA {"eventStatus":false,"at":{"dayOfYear":188,"dayOfMonth":7,"dayOfWeek":2,"era":1,"year":2015,"millisOfDay":34184655,"weekOfWeekyear":28,"yearOfCent
      9876.0 eventB {"eventStatus":true,"at":{"dayOfYear":188,"dayOfMonth":7,"dayOfWeek":2,"era":1,"year":2015,"millisOfDay":34185658,"weekOfWeekyear":28,"yearOfCentu
      9876.0 eventB {"eventStatus":false,"at":{"dayOfYear":188,"dayOfMonth":7,"dayOfWeek":2,"era":1,"year":2015,"millisOfDay":34186664,"weekOfWeekyear":28,"yearOfCent
      9876.0 eventA {"eventStatus":true,"at":{"dayOfYear":188,"dayOfMonth":7,"dayOfWeek":2,"era":1,"year":2015,"millisOfDay":34187670,"weekOfWeekyear":28,"yearOfCentu
      9876.0 eventA {"eventStatus":false,"at":{"dayOfYear":188,"dayOfMonth":7,"dayOfWeek":2,"era":1,"year":2015,"millisOfDay":34188676,"weekOfWeekyear":28,"yearOfCent
      9876.0 eventB {"eventStatus":true,"at":{"dayOfYear":188,"dayOfMonth":7,"dayOfWeek":2,"era":1,"year":2015,"millisOfDay":34189683,"weekOfWeekyear":28,"yearOfCentu
      9876.0 eventB {"eventStatus":false,"at":{"dayOfYear":188,"dayOfMonth":7,"dayOfWeek":2,"era":1,"year":2015,"millisOfDay":34190689,"weekOfWeekyear":28,"yearOfCent
      9876.0 eventA {"eventStatus":true,"at":{"dayOfYear":188,"dayOfMonth":7,"dayOfWeek":2,"era":1,"year":2015,"millisOfDay":34191694,"weekOfWeekyear":28,"yearOfCentu
      9876.0 eventA {"eventStatus":false,"at":{"dayOfYear":188,"dayOfMonth":7,"dayOfWeek":2,"era":1,"year":2015,"millisOfDay":34192696,"weekOfWeekyear":28,"yearOfCent
      9876.0 eventB {"eventStatus":true,"at":{"dayOfYear":188,"dayOfMonth":7,"dayOfWeek":2,"era":1,"year":2015,"millisOfDay":34193701,"weekOfWeekyear":28,"yearOfCentu
      9876.0 eventB {"eventStatus":false,"at":{"dayOfYear":188,"dayOfMonth":7,"dayOfWeek":2,"era":1,"year":2015,"millisOfDay":34194707,"weekOfWeekyear":28,"yearOfCent
      9876.0 eventA {"eventStatus":true,"at":{"dayOfYear":188,"dayOfMonth":7,"dayOfWeek":2,"era":1,"year":2015,"millisOfDay":34195712,"weekOfWeekyear":28,"yearOfCentu
      9876.0 eventA {"eventStatus":false,"at":{"dayOfYear":188,"dayOfMonth":7,"dayOfWeek":2,"era":1,"year":2015,"millisOfDay":34196718,"weekOfWeekyear":28,"yearOfCent
      9876.0 eventB {"eventStatus":true,"at":{"dayOfYear":188,"dayOfMonth":7,"dayOfWeek":2,"era":1,"year":2015,"millisOfDay":34197723,"weekOfWeekyear":28,"yearOfCentu

      ----------------------------------------------------------------------------------------------------------------------------------------------------------------
      15 rows selected (0.215 seconds)

      So it worked with Drill 1.0.0 but failed with 1.1.0.

        Activity

        Hide
        Filip23 Filip Grabowski added a comment -

        I have similar observations. At the moment I can only query elements that are strings. Numbers and dates do not work.

        Show
        Filip23 Filip Grabowski added a comment - I have similar observations. At the moment I can only query elements that are strings. Numbers and dates do not work.
        Hide
        jnadeau Jacques Nadeau added a comment -

        I'll take a look. The reading code didn't change much but we did upgrade the Mongo driver. I'm wondering if that caused an issue.

        Show
        jnadeau Jacques Nadeau added a comment - I'll take a look. The reading code didn't change much but we did upgrade the Mongo driver. I'm wondering if that caused an issue.

          People

          • Assignee:
            Unassigned
            Reporter:
            vksjoshi Vikas
          • Votes:
            6 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:

              Development