Apache Gora
  1. Apache Gora
  2. GORA-63

bin/compile-examples.sh does not work

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.2
    • Fix Version/s: 0.2.1
    • Component/s: None
    • Labels:
      None

      Description

      As reported by Lewis at gora-dev, bin/compile-examples.sh does not work. The reported exception is:

      lewis@lewis-01:~/ASF/gora$ bin/compile-examples.sh
      Compiling gora-core/src/examples/avro/employee.json
      Compiling gora-core/src/examples/avro/tokendatum.json
      Compiling gora-core/src/examples/avro/webpage.json
      Exception in thread "main" org.apache.avro.SchemaParseException: Map has no
      values type: {"type":"map","items":"string"}
         at org.apache.avro.Schema.parse(Schema.java:938)
         at org.apache.avro.Schema.parse(Schema.java:912)
         at org.apache.avro.Schema.parse(Schema.java:796)
         at
      org.apache.gora.compiler.GoraCompiler.compileSchema(GoraCompiler.java:62)
         at org.apache.gora.compiler.GoraCompiler.main(GoraCompiler.java:448)
      Compiling gora-core/src/examples/avro/webpage.json~
      Exception in thread "main" org.apache.avro.SchemaParseException: Array has
      no items type: {"type":"array","values":"string"}
         at org.apache.avro.Schema.parse(Schema.java:933)
         at org.apache.avro.Schema.parse(Schema.java:912)
         at org.apache.avro.Schema.parse(Schema.java:796)
         at
      org.apache.gora.compiler.GoraCompiler.compileSchema(GoraCompiler.java:62)
         at org.apache.gora.compiler.GoraCompiler.main(GoraCompiler.java:448)
      
      1. GORA-63_v1.patch
        0.8 kB
        Enis Soztutar

        Issue Links

          Activity

          Hide
          Enis Soztutar added a comment -

          It seems the problem is that in webpage.json, the parsedContent field is declared as an arrray, but in GoraCompiler, the Schema.getValuetype() is called instead of the correct Schema.getElementType(). This one-liner patch should fix the problem.

          Show
          Enis Soztutar added a comment - It seems the problem is that in webpage.json, the parsedContent field is declared as an arrray, but in GoraCompiler, the Schema.getValuetype() is called instead of the correct Schema.getElementType(). This one-liner patch should fix the problem.
          Hide
          Lewis John McGibbney added a comment -

          Hi Enis. I am not using Git, but when I applied your changes to my SVN working copy I get the following

          
          lewis@lewis-desktop:~/ASF/gora_trunk$ bin/compile-examples.sh
          Compiling gora-core/src/examples/avro/employee.json
          Compiling gora-core/src/examples/avro/tokendatum.json
          Compiling gora-core/src/examples/avro/webpage.json
          Exception in thread "main" org.apache.avro.AvroRuntimeException: Not a map: {"type":"array","items":"string"}
          	at org.apache.avro.Schema.getValueType(Schema.java:274)
          	at org.apache.gora.compiler.GoraCompiler.compile(GoraCompiler.java:327)
          	at org.apache.gora.compiler.GoraCompiler.compile(GoraCompiler.java:123)
          	at org.apache.gora.compiler.GoraCompiler.compileSchema(GoraCompiler.java:63)
          	at org.apache.gora.compiler.GoraCompiler.main(GoraCompiler.java:448)
          
          

          When I view the changes to my files locally, I get the following, quite strange output.

          lewis@lewis-desktop:~/ASF/gora_trunk$ svn status
          M       gora-core/src/main/java/org/apache/gora/compiler/GoraCompiler.java
          M       gora-core/src/examples/java/org/apache/gora/examples/generated/WebPage.java
          

          Then finally when I look at the changes to WebPage.java I get

          Index: gora-core/src/examples/java/org/apache/gora/examples/generated/WebPage.java
          ===================================================================
          --- gora-core/src/examples/java/org/apache/gora/examples/generated/WebPage.java(revision 1205345)
          +++ gora-core/src/examples/java/org/apache/gora/examples/generated/WebPage.java(working copy)
          @@ -93,33 +93,3 @@
             public void setContent(ByteBuffer value) {
               put(1, value);
             }
          -  public GenericArray<Utf8> getParsedContent() {
          -    return (GenericArray<Utf8>) get(2);
          -  }
          -  public void addToParsedContent(Utf8 element) {
          -    getStateManager().setDirty(this, 2);
          -    parsedContent.add(element);
          -  }
          -  public Map<Utf8, Utf8> getOutlinks() {
          -    return (Map<Utf8, Utf8>) get(3);
          -  }
          -  public Utf8 getFromOutlinks(Utf8 key) {
          -    if (outlinks == null) { return null; }
          -    return outlinks.get(key);
          -  }
          -  public void putToOutlinks(Utf8 key, Utf8 value) {
          -    getStateManager().setDirty(this, 3);
          -    outlinks.put(key, value);
          -  }
          -  public Utf8 removeFromOutlinks(Utf8 key) {
          -    if (outlinks == null) { return null; }
          -    getStateManager().setDirty(this, 3);
          -    return outlinks.remove(key);
          -  }
          -  public Metadata getMetadata() {
          -    return (Metadata) get(4);
          -  }
          -  public void setMetadata(Metadata value) {
          -    put(4, value);
          -  }
          -}
          
          
          
          Show
          Lewis John McGibbney added a comment - Hi Enis. I am not using Git, but when I applied your changes to my SVN working copy I get the following lewis@lewis-desktop:~/ASF/gora_trunk$ bin/compile-examples.sh Compiling gora-core/src/examples/avro/employee.json Compiling gora-core/src/examples/avro/tokendatum.json Compiling gora-core/src/examples/avro/webpage.json Exception in thread "main" org.apache.avro.AvroRuntimeException: Not a map: { "type" : "array" , "items" : "string" } at org.apache.avro.Schema.getValueType(Schema.java:274) at org.apache.gora.compiler.GoraCompiler.compile(GoraCompiler.java:327) at org.apache.gora.compiler.GoraCompiler.compile(GoraCompiler.java:123) at org.apache.gora.compiler.GoraCompiler.compileSchema(GoraCompiler.java:63) at org.apache.gora.compiler.GoraCompiler.main(GoraCompiler.java:448) When I view the changes to my files locally, I get the following, quite strange output. lewis@lewis-desktop:~/ASF/gora_trunk$ svn status M gora-core/src/main/java/org/apache/gora/compiler/GoraCompiler.java M gora-core/src/examples/java/org/apache/gora/examples/generated/WebPage.java Then finally when I look at the changes to WebPage.java I get Index: gora-core/src/examples/java/org/apache/gora/examples/generated/WebPage.java =================================================================== --- gora-core/src/examples/java/org/apache/gora/examples/generated/WebPage.java(revision 1205345) +++ gora-core/src/examples/java/org/apache/gora/examples/generated/WebPage.java(working copy) @@ -93,33 +93,3 @@ public void setContent(ByteBuffer value) { put(1, value); } - public GenericArray<Utf8> getParsedContent() { - return (GenericArray<Utf8>) get(2); - } - public void addToParsedContent(Utf8 element) { - getStateManager().setDirty( this , 2); - parsedContent.add(element); - } - public Map<Utf8, Utf8> getOutlinks() { - return (Map<Utf8, Utf8>) get(3); - } - public Utf8 getFromOutlinks(Utf8 key) { - if (outlinks == null ) { return null ; } - return outlinks.get(key); - } - public void putToOutlinks(Utf8 key, Utf8 value) { - getStateManager().setDirty( this , 3); - outlinks.put(key, value); - } - public Utf8 removeFromOutlinks(Utf8 key) { - if (outlinks == null ) { return null ; } - getStateManager().setDirty( this , 3); - return outlinks.remove(key); - } - public Metadata getMetadata() { - return (Metadata) get(4); - } - public void setMetadata(Metadata value) { - put(4, value); - } -}
          Hide
          Lewis John McGibbney added a comment -

          I can confirm that I still get the above errors after cleaning and reinstalling trunk.

          Show
          Lewis John McGibbney added a comment - I can confirm that I still get the above errors after cleaning and reinstalling trunk.
          Hide
          Enis Soztutar added a comment -

          I retried the patch w/o any problem. It seems that the error you are getting is the same as before, so I suspect maybe the patch is not applied correctly. What do you see when you do svn diff? You should only see the one line change from .getValueType() to .getElementType().

          Show
          Enis Soztutar added a comment - I retried the patch w/o any problem. It seems that the error you are getting is the same as before, so I suspect maybe the patch is not applied correctly. What do you see when you do svn diff? You should only see the one line change from .getValueType() to .getElementType().
          Hide
          Lewis John McGibbney added a comment -

          Hi Enis. This is pretty strange. It would be great if we could get one other person to try it out. I am still getting a failure to compile and have posted as much output as possible below.

          lewis@lewis-01:~/ASF/gora$ svn status
          ? target
          ? gora-sql/target
          ? gora-cassandra/target
          ? gora-tutorial/target
          ? gora-tutorial/src/main/resources/access.log
          ? gora-hbase/target
          ? gora-core/target
          M gora-core/src/main/java/org/apache/gora/compiler/GoraCompiler.java
          lewis@lewis-01:~/ASF/gora$ svn diff
          Index: gora-core/src/main/java/org/apache/gora/compiler/GoraCompiler.java
          ===================================================================
          — gora-core/src/main/java/org/apache/gora/compiler/GoraCompiler.java (revision 1209228)
          +++ gora-core/src/main/java/org/apache/gora/compiler/GoraCompiler.java (working copy)
          @@ -324,7 +324,7 @@
          break;
          case ARRAY:
          unboxed = unbox(fieldSchema.getElementType());

          • fieldType = type(fieldSchema.getValueType());
            + fieldType = type(fieldSchema.getElementType());
            line(1, "public GenericArray<"fieldType"> get"camelKey"() {"); line(2, "return (GenericArray<"+fieldType+">) get("+i+");"); line(1, "}

            ");
            lewis@lewis-01:~/ASF/gora$ bin/compile-examples.sh
            Compiling gora-core/src/examples/avro/employee.json
            Compiling gora-core/src/examples/avro/tokendatum.json
            Compiling gora-core/src/examples/avro/webpage.json
            Exception in thread "main" org.apache.avro.AvroRuntimeException: Not a map:

            {"type":"array","items":"string"}

            at org.apache.avro.Schema.getValueType(Schema.java:274)
            at org.apache.gora.compiler.GoraCompiler.compile(GoraCompiler.java:327)
            at org.apache.gora.compiler.GoraCompiler.compile(GoraCompiler.java:123)
            at org.apache.gora.compiler.GoraCompiler.compileSchema(GoraCompiler.java:63)
            at org.apache.gora.compiler.GoraCompiler.main(GoraCompiler.java:448)
            Compiling gora-core/src/examples/avro/webpage.json~
            Exception in thread "main" org.apache.avro.SchemaParseException: Array has no items type:

            {"type":"array","values":"string"}

            at org.apache.avro.Schema.parse(Schema.java:933)
            at org.apache.avro.Schema.parse(Schema.java:912)
            at org.apache.avro.Schema.parse(Schema.java:796)
            at org.apache.gora.compiler.GoraCompiler.compileSchema(GoraCompiler.java:62)
            at org.apache.gora.compiler.GoraCompiler.main(GoraCompiler.java:448)
            lewis@lewis-01:~/ASF/gora$

          What am I doing wrong here? BTW, after I try the compile-examples.sh and do svn diff I get the same error as above! e.g. WebPage.java has several lines deleted!

          Show
          Lewis John McGibbney added a comment - Hi Enis. This is pretty strange. It would be great if we could get one other person to try it out. I am still getting a failure to compile and have posted as much output as possible below. lewis@lewis-01:~/ASF/gora$ svn status ? target ? gora-sql/target ? gora-cassandra/target ? gora-tutorial/target ? gora-tutorial/src/main/resources/access.log ? gora-hbase/target ? gora-core/target M gora-core/src/main/java/org/apache/gora/compiler/GoraCompiler.java lewis@lewis-01:~/ASF/gora$ svn diff Index: gora-core/src/main/java/org/apache/gora/compiler/GoraCompiler.java =================================================================== — gora-core/src/main/java/org/apache/gora/compiler/GoraCompiler.java (revision 1209228) +++ gora-core/src/main/java/org/apache/gora/compiler/GoraCompiler.java (working copy) @@ -324,7 +324,7 @@ break; case ARRAY: unboxed = unbox(fieldSchema.getElementType()); fieldType = type(fieldSchema.getValueType()); + fieldType = type(fieldSchema.getElementType()); line(1, "public GenericArray<" fieldType "> get" camelKey "() {"); line(2, "return (GenericArray<"+fieldType+">) get("+i+");"); line(1, "} "); lewis@lewis-01:~/ASF/gora$ bin/compile-examples.sh Compiling gora-core/src/examples/avro/employee.json Compiling gora-core/src/examples/avro/tokendatum.json Compiling gora-core/src/examples/avro/webpage.json Exception in thread "main" org.apache.avro.AvroRuntimeException: Not a map: {"type":"array","items":"string"} at org.apache.avro.Schema.getValueType(Schema.java:274) at org.apache.gora.compiler.GoraCompiler.compile(GoraCompiler.java:327) at org.apache.gora.compiler.GoraCompiler.compile(GoraCompiler.java:123) at org.apache.gora.compiler.GoraCompiler.compileSchema(GoraCompiler.java:63) at org.apache.gora.compiler.GoraCompiler.main(GoraCompiler.java:448) Compiling gora-core/src/examples/avro/webpage.json~ Exception in thread "main" org.apache.avro.SchemaParseException: Array has no items type: {"type":"array","values":"string"} at org.apache.avro.Schema.parse(Schema.java:933) at org.apache.avro.Schema.parse(Schema.java:912) at org.apache.avro.Schema.parse(Schema.java:796) at org.apache.gora.compiler.GoraCompiler.compileSchema(GoraCompiler.java:62) at org.apache.gora.compiler.GoraCompiler.main(GoraCompiler.java:448) lewis@lewis-01:~/ASF/gora$ What am I doing wrong here? BTW, after I try the compile-examples.sh and do svn diff I get the same error as above! e.g. WebPage.java has several lines deleted!
          Hide
          Lewis John McGibbney added a comment -

          Has any tried this out? I think it's important we get to the bottom of it.

          Show
          Lewis John McGibbney added a comment - Has any tried this out? I think it's important we get to the bottom of it.
          Hide
          Lewis John McGibbney added a comment -

          Moving to 0.3 to prepare to release.

          Show
          Lewis John McGibbney added a comment - Moving to 0.3 to prepare to release.
          Hide
          Lewis John McGibbney added a comment -

          Linking issues and will try to get this committed ASAP.

          Show
          Lewis John McGibbney added a comment - Linking issues and will try to get this committed ASAP.
          Hide
          Lewis John McGibbney added a comment -

          Committed @revision 1339274 in trunk
          Thank you very much for the contribution. This one was well overdue and slipped through the net.

          Show
          Lewis John McGibbney added a comment - Committed @revision 1339274 in trunk Thank you very much for the contribution. This one was well overdue and slipped through the net.
          Hide
          Hudson added a comment -

          Integrated in gora-trunk #277 (See https://builds.apache.org/job/gora-trunk/277/)
          commit to address GORA-63 & 133 respectively and update to CHANGES.txt (Revision 1339274)

          Result = FAILURE
          lewismc :
          Files :

          • /gora/trunk/CHANGES.txt
          • /gora/trunk/bin/compile-examples.sh
          • /gora/trunk/bin/gora
          • /gora/trunk/gora-core/src/main/java/org/apache/gora/compiler/GoraCompiler.java
          Show
          Hudson added a comment - Integrated in gora-trunk #277 (See https://builds.apache.org/job/gora-trunk/277/ ) commit to address GORA-63 & 133 respectively and update to CHANGES.txt (Revision 1339274) Result = FAILURE lewismc : Files : /gora/trunk/CHANGES.txt /gora/trunk/bin/compile-examples.sh /gora/trunk/bin/gora /gora/trunk/gora-core/src/main/java/org/apache/gora/compiler/GoraCompiler.java
          Hide
          Hudson added a comment -

          Integrated in goraOracle #6 (See https://builds.apache.org/job/goraOracle/6/)
          commit to address GORA-63 & 133 respectively and update to CHANGES.txt (Revision 1b4ea508f6e4b01de5ed511cc7b39b8858248b55)

          Result = FAILURE
          lewismc :
          Files :

          • bin/gora
          • bin/compile-examples.sh
          • CHANGES.txt
          • gora-core/src/main/java/org/apache/gora/compiler/GoraCompiler.java
          Show
          Hudson added a comment - Integrated in goraOracle #6 (See https://builds.apache.org/job/goraOracle/6/ ) commit to address GORA-63 & 133 respectively and update to CHANGES.txt (Revision 1b4ea508f6e4b01de5ed511cc7b39b8858248b55) Result = FAILURE lewismc : Files : bin/gora bin/compile-examples.sh CHANGES.txt gora-core/src/main/java/org/apache/gora/compiler/GoraCompiler.java

            People

            • Assignee:
              Unassigned
              Reporter:
              Enis Soztutar
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development