XMLBeans
  1. XMLBeans
  2. XMLBEANS-307

XMLBeans scomp throws error "code too large"

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: Version 1.0.4
    • Fix Version/s: Version 2.2
    • Component/s: Compiler
    • Labels:
      None
    • Environment:
      Window XP Service pack 2

      Description

      I am trying to run scomp on my schema which import many other schema's. Out of many schema's one of them is having a size of 1.6 MB.
      Below is the error I get when i run scomp. It would be very much helpful if somebody gives me resolution for this problem.
      Thanks in advance
      Harish

      C:\xmlbeans-2.2.0\bin>scomp -out C:\WantedMissingInquiry\WantedMissingInquiryResponseIEPD\XMLSchemas\test.jar C:\WantedMissingInqui
      ry\WantedMissingInquiryResponseIEPD\XMLSchemas\NYSMessage.xsd
      Time to build schema type system: 2.875 seconds
      Time to generate code: 8.015 seconds
      C:\DOCUME~1\hswamy\LOCALS~1\Temp\xbean8664.d\src\gov\niem\niem\ncic2000\x10\VMACodeSimpleType.java:15268: code too large
      public static final org.apache.xmlbeans.StringEnumAbstractBase.Table table =
      ^
      1 error

      BUILD FAILED

        Activity

        Hide
        Jacob Danner added a comment -

        What are the memory settings for your version of java?
        I've had success compiling with large schemas, but every now and then I need to add a memory flag (-DXmx=...)

        From the error message you posted, it looks like the error you hit is actualy after the xmlbeans code has been generated. At this point, Xmlbeans basically just calls javac over the code and that looks like where your problem is occuring.
        Can you try increasing mem settings and letting us know the outcome?
        Thanks,
        -Jacobd

        Show
        Jacob Danner added a comment - What are the memory settings for your version of java? I've had success compiling with large schemas, but every now and then I need to add a memory flag (-DXmx=...) From the error message you posted, it looks like the error you hit is actualy after the xmlbeans code has been generated. At this point, Xmlbeans basically just calls javac over the code and that looks like where your problem is occuring. Can you try increasing mem settings and letting us know the outcome? Thanks, -Jacobd
        Hide
        Harish K Swamy added a comment -

        Hi Jacob,

        I tried changing the virtual memory size and run scomp util and found that I got the same results. I did the following

        -mx - maximum memory for external java compiler - Tired with 512m, 1024m, 1512m nothing worked.

        C:\xmlbeans-2.2.0\bin>scomp -mx 512m -out C:\WantedMissingInquiry\WantedMissingInquiryResponseIEPD\XMLSchemas\test.jar C:\WantedMissingInquiry\WantedMissingInquiryResponseIEPD\XMLSchemas\NYSMessage.xsd
        Time to build schema type system: 2.875 seconds
        Time to generate code: 8.015 seconds
        C:\DOCUME~1\hswamy\LOCALS~1\Temp\xbean8664.d\src\gov\niem\niem\ncic2000\x10\VMACodeSimpleType.java:15268: code too large
        public static final org.apache.xmlbeans.StringEnumAbstractBase.Table table =
        ^
        1 error

        BUILD FAILED

        Please let me know how are you adding memory flag.
        Thanks
        Harish

        Show
        Harish K Swamy added a comment - Hi Jacob, I tried changing the virtual memory size and run scomp util and found that I got the same results. I did the following -mx - maximum memory for external java compiler - Tired with 512m, 1024m, 1512m nothing worked. C:\xmlbeans-2.2.0\bin>scomp -mx 512m -out C:\WantedMissingInquiry\WantedMissingInquiryResponseIEPD\XMLSchemas\test.jar C:\WantedMissingInquiry\WantedMissingInquiryResponseIEPD\XMLSchemas\NYSMessage.xsd Time to build schema type system: 2.875 seconds Time to generate code: 8.015 seconds C:\DOCUME~1\hswamy\LOCALS~1\Temp\xbean8664.d\src\gov\niem\niem\ncic2000\x10\VMACodeSimpleType.java:15268: code too large public static final org.apache.xmlbeans.StringEnumAbstractBase.Table table = ^ 1 error BUILD FAILED Please let me know how are you adding memory flag. Thanks Harish
        Hide
        Jacob Danner added a comment -

        I think the syntax of your parameters is incorrect, try the following:
        scomp -Xmx512m

        from java -X
        -Xms<size> set initial Java heap size
        -Xmx<size> set maximum Java heap size

        Show
        Jacob Danner added a comment - I think the syntax of your parameters is incorrect, try the following: scomp -Xmx512m from java -X -Xms<size> set initial Java heap size -Xmx<size> set maximum Java heap size
        Hide
        Jacob Danner added a comment -

        correcting bug stats

        Show
        Jacob Danner added a comment - correcting bug stats
        Hide
        Harish K Swamy added a comment -

        As per your suggestion when i tried

        C:\xmlbeans-2.2.0\bin>scomp -Xmx512m -out C:\WantedMissingInquiry\WantedMissingInquiryResponseIEPD\XMLSchemas\test.jar C:\WantedMis
        singInquiry\WantedMissingInquiryResponseIEPD\XMLSchemas\NYSMessage.xsd
        Unrecognized option: -Xmx512m

        scomp does not recognise this option, As per the scomp syntex
        -ms - initial memory for external java compiler (default '8m')
        -mx - maximum memory for external java compiler (default '256m')

        Please advice

        Thanks
        Harish

        Show
        Harish K Swamy added a comment - As per your suggestion when i tried C:\xmlbeans-2.2.0\bin>scomp -Xmx512m -out C:\WantedMissingInquiry\WantedMissingInquiryResponseIEPD\XMLSchemas\test.jar C:\WantedMis singInquiry\WantedMissingInquiryResponseIEPD\XMLSchemas\NYSMessage.xsd Unrecognized option: -Xmx512m scomp does not recognise this option, As per the scomp syntex -ms - initial memory for external java compiler (default '8m') -mx - maximum memory for external java compiler (default '256m') Please advice Thanks Harish
        Hide
        Jacob Danner added a comment -

        Have you tried placing the src files in a dir and javac'ing them?
        I'm curious if you can reproduce this problem outside of scomp.

        Can you attach the schema so I can try?

        Show
        Jacob Danner added a comment - Have you tried placing the src files in a dir and javac'ing them? I'm curious if you can reproduce this problem outside of scomp. Can you attach the schema so I can try?
        Hide
        Harish K Swamy added a comment -

        Hi Jacob,

        When I printed verbose message for the scomp. The message below shows that -mx 1536m did set the virtual memory.
        Please let me know if I am making any mistakes here.

        created source: C:\DOCUME~1\hswamy\LOCALS~1\Temp\xbean63171.d\src\gov\niem\niem\structures\x10\RefAttribute.java
        created source: C:\DOCUME~1\hswamy\LOCALS~1\Temp\xbean63171.d\src\gov\niem\niem\structures\x10\impl\RefAttributeImpl.jav
        Time to generate code: 8.906 seconds

        compile command: C:\j2sdk1.4.2_08\jre\..\bin\javac.exe -J-Xmx1536m @C:\DOCUME~1\hswamy\LOCALS~1\Temp\javac63172
        ----------------------**************----------------------
        [parsing started C:\DOCUME~1\hswamy\LOCALS~1\Temp\xbean63171.d\src\us\ny\state\exchanges\common\NYSMessageType.java]
        [parsing completed 47ms]
        [parsing started C:\DOCUME~1\hswamy\LOCALS~1\Temp\xbean63171.d\src\us\ny\state\exchanges\common\impl\NYSMessageTypeImpl.
        [parsing completed 0ms]

        Thank you
        Harish

        Show
        Harish K Swamy added a comment - Hi Jacob, When I printed verbose message for the scomp. The message below shows that -mx 1536m did set the virtual memory. Please let me know if I am making any mistakes here. created source: C:\DOCUME~1\hswamy\LOCALS~1\Temp\xbean63171.d\src\gov\niem\niem\structures\x10\RefAttribute.java created source: C:\DOCUME~1\hswamy\LOCALS~1\Temp\xbean63171.d\src\gov\niem\niem\structures\x10\impl\RefAttributeImpl.jav Time to generate code: 8.906 seconds compile command: C:\j2sdk1.4.2_08\jre\..\bin\javac.exe -J-Xmx1536m @C:\DOCUME~1\hswamy\LOCALS~1\Temp\javac63172 ---- ------------------ ************** ---------------------- [parsing started C:\DOCUME~1\hswamy\LOCALS~1\Temp\xbean63171.d\src\us\ny\state\exchanges\common\NYSMessageType.java] [parsing completed 47ms] [parsing started C:\DOCUME~1\hswamy\LOCALS~1\Temp\xbean63171.d\src\us\ny\state\exchanges\common\impl\NYSMessageTypeImpl. [parsing completed 0ms] Thank you Harish
        Hide
        Harish K Swamy added a comment -

        Hi Jacob,

        There are nearly 10 schemas, The problem is with only one schema. If i comment out the content of this big scheam, everything works fine.
        But the schema which is too big ie 1.6MB is a coded values for different states and so on, So can not reduce the size of this schema.

        Sending the schema to you is not a good idea since it is company confidential resources.

        Please let me know is their any other way.

        Thanks
        Harish

        Show
        Harish K Swamy added a comment - Hi Jacob, There are nearly 10 schemas, The problem is with only one schema. If i comment out the content of this big scheam, everything works fine. But the schema which is too big ie 1.6MB is a coded values for different states and so on, So can not reduce the size of this schema. Sending the schema to you is not a good idea since it is company confidential resources. Please let me know is their any other way. Thanks Harish
        Hide
        Jacob Danner added a comment -

        hmm, well not beign able to see it certainly makes it more difficult.
        Have you tried generating the src and compiling as another step?

        scomp options:
        -d [dir] - target binary directory for .class and .xsb files
        -src [dir] - target directory for generated .java files
        -srconly - do not compile .java files or jar the output.

        also, can you decrease the length of your tmpdir. Right now you are building to a long path and to one with spaces
        C:\DOCUME~1\hswamy\LOCALS~1\Temp\xbean63171.d
        is actually
        C:\Documents and Settings\hswamy\...

        maybe try
        scomp * -Djava.io.tmpdir=<a path with no spaces, e.g, C:\Temp>

        Show
        Jacob Danner added a comment - hmm, well not beign able to see it certainly makes it more difficult. Have you tried generating the src and compiling as another step? scomp options: -d [dir] - target binary directory for .class and .xsb files -src [dir] - target directory for generated .java files -srconly - do not compile .java files or jar the output. also, can you decrease the length of your tmpdir. Right now you are building to a long path and to one with spaces C:\DOCUME~1\hswamy\LOCALS~1\Temp\xbean63171.d is actually C:\Documents and Settings\hswamy\... maybe try scomp * -Djava.io.tmpdir=<a path with no spaces, e.g, C:\Temp>
        Hide
        Harish K Swamy added a comment -

        I can attach verbose file, Please let me know if it helps

        Harish

        Show
        Harish K Swamy added a comment - I can attach verbose file, Please let me know if it helps Harish
        Hide
        Harish K Swamy added a comment -

        Ok I will try doing that. I will decrease the length of temdir and also i will try to generate only source and compile the source directory.
        Harish

        Show
        Harish K Swamy added a comment - Ok I will try doing that. I will decrease the length of temdir and also i will try to generate only source and compile the source directory. Harish
        Hide
        Harish K Swamy added a comment -

        Hi Jacob,

        I generated the source only option and tried compling the source folder, All the files got complied expect one(VMACodeSimpleType.java) and this the error I got,

        Severity Description Resource In Folder Location Creation Time
        2 The code for the static initializer is exceeding the 65535 bytes limit VMACodeSimpleType.java NYSMessageSource/src/gov/niem/niem/ncic2000/x10 line 10181 January 2, 2007 4:57:07 PM

        It says the code is exceeding 65535 bytes limit. Did you ever come across this problem. Please let me know what is the solution this.

        Show
        Harish K Swamy added a comment - Hi Jacob, I generated the source only option and tried compling the source folder, All the files got complied expect one(VMACodeSimpleType.java) and this the error I got, Severity Description Resource In Folder Location Creation Time 2 The code for the static initializer is exceeding the 65535 bytes limit VMACodeSimpleType.java NYSMessageSource/src/gov/niem/niem/ncic2000/x10 line 10181 January 2, 2007 4:57:07 PM It says the code is exceeding 65535 bytes limit. Did you ever come across this problem. Please let me know what is the solution this.
        Hide
        Wing Yew Poon added a comment -

        Does your simple type have a very long enumeration facet?
        This is a known limitation.

        Show
        Wing Yew Poon added a comment - Does your simple type have a very long enumeration facet? This is a known limitation.
        Hide
        Harish K Swamy added a comment -

        Hi Wing Yew Poon,

        Yes it does have very long enumeration facet. What could be the possible solution to over come this situvation?

        Thank you.
        Harish

        Show
        Harish K Swamy added a comment - Hi Wing Yew Poon, Yes it does have very long enumeration facet. What could be the possible solution to over come this situvation? Thank you. Harish
        Hide
        Wing Yew Poon added a comment -

        The limit on bytecode size is a limitation of the JVM.
        I don't have any good advice for you.
        You could remove the facet from the definition, for the purpose of generating the xmlbeans, i.e., just map your simple type to xs:string.

        Show
        Wing Yew Poon added a comment - The limit on bytecode size is a limitation of the JVM. I don't have any good advice for you. You could remove the facet from the definition, for the purpose of generating the xmlbeans, i.e., just map your simple type to xs:string.
        Hide
        Harish K Swamy added a comment -

        Thanks Wing and Jacob, I will try splitting the file to keep it under 64KB.

        Show
        Harish K Swamy added a comment - Thanks Wing and Jacob, I will try splitting the file to keep it under 64KB.
        Hide
        Harish K Swamy added a comment -

        Hi Wing and Jacob,

        I want to request that, Will it be possible for XMLBeans scomp util to identify such huge coded value source file to split into 64Kb JVM standard before doing the compiliation??
        May be this woud be additional feature for XMLBeans, Please advice.

        Thanks
        Harish

        Show
        Harish K Swamy added a comment - Hi Wing and Jacob, I want to request that, Will it be possible for XMLBeans scomp util to identify such huge coded value source file to split into 64Kb JVM standard before doing the compiliation?? May be this woud be additional feature for XMLBeans, Please advice. Thanks Harish
        Hide
        Wing Yew Poon added a comment -

        Harish,
        the issue is not the size of the .java file. The JVM has limits on the constant pool count, number of fields, number of local variables in a frame, number of methods, size of a method (the bytecode size must not exceed 64k), etc. It is beyond the scope of the schema compiler (scomp) to check for such things as how big the bytecode size of generated methods would be when they are compiled by the java compiler (javac).

        • Wing Yew
        Show
        Wing Yew Poon added a comment - Harish, the issue is not the size of the .java file. The JVM has limits on the constant pool count, number of fields, number of local variables in a frame, number of methods, size of a method (the bytecode size must not exceed 64k), etc. It is beyond the scope of the schema compiler (scomp) to check for such things as how big the bytecode size of generated methods would be when they are compiled by the java compiler (javac). Wing Yew
        Hide
        Harish K Swamy added a comment -

        Thanks Wing Yew, Thanks for the advice.

        Show
        Harish K Swamy added a comment - Thanks Wing Yew, Thanks for the advice.
        Hide
        Radu Preotiuc-Pietro added a comment -

        Wing Yew is right, this is a JVM limitation that is difficult to work around, because one doesn't know how the code is going to be compiled. What XMLBeans should do is give an option to skip generation of the enumeration class and use the base type instead; this way one wouldn't need to alter their Schema file to get classes generated.

        Show
        Radu Preotiuc-Pietro added a comment - Wing Yew is right, this is a JVM limitation that is difficult to work around, because one doesn't know how the code is going to be compiled. What XMLBeans should do is give an option to skip generation of the enumeration class and use the base type instead; this way one wouldn't need to alter their Schema file to get classes generated.
        Hide
        Andrew Fuentes added a comment -

        I'm getting the same problems as Harry. I think I may know why. We both are using the NIEM (National Information Exchange Model) schemas. Some can get quite large depending on the one you want to compile because many are imports of other schemas in the framework.

        Anyways, what might help you whoever has the task to fix the problem is actually downloading the NIEM from their web site. It is publicly available to download at www.niem.gov. If you download it get 1.0 with docs. It comes packaged as a zip file. I'm recommending you get the docs because it describes the architecture of the model. It may help with the testing and debugging. A lot of good technical advice is coming from the GJXDM (Global Justice XML folks and directing us to XMLBeans for development solutions.

        If you really want to stress test XMLBeans it may be helpful the use both NIEM and GJXDM. You can also download GJXDM publicly at http://www.it.ojp.gov/jxdm/.

        I hope this helps,
        Andrew

        Show
        Andrew Fuentes added a comment - I'm getting the same problems as Harry. I think I may know why. We both are using the NIEM (National Information Exchange Model) schemas. Some can get quite large depending on the one you want to compile because many are imports of other schemas in the framework. Anyways, what might help you whoever has the task to fix the problem is actually downloading the NIEM from their web site. It is publicly available to download at www.niem.gov. If you download it get 1.0 with docs. It comes packaged as a zip file. I'm recommending you get the docs because it describes the architecture of the model. It may help with the testing and debugging. A lot of good technical advice is coming from the GJXDM (Global Justice XML folks and directing us to XMLBeans for development solutions. If you really want to stress test XMLBeans it may be helpful the use both NIEM and GJXDM. You can also download GJXDM publicly at http://www.it.ojp.gov/jxdm/ . I hope this helps, Andrew
        Hide
        Jacob Danner added a comment -

        I've added an FAQ section on the wiki for this issue.
        http://wiki.apache.org/xmlbeans/XmlBeansFaq

        Show
        Jacob Danner added a comment - I've added an FAQ section on the wiki for this issue. http://wiki.apache.org/xmlbeans/XmlBeansFaq
        Hide
        Jacob Danner added a comment -

        I think I've got a workaround for this issue in the attached patch.
        I've attached a unit test also and have verified compilation of both sets of schemas mentions in this issue (niem && jxdm).

        Show
        Jacob Danner added a comment - I think I've got a workaround for this issue in the attached patch. I've attached a unit test also and have verified compilation of both sets of schemas mentions in this issue (niem && jxdm).
        Hide
        Jacob Danner added a comment -

        Please check out my patch, I think it works around the issue.
        The error occurs whenever the enum value is > 3668.

        Show
        Jacob Danner added a comment - Please check out my patch, I think it works around the issue. The error occurs whenever the enum value is > 3668.
        Hide
        Jacob Danner added a comment -

        Use this instead, maintains formatting with current src
        Also includes tests

        Show
        Jacob Danner added a comment - Use this instead, maintains formatting with current src Also includes tests
        Hide
        Radu Preotiuc-Pietro added a comment -

        Checked in with svn revision 532957.

        Show
        Radu Preotiuc-Pietro added a comment - Checked in with svn revision 532957.

          People

          • Assignee:
            Radu Preotiuc-Pietro
            Reporter:
            Harish K Swamy
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development