Created attachment 29082 [details] The above text is original FOP output. The below text is CTL enbaled text 1. In order to make Thai glyph substitution work, the language tag must be "dflt" instead of "th". This is because many popular Thai font use "dflt". 2. Using "dflt" language, both GSUB and GPOS are used. The problem is after it used, all glyph are placed further to the right. Notes: 1. Many Thai combining glyph have negative alignment. I suspect this is involved in miss alignment calculation. 2. The ratio of displacement seem irrational. If it shifted to the left by the same ratio, it is still invalid. Question: Should "dflt" being load by default ? XML Data for testing: <?xml version="1.0" encoding="UTF-8"?> <data>กุญญูฐฐูฏฏูฎฎูบบ่บี่ปป่ปี่กำปำถํ้า</data> XSLT for testing: <?xml version="1.0" encoding="ISO-8859-1"?> <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"> <xsl:template match="/"> <fo:root xmlns:fo="http://www.w3.org/1999/XSL/Format"> <fo:layout-master-set> <fo:simple-page-master master-name="A4"> <fo:region-body /> </fo:simple-page-master> </fo:layout-master-set> <fo:page-sequence master-reference="A4"> <fo:flow flow-name="xsl-region-body"> <fo:block script="thai" language="dflt" font-size="32pt" font-family="AngsanaUPC"> <xsl:value-of select="data" /> </fo:block> </fo:flow> </fo:page-sequence> </fo:root> </xsl:template> </xsl:stylesheet> The fop.xconf has added font into fonts section of applicaton/pdf renderer: <font embed-url="file:///c:/windows/fonts/angsau.ttf" kerning="yes"> <font-triplet name="AngsanaUPC" style="normal" weight="normal"/> </font> The Java code used to reproduce: FopFactory fopFac = FopFactory.newInstance(new File("fop.xconf")); OutputStream outputStream = new BufferedOutputStream(new FileOutputStream("c:\\temp\\fopThai.pdf")); Fop fop = fopFac.newFop(MimeConstants.MIME_PDF, outputStream); TransformerFactory transFac = TransformerFactory.newInstance(); Source xslData = new StreamSource(new File("data.xslt")); Transformer trans = transFac.newTransformer(xslData); Result res = new SAXResult(fop.getDefaultHandler()); Source xmlData = new StreamSource(new File("thaidata.xml")); trans.transform(xmlData, res); outputStream.close();
This certainly seems like an interesting issue and you'll have to excuse my ignorance on this matter, but are the GPOS and GSUB absolutely necessary for ALL Thai script? I'm sure we have users working with Thai that used FOP prior to the inclusion of the complex scripts feature (which takes GPOS/GSUB into account.) If I've just exposed my ignorance, my apologies, but my suggestion was going to be to try using FOP-1.0 and see if it behaves as expected. You will however have to change your Java slightly due to modifications to the API, but there are plenty of examples and documentation on the website about how to do this.
Thanks for the promptly reply. Yes, i have tested with FOP 1.0 and it product exactly the same as the first line of the picture i attached. There are few Thai fonts that aren't using GSUB and GPOS. Those few deploy a default glyph position that will never collide with other glyph. However, it gave a result with problem that call "floating tone mark". These few fonts also leave the glyph determination work to LayoutEngine. It expects LayoutEngine to pick correct glyph by itself. In order to accomplish this, it need State Machine. You can take a peek at ICU4C LayoutEngine. There is a ThaiLayoutEngine there. However, ICU4J doesn't have one. Note: 1. In order to avoid GSUB and GPOS it require the font to use Unicode PVA. Adobe recommends to avoid it as it is non-standard. The best example of problem is ICU4C uses different code point compare to Adobe that assign to Thai glyph.ICU gives more correct result. 2. All known Thai fonts in MS.Windows use GSUB and GPOS except Arial Unicode. 3. All known True Type fonts in Linux use GSUB and GPOS.
Please provide the following: * input FO file (not XSLT/XML files) * output PDF file * version of FOP you are using * any console output from FOP Also, note that you should not specify script="thai" or language="dflt", as doing so may prevent application of correct GSUB/GPOS tables (assuming you are using FOP1.1rc1 or current trunk). Also, note that [1] does not yet list thai as supported (by the recent complex scripts addition). In fact, the GSUB/GPOS features of thai fonts will be enabled if you do not specify script or specify script="auto" and if you do not specify language; however, line breaking does not yet take into account legitimate thai break candidates, and I have not yet tested thai to my satisfaction, so I did not list thai as supported yet. http://xmlgraphics.apache.org/fop/1.1rc1/complexscripts.html#supported_scripts
Created attachment 29090 [details] test FO file showing gsub/gpos application This FO shows the same data with Angsana UPC with various settings of language and script. Note that with or without language='dflt' and with or without script='thai' does not impact the results. However, specifying language='th' does prevent gsub/gpos application.
Created attachment 29091 [details] test PDF file showing gsub/gpos application This file shows the output using FOP1.1rc1 with the attachment 29090 [details]. Clearly the gsub/gpos features *are* working given acceptable language/script settings.
The first 3 lines of generated PDF from you give a near correct look. The problem is at character 26, Mai-Ek. It ship too far to the left. I have few questions regarding to your output. I changed my fo:block element removing all attribute except font-family. My result is different from what you got. It is look similar to the second line in first jpg attachment. Why the different ? Another question is whether the "dflt" shall be loaded when specific language is given ? Most Thai fonts gsub/gpos information use "dflt" language and "thai" script. Please correct me if i'm wrong. Per http://www.microsoft.com/typography/otfntdev/standot/appen.aspx, my understanding is when the "th" language is missing from font info, it shall fall back to "dflt".
Created attachment 29094 [details] the xml data used to reproduce a problem
Created attachment 29095 [details] the xslt which transform thaidata.xml into fo
Created attachment 29096 [details] The generated PDF file.
(In reply to comment #6) > The first 3 lines of generated PDF from you give a near correct look. > The problem is at character 26, Mai-Ek. > It ship too far to the left. I understand. I will investigate this further. > I have few questions regarding to your output. > I changed my fo:block element removing all attribute except font-family. > My result is different from what you got. > It is look similar to the second line in first jpg attachment. > Why the different ? That's hard to say. I am using FOP1.1rc1 and Version 5.0 of the AngsanaUPC font (from Windows 7). If you are using different versions, then that could cause the difference. > > Another question is whether the "dflt" shall be loaded when specific > language is given ? > Most Thai fonts gsub/gpos information use "dflt" language and "thai" script. > Please correct me if i'm wrong. Per > http://www.microsoft.com/typography/otfntdev/standot/appen.aspx, my > understanding is when the "th" language is missing from font info, it shall > fall back to "dflt". The current implementation in FOP will use 'dflt' for the OT Language lookup if the value of the language attribute is not specified or is specified as an empty string. Similarly, it will use 'thai' for the OT Script lookup if the dominant characters are Thai, the script attribute is not specified, is specified as the empty string, as 'auto', or as 'thai'. If 'th' or 'TH' is specified as the language, then presumably it should also produce a successful lookup. However, this will depend on whether the font designer/creator provides an OT lookup for language = 'th'|'TH' and script = 'thai'. Taking a look at AngsanaUPC (using the Adobe TTX tool), I see that the GSUB table only lists the default language system, and does not include 'TH', so that would explain why specifying language='TH' results in no lookup. <ScriptList> <!-- ScriptCount=1 --> <ScriptRecord index="0"> <ScriptTag value="thai"/> <Script> <DefaultLangSys> <ReqFeatureIndex value="65535"/> <!-- FeatureCount=1 --> <FeatureIndex index="0" value="0"/> </DefaultLangSys> <!-- LangSysCount=0 --> </Script> </ScriptRecord> </ScriptList>
(In reply to comment #8) > Created attachment 29095 [details] > the xslt which transform thaidata.xml into fo Please provide an FO input file. I don't care about XSLT/XML files (they are just noise).
Created attachment 29097 [details] FO file for reproduce the problem
(In reply to comment #10) > (In reply to comment #6) > > The first 3 lines of generated PDF from you give a near correct look. > > The problem is at character 26, Mai-Ek. > > It ship too far to the left. > > I understand. I will investigate this further. > > > I have few questions regarding to your output. > > I changed my fo:block element removing all attribute except font-family. > > My result is different from what you got. > > It is look similar to the second line in first jpg attachment. > > Why the different ? > > That's hard to say. I am using FOP1.1rc1 and Version 5.0 of the AngsanaUPC > font (from Windows 7). If you are using different versions, then that could > cause the difference. > I'm using AngsanaUPC 5.0 from Windows 7 SP1 64 bits [Version 6.1.7601] I uses FOP from SVN trunk, checked out today.
(In reply to comment #13) > (In reply to comment #10) > > (In reply to comment #6) > > > The first 3 lines of generated PDF from you give a near correct look. > > > The problem is at character 26, Mai-Ek. > > > It ship too far to the left. > > > > I understand. I will investigate this further. > > > > > I have few questions regarding to your output. > > > I changed my fo:block element removing all attribute except font-family. > > > My result is different from what you got. > > > It is look similar to the second line in first jpg attachment. > > > Why the different ? > > > > That's hard to say. I am using FOP1.1rc1 and Version 5.0 of the AngsanaUPC > > font (from Windows 7). If you are using different versions, then that could > > cause the difference. > > > I'm using AngsanaUPC 5.0 from Windows 7 SP1 64 bits [Version 6.1.7601] > I uses FOP from SVN trunk, checked out today. Hmmm. Strange. Here is the config I am using (fop.xconf): <?xml version="1.0"?> <fop version="1.0"> <use-cache>false</use-cache> <base>./</base> <strict-configuration>true</strict-configuration> <renderers> <renderer mime="application/pdf"> <filterList> <value>flate</value> </filterList> <fonts> <directory>./fonts</directory> </fonts> </renderer> </renderers> </fop> I placed the angsanu.ttf file in a fonts directory in the directory that contains this config. I turned of the font-cache to make sure it is using that font. Perhaps you might try this also?
(In reply to comment #14) > (In reply to comment #13) > > (In reply to comment #10) > > > (In reply to comment #6) > > > > The first 3 lines of generated PDF from you give a near correct look. > > > > The problem is at character 26, Mai-Ek. > > > > It ship too far to the left. > > > > > > I understand. I will investigate this further. > > > > > > > I have few questions regarding to your output. > > > > I changed my fo:block element removing all attribute except font-family. > > > > My result is different from what you got. > > > > It is look similar to the second line in first jpg attachment. > > > > Why the different ? > > > > > > That's hard to say. I am using FOP1.1rc1 and Version 5.0 of the AngsanaUPC > > > font (from Windows 7). If you are using different versions, then that could > > > cause the difference. > > > > > I'm using AngsanaUPC 5.0 from Windows 7 SP1 64 bits [Version 6.1.7601] > > I uses FOP from SVN trunk, checked out today. > > Hmmm. Strange. Here is the config I am using (fop.xconf): > > <?xml version="1.0"?> > <fop version="1.0"> > <use-cache>false</use-cache> > <base>./</base> > <strict-configuration>true</strict-configuration> > <renderers> > <renderer mime="application/pdf"> > <filterList> > <value>flate</value> > </filterList> > <fonts> > <directory>./fonts</directory> > </fonts> > </renderer> > </renderers> > </fop> > > I placed the angsanu.ttf file in a fonts directory in the directory that > contains this config. I turned of the font-cache to make sure it is using > that font. > > Perhaps you might try this also? Tried it. Nothing change. I might forget to mention you that i embed FOP into java application. I copied fop.xconf from FOP 1.0 directory and put it right into my java project. The only section i changed is all those you mention above. Do i have to set anything other than that for embedded use ?
(In reply to comment #15) > (In reply to comment #14) > > (In reply to comment #13) > > > (In reply to comment #10) > > > > (In reply to comment #6) > > > > > The first 3 lines of generated PDF from you give a near correct look. > > > > > The problem is at character 26, Mai-Ek. > > > > > It ship too far to the left. > > > > > > > > I understand. I will investigate this further. > > > > > > > > > I have few questions regarding to your output. > > > > > I changed my fo:block element removing all attribute except font-family. > > > > > My result is different from what you got. > > > > > It is look similar to the second line in first jpg attachment. > > > > > Why the different ? > > > > > > > > That's hard to say. I am using FOP1.1rc1 and Version 5.0 of the AngsanaUPC > > > > font (from Windows 7). If you are using different versions, then that could > > > > cause the difference. > > > > > > > I'm using AngsanaUPC 5.0 from Windows 7 SP1 64 bits [Version 6.1.7601] > > > I uses FOP from SVN trunk, checked out today. > > > > Hmmm. Strange. Here is the config I am using (fop.xconf): > > > > <?xml version="1.0"?> > > <fop version="1.0"> > > <use-cache>false</use-cache> > > <base>./</base> > > <strict-configuration>true</strict-configuration> > > <renderers> > > <renderer mime="application/pdf"> > > <filterList> > > <value>flate</value> > > </filterList> > > <fonts> > > <directory>./fonts</directory> > > </fonts> > > </renderer> > > </renderers> > > </fop> > > > > I placed the angsanu.ttf file in a fonts directory in the directory that > > contains this config. I turned of the font-cache to make sure it is using > > that font. > > > > Perhaps you might try this also? > > Tried it. Nothing change. I might forget to mention you that i embed FOP > into java application. I copied fop.xconf from FOP 1.0 directory and put it > right into my java project. The only section i changed is all those you > mention above. > Do i have to set anything other than that for embedded use ? What results do you get when running FOP directly from command line using fop script or directly from Java command? Here is a debug output from my invocation (I'm using Ant to invoke java directly): [java] Executing '/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/bin/java' with arguments: [java] '-Xms512m' [java] '-Xmx2048m' [java] '-Djava.awt.headless=true' [java] '-Djava.util.logging.config.file=logging.properties' [java] '-ea:org.apache.fop...' [java] '-classpath' [java] '/Users/glenn/work/fop/build/fop.jar:/Users/glenn/work/fop/lib/avalon-framework-4.2.0.jar:/Users/glenn/work/fop/lib/batik-all-1.7.jar:/Users/glenn/work/fop/lib/commons-io-1.3.1.jar:/Users/gl\ enn/work/fop/lib/commons-logging-1.0.4.jar:/Users/glenn/work/fop/lib/fop-hyph.jar:/Users/glenn/work/fop/lib/serializer-2.7.0.jar:/Users/glenn/work/fop/lib/servlet-2.2.jar:/Users/glenn/work/fop/lib/xalan\ -2.7.0-sources.jar:/Users/glenn/work/fop/lib/xalan-2.7.0.jar:/Users/glenn/work/fop/lib/xercesImpl-2.7.1.jar:/Users/glenn/work/fop/lib/xml-apis-1.3.04.jar:/Users/glenn/work/fop/lib/xml-apis-ext-1.3.04.ja\ r:/Users/glenn/work/fop/lib/xmlgraphics-commons-1.5svn.jar' [java] 'org.apache.fop.cli.Main' [java] '-q' [java] '-c' [java] 'fop.xconf' [java] '-fo' [java] './test.fo.xml' [java] '-pdf' [java] './test.pdf'
Created attachment 29102 [details] The debug mode output from fop.bat The result of using fop.bat execution is the same as previously upload PDF file. I suspect there must be some configuration different or my trunk version is different from yours. BTW, i tried fop.bat -version and it return 1.0beta2.
(In reply to comment #17) > Created attachment 29102 [details] > The debug mode output from fop.bat > > The result of using fop.bat execution is the same as previously upload PDF > file. > I suspect there must be some configuration different or my trunk version is > different from yours. > BTW, i tried fop.bat -version and it return 1.0beta2. I can tell already that you aren't using the trunk version, since you should not get 1.0beta2. Please do the following (or equivalent) from the root level of what you think is the current trunk HEAD and post a reply here: $ svn info Path: . Working Copy Root Path: /Users/glenn/work/xmlgraphics/fop/trunk URL: https://svn.apache.org/repos/asf/xmlgraphics/fop/trunk Repository Root: https://svn.apache.org/repos/asf Repository UUID: 13f79535-47bb-0310-9956-ffa450edef68 Revision: 1364879 Node Kind: directory Schedule: normal Last Changed Author: mehdi Last Changed Rev: 1364567 Last Changed Date: 2012-07-23 04:21:23 -0600 (Mon, 23 Jul 2012) $ java -version java version "1.6.0_33" Java(TM) SE Runtime Environment (build 1.6.0_33-b03-424-11M3720) Java HotSpot(TM) 64-Bit Server VM (build 20.8-b03-424, mixed mode) $ java -jar build/fop.jar -version FOP Version svn-trunk $ md5 fonts/angsau.ttf MD5 (fonts/angsau.ttf) = 070f54e1790ee0a495de46729b974b57
> I can tell already that you aren't using the trunk version, since you should > not get 1.0beta2. > > Please do the following (or equivalent) from the root level of what you > think is the current trunk HEAD and post a reply here: > > $ svn info > Path: . > Working Copy Root Path: /Users/glenn/work/xmlgraphics/fop/trunk > URL: https://svn.apache.org/repos/asf/xmlgraphics/fop/trunk > Repository Root: https://svn.apache.org/repos/asf > Repository UUID: 13f79535-47bb-0310-9956-ffa450edef68 > Revision: 1364879 > Node Kind: directory > Schedule: normal > Last Changed Author: mehdi > Last Changed Rev: 1364567 > Last Changed Date: 2012-07-23 04:21:23 -0600 (Mon, 23 Jul 2012) > > $ java -version > java version "1.6.0_33" > Java(TM) SE Runtime Environment (build 1.6.0_33-b03-424-11M3720) > Java HotSpot(TM) 64-Bit Server VM (build 20.8-b03-424, mixed mode) > > $ java -jar build/fop.jar -version > FOP Version svn-trunk > > $ md5 fonts/angsau.ttf > MD5 (fonts/angsau.ttf) = 070f54e1790ee0a495de46729b974b57 I'm using subclipse to checkout from trunk. The trunk URL is http://svn.apache.org/repos/asf/xmlgraphics/fop/trunk Revision: 1364910 Last rev: 1364567 by mehdi java -version return: Java(TM) SE Runtime Environment (build 1.7.0-b147) java -jar build/fop.jar -version FOP Version svn-trunk fop -version FOP Version 1.0beta2 I don't install md5
(In reply to comment #19) > fop -version > FOP Version 1.0beta2 There was never been such FOP version between 0.95 final and 1.0 final. The branch 1.0 was used only to prepare the tag 1.0 final, and only that. can you check how the script "fop" (one of fop.sh, fop.cmd, fop.bat, fop.js, ...) recovers this info?
(In reply to comment #20) > (In reply to comment #19) > > fop -version > > FOP Version 1.0beta2 > > There was never been such FOP version between 0.95 final and 1.0 final. > The branch 1.0 was used only to prepare the tag 1.0 final, and only that. > can you check how the script "fop" (one of fop.sh, fop.cmd, fop.bat, fop.js, > ...) recovers this info? In fop.bat, i remove "rem" and let it echo the final command. It is: "java" -cp ";C:\svn tempspace\fop svn\build\fop-hyph.jar;C:\svn tempspace\fop svn\build\fop-sandbox.jar;C:\svn tempspace\fop svn\build\fop-transcoder-allinone .jar;C:\svn tempspace\fop svn\build\fop-transcoder.jar;C:\svn tempspace\fop svn\ build\fop.jar;C:\svn tempspace\fop svn\lib\avalon-framework-4.2.0.jar;C:\svn tem pspace\fop svn\lib\batik-all-1.7.jar;C:\svn tempspace\fop svn\lib\commons-io-1.3 .1.jar;C:\svn tempspace\fop svn\lib\commons-logging-1.0.4.jar;C:\svn tempspace\f op svn\lib\serializer-2.7.0.jar;C:\svn tempspace\fop svn\lib\servlet-2.2.jar;C:\ svn tempspace\fop svn\lib\xalan-2.7.0.jar;C:\svn tempspace\fop svn\lib\xercesImp l-2.7.1.jar;C:\svn tempspace\fop svn\lib\xml-apis-1.3.04.jar;C:\svn tempspace\fo p svn\lib\xml-apis-ext-1.3.04.jar;C:\svn tempspace\fop svn\lib\xmlgraphics-commo ns-1.5svn.jar" org.apache.fop.cli.Main -version It return: FOP Version 1.0beta2 However, i tried executing following command. C:\svn tempspace\fop svn\build>java -cp fop.jar org.apache.fop.cli.Main -version It return: FOP Version svn-trunk
(In reply to comment #21) > However, i tried executing following command. > C:\svn tempspace\fop svn\build>java -cp fop.jar org.apache.fop.cli.Main > -version > > It return: > FOP Version svn-trunk you should try adding one-by-one other jar, you probably will identify witch overrides fop.
(In reply to comment #21) > It return: > FOP Version 1.0beta2 > > It return: > FOP Version svn-trunk It is clear you have a local configuration issue to figure out and fix. Your tests appear to be using FOP1.0 or some prior (beta2) version, and not the current trunk version of FOP. This could clearly account for the differences we are seeing. Also, I would suggest avoiding the use of directory names containing spaces. I have encountered various problems on WIN bat/cmd files due to this usage in the past. At this point, I'm going to investigate the apparent MAI EK problem and will open a new bug against that issue if it turns out to be a code bug as opposed to a font bug.
(In reply to comment #23) > (In reply to comment #21) > > It return: > > FOP Version 1.0beta2 > > > > It return: > > FOP Version svn-trunk > > It is clear you have a local configuration issue to figure out and fix. Your > tests appear to be using FOP1.0 or some prior (beta2) version, and not the > current trunk version of FOP. This could clearly account for the differences > we are seeing. > > Also, I would suggest avoiding the use of directory names containing spaces. > I have encountered various problems on WIN bat/cmd files due to this usage > in the past. > > At this point, I'm going to investigate the apparent MAI EK problem and will > open a new bug against that issue if it turns out to be a code bug as > opposed to a font bug. I agree that it is another problem which shall not be discuss in this PMR because it is unrelated to the original problem. Please notice me when you open a new PMR. Thanks.