patch for POI 3.13 Beta 2 integration.
(Although its origin is the common_sl change, I haven't moved the code to a common extractor for both formats (HSLF/XSLF) yet)

Looks good so far to me!

One quick thing though - any chance you could review http://tika.apache.org/contribute.html#Code_Formatting and tweak your formatting settings accordingly? (I think your IDE isn't quite right for Tika, for imports at least)

ok ... I've changed the import settings and executed organize imports.
I couldn't validate the tests again, as the new grobid dependency somehow broke my build ... but apart of that, the rest is the same ...

The build is hopefully working again now. If you could re-test, that'd be wonderful!

The affected test cases are ok now ... I haven't tried the full fledged tika test suite, as my JRE chokes on the 2GB heap settings, but tika-parsers seems to be ok with 1GB

r1706079.

Thank you, Andreas Beeker!

Apologies to you and Nick Burch for not remembering that you had already fixed the upgrades with this patch when I opened the duplicate TIKA-1748.

SUCCESS: Integrated in tika-trunk-jdk1.7 #860 (See https://builds.apache.org/job/tika-trunk-jdk1.7/860/)
TIKA-1707: upgrade to POI 3.13 (tallison: http://svn.apache.org/viewvc/tika/trunk/?view=rev&rev=1706079)

• trunk/CHANGES.txt
• trunk/tika-parsers/pom.xml
• trunk/tika-parsers/src/main/java/org/apache/tika/parser/microsoft/HSLFExtractor.java
• trunk/tika-parsers/src/main/java/org/apache/tika/parser/microsoft/ooxml/XSLFPowerPointExtractorDecorator.java
• trunk/tika-parsers/src/test/java/org/apache/tika/parser/microsoft/PowerPointParserTest.java

Andreas Beeker, we found a regression in spacing around differently formatted runs in ppt (here and TIKA-1778). Do you see any problems if we don't trim here:

for (HSLFTextRun htr : htp.getTextRuns()) {
    String line = htr.getRawText();
        if (line != null) {
            for (String fragment : line.split("\\u000b")){
                   ....
                xhtml.characters(fragment.trim());
...

If we drop it, will we get spaces where we shouldn't?

Perhaps trim the trailing carriage return?

fragment.replaceFirst("\r$", " ")

I would replace it with the empty string and use the regex escape for the line break

    fragment.replaceFirst("\\r$", "")

Apart of that, I've added a patch for bullet lists.
Currently HSLF always returns false for super/subscript ... I need to change this in POI.

Please comment, if it makes sense to add further markup information.

Patch for trim-replacement and bullet lists

Thank you! That fixed the vast majority of content diffs. There are still 6 ppts where we aren't adding a break between lines and we're getting improper concatenation of terms across lines. Should we move boolean isFirst = true; above for (HSLFTextRun htr : textRuns) {?

Example file...

Jeff Heimerman is one line and Senior ... is another line, but we aren't currently adding a <br/>

That was a bad idea.

The issue was that the first run ended with \u000b, and the split was hiding that "paragraph break" before the next run.

So, how about adding the if line.endsWith("\u000b"):

                   if (line != null) {
                        boolean isfirst = true;
                        for (String fragment : line.split("\\u000b")) {
                            if (!isfirst) {
                                xhtml.startElement("br");
                                xhtml.endElement("br");
                            }
                            isfirst = false;
                            xhtml.characters(removePBreak(fragment));
                        }
                        if (line.endsWith("\u000b")) {
                            xhtml.startElement("br");
                            xhtml.endElement("br");
                        }
                    }