Uploaded image for project: 'Tika'
  1. Tika
  2. TIKA-2306

Update Inception v3 to Inception v4 in Object recognition parser

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 1.14
    • Fix Version/s: None
    • Component/s: parser
    • Flags:
      Patch

      Description

      Object Recognition Parser currently uses Inception V3 model for the object classification task. Google released a newer Inception V4 model [1][2].
      It has an improved Top -1 accuracy of 80.2 and Top-5 accuracy of 95.2 [3].

      I believe that Tika community would benefit from it. I would be working on this issue in the next few days.

      [1] https://research.googleblog.com/2016/08/improving-inception-and-image.html
      [2] https://arxiv.org/abs/1602.07261
      [3] https://github.com/tensorflow/models/tree/master/slim

        Activity

        Hide
        githubbot ASF GitHub Bot added a comment -

        KranthiGV commented on issue #163: TIKA-2306: Update Inception v3 to Inception v4 in Object recognition parser
        URL: https://github.com/apache/tika/pull/163#issuecomment-295912340

        Thanks for updating the URL in dockerfile.
        I have made the minor changes on Wiki page now (V3 to V4, etc).
        I will be updating the results of running the parser by today.
        Thank you.

        ----------------------------------------------------------------
        This is an automated message from the Apache Git Service.
        To respond to the message, please log on GitHub and use the
        URL above to go to the specific comment.

        For queries about this service, please contact Infrastructure at:
        users@infra.apache.org

        Show
        githubbot ASF GitHub Bot added a comment - KranthiGV commented on issue #163: TIKA-2306 : Update Inception v3 to Inception v4 in Object recognition parser URL: https://github.com/apache/tika/pull/163#issuecomment-295912340 Thanks for updating the URL in dockerfile. I have made the minor changes on Wiki page now (V3 to V4, etc). I will be updating the results of running the parser by today. Thank you. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org
        Hide
        kranthigv Kranthi Kiran GV added a comment -

        Chris A. Mattmann
        Apologies! I have updated the wiki page at https://wiki.apache.org/tika/TikaAndVision
        Everything is updated and in place now.
        Thank you. Glad you liked the work, Chris!

        Show
        kranthigv Kranthi Kiran GV added a comment - Chris A. Mattmann Apologies! I have updated the wiki page at https://wiki.apache.org/tika/TikaAndVision Everything is updated and in place now. Thank you. Glad you liked the work, Chris!
        Hide
        chrismattmann Chris A. Mattmann added a comment -

        the wiki update is critical BTW. Normally I would say err on the side of updating the wiki before committing. That way our documentation is an explicit pre-requisite to the code.

        Show
        chrismattmann Chris A. Mattmann added a comment - the wiki update is critical BTW. Normally I would say err on the side of updating the wiki before committing. That way our documentation is an explicit pre-requisite to the code.
        Hide
        chrismattmann Chris A. Mattmann added a comment -
        Show
        chrismattmann Chris A. Mattmann added a comment - great work Kranthi Kiran GV and Thamme Gowda !
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Tika-trunk #1245 (See https://builds.apache.org/job/Tika-trunk/1245/)
        fix for TIKA-2306 contributed by kranthigv (kranthi.gv: https://github.com/apache/tika/commit/236db96393d94756dbc2e3f40b318f8f93b95dff)

        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Tika-trunk #1245 (See https://builds.apache.org/job/Tika-trunk/1245/ ) fix for TIKA-2306 contributed by kranthigv (kranthi.gv: https://github.com/apache/tika/commit/236db96393d94756dbc2e3f40b318f8f93b95dff ) (edit) tika-parsers/src/main/resources/org/apache/tika/parser/recognition/tf/classify_image.py fix for TIKA-2306 contributed by kranthigv (kranthi.gv: https://github.com/apache/tika/commit/0c0bd4bec2312355d2bc48426f8ec94306d0e4a0 ) (edit) tika-parsers/src/main/resources/org/apache/tika/parser/recognition/tf/classify_image.py fix for TIKA-2306 contributed by kranthigv (kranthi.gv: https://github.com/apache/tika/commit/09cb2df973f20e3a877ca1309b67384264650be0 ) (edit) tika-parsers/src/main/resources/org/apache/tika/parser/recognition/tf/inceptionapi.py (edit) tika-parsers/src/main/java/org/apache/tika/parser/recognition/tf/TensorflowRESTRecogniser.java (edit) tika-parsers/src/main/resources/org/apache/tika/parser/recognition/tf/InceptionRestDockerfile fix for TIKA-2306 contributed by kranthigv (kranthi.gv: https://github.com/apache/tika/commit/f92809ac19d5bef903ef1ac393092e6a13884fc0 ) (edit) tika-parsers/src/main/resources/org/apache/tika/parser/recognition/tf/InceptionRestDockerfile fix for TIKA-2306 contributed by kranthigv (kranthi.gv: https://github.com/apache/tika/commit/be773cacaf3c344c11fff9b85ebaf1d0dc8b5174 ) (edit) tika-parsers/src/test/java/org/apache/tika/parser/recognition/ObjectRecognitionParserTest.java (edit) tika-parsers/src/test/java/org/apache/tika/parser/recognition/tf/TensorflowImageRecParserTest.java
        Hide
        githubbot ASF GitHub Bot added a comment -

        KranthiGV commented on issue #163: TIKA-2306: Update Inception v3 to Inception v4 in Object recognition parser
        URL: https://github.com/apache/tika/pull/163#issuecomment-295912340

        Thanks for updating the URL in dockerfile.
        I have made the minor changes on Wiki page now (V3 to V4, etc).
        I will be updating the results of running the parser by today.
        Thank you.

        ----------------------------------------------------------------
        This is an automated message from the Apache Git Service.
        To respond to the message, please log on GitHub and use the
        URL above to go to the specific comment.

        For queries about this service, please contact Infrastructure at:
        users@infra.apache.org

        Show
        githubbot ASF GitHub Bot added a comment - KranthiGV commented on issue #163: TIKA-2306 : Update Inception v3 to Inception v4 in Object recognition parser URL: https://github.com/apache/tika/pull/163#issuecomment-295912340 Thanks for updating the URL in dockerfile. I have made the minor changes on Wiki page now (V3 to V4, etc). I will be updating the results of running the parser by today. Thank you. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org
        Hide
        githubbot ASF GitHub Bot added a comment -

        thammegowda commented on issue #163: TIKA-2306: Update Inception v3 to Inception v4 in Object recognition parser
        URL: https://github.com/apache/tika/pull/163#issuecomment-295906412

        @KranthiGV It is now merged. I updated the URL in dockerfile to the ASF repo.

        Could you please update the URLs in the wiki https://wiki.apache.org/tika/TikaAndVision
        Inception/v3 URLS should now be replaced by inception/v4

        Let us know if you need access to wiki.

        ----------------------------------------------------------------
        This is an automated message from the Apache Git Service.
        To respond to the message, please log on GitHub and use the
        URL above to go to the specific comment.

        For queries about this service, please contact Infrastructure at:
        users@infra.apache.org

        Show
        githubbot ASF GitHub Bot added a comment - thammegowda commented on issue #163: TIKA-2306 : Update Inception v3 to Inception v4 in Object recognition parser URL: https://github.com/apache/tika/pull/163#issuecomment-295906412 @KranthiGV It is now merged. I updated the URL in dockerfile to the ASF repo. Could you please update the URLs in the wiki https://wiki.apache.org/tika/TikaAndVision Inception/v3 URLS should now be replaced by inception/v4 Let us know if you need access to wiki. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org
        Hide
        githubbot ASF GitHub Bot added a comment -

        thammegowda closed pull request #163: TIKA-2306: Update Inception v3 to Inception v4 in Object recognition parser
        URL: https://github.com/apache/tika/pull/163

        ----------------------------------------------------------------
        This is an automated message from the Apache Git Service.
        To respond to the message, please log on GitHub and use the
        URL above to go to the specific comment.

        For queries about this service, please contact Infrastructure at:
        users@infra.apache.org

        Show
        githubbot ASF GitHub Bot added a comment - thammegowda closed pull request #163: TIKA-2306 : Update Inception v3 to Inception v4 in Object recognition parser URL: https://github.com/apache/tika/pull/163 ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org
        Hide
        githubbot ASF GitHub Bot added a comment -

        KranthiGV commented on issue #163: TIKA-2306: Update Inception v3 to Inception v4 in Object recognition parser
        URL: https://github.com/apache/tika/pull/163#issuecomment-295430728

        That sounds great! I'm looking at _TMP_IMAGE_METADATA_PARSER and the CompositeParser.

        ----------------------------------------------------------------
        This is an automated message from the Apache Git Service.
        To respond to the message, please log on GitHub and use the
        URL above to go to the specific comment.

        For queries about this service, please contact Infrastructure at:
        users@infra.apache.org

        Show
        githubbot ASF GitHub Bot added a comment - KranthiGV commented on issue #163: TIKA-2306 : Update Inception v3 to Inception v4 in Object recognition parser URL: https://github.com/apache/tika/pull/163#issuecomment-295430728 That sounds great! I'm looking at _TMP_IMAGE_METADATA_PARSER and the CompositeParser. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org
        Hide
        githubbot ASF GitHub Bot added a comment -

        thammegowda commented on issue #163: TIKA-2306: Update Inception v3 to Inception v4 in Object recognition parser
        URL: https://github.com/apache/tika/pull/163#issuecomment-295408367

        Got it. users who aren't using Inception will not lose JPEG metadata.
        We can merge this PR and support the composition of JPEG and ObjectRecognition in a new PR.

        ----------------------------------------------------------------
        This is an automated message from the Apache Git Service.
        To respond to the message, please log on GitHub and use the
        URL above to go to the specific comment.

        For queries about this service, please contact Infrastructure at:
        users@infra.apache.org

        Show
        githubbot ASF GitHub Bot added a comment - thammegowda commented on issue #163: TIKA-2306 : Update Inception v3 to Inception v4 in Object recognition parser URL: https://github.com/apache/tika/pull/163#issuecomment-295408367 Got it. users who aren't using Inception will not lose JPEG metadata. We can merge this PR and support the composition of JPEG and ObjectRecognition in a new PR. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org
        Hide
        githubbot ASF GitHub Bot added a comment -

        tballison commented on issue #163: TIKA-2306: Update Inception v3 to Inception v4 in Object recognition parser
        URL: https://github.com/apache/tika/pull/163#issuecomment-295382171

        > That's right.
        Ok, I'm ok with this behavior. Users who want inception lose JPEG metadata (for now). What I don't want to repeat is users who aren't using the ObjectRecognizer losing JPEG metadata.

        Bonus points for copy/paste of TesseractOCRParser's metadata insertion, but not required, IMHO.

        ----------------------------------------------------------------
        This is an automated message from the Apache Git Service.
        To respond to the message, please log on GitHub and use the
        URL above to go to the specific comment.

        For queries about this service, please contact Infrastructure at:
        users@infra.apache.org

        Show
        githubbot ASF GitHub Bot added a comment - tballison commented on issue #163: TIKA-2306 : Update Inception v3 to Inception v4 in Object recognition parser URL: https://github.com/apache/tika/pull/163#issuecomment-295382171 > That's right. Ok, I'm ok with this behavior. Users who want inception lose JPEG metadata (for now). What I don't want to repeat is users who aren't using the ObjectRecognizer losing JPEG metadata. Bonus points for copy/paste of TesseractOCRParser's metadata insertion, but not required, IMHO. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org
        Hide
        githubbot ASF GitHub Bot added a comment -

        thammegowda commented on issue #163: TIKA-2306: Update Inception v3 to Inception v4 in Object recognition parser
        URL: https://github.com/apache/tika/pull/163#issuecomment-295379808

        > I looks like you have to turn the ObjectRecognitionParser on via config, etc? So, the default behavior is that the JPEGParser is called and the ObjectRecognitionParser is not called, right?

        That's right.

        > To merge metadata, see: _TMP_IMAGE_METADATA_PARSER in TesseractOCRParser

        Thanks! I will have a look

        ----------------------------------------------------------------
        This is an automated message from the Apache Git Service.
        To respond to the message, please log on GitHub and use the
        URL above to go to the specific comment.

        For queries about this service, please contact Infrastructure at:
        users@infra.apache.org

        Show
        githubbot ASF GitHub Bot added a comment - thammegowda commented on issue #163: TIKA-2306 : Update Inception v3 to Inception v4 in Object recognition parser URL: https://github.com/apache/tika/pull/163#issuecomment-295379808 > I looks like you have to turn the ObjectRecognitionParser on via config, etc? So, the default behavior is that the JPEGParser is called and the ObjectRecognitionParser is not called, right? That's right. > To merge metadata, see: _TMP_IMAGE_METADATA_PARSER in TesseractOCRParser Thanks! I will have a look ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org
        Hide
        githubbot ASF GitHub Bot added a comment -

        tballison commented on issue #163: TIKA-2306: Update Inception v3 to Inception v4 in Object recognition parser
        URL: https://github.com/apache/tika/pull/163#issuecomment-295379227

        I looks like you have to turn the ObjectRecognitionParser on via config, etc? So, the default behavior is that the JPEGParser is called and the ObjectRecognitionParser is not called, right?

        To merge metadata, see: _TMP_IMAGE_METADATA_PARSER in TesseractOCRParser

        ----------------------------------------------------------------
        This is an automated message from the Apache Git Service.
        To respond to the message, please log on GitHub and use the
        URL above to go to the specific comment.

        For queries about this service, please contact Infrastructure at:
        users@infra.apache.org

        Show
        githubbot ASF GitHub Bot added a comment - tballison commented on issue #163: TIKA-2306 : Update Inception v3 to Inception v4 in Object recognition parser URL: https://github.com/apache/tika/pull/163#issuecomment-295379227 I looks like you have to turn the ObjectRecognitionParser on via config, etc? So, the default behavior is that the JPEGParser is called and the ObjectRecognitionParser is not called, right? To merge metadata, see: _TMP_IMAGE_METADATA_PARSER in TesseractOCRParser ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org
        Hide
        githubbot ASF GitHub Bot added a comment -

        tballison commented on issue #163: TIKA-2306: Update Inception v3 to Inception v4 in Object recognition parser
        URL: https://github.com/apache/tika/pull/163#issuecomment-295379227

        I looks like you have to turn the ObjectRecognitionParser on via config, etc? So, the default behavior is that the JPEGParser is called and the ObjectRecognitionParser is not called, right?

        To merge metadata, see: _TMP_IMAGE_METADATA_PARSER in TesseractOCRParser

        ----------------------------------------------------------------
        This is an automated message from the Apache Git Service.
        To respond to the message, please log on GitHub and use the
        URL above to go to the specific comment.

        For queries about this service, please contact Infrastructure at:
        users@infra.apache.org

        Show
        githubbot ASF GitHub Bot added a comment - tballison commented on issue #163: TIKA-2306 : Update Inception v3 to Inception v4 in Object recognition parser URL: https://github.com/apache/tika/pull/163#issuecomment-295379227 I looks like you have to turn the ObjectRecognitionParser on via config, etc? So, the default behavior is that the JPEGParser is called and the ObjectRecognitionParser is not called, right? To merge metadata, see: _TMP_IMAGE_METADATA_PARSER in TesseractOCRParser ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org
        Hide
        githubbot ASF GitHub Bot added a comment -

        thammegowda commented on issue #163: TIKA-2306: Update Inception v3 to Inception v4 in Object recognition parser
        URL: https://github.com/apache/tika/pull/163#issuecomment-295358397

        > Wait, does this prevent jpeg metadata from being extracted?

        Oh yes! That is the case with existing ObjectRecognition Parser.

        How do we run two different parsers for the same file and merge the metadata?

        ----------------------------------------------------------------
        This is an automated message from the Apache Git Service.
        To respond to the message, please log on GitHub and use the
        URL above to go to the specific comment.

        For queries about this service, please contact Infrastructure at:
        users@infra.apache.org

        Show
        githubbot ASF GitHub Bot added a comment - thammegowda commented on issue #163: TIKA-2306 : Update Inception v3 to Inception v4 in Object recognition parser URL: https://github.com/apache/tika/pull/163#issuecomment-295358397 > Wait, does this prevent jpeg metadata from being extracted? Oh yes! That is the case with existing ObjectRecognition Parser. How do we run two different parsers for the same file and merge the metadata? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org
        Hide
        githubbot ASF GitHub Bot added a comment -

        thammegowda commented on issue #163: TIKA-2306: Update Inception v3 to Inception v4 in Object recognition parser
        URL: https://github.com/apache/tika/pull/163#issuecomment-295358397

        > Wait, does this prevent jpeg metadata from being extracted?

        Oh yes! That is the case with existing ObjectRecognition Parser.

        How do we run two different parsers for the same file and merge the metadata?

        ----------------------------------------------------------------
        This is an automated message from the Apache Git Service.
        To respond to the message, please log on GitHub and use the
        URL above to go to the specific comment.

        For queries about this service, please contact Infrastructure at:
        users@infra.apache.org

        Show
        githubbot ASF GitHub Bot added a comment - thammegowda commented on issue #163: TIKA-2306 : Update Inception v3 to Inception v4 in Object recognition parser URL: https://github.com/apache/tika/pull/163#issuecomment-295358397 > Wait, does this prevent jpeg metadata from being extracted? Oh yes! That is the case with existing ObjectRecognition Parser. How do we run two different parsers for the same file and merge the metadata? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org
        Hide
        githubbot ASF GitHub Bot added a comment -

        tballison commented on issue #163: TIKA-2306: Update Inception v3 to Inception v4 in Object recognition parser
        URL: https://github.com/apache/tika/pull/163#issuecomment-295354514

        What's the worst that could happen? Go for it...

        ----------------------------------------------------------------
        This is an automated message from the Apache Git Service.
        To respond to the message, please log on GitHub and use the
        URL above to go to the specific comment.

        For queries about this service, please contact Infrastructure at:
        users@infra.apache.org

        Show
        githubbot ASF GitHub Bot added a comment - tballison commented on issue #163: TIKA-2306 : Update Inception v3 to Inception v4 in Object recognition parser URL: https://github.com/apache/tika/pull/163#issuecomment-295354514 What's the worst that could happen? Go for it... ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org
        Hide
        githubbot ASF GitHub Bot added a comment -

        tballison commented on issue #163: TIKA-2306: Update Inception v3 to Inception v4 in Object recognition parser
        URL: https://github.com/apache/tika/pull/163#issuecomment-295354514

        What's the worst that could happen? Go for it...

        ----------------------------------------------------------------
        This is an automated message from the Apache Git Service.
        To respond to the message, please log on GitHub and use the
        URL above to go to the specific comment.

        For queries about this service, please contact Infrastructure at:
        users@infra.apache.org

        Show
        githubbot ASF GitHub Bot added a comment - tballison commented on issue #163: TIKA-2306 : Update Inception v3 to Inception v4 in Object recognition parser URL: https://github.com/apache/tika/pull/163#issuecomment-295354514 What's the worst that could happen? Go for it... ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org
        Hide
        githubbot ASF GitHub Bot added a comment -

        thammegowda commented on issue #163: TIKA-2306: Update Inception v3 to Inception v4 in Object recognition parser
        URL: https://github.com/apache/tika/pull/163#issuecomment-295351578

        @tballison Can we merge this before the Tika 1.15 release.
        It's ready to be merged, I had it tested.

        ----------------------------------------------------------------
        This is an automated message from the Apache Git Service.
        To respond to the message, please log on GitHub and use the
        URL above to go to the specific comment.

        For queries about this service, please contact Infrastructure at:
        users@infra.apache.org

        Show
        githubbot ASF GitHub Bot added a comment - thammegowda commented on issue #163: TIKA-2306 : Update Inception v3 to Inception v4 in Object recognition parser URL: https://github.com/apache/tika/pull/163#issuecomment-295351578 @tballison Can we merge this before the Tika 1.15 release. It's ready to be merged, I had it tested. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org
        Hide
        githubbot ASF GitHub Bot added a comment -

        thammegowda commented on issue #163: TIKA-2306: Update Inception v3 to Inception v4 in Object recognition parser
        URL: https://github.com/apache/tika/pull/163#issuecomment-294238344

        Thanks, @KranthiGV.
        LGTM!

        ----------------------------------------------------------------
        This is an automated message from the Apache Git Service.
        To respond to the message, please log on GitHub and use the
        URL above to go to the specific comment.

        For queries about this service, please contact Infrastructure at:
        users@infra.apache.org

        Show
        githubbot ASF GitHub Bot added a comment - thammegowda commented on issue #163: TIKA-2306 : Update Inception v3 to Inception v4 in Object recognition parser URL: https://github.com/apache/tika/pull/163#issuecomment-294238344 Thanks, @KranthiGV. LGTM! ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org
        Hide
        githubbot ASF GitHub Bot added a comment -

        KranthiGV commented on issue #163: TIKA-2306: Update Inception v3 to Inception v4 in Object recognition parser
        URL: https://github.com/apache/tika/pull/163#issuecomment-294143638

        @thammegowda
        I have made the necessary changes at Reduced disk I/O commit (https://github.com/apache/tika/pull/163/commits/db8c81410b8468e1a524ed2dbb28abc7154fb11d)
        The performance is compared by running it 50 times on an image.
        ```
        script.sh:
        n=0; while [[ $n -lt 50 ]]; do java -jar tika-app/target/tika-app-1.15-SNAPSHOT.jar --config=tika-parsers/src/test/resources/org/apache/tika/parser/recognition/tika-config-tflow-rest.xml http://www.trbimg.com/img-57226a08/turbine/ct-tesla-model-3-unveiling-20160404/650/650x366; n=$((n+1)); done

        time ./script.sh
        ```
        Before changes:
        `real 4m33.334s`

        After changes:
        `real 1m0.736s`

        TODO after merging:
        1) Update the inceptionapi.py link to Apache's repo link.
        2) Document the usage in Wiki.

        ----------------------------------------------------------------
        This is an automated message from the Apache Git Service.
        To respond to the message, please log on GitHub and use the
        URL above to go to the specific comment.

        For queries about this service, please contact Infrastructure at:
        users@infra.apache.org

        Show
        githubbot ASF GitHub Bot added a comment - KranthiGV commented on issue #163: TIKA-2306 : Update Inception v3 to Inception v4 in Object recognition parser URL: https://github.com/apache/tika/pull/163#issuecomment-294143638 @thammegowda I have made the necessary changes at Reduced disk I/O commit ( https://github.com/apache/tika/pull/163/commits/db8c81410b8468e1a524ed2dbb28abc7154fb11d ) The performance is compared by running it 50 times on an image. ``` script.sh: n=0; while [[ $n -lt 50 ]]; do java -jar tika-app/target/tika-app-1.15-SNAPSHOT.jar --config=tika-parsers/src/test/resources/org/apache/tika/parser/recognition/tika-config-tflow-rest.xml http://www.trbimg.com/img-57226a08/turbine/ct-tesla-model-3-unveiling-20160404/650/650x366 ; n=$((n+1)); done time ./script.sh ``` Before changes: `real 4m33.334s` After changes: `real 1m0.736s` TODO after merging: 1) Update the inceptionapi.py link to Apache's repo link. 2) Document the usage in Wiki. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org
        Hide
        githubbot ASF GitHub Bot added a comment -

        KranthiGV commented on a change in pull request #163: TIKA-2306: Update Inception v3 to Inception v4 in Object recognition parser
        URL: https://github.com/apache/tika/pull/163#discussion_r111530356

        ##########
        File path: tika-parsers/src/main/resources/org/apache/tika/parser/recognition/tf/classify_image.py
        ##########
        @@ -44,169 +49,205 @@
        from six.moves import urllib
        import tensorflow as tf

        +from datasets import imagenet, dataset_utils
        +from nets import inception
        +from preprocessing import inception_preprocessing
        +
        +slim = tf.contrib.slim
        +
        FLAGS = tf.app.flags.FLAGS

          1. classify_image_graph_def.pb:
          2. Binary representation of the GraphDef protocol buffer.
          3. imagenet_synset_to_human_label_map.txt:
            +# inception_v4.ckpt
            +# Inception V4 checkpoint file.
            +# imagenet_metadata.txt
        1. Map from synset ID to a human readable string.
          1. imagenet_2012_challenge_label_map_proto.pbtxt:
            +# imagenet_lsvrc_2015_synsets.txt
        2. Text representation of a protocol buffer mapping a label to synset ID.
          tf.app.flags.DEFINE_string(
          'model_dir', '/tmp/imagenet',
        • """Path to classify_image_graph_def.pb, """
        • """imagenet_synset_to_human_label_map.txt, and """
        • """imagenet_2012_challenge_label_map_proto.pbtxt.""")
          + """Path to inception_v4.ckpt, """
          + """imagenet_lsvrc_2015_synsets.txt, and """
          + """imagenet_metadata.txt.""")
          tf.app.flags.DEFINE_string('image_file', '',
          """Absolute path to image file.""")
          tf.app.flags.DEFINE_integer('num_top_predictions', 5,
          """Display this many predictions.""")
        1. pylint: disable=line-too-long
          -DATA_URL = 'http://download.tensorflow.org/models/image/imagenet/inception-2015-12-05.tgz'
          +DATA_URL = 'http://download.tensorflow.org/models/inception_v4_2016_09_09.tar.gz'
        2. pylint: enable=line-too-long

        -class NodeLookup(object):

        • """Converts integer node ID's to human readable labels."""
          +def create_readable_names_for_imagenet_labels():
          + """Create a dict mapping label id to human readable string.
          +
          + Returns:
          + labels_to_names: dictionary where keys are integers from to 1000
          + and values are human-readable names.
          +
          + We retrieve a synset file, which contains a list of valid synset labels used
          + by ILSVRC competition. There is one synset one per line, eg.
          + # n01440764
          + # n01443537
          + We also retrieve a synset_to_human_file, which contains a mapping from synsets
          + to human-readable names for every synset in Imagenet. These are stored in a
          + tsv format, as follows:
          + # n02119247 black fox
          + # n02119359 silver fox
          + We assign each synset (in alphabetical order) an integer, starting from 1
          + (since 0 is reserved for the background class).
          +
          + Code is based on
          + https://github.com/tensorflow/models/blob/master/inception/inception/data/build_imagenet_data.py#L463
          + """
          +
          + # pylint: disable=line-too-long
          +
          + dest_directory = FLAGS.model_dir
          +
          + synset_list = [s.strip() for s in open(os.path.join(
          + dest_directory, 'imagenet_lsvrc_2015_synsets.txt')).readlines()]
          + num_synsets_in_ilsvrc = len(synset_list)
          + assert num_synsets_in_ilsvrc == 1000
        • def _init_(self,
        • label_lookup_path=None,
        • uid_lookup_path=None):
        • if not label_lookup_path:
        • label_lookup_path = os.path.join(
        • FLAGS.model_dir, 'imagenet_2012_challenge_label_map_proto.pbtxt')
        • if not uid_lookup_path:
        • uid_lookup_path = os.path.join(
        • FLAGS.model_dir, 'imagenet_synset_to_human_label_map.txt')
        • self.node_lookup = self.load(label_lookup_path, uid_lookup_path)
          + synset_to_human_list = open(os.path.join(
          + dest_directory, 'imagenet_metadata.txt')).readlines()
          + num_synsets_in_all_imagenet = len(synset_to_human_list)
          + assert num_synsets_in_all_imagenet == 21842
        • def load(self, label_lookup_path, uid_lookup_path):
        • """Loads a human readable English name for each softmax node.
          + synset_to_human = {}
          + for s in synset_to_human_list:
          + parts = s.strip().split('\t')
          + assert len(parts) == 2
          + synset = parts[0]
          + human = parts[1]
          + synset_to_human[synset] = human
          +
          + label_index = 1
          + labels_to_names = {0: 'background'}

          + for synset in synset_list:
          + name = synset_to_human[synset]
          + labels_to_names[label_index] = name
          + label_index += 1
          +
          + return labels_to_names
          +
          +
          +def run_inference_on_image(image):
          + """Runs inference on an image.

        Args:

        • label_lookup_path: string UID to integer node ID.
        • uid_lookup_path: string UID to human-readable string.
          + image: Image file name.

        Returns:

        • dict from integer node ID to human-readable string.
          + Nothing
          """
        • if not tf.gfile.Exists(uid_lookup_path):
        • tf.logging.fatal('File does not exist %s', uid_lookup_path)
        • if not tf.gfile.Exists(label_lookup_path):
        • tf.logging.fatal('File does not exist %s', label_lookup_path)
          -
        • # Loads mapping from string UID to human-readable string
        • proto_as_ascii_lines = tf.gfile.GFile(uid_lookup_path).readlines()
        • uid_to_human = {}
        • p = re.compile(r'[n\d][ \S,]')
        • for line in proto_as_ascii_lines:
        • parsed_items = p.findall(line)
        • uid = parsed_items[0]
        • human_string = parsed_items[2]
        • uid_to_human[uid] = human_string
          -
        • # Loads mapping from string UID to integer node ID.
        • node_id_to_uid = {}
        • proto_as_ascii = tf.gfile.GFile(label_lookup_path).readlines()
        • for line in proto_as_ascii:
        • if line.startswith(' target_class:'):
        • target_class = int(line.split(': ')[1])
        • if line.startswith(' target_class_string:'):
        • target_class_string = line.split(': ')[1]
        • node_id_to_uid[target_class] = target_class_string[1:-2]
          -
        • # Loads the final mapping of integer node ID to human-readable string
        • node_id_to_name = {}
        • for key, val in node_id_to_uid.items():
        • if val not in uid_to_human:
        • tf.logging.fatal('Failed to locate: %s', val)
        • name = uid_to_human[val]
        • node_id_to_name[key] = name
          -
        • return node_id_to_name
          -
        • def id_to_string(self, node_id):
        • if node_id not in self.node_lookup:
        • return ''
        • return self.node_lookup[node_id]
          -
          -
          -def create_graph():
        • """Creates a graph from saved GraphDef file and returns a saver."""
        • # Creates graph from saved graph_def.pb.
        • with tf.gfile.FastGFile(os.path.join(
        • FLAGS.model_dir, 'classify_image_graph_def.pb'), 'rb') as f:
        • graph_def = tf.GraphDef()
        • graph_def.ParseFromString(f.read())
        • _ = tf.import_graph_def(graph_def, name='')
          + dest_directory = FLAGS.model_dir

        + image_size = inception.inception_v4.default_image_size

        -def run_inference_on_image(image):

        • """Runs inference on an image.
          -
        • Args:
        • image: Image file name.
          -
        • Returns:
        • Nothing
        • """
        • if not tf.gfile.Exists(image):
        • tf.logging.fatal('File does not exist %s', image)
        • image_data = tf.gfile.FastGFile(image, 'rb').read()
          -
        • # Creates graph from saved GraphDef.
        • create_graph()
          -
        • with tf.Session() as sess:
        • # Some useful tensors:
        • # 'softmax:0': A tensor containing the normalized prediction across
        • # 1000 labels.
        • # 'pool_3:0': A tensor containing the next-to-last layer containing 2048
        • # float description of the image.
        • # 'DecodeJpeg/contents:0': A tensor containing a string providing JPEG
        • # encoding of the image.
        • # Runs the softmax tensor by feeding the image_data as input to the graph.
        • softmax_tensor = sess.graph.get_tensor_by_name('softmax:0')
        • predictions = sess.run(softmax_tensor,
        • {'DecodeJpeg/contents:0': image_data}

          )

        • predictions = np.squeeze(predictions)
          -
        • # Creates node ID --> English string lookup.
        • node_lookup = NodeLookup()
          -
        • top_k = predictions.argsort()[-FLAGS.num_top_predictions:][::-1]
        • for node_id in top_k:
        • human_string = node_lookup.id_to_string(node_id)
        • score = predictions[node_id]
        • print('%s (score = %.5f)' % (human_string, score))
          + if not tf.gfile.Exists(image):
          + tf.logging.fatal('File does not exist %s', image)
          + image_string = tf.gfile.FastGFile(image, 'rb').read()

        + with tf.Graph().as_default():
        + image = tf.image.decode_jpeg(image_string, channels=3)
        + processed_image = inception_preprocessing.preprocess_image(
        + image, image_size, image_size, is_training=False)
        + processed_images = tf.expand_dims(processed_image, 0)
        +
        + # Create the model, use the default arg scope to configure the batch
        + # norm parameters.
        + with slim.arg_scope(inception.inception_v4_arg_scope()):
        + logits, _ = inception.inception_v4(
        + processed_images, num_classes=1001, is_training=False)
        + probabilities = tf.nn.softmax(logits)
        +
        + init_fn = slim.assign_from_checkpoint_fn(
        + os.path.join(dest_directory, 'inception_v4.ckpt'),
        + slim.get_model_variables('InceptionV4'))
        +
        + with tf.Session() as sess:
        + init_fn(sess)
        + _, probabilities = sess.run([image, probabilities])

        Review comment:
        @thammegowda It makes sense. Thanks for clarifying. I'll work on it now.

        ----------------------------------------------------------------
        This is an automated message from the Apache Git Service.
        To respond to the message, please log on GitHub and use the
        URL above to go to the specific comment.

        For queries about this service, please contact Infrastructure at:
        users@infra.apache.org

        Show
        githubbot ASF GitHub Bot added a comment - KranthiGV commented on a change in pull request #163: TIKA-2306 : Update Inception v3 to Inception v4 in Object recognition parser URL: https://github.com/apache/tika/pull/163#discussion_r111530356 ########## File path: tika-parsers/src/main/resources/org/apache/tika/parser/recognition/tf/classify_image.py ########## @@ -44,169 +49,205 @@ from six.moves import urllib import tensorflow as tf +from datasets import imagenet, dataset_utils +from nets import inception +from preprocessing import inception_preprocessing + +slim = tf.contrib.slim + FLAGS = tf.app.flags.FLAGS classify_image_graph_def.pb: Binary representation of the GraphDef protocol buffer. imagenet_synset_to_human_label_map.txt: +# inception_v4.ckpt +# Inception V4 checkpoint file. +# imagenet_metadata.txt Map from synset ID to a human readable string. imagenet_2012_challenge_label_map_proto.pbtxt: +# imagenet_lsvrc_2015_synsets.txt Text representation of a protocol buffer mapping a label to synset ID. tf.app.flags.DEFINE_string( 'model_dir', '/tmp/imagenet', """Path to classify_image_graph_def.pb, """ """imagenet_synset_to_human_label_map.txt, and """ """imagenet_2012_challenge_label_map_proto.pbtxt.""") + """Path to inception_v4.ckpt, """ + """imagenet_lsvrc_2015_synsets.txt, and """ + """imagenet_metadata.txt.""") tf.app.flags.DEFINE_string('image_file', '', """Absolute path to image file.""") tf.app.flags.DEFINE_integer('num_top_predictions', 5, """Display this many predictions.""") pylint: disable=line-too-long -DATA_URL = 'http://download.tensorflow.org/models/image/imagenet/inception-2015-12-05.tgz' +DATA_URL = 'http://download.tensorflow.org/models/inception_v4_2016_09_09.tar.gz' pylint: enable=line-too-long -class NodeLookup(object): """Converts integer node ID's to human readable labels.""" +def create_readable_names_for_imagenet_labels(): + """Create a dict mapping label id to human readable string. + + Returns: + labels_to_names: dictionary where keys are integers from to 1000 + and values are human-readable names. + + We retrieve a synset file, which contains a list of valid synset labels used + by ILSVRC competition. There is one synset one per line, eg. + # n01440764 + # n01443537 + We also retrieve a synset_to_human_file, which contains a mapping from synsets + to human-readable names for every synset in Imagenet. These are stored in a + tsv format, as follows: + # n02119247 black fox + # n02119359 silver fox + We assign each synset (in alphabetical order) an integer, starting from 1 + (since 0 is reserved for the background class). + + Code is based on + https://github.com/tensorflow/models/blob/master/inception/inception/data/build_imagenet_data.py#L463 + """ + + # pylint: disable=line-too-long + + dest_directory = FLAGS.model_dir + + synset_list = [s.strip() for s in open(os.path.join( + dest_directory, 'imagenet_lsvrc_2015_synsets.txt')).readlines()] + num_synsets_in_ilsvrc = len(synset_list) + assert num_synsets_in_ilsvrc == 1000 def _ init _(self, label_lookup_path=None, uid_lookup_path=None): if not label_lookup_path: label_lookup_path = os.path.join( FLAGS.model_dir, 'imagenet_2012_challenge_label_map_proto.pbtxt') if not uid_lookup_path: uid_lookup_path = os.path.join( FLAGS.model_dir, 'imagenet_synset_to_human_label_map.txt') self.node_lookup = self.load(label_lookup_path, uid_lookup_path) + synset_to_human_list = open(os.path.join( + dest_directory, 'imagenet_metadata.txt')).readlines() + num_synsets_in_all_imagenet = len(synset_to_human_list) + assert num_synsets_in_all_imagenet == 21842 def load(self, label_lookup_path, uid_lookup_path): """Loads a human readable English name for each softmax node. + synset_to_human = {} + for s in synset_to_human_list: + parts = s.strip().split('\t') + assert len(parts) == 2 + synset = parts [0] + human = parts [1] + synset_to_human [synset] = human + + label_index = 1 + labels_to_names = {0: 'background'} + for synset in synset_list: + name = synset_to_human [synset] + labels_to_names [label_index] = name + label_index += 1 + + return labels_to_names + + +def run_inference_on_image(image): + """Runs inference on an image. Args: label_lookup_path: string UID to integer node ID. uid_lookup_path: string UID to human-readable string. + image: Image file name. Returns: dict from integer node ID to human-readable string. + Nothing """ if not tf.gfile.Exists(uid_lookup_path): tf.logging.fatal('File does not exist %s', uid_lookup_path) if not tf.gfile.Exists(label_lookup_path): tf.logging.fatal('File does not exist %s', label_lookup_path) - # Loads mapping from string UID to human-readable string proto_as_ascii_lines = tf.gfile.GFile(uid_lookup_path).readlines() uid_to_human = {} p = re.compile(r' [n\d] [ \S,] ') for line in proto_as_ascii_lines: parsed_items = p.findall(line) uid = parsed_items [0] human_string = parsed_items [2] uid_to_human [uid] = human_string - # Loads mapping from string UID to integer node ID. node_id_to_uid = {} proto_as_ascii = tf.gfile.GFile(label_lookup_path).readlines() for line in proto_as_ascii: if line.startswith(' target_class:'): target_class = int(line.split(': ') [1] ) if line.startswith(' target_class_string:'): target_class_string = line.split(': ') [1] node_id_to_uid [target_class] = target_class_string [1:-2] - # Loads the final mapping of integer node ID to human-readable string node_id_to_name = {} for key, val in node_id_to_uid.items(): if val not in uid_to_human: tf.logging.fatal('Failed to locate: %s', val) name = uid_to_human [val] node_id_to_name [key] = name - return node_id_to_name - def id_to_string(self, node_id): if node_id not in self.node_lookup: return '' return self.node_lookup [node_id] - - -def create_graph(): """Creates a graph from saved GraphDef file and returns a saver.""" # Creates graph from saved graph_def.pb. with tf.gfile.FastGFile(os.path.join( FLAGS.model_dir, 'classify_image_graph_def.pb'), 'rb') as f: graph_def = tf.GraphDef() graph_def.ParseFromString(f.read()) _ = tf.import_graph_def(graph_def, name='') + dest_directory = FLAGS.model_dir + image_size = inception.inception_v4.default_image_size -def run_inference_on_image(image): """Runs inference on an image. - Args: image: Image file name. - Returns: Nothing """ if not tf.gfile.Exists(image): tf.logging.fatal('File does not exist %s', image) image_data = tf.gfile.FastGFile(image, 'rb').read() - # Creates graph from saved GraphDef. create_graph() - with tf.Session() as sess: # Some useful tensors: # 'softmax:0': A tensor containing the normalized prediction across # 1000 labels. # 'pool_3:0': A tensor containing the next-to-last layer containing 2048 # float description of the image. # 'DecodeJpeg/contents:0': A tensor containing a string providing JPEG # encoding of the image. # Runs the softmax tensor by feeding the image_data as input to the graph. softmax_tensor = sess.graph.get_tensor_by_name('softmax:0') predictions = sess.run(softmax_tensor, {'DecodeJpeg/contents:0': image_data} ) predictions = np.squeeze(predictions) - # Creates node ID --> English string lookup. node_lookup = NodeLookup() - top_k = predictions.argsort() [-FLAGS.num_top_predictions:] [::-1] for node_id in top_k: human_string = node_lookup.id_to_string(node_id) score = predictions [node_id] print('%s (score = %.5f)' % (human_string, score)) + if not tf.gfile.Exists(image): + tf.logging.fatal('File does not exist %s', image) + image_string = tf.gfile.FastGFile(image, 'rb').read() + with tf.Graph().as_default(): + image = tf.image.decode_jpeg(image_string, channels=3) + processed_image = inception_preprocessing.preprocess_image( + image, image_size, image_size, is_training=False) + processed_images = tf.expand_dims(processed_image, 0) + + # Create the model, use the default arg scope to configure the batch + # norm parameters. + with slim.arg_scope(inception.inception_v4_arg_scope()): + logits, _ = inception.inception_v4( + processed_images, num_classes=1001, is_training=False) + probabilities = tf.nn.softmax(logits) + + init_fn = slim.assign_from_checkpoint_fn( + os.path.join(dest_directory, 'inception_v4.ckpt'), + slim.get_model_variables('InceptionV4')) + + with tf.Session() as sess: + init_fn(sess) + _, probabilities = sess.run( [image, probabilities] ) Review comment: @thammegowda It makes sense. Thanks for clarifying. I'll work on it now. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org
        Hide
        githubbot ASF GitHub Bot added a comment -

        thammegowda commented on a change in pull request #163: TIKA-2306: Update Inception v3 to Inception v4 in Object recognition parser
        URL: https://github.com/apache/tika/pull/163#discussion_r111530205

        ##########
        File path: tika-parsers/src/main/resources/org/apache/tika/parser/recognition/tf/classify_image.py
        ##########
        @@ -44,169 +49,205 @@
        from six.moves import urllib
        import tensorflow as tf

        +from datasets import imagenet, dataset_utils
        +from nets import inception
        +from preprocessing import inception_preprocessing
        +
        +slim = tf.contrib.slim
        +
        FLAGS = tf.app.flags.FLAGS

          1. classify_image_graph_def.pb:
          2. Binary representation of the GraphDef protocol buffer.
          3. imagenet_synset_to_human_label_map.txt:
            +# inception_v4.ckpt
            +# Inception V4 checkpoint file.
            +# imagenet_metadata.txt
        1. Map from synset ID to a human readable string.
          1. imagenet_2012_challenge_label_map_proto.pbtxt:
            +# imagenet_lsvrc_2015_synsets.txt
        2. Text representation of a protocol buffer mapping a label to synset ID.
          tf.app.flags.DEFINE_string(
          'model_dir', '/tmp/imagenet',
        • """Path to classify_image_graph_def.pb, """
        • """imagenet_synset_to_human_label_map.txt, and """
        • """imagenet_2012_challenge_label_map_proto.pbtxt.""")
          + """Path to inception_v4.ckpt, """
          + """imagenet_lsvrc_2015_synsets.txt, and """
          + """imagenet_metadata.txt.""")
          tf.app.flags.DEFINE_string('image_file', '',
          """Absolute path to image file.""")
          tf.app.flags.DEFINE_integer('num_top_predictions', 5,
          """Display this many predictions.""")
        1. pylint: disable=line-too-long
          -DATA_URL = 'http://download.tensorflow.org/models/image/imagenet/inception-2015-12-05.tgz'
          +DATA_URL = 'http://download.tensorflow.org/models/inception_v4_2016_09_09.tar.gz'
        2. pylint: enable=line-too-long

        -class NodeLookup(object):

        • """Converts integer node ID's to human readable labels."""
          +def create_readable_names_for_imagenet_labels():
          + """Create a dict mapping label id to human readable string.
          +
          + Returns:
          + labels_to_names: dictionary where keys are integers from to 1000
          + and values are human-readable names.
          +
          + We retrieve a synset file, which contains a list of valid synset labels used
          + by ILSVRC competition. There is one synset one per line, eg.
          + # n01440764
          + # n01443537
          + We also retrieve a synset_to_human_file, which contains a mapping from synsets
          + to human-readable names for every synset in Imagenet. These are stored in a
          + tsv format, as follows:
          + # n02119247 black fox
          + # n02119359 silver fox
          + We assign each synset (in alphabetical order) an integer, starting from 1
          + (since 0 is reserved for the background class).
          +
          + Code is based on
          + https://github.com/tensorflow/models/blob/master/inception/inception/data/build_imagenet_data.py#L463
          + """
          +
          + # pylint: disable=line-too-long
          +
          + dest_directory = FLAGS.model_dir
          +
          + synset_list = [s.strip() for s in open(os.path.join(
          + dest_directory, 'imagenet_lsvrc_2015_synsets.txt')).readlines()]
          + num_synsets_in_ilsvrc = len(synset_list)
          + assert num_synsets_in_ilsvrc == 1000
        • def _init_(self,
        • label_lookup_path=None,
        • uid_lookup_path=None):
        • if not label_lookup_path:
        • label_lookup_path = os.path.join(
        • FLAGS.model_dir, 'imagenet_2012_challenge_label_map_proto.pbtxt')
        • if not uid_lookup_path:
        • uid_lookup_path = os.path.join(
        • FLAGS.model_dir, 'imagenet_synset_to_human_label_map.txt')
        • self.node_lookup = self.load(label_lookup_path, uid_lookup_path)
          + synset_to_human_list = open(os.path.join(
          + dest_directory, 'imagenet_metadata.txt')).readlines()
          + num_synsets_in_all_imagenet = len(synset_to_human_list)
          + assert num_synsets_in_all_imagenet == 21842
        • def load(self, label_lookup_path, uid_lookup_path):
        • """Loads a human readable English name for each softmax node.
          + synset_to_human = {}
          + for s in synset_to_human_list:
          + parts = s.strip().split('\t')
          + assert len(parts) == 2
          + synset = parts[0]
          + human = parts[1]
          + synset_to_human[synset] = human
          +
          + label_index = 1
          + labels_to_names = {0: 'background'}

          + for synset in synset_list:
          + name = synset_to_human[synset]
          + labels_to_names[label_index] = name
          + label_index += 1
          +
          + return labels_to_names
          +
          +
          +def run_inference_on_image(image):
          + """Runs inference on an image.

        Args:

        • label_lookup_path: string UID to integer node ID.
        • uid_lookup_path: string UID to human-readable string.
          + image: Image file name.

        Returns:

        • dict from integer node ID to human-readable string.
          + Nothing
          """
        • if not tf.gfile.Exists(uid_lookup_path):
        • tf.logging.fatal('File does not exist %s', uid_lookup_path)
        • if not tf.gfile.Exists(label_lookup_path):
        • tf.logging.fatal('File does not exist %s', label_lookup_path)
          -
        • # Loads mapping from string UID to human-readable string
        • proto_as_ascii_lines = tf.gfile.GFile(uid_lookup_path).readlines()
        • uid_to_human = {}
        • p = re.compile(r'[n\d][ \S,]')
        • for line in proto_as_ascii_lines:
        • parsed_items = p.findall(line)
        • uid = parsed_items[0]
        • human_string = parsed_items[2]
        • uid_to_human[uid] = human_string
          -
        • # Loads mapping from string UID to integer node ID.
        • node_id_to_uid = {}
        • proto_as_ascii = tf.gfile.GFile(label_lookup_path).readlines()
        • for line in proto_as_ascii:
        • if line.startswith(' target_class:'):
        • target_class = int(line.split(': ')[1])
        • if line.startswith(' target_class_string:'):
        • target_class_string = line.split(': ')[1]
        • node_id_to_uid[target_class] = target_class_string[1:-2]
          -
        • # Loads the final mapping of integer node ID to human-readable string
        • node_id_to_name = {}
        • for key, val in node_id_to_uid.items():
        • if val not in uid_to_human:
        • tf.logging.fatal('Failed to locate: %s', val)
        • name = uid_to_human[val]
        • node_id_to_name[key] = name
          -
        • return node_id_to_name
          -
        • def id_to_string(self, node_id):
        • if node_id not in self.node_lookup:
        • return ''
        • return self.node_lookup[node_id]
          -
          -
          -def create_graph():
        • """Creates a graph from saved GraphDef file and returns a saver."""
        • # Creates graph from saved graph_def.pb.
        • with tf.gfile.FastGFile(os.path.join(
        • FLAGS.model_dir, 'classify_image_graph_def.pb'), 'rb') as f:
        • graph_def = tf.GraphDef()
        • graph_def.ParseFromString(f.read())
        • _ = tf.import_graph_def(graph_def, name='')
          + dest_directory = FLAGS.model_dir

        + image_size = inception.inception_v4.default_image_size

        -def run_inference_on_image(image):

        • """Runs inference on an image.
          -
        • Args:
        • image: Image file name.
          -
        • Returns:
        • Nothing
        • """
        • if not tf.gfile.Exists(image):
        • tf.logging.fatal('File does not exist %s', image)
        • image_data = tf.gfile.FastGFile(image, 'rb').read()
          -
        • # Creates graph from saved GraphDef.
        • create_graph()
          -
        • with tf.Session() as sess:
        • # Some useful tensors:
        • # 'softmax:0': A tensor containing the normalized prediction across
        • # 1000 labels.
        • # 'pool_3:0': A tensor containing the next-to-last layer containing 2048
        • # float description of the image.
        • # 'DecodeJpeg/contents:0': A tensor containing a string providing JPEG
        • # encoding of the image.
        • # Runs the softmax tensor by feeding the image_data as input to the graph.
        • softmax_tensor = sess.graph.get_tensor_by_name('softmax:0')
        • predictions = sess.run(softmax_tensor,
        • {'DecodeJpeg/contents:0': image_data}

          )

        • predictions = np.squeeze(predictions)
          -
        • # Creates node ID --> English string lookup.
        • node_lookup = NodeLookup()
          -
        • top_k = predictions.argsort()[-FLAGS.num_top_predictions:][::-1]
        • for node_id in top_k:
        • human_string = node_lookup.id_to_string(node_id)
        • score = predictions[node_id]
        • print('%s (score = %.5f)' % (human_string, score))
          + if not tf.gfile.Exists(image):
          + tf.logging.fatal('File does not exist %s', image)
          + image_string = tf.gfile.FastGFile(image, 'rb').read()

        + with tf.Graph().as_default():
        + image = tf.image.decode_jpeg(image_string, channels=3)
        + processed_image = inception_preprocessing.preprocess_image(
        + image, image_size, image_size, is_training=False)
        + processed_images = tf.expand_dims(processed_image, 0)
        +
        + # Create the model, use the default arg scope to configure the batch
        + # norm parameters.
        + with slim.arg_scope(inception.inception_v4_arg_scope()):
        + logits, _ = inception.inception_v4(
        + processed_images, num_classes=1001, is_training=False)
        + probabilities = tf.nn.softmax(logits)
        +
        + init_fn = slim.assign_from_checkpoint_fn(
        + os.path.join(dest_directory, 'inception_v4.ckpt'),
        + slim.get_model_variables('InceptionV4'))
        +
        + with tf.Session() as sess:
        + init_fn(sess)
        + _, probabilities = sess.run([image, probabilities])

        Review comment:
        Nope, it doesn't affect classify_image.py. Because in CLI there is no way to keep the model in memory for subsequent parse calls, we are fine with that. CLI is meant for testing few images and not meant for production.

        In REST API we can keep the model in memory for subsequent parse calls

        ----------------------------------------------------------------
        This is an automated message from the Apache Git Service.
        To respond to the message, please log on GitHub and use the
        URL above to go to the specific comment.

        For queries about this service, please contact Infrastructure at:
        users@infra.apache.org

        Show
        githubbot ASF GitHub Bot added a comment - thammegowda commented on a change in pull request #163: TIKA-2306 : Update Inception v3 to Inception v4 in Object recognition parser URL: https://github.com/apache/tika/pull/163#discussion_r111530205 ########## File path: tika-parsers/src/main/resources/org/apache/tika/parser/recognition/tf/classify_image.py ########## @@ -44,169 +49,205 @@ from six.moves import urllib import tensorflow as tf +from datasets import imagenet, dataset_utils +from nets import inception +from preprocessing import inception_preprocessing + +slim = tf.contrib.slim + FLAGS = tf.app.flags.FLAGS classify_image_graph_def.pb: Binary representation of the GraphDef protocol buffer. imagenet_synset_to_human_label_map.txt: +# inception_v4.ckpt +# Inception V4 checkpoint file. +# imagenet_metadata.txt Map from synset ID to a human readable string. imagenet_2012_challenge_label_map_proto.pbtxt: +# imagenet_lsvrc_2015_synsets.txt Text representation of a protocol buffer mapping a label to synset ID. tf.app.flags.DEFINE_string( 'model_dir', '/tmp/imagenet', """Path to classify_image_graph_def.pb, """ """imagenet_synset_to_human_label_map.txt, and """ """imagenet_2012_challenge_label_map_proto.pbtxt.""") + """Path to inception_v4.ckpt, """ + """imagenet_lsvrc_2015_synsets.txt, and """ + """imagenet_metadata.txt.""") tf.app.flags.DEFINE_string('image_file', '', """Absolute path to image file.""") tf.app.flags.DEFINE_integer('num_top_predictions', 5, """Display this many predictions.""") pylint: disable=line-too-long -DATA_URL = 'http://download.tensorflow.org/models/image/imagenet/inception-2015-12-05.tgz' +DATA_URL = 'http://download.tensorflow.org/models/inception_v4_2016_09_09.tar.gz' pylint: enable=line-too-long -class NodeLookup(object): """Converts integer node ID's to human readable labels.""" +def create_readable_names_for_imagenet_labels(): + """Create a dict mapping label id to human readable string. + + Returns: + labels_to_names: dictionary where keys are integers from to 1000 + and values are human-readable names. + + We retrieve a synset file, which contains a list of valid synset labels used + by ILSVRC competition. There is one synset one per line, eg. + # n01440764 + # n01443537 + We also retrieve a synset_to_human_file, which contains a mapping from synsets + to human-readable names for every synset in Imagenet. These are stored in a + tsv format, as follows: + # n02119247 black fox + # n02119359 silver fox + We assign each synset (in alphabetical order) an integer, starting from 1 + (since 0 is reserved for the background class). + + Code is based on + https://github.com/tensorflow/models/blob/master/inception/inception/data/build_imagenet_data.py#L463 + """ + + # pylint: disable=line-too-long + + dest_directory = FLAGS.model_dir + + synset_list = [s.strip() for s in open(os.path.join( + dest_directory, 'imagenet_lsvrc_2015_synsets.txt')).readlines()] + num_synsets_in_ilsvrc = len(synset_list) + assert num_synsets_in_ilsvrc == 1000 def _ init _(self, label_lookup_path=None, uid_lookup_path=None): if not label_lookup_path: label_lookup_path = os.path.join( FLAGS.model_dir, 'imagenet_2012_challenge_label_map_proto.pbtxt') if not uid_lookup_path: uid_lookup_path = os.path.join( FLAGS.model_dir, 'imagenet_synset_to_human_label_map.txt') self.node_lookup = self.load(label_lookup_path, uid_lookup_path) + synset_to_human_list = open(os.path.join( + dest_directory, 'imagenet_metadata.txt')).readlines() + num_synsets_in_all_imagenet = len(synset_to_human_list) + assert num_synsets_in_all_imagenet == 21842 def load(self, label_lookup_path, uid_lookup_path): """Loads a human readable English name for each softmax node. + synset_to_human = {} + for s in synset_to_human_list: + parts = s.strip().split('\t') + assert len(parts) == 2 + synset = parts [0] + human = parts [1] + synset_to_human [synset] = human + + label_index = 1 + labels_to_names = {0: 'background'} + for synset in synset_list: + name = synset_to_human [synset] + labels_to_names [label_index] = name + label_index += 1 + + return labels_to_names + + +def run_inference_on_image(image): + """Runs inference on an image. Args: label_lookup_path: string UID to integer node ID. uid_lookup_path: string UID to human-readable string. + image: Image file name. Returns: dict from integer node ID to human-readable string. + Nothing """ if not tf.gfile.Exists(uid_lookup_path): tf.logging.fatal('File does not exist %s', uid_lookup_path) if not tf.gfile.Exists(label_lookup_path): tf.logging.fatal('File does not exist %s', label_lookup_path) - # Loads mapping from string UID to human-readable string proto_as_ascii_lines = tf.gfile.GFile(uid_lookup_path).readlines() uid_to_human = {} p = re.compile(r' [n\d] [ \S,] ') for line in proto_as_ascii_lines: parsed_items = p.findall(line) uid = parsed_items [0] human_string = parsed_items [2] uid_to_human [uid] = human_string - # Loads mapping from string UID to integer node ID. node_id_to_uid = {} proto_as_ascii = tf.gfile.GFile(label_lookup_path).readlines() for line in proto_as_ascii: if line.startswith(' target_class:'): target_class = int(line.split(': ') [1] ) if line.startswith(' target_class_string:'): target_class_string = line.split(': ') [1] node_id_to_uid [target_class] = target_class_string [1:-2] - # Loads the final mapping of integer node ID to human-readable string node_id_to_name = {} for key, val in node_id_to_uid.items(): if val not in uid_to_human: tf.logging.fatal('Failed to locate: %s', val) name = uid_to_human [val] node_id_to_name [key] = name - return node_id_to_name - def id_to_string(self, node_id): if node_id not in self.node_lookup: return '' return self.node_lookup [node_id] - - -def create_graph(): """Creates a graph from saved GraphDef file and returns a saver.""" # Creates graph from saved graph_def.pb. with tf.gfile.FastGFile(os.path.join( FLAGS.model_dir, 'classify_image_graph_def.pb'), 'rb') as f: graph_def = tf.GraphDef() graph_def.ParseFromString(f.read()) _ = tf.import_graph_def(graph_def, name='') + dest_directory = FLAGS.model_dir + image_size = inception.inception_v4.default_image_size -def run_inference_on_image(image): """Runs inference on an image. - Args: image: Image file name. - Returns: Nothing """ if not tf.gfile.Exists(image): tf.logging.fatal('File does not exist %s', image) image_data = tf.gfile.FastGFile(image, 'rb').read() - # Creates graph from saved GraphDef. create_graph() - with tf.Session() as sess: # Some useful tensors: # 'softmax:0': A tensor containing the normalized prediction across # 1000 labels. # 'pool_3:0': A tensor containing the next-to-last layer containing 2048 # float description of the image. # 'DecodeJpeg/contents:0': A tensor containing a string providing JPEG # encoding of the image. # Runs the softmax tensor by feeding the image_data as input to the graph. softmax_tensor = sess.graph.get_tensor_by_name('softmax:0') predictions = sess.run(softmax_tensor, {'DecodeJpeg/contents:0': image_data} ) predictions = np.squeeze(predictions) - # Creates node ID --> English string lookup. node_lookup = NodeLookup() - top_k = predictions.argsort() [-FLAGS.num_top_predictions:] [::-1] for node_id in top_k: human_string = node_lookup.id_to_string(node_id) score = predictions [node_id] print('%s (score = %.5f)' % (human_string, score)) + if not tf.gfile.Exists(image): + tf.logging.fatal('File does not exist %s', image) + image_string = tf.gfile.FastGFile(image, 'rb').read() + with tf.Graph().as_default(): + image = tf.image.decode_jpeg(image_string, channels=3) + processed_image = inception_preprocessing.preprocess_image( + image, image_size, image_size, is_training=False) + processed_images = tf.expand_dims(processed_image, 0) + + # Create the model, use the default arg scope to configure the batch + # norm parameters. + with slim.arg_scope(inception.inception_v4_arg_scope()): + logits, _ = inception.inception_v4( + processed_images, num_classes=1001, is_training=False) + probabilities = tf.nn.softmax(logits) + + init_fn = slim.assign_from_checkpoint_fn( + os.path.join(dest_directory, 'inception_v4.ckpt'), + slim.get_model_variables('InceptionV4')) + + with tf.Session() as sess: + init_fn(sess) + _, probabilities = sess.run( [image, probabilities] ) Review comment: Nope, it doesn't affect classify_image.py. Because in CLI there is no way to keep the model in memory for subsequent parse calls, we are fine with that. CLI is meant for testing few images and not meant for production. In REST API we can keep the model in memory for subsequent parse calls ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org
        Hide
        githubbot ASF GitHub Bot added a comment -

        KranthiGV commented on a change in pull request #163: TIKA-2306: Update Inception v3 to Inception v4 in Object recognition parser
        URL: https://github.com/apache/tika/pull/163#discussion_r111530004

        ##########
        File path: tika-parsers/src/main/resources/org/apache/tika/parser/recognition/tf/classify_image.py
        ##########
        @@ -44,169 +49,205 @@
        from six.moves import urllib
        import tensorflow as tf

        +from datasets import imagenet, dataset_utils
        +from nets import inception
        +from preprocessing import inception_preprocessing
        +
        +slim = tf.contrib.slim
        +
        FLAGS = tf.app.flags.FLAGS

          1. classify_image_graph_def.pb:
          2. Binary representation of the GraphDef protocol buffer.
          3. imagenet_synset_to_human_label_map.txt:
            +# inception_v4.ckpt
            +# Inception V4 checkpoint file.
            +# imagenet_metadata.txt
        1. Map from synset ID to a human readable string.
          1. imagenet_2012_challenge_label_map_proto.pbtxt:
            +# imagenet_lsvrc_2015_synsets.txt
        2. Text representation of a protocol buffer mapping a label to synset ID.
          tf.app.flags.DEFINE_string(
          'model_dir', '/tmp/imagenet',
        • """Path to classify_image_graph_def.pb, """
        • """imagenet_synset_to_human_label_map.txt, and """
        • """imagenet_2012_challenge_label_map_proto.pbtxt.""")
          + """Path to inception_v4.ckpt, """
          + """imagenet_lsvrc_2015_synsets.txt, and """
          + """imagenet_metadata.txt.""")
          tf.app.flags.DEFINE_string('image_file', '',
          """Absolute path to image file.""")
          tf.app.flags.DEFINE_integer('num_top_predictions', 5,
          """Display this many predictions.""")
        1. pylint: disable=line-too-long
          -DATA_URL = 'http://download.tensorflow.org/models/image/imagenet/inception-2015-12-05.tgz'
          +DATA_URL = 'http://download.tensorflow.org/models/inception_v4_2016_09_09.tar.gz'
        2. pylint: enable=line-too-long

        -class NodeLookup(object):

        • """Converts integer node ID's to human readable labels."""
          +def create_readable_names_for_imagenet_labels():
          + """Create a dict mapping label id to human readable string.
          +
          + Returns:
          + labels_to_names: dictionary where keys are integers from to 1000
          + and values are human-readable names.
          +
          + We retrieve a synset file, which contains a list of valid synset labels used
          + by ILSVRC competition. There is one synset one per line, eg.
          + # n01440764
          + # n01443537
          + We also retrieve a synset_to_human_file, which contains a mapping from synsets
          + to human-readable names for every synset in Imagenet. These are stored in a
          + tsv format, as follows:
          + # n02119247 black fox
          + # n02119359 silver fox
          + We assign each synset (in alphabetical order) an integer, starting from 1
          + (since 0 is reserved for the background class).
          +
          + Code is based on
          + https://github.com/tensorflow/models/blob/master/inception/inception/data/build_imagenet_data.py#L463
          + """
          +
          + # pylint: disable=line-too-long
          +
          + dest_directory = FLAGS.model_dir
          +
          + synset_list = [s.strip() for s in open(os.path.join(
          + dest_directory, 'imagenet_lsvrc_2015_synsets.txt')).readlines()]
          + num_synsets_in_ilsvrc = len(synset_list)
          + assert num_synsets_in_ilsvrc == 1000
        • def _init_(self,
        • label_lookup_path=None,
        • uid_lookup_path=None):
        • if not label_lookup_path:
        • label_lookup_path = os.path.join(
        • FLAGS.model_dir, 'imagenet_2012_challenge_label_map_proto.pbtxt')
        • if not uid_lookup_path:
        • uid_lookup_path = os.path.join(
        • FLAGS.model_dir, 'imagenet_synset_to_human_label_map.txt')
        • self.node_lookup = self.load(label_lookup_path, uid_lookup_path)
          + synset_to_human_list = open(os.path.join(
          + dest_directory, 'imagenet_metadata.txt')).readlines()
          + num_synsets_in_all_imagenet = len(synset_to_human_list)
          + assert num_synsets_in_all_imagenet == 21842
        • def load(self, label_lookup_path, uid_lookup_path):
        • """Loads a human readable English name for each softmax node.
          + synset_to_human = {}
          + for s in synset_to_human_list:
          + parts = s.strip().split('\t')
          + assert len(parts) == 2
          + synset = parts[0]
          + human = parts[1]
          + synset_to_human[synset] = human
          +
          + label_index = 1
          + labels_to_names = {0: 'background'}

          + for synset in synset_list:
          + name = synset_to_human[synset]
          + labels_to_names[label_index] = name
          + label_index += 1
          +
          + return labels_to_names
          +
          +
          +def run_inference_on_image(image):
          + """Runs inference on an image.

        Args:

        • label_lookup_path: string UID to integer node ID.
        • uid_lookup_path: string UID to human-readable string.
          + image: Image file name.

        Returns:

        • dict from integer node ID to human-readable string.
          + Nothing
          """
        • if not tf.gfile.Exists(uid_lookup_path):
        • tf.logging.fatal('File does not exist %s', uid_lookup_path)
        • if not tf.gfile.Exists(label_lookup_path):
        • tf.logging.fatal('File does not exist %s', label_lookup_path)
          -
        • # Loads mapping from string UID to human-readable string
        • proto_as_ascii_lines = tf.gfile.GFile(uid_lookup_path).readlines()
        • uid_to_human = {}
        • p = re.compile(r'[n\d][ \S,]')
        • for line in proto_as_ascii_lines:
        • parsed_items = p.findall(line)
        • uid = parsed_items[0]
        • human_string = parsed_items[2]
        • uid_to_human[uid] = human_string
          -
        • # Loads mapping from string UID to integer node ID.
        • node_id_to_uid = {}
        • proto_as_ascii = tf.gfile.GFile(label_lookup_path).readlines()
        • for line in proto_as_ascii:
        • if line.startswith(' target_class:'):
        • target_class = int(line.split(': ')[1])
        • if line.startswith(' target_class_string:'):
        • target_class_string = line.split(': ')[1]
        • node_id_to_uid[target_class] = target_class_string[1:-2]
          -
        • # Loads the final mapping of integer node ID to human-readable string
        • node_id_to_name = {}
        • for key, val in node_id_to_uid.items():
        • if val not in uid_to_human:
        • tf.logging.fatal('Failed to locate: %s', val)
        • name = uid_to_human[val]
        • node_id_to_name[key] = name
          -
        • return node_id_to_name
          -
        • def id_to_string(self, node_id):
        • if node_id not in self.node_lookup:
        • return ''
        • return self.node_lookup[node_id]
          -
          -
          -def create_graph():
        • """Creates a graph from saved GraphDef file and returns a saver."""
        • # Creates graph from saved graph_def.pb.
        • with tf.gfile.FastGFile(os.path.join(
        • FLAGS.model_dir, 'classify_image_graph_def.pb'), 'rb') as f:
        • graph_def = tf.GraphDef()
        • graph_def.ParseFromString(f.read())
        • _ = tf.import_graph_def(graph_def, name='')
          + dest_directory = FLAGS.model_dir

        + image_size = inception.inception_v4.default_image_size

        -def run_inference_on_image(image):

        • """Runs inference on an image.
          -
        • Args:
        • image: Image file name.
          -
        • Returns:
        • Nothing
        • """
        • if not tf.gfile.Exists(image):
        • tf.logging.fatal('File does not exist %s', image)
        • image_data = tf.gfile.FastGFile(image, 'rb').read()
          -
        • # Creates graph from saved GraphDef.
        • create_graph()
          -
        • with tf.Session() as sess:
        • # Some useful tensors:
        • # 'softmax:0': A tensor containing the normalized prediction across
        • # 1000 labels.
        • # 'pool_3:0': A tensor containing the next-to-last layer containing 2048
        • # float description of the image.
        • # 'DecodeJpeg/contents:0': A tensor containing a string providing JPEG
        • # encoding of the image.
        • # Runs the softmax tensor by feeding the image_data as input to the graph.
        • softmax_tensor = sess.graph.get_tensor_by_name('softmax:0')
        • predictions = sess.run(softmax_tensor,
        • {'DecodeJpeg/contents:0': image_data}

          )

        • predictions = np.squeeze(predictions)
          -
        • # Creates node ID --> English string lookup.
        • node_lookup = NodeLookup()
          -
        • top_k = predictions.argsort()[-FLAGS.num_top_predictions:][::-1]
        • for node_id in top_k:
        • human_string = node_lookup.id_to_string(node_id)
        • score = predictions[node_id]
        • print('%s (score = %.5f)' % (human_string, score))
          + if not tf.gfile.Exists(image):
          + tf.logging.fatal('File does not exist %s', image)
          + image_string = tf.gfile.FastGFile(image, 'rb').read()

        + with tf.Graph().as_default():
        + image = tf.image.decode_jpeg(image_string, channels=3)
        + processed_image = inception_preprocessing.preprocess_image(
        + image, image_size, image_size, is_training=False)
        + processed_images = tf.expand_dims(processed_image, 0)
        +
        + # Create the model, use the default arg scope to configure the batch
        + # norm parameters.
        + with slim.arg_scope(inception.inception_v4_arg_scope()):
        + logits, _ = inception.inception_v4(
        + processed_images, num_classes=1001, is_training=False)
        + probabilities = tf.nn.softmax(logits)
        +
        + init_fn = slim.assign_from_checkpoint_fn(
        + os.path.join(dest_directory, 'inception_v4.ckpt'),
        + slim.get_model_variables('InceptionV4'))
        +
        + with tf.Session() as sess:
        + init_fn(sess)
        + _, probabilities = sess.run([image, probabilities])

        Review comment:
        @thammegowda

        ----------------------------------------------------------------
        This is an automated message from the Apache Git Service.
        To respond to the message, please log on GitHub and use the
        URL above to go to the specific comment.

        For queries about this service, please contact Infrastructure at:
        users@infra.apache.org

        Show
        githubbot ASF GitHub Bot added a comment - KranthiGV commented on a change in pull request #163: TIKA-2306 : Update Inception v3 to Inception v4 in Object recognition parser URL: https://github.com/apache/tika/pull/163#discussion_r111530004 ########## File path: tika-parsers/src/main/resources/org/apache/tika/parser/recognition/tf/classify_image.py ########## @@ -44,169 +49,205 @@ from six.moves import urllib import tensorflow as tf +from datasets import imagenet, dataset_utils +from nets import inception +from preprocessing import inception_preprocessing + +slim = tf.contrib.slim + FLAGS = tf.app.flags.FLAGS classify_image_graph_def.pb: Binary representation of the GraphDef protocol buffer. imagenet_synset_to_human_label_map.txt: +# inception_v4.ckpt +# Inception V4 checkpoint file. +# imagenet_metadata.txt Map from synset ID to a human readable string. imagenet_2012_challenge_label_map_proto.pbtxt: +# imagenet_lsvrc_2015_synsets.txt Text representation of a protocol buffer mapping a label to synset ID. tf.app.flags.DEFINE_string( 'model_dir', '/tmp/imagenet', """Path to classify_image_graph_def.pb, """ """imagenet_synset_to_human_label_map.txt, and """ """imagenet_2012_challenge_label_map_proto.pbtxt.""") + """Path to inception_v4.ckpt, """ + """imagenet_lsvrc_2015_synsets.txt, and """ + """imagenet_metadata.txt.""") tf.app.flags.DEFINE_string('image_file', '', """Absolute path to image file.""") tf.app.flags.DEFINE_integer('num_top_predictions', 5, """Display this many predictions.""") pylint: disable=line-too-long -DATA_URL = 'http://download.tensorflow.org/models/image/imagenet/inception-2015-12-05.tgz' +DATA_URL = 'http://download.tensorflow.org/models/inception_v4_2016_09_09.tar.gz' pylint: enable=line-too-long -class NodeLookup(object): """Converts integer node ID's to human readable labels.""" +def create_readable_names_for_imagenet_labels(): + """Create a dict mapping label id to human readable string. + + Returns: + labels_to_names: dictionary where keys are integers from to 1000 + and values are human-readable names. + + We retrieve a synset file, which contains a list of valid synset labels used + by ILSVRC competition. There is one synset one per line, eg. + # n01440764 + # n01443537 + We also retrieve a synset_to_human_file, which contains a mapping from synsets + to human-readable names for every synset in Imagenet. These are stored in a + tsv format, as follows: + # n02119247 black fox + # n02119359 silver fox + We assign each synset (in alphabetical order) an integer, starting from 1 + (since 0 is reserved for the background class). + + Code is based on + https://github.com/tensorflow/models/blob/master/inception/inception/data/build_imagenet_data.py#L463 + """ + + # pylint: disable=line-too-long + + dest_directory = FLAGS.model_dir + + synset_list = [s.strip() for s in open(os.path.join( + dest_directory, 'imagenet_lsvrc_2015_synsets.txt')).readlines()] + num_synsets_in_ilsvrc = len(synset_list) + assert num_synsets_in_ilsvrc == 1000 def _ init _(self, label_lookup_path=None, uid_lookup_path=None): if not label_lookup_path: label_lookup_path = os.path.join( FLAGS.model_dir, 'imagenet_2012_challenge_label_map_proto.pbtxt') if not uid_lookup_path: uid_lookup_path = os.path.join( FLAGS.model_dir, 'imagenet_synset_to_human_label_map.txt') self.node_lookup = self.load(label_lookup_path, uid_lookup_path) + synset_to_human_list = open(os.path.join( + dest_directory, 'imagenet_metadata.txt')).readlines() + num_synsets_in_all_imagenet = len(synset_to_human_list) + assert num_synsets_in_all_imagenet == 21842 def load(self, label_lookup_path, uid_lookup_path): """Loads a human readable English name for each softmax node. + synset_to_human = {} + for s in synset_to_human_list: + parts = s.strip().split('\t') + assert len(parts) == 2 + synset = parts [0] + human = parts [1] + synset_to_human [synset] = human + + label_index = 1 + labels_to_names = {0: 'background'} + for synset in synset_list: + name = synset_to_human [synset] + labels_to_names [label_index] = name + label_index += 1 + + return labels_to_names + + +def run_inference_on_image(image): + """Runs inference on an image. Args: label_lookup_path: string UID to integer node ID. uid_lookup_path: string UID to human-readable string. + image: Image file name. Returns: dict from integer node ID to human-readable string. + Nothing """ if not tf.gfile.Exists(uid_lookup_path): tf.logging.fatal('File does not exist %s', uid_lookup_path) if not tf.gfile.Exists(label_lookup_path): tf.logging.fatal('File does not exist %s', label_lookup_path) - # Loads mapping from string UID to human-readable string proto_as_ascii_lines = tf.gfile.GFile(uid_lookup_path).readlines() uid_to_human = {} p = re.compile(r' [n\d] [ \S,] ') for line in proto_as_ascii_lines: parsed_items = p.findall(line) uid = parsed_items [0] human_string = parsed_items [2] uid_to_human [uid] = human_string - # Loads mapping from string UID to integer node ID. node_id_to_uid = {} proto_as_ascii = tf.gfile.GFile(label_lookup_path).readlines() for line in proto_as_ascii: if line.startswith(' target_class:'): target_class = int(line.split(': ') [1] ) if line.startswith(' target_class_string:'): target_class_string = line.split(': ') [1] node_id_to_uid [target_class] = target_class_string [1:-2] - # Loads the final mapping of integer node ID to human-readable string node_id_to_name = {} for key, val in node_id_to_uid.items(): if val not in uid_to_human: tf.logging.fatal('Failed to locate: %s', val) name = uid_to_human [val] node_id_to_name [key] = name - return node_id_to_name - def id_to_string(self, node_id): if node_id not in self.node_lookup: return '' return self.node_lookup [node_id] - - -def create_graph(): """Creates a graph from saved GraphDef file and returns a saver.""" # Creates graph from saved graph_def.pb. with tf.gfile.FastGFile(os.path.join( FLAGS.model_dir, 'classify_image_graph_def.pb'), 'rb') as f: graph_def = tf.GraphDef() graph_def.ParseFromString(f.read()) _ = tf.import_graph_def(graph_def, name='') + dest_directory = FLAGS.model_dir + image_size = inception.inception_v4.default_image_size -def run_inference_on_image(image): """Runs inference on an image. - Args: image: Image file name. - Returns: Nothing """ if not tf.gfile.Exists(image): tf.logging.fatal('File does not exist %s', image) image_data = tf.gfile.FastGFile(image, 'rb').read() - # Creates graph from saved GraphDef. create_graph() - with tf.Session() as sess: # Some useful tensors: # 'softmax:0': A tensor containing the normalized prediction across # 1000 labels. # 'pool_3:0': A tensor containing the next-to-last layer containing 2048 # float description of the image. # 'DecodeJpeg/contents:0': A tensor containing a string providing JPEG # encoding of the image. # Runs the softmax tensor by feeding the image_data as input to the graph. softmax_tensor = sess.graph.get_tensor_by_name('softmax:0') predictions = sess.run(softmax_tensor, {'DecodeJpeg/contents:0': image_data} ) predictions = np.squeeze(predictions) - # Creates node ID --> English string lookup. node_lookup = NodeLookup() - top_k = predictions.argsort() [-FLAGS.num_top_predictions:] [::-1] for node_id in top_k: human_string = node_lookup.id_to_string(node_id) score = predictions [node_id] print('%s (score = %.5f)' % (human_string, score)) + if not tf.gfile.Exists(image): + tf.logging.fatal('File does not exist %s', image) + image_string = tf.gfile.FastGFile(image, 'rb').read() + with tf.Graph().as_default(): + image = tf.image.decode_jpeg(image_string, channels=3) + processed_image = inception_preprocessing.preprocess_image( + image, image_size, image_size, is_training=False) + processed_images = tf.expand_dims(processed_image, 0) + + # Create the model, use the default arg scope to configure the batch + # norm parameters. + with slim.arg_scope(inception.inception_v4_arg_scope()): + logits, _ = inception.inception_v4( + processed_images, num_classes=1001, is_training=False) + probabilities = tf.nn.softmax(logits) + + init_fn = slim.assign_from_checkpoint_fn( + os.path.join(dest_directory, 'inception_v4.ckpt'), + slim.get_model_variables('InceptionV4')) + + with tf.Session() as sess: + init_fn(sess) + _, probabilities = sess.run( [image, probabilities] ) Review comment: @thammegowda ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org
        Hide
        githubbot ASF GitHub Bot added a comment -

        thammegowda commented on issue #163: TIKA-2306: Update Inception v3 to Inception v4 in Object recognition parser
        URL: https://github.com/apache/tika/pull/163#issuecomment-294091086

        @KranthiGV Also please pull changes on master branch and merge it with this branch.

        ----------------------------------------------------------------
        This is an automated message from the Apache Git Service.
        To respond to the message, please log on GitHub and use the
        URL above to go to the specific comment.

        For queries about this service, please contact Infrastructure at:
        users@infra.apache.org

        Show
        githubbot ASF GitHub Bot added a comment - thammegowda commented on issue #163: TIKA-2306 : Update Inception v3 to Inception v4 in Object recognition parser URL: https://github.com/apache/tika/pull/163#issuecomment-294091086 @KranthiGV Also please pull changes on master branch and merge it with this branch. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org
        Hide
        githubbot ASF GitHub Bot added a comment -

        KranthiGV commented on a change in pull request #163: TIKA-2306: Update Inception v3 to Inception v4 in Object recognition parser
        URL: https://github.com/apache/tika/pull/163#discussion_r111529725

        ##########
        File path: tika-parsers/src/main/resources/org/apache/tika/parser/recognition/tf/classify_image.py
        ##########
        @@ -44,169 +49,205 @@
        from six.moves import urllib
        import tensorflow as tf

        +from datasets import imagenet, dataset_utils
        +from nets import inception
        +from preprocessing import inception_preprocessing
        +
        +slim = tf.contrib.slim
        +
        FLAGS = tf.app.flags.FLAGS

          1. classify_image_graph_def.pb:
          2. Binary representation of the GraphDef protocol buffer.
          3. imagenet_synset_to_human_label_map.txt:
            +# inception_v4.ckpt
            +# Inception V4 checkpoint file.
            +# imagenet_metadata.txt
        1. Map from synset ID to a human readable string.
          1. imagenet_2012_challenge_label_map_proto.pbtxt:
            +# imagenet_lsvrc_2015_synsets.txt
        2. Text representation of a protocol buffer mapping a label to synset ID.
          tf.app.flags.DEFINE_string(
          'model_dir', '/tmp/imagenet',
        • """Path to classify_image_graph_def.pb, """
        • """imagenet_synset_to_human_label_map.txt, and """
        • """imagenet_2012_challenge_label_map_proto.pbtxt.""")
          + """Path to inception_v4.ckpt, """
          + """imagenet_lsvrc_2015_synsets.txt, and """
          + """imagenet_metadata.txt.""")
          tf.app.flags.DEFINE_string('image_file', '',
          """Absolute path to image file.""")
          tf.app.flags.DEFINE_integer('num_top_predictions', 5,
          """Display this many predictions.""")
        1. pylint: disable=line-too-long
          -DATA_URL = 'http://download.tensorflow.org/models/image/imagenet/inception-2015-12-05.tgz'
          +DATA_URL = 'http://download.tensorflow.org/models/inception_v4_2016_09_09.tar.gz'
        2. pylint: enable=line-too-long

        -class NodeLookup(object):

        • """Converts integer node ID's to human readable labels."""
          +def create_readable_names_for_imagenet_labels():
          + """Create a dict mapping label id to human readable string.
          +
          + Returns:
          + labels_to_names: dictionary where keys are integers from to 1000
          + and values are human-readable names.
          +
          + We retrieve a synset file, which contains a list of valid synset labels used
          + by ILSVRC competition. There is one synset one per line, eg.
          + # n01440764
          + # n01443537
          + We also retrieve a synset_to_human_file, which contains a mapping from synsets
          + to human-readable names for every synset in Imagenet. These are stored in a
          + tsv format, as follows:
          + # n02119247 black fox
          + # n02119359 silver fox
          + We assign each synset (in alphabetical order) an integer, starting from 1
          + (since 0 is reserved for the background class).
          +
          + Code is based on
          + https://github.com/tensorflow/models/blob/master/inception/inception/data/build_imagenet_data.py#L463
          + """
          +
          + # pylint: disable=line-too-long
          +
          + dest_directory = FLAGS.model_dir
          +
          + synset_list = [s.strip() for s in open(os.path.join(
          + dest_directory, 'imagenet_lsvrc_2015_synsets.txt')).readlines()]
          + num_synsets_in_ilsvrc = len(synset_list)
          + assert num_synsets_in_ilsvrc == 1000
        • def _init_(self,
        • label_lookup_path=None,
        • uid_lookup_path=None):
        • if not label_lookup_path:
        • label_lookup_path = os.path.join(
        • FLAGS.model_dir, 'imagenet_2012_challenge_label_map_proto.pbtxt')
        • if not uid_lookup_path:
        • uid_lookup_path = os.path.join(
        • FLAGS.model_dir, 'imagenet_synset_to_human_label_map.txt')
        • self.node_lookup = self.load(label_lookup_path, uid_lookup_path)
          + synset_to_human_list = open(os.path.join(
          + dest_directory, 'imagenet_metadata.txt')).readlines()
          + num_synsets_in_all_imagenet = len(synset_to_human_list)
          + assert num_synsets_in_all_imagenet == 21842
        • def load(self, label_lookup_path, uid_lookup_path):
        • """Loads a human readable English name for each softmax node.
          + synset_to_human = {}
          + for s in synset_to_human_list:
          + parts = s.strip().split('\t')
          + assert len(parts) == 2
          + synset = parts[0]
          + human = parts[1]
          + synset_to_human[synset] = human
          +
          + label_index = 1
          + labels_to_names = {0: 'background'}

          + for synset in synset_list:
          + name = synset_to_human[synset]
          + labels_to_names[label_index] = name
          + label_index += 1
          +
          + return labels_to_names
          +
          +
          +def run_inference_on_image(image):
          + """Runs inference on an image.

        Args:

        • label_lookup_path: string UID to integer node ID.
        • uid_lookup_path: string UID to human-readable string.
          + image: Image file name.

        Returns:

        • dict from integer node ID to human-readable string.
          + Nothing
          """
        • if not tf.gfile.Exists(uid_lookup_path):
        • tf.logging.fatal('File does not exist %s', uid_lookup_path)
        • if not tf.gfile.Exists(label_lookup_path):
        • tf.logging.fatal('File does not exist %s', label_lookup_path)
          -
        • # Loads mapping from string UID to human-readable string
        • proto_as_ascii_lines = tf.gfile.GFile(uid_lookup_path).readlines()
        • uid_to_human = {}
        • p = re.compile(r'[n\d][ \S,]')
        • for line in proto_as_ascii_lines:
        • parsed_items = p.findall(line)
        • uid = parsed_items[0]
        • human_string = parsed_items[2]
        • uid_to_human[uid] = human_string
          -
        • # Loads mapping from string UID to integer node ID.
        • node_id_to_uid = {}
        • proto_as_ascii = tf.gfile.GFile(label_lookup_path).readlines()
        • for line in proto_as_ascii:
        • if line.startswith(' target_class:'):
        • target_class = int(line.split(': ')[1])
        • if line.startswith(' target_class_string:'):
        • target_class_string = line.split(': ')[1]
        • node_id_to_uid[target_class] = target_class_string[1:-2]
          -
        • # Loads the final mapping of integer node ID to human-readable string
        • node_id_to_name = {}
        • for key, val in node_id_to_uid.items():
        • if val not in uid_to_human:
        • tf.logging.fatal('Failed to locate: %s', val)
        • name = uid_to_human[val]
        • node_id_to_name[key] = name
          -
        • return node_id_to_name
          -
        • def id_to_string(self, node_id):
        • if node_id not in self.node_lookup:
        • return ''
        • return self.node_lookup[node_id]
          -
          -
          -def create_graph():
        • """Creates a graph from saved GraphDef file and returns a saver."""
        • # Creates graph from saved graph_def.pb.
        • with tf.gfile.FastGFile(os.path.join(
        • FLAGS.model_dir, 'classify_image_graph_def.pb'), 'rb') as f:
        • graph_def = tf.GraphDef()
        • graph_def.ParseFromString(f.read())
        • _ = tf.import_graph_def(graph_def, name='')
          + dest_directory = FLAGS.model_dir

        + image_size = inception.inception_v4.default_image_size

        -def run_inference_on_image(image):

        • """Runs inference on an image.
          -
        • Args:
        • image: Image file name.
          -
        • Returns:
        • Nothing
        • """
        • if not tf.gfile.Exists(image):
        • tf.logging.fatal('File does not exist %s', image)
        • image_data = tf.gfile.FastGFile(image, 'rb').read()
          -
        • # Creates graph from saved GraphDef.
        • create_graph()
          -
        • with tf.Session() as sess:
        • # Some useful tensors:
        • # 'softmax:0': A tensor containing the normalized prediction across
        • # 1000 labels.
        • # 'pool_3:0': A tensor containing the next-to-last layer containing 2048
        • # float description of the image.
        • # 'DecodeJpeg/contents:0': A tensor containing a string providing JPEG
        • # encoding of the image.
        • # Runs the softmax tensor by feeding the image_data as input to the graph.
        • softmax_tensor = sess.graph.get_tensor_by_name('softmax:0')
        • predictions = sess.run(softmax_tensor,
        • {'DecodeJpeg/contents:0': image_data}

          )

        • predictions = np.squeeze(predictions)
          -
        • # Creates node ID --> English string lookup.
        • node_lookup = NodeLookup()
          -
        • top_k = predictions.argsort()[-FLAGS.num_top_predictions:][::-1]
        • for node_id in top_k:
        • human_string = node_lookup.id_to_string(node_id)
        • score = predictions[node_id]
        • print('%s (score = %.5f)' % (human_string, score))
          + if not tf.gfile.Exists(image):
          + tf.logging.fatal('File does not exist %s', image)
          + image_string = tf.gfile.FastGFile(image, 'rb').read()

        + with tf.Graph().as_default():
        + image = tf.image.decode_jpeg(image_string, channels=3)
        + processed_image = inception_preprocessing.preprocess_image(
        + image, image_size, image_size, is_training=False)
        + processed_images = tf.expand_dims(processed_image, 0)
        +
        + # Create the model, use the default arg scope to configure the batch
        + # norm parameters.
        + with slim.arg_scope(inception.inception_v4_arg_scope()):
        + logits, _ = inception.inception_v4(
        + processed_images, num_classes=1001, is_training=False)
        + probabilities = tf.nn.softmax(logits)
        +
        + init_fn = slim.assign_from_checkpoint_fn(
        + os.path.join(dest_directory, 'inception_v4.ckpt'),
        + slim.get_model_variables('InceptionV4'))
        +
        + with tf.Session() as sess:
        + init_fn(sess)
        + _, probabilities = sess.run([image, probabilities])

        Review comment:
        Thanks for the review. I'll work on having a constructor method as expected. I have a query. Don't you think it wouldn't affect classify_image.py? Because we will have run that python file every time we call for recognition through cli?
        What you suggested seems relevant to the REST api in which we can load the model once and reuse it. Correct me if I overlooked something.

        ----------------------------------------------------------------
        This is an automated message from the Apache Git Service.
        To respond to the message, please log on GitHub and use the
        URL above to go to the specific comment.

        For queries about this service, please contact Infrastructure at:
        users@infra.apache.org

        Show
        githubbot ASF GitHub Bot added a comment - KranthiGV commented on a change in pull request #163: TIKA-2306 : Update Inception v3 to Inception v4 in Object recognition parser URL: https://github.com/apache/tika/pull/163#discussion_r111529725 ########## File path: tika-parsers/src/main/resources/org/apache/tika/parser/recognition/tf/classify_image.py ########## @@ -44,169 +49,205 @@ from six.moves import urllib import tensorflow as tf +from datasets import imagenet, dataset_utils +from nets import inception +from preprocessing import inception_preprocessing + +slim = tf.contrib.slim + FLAGS = tf.app.flags.FLAGS classify_image_graph_def.pb: Binary representation of the GraphDef protocol buffer. imagenet_synset_to_human_label_map.txt: +# inception_v4.ckpt +# Inception V4 checkpoint file. +# imagenet_metadata.txt Map from synset ID to a human readable string. imagenet_2012_challenge_label_map_proto.pbtxt: +# imagenet_lsvrc_2015_synsets.txt Text representation of a protocol buffer mapping a label to synset ID. tf.app.flags.DEFINE_string( 'model_dir', '/tmp/imagenet', """Path to classify_image_graph_def.pb, """ """imagenet_synset_to_human_label_map.txt, and """ """imagenet_2012_challenge_label_map_proto.pbtxt.""") + """Path to inception_v4.ckpt, """ + """imagenet_lsvrc_2015_synsets.txt, and """ + """imagenet_metadata.txt.""") tf.app.flags.DEFINE_string('image_file', '', """Absolute path to image file.""") tf.app.flags.DEFINE_integer('num_top_predictions', 5, """Display this many predictions.""") pylint: disable=line-too-long -DATA_URL = 'http://download.tensorflow.org/models/image/imagenet/inception-2015-12-05.tgz' +DATA_URL = 'http://download.tensorflow.org/models/inception_v4_2016_09_09.tar.gz' pylint: enable=line-too-long -class NodeLookup(object): """Converts integer node ID's to human readable labels.""" +def create_readable_names_for_imagenet_labels(): + """Create a dict mapping label id to human readable string. + + Returns: + labels_to_names: dictionary where keys are integers from to 1000 + and values are human-readable names. + + We retrieve a synset file, which contains a list of valid synset labels used + by ILSVRC competition. There is one synset one per line, eg. + # n01440764 + # n01443537 + We also retrieve a synset_to_human_file, which contains a mapping from synsets + to human-readable names for every synset in Imagenet. These are stored in a + tsv format, as follows: + # n02119247 black fox + # n02119359 silver fox + We assign each synset (in alphabetical order) an integer, starting from 1 + (since 0 is reserved for the background class). + + Code is based on + https://github.com/tensorflow/models/blob/master/inception/inception/data/build_imagenet_data.py#L463 + """ + + # pylint: disable=line-too-long + + dest_directory = FLAGS.model_dir + + synset_list = [s.strip() for s in open(os.path.join( + dest_directory, 'imagenet_lsvrc_2015_synsets.txt')).readlines()] + num_synsets_in_ilsvrc = len(synset_list) + assert num_synsets_in_ilsvrc == 1000 def _ init _(self, label_lookup_path=None, uid_lookup_path=None): if not label_lookup_path: label_lookup_path = os.path.join( FLAGS.model_dir, 'imagenet_2012_challenge_label_map_proto.pbtxt') if not uid_lookup_path: uid_lookup_path = os.path.join( FLAGS.model_dir, 'imagenet_synset_to_human_label_map.txt') self.node_lookup = self.load(label_lookup_path, uid_lookup_path) + synset_to_human_list = open(os.path.join( + dest_directory, 'imagenet_metadata.txt')).readlines() + num_synsets_in_all_imagenet = len(synset_to_human_list) + assert num_synsets_in_all_imagenet == 21842 def load(self, label_lookup_path, uid_lookup_path): """Loads a human readable English name for each softmax node. + synset_to_human = {} + for s in synset_to_human_list: + parts = s.strip().split('\t') + assert len(parts) == 2 + synset = parts [0] + human = parts [1] + synset_to_human [synset] = human + + label_index = 1 + labels_to_names = {0: 'background'} + for synset in synset_list: + name = synset_to_human [synset] + labels_to_names [label_index] = name + label_index += 1 + + return labels_to_names + + +def run_inference_on_image(image): + """Runs inference on an image. Args: label_lookup_path: string UID to integer node ID. uid_lookup_path: string UID to human-readable string. + image: Image file name. Returns: dict from integer node ID to human-readable string. + Nothing """ if not tf.gfile.Exists(uid_lookup_path): tf.logging.fatal('File does not exist %s', uid_lookup_path) if not tf.gfile.Exists(label_lookup_path): tf.logging.fatal('File does not exist %s', label_lookup_path) - # Loads mapping from string UID to human-readable string proto_as_ascii_lines = tf.gfile.GFile(uid_lookup_path).readlines() uid_to_human = {} p = re.compile(r' [n\d] [ \S,] ') for line in proto_as_ascii_lines: parsed_items = p.findall(line) uid = parsed_items [0] human_string = parsed_items [2] uid_to_human [uid] = human_string - # Loads mapping from string UID to integer node ID. node_id_to_uid = {} proto_as_ascii = tf.gfile.GFile(label_lookup_path).readlines() for line in proto_as_ascii: if line.startswith(' target_class:'): target_class = int(line.split(': ') [1] ) if line.startswith(' target_class_string:'): target_class_string = line.split(': ') [1] node_id_to_uid [target_class] = target_class_string [1:-2] - # Loads the final mapping of integer node ID to human-readable string node_id_to_name = {} for key, val in node_id_to_uid.items(): if val not in uid_to_human: tf.logging.fatal('Failed to locate: %s', val) name = uid_to_human [val] node_id_to_name [key] = name - return node_id_to_name - def id_to_string(self, node_id): if node_id not in self.node_lookup: return '' return self.node_lookup [node_id] - - -def create_graph(): """Creates a graph from saved GraphDef file and returns a saver.""" # Creates graph from saved graph_def.pb. with tf.gfile.FastGFile(os.path.join( FLAGS.model_dir, 'classify_image_graph_def.pb'), 'rb') as f: graph_def = tf.GraphDef() graph_def.ParseFromString(f.read()) _ = tf.import_graph_def(graph_def, name='') + dest_directory = FLAGS.model_dir + image_size = inception.inception_v4.default_image_size -def run_inference_on_image(image): """Runs inference on an image. - Args: image: Image file name. - Returns: Nothing """ if not tf.gfile.Exists(image): tf.logging.fatal('File does not exist %s', image) image_data = tf.gfile.FastGFile(image, 'rb').read() - # Creates graph from saved GraphDef. create_graph() - with tf.Session() as sess: # Some useful tensors: # 'softmax:0': A tensor containing the normalized prediction across # 1000 labels. # 'pool_3:0': A tensor containing the next-to-last layer containing 2048 # float description of the image. # 'DecodeJpeg/contents:0': A tensor containing a string providing JPEG # encoding of the image. # Runs the softmax tensor by feeding the image_data as input to the graph. softmax_tensor = sess.graph.get_tensor_by_name('softmax:0') predictions = sess.run(softmax_tensor, {'DecodeJpeg/contents:0': image_data} ) predictions = np.squeeze(predictions) - # Creates node ID --> English string lookup. node_lookup = NodeLookup() - top_k = predictions.argsort() [-FLAGS.num_top_predictions:] [::-1] for node_id in top_k: human_string = node_lookup.id_to_string(node_id) score = predictions [node_id] print('%s (score = %.5f)' % (human_string, score)) + if not tf.gfile.Exists(image): + tf.logging.fatal('File does not exist %s', image) + image_string = tf.gfile.FastGFile(image, 'rb').read() + with tf.Graph().as_default(): + image = tf.image.decode_jpeg(image_string, channels=3) + processed_image = inception_preprocessing.preprocess_image( + image, image_size, image_size, is_training=False) + processed_images = tf.expand_dims(processed_image, 0) + + # Create the model, use the default arg scope to configure the batch + # norm parameters. + with slim.arg_scope(inception.inception_v4_arg_scope()): + logits, _ = inception.inception_v4( + processed_images, num_classes=1001, is_training=False) + probabilities = tf.nn.softmax(logits) + + init_fn = slim.assign_from_checkpoint_fn( + os.path.join(dest_directory, 'inception_v4.ckpt'), + slim.get_model_variables('InceptionV4')) + + with tf.Session() as sess: + init_fn(sess) + _, probabilities = sess.run( [image, probabilities] ) Review comment: Thanks for the review. I'll work on having a constructor method as expected. I have a query. Don't you think it wouldn't affect classify_image.py? Because we will have run that python file every time we call for recognition through cli? What you suggested seems relevant to the REST api in which we can load the model once and reuse it. Correct me if I overlooked something. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org
        Hide
        githubbot ASF GitHub Bot added a comment -

        thammegowda commented on a change in pull request #163: TIKA-2306: Update Inception v3 to Inception v4 in Object recognition parser
        URL: https://github.com/apache/tika/pull/163#discussion_r111529125

        ##########
        File path: tika-parsers/src/main/resources/org/apache/tika/parser/recognition/tf/classify_image.py
        ##########
        @@ -44,169 +49,205 @@
        from six.moves import urllib
        import tensorflow as tf

        +from datasets import imagenet, dataset_utils
        +from nets import inception
        +from preprocessing import inception_preprocessing
        +
        +slim = tf.contrib.slim
        +
        FLAGS = tf.app.flags.FLAGS

          1. classify_image_graph_def.pb:
          2. Binary representation of the GraphDef protocol buffer.
          3. imagenet_synset_to_human_label_map.txt:
            +# inception_v4.ckpt
            +# Inception V4 checkpoint file.
            +# imagenet_metadata.txt
        1. Map from synset ID to a human readable string.
          1. imagenet_2012_challenge_label_map_proto.pbtxt:
            +# imagenet_lsvrc_2015_synsets.txt
        2. Text representation of a protocol buffer mapping a label to synset ID.
          tf.app.flags.DEFINE_string(
          'model_dir', '/tmp/imagenet',
        • """Path to classify_image_graph_def.pb, """
        • """imagenet_synset_to_human_label_map.txt, and """
        • """imagenet_2012_challenge_label_map_proto.pbtxt.""")
          + """Path to inception_v4.ckpt, """
          + """imagenet_lsvrc_2015_synsets.txt, and """
          + """imagenet_metadata.txt.""")
          tf.app.flags.DEFINE_string('image_file', '',
          """Absolute path to image file.""")
          tf.app.flags.DEFINE_integer('num_top_predictions', 5,
          """Display this many predictions.""")
        1. pylint: disable=line-too-long
          -DATA_URL = 'http://download.tensorflow.org/models/image/imagenet/inception-2015-12-05.tgz'
          +DATA_URL = 'http://download.tensorflow.org/models/inception_v4_2016_09_09.tar.gz'
        2. pylint: enable=line-too-long

        -class NodeLookup(object):

        • """Converts integer node ID's to human readable labels."""
          +def create_readable_names_for_imagenet_labels():
          + """Create a dict mapping label id to human readable string.
          +
          + Returns:
          + labels_to_names: dictionary where keys are integers from to 1000
          + and values are human-readable names.
          +
          + We retrieve a synset file, which contains a list of valid synset labels used
          + by ILSVRC competition. There is one synset one per line, eg.
          + # n01440764
          + # n01443537
          + We also retrieve a synset_to_human_file, which contains a mapping from synsets
          + to human-readable names for every synset in Imagenet. These are stored in a
          + tsv format, as follows:
          + # n02119247 black fox
          + # n02119359 silver fox
          + We assign each synset (in alphabetical order) an integer, starting from 1
          + (since 0 is reserved for the background class).
          +
          + Code is based on
          + https://github.com/tensorflow/models/blob/master/inception/inception/data/build_imagenet_data.py#L463
          + """
          +
          + # pylint: disable=line-too-long
          +
          + dest_directory = FLAGS.model_dir
          +
          + synset_list = [s.strip() for s in open(os.path.join(
          + dest_directory, 'imagenet_lsvrc_2015_synsets.txt')).readlines()]
          + num_synsets_in_ilsvrc = len(synset_list)
          + assert num_synsets_in_ilsvrc == 1000
        • def _init_(self,
        • label_lookup_path=None,
        • uid_lookup_path=None):
        • if not label_lookup_path:
        • label_lookup_path = os.path.join(
        • FLAGS.model_dir, 'imagenet_2012_challenge_label_map_proto.pbtxt')
        • if not uid_lookup_path:
        • uid_lookup_path = os.path.join(
        • FLAGS.model_dir, 'imagenet_synset_to_human_label_map.txt')
        • self.node_lookup = self.load(label_lookup_path, uid_lookup_path)
          + synset_to_human_list = open(os.path.join(
          + dest_directory, 'imagenet_metadata.txt')).readlines()
          + num_synsets_in_all_imagenet = len(synset_to_human_list)
          + assert num_synsets_in_all_imagenet == 21842
        • def load(self, label_lookup_path, uid_lookup_path):
        • """Loads a human readable English name for each softmax node.
          + synset_to_human = {}
          + for s in synset_to_human_list:
          + parts = s.strip().split('\t')
          + assert len(parts) == 2
          + synset = parts[0]
          + human = parts[1]
          + synset_to_human[synset] = human
          +
          + label_index = 1
          + labels_to_names = {0: 'background'}

          + for synset in synset_list:
          + name = synset_to_human[synset]
          + labels_to_names[label_index] = name
          + label_index += 1
          +
          + return labels_to_names
          +
          +
          +def run_inference_on_image(image):
          + """Runs inference on an image.

        Args:

        • label_lookup_path: string UID to integer node ID.
        • uid_lookup_path: string UID to human-readable string.
          + image: Image file name.

        Returns:

        • dict from integer node ID to human-readable string.
          + Nothing
          """
        • if not tf.gfile.Exists(uid_lookup_path):
        • tf.logging.fatal('File does not exist %s', uid_lookup_path)
        • if not tf.gfile.Exists(label_lookup_path):
        • tf.logging.fatal('File does not exist %s', label_lookup_path)
          -
        • # Loads mapping from string UID to human-readable string
        • proto_as_ascii_lines = tf.gfile.GFile(uid_lookup_path).readlines()
        • uid_to_human = {}
        • p = re.compile(r'[n\d][ \S,]')
        • for line in proto_as_ascii_lines:
        • parsed_items = p.findall(line)
        • uid = parsed_items[0]
        • human_string = parsed_items[2]
        • uid_to_human[uid] = human_string
          -
        • # Loads mapping from string UID to integer node ID.
        • node_id_to_uid = {}
        • proto_as_ascii = tf.gfile.GFile(label_lookup_path).readlines()
        • for line in proto_as_ascii:
        • if line.startswith(' target_class:'):
        • target_class = int(line.split(': ')[1])
        • if line.startswith(' target_class_string:'):
        • target_class_string = line.split(': ')[1]
        • node_id_to_uid[target_class] = target_class_string[1:-2]
          -
        • # Loads the final mapping of integer node ID to human-readable string
        • node_id_to_name = {}
        • for key, val in node_id_to_uid.items():
        • if val not in uid_to_human:
        • tf.logging.fatal('Failed to locate: %s', val)
        • name = uid_to_human[val]
        • node_id_to_name[key] = name
          -
        • return node_id_to_name
          -
        • def id_to_string(self, node_id):
        • if node_id not in self.node_lookup:
        • return ''
        • return self.node_lookup[node_id]
          -
          -
          -def create_graph():
        • """Creates a graph from saved GraphDef file and returns a saver."""
        • # Creates graph from saved graph_def.pb.
        • with tf.gfile.FastGFile(os.path.join(
        • FLAGS.model_dir, 'classify_image_graph_def.pb'), 'rb') as f:
        • graph_def = tf.GraphDef()
        • graph_def.ParseFromString(f.read())
        • _ = tf.import_graph_def(graph_def, name='')
          + dest_directory = FLAGS.model_dir

        + image_size = inception.inception_v4.default_image_size

        -def run_inference_on_image(image):

        • """Runs inference on an image.
          -
        • Args:
        • image: Image file name.
          -
        • Returns:
        • Nothing
        • """
        • if not tf.gfile.Exists(image):
        • tf.logging.fatal('File does not exist %s', image)
        • image_data = tf.gfile.FastGFile(image, 'rb').read()
          -
        • # Creates graph from saved GraphDef.
        • create_graph()
          -
        • with tf.Session() as sess:
        • # Some useful tensors:
        • # 'softmax:0': A tensor containing the normalized prediction across
        • # 1000 labels.
        • # 'pool_3:0': A tensor containing the next-to-last layer containing 2048
        • # float description of the image.
        • # 'DecodeJpeg/contents:0': A tensor containing a string providing JPEG
        • # encoding of the image.
        • # Runs the softmax tensor by feeding the image_data as input to the graph.
        • softmax_tensor = sess.graph.get_tensor_by_name('softmax:0')
        • predictions = sess.run(softmax_tensor,
        • {'DecodeJpeg/contents:0': image_data}

          )

        • predictions = np.squeeze(predictions)
          -
        • # Creates node ID --> English string lookup.
        • node_lookup = NodeLookup()
          -
        • top_k = predictions.argsort()[-FLAGS.num_top_predictions:][::-1]
        • for node_id in top_k:
        • human_string = node_lookup.id_to_string(node_id)
        • score = predictions[node_id]
        • print('%s (score = %.5f)' % (human_string, score))
          + if not tf.gfile.Exists(image):
          + tf.logging.fatal('File does not exist %s', image)
          + image_string = tf.gfile.FastGFile(image, 'rb').read()

        + with tf.Graph().as_default():
        + image = tf.image.decode_jpeg(image_string, channels=3)
        + processed_image = inception_preprocessing.preprocess_image(
        + image, image_size, image_size, is_training=False)
        + processed_images = tf.expand_dims(processed_image, 0)
        +
        + # Create the model, use the default arg scope to configure the batch
        + # norm parameters.
        + with slim.arg_scope(inception.inception_v4_arg_scope()):
        + logits, _ = inception.inception_v4(
        + processed_images, num_classes=1001, is_training=False)
        + probabilities = tf.nn.softmax(logits)
        +
        + init_fn = slim.assign_from_checkpoint_fn(
        + os.path.join(dest_directory, 'inception_v4.ckpt'),
        + slim.get_model_variables('InceptionV4'))
        +
        + with tf.Session() as sess:
        + init_fn(sess)
        + _, probabilities = sess.run([image, probabilities])

        Review comment:
        This code is lot inefficient. It takes about 15 seconds on my super powerful machine.

        *Problem here*
        Sessions can be handled in a better way to reduce disk IO (Loading model is a disk IO).
        Initializing the model and restoring the checkpoint is costly operation compared to the classification itself.

        *How to fix:*
        Please reimagine Classifier service as an object in Object oriented system.
        Get the tensorflow session initialized in the constructor
        Initialize the model and restore checkpoint in the constructor.
        It is perfectly okay to not safely closing the session, here we are not modifying the model.

        Then in the classify action, simply use pre-initialized session and the model to classify image.
        Note that classify action/method may be called a million times.

        You may refer the old code with these points you will see the pattern. However, if you face difficulty ping me , I will have another closer look in newer version of TF how to do this.

        @KranthiGV

        ----------------------------------------------------------------
        This is an automated message from the Apache Git Service.
        To respond to the message, please log on GitHub and use the
        URL above to go to the specific comment.

        For queries about this service, please contact Infrastructure at:
        users@infra.apache.org

        Show
        githubbot ASF GitHub Bot added a comment - thammegowda commented on a change in pull request #163: TIKA-2306 : Update Inception v3 to Inception v4 in Object recognition parser URL: https://github.com/apache/tika/pull/163#discussion_r111529125 ########## File path: tika-parsers/src/main/resources/org/apache/tika/parser/recognition/tf/classify_image.py ########## @@ -44,169 +49,205 @@ from six.moves import urllib import tensorflow as tf +from datasets import imagenet, dataset_utils +from nets import inception +from preprocessing import inception_preprocessing + +slim = tf.contrib.slim + FLAGS = tf.app.flags.FLAGS classify_image_graph_def.pb: Binary representation of the GraphDef protocol buffer. imagenet_synset_to_human_label_map.txt: +# inception_v4.ckpt +# Inception V4 checkpoint file. +# imagenet_metadata.txt Map from synset ID to a human readable string. imagenet_2012_challenge_label_map_proto.pbtxt: +# imagenet_lsvrc_2015_synsets.txt Text representation of a protocol buffer mapping a label to synset ID. tf.app.flags.DEFINE_string( 'model_dir', '/tmp/imagenet', """Path to classify_image_graph_def.pb, """ """imagenet_synset_to_human_label_map.txt, and """ """imagenet_2012_challenge_label_map_proto.pbtxt.""") + """Path to inception_v4.ckpt, """ + """imagenet_lsvrc_2015_synsets.txt, and """ + """imagenet_metadata.txt.""") tf.app.flags.DEFINE_string('image_file', '', """Absolute path to image file.""") tf.app.flags.DEFINE_integer('num_top_predictions', 5, """Display this many predictions.""") pylint: disable=line-too-long -DATA_URL = 'http://download.tensorflow.org/models/image/imagenet/inception-2015-12-05.tgz' +DATA_URL = 'http://download.tensorflow.org/models/inception_v4_2016_09_09.tar.gz' pylint: enable=line-too-long -class NodeLookup(object): """Converts integer node ID's to human readable labels.""" +def create_readable_names_for_imagenet_labels(): + """Create a dict mapping label id to human readable string. + + Returns: + labels_to_names: dictionary where keys are integers from to 1000 + and values are human-readable names. + + We retrieve a synset file, which contains a list of valid synset labels used + by ILSVRC competition. There is one synset one per line, eg. + # n01440764 + # n01443537 + We also retrieve a synset_to_human_file, which contains a mapping from synsets + to human-readable names for every synset in Imagenet. These are stored in a + tsv format, as follows: + # n02119247 black fox + # n02119359 silver fox + We assign each synset (in alphabetical order) an integer, starting from 1 + (since 0 is reserved for the background class). + + Code is based on + https://github.com/tensorflow/models/blob/master/inception/inception/data/build_imagenet_data.py#L463 + """ + + # pylint: disable=line-too-long + + dest_directory = FLAGS.model_dir + + synset_list = [s.strip() for s in open(os.path.join( + dest_directory, 'imagenet_lsvrc_2015_synsets.txt')).readlines()] + num_synsets_in_ilsvrc = len(synset_list) + assert num_synsets_in_ilsvrc == 1000 def _ init _(self, label_lookup_path=None, uid_lookup_path=None): if not label_lookup_path: label_lookup_path = os.path.join( FLAGS.model_dir, 'imagenet_2012_challenge_label_map_proto.pbtxt') if not uid_lookup_path: uid_lookup_path = os.path.join( FLAGS.model_dir, 'imagenet_synset_to_human_label_map.txt') self.node_lookup = self.load(label_lookup_path, uid_lookup_path) + synset_to_human_list = open(os.path.join( + dest_directory, 'imagenet_metadata.txt')).readlines() + num_synsets_in_all_imagenet = len(synset_to_human_list) + assert num_synsets_in_all_imagenet == 21842 def load(self, label_lookup_path, uid_lookup_path): """Loads a human readable English name for each softmax node. + synset_to_human = {} + for s in synset_to_human_list: + parts = s.strip().split('\t') + assert len(parts) == 2 + synset = parts [0] + human = parts [1] + synset_to_human [synset] = human + + label_index = 1 + labels_to_names = {0: 'background'} + for synset in synset_list: + name = synset_to_human [synset] + labels_to_names [label_index] = name + label_index += 1 + + return labels_to_names + + +def run_inference_on_image(image): + """Runs inference on an image. Args: label_lookup_path: string UID to integer node ID. uid_lookup_path: string UID to human-readable string. + image: Image file name. Returns: dict from integer node ID to human-readable string. + Nothing """ if not tf.gfile.Exists(uid_lookup_path): tf.logging.fatal('File does not exist %s', uid_lookup_path) if not tf.gfile.Exists(label_lookup_path): tf.logging.fatal('File does not exist %s', label_lookup_path) - # Loads mapping from string UID to human-readable string proto_as_ascii_lines = tf.gfile.GFile(uid_lookup_path).readlines() uid_to_human = {} p = re.compile(r' [n\d] [ \S,] ') for line in proto_as_ascii_lines: parsed_items = p.findall(line) uid = parsed_items [0] human_string = parsed_items [2] uid_to_human [uid] = human_string - # Loads mapping from string UID to integer node ID. node_id_to_uid = {} proto_as_ascii = tf.gfile.GFile(label_lookup_path).readlines() for line in proto_as_ascii: if line.startswith(' target_class:'): target_class = int(line.split(': ') [1] ) if line.startswith(' target_class_string:'): target_class_string = line.split(': ') [1] node_id_to_uid [target_class] = target_class_string [1:-2] - # Loads the final mapping of integer node ID to human-readable string node_id_to_name = {} for key, val in node_id_to_uid.items(): if val not in uid_to_human: tf.logging.fatal('Failed to locate: %s', val) name = uid_to_human [val] node_id_to_name [key] = name - return node_id_to_name - def id_to_string(self, node_id): if node_id not in self.node_lookup: return '' return self.node_lookup [node_id] - - -def create_graph(): """Creates a graph from saved GraphDef file and returns a saver.""" # Creates graph from saved graph_def.pb. with tf.gfile.FastGFile(os.path.join( FLAGS.model_dir, 'classify_image_graph_def.pb'), 'rb') as f: graph_def = tf.GraphDef() graph_def.ParseFromString(f.read()) _ = tf.import_graph_def(graph_def, name='') + dest_directory = FLAGS.model_dir + image_size = inception.inception_v4.default_image_size -def run_inference_on_image(image): """Runs inference on an image. - Args: image: Image file name. - Returns: Nothing """ if not tf.gfile.Exists(image): tf.logging.fatal('File does not exist %s', image) image_data = tf.gfile.FastGFile(image, 'rb').read() - # Creates graph from saved GraphDef. create_graph() - with tf.Session() as sess: # Some useful tensors: # 'softmax:0': A tensor containing the normalized prediction across # 1000 labels. # 'pool_3:0': A tensor containing the next-to-last layer containing 2048 # float description of the image. # 'DecodeJpeg/contents:0': A tensor containing a string providing JPEG # encoding of the image. # Runs the softmax tensor by feeding the image_data as input to the graph. softmax_tensor = sess.graph.get_tensor_by_name('softmax:0') predictions = sess.run(softmax_tensor, {'DecodeJpeg/contents:0': image_data} ) predictions = np.squeeze(predictions) - # Creates node ID --> English string lookup. node_lookup = NodeLookup() - top_k = predictions.argsort() [-FLAGS.num_top_predictions:] [::-1] for node_id in top_k: human_string = node_lookup.id_to_string(node_id) score = predictions [node_id] print('%s (score = %.5f)' % (human_string, score)) + if not tf.gfile.Exists(image): + tf.logging.fatal('File does not exist %s', image) + image_string = tf.gfile.FastGFile(image, 'rb').read() + with tf.Graph().as_default(): + image = tf.image.decode_jpeg(image_string, channels=3) + processed_image = inception_preprocessing.preprocess_image( + image, image_size, image_size, is_training=False) + processed_images = tf.expand_dims(processed_image, 0) + + # Create the model, use the default arg scope to configure the batch + # norm parameters. + with slim.arg_scope(inception.inception_v4_arg_scope()): + logits, _ = inception.inception_v4( + processed_images, num_classes=1001, is_training=False) + probabilities = tf.nn.softmax(logits) + + init_fn = slim.assign_from_checkpoint_fn( + os.path.join(dest_directory, 'inception_v4.ckpt'), + slim.get_model_variables('InceptionV4')) + + with tf.Session() as sess: + init_fn(sess) + _, probabilities = sess.run( [image, probabilities] ) Review comment: This code is lot inefficient. It takes about 15 seconds on my super powerful machine. * Problem here * Sessions can be handled in a better way to reduce disk IO (Loading model is a disk IO). Initializing the model and restoring the checkpoint is costly operation compared to the classification itself. * How to fix: * Please reimagine Classifier service as an object in Object oriented system. Get the tensorflow session initialized in the constructor Initialize the model and restore checkpoint in the constructor. It is perfectly okay to not safely closing the session, here we are not modifying the model. Then in the classify action, simply use pre-initialized session and the model to classify image. Note that classify action/method may be called a million times. You may refer the old code with these points you will see the pattern. However, if you face difficulty ping me , I will have another closer look in newer version of TF how to do this. @KranthiGV ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org
        Hide
        githubbot ASF GitHub Bot added a comment -

        thammegowda commented on a change in pull request #163: TIKA-2306: Update Inception v3 to Inception v4 in Object recognition parser
        URL: https://github.com/apache/tika/pull/163#discussion_r111525259

        ##########
        File path: tika-parsers/src/main/resources/org/apache/tika/parser/recognition/tf/InceptionRestDockerfile
        ##########
        @@ -16,26 +16,34 @@
        #
        FROM ubuntu

        -MAINTAINER Thamme Gowda <thammegowda@apache.org>
        +MAINTAINER Thamme Gowda <thammegowda@apache.org> and Kranthi Kiran GV <kkranthi@student.nitw.ac.in>

          1. install missing part of ubuntu core + python stuff
            +# Install missing part of ubuntu core + python stuff
            RUN apt-get update && \
        • apt-get install -y python-pip python-dev wget
          + apt-get install -y python-pip python-dev wget libtcmalloc-minimal4 git
        1. Install tensorflow and other dependencies
          RUN \

        +# Get the TF-slim dependencies
        +# Forked for future compatibility
        +RUN git clone https://github.com/KranthiGV/models
        +

        1. TODO: Change the URL to Apache/Tika Repo when this PR gets merged
          RUN \

        Review comment:
        TODO: when we merge this PR, we modify it and then merge it

        ----------------------------------------------------------------
        This is an automated message from the Apache Git Service.
        To respond to the message, please log on GitHub and use the
        URL above to go to the specific comment.

        For queries about this service, please contact Infrastructure at:
        users@infra.apache.org

        Show
        githubbot ASF GitHub Bot added a comment - thammegowda commented on a change in pull request #163: TIKA-2306 : Update Inception v3 to Inception v4 in Object recognition parser URL: https://github.com/apache/tika/pull/163#discussion_r111525259 ########## File path: tika-parsers/src/main/resources/org/apache/tika/parser/recognition/tf/InceptionRestDockerfile ########## @@ -16,26 +16,34 @@ # FROM ubuntu -MAINTAINER Thamme Gowda <thammegowda@apache.org> +MAINTAINER Thamme Gowda <thammegowda@apache.org> and Kranthi Kiran GV <kkranthi@student.nitw.ac.in> install missing part of ubuntu core + python stuff +# Install missing part of ubuntu core + python stuff RUN apt-get update && \ apt-get install -y python-pip python-dev wget + apt-get install -y python-pip python-dev wget libtcmalloc-minimal4 git Install tensorflow and other dependencies RUN \ pip install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.9.0-cp27-none-linux_x86_64.whl && \ + pip install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.0.1-cp27-none-linux_x86_64.whl && \ pip install flask requests +# Get the TF-slim dependencies +# Forked for future compatibility +RUN git clone https://github.com/KranthiGV/models + TODO: Change the URL to Apache/Tika Repo when this PR gets merged RUN \ wget https://raw.githubusercontent.com/thammegowda/tika/TIKA-1993/tika-parsers/src/main/resources/org/apache/tika/parser/recognition/tf/inceptionapi.py -O /usr/bin/inceptionapi.py && \ + wget https://raw.githubusercontent.com/KranthiGV/tika/TIKA-2306/tika-parsers/src/main/resources/org/apache/tika/parser/recognition/tf/inceptionapi.py -O /usr/bin/inceptionapi.py && \ Review comment: TODO: when we merge this PR, we modify it and then merge it ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org
        Hide
        githubbot ASF GitHub Bot added a comment -

        thammegowda commented on issue #163: TIKA-2306: Update Inception v3 to Inception v4 in Object recognition parser
        URL: https://github.com/apache/tika/pull/163#issuecomment-294076746

        Reviewing this today.

        ----------------------------------------------------------------
        This is an automated message from the Apache Git Service.
        To respond to the message, please log on GitHub and use the
        URL above to go to the specific comment.

        For queries about this service, please contact Infrastructure at:
        users@infra.apache.org

        Show
        githubbot ASF GitHub Bot added a comment - thammegowda commented on issue #163: TIKA-2306 : Update Inception v3 to Inception v4 in Object recognition parser URL: https://github.com/apache/tika/pull/163#issuecomment-294076746 Reviewing this today. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org
        Hide
        githubbot ASF GitHub Bot added a comment -

        KranthiGV commented on issue #163: TIKA-2306: Update Inception v3 to Inception v4 in Object recognition parser
        URL: https://github.com/apache/tika/pull/163#issuecomment-293681965

        @thammegowda @grossws
        Can you please review this PR?
        It would enable me to work on TIKA-2308(https://issues.apache.org/jira/browse/TIKA-2308) and also adding support for other image MIME types.
        Since the current and this implementations are different, it would save us the trouble of re-doing a lot of work again when this PR gets merged.

        ----------------------------------------------------------------
        This is an automated message from the Apache Git Service.
        To respond to the message, please log on GitHub and use the
        URL above to go to the specific comment.

        For queries about this service, please contact Infrastructure at:
        users@infra.apache.org

        Show
        githubbot ASF GitHub Bot added a comment - KranthiGV commented on issue #163: TIKA-2306 : Update Inception v3 to Inception v4 in Object recognition parser URL: https://github.com/apache/tika/pull/163#issuecomment-293681965 @thammegowda @grossws Can you please review this PR? It would enable me to work on TIKA-2308 ( https://issues.apache.org/jira/browse/TIKA-2308 ) and also adding support for other image MIME types. Since the current and this implementations are different, it would save us the trouble of re-doing a lot of work again when this PR gets merged. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org
        Hide
        githubbot ASF GitHub Bot added a comment -

        KranthiGV commented on issue #163: TIKA-2306: Update Inception v3 to Inception v4 in Object recognition parser
        URL: https://github.com/apache/tika/pull/163#issuecomment-293681965

        @thammegowda
        Can you please review this PR?
        It would enable me to work on TIKA-2308(https://issues.apache.org/jira/browse/TIKA-2308) and also adding support for other image MIME types.
        Since the current and this implementations are different, it would save us the trouble of re-doing a lot of work again when this PR gets merged.

        ----------------------------------------------------------------
        This is an automated message from the Apache Git Service.
        To respond to the message, please log on GitHub and use the
        URL above to go to the specific comment.

        For queries about this service, please contact Infrastructure at:
        users@infra.apache.org

        Show
        githubbot ASF GitHub Bot added a comment - KranthiGV commented on issue #163: TIKA-2306 : Update Inception v3 to Inception v4 in Object recognition parser URL: https://github.com/apache/tika/pull/163#issuecomment-293681965 @thammegowda Can you please review this PR? It would enable me to work on TIKA-2308 ( https://issues.apache.org/jira/browse/TIKA-2308 ) and also adding support for other image MIME types. Since the current and this implementations are different, it would save us the trouble of re-doing a lot of work again when this PR gets merged. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org
        Hide
        githubbot ASF GitHub Bot added a comment -

        KranthiGV commented on issue #163: TIKA-2306: Update Inception v3 to Inception v4 in Object recognition parser
        URL: https://github.com/apache/tika/pull/163#issuecomment-293681965

        @thammegowda
        Can you please review this PR?
        It would enable me to work on TIKA-2308(https://issues.apache.org/jira/browse/TIKA-2308) and also adding support for other image MIME types.
        Since the current and this implementations are different, it would save us the trouble of re-doing a lot of work again when this PR gets merged.

        ----------------------------------------------------------------
        This is an automated message from the Apache Git Service.
        To respond to the message, please log on GitHub and use the
        URL above to go to the specific comment.

        For queries about this service, please contact Infrastructure at:
        users@infra.apache.org

        Show
        githubbot ASF GitHub Bot added a comment - KranthiGV commented on issue #163: TIKA-2306 : Update Inception v3 to Inception v4 in Object recognition parser URL: https://github.com/apache/tika/pull/163#issuecomment-293681965 @thammegowda Can you please review this PR? It would enable me to work on TIKA-2308 ( https://issues.apache.org/jira/browse/TIKA-2308 ) and also adding support for other image MIME types. Since the current and this implementations are different, it would save us the trouble of re-doing a lot of work again when this PR gets merged. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org
        Show
        kranthigv Kranthi Kiran GV added a comment - - edited Chris A. Mattmann Thamme Gowda | ( Thamme Gowda Mattmann, Chris A (388J) ) Please review PR at https://github.com/apache/tika/pull/163
        Hide
        raunaq.abhyankar Raunaq Abhyankar added a comment -

        Hi
        I have sent in a PR. Do review it and suggest necessary changes.
        Haven't changed the inceptionapi.py script. But its implementation shouldn't be much difficult & can be done once the PR is reviewed.
        Thamme Gowda

        Show
        raunaq.abhyankar Raunaq Abhyankar added a comment - Hi I have sent in a PR. Do review it and suggest necessary changes. Haven't changed the inceptionapi.py script. But its implementation shouldn't be much difficult & can be done once the PR is reviewed. Thamme Gowda
        Hide
        raunaq.abhyankar Raunaq Abhyankar added a comment -

        Hi!
        I was able to successfully classify image using Inception v4 and the results are better than Inception v3! However, I'm finding it difficult to merge the independent classification script with the existing Tika codebase.
        Any tips on the flow of code in Tika? How exactly is "classify_image.py" run?

        Show
        raunaq.abhyankar Raunaq Abhyankar added a comment - Hi! I was able to successfully classify image using Inception v4 and the results are better than Inception v3! However, I'm finding it difficult to merge the independent classification script with the existing Tika codebase. Any tips on the flow of code in Tika? How exactly is "classify_image.py" run?

          People

          • Assignee:
            thammegowda Thamme Gowda
            Reporter:
            kranthigv Kranthi Kiran GV
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:

              Time Tracking

              Estimated:
              Original Estimate - 72h
              72h
              Remaining:
              Remaining Estimate - 72h
              72h
              Logged:
              Time Spent - Not Specified
              Not Specified

                Development