Uploaded image for project: 'ManifoldCF'
  1. ManifoldCF
  2. CONNECTORS-1233

AmazonS3 Repository Connector

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: ManifoldCF 2.3
    • Component/s: None
    • Labels:
    • Flags:
      Patch, Important

      Description

      Feature Patch

      AmazonS3 Repository Connector
      AmazonS3 Repository Connector

      A. Overview
      1. Connects to Amazons3 buckets, and indexes the artifact. if any buckets to be avoided it can be skipped ( it can be configured in job)
      2. Internally documents are parsed and meta data are extracted using Tika
      3. Support Locale - English US ( Currently common_en_US.properties, available, looking for support from some to do the translation for the keys)

      B. Documentation - Work in progress, will be attached issue on the following days

      C. Dependencies - (common-lib)
      1. aws-java-sdk-

      {version}.jar
      2. aws-java-sdk-core-{version}

      .jar
      3. aws-java-sdk-s3-

      {version}

      .jar
      4. joda-time-2.2.jar

      D. Connectors.xml
      <!-- Add your authority connectors here -->
      <authorityconnector name="Amazons3" class="org.apache.manifoldcf.authorities.authorities.amazons3.AmazonS3Authority"/>

      <!-- Add your repository connectors here -->
      <repositoryconnector name="AmazonS3" class="org.apache.manifoldcf.crawler.connectors.amazons3.AmazonS3Connector"/>

        Attachments

        1. patch-unbounded-new2.diff
          14 kB
          Gunaratnam Kuhajeyan
        2. patch-tikaremoved.diff
          12 kB
          Gunaratnam Kuhajeyan
        3. patch-removed-unwanted-dependencies-connector-1233.diff
          3 kB
          Gunaratnam Kuhajeyan
        4. patch-exceptionhandle.diff
          17 kB
          Gunaratnam Kuhajeyan
        5. dependencies.docx
          13 kB
          Gunaratnam Kuhajeyan
        6. amazons3patchnew1.diff
          40 kB
          Gunaratnam Kuhajeyan
        7. amazons3patch-fixunboundedsize.diff
          18 kB
          Gunaratnam Kuhajeyan
        8. amazons3patch.diff
          136 kB
          Gunaratnam Kuhajeyan

          Issue Links

            Activity

              People

              • Assignee:
                kwright@metacarta.com Karl Wright
                Reporter:
                kbird Gunaratnam Kuhajeyan
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - 240h
                  240h
                  Remaining:
                  Remaining Estimate - 240h
                  240h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified