Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-3268

Add AUTO_INCREMENT column in GenerateTableFetch to benefit index

    Details

    • Type: Improvement
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.1.1
    • Fix Version/s: None
    • Component/s: Core Framework
    • Labels:
    • Environment:
      - ubuntu 16.04
      - java version "1.8.0_111"
      - Java(TM) SE Runtime Environment (build 1.8.0_111-b14)
      - Java HotSpot(TM) 64-Bit Server VM (build 25.111-b14, mixed mode)

      Description

      I added AUTO_INCREMENT column in GenerateTableFetch to benefit index column
      By default this processor uses OFFSET, i have problems with large data. somme column has index so we could use index to speed up query time.

      I posted question here :

      https://community.hortonworks.com/questions/72586/how-can-i-use-an-array-with-putelasticsearch.html

      If you indexed un column (id), you could use this sql

      ```
      select xxx
      From xxxxx
      where 200000=>id
      order by id
      limit 200000
      ```

      “OFFSET is bad for skipping previous rows.” [Online]. Available: http://Use-The-Index-Luke.com/sql/partial-results/fetch-next-page. [Accessed: 27-Dec-2016].

      Thank you in advance

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                qfdk qfdk
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated: