[NIFI-3268] Add AUTO_INCREMENT column in GenerateTableFetch to benefit index

• Type: Improvement
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.1.1
• Component/s: Core Framework
• Labels: easyfix
• Environment: - ubuntu 16.04; - java version "1.8.0_111"; - Java(TM) SE Runtime Environment (build 1.8.0_111-b14); - Java HotSpot(TM) 64-Bit Server VM (build 25.111-b14, mixed mode)
• Assignee: Unassigned
• Reporter: qfdk
• Created: 30/Dec/16 10:19
• Updated: 25/Mar/21 17:39

Description

I added AUTO_INCREMENT column in GenerateTableFetch to benefit index column
By default this processor uses OFFSET, i have problems with large data. somme column has index so we could use index to speed up query time.

I posted question here :

https://community.hortonworks.com/questions/72586/how-can-i-use-an-array-with-putelasticsearch.html

If you indexed un column (id), you could use this sql

```
select xxx
From xxxxx
where 200000=>id
order by id
limit 200000
```

“OFFSET is bad for skipping previous rows.” [Online]. Available: http://Use-The-Index-Luke.com/sql/partial-results/fetch-next-page. [Accessed: 27-Dec-2016].

Thank you in advance

Issue Links

• links to: GitHub Pull Request #1376
• links to: GitHub Pull Request #1376