[HUDI-485] Check for where clause is wrong in HiveIncrementalPuller

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Unresolved
• Fix Version/s: 0.11.0
• Component/s: incremental-query
• Labels: new-to-hudi pull-request-available query-eng sev:high
• Assignee: Pratyaksh Sharma
• Reporter: Pratyaksh Sharma
• Created: 30/Dec/19 14:59
• Updated: 16/Jan/22 06:29

Description

HiveIncrementalPuller checks the clause in incrementalSqlFile like this -> 

if (!incrementalSQL.contains("`_hoodie_commit_time` > '%targetBasePath'"))

{ LOG.info("Incremental SQL : " + incrementalSQL + " does not contain `_hoodie_commit_time` > %targetBasePath. Please add " + "this clause for incremental to work properly."); throw new HoodieIncrementalPullSQLException( "Incremental SQL does not have clause `_hoodie_commit_time` > '%targetBasePath', which " + "means its not pulling incrementally"); }

Basically we are trying to add a placeholder here which is later replaced with config.fromCommitTime here - 

incrementalPullSQLtemplate.add("incrementalSQL", String.format(incrementalSQL, config.fromCommitTime));

Hence, the above check needs to replaced with `_hoodie_commit_time` > %s

Issue Links

• links to: GitHub Pull Request #2768