[LUCY-327] t/core/035-json.t aborts when compiled with lemon 3.22.0

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 0.6.1
• Fix Version/s: 0.7.0, 0.6.2
• Component/s: Perl bindings
• Assignee: Unassigned
• Reporter: Petr Pisar
• Created: 08/Feb/18 08:19
• Updated: 08/Feb/18 15:35
• Resolved: 08/Feb/18 15:35

Description

When building Lucy-0.6.1 using lemon-3.22.0, t/core/035-json.t aborts:

t/core/033-index_file_names.t ......... ok
perl: cfcore/Lucy/Util/Json/JsonParser.c:681: yy_find_shift_action: Assertion `iLookAhead < YYNTOKEN' failed.
t/core/035-json.t ..................... 
Failed 43/135 subtests 

This does not happen with lemon-3.20.1.

The generated yy_find_shift_action() function has changed like this:

@@ -617,13 +669,18 @@ static unsigned int yy_find_shift_action
   int i;
   int stateno = pParser->yytos->stateno;
  
-  if( stateno>=YY_MIN_REDUCE ) return stateno;
+  if( stateno>YY_MAX_SHIFT ) return stateno;
   assert( stateno <= YY_SHIFT_COUNT );
+#if defined(YYCOVERAGE)
+  yycoverage[stateno][iLookAhead] = 1;
+#endif
   do{
     i = yy_shift_ofst[stateno];
+    assert( i>=0 && i+YYNTOKEN<=sizeof(yy_lookahead)/sizeof(yy_lookahead[0]) );
     assert( iLookAhead!=YYNOCODE );
+    assert( iLookAhead < YYNTOKEN );
     i += iLookAhead;
-    if( i<0 || i>=YY_ACTTAB_COUNT || yy_lookahead[i]!=iLookAhead ){
+    if( yy_lookahead[i]!=iLookAhead ){
 #ifdef YYFALLBACK
       YYCODETYPE iFallback;            /* Fallback token */
       if( iLookAhead<sizeof(yyFallback)/sizeof(yyFallback[0])

I don't know if this is a bug in the lemon or Lucy. sqlite-3.22.0 changelog only mentions lemon performance improvements. I believe this is in issue in the lemon, but I think you should know about that.