[VXQUERY-188] Fully integrate Lucene Indexing into VXQuery

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Labels: GSOC2016 HDFS Indexing Lucene XML XQuery
• Assignee: Unassigned
• Reporter: Steven Jacobs
• Created: 14/Mar/16 19:34
• Updated: 19/Sep/16 21:17

Description

Currently, the indexing project for VXQuery is able to perform two tasks:

1) Create a Lucene index from an XML collection (folder).
2) Specify in a query to use this index to perform the query.

There are several more desired capabilities to fully integrate Lucene indexing:

1) Allow updates to collection indexes (when Adding/Deleting/Modifying XML files).
2) Extend indexing to HDFS folders.
3) Benchmark the difference between index and non-index plans.
4) Enable queries to dynamically decide at runtime when to use indexes.

Issue Links

• incorporates: VXQUERY-196 Move XPath out of arguments for collection-from-index
• incorporates: VXQUERY-199 Find a way to index large XML files
• incorporates: VXQUERY-200 Index search not optimized within documents
• incorporates: VXQUERY-201 1,000,000 doc limit on index search
• incorporates: VXQUERY-202 Support variable replacement in XTest
• incorporates: VXQUERY-203 Create index from HDFS files
• incorporates: VXQUERY-204 Hide the index usage from the user
• incorporates: VXQUERY-205 Store indexes on HDFS
• incorporates: VXQUERY-197 Move lucene indexes to a single folder
• incorporates: VXQUERY-198 Create an "update-index" statement
• incorporates: VXQUERY-209 Parallel Index creation and access
• incorporates: VXQUERY-216 Create a JUnit test to test indexing code functionality
• incorporates: VXQUERY-218 Add delete-index function
• incorporates: VXQUERY-220 Add show-indexes function
• incorporates: VXQUERY-206 Benchmark index queries
• incorporates: VXQUERY-208 Lucene Index Filter for Values
• incorporates: VXQUERY-207 Data and Index Directories Set In Configuration