Issue 120847

Summary: Optimizing algorithm of creating pivot table cache when loading a xls file with pivot table
Product: Calc Reporter: Tan Li <litan.test>
Component: open-importAssignee: AOO issues mailing list <issues>
Status: CLOSED FIXED QA Contact:
Severity: Normal    
Priority: P3 CC: arielch, leiw, liushenf, pfg
Version: 3.4.0   
Target Milestone: 4.0.0   
Hardware: All   
OS: All   
Issue Type: ENHANCEMENT Latest Confirmation in: ---
Developer Difficulty: ---
Attachments:
Description Flags
patch for this issue
none
update previous patch litan.test: review?

Description Tan Li 2012-09-07 07:56:33 UTC
Optimizing algorithm of creating cache: Current algorithm will visit all cells one by one to generate cache, need binary search in each column.  New algorithm will create cache for each column
Comment 1 Tan Li 2012-09-07 08:03:05 UTC
Created attachment 79350 [details]
patch for this issue
Comment 2 Tan Li 2012-09-07 09:20:43 UTC
Created attachment 79353 [details]
update previous patch
Comment 3 SVN Robot 2012-09-07 09:27:28 UTC
"leiw" committed SVN revision 1381946 into trunk:
#i120847 Optimizing algorithm of creating pivot table cache when loading a xl...
Comment 4 Wang Lei 2012-09-07 09:29:25 UTC
Confirm it
Comment 5 Wang Lei 2012-09-07 09:30:10 UTC
Submit in revision 1381946
Comment 6 Ariel Constenla-Haile 2012-09-08 02:44:47 UTC
Reopened.

Changes break the build on Linux:

Checking DLL ../unxlngi6.pro/lib/check_libscd.so ...-rwxrwxr-x 1 buildslave20 buildslave20 62785 Sep  8 02:03 ../unxlngi6.pro/lib/libscd.so
../unxlngi6.pro/slo/column3.o: In function `ScColumn::FillDPCache(ScDPTableDataCache*, long, long, long)':
column3.cxx:(.text+0x3f08): undefined reference to `unsigned char ScDPTableDataCache::AddData<false>(long, ScDPItemData*)'
collect2: ld returned 1 exit status
dmake:  Error code 1, while making '../unxlngi6.pro/lib/libsc.so'


See bug 120854
Comment 7 SVN Robot 2012-09-12 07:56:14 UTC
"leiw" committed SVN revision 1383822 into trunk:
#i120847 Resubmit to take back some codes which is roll back to fix a build b...
Comment 8 Wang Lei 2012-09-12 08:17:29 UTC
Resubmit to take back some codes which is roll back to fix a build break bug 120854
Comment 9 Tan Li 2012-09-12 09:13:50 UTC
(In reply to comment #0)
> Optimizing algorithm of creating cache: Current algorithm will visit all
> cells one by one to generate cache, need binary search in each column.  New
> algorithm will create cache for each column

Previous algorithm use "new ScDPItemData( pDoc, nRow, nCol, nDocTab  )" for each cell to construct cache item "ScDPItemData", this will call "ScColumn::GetCell( SCROW nRow )" ,"pDoc->HasValueData( nCol, nRow, nDocTab )", "pDoc->HasData( nCol,nRow, nDocTab )" , and all these three function use binary search to locate the cell and its properties. Too many redundant search is low efficient.

In our solution, for one column, only using binary search for one time to locate cell in start row, for each "ColEntry pItems[nIndex]" in column,using "ScDPItemData( sal_uLong nNF, const String & rS, double fV, sal_uInt8 bF )",	or "ScDPItemData( const String& rS, double fV, sal_Bool bHV, const sal_uLong nNumFormat, sal_Bool bData )" for construction; and for each cell between "ColEntry pItems[nIndex]",using"ScDPItemData()" for construction, and all of these function avoid binary search. "pCell->HasValueData()", "pCell->IsBlank()" are used for replacement. ScDPTableDataCache::AddData(...) is also modified to avoid redundant number format process.
Comment 10 Pedro Giffuni 2012-09-14 19:10:10 UTC
Hello;

This change uses headers that are not available in modern versions of boost (1.48).

Boost is being updated because the newer versions have many bugfixes and work with clang. Please try to use more portable headers or revert until after we finish updating boost.
Comment 11 Pedro Giffuni 2012-09-15 13:42:36 UTC
False alarm: I just build the latest version using the external boost 1.48 and the build was successful. The problems building with an updated internal boost are unrelated. (sorry)
Comment 12 Shenfeng Liu 2012-10-10 07:26:54 UTC
set Target Milestone to AOO 3.5.0 for PM purpose.