Issue 108228 - table: bool has negative sign when used in formula expression
table: bool has negative sign when used in formula expression
Status: RESOLVED FIXED
Product: Writer
Classification: Application
Component: ui
4.0.0-dev
All All
: P3 trivial (vote)
: 4.0.0
Assigned To: Pedro Giffuni
issues@sw
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2010-01-11 18:13 UTC by Joe Smith
Modified: 2013-07-13 14:24 UTC (History)
1 user (show)

See Also:
Issue Type: DEFECT
Latest Confirmation on: ---
Developer Difficulty: ---


Attachments
Sample document showing problem (10.18 KB, application/vnd.oasis.opendocument.text)
2010-01-11 18:14 UTC, Joe Smith
no flags Details

Note You need to log in before you can comment on or make changes to this issue.
Description Joe Smith 2010-01-11 18:13:52 UTC
Formulas in Writer may use conditional expressions, which produce a boolean
result. By convention, the true/false result is represented by the numeric
values 1 and 0. This is also the case in a Writer if a formula contains a bare
conditional expression:
1 eq 1 -> 1
1 eq 0 -> 0

However, if the conditional expression is used as part of a larger formula, and
the conditional produces a true result, the conditional result has a negative
sign in the larger expression, and causes incorrect results:
1*(1 eq 1) -> -1 (should be 1)
1-(1 eq 1) -> 2 (should be 0)

Tested in OOo 3.2rc1, 3.1.1 and 2.4.2 on Fedora Linux 11.
Comment 1 Joe Smith 2010-01-11 18:14:50 UTC
Created attachment 67127 [details]
Sample document showing problem
Comment 2 michael.ruess 2010-01-12 12:13:36 UTC
MRU->OS: see attached document; a formula like 1*(1 eq 1) will give -1 as result.
Comment 3 Oliver Specht 2010-01-12 14:37:22 UTC
->jes: What makes you think a boolean TRUE has to be +1? 
Comment 4 Joe Smith 2010-01-12 16:34:38 UTC
@os:
Of course the _test_ for T/F may check for zero/non-zero and the sign does not
matter, but when the result of the expression is used for doing math, the sign
matters a great deal.

First, it is not internally consistent: the expression returns +1 when used
alone, but -1 when used in a larger expression.

Second, it is not consistent with common use, including Calc, where boolean
expressions produce 0/1 when used in an expression.

Third, the status quo is less useful than the consistent production of 0/1:
since Writer provides no explicit conditional expression the value of the
conditional itself can be used as a workaround in some cases. Having a
consistent and familiar behavior would surely improve any such use.

Of course, fixing this would be a low priority, but at least the unexpected
behavior is on record.
Comment 5 Pedro Giffuni 2013-01-23 16:26:52 UTC
The issue is very well explained in the LibreOffice list:
http://lists.freedesktop.org/archives/libreoffice/2010-December/004303.html

"The problem behaviour occurs because the boolean expression "true"
evaluates to -1 rather than +1 in formulas in Writer. So if, for
example, you create a table formula in Writer, (Table->Formula) and
enter something like 1+(2==2), it evaluates to 0 rather than 2. It
turns out the reason for this is buried in sbxdef.hxx, where we have

// The numeric values of TRUE and FALSE
enum SbxBOOL { SbxFALSE = 0, SbxTRUE = -1 };

..."

The proposed fix is in file
sw/source/core/bastyp/calc.cxx function SwSbxValue::MakeDouble

change the comparison in 1763 so that the action happens
when GetType() is equal to either SbxSTRING or SbxBOOL
Comment 6 Pedro Giffuni 2013-01-23 23:02:46 UTC
(In reply to comment #5)
> The issue is very well explained in the LibreOffice list:
> http://lists.freedesktop.org/archives/libreoffice/2010-December/004303.html
> 
...
> 
> The proposed fix is in file
> sw/source/core/bastyp/calc.cxx function SwSbxValue::MakeDouble
> 
> change the comparison in 1763 so that the action happens
> when GetType() is equal to either SbxSTRING or SbxBOOL

Mattias Johnnson has given permission to adopt his patch under ALv2.

I have a small variant that I will be committing after I confirm it works.
Comment 7 SVN Robot 2013-01-24 02:43:26 UTC
"pfg" committed SVN revision 1437842 into trunk:
i108228 - table: bool has negative sign when used in formula expression.
Comment 8 Pedro Giffuni 2013-01-24 02:45:25 UTC
Verified to work with the test document.
Thanks to Mattias!
Comment 9 hdu@apache.org 2013-07-11 06:47:35 UTC
Adjusting the target to the release that will contain the fix.
Comment 10 Joe Smith 2013-07-13 14:24:37 UTC
Testing
AOO400m3(Build:9702)  -  Rev. 1502185
2013-07-11 08:22:42 (Thu, 11 Jul 2013) - Linux i686
on Fedora 17

Looks great: all tests in sample document give expected results now, although it requires Tools > Update > Update All (or Fields) to refresh the calculations.

Many thanks to everyone who helped fix this!