Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-9224

Figure out a better default float precision rule for cqlsh

    XMLWordPrintableJSON

    Details

      Description

      We currently use a DEFAULT_FLOAT_PRECISION of 5 in cqlsh with formatting '%.*g' % (float_precision, val). In practice, this is way too low. For example, 12345.5 will show up as 123456. Since the float precision is used for cqlsh's COPY TO, it's particularly important that we maintain as much precision as is practical by default.

      There are some other tricky considerations, though. If the precision is too high, python will do something like this:

      > '%.25g' % (12345.5555555555555555,)
      '12345.55555555555474711582'
      

      That's not terrible, but it would be nice to avoid if we can.

        Attachments

        1. 9224-2.1.txt
          2 kB
          Tom Hobbs

          Issue Links

            Activity

              People

              • Assignee:
                stefania Stefania Alborghetti
                Reporter:
                thobbs Tom Hobbs
                Authors:
                Stefania Alborghetti
                Reviewers:
                Tom Hobbs
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: