Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-9224

Figure out a better default float precision rule for cqlsh

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      We currently use a DEFAULT_FLOAT_PRECISION of 5 in cqlsh with formatting '%.*g' % (float_precision, val). In practice, this is way too low. For example, 12345.5 will show up as 123456. Since the float precision is used for cqlsh's COPY TO, it's particularly important that we maintain as much precision as is practical by default.

      There are some other tricky considerations, though. If the precision is too high, python will do something like this:

      > '%.25g' % (12345.5555555555555555,)
      '12345.55555555555474711582'
      

      That's not terrible, but it would be nice to avoid if we can.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            stefania Stefania Alborghetti Assign to me
            thobbs Tom Hobbs
            Stefania Alborghetti
            Tom Hobbs
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment