Uploaded image for project: 'Kudu'
  1. Kudu
  2. KUDU-1891

Uploading 100,000 rows x 20 columns results in not enough mutation buffer space when uploading data using Python

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Not A Problem
    • Affects Version/s: 1.2.0
    • Fix Version/s: n/a
    • Component/s: python
    • Labels:
      None
    • Environment:
      Ubuntu 16.04

      Description

      The table had one timestamp column and 19 single precision columns with only the timestamp as the primary key.
      The tuples were uploaded in the following way:

      table = client.table('new_table')
      session = client.new_session()
      for t in tuples[:100000]:
          session.apply(table.new_insert(t))
      

      Please note that the default flush mode in Python is manual.

      This resulted in the bellow error:

      ---------------------------------------------------------------------------
      KuduBadStatus                             Traceback (most recent call last)
      <ipython-input-8-0dc8979dc1a0> in <module>()
            2 session = client.new_session()
            3 for t in tuples[:100000]:
      ----> 4     session.apply(table.new_insert(t))
            5 
            6 try:
      
      /root/anaconda3/envs/sifr-repository/lib/python3.5/site-packages/kudu/client.pyx in kudu.client.Session.apply (kudu/client.cpp:15185)()
      
      /root/anaconda3/envs/sifr-repository/lib/python3.5/site-packages/kudu/client.pyx in kudu.client.WriteOperation.add_to_session (kudu/client.cpp:27992)()
      
      /root/anaconda3/envs/sifr-repository/lib/python3.5/site-packages/kudu/errors.pyx in kudu.errors.check_status (kudu/errors.cpp:1314)()
      
      KuduBadStatus: b'Incomplete: not enough mutation buffer space remaining for operation: required additional 225 when 7339950 of 7340032 already used'
      

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              rogerthat Roger
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: