Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-6682

[C#] Arrow R/C++ hangs reading binary file generated by C#

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.14.1
    • Fix Version/s: 0.15.0
    • Component/s: C#

      Description

      I get random hangs on arrow_read in R (windows) when using a very large file (10-12gb). (the file has 37 columns)

      I have memory dumps - All threads seem to be in wait handles.

      Are there debug symbols somewhere? 

      Is there a way to get the C++ code to produce diagnostic logging from R? 

       

      UPDATE: it seems that the hangs are not related to file size, row counts, or # of record batches, but rather the number of columns

        Attachments

        1. Generated_4000Batch_50Columns_100Rows_PerBatch.rar
          60 kB
          Anthony Abate
        2. Generated_4000Batch_50Columns_100Rows_PerBatch.zip
          2.98 MB
          Anthony Abate
        3. script.runner.ps1
          0.3 kB
          Anthony Abate
        4. arrow.benchmark.r
          0.3 kB
          Anthony Abate

          Issue Links

            Activity

              People

              • Assignee:
                eerhardt Eric Erhardt
                Reporter:
                abbot Anthony Abate
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m