Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-4452

[Python] Serializing sparse torch tensors

    XMLWordPrintableJSON

    Details

      Description

      Using the pytorch serialization handler on sparse Tensors:

      import torch
      i = torch.LongTensor([[0, 2], [1, 0], [1, 2]])
      v = torch.FloatTensor([3,      4,      5    ])
      tensor = torch.sparse.FloatTensor(i.t(), v, torch.Size([2,3]))
      
      pyarrow.serialization.register_torch_serialization_handlers(pyarrow.serialization._default_serialization_context)
      
      s = pyarrow.serialize(tensor, context=pyarrow.serialization._default_serialization_context) 

      Produces this result:

      TypeError: can't convert sparse tensor to numpy. Use Tensor.to_dense() to convert to a dense tensor first.

      We should provide a way to serialize sparse torch tensors, especially now that we are getting support for sparse Tensors.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                pcmoritz Philipp Moritz
                Reporter:
                pcmoritz Philipp Moritz
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 20m
                  1h 20m