Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-8453

[Integration][Go] Recursive nested types unsupported

    XMLWordPrintableJSON

Details

    Description

      The Go JSON integration implementation doesn't support recursive nested types, e.g. "list(list(int32))".

      Here is an example traceback when Go is the consumer:

      panic: runtime error: index out of range
      
      goroutine 1 [running]:
      github.com/apache/arrow/go/arrow/internal/arrjson.dtypeFromJSON(0xc00001687c, 0x4, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
      	/arrow/go/arrow/internal/arrjson/arrjson.go:238 +0x1710
      github.com/apache/arrow/go/arrow/internal/arrjson.dtypeFromJSON(0xc000016858, 0x4, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
      	/arrow/go/arrow/internal/arrjson/arrjson.go:238 +0x838
      github.com/apache/arrow/go/arrow/internal/arrjson.fieldFromJSON(0xc000016860, 0xb, 0xc000016858, 0x4, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
      	/arrow/go/arrow/internal/arrjson/arrjson.go:309 +0xb5
      github.com/apache/arrow/go/arrow/internal/arrjson.fieldsFromJSON(0xc0000ca280, 0x4, 0x4, 0x0, 0x6f6d08, 0xc00000db60)
      	/arrow/go/arrow/internal/arrjson/arrjson.go:301 +0xfe
      github.com/apache/arrow/go/arrow/internal/arrjson.schemaFromJSON(0xc0000ca280, 0x4, 0x4, 0xc00000db60)
      	/arrow/go/arrow/internal/arrjson/arrjson.go:274 +0x3f
      github.com/apache/arrow/go/arrow/internal/arrjson.NewReader(0x5b4700, 0xc00000e028, 0x0, 0x0, 0x0, 0x0, 0x0, 0xd0)
      	/arrow/go/arrow/internal/arrjson/reader.go:56 +0x13d
      main.validate(0x7ffbffffc819, 0x37, 0x7ffbffffc857, 0x26, 0x4acf01, 0x0, 0x0)
      	/arrow/go/arrow/ipc/cmd/arrow-json-integration-test/main.go:181 +0x1c8
      main.runCommand(0x7ffbffffc857, 0x26, 0x7ffbffffc819, 0x37, 0x7ffbffffc884, 0x8, 0xc000016101, 0xc000086260, 0x40568f)
      	/arrow/go/arrow/ipc/cmd/arrow-json-integration-test/main.go:65 +0x228
      main.main()
      	/arrow/go/arrow/ipc/cmd/arrow-json-integration-test/main.go:44 +0x24e
      

      When Go is the producer:

      panic: runtime error: index out of range
      
      goroutine 1 [running]:
      github.com/apache/arrow/go/arrow/internal/arrjson.dtypeFromJSON(0xc00001687c, 0x4, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
      	/arrow/go/arrow/internal/arrjson/arrjson.go:238 +0x1710
      github.com/apache/arrow/go/arrow/internal/arrjson.dtypeFromJSON(0xc00001686c, 0x4, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
      	/arrow/go/arrow/internal/arrjson/arrjson.go:238 +0x838
      github.com/apache/arrow/go/arrow/internal/arrjson.fieldFromJSON(0xc000016860, 0xb, 0xc00001686c, 0x4, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
      	/arrow/go/arrow/internal/arrjson/arrjson.go:309 +0xb5
      github.com/apache/arrow/go/arrow/internal/arrjson.fieldsFromJSON(0xc0000ca280, 0x4, 0x4, 0x0, 0x6f6d08, 0xc00000db60)
      	/arrow/go/arrow/internal/arrjson/arrjson.go:301 +0xfe
      github.com/apache/arrow/go/arrow/internal/arrjson.schemaFromJSON(0xc0000ca280, 0x4, 0x4, 0xc00000db60)
      	/arrow/go/arrow/internal/arrjson/arrjson.go:274 +0x3f
      github.com/apache/arrow/go/arrow/internal/arrjson.NewReader(0x5b4700, 0xc00000e028, 0x0, 0x0, 0x0, 0x0, 0x0, 0xcc37a1760fc5b719)
      	/arrow/go/arrow/internal/arrjson/reader.go:56 +0x13d
      main.cnvToARROW(0x7ffbffffc814, 0x37, 0x7ffbffffc852, 0x26, 0x4acf01, 0x0, 0x0)
      	/arrow/go/arrow/ipc/cmd/arrow-json-integration-test/main.go:137 +0x319
      main.runCommand(0x7ffbffffc852, 0x26, 0x7ffbffffc814, 0x37, 0x7ffbffffc87f, 0xd, 0xc000016101, 0xc000086260, 0x40568f)
      	/arrow/go/arrow/ipc/cmd/arrow-json-integration-test/main.go:63 +0x172
      main.main()
      	/arrow/go/arrow/ipc/cmd/arrow-json-integration-test/main.go:44 +0x24e
      

      Attachments

        Issue Links

          Activity

            People

              zeroshade Matthew Topol
              apitrou Antoine Pitrou
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 50m
                  1h 50m