Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-5907

RowCsvInputFormat bug on parsing tsv

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.2.0
    • Fix Version/s: 1.3.0, 1.2.1
    • Component/s: Java API
    • Labels:

      Description

      The following snippet reproduce the problem (using the attached file as input):

      char fieldDelim = '\t';
          TypeInformation<?>[] fieldTypes = new TypeInformation<?>[51];
          for (int i = 0; i < fieldTypes.length; i++) {
            fieldTypes[i] = BasicTypeInfo.STRING_TYPE_INFO;
          }
          int[] fieldMask = new int[fieldTypes.length];
          for (int i = 0; i < fieldMask.length; i++) {
            fieldMask[i] = i;
          }
          RowCsvInputFormat csvIF = new RowCsvInputFormat(new Path(testCsv), fieldTypes, "\n", fieldDelim +"", 
             fieldMask, true);
          csvIF.setNestedFileEnumeration(true);
          DataSet<Row> csv = env.createInput(csvIF);
         csv.print()
      

        Attachments

        1. test.tsv
          0.2 kB
          Flavio Pompermaier

          Issue Links

            Activity

              People

              • Assignee:
                ykt836 Kurt Young
                Reporter:
                f.pompermaier Flavio Pompermaier
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: