Uploaded image for project: 'Thrift'
  1. Thrift
  2. THRIFT-4677

UnicodeDecodeError in Python3

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Python - Library
    • Labels:
      None
    • Environment:

      Operating System: Windows 10 Pro (Simplified Chinese)

      Python Interpreter: Python 3.6.6

      osquery Version: 3.3.0

      osquery-python Version: 3.0.5

       

      Description

      This is an issue occurred when using osquery-python (Python binding of osquery by Facebook).

      When querying, UnicodeDecodeError raised with error message: "'utf-8' codec can't decode byte 0xc3 in position 0: invalid continuation byte" from thrift.compat.binary_to_str, which is because the encoding of bin_val parameter should be "gbk".

      Possible approaches are:

      • add a parameter for user to determine encodings
      • get the system encoding through locale.getpreferredencoding()
      • call bin_val.decode with errors='replace' or errors='ignore' parameter
      • introduce chardet to try and resolve encoding problems

      The attachment is my hack solution to this issue (through not perfect).

        Attachments

        1. osquery_all_mp.py
          3 kB
          Jarry Shaw
        2. compat.py
          1 kB
          Jarry Shaw

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              jarryshaw Jarry Shaw
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:

                Time Tracking

                Estimated:
                Original Estimate - 0.5h
                0.5h
                Remaining:
                Remaining Estimate - 0.5h
                0.5h
                Logged:
                Time Spent - Not Specified
                Not Specified