Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 1.2
    • Fix Version/s: None
    • Labels:
    • Skill Level:
      Regular Contributors Level (Easy to Medium)

      Description

      Jiffy is a NIF for JSON serialization. I wrote it quite a while ago but have finally gotten around to writing the integration for CouchDB. This is a direct replacement for ejson which does much less parsing in C. Jiffy is as close to mochijson2 (with our custom objects) as humanly possible. At the moment there are no known differences in output. Jiffy itself has been tested fairly extensively by a couple different groups so I'm fairly confident there aren't any major bugs left.

      1. 0001-Import-Jiffy.patch
        98 kB
        Paul Joseph Davis
      2. 0002-Remove-old-ejson-app.patch
        186 kB
        Paul Joseph Davis

        Activity

        Hide
        Paul Joseph Davis added a comment -

        Le patches

        Show
        Paul Joseph Davis added a comment - Le patches
        Show
        Paul Joseph Davis added a comment - http://www.youtube.com/watch?v=NP0mQeLWCCo&feature=related
        Hide
        Benoit Chesneau added a comment -

        just tested .It need a little upgrade for last trunk. Anyway I get some errors in js tests:

        [debug] [<0.178.0>] OS Process #Port<0.2856> Input :: ["reset",

        {"reduce_limit":true,"timeout":5000}

        ]
        [debug] [<0.178.0>] OS Process #Port<0.2856> Output :: true
        [debug] [<0.178.0>] OS Process #Port<0.2856> Input :: ["add_fun","function (doc)

        {emit(doc.integer, doc.integer)};"]
        [debug] [<0.178.0>] OS Process #Port<0.2856> Output :: ["error","compilation_error","Expression does not eval to a function. ((new String(\"(function (doc) {emit(doc.integer, doc.integer)}

        \")))"]
        [error] [<0.218.0>] OS Process Error <0.178.0> :: {<<"compilation_error">>,
        <<"Expression does not eval to a function. ((new String(\"(function (doc)

        {emit(doc.integer, doc.integer)}\")))">>}
        [error] [emulator] Error in process <0.218.0> with exit value: nocatch,{<<17 bytes>>,<<108 bytes>>,[{couch_os_process,prompt,2},{couch_query_servers,proc_prompt,2},{couch_query_servers,'-start_doc_map/3-fun-0-',2},{lists,foreach,2},{couch_query_servers,start_doc_map,3},{couch_mrview_updater,start_query_server...



        =ERROR REPORT==== 30-Sep-2011::06:25:07 ===
        Error in process <0.218.0> with exit value: nocatch,{<<17 bytes>>,<<108 bytes>>,[{couch_os_process,prompt,2},{couch_query_servers,proc_prompt,2},{couch_query_servers,'-start_doc_map/3-fun-0-',2},{lists,foreach,2},{couch_query_servers,start_doc_map,3},{couch_mrview_updater,start_query_server...

        [debug] [<0.123.0>] Linked process died abnormally: <0.178.0> (reason: {{nocatch,
        {<<"compilation_error">>,
        <<"Expression does not eval to a function. ((new String(\"(function (doc) {emit(doc.integer, doc.integer)}

        \")))">>}},
        [

        {couch_os_process, prompt, 2}

        ,

        {couch_query_servers, proc_prompt, 2}

        ,

        {couch_query_servers, '-start_doc_map/3-fun-0-', 2}

        ,

        {lists, foreach, 2}

        ,

        {couch_query_servers, start_doc_map, 3}

        ,

        {couch_mrview_updater, start_query_server, 1}

        ,

        {couch_mrview_updater, map_docs, 2}

        ]})
        [debug] [<0.138.0>] Minor error in HTTP request: {<<"compilation_error">>,
        <<"Expression does not eval to a function. ((new String(\"(function (doc)

        {emit(doc.integer, doc.integer)}\")))">>}
        [debug] [<0.138.0>] Stacktrace: [{couch_mrview_util,get_view,4},
        {couch_mrview,query_view,6},
        {couch_httpd,etag_maybe,2},
        {couch_mrview_http,design_doc_view,5},
        {couch_httpd_db,do_db_req,2},
        {couch_httpd,handle_request_int,5},
        {mochiweb_http,headers,5},
        {proc_lib,init_p_do_apply,3}]
        [info] [<0.138.0>] 127.0.0.1 - - GET /test_suite_db/_design/test/_view/no_reduce?reduce=false 500
        [debug] [<0.138.0>] httpd 500 error response:
        {"error":"compilation_error","reason":"Expression does not eval to a function. ((new String(\"(function (doc) {emit(doc.integer, doc.integer)}

        \")))"}

        on view_errors for example

        same kind of error in other tests.

        Show
        Benoit Chesneau added a comment - just tested .It need a little upgrade for last trunk. Anyway I get some errors in js tests: [debug] [<0.178.0>] OS Process #Port<0.2856> Input :: ["reset", {"reduce_limit":true,"timeout":5000} ] [debug] [<0.178.0>] OS Process #Port<0.2856> Output :: true [debug] [<0.178.0>] OS Process #Port<0.2856> Input :: ["add_fun","function (doc) {emit(doc.integer, doc.integer)};"] [debug] [<0.178.0>] OS Process #Port<0.2856> Output :: ["error","compilation_error","Expression does not eval to a function. ((new String(\"(function (doc) {emit(doc.integer, doc.integer)} \")))"] [error] [<0.218.0>] OS Process Error <0.178.0> :: {<<"compilation_error">>, <<"Expression does not eval to a function. ((new String(\"(function (doc) {emit(doc.integer, doc.integer)} \")))">>} [error] [emulator] Error in process <0.218.0> with exit value: nocatch,{<<17 bytes>>,<<108 bytes>> ,[{couch_os_process,prompt,2},{couch_query_servers,proc_prompt,2},{couch_query_servers,'-start_doc_map/3-fun-0-',2},{lists,foreach,2},{couch_query_servers,start_doc_map,3},{couch_mrview_updater,start_query_server... =ERROR REPORT==== 30-Sep-2011::06:25:07 === Error in process <0.218.0> with exit value: nocatch,{<<17 bytes>>,<<108 bytes>> ,[{couch_os_process,prompt,2},{couch_query_servers,proc_prompt,2},{couch_query_servers,'-start_doc_map/3-fun-0-',2},{lists,foreach,2},{couch_query_servers,start_doc_map,3},{couch_mrview_updater,start_query_server... [debug] [<0.123.0>] Linked process died abnormally: <0.178.0> (reason: {{nocatch, {<<"compilation_error">>, <<"Expression does not eval to a function. ((new String(\"(function (doc) {emit(doc.integer, doc.integer)} \")))">>}}, [ {couch_os_process, prompt, 2} , {couch_query_servers, proc_prompt, 2} , {couch_query_servers, '-start_doc_map/3-fun-0-', 2} , {lists, foreach, 2} , {couch_query_servers, start_doc_map, 3} , {couch_mrview_updater, start_query_server, 1} , {couch_mrview_updater, map_docs, 2} ]}) [debug] [<0.138.0>] Minor error in HTTP request: {<<"compilation_error">>, <<"Expression does not eval to a function. ((new String(\"(function (doc) {emit(doc.integer, doc.integer)} \")))">>} [debug] [<0.138.0>] Stacktrace: [{couch_mrview_util,get_view,4}, {couch_mrview,query_view,6}, {couch_httpd,etag_maybe,2}, {couch_mrview_http,design_doc_view,5}, {couch_httpd_db,do_db_req,2}, {couch_httpd,handle_request_int,5}, {mochiweb_http,headers,5}, {proc_lib,init_p_do_apply,3}] [info] [<0.138.0>] 127.0.0.1 - - GET /test_suite_db/_design/test/_view/no_reduce?reduce=false 500 [debug] [<0.138.0>] httpd 500 error response: {"error":"compilation_error","reason":"Expression does not eval to a function. ((new String(\"(function (doc) {emit(doc.integer, doc.integer)} \")))"} on view_errors for example same kind of error in other tests.
        Hide
        Benoit Chesneau added a comment -

        just tested .It need a little upgrade for last trunk. Anyway I get some errors in js tests:

        [debug] [<0.178.0>] OS Process #Port<0.2856> Input :: ["reset",

        {"reduce_limit":true,"timeout":5000}

        ]
        [debug] [<0.178.0>] OS Process #Port<0.2856> Output :: true
        [debug] [<0.178.0>] OS Process #Port<0.2856> Input :: ["add_fun","function (doc)

        {emit(doc.integer, doc.integer)};"]
        [debug] [<0.178.0>] OS Process #Port<0.2856> Output :: ["error","compilation_error","Expression does not eval to a function. ((new String(\"(function (doc) {emit(doc.integer, doc.integer)}

        \")))"]
        [error] [<0.218.0>] OS Process Error <0.178.0> :: {<<"compilation_error">>,
        <<"Expression does not eval to a function. ((new String(\"(function (doc)

        {emit(doc.integer, doc.integer)}\")))">>}
        [error] [emulator] Error in process <0.218.0> with exit value: nocatch,{<<17 bytes>>,<<108 bytes>>,[{couch_os_process,prompt,2},{couch_query_servers,proc_prompt,2},{couch_query_servers,'-start_doc_map/3-fun-0-',2},{lists,foreach,2},{couch_query_servers,start_doc_map,3},{couch_mrview_updater,start_query_server...



        =ERROR REPORT==== 30-Sep-2011::06:25:07 ===
        Error in process <0.218.0> with exit value: nocatch,{<<17 bytes>>,<<108 bytes>>,[{couch_os_process,prompt,2},{couch_query_servers,proc_prompt,2},{couch_query_servers,'-start_doc_map/3-fun-0-',2},{lists,foreach,2},{couch_query_servers,start_doc_map,3},{couch_mrview_updater,start_query_server...

        [debug] [<0.123.0>] Linked process died abnormally: <0.178.0> (reason: {{nocatch,
        {<<"compilation_error">>,
        <<"Expression does not eval to a function. ((new String(\"(function (doc) {emit(doc.integer, doc.integer)}

        \")))">>}},
        [

        {couch_os_process, prompt, 2}

        ,

        {couch_query_servers, proc_prompt, 2}

        ,

        {couch_query_servers, '-start_doc_map/3-fun-0-', 2}

        ,

        {lists, foreach, 2}

        ,

        {couch_query_servers, start_doc_map, 3}

        ,

        {couch_mrview_updater, start_query_server, 1}

        ,

        {couch_mrview_updater, map_docs, 2}

        ]})
        [debug] [<0.138.0>] Minor error in HTTP request: {<<"compilation_error">>,
        <<"Expression does not eval to a function. ((new String(\"(function (doc)

        {emit(doc.integer, doc.integer)}\")))">>}
        [debug] [<0.138.0>] Stacktrace: [{couch_mrview_util,get_view,4},
        {couch_mrview,query_view,6},
        {couch_httpd,etag_maybe,2},
        {couch_mrview_http,design_doc_view,5},
        {couch_httpd_db,do_db_req,2},
        {couch_httpd,handle_request_int,5},
        {mochiweb_http,headers,5},
        {proc_lib,init_p_do_apply,3}]
        [info] [<0.138.0>] 127.0.0.1 - - GET /test_suite_db/_design/test/_view/no_reduce?reduce=false 500
        [debug] [<0.138.0>] httpd 500 error response:
        {"error":"compilation_error","reason":"Expression does not eval to a function. ((new String(\"(function (doc) {emit(doc.integer, doc.integer)}

        \")))"}

        on view_errors for example

        same kind of error in other tests.

        Show
        Benoit Chesneau added a comment - just tested .It need a little upgrade for last trunk. Anyway I get some errors in js tests: [debug] [<0.178.0>] OS Process #Port<0.2856> Input :: ["reset", {"reduce_limit":true,"timeout":5000} ] [debug] [<0.178.0>] OS Process #Port<0.2856> Output :: true [debug] [<0.178.0>] OS Process #Port<0.2856> Input :: ["add_fun","function (doc) {emit(doc.integer, doc.integer)};"] [debug] [<0.178.0>] OS Process #Port<0.2856> Output :: ["error","compilation_error","Expression does not eval to a function. ((new String(\"(function (doc) {emit(doc.integer, doc.integer)} \")))"] [error] [<0.218.0>] OS Process Error <0.178.0> :: {<<"compilation_error">>, <<"Expression does not eval to a function. ((new String(\"(function (doc) {emit(doc.integer, doc.integer)} \")))">>} [error] [emulator] Error in process <0.218.0> with exit value: nocatch,{<<17 bytes>>,<<108 bytes>> ,[{couch_os_process,prompt,2},{couch_query_servers,proc_prompt,2},{couch_query_servers,'-start_doc_map/3-fun-0-',2},{lists,foreach,2},{couch_query_servers,start_doc_map,3},{couch_mrview_updater,start_query_server... =ERROR REPORT==== 30-Sep-2011::06:25:07 === Error in process <0.218.0> with exit value: nocatch,{<<17 bytes>>,<<108 bytes>> ,[{couch_os_process,prompt,2},{couch_query_servers,proc_prompt,2},{couch_query_servers,'-start_doc_map/3-fun-0-',2},{lists,foreach,2},{couch_query_servers,start_doc_map,3},{couch_mrview_updater,start_query_server... [debug] [<0.123.0>] Linked process died abnormally: <0.178.0> (reason: {{nocatch, {<<"compilation_error">>, <<"Expression does not eval to a function. ((new String(\"(function (doc) {emit(doc.integer, doc.integer)} \")))">>}}, [ {couch_os_process, prompt, 2} , {couch_query_servers, proc_prompt, 2} , {couch_query_servers, '-start_doc_map/3-fun-0-', 2} , {lists, foreach, 2} , {couch_query_servers, start_doc_map, 3} , {couch_mrview_updater, start_query_server, 1} , {couch_mrview_updater, map_docs, 2} ]}) [debug] [<0.138.0>] Minor error in HTTP request: {<<"compilation_error">>, <<"Expression does not eval to a function. ((new String(\"(function (doc) {emit(doc.integer, doc.integer)} \")))">>} [debug] [<0.138.0>] Stacktrace: [{couch_mrview_util,get_view,4}, {couch_mrview,query_view,6}, {couch_httpd,etag_maybe,2}, {couch_mrview_http,design_doc_view,5}, {couch_httpd_db,do_db_req,2}, {couch_httpd,handle_request_int,5}, {mochiweb_http,headers,5}, {proc_lib,init_p_do_apply,3}] [info] [<0.138.0>] 127.0.0.1 - - GET /test_suite_db/_design/test/_view/no_reduce?reduce=false 500 [debug] [<0.138.0>] httpd 500 error response: {"error":"compilation_error","reason":"Expression does not eval to a function. ((new String(\"(function (doc) {emit(doc.integer, doc.integer)} \")))"} on view_errors for example same kind of error in other tests.
        Hide
        Paul Joseph Davis added a comment -

        Most odd. I think this is a bit tied into that last fix I did. Either way, this is a SM error and not a Jiffy error. I'm quite perplexed on what the interplay might be though.

        Also, the SM fix I did the other day seems like less and less of a fix.

        Show
        Paul Joseph Davis added a comment - Most odd. I think this is a bit tied into that last fix I did. Either way, this is a SM error and not a Jiffy error. I'm quite perplexed on what the interplay might be though. Also, the SM fix I did the other day seems like less and less of a fix.
        Hide
        Benoit Chesneau added a comment -

        I don't reproduce it though on latest trunk. not sure what happened. Other question I have is that you don't seem to fallback to mochijson? Or did I miss something?

        Show
        Benoit Chesneau added a comment - I don't reproduce it though on latest trunk. not sure what happened. Other question I have is that you don't seem to fallback to mochijson? Or did I miss something?
        Hide
        Benoit Chesneau added a comment -

        for teh record, full patch is here : https://github.com/benoitc/couchdb/compare/jiffy

        Show
        Benoit Chesneau added a comment - for teh record, full patch is here : https://github.com/benoitc/couchdb/compare/jiffy
        Hide
        Paul Joseph Davis added a comment -

        The fallback is based on configure declaring a compile time definition that couch_db.hrl uses to determine which JSON encoder to use.

        Show
        Paul Joseph Davis added a comment - The fallback is based on configure declaring a compile time definition that couch_db.hrl uses to determine which JSON encoder to use.
        Hide
        Dave Cottlehuber added a comment -

        From recent discussion on user@ we should get jiffy in and also confirm if we should reject JSON with duplicate names inside objects. Per [JSON spec], ` The names within an object SHOULD be unique.`.

        {{

        }}
        (term@akai)16> jiffy:decode(<< "{\"foo\":\"bar\", \"few\":\"bar\"}" >>).
        {[{<<"foo">>,<<"bar">>},{<<"few">>,<<"bar">>}]}
        
        (term@akai)17> jiffy:decode(<< "{\"foo\":\"bar\", \"foo\":\"bar\"}" >>).
        {[{<<"foo">>,<<"bar">>},{<<"foo">>,<<"bar">>}]}
        {{

        }}
        [JSON spec]: http://tools.ietf.org/html/rfc4627#section-2.2

        Show
        Dave Cottlehuber added a comment - From recent discussion on user@ we should get jiffy in and also confirm if we should reject JSON with duplicate names inside objects. Per [JSON spec] , ` The names within an object SHOULD be unique.`. {{ }} (term@akai)16> jiffy:decode(<< "{\" foo\ ":\" bar\ ", \" few\ ":\" bar\ "}" >>). {[{<< "foo" >>,<< "bar" >>},{<< "few" >>,<< "bar" >>}]} (term@akai)17> jiffy:decode(<< "{\" foo\ ":\" bar\ ", \" foo\ ":\" bar\ "}" >>). {[{<< "foo" >>,<< "bar" >>},{<< "foo" >>,<< "bar" >>}]} {{ }} [JSON spec] : http://tools.ietf.org/html/rfc4627#section-2.2

          People

          • Assignee:
            Unassigned
            Reporter:
            Paul Joseph Davis
          • Votes:
            1 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:

              Development