JSON decode (PHP Backend)

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

JSON decode (PHP Backend)

Peter Schneider
Hi Group,

When I input a backslash followed by double-quotas and try to send it via JSON
to my (PHP-)backend, the "decode()" Method of JSON class fails :(
The input comes from an TextField, where this has to be allowed!
The frontend side seems to encode the data correct (see below):

{"service":"test_n_debug.rpcTest","method":"echo","id":10,"params":["\\\""],"server_data":null}

JSONlint [1] also validates this as "Valid JSON". But the backend fails to
decode it.
Does anybody have the knowledge why the PHP backend chokes on this?
Any help/tip/hint would be appreciated. I don't know if this issue is present
in the other backend implementations (Java,Pearl,Pyhon) too, but should be
checked anyway.

Regards,
  Peter


[1] http://www.jsonlint.com/


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
qooxdoo-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Reply | Threaded
Open this post in threaded view
|

Re: JSON decode (PHP Backend)

Andreas Junghans
Hi,

Am 15.07.2008 um 09:59 schrieb Peter Schneider:

> When I input a backslash followed by double-quotas and try to send  
> it via JSON
> to my (PHP-)backend, the "decode()" Method of JSON class fails :(
> The input comes from an TextField, where this has to be allowed!
> The frontend side seems to encode the data correct (see below):
>
> {"service":"test_n_debug.rpcTest","method":"echo","id":10,"params":
> ["\\\""],"server_data":null}
>
> JSONlint [1] also validates this as "Valid JSON". But the backend  
> fails to
> decode it.
> Does anybody have the knowledge why the PHP backend chokes on this?
> Any help/tip/hint would be appreciated. I don't know if this issue  
> is present
> in the other backend implementations (Java,Pearl,Pyhon) too, but  
> should be
> checked anyway.

I just tested it using the Java backend, and the string is decoded  
correctly there.

Regards,

   Andreas


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
qooxdoo-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Reply | Threaded
Open this post in threaded view
|

Re: JSON decode (PHP Backend)

Peter Schneider
In reply to this post by Peter Schneider
Hi,

I've investigated a bit and found the place where the problem occurs.

JSON.php (lines 716-721):
<code>
716:  } elseif (($chrs{$c} == $top['delim']) &&
717:           ($top['what'] == SERVICES_JSON_IN_STR) &&
718:           (($chrs{$c - 1} != '\\') ||
719:           ($chrs{$c - 1} == '\\' && $chrs{$c - 2} == '\\'))) {
720:      // found a quote, we're in a string, and it's not escaped
721:      array_pop($stk);
</code>

The problem is that the expression in line 719 evaluates to true.
The line looks like it tries to handle any escaped backslash (\\) inside the
string just prior to string-end delimiter ("). But in my case it falsely
evaluates to true.
I could easily add another special check somewhere between lines 718 and 719
that handles my special case, but there should be a general solution, I think.

For the overview, here's the situation when we reach the problem:

<pre>

 +--------------
 |  +----------- $chrs{ c-2 }
 |  |  +-------- $chrs{ c-1 }
 |  |  |  +----- $chrs{ c   }
 |  |  |  |  +-- $chrs{ c+1 }
 v  v  v  v  v
[\][\][\]["]["]

</pre>

Is there anybody that knows a good solution for this? I am not happy with my
'special check' that handles just my special issue.


Peter


> Hi Group,
>
> When I input a backslash followed by double-quotas and try to send it via JSON
> to my (PHP-)backend, the "decode()" Method of JSON class fails :(
> The input comes from an TextField, where this has to be allowed!
> The frontend side seems to encode the data correct (see below):
>
> {"service":"test_n_debug.rpcTest","method":"echo","id":10,"params":["\\\""],"server_data":null}
>
> JSONlint [1] also validates this as "Valid JSON". But the backend fails to
> decode it.
> Does anybody have the knowledge why the PHP backend chokes on this?
> Any help/tip/hint would be appreciated. I don't know if this issue is present
> in the other backend implementations (Java,Pearl,Pyhon) too, but should be
> checked anyway.
>
> Regards,
>   Peter
>
>
> [1] http://www.jsonlint.com/


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
qooxdoo-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Reply | Threaded
Open this post in threaded view
|

Re: JSON decode (PHP Backend)

Helder Magalhães


Peter Schneider-2 wrote:
>
> Is there anybody that knows a good solution for this? I am not happy with
> my
> 'special check' that handles just my special issue.
>

I haven't confirmed this but... Are you experiencing this using the SVN code
or a specific release? As Andreas has said it worked for him, you might be
suffering from an issue which may already be fixed in trunk. Please post
qooxdoo release version or SVN revision.

Hope this helps,

 Helder Magalhães
--
View this message in context: http://www.nabble.com/JSON-decode-%28PHP-Backend%29-tp18459901p18461283.html
Sent from the qooxdoo-devel mailing list archive at Nabble.com.


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
qooxdoo-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Reply | Threaded
Open this post in threaded view
|

Re: JSON decode (PHP Backend)

Andreas Junghans
Hi,

Am 15.07.2008 um 11:34 schrieb Helder Magalhães:

>> Is there anybody that knows a good solution for this? I am not  
>> happy with
>> my
>> 'special check' that handles just my special issue.
>>
>
> I haven't confirmed this but... Are you experiencing this using the  
> SVN code
> or a specific release? As Andreas has said it worked for him, you  
> might be
> suffering from an issue which may already be fixed in trunk.

I was talking about the Java backend (since Peter mentioned that other  
backends should also be checked for the problem).

Regarding the PHP code, maybe the Java backend could serve as  
inspiration for a fix? The code that deals with string parsing looks  
like this:

     public String nextString(char quote) throws ParseException {
         char c;
         StringBuffer sb = new StringBuffer();
         while (true) {
             c = next();
             switch (c) {
             case 0:
             case '\n':
             case '\r':
                 throw syntaxError("Unterminated string");
             case '\\':
                 c = next();
                 switch (c) {
                 case 'b':
                     sb.append('\b');
                     break;
                 case 't':
                     sb.append('\t');
                     break;
                 case 'n':
                     sb.append('\n');
                     break;
                 case 'f':
                     sb.append('\f');
                     break;
                 case 'r':
                     sb.append('\r');
                     break;
                 case 'u':
                     sb.append((char)Integer.parseInt(next(4), 16));
                     break;
                 case 'x' :
                     sb.append((char) Integer.parseInt(next(2), 16));
                     break;
                 default:
                     sb.append(c);
                 }
                 break;
             default:
                 if (c == quote) {
                     return sb.toString();
                 }
                 sb.append(c);
             }
         }
     }

The code originally comes from json.org, and it looks simple and clean  
to me. However, I haven't looked at the PHP backend code (and I don't  
know PHP very well), so something like the above might not be  
applicable.

Regards,

   Andreas


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
qooxdoo-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Reply | Threaded
Open this post in threaded view
|

Re: JSON decode (PHP Backend)

Peter Schneider
In reply to this post by Helder Magalhães
Hi Helder,

> [...]
> I haven't confirmed this but... Are you experiencing this using the SVN code
> or a specific release?
> [...]

Sorry, forgot to mention that (as always, hmmmm):
I am using qooxdoo 0.7.4-pre (trunk of the legacy branch).


> [...] As Andreas has said it worked for him, you might be
> suffering from an issue which may already be fixed in trunk. Please post
> qooxdoo release version or SVN revision.

Andreas' Post was about the Java backend implementation, my problem seems to be
specific to (at least) the PHP backend implementation.
Unfortunately the decoding seems to be pretty different there (Java), so I was
not yet able to take any advantage from looking at the Java implementation :-/

> Hope this helps,
>
>  Helder Magalhães

Unfortunately, not ;) But thanks anyway.

  Peter


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
qooxdoo-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Reply | Threaded
Open this post in threaded view
|

Re: JSON decode (PHP Backend)

Peter Schneider
In reply to this post by Andreas Junghans
Hi Andreas,


> [...]
> I was talking about the Java backend (since Peter mentioned that other  
> backends should also be checked for the problem).
>
> Regarding the PHP code, maybe the Java backend could serve as  
> inspiration for a fix? The code that deals with string parsing looks  
> like this:
>
>      public String nextString(char quote) throws ParseException {
>          char c;
>          StringBuffer sb = new StringBuffer();
>          while (true) {
>              c = next();
>              switch (c) {
>              case 0:
>              case '\n':
>              case '\r':
>                  throw syntaxError("Unterminated string");
>              case '\\':
> [...]
>          }
>      }
>
> The code originally comes from json.org, and it looks simple and clean  
> to me. However, I haven't looked at the PHP backend code (and I don't  
> know PHP very well), so something like the above might not be  
> applicable.

Yeah, you're right. I was looking at that, too. There is a similar part of code
in the PHP implementation...
Therefore I am currently looking at some other place in the JSON.phps file. The
regular expression in line 522 might be too strict...

> Regards,
>
>    Andreas

Thanks for the hint,

  Peter


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
qooxdoo-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Reply | Threaded
Open this post in threaded view
|

Re: JSON decode (PHP Backend)

Stefan Volbers
In reply to this post by Peter Schneider
Hi Peter,

just in case you are using a PHP version of at least 5.2:
please consider letting PHP do the JSON en-/decoding natively. It might
save you a huge amount of server power, as the encoding and decoding is
then done in current PHP's own function; the JSON.phps would become
redundant.

I actually thought that the use of the native function would be
incorporated in the backend by now (hadn't checked it out in a while).
There had been a discussion on this topic many months ago in this list,
IIRC.

Anyway, my recommendation for you, in case your PHP version matches, is
to back up your backend's index.php file and then search and replace any
$json->decode with json_decode and likewise $json->encode with json_encode.

I assume that the PHP native function would be free of the mentioned
error (ain't no means to test it right now to prove that right now), so
you might get rid of the fault the easy way.

Good luck,

Stefan

Peter Schneider wrote:

> Hi Helder,
>
>> [...]
>> I haven't confirmed this but... Are you experiencing this using the SVN code
>> or a specific release?
>> [...]
>
> Sorry, forgot to mention that (as always, hmmmm):
> I am using qooxdoo 0.7.4-pre (trunk of the legacy branch).
>
>
>> [...] As Andreas has said it worked for him, you might be
>> suffering from an issue which may already be fixed in trunk. Please post
>> qooxdoo release version or SVN revision.
>
> Andreas' Post was about the Java backend implementation, my problem seems to be
> specific to (at least) the PHP backend implementation.
> Unfortunately the decoding seems to be pretty different there (Java), so I was
> not yet able to take any advantage from looking at the Java implementation :-/
>
>> Hope this helps,
>>
>>  Helder Magalhães
>
> Unfortunately, not ;) But thanks anyway.
>
>   Peter
>
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> qooxdoo-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
qooxdoo-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Reply | Threaded
Open this post in threaded view
|

Re: JSON decode (PHP Backend)

Peter Schneider
Hi Stefan,

thanks a lot for that tip! That will definitely do the trick when using PHP
version >= 5.2!

Could the attached patch be applied to the 0.7.x legacy branch, please
(Derrell?)

Unfortunately we run PHP versions less than 5.2, so the problem still appears
on those systems.
So, if anybody can still help fixing the cause in JSON.phps, please do so ;)
I am currently not 100% sure where to look at...and regex have never been on
my personal list of "things I like to analyze" ;)

Thanks again for the tip,

  Peter



> Hi Peter,
>
> just in case you are using a PHP version of at least 5.2:
> please consider letting PHP do the JSON en-/decoding natively. It might
> save you a huge amount of server power, as the encoding and decoding is
> then done in current PHP's own function; the JSON.phps would become
> redundant.
>
> I actually thought that the use of the native function would be
> incorporated in the backend by now (hadn't checked it out in a while).
> There had been a discussion on this topic many months ago in this list,
> IIRC.
>
> Anyway, my recommendation for you, in case your PHP version matches, is
> to back up your backend's index.php file and then search and replace any
> $json->decode with json_decode and likewise $json->encode with json_encode.
>
> I assume that the PHP native function would be free of the mentioned
> error (ain't no means to test it right now to prove that right now), so
> you might get rid of the fault the easy way.
>
> Good luck,
>
> Stefan
>
> Peter Schneider wrote:
>> Hi Helder,
>>
>>> [...]
>>> I haven't confirmed this but... Are you experiencing this using the SVN code
>>> or a specific release?
>>> [...]
>> Sorry, forgot to mention that (as always, hmmmm):
>> I am using qooxdoo 0.7.4-pre (trunk of the legacy branch).
>>
>>
>>> [...] As Andreas has said it worked for him, you might be
>>> suffering from an issue which may already be fixed in trunk. Please post
>>> qooxdoo release version or SVN revision.
>> Andreas' Post was about the Java backend implementation, my problem seems to be
>> specific to (at least) the PHP backend implementation.
>> Unfortunately the decoding seems to be pretty different there (Java), so I was
>> not yet able to take any advantage from looking at the Java implementation :-/
>>
>>> Hope this helps,
>>>
>>>  Helder Magalhães
>> Unfortunately, not ;) But thanks anyway.
>>
>>   Peter

Index: qooxdoo/backend/php/services/index.php
===================================================================
--- qooxdoo/backend/php/services/index.php (revision 14514)
+++ qooxdoo/backend/php/services/index.php (working copy)
@@ -497,7 +497,12 @@
     case "application/json":
         /* We found literal POSTed json-rpc data (we hope) */
         $input = file_get_contents('php://input');
-        $jsonInput = $json->decode($input);
+        if (function_exists("json_decode"))
+        {
+          $jsonInput = json_decode($input);
+        } else {
+          $jsonInput = $json->decode($input);
+        }
         break;
     
     default:
@@ -520,9 +525,16 @@
     $scriptTransportId = $_GET["_ScriptTransport_id"];
     $error->SetScriptTransportId($scriptTransportId);
     $input = $_GET["_ScriptTransport_data"];
-    $jsonInput = $json->decode(get_magic_quotes_gpc()
+    if (function_exists("json_decode"))
+    {
+      $jsonInput = json_decode(get_magic_quotes_gpc()
                                ? stripslashes($input)
                                : $input);
+    } else {
+      $jsonInput = $json->decode(get_magic_quotes_gpc()
+                                 ? stripslashes($input)
+                                 : $input);
+    }
 }
 else
 {

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
qooxdoo-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Reply | Threaded
Open this post in threaded view
|

Re: JSON decode (PHP Backend)

Derrell Lipman
In reply to this post by Stefan Volbers
On Tue, Jul 15, 2008 at 6:27 AM, Stefan Volbers <[hidden email]> wrote:

just in case you are using a PHP version of at least 5.2:
please consider letting PHP do the JSON en-/decoding natively. It might
save you a huge amount of server power, as the encoding and decoding is
then done in current PHP's own function; the JSON.phps would become
redundant.

This was discussed months ago, and I was almost certain that I had implemented it. I will revisit this!  I'll also compare the PHP parser that we're using against the latest from json.org to see what has changed.  Unless something with my plans for today changes, I should be able to work on this later today.

Derrell


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
qooxdoo-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Reply | Threaded
Open this post in threaded view
|

Re: JSON decode (PHP Backend)

Derrell Lipman
On Tue, Jul 15, 2008 at 7:59 AM, Derrell Lipman <[hidden email]> wrote:

I'll also compare the PHP parser that we're using against the latest from json.org to see what has changed.

Would you please try the attached patch and let me know if it fixes the problem.

Derrell


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
qooxdoo-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel

x.patch (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: JSON decode (PHP Backend)

Peter Schneider
Hi Derrell,

1) thank you very much!

2) The patch fixes the original problem!

3) A short (10-15 minutes) test in our application did not show any new
   problems.

4) To me, your fix looks like it make sense (yes I do read comments ;) )


Again, thank you yery much for your work,

  Peter


> On Tue, Jul 15, 2008 at 7:59 AM, Derrell Lipman <
> [hidden email]> wrote:
>
>> I'll also compare the PHP parser that we're using against the latest from
>> json.org to see what has changed.
>>
>
> Would you please try the attached patch and let me know if it fixes the
> problem.
>
> Derrell


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
qooxdoo-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Reply | Threaded
Open this post in threaded view
|

Re: JSON decode (PHP Backend)

Derrell Lipman


On Tue, Jul 15, 2008 at 10:17 AM, Peter Schneider <[hidden email]> wrote:
Hi Derrell,

1) thank you very much!

2) The patch fixes the original problem!

3) A short (10-15 minutes) test in our application did not show any new
  problems.

4) To me, your fix looks like it make sense (yes I do read comments ;) )

Thanks.  I just switched my OS environment and don't yet have a test environment re-set up for this.  If you could run demobrowser test/RPC_3.html and RPC_4.html and send me the debug output from those, I'd much appreciate it.  I want to just confirm that the basic set of functionality hasn't broken before I check this in.

Cheers,

Derrell


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
qooxdoo-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel