Require a "live" form; what are my options?

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Require a "live" form; what are my options?

Nick Watkins
I can't seem to find anything regarding the subject, or perhaps I'm not searching for the right thing. For an application I'm developing, I require a form whose elements are connected to data in a database. Changes on the client side are reflected in the database in real-time, but not necessarily the other way around (simultaneous edits of the same form data by different administrators could be possible but not common, though some form of "push" would be great for this). I'm not sure what my options are other than coding the construct from scratch.

Qcl looks very promising, but I'm not sure to what degree it works and is stable. Not to mention that the documentation is sparse outside the API docs, and tutorials non-existent. If there was a barebones example of a form widget interacting with a database, that would be wonderful.

Any suggestions? Thanks, Qooxdoo community.
Reply | Threaded
Open this post in threaded view
|

Re: Require a "live" form; what are my options?

Werner Thie
Nick Watkins wrote:

> I can't seem to find anything regarding the subject, or perhaps I'm not
> searching for the right thing. For an application I'm developing, I require
> a form whose elements are connected to data in a database. Changes on the
> client side are reflected in the database in real-time, but not necessarily
> the other way around (simultaneous edits of the same form data by different
> administrators could be possible but not common, though some form of "push"
> would be great for this). I'm not sure what my options are other than coding
> the construct from scratch.
>
> Qcl looks very promising, but I'm not sure to what degree it works and is
> stable. Not to mention that the documentation is sparse outside the API
> docs, and tutorials non-existent. If there was a barebones example of a form
> widget interacting with a database, that would be wonderful.
>
> Any suggestions? Thanks, Qooxdoo community.

Hi Nick

have a peek at twisted and on top Nevow/Athena and the LivePage
examples. I coupled qooxdoo and twisted/Nevow/Athena quite loosly and
think this is the best combination since presliced bread and butter were
invented. Nevow/LivePage basically gives you a single webpage into which
you inject/remove widgets of various complexity (think qooxdoo
aggregates like a dialog. Along with the widget in the browser goes an
object on the server which exposes methods which can be called with
callRemote('doSomething', p1, p2, p3), callRemote returning immediately
with a deferred onto which you tack a callBack function being called
when the server method returns its data and an errorBack function which
is called whenever there was something not foreseen happening. Likewise
the server object can call the browser the same way anytime he likes.

So, recollecting the briefly mentioned parts in the above interaction
you get with

- twisted: one of the most mature asynchronous, coherent and tested
networking frameworks available (no its not PHP, its Python based)
implementing everything you find as protocol in the TCP/IP world and if
not, if gives you the tools to do a very clean and high performance
implementation of your own protocols. DB interaction in the same
asynchronous way is of course included, but no ORM support (there are a
few projects hanging out there trying to do so)

http://www.twistedmatrix.com

- nevow: a very comprehensive web framework sitting on top twisted,
which you almost do not use with qooxdoo. A comprehensive object
orientation which rivals and surpasses in my opinion the qooxdoo
implementation of object orientation. You can use both of these with ease

http://www.divmod.org

- nevow/LivePage: asynchronous transparent back and forth calling
between server and browser objects, error handling, JSON marshalling

If you read that far give it a try and hit http://nqx.thieprojects.ch 
which gives you a few buttons. If you hit the calculator, you see the
above mentioned technology in action. The calculator does its
calculations on the server, all this object does is sending the
keystrokes to the server via the described RPC implementation. Feel free
to poke around in the source (it's not obfuscated) and ask back if you
like what you see. The other buttons start a few experiments I did for
testing how quick I could throw together more complicated qooxdoo
aggregates (like the telinfo demo button)

HTH, Werner





------------------------------------------------------------------------------

_______________________________________________
qooxdoo-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Reply | Threaded
Open this post in threaded view
|

Re: Require a "live" form; what are my options?

Nick Watkins
Thanks for the suggestion. That project looks promising. I took a look at your example and it seems to replicate basically the functionality I'm looking for. Is there a source version you have available? The qooxdoo.js is a lot to muck through to find the "meat" of the example. The only thing I have reservations about is that my Python knowledge is fundamental at best. I was really hoping for something implemented with a PHP back-end through soemthing like JsonRPC.
Reply | Threaded
Open this post in threaded view
|

Re: Require a "live" form; what are my options?

Werner Thie
Hi Nick

The meat is not in qooxdoo.js. Without knowing all too much of qooxdoo
this demo took only a few hours to write. The source is there for the
client side, JScript can be delivered on the fly when injecting widgets.
Poke around with Firebug and you'll see the meat in the code islands
named qooxdoo.qxcalculator, web.playground, ...

For projects at the complexity level you will arrive at very early in
your project a PHP backend and 'whatever'RPC is simply not cutting to
the meat in my opinion, because of a lot of missing concepts in PHP.
Code in PHP becomes usually muddled really fast when solving that type
of problem, that there are simply no solutions around that I know of.
Usually it's resorting to Java or Python and if you want to go massively
parallel then Python and stackless are doing its job nicely as you can
see for instance with project EVE http://play.eveonline.com/en/home.aspx

So if you want to build RPC/Comet based apps in the browser I would
suggest you scan the archives of http://cometdaily.com/
You will see that the resources are few in this particular area (and as
an aside: WebSockets will not be any easier to handle, things stay
highly asynchronous). If you still feel inclined to go with what I
suggested I can point you in the right direction and help you along,
though I do not have an 'industrial grade' solution framework, you can
download.

Good luck, Werner

The server side coding of qxcalculator:

#! /usr/bin/env python
#-*- coding: iso-8859-1 -*-

"""
qxcalculator.py - a minimal Qooxdoo calculator

author     : Werner Thie, wth
last edit  : wth, 24.03.2010
modhistory :
   11.03.2010 - wth, created
"""

import sys, os

from twisted.internet import reactor
from twisted.python import log

from nevow import static, athena, loaders, inevow, url, tags as T,
inevow, guard

from nevow.page import Element

from asite.common.helpers import uc

class qxCalculator(athena.LiveElement):
   jsClass = u'qooxdoo.qxcalculator.qxCalculator'

   docFactory = loaders.xmlstr("""
     <div name='qxcalculator'
xmlns:nevow='http://nevow.com/ns/nevow/0.1' nevow:render='liveElement'>
     </div>
   """)

   def __init__(self, qooxdoopath, *a, **kw):
     super(qxCalculator, self).__init__(*a, **kw)
     self.qooxdoopath  = qooxdoopath
     self.result    = 0.0
     self.comma     = 0.0
     self.scratch   = 0.0
     self.operator  = ''
     self.opPending = False
     self.memory    = 0.0

   @athena.expose
   def keyPressed(self, key):
     if key == 'C':
       self.result  = 0.0
       self.comma   = 0.0
       self.scratch = 0.0
       self.opPending = False
     elif key in ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9']:
       if self.opPending:
         self.result = 0.0
         self.comma  = 0.0
         self.opPending = False
       if self.comma > 0.0:
         self.result = self.result + (float(key) / self.comma)
         self.comma = self.comma * 10.0
       else:
         self.result = (self.result * 10.0) + float(key)
     elif key == '.' and self.comma == 0.0:
       self.comma = 10.0
     elif key == 'MC':
       self.memory = 0.0
     elif key == 'MR':
       self.result = self.memory
     elif key == 'M+':
       self.memory += self.result
     elif key == 'M-':
       self.memory -= self.result
     elif key in ['+', '-', '*', '/', '=']:
       if self.scratch != 0.0:
         if self.operator == '+':
           self.result += self.scratch
         elif self.operator == '-':
           self.result = self.scratch - self.result
         elif self.operator == '*':
           self.result *= self.scratch
         elif self.operator == '/':
           self.result = self.scratch / self.result
         self.operator = ''
       self.opPending = True
       if key == '=':
         self.operator = ''
         self.scratch = 0.0
       else:
         self.operator = key
         self.scratch = self.result

     log.msg('---- key pressed: ', key)
     log.msg('result   : ', self.result)
     log.msg('scratch  : ', self.scratch)
     log.msg('comma    : ', self.comma)
     log.msg('operator : ', self.operator)
     log.msg('pending  : ', self.opPending)

     return self.result

   def detached(self):
     #clean up whatever needs cleaning...
     log.msg('qxCalculator Object was detached cleanly')



Nick Watkins wrote:
> Thanks for the suggestion. That project looks promising. I took a look at
> your example and it seems to replicate basically the functionality I'm
> looking for. Is there a source version you have available? The qooxdoo.js is
> a lot to muck through to find the "meat" of the example. The only thing I
> have reservations about is that my Python knowledge is fundamental at best.
> I was really hoping for something implemented with a PHP back-end through
> soemthing like JsonRPC.


------------------------------------------------------------------------------

_______________________________________________
qooxdoo-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Reply | Threaded
Open this post in threaded view
|

Re: Require a "live" form; what are my options?

Gregory Beaver
>
> Hi Nick
>
>  
<snip>

> For projects at the complexity level you will arrive at very early in
> your project a PHP backend and 'whatever'RPC is simply not cutting to
> the meat in my opinion, because of a lot of missing concepts in PHP.
> Code in PHP becomes usually muddled really fast when solving that type
> of problem, that there are simply no solutions around that I know of.
> Usually it's resorting to Java or Python and if you want to go massively
> parallel then Python and stackless are doing its job nicely as you can
> see for instance with project EVE http://play.eveonline.com/en/home.aspx
>
>  
Hi Nick and Werner,

It sounds like Nick is looking for a pretty straightforward data entry
form, not a massive online multiplayer game like Eve.  In that case,
there are no intrinsic advantages to passing everything to the server,
and in fact there may be some disadvantages.  For instance, in the
calculator demo, there is a noticeable lag between keypresses and the
numbers appearing, which would be a non-starter for my tech-unsavvy
users, particularly in a lagging network environment.  To say that PHP
can't cut it is also doing Nick a disservice, he clearly stated that his
expertise lies there.  Rather than force Nick to learn 2 completely new
toolsets (both qooxdoo and Python), it seems to me the best route would
be to help him solve the problem the fastest way possible.  As for
whether PHP is capable of solving problems, that is a useless canard.
 Having actually coded a PHP backend using a "whatever" RPC, I can say
quite definitively that it works wonderfully, efficiently, and is in
fact the easiest part of the application.  Getting the UI right is much
harder, and this is where qooxoo excels.

Nick: if you'd care to elaborate on your requirements, it might be
possible to answer your questions better.  What kind of syncing are you
needing?  You'd like data records to be updated field by field as the
user types and a field loses focus?  Do you want forms to have a "Save"
button (it sounds like not)?

Greg

------------------------------------------------------------------------------

_______________________________________________
qooxdoo-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Reply | Threaded
Open this post in threaded view
|

Re: Require a "live" form; what are my options?

Nick Watkins
Thank you both for your input. Let me take a sec to try and iterate what I'm looking for more clearly.

The only thing I'm focusing on is user interaction with the form. Greg, you're correct in that I'm looking for the form to lack a save button. When the user changes data in the form, that change is made asynchronously in the record in the database that the form represents. It doesn't matter how that change gets initiated (maybe form widgets triggering an "onblur" event, whatever). I don't necessarily need a persistent connection or socket or anything fancy like that; just a simple way to truck data from the form to the database, initiated by user interaction with individual form widgets. I'm not sure if anyone's had this requirement yet.
Reply | Threaded
Open this post in threaded view
|

Re: Require a "live" form; what are my options?

Gregory Beaver
>
> Thank you both for your input. Let me take a sec to try and iterate what I'm
> looking for more clearly.
>
> The only thing I'm focusing on is user interaction with the form. Greg,
> you're correct in that I'm looking for the form to lack a save button. When
> the user changes data in the form, that change is made asynchronously in the
> record in the database that the form represents. It doesn't matter how that
> change gets initiated (maybe form widgets triggering an "onblur" event,
> whatever). I don't necessarily need a persistent connection or socket or
> anything fancy like that; just a simple way to truck data from the form to
> the database, initiated by user interaction with individual form widgets.
> I'm not sure if anyone's had this requirement yet.
>  
Hi Nick,

This is a relatively common need, and I think qooxdoo is well-designed
for this, in my limited experience with qooxdoo.  In my app, I have a
checkbox as part of a tree item that toggles a setting on the server
through exactly what you describe:

1) listen to the "changeValue" event
2) in the event handler, contact the rpc server asynchronously with the
new value
2a) the server runs a database update and returns a result
3) with the rpc callback, check for an exception, and alert the user if
any, including timeout for network problems.

For security reasons, either use a whitelist of database tables, or
define a different callback method for each database table on the
server, i.e.

class whatever {
protected $fields = array(
   'id' => true,
   'something' => true,
   // ...
);
// set up db object here
function saveThing($field, $value, $id)
{
    if (!isset($this->fields[$field])) {
        throw new Exception("Invalid field: " . $field . ", not in table
'thing'");
    }
    // update the database here
}
// ...
}

If you're using PHP 5.3 or newer on the server, I can send you the RPC
server I wrote that makes use of the native json extension as well as
slimmer structure for better speed and native conversion of all PHP
errors and exceptions into a qooxdoo exception format.  The only thing
it doesn't do is the weird qooxdoo Date object, it expects you to
serialize it prior to transmission into a string, and to process them in
your code.  This results in MUCH smaller code, and so is worth the
tradeoff in my opinion, but won't fit everyone's needs.

In your qooxdoo code, I would probably define a mixin that contains
generic code for calling the RPC server so you can re-use the logic for
any kind of form element.

There are a lot of ways to do this on the frontend (you might instead
use a table, for instance) using the standard qooxdoo stuff.  I'd start
with a snippet from the demo browser that looks similar to what you want
and add the rpc portion to it.

One important note regarding the server-side.  If there really is a
chance of a concurrency issue, then you'd do well to add a field to each
table that defines the record version.  Thus, if Joe and Mary both grab
record version 1000, and Joe edits it, resulting in record version 1001,
and then Mary tries to edit it, you can detect this problem.  Simply add
the record version to the WHERE clause of your update statement and
monitor the affected row count.  Your update would look something like:

UPDATE thing SET field=value, recordversion=1001 WHERE id=123 AND
recordversion=1000;

If affected rows is 0, then you should alert the user to refresh their
work.  You might also want to save who edited what in the record as
well, so you can ask something like "Joe has saved a newer version of
this record, refresh or overwrite?"

This is assuming you're using a sql database.  The rules change for
nosql databases as they have this kind of concurrency check built into
the design.

Greg

------------------------------------------------------------------------------

_______________________________________________
qooxdoo-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Reply | Threaded
Open this post in threaded view
|

Re: Require a "live" form; what are my options?

Werner Thie
Hi

never thought that discussing solutions and voicing opinions would be
called 'disservice': I always thought that this is the point in having
mailing lists.

But you setting me straight I humbly apologize for having voiced my
clearly not counting opinion on this list.

Werner

Greg Beaver wrote:

>> Thank you both for your input. Let me take a sec to try and iterate what I'm
>> looking for more clearly.
>>
>> The only thing I'm focusing on is user interaction with the form. Greg,
>> you're correct in that I'm looking for the form to lack a save button. When
>> the user changes data in the form, that change is made asynchronously in the
>> record in the database that the form represents. It doesn't matter how that
>> change gets initiated (maybe form widgets triggering an "onblur" event,
>> whatever). I don't necessarily need a persistent connection or socket or
>> anything fancy like that; just a simple way to truck data from the form to
>> the database, initiated by user interaction with individual form widgets.
>> I'm not sure if anyone's had this requirement yet.
>>  
> Hi Nick,
>
> This is a relatively common need, and I think qooxdoo is well-designed
> for this, in my limited experience with qooxdoo.  In my app, I have a
> checkbox as part of a tree item that toggles a setting on the server
> through exactly what you describe:
>
> 1) listen to the "changeValue" event
> 2) in the event handler, contact the rpc server asynchronously with the
> new value
> 2a) the server runs a database update and returns a result
> 3) with the rpc callback, check for an exception, and alert the user if
> any, including timeout for network problems.
>
> For security reasons, either use a whitelist of database tables, or
> define a different callback method for each database table on the
> server, i.e.
>
> class whatever {
> protected $fields = array(
>    'id' => true,
>    'something' => true,
>    // ...
> );
> // set up db object here
> function saveThing($field, $value, $id)
> {
>     if (!isset($this->fields[$field])) {
>         throw new Exception("Invalid field: " . $field . ", not in table
> 'thing'");
>     }
>     // update the database here
> }
> // ...
> }
>
> If you're using PHP 5.3 or newer on the server, I can send you the RPC
> server I wrote that makes use of the native json extension as well as
> slimmer structure for better speed and native conversion of all PHP
> errors and exceptions into a qooxdoo exception format.  The only thing
> it doesn't do is the weird qooxdoo Date object, it expects you to
> serialize it prior to transmission into a string, and to process them in
> your code.  This results in MUCH smaller code, and so is worth the
> tradeoff in my opinion, but won't fit everyone's needs.
>
> In your qooxdoo code, I would probably define a mixin that contains
> generic code for calling the RPC server so you can re-use the logic for
> any kind of form element.
>
> There are a lot of ways to do this on the frontend (you might instead
> use a table, for instance) using the standard qooxdoo stuff.  I'd start
> with a snippet from the demo browser that looks similar to what you want
> and add the rpc portion to it.
>
> One important note regarding the server-side.  If there really is a
> chance of a concurrency issue, then you'd do well to add a field to each
> table that defines the record version.  Thus, if Joe and Mary both grab
> record version 1000, and Joe edits it, resulting in record version 1001,
> and then Mary tries to edit it, you can detect this problem.  Simply add
> the record version to the WHERE clause of your update statement and
> monitor the affected row count.  Your update would look something like:
>
> UPDATE thing SET field=value, recordversion=1001 WHERE id=123 AND
> recordversion=1000;
>
> If affected rows is 0, then you should alert the user to refresh their
> work.  You might also want to save who edited what in the record as
> well, so you can ask something like "Joe has saved a newer version of
> this record, refresh or overwrite?"
>
> This is assuming you're using a sql database.  The rules change for
> nosql databases as they have this kind of concurrency check built into
> the design.
>
> Greg
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> qooxdoo-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel


------------------------------------------------------------------------------

_______________________________________________
qooxdoo-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Reply | Threaded
Open this post in threaded view
|

Re: Require a "live" form; what are my options?

Thomas Herchenroeder
In reply to this post by Werner Thie

> If you read that far give it a try and hit http://nqx.thieprojects.ch 
> which gives you a few buttons...

Werner, going to that URL it just shows a "Usually not shown" message !?

T.

------------------------------------------------------------------------------

_______________________________________________
qooxdoo-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Reply | Threaded
Open this post in threaded view
|

Re: Require a "live" form; what are my options?

Werner Thie
Correct, I stopped the server
Werner

thron7 wrote:

>> If you read that far give it a try and hit http://nqx.thieprojects.ch 
>> which gives you a few buttons...
>
> Werner, going to that URL it just shows a "Usually not shown" message !?
>
> T.
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> qooxdoo-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel


------------------------------------------------------------------------------

_______________________________________________
qooxdoo-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel