contrib 2.0 infrastructure

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

contrib 2.0 infrastructure

panyasan
I have just updated two contributions (will announce it when they are in the contrib demobrowser) and it made me think about why so few contributions have been updated to the new contrib infrastructure. I think that the qooxdoo team should invest some more time into this to get more contrib authors on board.

- the documentation at  http://qooxdoo.org/contrib is not up-to-date. The information on the new infrastructure is in the manual (http://manual.qooxdoo.org/3.5.x/pages/development/contrib.html). However, this is quite hard to find and will not show up when you search "qooxdoo contrib" on google, for example.  

- I found the process of having to clone the registry, apply changes and create a pull request quite complicated. Probably it will be much quicker and easier the next time, but it seems to me that this will scare off newcomers. In particular, having to create a checksum manually seemed to be strange.

- Keeping redundant information in three different places (contrib wiki, repository Manifest, registry manifest) not only violates the DRY principle :-) but also is cumbersome and error-prone. Ideally, all information on the contrib would be kept in one place - the location of the specific version of the published contrib. From this info, all other info can be generated (contrib catalog, demo, registry, etc.). I understand if there are technical obstacles to this, but welcome if the redundancy could be reduced. NPM is a perfect example how to solve the versioning, dependency, discovery, and publication problems.

I know that the current solution is a very flexible one that allows to integrate contribs from very different sources (github, sourceforge, ...), and you have thought long and hard about it. But I do think you need to supply more automation tools to the authors - through the generator or a web application. I've seen in the qooxdoo repo that a web UI is in planning - any info on the ETA?

Finally, I think some active curation would create more value to the whole contrib infrastructure - there is author info in the contribs - why don't you write to authors of the outdated contribs and invite them to update their contribution to the current qooxdoo version - that wasn't hard (almost no changes except changing #asset to @asset)...

My two cents ... any thoughts?

Christian
Reply | Threaded
Open this post in threaded view
|

Re: contrib 2.0 infrastructure

Richard Sternagel
Hey Christian,

first of all thanks a lot for your feedback - it's even more valuable
cause in my mind all of your points are valid and true! :)


* documentation scattered

Short term solution:
I've added a note regarding the more current manual page about contribs.

Long term solution:
I've filed a bug (#8263) for an overall overhaul of the contrib wiki
section.


* process complicated

It's right that it takes now more effort to distribute your contrib with
the contrib catalog. But it also gives you more freedom (e.g. where your
code is hosted), and will make it easier to find new contribs in the
future, when the web interface [1] is in place.

The checksum is indeed a high barrier to entry for newcomers and we
consider to remove it (we favored security over convenience in the
past), at least for the master/trunk version of the contribs. We should
also add more documentation on how to generate one.


* violating DRY

You're right on this too regarding the current state (contrib wiki as it
exists now in co-existence with the catalog), we had not the time to
accomplish the next milestone, but we had DRY in mind:

- the idea is to boil down the contrib wiki section once the web
interface [1] for the contribs is live.

- the web interface consumes (i.e. reuses) the catalog content and
besides that should have no manually maintained content.

- the catalog content, i.e. the "Manifest.json" files, should only
slightly differ from the "Manifest.json" file the author maintains with
his contrib. So on the one hand the contrib author has to deal with the
catalog but on the other hand he/she also saves time because there would
be no need to edit the wiki page of all available contribs manually [2]
anymore.


* active curation

Good point, we'll consider that in the future. We wanted to first launch
the web interface in order to encourage contrib authors to update their
contribs.

[1]
https://github.com/qooxdoo/contrib-platform/tree/master/catalog-webinterface
[2] http://qooxdoo.org/contrib/project

Regards
Richard

------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/NeoTech
_______________________________________________
qooxdoo-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Reply | Threaded
Open this post in threaded view
|

Re: contrib 2.0 infrastructure

panyasan
Hi Richard,

thank you for the detailed information. I am looking forward to seeing the new web interface - when will that be online?

One more question: how long does it take until a contribution's demo is in the demo browser? Is that an automated process or do you have to manually upload the demos?

Best,
Christian


2014-04-16 12:20 GMT+02:00 Richard Sternagel [via qooxdoo] <[hidden email]>:
Hey Christian,

first of all thanks a lot for your feedback - it's even more valuable
cause in my mind all of your points are valid and true! :)


* documentation scattered

Short term solution:
I've added a note regarding the more current manual page about contribs.

Long term solution:
I've filed a bug (#8263) for an overall overhaul of the contrib wiki
section.


* process complicated

It's right that it takes now more effort to distribute your contrib with
the contrib catalog. But it also gives you more freedom (e.g. where your
code is hosted), and will make it easier to find new contribs in the
future, when the web interface [1] is in place.

The checksum is indeed a high barrier to entry for newcomers and we
consider to remove it (we favored security over convenience in the
past), at least for the master/trunk version of the contribs. We should
also add more documentation on how to generate one.


* violating DRY

You're right on this too regarding the current state (contrib wiki as it
exists now in co-existence with the catalog), we had not the time to
accomplish the next milestone, but we had DRY in mind:

- the idea is to boil down the contrib wiki section once the web
interface [1] for the contribs is live.

- the web interface consumes (i.e. reuses) the catalog content and
besides that should have no manually maintained content.

- the catalog content, i.e. the "Manifest.json" files, should only
slightly differ from the "Manifest.json" file the author maintains with
his contrib. So on the one hand the contrib author has to deal with the
catalog but on the other hand he/she also saves time because there would
be no need to edit the wiki page of all available contribs manually [2]
anymore.


* active curation

Good point, we'll consider that in the future. We wanted to first launch
the web interface in order to encourage contrib authors to update their
contribs.

[1]
https://github.com/qooxdoo/contrib-platform/tree/master/catalog-webinterface
[2] http://qooxdoo.org/contrib/project

Regards
Richard

------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/NeoTech
_______________________________________________
qooxdoo-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel



If you reply to this email, your message will be added to the discussion below:
http://qooxdoo.678.n2.nabble.com/contrib-2-0-infrastructure-tp7585559p7585561.html
To unsubscribe from contrib 2.0 infrastructure, click here.
NAML

Reply | Threaded
Open this post in threaded view
|

Re: contrib 2.0 infrastructure

Richard Sternagel
Hi Christian,

 > [...] new web interface - when will that be online?

As soon as it's prioritized again. Can't give you a date.

 > [...] how long does it take until a contribution's demo
 > is in the demo browser? Is that an automated process or do you have
 > to manually upload the demos?

Most of the time we trigger it manually when there is activity
in the old SourceForge qooxdoo-contrib repository. Only demos of
contribs which still reside in the old SourceForge qooxdoo-contrib
repository are build.

However, we encourage contrib authors to host their own demos (and
include links to them in the "Manifest.json") so we can hyperlink them
within the new web interface.

Regards
Richard

------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/NeoTech
_______________________________________________
qooxdoo-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Reply | Threaded
Open this post in threaded view
|

Re: contrib 2.0 infrastructure

panyasan
Ah ok. I don't think that external demos are  good idea - again, I think you create a stumbling block that takes away an incentive to pontential contrib authors. If your goal is to have a lot of high-quality contribs that fill the infrastructure with life that you have put so much thought and work into, the infrastructure should take care of everything that can be automated. The tarball contains the demo code, too, so it shouldn't be hard to create a nightly job that generates the contrib browser code. You can even use the qooxdoo tools to see if the demo works or breaks, and automatically notify the author. That is, honestly, what I would expect from such a framework, and to which I am motivated to contribute. Maybe I am asking for too much, but in my humble opinion, this would be one element in motivating people to contribute...

Best,
Christian




2014-04-16 16:19 GMT+02:00 Richard Sternagel [via qooxdoo] <[hidden email]>:
Hi Christian,

 > [...] new web interface - when will that be online?

As soon as it's prioritized again. Can't give you a date.

 > [...] how long does it take until a contribution's demo
 > is in the demo browser? Is that an automated process or do you have
 > to manually upload the demos?

Most of the time we trigger it manually when there is activity
in the old SourceForge qooxdoo-contrib repository. Only demos of
contribs which still reside in the old SourceForge qooxdoo-contrib
repository are build.

However, we encourage contrib authors to host their own demos (and
include links to them in the "Manifest.json") so we can hyperlink them
within the new web interface.

Regards
Richard

------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/NeoTech
_______________________________________________
qooxdoo-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel



If you reply to this email, your message will be added to the discussion below:
http://qooxdoo.678.n2.nabble.com/contrib-2-0-infrastructure-tp7585559p7585563.html
To unsubscribe from contrib 2.0 infrastructure, click here.
NAML

Reply | Threaded
Open this post in threaded view
|

Re: contrib 2.0 infrastructure

Richard Sternagel
Hi Christian,

interesting point of view. It surely would motivate contrib
authors if all such automation would exist and I'm not denying the
usefulness of such an infrastructure. But if I would be a contrib author
(which I am not) I would also welcome as much freedom as I can get, i.e.
I wouldn't need that.

IMO the contrib demo browser is a good tool, but it also brings
obligations with it. For example a rigid demo directory structure.
Furthermore there is a dependency between us and you, which shouldn't be
there. If you want to change your demo you should be able to do so and
publish it right away. There should be no established process, where you
have to wait for a cronjob or the like.

You've mentioned npm: I think the success of npm and Node.js lies in
it's simplicity, there is not much which could be removed because barely
anything is superfluous.

Also keep in mind, that we have to balance our time between moving the
framework itself forward and amongst other things building an
infrastructure for contributors.

In my mind we attract more people with a great framework itself than
with a great infrastructure around it. The latter is without doubt
important but the former is crucial.

To sum up in the context of contribs: I would rather remove
cumbersomeness than write code for more convenience. :)

Best
Richard

------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/NeoTech
_______________________________________________
qooxdoo-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Reply | Threaded
Open this post in threaded view
|

Re: contrib 2.0 infrastructure

panyasan
… as a final note: I had in mind something along the lines of (pseudocode)

for each (contrib-catalog/contributions/*/*/Manifest.json) as manifestObject
 if ! manifestObject.buildDemo skip to next contrib
 download and unzip file at ${manifestObject.download}
 demoPath = "${manifestObject.name}/${manifestObject.version}/demo/default/“
 if demoPath exists in the file system
   cd demopath
   ./generate.py build
   if exit error, send email to authors „generation of contrib failed, please fix or remove buildDemo flag“
      and skip to next contrib
   open application in phantom browser and check for javascript errors
   if javascript errors, send email to authors „demo wasn’t working, please fix or remove buildDemo flag“
      and skip to next contrib
   copy build directory to demo browser and add to demo browser tree data

I am unsure about the ax-phantom stuff since I know nothing about that - wonder if it is possible to load a complete app (not a test suite) and check for errors.

I am sure I underestimate the complexity of what I would love to have, so I am not expecting that this will happen. I just think it would be a wonderful way of rewarding the work of people who maybe have the time to work occasionally on single contributions, but not more.

But let that be enough on the subject - just wanted to let you know what I was thinking about.

Happy Easter!

Christian
Reply | Threaded
Open this post in threaded view
|

Re: contrib 2.0 infrastructure

panyasan
In reply to this post by Richard Sternagel
Just for fun and to refresh my bash skill, I wrote this little script to build all demos (even the new ones hosted on github). It’s not as nice as the contrib demo browser, but it let’s you try all available contributions with their respective qooxdoo versions:

https://gist.github.com/cboulanger/11186595

See the result here (won’t be there for long, though:)

https://www.panya.de/qooxdoo-contrib/

Cheers,
Christian


Am 17.04.2014 um 19:10 schrieb Boulanger Christian <[hidden email]>:

> … as a final note: I had in mind something along the lines of (pseudocode)
>
> for each (contrib-catalog/contributions/*/*/Manifest.json) as manifestObject
> if ! manifestObject.buildDemo skip to next contrib
> download and unzip file at ${manifestObject.download}
> demoPath = "${manifestObject.name}/${manifestObject.version}/demo/default/“
> if demoPath exists in the file system
>   cd demopath
>   ./generate.py build
>   if exit error, send email to authors „generation of contrib failed, please fix or remove buildDemo flag“
>      and skip to next contrib
>   open application in phantom browser and check for javascript errors
>   if javascript errors, send email to authors „demo wasn’t working, please fix or remove buildDemo flag“
>      and skip to next contrib
>   copy build directory to demo browser and add to demo browser tree data
>
> I am unsure about the ax-phantom stuff since I know nothing about that - wonder if it is possible to load a complete app (not a test suite) and check for errors.
>
> I am sure I underestimate the complexity of what I would love to have, so I am not expecting that this will happen. I just think it would be a wonderful way of rewarding the work of people who maybe have the time to work occasionally on single contributions, but not more.
>
> But let that be enough on the subject - just wanted to let you know what I was thinking about.
>
> Happy Easter!
>
> Christian