Re: [Gluster-devel] [Gluster-users] uWSGI plugin and some question

[Anand Avati]

On Mon, Jul 29, 2013 at 10:55 PM, Anand Avati <address@hidden> wrote:

On Mon, Jul 29, 2013 at 8:36 AM, Roberto De Ioris <address@hidden> wrote:

Hi everyone, i have just committed a plugin for the uWSGI application server
for exposing glusterfs filesystems using the new native api:

https://github.com/unbit/uwsgi-docs/blob/master/GlusterFS.rst

Currently it is very simple, but works really well.

I have studied the whole api, and i have two questions:

why there is no glfs_stat_async() ?

if i understand the code well, even stat() is a blocking operation.

Can you show some code in uwsgi which makes use of asynchronous stat calls? Adding an async stat call in gfapi is not hard, but the use case hasn't been clear.

 

My objective is avoiding the use of threads and processes and use the
uWSGI async api to implement a non blocking-approach (mixable with other
engines like gevent or Coro::AnyEvent)

Are there any requirements that the callback happen only in specific threads? That is typically a common requirement, and the async callbacks would end up requiring special "wiring" to bring the callbacks to desired threads. But I guess that wiring would already be done with the IO callbacks anyways in your case.

Do you have some prototype of the module using gfapi out somewhere? I'm hoping to understand the use case of gfapi and see if something can be done to make it integrate with Coro::AnyEvent more "naturally".

I am assuming the module in question is this - https://github.com/unbit/uwsgi/blob/master/plugins/glusterfs/glusterfs.c. I see that you are not using the async variants of any of the glfs calls so far. I also believe you would like these "synchronous" calls to play nicely with Coro:: by yielding in a compatible way (and getting woken up when response arrives in a compatible way) - rather than implementing an explicit glfs_stat_async(). The ->request() method does not seem to be be naturally allowing the use of "explictly asynchronous" calls within.

Can you provide some details of the event/request management in use? If possible, I would like to provide hooks for yield and wakeup primitives in gfapi (which you can wire with Coro:: or anything else) such that these seemingly synchronous calls (glfs_open, glfs_stat etc.) don't starve the app thread without yielding.

I can see those hooks having a benefit in the qemu gfapi driver too, removing a bit of code there which integrates callbacks into the event loop using pipes.

Avati

 

Another thing is the bison/yacc nameclash. uWSGI allows you to load
various external libraries and the use of the default 'yy' prefix causes
nameclashes with common libraries (like matheval).

I understand that matheval too should choose a better approach, but why
not prefixing it like glusterfsyy ? This would avoid headaches, even for
when people will start using the library in higher level languages.

Currently i have tried the YFLAGS env var hack for ./configure but it did
not work (i am using bison)

YFLAGS="-Dapi.prefix=glusterfsyy -d" ./configure --prefix=/opt/glusterfs/

Hmm, this is nice to get fixed. Do you already have a patch which you have used (other than just the "technique" shown above)?

Thanks!

Avati