fab-user
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Fab-user] Timeout opening channel with fabric 1.13.2


From: Jeff Forcier
Subject: Re: [Fab-user] Timeout opening channel with fabric 1.13.2
Date: Fri, 15 Jun 2018 09:17:09 -0700

Hi Rob,

Fabric 1 has its own multiprocessing setup, but it needs to do various things to the objects getting cloned across subprocess barriers - e.g. socket cleanup, fabric.state.env contents tweaking, and the like - and I'm guessing if you are doing your own multiprocessing work, you may be skipping all of that.

See e.g. some of the stuff done here: https://github.com/fabric/fabric/blob/d91b86ecc0c91357e7befe3dd5b67efc00682aeb/fabric/tasks.py#L225 - you might need to look higher up in the call stack too but that's the guts of it re: disconnecting client objects in the cache.

That said, it's possible that Fabric 1's design straight up precludes the approach you're taking - it's not thread safe, it's not greenlet/coroutine/etc async safe, etc because of the emphasis on global module state. But as you're using multiprocessing, it depends on exactly what 'async' means in this context - I'm guessing it's more about the behavior of launching the subprocesses from the master, in which case it might work fine.

I don't have a ton of spare time at the moment to get deep into this if you encounter more problems, but hopefully the above gets you pointed in the right direction!

Best,
Jeff

On Fri, Jun 15, 2018 at 8:31 AM, Rob Marshall <address@hidden> wrote:
Hi,

I have a "wrapper" for Fabric 1.13.2 that I use to make running fabric
commands a little easier in some senses. I can setup an object that
has the host_string, etc., all setup when I instantiate the object.
I'm attempting to use it to run multiple processes simultaneously on
multiple hosts. This seems to work fine for some things, but I'm
running into a problem with a particular command that I seem to be
able to run just fine by itself, i.e. not using multiprocessing, but
that gets a connection timeout when being run via
multiprocess.pool.apply_async(). Is there a known issue with running
Fabric with multiprocessing pools?

Thanks,

Rob

_______________________________________________
Fab-user mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/fab-user



--
Jeff Forcier
Unix sysadmin; Python engineer
http://bitprophet.org

reply via email to

[Prev in Thread] Current Thread [Next in Thread]