Re: [Fab-user] How important is making fabric parallel to everyone else?

[Jorge Vargas]

Jumping late on this. I'm really interested in this feature but for
another reason. And in fact I'm using fabric for something a little
different than automatic deployment. I'm writing a set of tools to
automate several annoying tasks, so rather than building one product
(like you will do on say deployment your facefront web application)
I'm building several that interject in different ways.

The problem is these servers used to be very disparate and now we are
trying to get them into the same/similar setup, just to get you a
picture we have a ton of OS packages, our internal packages, compiled
by hand, easy_installed, gems, etc. software which varies from server
to server in almost 100 boxes.

My next step will be to write some sort of "query tool" which I can
start running on all the servers to get they stats, for example how
many servers have "python25" installed and in which places and by who?
So basically what I'm looking for is more like a "fire and forget"
type of executing system that will report back on errors rather than
exit (as it happens now), perhaps the real solution will be to write
some module that will invoke fabric instead of trying to create the
"modes" approach it used to have. Something like a "fabric-running"
just like a webserver will spawn threads or processes to server
different requests of the same app.

On Sun, Aug 9, 2009 at 11:32 AM, Jeff Forcier<address@hidden> wrote:
> Just a quick note to anyone with an interest in this topic, it was
> entered into Fabric's issue tracker a wile back as #19, with a high
> priority:
>
>    http://code.fabfile.org/issues/show/19
>
> I've also linked back to this thread for future reference.
>
> Apologies for thread necromancy, I'm going through all my marked
> emails and didn't want to leave this hanging :)
>
> Best,
> Jeff
>
> On Sun, Jul 19, 2009 at 6:55 AM, Christian Vest
> Hansen<address@hidden> wrote:
>> This is a concern for me too. A setting to limit the parallelism to a
>> fixed number of concurrent processes or threads would be quite useful
>> in my opinion.
>>
>> On Fri, Jul 17, 2009 at 9:30 PM, Xinan Wu<address@hidden> wrote:
>>> I am interested, and my usage case is actually a little bit more
>>> complicated: We have many machines in a production cluster. We don't
>>> want to deploy all in parallel, but it's also very inefficient to do
>>> it completely in series. So we deploy four at a time (pull out from
>>> load balancer, wait a minute, deploy, restart, put back in load
>>> balancer). After all this is done, we send a notification email. This
>>> whole sequence is too fancy to implement with fabric but we managed to
>>> write everything in fabfile with fabric 0.1.1 (basically fab call
>>> os.system to start fab subprocess by threads and then join()
>>> afterwards.) However I found it'd be very difficult (if not
>>> impossible) to implement it with the new fabric 0.9 unless I change
>>> fabric code or use a separate wrapper python script.
>>>
>>> I think people are talking about adding parallel execution in fabric
>>> but it's not very straightforward.
>>>
>>> On Fri, Jul 17, 2009 at 11:43 AM, Matthew Wilson<address@hidden> wrote:
>>>> I was surprised when I saw how fabric iterated through all the hosts
>>>> in sequence rather than in parallel.  For more than a few boxes and
>>>> for a fairly long set of tasks, my upgrades are going to take a long
>>>> time.
>>>>
>>>> Am I alone here?
>>>>
>>>> I've got some vague ideas about how to use subprocess or
>>>> multiprocessing to add parallel work to fabric.  Anyone already
>>>> working on this?  Maybe we could get some of this part figured out
>>>> ahead of time.
>>>>
>>>> Matt
>>>>
>>>> --
>>>> Matthew Wilson
>>>> address@hidden
>>>> http://tplus1.com
>>>>
>>>>
>>>> _______________________________________________
>>>> Fab-user mailing list
>>>> address@hidden
>>>> http://lists.nongnu.org/mailman/listinfo/fab-user
>>>>
>>>
>>>
>>> _______________________________________________
>>> Fab-user mailing list
>>> address@hidden
>>> http://lists.nongnu.org/mailman/listinfo/fab-user
>>>
>>
>>
>>
>> --
>> Venlig hilsen / Kind regards,
>> Christian Vest Hansen.
>>
>>
>> _______________________________________________
>> Fab-user mailing list
>> address@hidden
>> http://lists.nongnu.org/mailman/listinfo/fab-user
>>
>
>
> _______________________________________________
> Fab-user mailing list
> address@hidden
> http://lists.nongnu.org/mailman/listinfo/fab-user
>