Re: [Fab-user] Wait for parallel task to complete

[anatoly techtonik]

Nice hack. I will try it.


On Thu, May 10, 2012 at 11:58 PM, Morgan Goose <address@hidden> wrote:
> Just asked about this, and I was wrong. But you can also use:
>
> execute(parallel(foo), ...)
>
> as the parallel decorator can just be called as a wrapping function.
>
> -goose
>
> On Thu, May 10, 2012 at 1:54 PM, Morgan Goose <address@hidden> wrote:
>> Hmm, I think perhaps I may have been wrong that it takes all env
>> params. Will look and see if there is a bug or feature request for
>> that. But decorating @parallel should still be honored.
>>
>> -goose
>>
>> On Thu, May 10, 2012 at 11:16 AM, anatoly techtonik <address@hidden> wrote:
>>> Hi Morgan,
>>>
>>> Thanks for the advice, but keyword arguments from the environment
>>> don't work with execute(). The code below gives a traceback:
>>>
>>> @task
>>> def test():
>>>  pass
>>> @task
>>> def deploy():
>>>   execute(test, parallel=True)
>>>
>>> Traceback (most recent call last):
>>>  File "/usr/lib/python2.7/site-packages/fabric/main.py", line 712, in main
>>>    *args, **kwargs
>>>  File "/usr/lib/python2.7/site-packages/fabric/tasks.py", line 327, in 
>>> execute
>>>    results['<local-only>'] = task.run(*args, **new_kwargs)
>>>  File "/usr/lib/python2.7/site-packages/fabric/tasks.py", line 112, in run
>>>    return self.wrapped(*args, **kwargs)
>>>  File "/prdir/fabfile.py", line 95, in deploy
>>>    execute(test, parallel=True)
>>>  File "/usr/lib/python2.7/site-packages/fabric/tasks.py", line 327, in 
>>> execute
>>>    results['<local-only>'] = task.run(*args, **new_kwargs)
>>>  File "/usr/lib/python2.7/site-packages/fabric/tasks.py", line 112, in run
>>>    return self.wrapped(*args, **kwargs)
>>> TypeError: test() takes no arguments (1 given)
>>>
>>>
>>> On Wed, May 2, 2012 at 2:02 AM, Morgan Goose <address@hidden> wrote:
>>>> Notice the section in
>>>> http://docs.fabfile.org/en/1.4.1/api/core/tasks.html#fabric.tasks.execute
>>>> where it mentions that "Any other arguments or keyword arguments will
>>>> be passed verbatim into task when it is called, so execute(mytask,
>>>> 'arg1', kwarg1='value') will (once per host) invoke mytask('arg1',
>>>> kwarg1='value')."
>>>>
>>>> It is refering, albeit not clearly, to the fact that any keyword that
>>>> is given will be used as if it were a member of the env dict. So you
>>>> can use any of these env var names to specify things, eg:
>>>> parallel=True; 
>>>> http://docs.fabfile.org/en/1.4.1/usage/env.html#full-list-of-env-vars
>>>>
>>>> Also I fumbled the name of the decorator. It's serial:
>>>> http://docs.fabfile.org/en/1.4.1/api/core/decorators.html#fabric.decorators.serial
>>>>
>>>> You can also specify custom args to fab tasks:
>>>> http://docs.fabfile.org/en/1.4.1/usage/fab.html#per-task-arguments
>>>>
>>>> And with that you could make the deploy task take a role name, or list
>>>> of hosts to then use as the lists, local and remote, to use in your
>>>> script.
>>>>
>>>> -goose
>>>>
>>>> On Tue, May 1, 2012 at 12:01 PM, anatoly techtonik <address@hidden> wrote:
>>>>> Hi Morgan,
>>>>>
>>>>> I've tried your example, which is:
>>>>>
>>>>>  local = ['a','b']
>>>>>  remote = ['c','d']
>>>>>
>>>>>  def test:pass
>>>>>  def update:pass
>>>>>
>>>>> address@hidden
>>>>> address@hidden
>>>>>  def deploy():
>>>>>     execute(test, hosts=local+remote)
>>>>>     execute(update, hosts=local+remote)
>>>>>
>>>>> At first it didn't run at all with Fabric 1.4.1, unable to call
>>>>> @sequential decorator:
>>>>>
>>>>>  Traceback (most recent call last):
>>>>>    ...
>>>>>    File "/checkout83/fabfile.py", line 20, in <module>
>>>>>     address@hidden
>>>>>  NameError: name 'sequential' is not defined
>>>>>
>>>>> I commented the @sequential, after which `fab deploy` run sequentially:
>>>>>  local test
>>>>>  remote test
>>>>>  local update
>>>>>  remote update
>>>>>
>>>>> That's much better, but it still impossible to specify hosts from
>>>>> command line. `fab -H local deploy` still runs additional command on
>>>>> the remote, and `fab -H local,remote deploy` run everything twice on
>>>>> both hosts.
>>>>>
>>>>> I couldn't find parallel among params for execute -
>>>>> http://docs.fabfile.org/en/1.4.1/api/core/tasks.html#fabric.tasks.execute
>>>>> so I've just decorated functions:
>>>>>
>>>>>  local = ['a','b']
>>>>>  remote = ['c','d']
>>>>>
>>>>> address@hidden
>>>>> address@hidden
>>>>>  def test:pass
>>>>> address@hidden
>>>>> address@hidden
>>>>>  def update:pass
>>>>>
>>>>> address@hidden
>>>>>  def deploy():
>>>>>     execute(test, hosts=local+remote)
>>>>>     execute(update, hosts=local+remote)
>>>>>
>>>>> I must admit that although not ideally, but this works. I just need to
>>>>> make sure that deploy is always called with empty host list and I need
>>>>> to find a way to specify hosts from command line for the subtasks.
>>>>> --
>>>>> anatoly t.
>>>>>
>>>>>
>>>>> On Fri, Apr 27, 2012 at 11:48 PM, Morgan Goose <address@hidden> wrote:
>>>>>> The example I gave you and how to run it did all of those things, sans
>>>>>> the parallel. Execute has parallel as a param. Use that in your master
>>>>>> task's executes, and you'll have everything. If that still isn't what
>>>>>> you're looking for, we're going to need more information. Eg, like you
>>>>>> did before where you said how you're seeing it run, and how you'd
>>>>>> instead like it to run, complete with how you ran it with fab, and how
>>>>>> the fabfile looks.
>>>>>>
>>>>>> -goose
>>>>>>
>>>>>> On Thu, Apr 26, 2012 at 10:32 PM, anatoly techtonik <address@hidden> 
>>>>>> wrote:
>>>>>>> Thanks for the explanation. I read the tutorial. Still I see I can't
>>>>>>> construct the system that will satisfy the following usability
>>>>>>> requirements:
>>>>>>>
>>>>>>>  1. Define (or override) hosts from command line
>>>>>>>  2. Execute each scheduled task in parallel
>>>>>>>  3. Wait until previous task completes successfully on all servers
>>>>>>> before moving to the next one
>>>>>>>  4. Ability to run each task separately or use master task as a helper 
>>>>>>> command
>>>>>>>
>>>>>>> Command line hosts (1) is needed to test new nodes and control
>>>>>>> deployment from 3rd party application.
>>>>>>> Parallel execution (2) and (3) is critical to minimize servers downtime.
>>>>>>> Master task (4) is also highly desired, because full deployment process
>>>>>>> contains a lot of steps.
>>>>>>>
>>>>>>> It seems that this could be real with the @mastertask decorator that
>>>>>>> would make task run without servers at all. With it the following
>>>>>>> could work as expected.
>>>>>>>
>>>>>>> @task
>>>>>>> @parallel
>>>>>>> def test:pass
>>>>>>>
>>>>>>> @task
>>>>>>> @parallel
>>>>>>> def update:pass
>>>>>>>
>>>>>>> @mastertask
>>>>>>> def deploy():
>>>>>>>   execute(test)
>>>>>>>   execute(update)
>>>>>>>
>>>>>>> --
>>>>>>> anatoly t.
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Apr 27, 2012 at 1:28 AM, Morgan Goose <address@hidden> wrote:
>>>>>>>> I just read that last line, where you said you wanted to run them with
>>>>>>>> hosts defined at runtime. You can either use functions to generate the
>>>>>>>> hosts lists or with said example i gave change it up to get rid of the
>>>>>>>> deploy task, and then run the fabfile like:
>>>>>>>>    $ fab -H local,remote test update
>>>>>>>>
>>>>>>>> As that will be sequential and honor the host list as well. It will
>>>>>>>> also run the test on all hosts before moving to the next task listed,
>>>>>>>> update.
>>>>>>>>
>>>>>>>> -goose
>>>>>>>>
>>>>>>>> On Thu, Apr 26, 2012 at 3:25 PM, Morgan Goose <address@hidden> wrote:
>>>>>>>>> First have you read the tutorial? This is the section that will get
>>>>>>>>> you in on host list defining, and point you the the more detailed
>>>>>>>>> information that I link to below it:
>>>>>>>>> http://docs.fabfile.org/en/1.4.1/tutorial.html#defining-connections-beforehand
>>>>>>>>> http://docs.fabfile.org/en/1.4.1/usage/execution.html#host-lists
>>>>>>>>>
>>>>>>>>> As to what you want to do. Each task will loop over it's list of hosts
>>>>>>>>> to run on. So top down it's TaskA over all HostListA's hosts. Then if
>>>>>>>>> no errors move to the other task with the same or unique host list
>>>>>>>>> (defined either globally or per task).
>>>>>>>>>
>>>>>>>>> From what you're wanting to do you will construct the fabfile like 
>>>>>>>>> this:
>>>>>>>>>
>>>>>>>>> local = ['a','b']
>>>>>>>>> remote = ['c','d']
>>>>>>>>>
>>>>>>>>> def test:pass
>>>>>>>>> def update:pass
>>>>>>>>>
>>>>>>>>> @task
>>>>>>>>> @sequential
>>>>>>>>> def deploy():
>>>>>>>>>    execute(test, hosts=local+remote)
>>>>>>>>>    execute(update, hosts=local+remote)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Then all hosts will have the test task run on them before moving to
>>>>>>>>> looping over the two hosts lists and running the update. You then call
>>>>>>>>> it simply with:
>>>>>>>>>
>>>>>>>>> $ fab deploy
>>>>>>>>>
>>>>>>>>> And you never run the test and update tasks themselves. Nor do you
>>>>>>>>> define anything to be parallel. As parallel runs on the task would
>>>>>>>>> test on both hosts in parallel, and then deploy to both hosts in
>>>>>>>>> parallel. This was, running sequential and explicitly defined to do
>>>>>>>>> so, will fail fast on the test hosts if one dies. Because the host
>>>>>>>>> list's order is honored.
>>>>>>>>>
>>>>>>>>> -goose
>>>>>>>>>
>>>>>>>>> On Wed, Apr 25, 2012 at 1:13 PM, anatoly techtonik <address@hidden> 
>>>>>>>>> wrote:
>>>>>>>>>> On Sat, Apr 21, 2012 at 7:51 PM, Jeff Forcier <address@hidden> wrote:
>>>>>>>>>>> On Fri, Apr 20, 2012 at 5:31 AM, anatoly techtonik <address@hidden> 
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Is it possible in fabric to wait until a subtask completes on all
>>>>>>>>>>>> servers successfully before moving to next step?
>>>>>>>>>>>
>>>>>>>>>>> You need to use execute() to treat subroutines as if they were full
>>>>>>>>>>> fledged tasks. execute() is the machinery that says "take this
>>>>>>>>>>> function and run it once per host in this list of hosts." Right now
>>>>>>>>>>> that machinery is just applying implicitly to your deploy() task and
>>>>>>>>>>> you're probably using env.hosts to set your host list.
>>>>>>>>>>>
>>>>>>>>>>> Remove env.hosts and maybe make that host list a role in 
>>>>>>>>>>> env.roledefs.
>>>>>>>>>>> Then you can do this:
>>>>>>>>>>>
>>>>>>>>>>>    env.roledefs = {'myrole': ['a', 'b', 'c']}
>>>>>>>>>>>
>>>>>>>>>>>    def test(): ...
>>>>>>>>>>>    def update(): ...
>>>>>>>>>>>
>>>>>>>>>>>    def deploy():
>>>>>>>>>>>        execute(test, role='myrole')
>>>>>>>>>>>        execute(update, role='myrole')
>>>>>>>>>>>
>>>>>>>>>>> That should have the effect you want: "fab deploy" => first test()
>>>>>>>>>>> runs once per host, and when it's all done, update() will run once 
>>>>>>>>>>> per
>>>>>>>>>>> host. deploy() itself will end up running only one time total -- 
>>>>>>>>>>> it's
>>>>>>>>>>> just a "meta" task now.
>>>>>>>>>>
>>>>>>>>>> It took time to realize the that execute() iterates over the global
>>>>>>>>>> list of hosts. I expected that the following two to be equivalent, 
>>>>>>>>>> but
>>>>>>>>>> they were not:
>>>>>>>>>>
>>>>>>>>>>  1. fab -H local,remote test update
>>>>>>>>>>  2. fab -H local,remote deploy
>>>>>>>>>>
>>>>>>>>>> I used the script without roles:
>>>>>>>>>>
>>>>>>>>>>  def test(): ...
>>>>>>>>>>  def update(): ...
>>>>>>>>>>  def deploy():
>>>>>>>>>>      execute(test)
>>>>>>>>>>      execute(update)
>>>>>>>>>>
>>>>>>>>>> 1st execution variant is fully synchronous (i.e. next task doesn't
>>>>>>>>>> start until previous finishes) and gave the sequence:
>>>>>>>>>>
>>>>>>>>>>  local test
>>>>>>>>>>  local update
>>>>>>>>>>  remote test
>>>>>>>>>>  remove update
>>>>>>>>>>
>>>>>>>>>> but the 2nd variant with subtasks was confusing (I indented to see
>>>>>>>>>> what's going on):
>>>>>>>>>>
>>>>>>>>>>  local deploy
>>>>>>>>>>    local test
>>>>>>>>>>    remote test
>>>>>>>>>>    local update
>>>>>>>>>>    remote update
>>>>>>>>>>  remote deploy
>>>>>>>>>>    local test
>>>>>>>>>>    remote test
>>>>>>>>>>    local update
>>>>>>>>>>    remote update
>>>>>>>>>>
>>>>>>>>>> I found fabric pretty counter-intuitive in this case. I tried to fix
>>>>>>>>>> that without roles by explicitly passing current host:
>>>>>>>>>>
>>>>>>>>>>  def test(): ...
>>>>>>>>>>  def update(): ...
>>>>>>>>>>  def deploy():
>>>>>>>>>>      execute(test, host=env.host_string)
>>>>>>>>>>      execute(update, host=env.host_string)
>>>>>>>>>>
>>>>>>>>>> This gives:
>>>>>>>>>>  local deploy
>>>>>>>>>>    local test
>>>>>>>>>>    local update
>>>>>>>>>>  remote deploy
>>>>>>>>>>    remote test
>>>>>>>>>>    remove update
>>>>>>>>>>
>>>>>>>>>> Still not the desired behavior. The desired is:
>>>>>>>>>>  deploy
>>>>>>>>>>    local test
>>>>>>>>>>    remote test
>>>>>>>>>>    wait
>>>>>>>>>>    local update
>>>>>>>>>>    remote update
>>>>>>>>>>
>>>>>>>>>> I've tried using @parallel decorator for deploy task and it seemed to
>>>>>>>>>> work fine at first.
>>>>>>>>>>  local deploy
>>>>>>>>>>  remote deploy
>>>>>>>>>>  remote test
>>>>>>>>>>  local test
>>>>>>>>>>  local update
>>>>>>>>>>  remote update
>>>>>>>>>>
>>>>>>>>>> But the test step didn't not synchronize - local update executed 
>>>>>>>>>> while
>>>>>>>>>> remote test was still running. It looks like the roles is the only
>>>>>>>>>> chance, but I'd like to avoid hardcoding server names at all costs. 
>>>>>>>>>> Is
>>>>>>>>>> it possible?
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> anatoly t.
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Fab-user mailing list
>>>>>>>>>> address@hidden
>>>>>>>>>> https://lists.nongnu.org/mailman/listinfo/fab-user