Re: [Fab-user] Run multiple commands on single host in parallel

[Rob Marshall]

Hi Brandon,

As far as the solution to not finding the task, I found that via
google search, so it's not really "mine". :-)

Thank-you very much. You've been very helpful.

Rob
On Tue, Jun 19, 2018 at 10:59 AM Brandon Whaley <address@hidden> wrote:
>
> Thanks for dropping exec :)
>
> I see what you mean about the task lookup problem.  I guess you do need to 
> load the file as a fab file if you don't intend to use the "fab" command line 
> tool if you want string based task lookups in execute().  Sorry for not 
> realizing that!
>
> If you hadn't found the fabfile loading workaround, you could also have tried 
> to look up the task function itself in globals(), since execute can take an 
> actual callable instead of just a string:
>
> >>> def test():
> ...     pass
> ...
> >>> globals()['test']
> <function test at 0x7ff6150b5c80>
>
> I'm partial to your solution though.
>
> On Tue, Jun 19, 2018 at 10:15 AM Rob Marshall <address@hidden> wrote:
>>
>> Hi Brandon,
>>
>> As a personal favor :-) Which actually makes it easier to check the args...
>>
>> @task
>> def task_choser():
>>     host, values, task = env.host_string.split('__')
>>     args = { k:v for k, v in [ arg.split('=', 1) for arg in
>> values.split(',') ] }
>>
>>     if task == 'monitor_task':
>>         if not args.has_key('rackname'):
>>             raise ValueError('A rackname is required for monitor_task')
>>         return execute(task,hosts=[host],rackname=args['rackname'])[host]
>>     else:
>>         for argkey in ('load_node','load_base','load_max'):
>>             if not args.has_key(argkey):
>>                 raise ValueError('A %s is required for run_load')
>>         return 
>> execute(task,hosts=[host],load_node=args['load_node'],load_base=args['load_base'],
>>                        load_max=args['load_max'])[host]
>>
>> I essentially took your test script as (BTW, I'm running Python 2.7.12
>> and Fabric 1.13.2):
>>
>> address@hidden: [Projects]$ cat test_brandon.py
>> #!/usr/bin/python
>>
>> from fabric.api import *
>> from pprint import pprint
>>
>> @task
>> def hostname():
>>     return run('hostname')
>>
>> @task
>> def uname():
>>     return run('uname -a')
>>
>> @task
>> def task_chooser():
>>     # only consider up to the first underscore to be host data
>>     host, task = env.host_string.split('_', 1)
>>     results = execute('%s' % task, hosts=[host])
>>     return results
>>
>> @task
>> def parallel_runner():
>>     host_list=[
>>         '10.245.129.185_hostname',
>>         '10.245.129.185_uname',
>>         '10.245.129.186_hostname',
>>         '10.245.129.186_uname'
>>     ]
>>     with settings(parallel=True):
>>         results = execute(task_chooser, hosts=host_list)
>>
>>     pprint(results)
>>     return results
>>
>> if __name__ == '__main__':
>>     execute(parallel_runner)
>>
>> When I run it I get:
>>
>> address@hidden: [Projects]$ test_brandon.py
>> [10.245.129.185_hostname] Executing task 'task_chooser'
>> [10.245.129.185_uname] Executing task 'task_chooser'
>> [10.245.129.186_hostname] Executing task 'task_chooser'
>> [10.245.129.186_uname] Executing task 'task_chooser'
>>
>> Fatal error: 'uname' is not callable or a valid task name
>>
>> Aborting.
>>
>> Fatal error: 'hostname' is not callable or a valid task name
>>
>> Aborting.
>>
>> Fatal error: 'uname' is not callable or a valid task name
>>
>> Aborting.
>>
>> Fatal error: 'hostname' is not callable or a valid task name
>>
>> Aborting.
>>
>> Fatal error: One or more hosts failed while executing task 'task_chooser'
>>
>> Aborting.
>> On Tue, Jun 19, 2018 at 2:47 AM Brandon Whaley <address@hidden> wrote:
>> >
>> > Hmm, I'm not sure why run_parallel would throw an error like that.  I'd be 
>> > interested to see the full stack trace.  You actually shouldn't need to 
>> > use load_fabfile or commands.update, just using execute(run_parallel) 
>> > should work.  I'll take some time tomorrow and try to replicate your issue.
>> >
>> >
>> > P.S.
>> > As a personal favor for my sanity, I ask that you not use exec().  Here's 
>> > an example of parsing an argument list like the one you're using exec() on:
>> >
>> > >>> import json
>> > >>> values = 'load_node=10.10.0.1,load_base=0,load_max=1000'
>> > >>> args = { k: v for k, v in [ arg.split('=', 1) for arg in 
>> > >>> values.split(',') ] }
>> > >>> print json.dumps(args, indent=4)
>> > {
>> >     "load_node": "10.10.0.1",
>> >     "load_base": "0",
>> >     "load_max": "1000"
>> > }
>> >
>> > You'd then check for args['load_node'] instead of using the local variable 
>> > load_node.
>> >
>> >
>> > On Tue, Jun 19, 2018 at 1:43 AM Rob Marshall <address@hidden> wrote:
>> >>
>> >> Hi,
>> >>
>> >> So I modified your code a bit and ended up with something like this:
>> >>
>> >> @task
>> >> def monitor_task(rackname):
>> >>     cmd = [
>> >>            'run_rack_monitor',
>> >>            '--rack',rackname
>> >>            ]
>> >>
>> >>     return run(' '.join(cmd))
>> >>
>> >> @task
>> >> def run_load(load_node,load_base,load_max):
>> >>     cmd = [
>> >>            'run_system_load',
>> >>            '--datanode',load_node,
>> >>            '--base-value',str(load_base),
>> >>            '--max-value',str(load_max),
>> >>            ]
>> >>
>> >>     return run(' '.join(cmd))
>> >>
>> >> @task
>> >> def task_choser():
>> >>     host, values, task = env.host_string.split('__')
>> >>     for value in values.split(','):
>> >>         exec(value)
>> >>
>> >>     if task == 'monitor_task':
>> >>         return execute(task,hosts=[host],rackname=rackname)
>> >>     else:
>> >>         return 
>> >> execute(task,hosts=[host],load_node=load_node,load_base=load_base,load_max=load_max)
>> >>
>> >> @task
>> >> def run_parallel():
>> >>     host_list = [
>> >>                  '10.10.0.2__rackname="rackname01"__monitor_task',
>> >>                  '10.10.0.2__rackname="rackname02"__monitor_task',
>> >>                  '10.10.0.2__rackname="rackname03"__monitor_task',
>> >>
>> >> '10.10.0.1__load_node="10.10.0.1",load_base=0,load_max=1000__run_load',
>> >>
>> >> '10.10.0.2__load_node="10.10.0.2",load_base=1000,load_max=2000__run_load',
>> >>
>> >> '10.10.0.3__load_node="10.10.0.3",load_base=2000,load_max=3000__run_load',
>> >>
>> >> '10.10.0.4__load_node="10.10.0.4",load_base=3000,load_max=4000__run_load',
>> >>
>> >> '10.10.0.5__load_node="10.10.0.5",load_base=4000,load_max=5000__run_load',
>> >>
>> >> '10.10.0.6__load_node="10.10.0.6",load_base=5000,load_max=6000__run_load',
>> >>                  ]
>> >>
>> >>     with settings(parallel=True):
>> >>         results = execute(task_choser,hosts=host_list)
>> >>
>> >>     return results
>> >>
>> >> Which allows me to pass in arguments to the tasks. I did run into one
>> >> odd thing: If I just tried to run run_parallel() as a function I got
>> >> an error:
>> >>
>> >> Fatal error: '...' is not callable or a valid task name
>> >>
>> >> So what I ended up doing (not sure if there's a better way) was:
>> >>
>> >> from fabric.main import load_fabfile
>> >> from fabric.state import commands
>> >> ...
>> >>
>> >>     docstring, callables, default = load_fabfile(__file__)
>> >>     commands.update(callables)
>> >>
>> >>     with 
>> >> settings(hide('everything'),user='username',password='password1'):
>> >>         results = execute('run_parallel')
>> >>
>> >> That seemed to work.
>> >>
>> >> Thanks,
>> >>
>> >> Rob
>> >> On Mon, Jun 18, 2018 at 4:57 PM Brandon Whaley <address@hidden> wrote:
>> >> >
>> >> > Hi Rob, I've done this as a hack in the past by adding data to the host 
>> >> > list and parsing it before execution to determine what to run.  I've 
>> >> > built a simple example to give you an idea:
>> >> >
>> >> > @task
>> >> > def hostname():
>> >> >     return run('hostname')
>> >> >
>> >> > @task
>> >> > def uname():
>> >> >     return run('uname -a')
>> >> >
>> >> > @task
>> >> > def task_chooser():
>> >> >     # only consider up to the first underscore to be host data
>> >> >     host, task = env.host_string.split('_', 1)
>> >> >     return execute(task, hosts=[host])[host]
>> >> >
>> >> > @task
>> >> > def parallel_runner():
>> >> >     host_list=[
>> >> >         'host1_hostname',
>> >> >         'host1_uname',
>> >> >         'host2_hostname',
>> >> >         'host2_uname'
>> >> >     ]
>> >> >     with settings(parallel=True):
>> >> >         execute(task_chooser, hosts=host_list)
>> >> >
>> >> > [host1_hostname] Executing task 'task_chooser'
>> >> > [host1_uname] Executing task 'task_chooser'
>> >> > [host2_hostname] Executing task 'task_chooser'
>> >> > [host2_uname] Executing task 'task_chooser'
>> >> > [host2] Executing task 'uname'
>> >> > [host2] Executing task 'hostname'
>> >> > [host1] Executing task 'uname'
>> >> > [host2] run: uname -a
>> >> > [host1] Executing task 'hostname'
>> >> > [host2] run: hostname
>> >> > [host1] run: uname -a
>> >> > [host1] run: hostname
>> >> > [host1] out: Linux host1 4.4.0-104-generic #127-Ubuntu SMP Mon Dec 11 
>> >> > 12:16:42 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
>> >> > [host1] out:
>> >> >
>> >> > [host2] out: host2
>> >> > [host2] out:
>> >> >
>> >> > [host2] out: Linux host2 4.4.0-63-generic #84-Ubuntu SMP Wed Feb 1 
>> >> > 17:20:32 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
>> >> > [host2] out:
>> >> >
>> >> > [host1] out: host1
>> >> > [host1] out:
>> >> >
>> >> >
>> >> > Done.
>> >> >
>> >> >
>> >> > On Mon, Jun 18, 2018 at 3:00 PM Rob Marshall <address@hidden> wrote:
>> >> >>
>> >> >> Hi,
>> >> >>
>> >> >> I'm trying to run multiple commands on the same host in parallel but
>> >> >> if I try to run a list of commands based on env.host_string it doesn't
>> >> >> run those commands in parallel. Is there a way to do that?
>> >> >>
>> >> >> I guess, in essence, I'd like to "nest" parallel commands. I
>> >> >> originally attempted to place the host in the hosts list multiple
>> >> >> times, but it looks like parallel removes duplicates (I assume this
>> >> >> has to do with separating results by host).
>> >> >>
>> >> >> Thanks,
>> >> >>
>> >> >> Rob
>> >> >>
>> >> >> _______________________________________________
>> >> >> Fab-user mailing list
>> >> >> address@hidden
>> >> >> https://lists.nongnu.org/mailman/listinfo/fab-user