首页 文章

传递结构执行任务的命令行参数

提问于
浏览 次
0

我是 fabric API的新手，我现在就在那里 . 目前，下面的位在三台主机上的每台主机上并行正确运行脚本 get_num_reviews_aws.py .

hosts = [ubuntu@54.xxx.xx.xx,
         ubuntu@52.xx.xxx.xx,
         ubuntu@54.xx.xxx.xx]

#%%
from fabric.api import run, parallel
from fabric.tasks import execute


%%
@parallel
def webscraper():
    run("python get_num_reviews_aws.py")


#%% run on hosts
execute(webscraper, hosts=hosts)

我正在寻找的是能够将命令行参数传递给python脚本，这些参数对于每个主机都是不同的，但仍然可以并行运行 . 所以像这样：

@parallel
def webscraper(start, end):
    run("python get_num_reviews_aws.py %s %s" % (start, end))

然后基本上每个主机都有一组不同的 start 和 end . 什么's hanging me up is I pass a list of hosts but I don' t我认为我为每个命令行参数传递一个列表，例如：

start = [1, 2, 3]
end = [4, 5, 6]

execute(webscraper, start, end, hosts=hosts)
python amazon-ec2 fabric

1 回答

  • 0

    在看了一些其他的similar questions on SO后，这对我有用 .

    hosts = [ubuntu@54.xxx.xx.xx,
         ubuntu@52.xx.xxx.xx,
         ubuntu@54.xx.xxx.xx]


#%% get username and host
hosts = ["ubuntu@" + ip.ip_address for ip in instance_lst]


#%% Create my command line arguments and store them in a dict
# use the hostname as the key and store arguments in a tuple
host_dict = {host: (0, 10 + x) for x, host in enumerate(hosts)}


#%%
from fabric.api import run, parallel, env
from fabric.tasks import execute


#%% set environment
env.hosts = hosts


#%%
@parallel
def webscraper(host_dict):
    host = "ubuntu@" + env.host # cuts off the username, so have to re-add
    start_end = host_dict[host] # get the start and end for each host
    run("python get_num_reviews_aws.py %s %s" % start_end)


#%% run on hosts
# since I stored the hosts in an environment, don't need to pass hosts
execute(webscraper, host_dict)
    回复于

相关问题