在azure批处理节点中安装python包

时间:2018-03-12 13:04:38

标签: python linux azure

我如何设法在azure批处理节点中安装python包,如 pip install azure-storage-blob 。我试图在启动任务中编写该命令,但是我导致严峻的任务失败了?

1 个答案:

答案 0 :(得分:-1)

定义池时,必须指定池启动任务。此启动任务将在每个节点创建时运行并加入池(或重新启动时,重新映像)。

您可以在池下的控制台中执行此操作。或者,如果您使用Python,请执行此操作(这是一个示例,根据需要进行调整!)。看看如何定义tast_commands变量(需要curl):

def create_pool(batch_service_client, pool_id,
            resource_files, publisher, offer, sku):
"""
Creates a pool of compute nodes with the specified OS settings.

:param batch_service_client: A Batch service client.
:type batch_service_client: `azure.batch.BatchServiceClient`
:param str pool_id: An ID for the new pool.
:param list resource_files: A collection of resource files for the pool's
start task.
:param str publisher: Marketplace image publisher
:param str offer: Marketplace image offer
:param str sku: Marketplace image sku
"""
print('Creating pool [{}]...'.format(pool_id))

# Create a new pool of Linux compute nodes using an Azure Virtual Machines
# Marketplace image. For more information about creating pools of Linux
# nodes, see:
# https://azure.microsoft.com/documentation/articles/batch-linux-nodes/

# Specify the commands for the pool's start task. The start task is run
# on each node as it joins the pool, and when it's rebooted or re-imaged.
# We use the start task to prep the node for running our task script.
task_commands = [
    'cp -p {} $AZ_BATCH_NODE_SHARED_DIR'.format('YOUR-PYTHON-SCRIPT.py'),

    # Install pip
    'curl -fSsL https://bootstrap.pypa.io/get-pip.py | python',
    # Install the azure-storage module so that the task script can access
    # Azure Blob storage, pre-cryptography version
    'pip install azure-storage==0.32.0']

# Get the node agent SKU and image reference for the virtual machine
# configuration.
# For more information about the virtual machine configuration, see:
# https://azure.microsoft.com/documentation/articles/batch-linux-nodes/
sku_to_use, image_ref_to_use = \
    helpers.select_latest_verified_vm_image_with_node_agent_sku(
        batch_service_client, publisher, offer, sku)

user = batchmodels.AutoUserSpecification(
    scope=batchmodels.AutoUserScope.pool,
    elevation_level=batchmodels.ElevationLevel.admin)

new_pool = batch.models.PoolAddParameter(
    id=pool_id,
    virtual_machine_configuration=batchmodels.VirtualMachineConfiguration(
        image_reference=image_ref_to_use,
        node_agent_sku_id=sku_to_use),
    vm_size=_POOL_VM_SIZE,
    target_dedicated_nodes=_POOL_NODE_COUNT,
    start_task=batch.models.StartTask(
        command_line=helpers.wrap_commands_in_shell('linux',
                                                           task_commands),
        user_identity=batchmodels.UserIdentity(auto_user=user),
        wait_for_success=True,
        resource_files=resource_files),
)

try:
    batch_service_client.pool.add(new_pool)
except batchmodels.batch_error.BatchErrorException as err:
    print_batch_exception(err)
    raise

免责声明:基于Microsoft的python_tutorial_client.py,查找一下,它会帮助您入门!