Deploy applications to compute nodes with Batch awarding packages

  • Commodity
  • 13 minutes to read

Application packages tin simplify the code in your Azure Batch solution and go far easier to manage the applications that your tasks run. With application packages, you can upload and manage multiple versions of applications your tasks run, including their supporting files. Y'all tin can and so automatically deploy one or more than of these applications to the compute nodes in your pool.

The APIs for creating and managing application packages are function of the Batch Management .Internet library. The APIs for installing application packages on a compute node are part of the Batch .Internet library. Comparable features are in the bachelor Batch APIs for other languages.

This article explains how to upload and manage awarding packages in the Azure portal. It besides shows how to install them on a pool's compute nodes with the Batch .NET library.

Application parcel requirements

To apply application packages, you demand to link an Azure Storage account to your Batch business relationship.

There are restrictions on the number of applications and application packages within a Batch account and on the maximum awarding package size. For more than information, encounter Quotas and limits for the Azure Batch service.

Note

Batch pools created prior to July 5, 2017 do non support awarding packages (unless they were created after March x, 2016 using Cloud Services Configuration). The application packages feature described here supersedes the Batch Apps feature available in previous versions of the service.

Understand applications and awarding packages

Within Azure Batch, an application refers to a set of versioned binaries that can be automatically downloaded to the compute nodes in your pool. An awarding contains i or more application packages, which correspond dissimilar versions of the application.

Each application package is a .cypher file that contains the application binaries and any supporting files. Just the .zip format is supported.

Diagram showing a high-level view of applications and application packages.

You tin specify application packages at the pool or task level.

  • Puddle awarding packages are deployed to every node in the pool. Applications are deployed when a node joins a pool, and when it is rebooted or reimaged.

    Pool application packages are appropriate when all nodes in a pool volition execute a job'south tasks. You can specify one or more application packages to deploy when you create a pool. Y'all can also add or update an existing pool'southward packages. To install a new package to an existing pool, you must restart its nodes.

  • Task application packages are deployed merely to a compute node scheduled to run a task, merely before running the task's control line. If the specified application package and version is already on the node, information technology is not redeployed and the existing package is used.

    Job application packages are useful in shared-pool environments, where different jobs run on i pool, and the pool isn't deleted when a job completes. If your task has fewer tasks than nodes in the puddle, job application packages tin minimize data transfer, since your application is deployed only to the nodes that run tasks.

    Other scenarios that can do good from task awarding packages are jobs that run a large application, but for merely a few tasks. For example, chore applications may be useful for a heavyweight pre-processing stage or a merge task.

With awarding packages, your pool'southward start job doesn't have to specify a long listing of private resources files to install on the nodes. Yous don't have to manually manage multiple versions of your awarding files in Azure Storage, or on your nodes. And you don't need to worry about generating SAS URLs to provide access to the files in your Storage business relationship. Batch works in the background with Azure Storage to store awarding packages and deploy them to compute nodes.

Note

The total size of a start job must be less than or equal to 32768 characters, including resource files and environment variables. If your start task exceeds this limit, using application packages is another option. You can too create a .zip file containing your resources files, upload information technology as a blob to Azure Storage, and so unzip it from the control line of your commencement chore.

Upload and manage applications

You lot tin use the Azure portal or the Batch Management APIs to manage the application packages in your Batch account. The following sections explain how to link a storage business relationship, and how to add and manage applications and application packages in the Azure portal.

Note

While you can define application values in the Microsoft.Batch/batchAccounts resource of an ARM template, it's non currently possible to use an ARM template to upload application packages to use in your Batch account. You must upload them to your linked storage business relationship as described below.

To utilize application packages, you must link an Azure Storage account to your Batch account. The Batch service will use the associated storage account to store your application packages. Nosotros recommend that you create a storage account specifically for utilise with your Batch account.

If you lot have not yet configured a storage account, the Azure portal displays a warning the first time y'all select Applications in your Batch account. To link a storage account to your Batch account, select Storage account on the Alert window, and so select Storage Account again.

After you've linked the two accounts, Batch tin automatically deploy the packages stored in the linked Storage account to your compute nodes.

Important

You can't use application packages with Azure Storage accounts configured with firewall rules, or with Hierarchical namespace set to Enabled.

The Batch service uses Azure Storage to store your awarding packages equally block blobs. You lot are charged as normal for the cake blob information, and the size of each package can't exceed the maximum block blob size. For more information, see Azure Storage scalability and functioning targets for storage accounts. To minimize costs, be sure to consider the size and number of your application packages, and periodically remove deprecated packages.

View current applications

To view the applications in your Batch account, select Applications in the left navigation menu.

Screenshot of the Applications menu item in the Azure portal.

Selecting this bill of fare selection opens the Applications window. This window displays the ID of each awarding in your account and the following backdrop:

  • Packages: The number of versions associated with this awarding.
  • Default version: If applicable, the application version that volition be installed if no version is specified when deploying the application.
  • Allow updates: Specifies whether packet updates and deletions are allowed.

To see the file structure of the application package on a compute node, navigate to your Batch account in the Azure portal. Select Pools. then select the pool that contains the compute node. Select the compute node on which the application package is installed and open up the applications folder.

View application details

To see the details for an application, select information technology in the Applications window. You can configure the following settings for your application.

  • Permit updates: Indicates whether application packages tin be updated or deleted. The default is Yes. If ready to No, existing application packages tin't be updated or deleted, but new application package versions can still be added.
  • Default version: The default application parcel to utilise when the application is deployed, if no version is specified.
  • Display name: A friendly proper name that your Batch solution can use when it displays information about the awarding. For example, this proper name tin exist used in the UI of a service that you lot provide to your customers through Batch.

Add together a new application

To create a new application, you add together an application packet and specify a unique awarding ID.

In your Batch account, select Applications and then select Add.

Screenshot of the New application creation process in the Azure portal.

Enter the post-obit information:

  • Application ID: The ID of your new application.
  • Version": The version for the awarding package yous are uploading.
  • Awarding bundle: The .cipher file containing the application binaries and supporting files that are required to execute the application.

The Application ID and Version yous enter must follow these requirements:

  • On Windows nodes, the ID can contain any combination of alphanumeric characters, hyphens, and underscores. On Linux nodes, only alphanumeric characters and underscores are permitted.
  • Tin can't contain more than 64 characters.
  • Must be unique within the Batch account.
  • IDs are example-preserving and case-insensitive.

When you lot're prepare, select Submit. Later on the .cipher file has been uploaded to your Azure Storage account, the portal displays a notification. Depending on the size of the file that you are uploading and the speed of your network connection, this may take some fourth dimension.

Add together a new application parcel

To add together an awarding package version for an existing application, select the application in the Applications section of your Batch account, then select Add.

As you did for the new application, specify the Version for your new package, upload your .nix file in the Application package field, and then select Submit.

Update or delete an awarding bundle

To update or delete an existing application package, select the awarding in the Applications section of your Batch account. Select the ellipsis in the row of the application package that you desire to alter, then select the activeness that you desire to perform.

Screenshot showing the update and delete options for application packages in the Azure portal.

If you select Update, yous'll be able to upload a new .zip file. This will replace the previous .null file that yous uploaded for that version.

If you select Delete, you'll be prompted to confirm the deletion of that version. Once you select OK, Batch volition delete the .zippo file from your Azure Storage account. If you delete the default version of an awarding, the Default version setting is removed for that application.

Install applications on compute nodes

Now that you've learned how to manage application packages in the Azure portal, we can discuss how to deploy them to compute nodes and run them with Batch tasks.

Install pool awarding packages

To install an application parcel on all compute nodes in a puddle, specify one or more awarding package references for the pool. The awarding packages that you specify for a pool are installed on each compute node that joins the pool, and on any node that is rebooted or reimaged.

In Batch .Net, specify one or more CloudPool.ApplicationPackageReferences when yous create a new pool, or for an existing pool. The ApplicationPackageReference class specifies an application ID and version to install on a pool'due south compute nodes.

              // Create the unbound CloudPool CloudPool myCloudPool =     batchClient.PoolOperations.CreatePool(         poolId: "myPool",         targetDedicatedComputeNodes: 1,         virtualMachineSize: "standard_d1_v2",         VirtualMachineConfiguration: new VirtualMachineConfiguration(             imageReference: new ImageReference(                                 publisher: "MicrosoftWindowsServer",                                 offering: "WindowsServer",                                 sku: "2019-datacenter-cadre",                                 version: "latest"),             nodeAgentSkuId: "batch.node.windows amd64");  // Specify the application and version to install on the compute nodes myCloudPool.ApplicationPackageReferences = new List<ApplicationPackageReference> {     new ApplicationPackageReference {         ApplicationId = "litware",         Version = "ane.1001.2b" } };  // Commit the puddle then that it's created in the Batch service. Every bit the nodes bring together // the puddle, the specified awarding bundle is installed on each. await myCloudPool.CommitAsync();

Of import

If an application parcel deployment fails, the Batch service marks the node unusable, and no tasks are scheduled for execution on that node. If this happens, restart the node to reinitiate the bundle deployment. Restarting the node also enables task scheduling again on the node.

Install task application packages

Like to a puddle, yous specify application package references for a task. When a task is scheduled to run on a node, the package is downloaded and extracted merely before the task'due south command line is executed. If a specified bundle and version is already installed on the node, the package is not downloaded and the existing packet is used.

To install a task awarding package, configure the task'south CloudTask.ApplicationPackageReferences property:

              CloudTask job =     new CloudTask(         "litwaretask001",         "cmd /c %AZ_BATCH_APP_PACKAGE_LITWARE%\\litware.exe -args -here");  task.ApplicationPackageReferences = new List<ApplicationPackageReference> {     new ApplicationPackageReference     {         ApplicationId = "litware",         Version = "one.1001.2b"     } };

Execute the installed applications

The packages that you've specified for a pool or task are downloaded and extracted to a named directory within the AZ_BATCH_ROOT_DIR of the node. Batch also creates an environs variable that contains the path to the named directory. Your task command lines employ this environment variable when referencing the application on the node.

On Windows nodes, the variable is in the following format:

              Windows: AZ_BATCH_APP_PACKAGE_APPLICATIONID#version

On Linux nodes, the format is slightly dissimilar. Periods (.), hyphens (-) and number signs (#) are flattened to underscores in the environment variable. Also, note that the case of the application ID is preserved. For example:

              Linux: AZ_BATCH_APP_PACKAGE_applicationid_version

APPLICATIONID and version are values that correspond to the awarding and package version you've specified for deployment. For instance, if you specified that version 2.vii of awarding blender should be installed on Windows nodes, your task command lines would apply this surroundings variable to admission its files:

              Windows: AZ_BATCH_APP_PACKAGE_BLENDER#ii.7

On Linux nodes, specify the environs variable in this format. Flatten the periods (.), hyphens (-) and number signs (#) to underscores, and preserve the case of the application ID:

              Linux: AZ_BATCH_APP_PACKAGE_blender_2_7

When you lot upload an awarding parcel, you can specify a default version to deploy to your compute nodes. If you take specified a default version for an application, you can omit the version suffix when you lot reference the application. Yous tin specify the default awarding version in the Azure portal, in the Applications window, as shown in Upload and manage applications.

For case, if you gear up "2.7" every bit the default version for application blender, and your tasks reference the following environment variable, then your Windows nodes will execute version 2.7:

AZ_BATCH_APP_PACKAGE_BLENDER

The following code snippet shows an example job command line that launches the default version of the blender application:

              string taskId = "blendertask01"; cord commandLine =     @"cmd /c %AZ_BATCH_APP_PACKAGE_BLENDER%\blender.exe -args -hither"; CloudTask blenderTask = new CloudTask(taskId, commandLine);

Update a puddle's application packages

If an existing pool has already been configured with an awarding parcel, you can specify a new package for the puddle. This ways:

  • The Batch service installs the newly specified package on all new nodes that join the pool and on any existing node that is rebooted or reimaged.
  • Compute nodes that are already in the pool when you update the package references practise non automatically install the new application package. These compute nodes must be rebooted or reimaged to receive the new bundle.
  • When a new package is deployed, the created environment variables reflect the new application bundle references.

In this case, the existing pool has version two.7 of the blender application configured equally one of its CloudPool.ApplicationPackageReferences. To update the pool's nodes with version 2.76b, specify a new ApplicationPackageReference with the new version, and commit the alter.

              cord newVersion = "2.76b"; CloudPool boundPool = await batchClient.PoolOperations.GetPoolAsync("myPool"); boundPool.ApplicationPackageReferences = new Listing<ApplicationPackageReference> {     new ApplicationPackageReference {         ApplicationId = "blender",         Version = newVersion } }; await boundPool.CommitAsync();

Now that the new version has been configured, the Batch service installs version 2.76b to any new node that joins the pool. To install 2.76b on the nodes that are already in the puddle, reboot or reimage them. Notation that rebooted nodes retain files from previous parcel deployments.

List the applications in a Batch business relationship

You tin can list the applications and their packages in a Batch business relationship past using the ApplicationOperations.ListApplicationSummaries method.

              // List the applications and their application packages in the Batch account. List<ApplicationSummary> applications = await batchClient.ApplicationOperations.ListApplicationSummaries().ToListAsync(); foreach (ApplicationSummary app in applications) {     Console.WriteLine("ID: {0} | Display Proper name: {1}", app.Id, app.DisplayName);      foreach (string version in app.Versions)     {         Console.WriteLine("  {0}", version);     } }

Next steps

  • The Batch Remainder API likewise provides support to work with awarding packages. For instance, see the applicationPackageReferences element for how to specify packages to install, and Applications for how to obtain application information.
  • Learn how to programmatically manage Azure Batch accounts and quotas with Batch Management .NET. The Batch Management .NET library can enable account cosmos and deletion features for your Batch awarding or service.