# Enabling Azure resources
> This bundle contains all pages in the Enabling Azure resources section.
> Source: https://www.union.ai/docs/v2/union/deployment/byoc/enabling-azure-resources/

=== PAGE: https://www.union.ai/docs/v2/union/deployment/byoc/enabling-azure-resources ===

# Enabling Azure resources

> **📝 Note**
>
> An LLM-optimized bundle of this entire section is available at [`section.md`](https://www.union.ai/docs/v2/union/deployment/byoc/section.md).
> This single file contains all pages in this section, optimized for AI coding agent context.

Components of your Union.ai data plane will need to connect to and communicate with other resources in your Azure cloud environment, such as Azure [Blob Storage](https://azure.microsoft.com/en-ca/products/storage/blobs/) and [Container Registry](https://azure.microsoft.com/en-us/products/container-registry).

[Data plane setup on Azure](https://www.union.ai/docs/v2/union/deployment/byoc/data-plane-setup-on-azure/page.md) provides Union.ai with the necessary permissions to manage underlying Azure resources within your data plane. Access to non-Union.ai Azure resources is subject to Azure limitations and will require additional configuration.

As your projects evolve, your needs may change.
You can always contact the Union.ai team for help enabling additional resources as required.

## Types of access

There are two categories of access that you are likely to have to deal with:

* **Infrastructure access**:
  Enabling access to a resource for your data plane infrastructure.
  The most common case occurs when using your container registry task container images.
  In that case, refer to **BYOC deployment > Enabling Azure resources > Enabling Azure Container Registry (ACR)** to configure the Union.ai data plane to access that registry.
* **Task code access**:
  Enabling access to a resource for your task code.
  For example, your task code might need to access Azure Blob Storage at runtime.
  This involves granting permission to the [User-assigned managed identity](https://learn.microsoft.com/en-us/entra/identity/managed-identities-azure-resources/overview) attached to the Kubernetes cluster within which your task code runs.

## Infrastructure-level access

Infrastructure access with non-Union.ai-managed Azure resources will require additional configuration. Refer to **BYOC deployment > Enabling Azure resources > Enabling Azure Container Registry (ACR)** if you need access to images within an existing or non-Union.ai-managed container registry.

## Task code access

Union.ai tasks run within a Union.ai-managed Kubernetes pod in your data plane. Union.ai uses [Microsoft Entra Workload ID](https://learn.microsoft.com/en-us/azure/aks/workload-identity-overview?tabs=dotnet) to create user-assigned managed identities and access Union.ai-managed Azure resources. Additional permissions can be granted to the user-assigned managed identity to access Azure resources within the same Tenant.

Union.ai on Azure has two types of access arrangements:

* **Domain-scoped access**: With this arrangement, you define permissions you want to grant to your tasks, which are applied only to a specific Union.ai domain.
* **Global access**: With this arrangement, you define permissions you want to grant to your tasks, which are applied to an entire Azure subscription or resource group.

> [!NOTE] Azure only supports scoping by domain
> In AWS-based data planes, scoping by both project _and_ domain is supported.
> However, due to intrinsic architectural constraints, Azure-based data planes only support scoping by domain.

Global access is recommended for most use cases since it is simpler. Still, if you have a compelling reason to restrict access, then the subscription/resource group-domain-scoped access is available at the cost of additional complexity in setup.

> [!NOTE] Relationship with RBAC
> The permissions being discussed here are attached to a domain.
> This is independent of the permissions granted to users and machine applications through Union.ai's role-based access control (see the user management documentation).
> But, the two types of permissions are related.
>
> For example, for a user (or machine application) to have read access to a blob storage container, two things are required:
>
> * The user (or machine application) must have **execute** permission for the project and domain where the code that does the reading resides.
> * The domain must have read permission for the blob storage container.

## Domain-scoped access

**Because of the way that Azure works internally, domain-scoped access can only be configured by the Union.ai team.**

Please work directly with the Union.ai team if you have requirements that involve domain-scoped access to cloud resources.

## Globally-scoped access

Union.ai creates a managed identity prefixed with `flyteuser` within the resource group that contains the other Union.ai-managed data plane Azure resources. Navigate to [Azure portal Managed Identities](https://portal.azure.com/#view/HubsExtension/BrowseResource/resourceType/Microsoft.ManagedIdentity%2FuserAssignedIdentities) to find respective managed identity details.

Follow [Azure's official assigned roles documentation](https://learn.microsoft.com/en-us/azure/role-based-access-control/role-assignments-portal) to assign an appropriate role to scope.

=== PAGE: https://www.union.ai/docs/v2/union/deployment/byoc/enabling-azure-resources/enabling-azure-blob-storage ===

# Enabling Azure Blob Storage

For Union.ai customers whose data plane is in Azure, we walk through setting up access to your own Azure Blob Storage container.

> [!NOTE] Azure Blob Storage in the Union.ai environment
> Your data plane is set up with a Kubernetes cluster and other resources.
> Among these are a number of Azure Storage containers used internally by the Union.ai operator running in the cluster (see [Platform architecture](https://www.union.ai/docs/v2/union/deployment/byoc/platform-architecture)) to store things like workflow metadata.
>
> **These are not the Azure Blob Storage containers we are talking about in this section.**
>
> **We are discussing the case where you have **_**your own Azure Blob Storage container**_**that you set up to store input and output data used by your workflows.**

## Providing permissions to Azure Blob Storage container

Union.ai data plane tasks employ Azure Workload Identity Federation to access Azure resources using an Azure user-assigned identity. Access to Azure Blob Storage containers requires updating permissions to permit this Union.ai-managed user-assigned identity.

### Union.ai-managed permissions

The simplest, most flexible approach is to provide Union.ai the ability to add roles assignments against the blob storage container. [Create a role assignment](https://learn.microsoft.com/en-us/azure/role-based-access-control/role-assignments-portal) to allow Union.ai to assign roles to the blob storage container. These permissions should be scoped to the target container. Follow these steps to set up the required access:

1. Navigate to the Azure portal and locate the target storage container.
2. In the storage container's access control (IAM) section, create a new role assignment.
3. For the 'Assigned to' field, select the Union.ai application's service principal.
4. For the 'Role' field, you have two options:
  * Simplest approach: Assign the built-in Azure role `User Access Administrator`.
  * Advanced approach: Create a custom role with the following specific permissions:
    * `Microsoft.Authorization/roleAssignments/write`
    * `Microsoft.Authorization/roleAssignments/delete`
    * `Microsoft.Authorization/roleAssignments/read`
    * `Microsoft.Authorization/roleDefinitions/read`
5. Ensure the 'Scope' is set to the target blob storage container.
6. Complete the role assignment process.
7. Provide the blob storage container [resource ID](https://learn.microsoft.com/en-us/dotnet/api/microsoft.azure.management.storage.models.resource.id) to Union.ai support.

### Manage permissions directly

Managing permissions directly is required if it is not desirable to grant role assigning permissions to Union.ai. [Create a role assignment](https://learn.microsoft.com/en-us/azure/role-based-access-control/role-assignments-portal)) assigning the `Storage Blob Data Contributor` role to the `userflyterole` user assigned identity scoped the blob storage container.

> [!NOTE] Union.ai managed user-assigned identities
> Refer to [Azure portal's user assigned managed identitites](https://portal.azure.com/#view/HubsExtension/BrowseResource/resourceType/Microsoft.ManagedIdentity%2FuserAssignedIdentities) if assistance is required identifying the `userflyterole` user assigned managed identity within the same resource group as the Union.ai data plane.

=== PAGE: https://www.union.ai/docs/v2/union/deployment/byoc/enabling-azure-resources/enabling-azure-container-registry ===

# Enabling Azure Container Registry (ACR)

ACR can be used to store container images within Azure and accessed within your Azure-based Data Plane.

Union.ai leverages Azure Kubernetes Service (AKS) managed identities to authenticate with ACR.

Refer to [Azure documentation for more details](https://learn.microsoft.com/en-us/azure/container-registry/authenticate-kubernetes-options)

## Creating a container registry

### Creating a container registry outside of Union.ai

ACR instances that allow anonymous (I.E., public) access doesn't require additional configuration. Otherwise, the underlying AKS cluster must be granted permissions to pull from the container registry.

Private ACR for Union.ai images is only supported for ACRs within the same tenant as the Union.ai data plane. Refer to [Azure documentation](https://learn.microsoft.com/en-us/azure/container-registry/container-registry-get-started-portal?tabs=azure-cli) for creating Container Registries.

### Creating a Union.ai-managed container registry

Upon request, Union.ai can create a container registry within your data plane.

By default this Union.ai-managed ACR instance:

* Will be created within the same subscription and resource group of the Azure Kubernetes cluster instance.
* Union.ai will create necessary permissions for the Azure Kubernetes cluster to pull images from the container registry.
* Container registry will be created with **Basic** service tier.
* In order to mitigate excessive storage costs, Union.ai creates a weekly [scheduled container registry task](https://learn.microsoft.com/en-us/azure/container-registry/container-registry-tasks-scheduled) to [purge](https://learn.microsoft.com/en-us/azure/container-registry/container-registry-auto-purge#use-the-purge-command) **all** images with last modified dates older then 7 days. As a symptom, some 7 day old images will be rebuilt.

Upon request, Union.ai can:

* Configure the [Container Registry service tier](https://learn.microsoft.com/en-us/azure/container-registry/container-registry-skus).
* Disable the purge task to prevent automated image delettion.
* Configure the purge task to run daily, weekly, and monthly deleting tasks with last modified dates older then 1, 7, and 30 days respectively.
* Configure a [regexp2 with RE2 compatiblity](https://github.com/dlclark/regexp2) regular expression to filter for which repository to purge. For example, `^(?!keep-repo).*` will keep all images with repositories prefixed with keep-repo, E.G., `<CONTAINER_REGISTRY_NAME>/keep-repo/my-image:my-tag>`.

Union.ai will provide the created container registry Name and Login server for Docker authentication.

## Enable access to ACR in a different subscription within the same Azure tenant

Union.ai data plane resources will require permissions to pull images from your container registry.

### Allow Union.ai to manage permissions

The simplest, most flexible approach is to provide Union.ai the ability to add roles assignments against the container registry. [Create a role assignment](https://learn.microsoft.com/en-us/azure/role-based-access-control/role-assignments-portal) to allow Union.ai to assign roles to the container registry. These permissions should be scoped to the target container registry. Follow these steps to set up the required access:

1. Navigate to the Azure portal and locate the target container registry.
2. In the container registry's access control (IAM) section, create a new role assignment.
3. For the 'Assigned to' field, select the Union.ai application's service principal.
4. For the 'Role' field, you have two options:
    * Simplest approach: Assign the built-in Azure role `User Access Administrator`.
    * Advanced approach: Create a custom role with the following specific permissions:
      * `Microsoft.Authorization/roleAssignments/write`
      * `Microsoft.Authorization/roleAssignments/delete`
      * `Microsoft.Authorization/roleAssignments/read`
      * `Microsoft.Authorization/roleDefinitions/read`
5. Ensure the 'Scope' is set to the target container registry.
6. Complete the role assignment process.
7. Provide the container registry [resource ID](https://learn.microsoft.com/en-us/dotnet/api/microsoft.azure.management.storage.models.resource.id) to Union.ai support.

### Manage permissions directly

Managing permissions directly is required if it is not desirable to grant role assigning permissions to Union.ai. [Create a role assignment](https://learn.microsoft.com/en-us/azure/role-based-access-control/role-assignments-portal) assigning the `AcrPull` role to the underlying AKS cluster kubelet service principal ID. The service principal ID can be provided by Union.ai support.

Note, this process needs to be repeated every time the underlying Kubernetes cluster is changed or a new cluster is added.

## Enable access to ACR in a different Azure tenant

Please contact and work directly with Union.ai support.

## References

* [Azure - Authenticate with Azure Container Registry (ACR) from Azure Kubernetes Service (AKS)](https://learn.microsoft.com/en-us/azure/aks/cluster-container-registry-integration?toc=%2Fazure%2Fcontainer-registry%2Ftoc.json&bc=%2Fazure%2Fcontainer-registry%2Fbreadcrumb%2Ftoc.json&tabs=azure-cli)
* [Azure - Pull images from a container registry to an AKS cluster in a different Microsoft Entra tenant](https://learn.microsoft.com/en-us/azure/container-registry/authenticate-aks-cross-tenant)

=== PAGE: https://www.union.ai/docs/v2/union/deployment/byoc/enabling-azure-resources/enabling-azure-key-vault ===

# Enabling Azure Key Vault

> [!NOTE]
> This documentation exists for customers who must use Azure Key Vault for organizational reasons. For everyone else, we strongly recommend using the
> [Union.ai secrets manager](https://www.union.ai/docs/v2/union/user-guide/task-configuration/secrets) to manage secrets rather than Azure Key Vault.

The Union.ai-managed `userflyterole` identity must be granted permission to access [Azure Key Vault secrets](https://learn.microsoft.com/en-us/azure/key-vault/secrets/about-secrets).

> [!NOTE] Managing Azure Key Vault secrets
> Refer to [Azure official documentation](https://learn.microsoft.com/en-us/azure/key-vault/secrets/quick-create-portal) for details on creating and managing secrets.

## Providing permissions to Azure Key Vault

Union.ai data plane tasks employ Azure Workload Identity Federation to access Azure resources using an Azure user-assigned identity. Access to Azure Key Vault containers requires updating permissions to permit this Union.ai-managed user-assigned identity.

[Create a role assignment](https://learn.microsoft.com/en-us/azure/role-based-access-control/role-assignments-portal) assigning the `Key Vault Secrets User` role to the `userflyterole` user-assigned identity. Make sure it is scoped to the Azure Key Vault Secret.

> [!NOTE] Union.ai managed user-assigned identities
> Refer to [Azure portal's user assigned managed identitites](https://portal.azure.com/#view/HubsExtension/BrowseResource/resourceType/Microsoft.ManagedIdentity%2FuserAssignedIdentities) if assistance is required identifying the `userflyterole` user-assigned identity within the Union.ai data plane resource group.

## Accessing the secret within Union.ai

* Define a `Secret` object where
  * `Secret.group` is the a HTTP URI of the format `https://<KEY_VAULT_NAME>.vault.azure.net/secrets/<SECRET_NAME>`
  * `Secret.group_version` can be omitted to retrieve the latest version or set to an explicit secret version
  * `Secret.mount_requirement` is `Secret.MountType.FILE`
* Pass that `Secret` object in the `secret_requests` parameter of the `@union.task` decorator.
* Inside the task code, retrieve the value of the secret with:
  * `union.current_context().secrets.get(<SECRET_NAME>)` if `Secret.group_version` was omitted.
  * `union.current_context().secrets.get(<SECRET_NAME>, group_version=SECRET_GROUP_VERSION)` if `Secret.group_version` was specified.

Here are examples:

```python
import union

VAULT_NAME = "examplevault"
SECRET_NAME = "example-secret"

SECRET_GROUP = f"https://{VAULT_NAME}.vault.azure.net/secrets/{SECRET_NAME}"
SECRET_GROUP_VERSION = "12345"

SECRET_REQUEST_WITH_VERSION = union.Secret(
  group=SECRET_GROUP,
  group_version=SECRET_GROUP_VERSION,
  mount_requirement=union.Secret.MountType.FILE
)

@union.task(secret_requests=[SECRET_REQUEST_WITH_VERSION])
def task_with_versioned_secret():
    secret_val = union.current_context().secrets.get(
        SECRET_NAME,
        group_version=SECRET_GROUP_VERSION
    )

SECRET_REQUEST_FOR_LATEST = union.Secret(
  group=SECRET_GROUP,
  mount_requirement=union.Secret.MountType.FILE
)

@union.task(secret_requests=[SECRET_REQUEST_FOR_LATEST])
def task_with_latest_secret():
    secret_val = union.current_context().secrets.get(
        SECRET_NAME
    )
```