The future of nx-remotecache-*

Nrwl Nx just introduced us to Powerpack. It's the clear successor of the remote cache setup and officially supports custom caching solutions. This will mark the end of the nx-remotecache-* packages as custom caching solutions based on the filesystem won't work anymore starting with Nx 21.

Powerpack fills exactly the void that nx-remotecache-custom filled. Therefore I'm happy to give this topic back to the creators behind Nx. Thanks for the great ride – we reached over 114,000 weekly downloads on npm as I'm writing this 🥳

Feel free to read the Introduction to Powerpack by Jeff yourself. If you have any further questions checkout the pinned GitHub issue.

Cheers, Niklas 👋

nx-remotecache-azure

A task runner for @nrwl/nx that uses an Azure Blob Storage as a remote cache. This enables all team members and CI servers to share a single cache. The concept and benefits of computation caching are explained in the NX documentation.

This package was built with nx-remotecache-custom 🙌

Compatability

Setup

npm install --save-dev nx-remotecache-azure

{
  "tasksRunnerOptions": {
    "default": {
      "runner": "nx-remotecache-azure",
      "options": {
        // All of the azure specific options can also be inserted via environment variables! ⬆️
        "accountName": "MyAzureAccountName",
        "accountKey": "my-azure-account-key-11223-22..",
        "container": "test",
        "cacheableOperations": ["build", "test", "lint", "e2e"]
      }
    }
  }
}

Run it 🚀

Running tasks should now show the storage or retrieval from the remote cache:

------------------------------------------------------------------------
Built Angular Package
 - from: /Users/name/example-workspace/libs/example-lib
 - to:   /Users/name/example-workspace/dist/libs/example-lib
------------------------------------------------------------------------
------------------------------------------------------------------------
Stored output to remote cache: Azure Blob Storage
Hash: d3d2bea71ea0f3004304c5cc88cf91be50b02bb636ebbdfcc927626fd8edf1ae
------------------------------------------------------------------------

Advanced Configuration

"tasksRunnerOptions": {
  "default": {
    "options": {
      "name": "My Storage",
      "verbose": true,
      "silent": true
    }
  }
}

Cleanup

Over time, cached items accumulate on the remote storage, potentially leading to uncontrolled growth of storage usage. Failure to clean up outdated items can result in excessive costs. However, you can effectively manage storage usage and costs by configuring Azure storage with lifecycle rules.

To begin, ensure that the last access time of stored items is monitored. Access the Lifecycle management section within your storage account and enable monitoring as illustrated below:

Next, tailor the rules according to your requirements. For example, consider the following rules that will manage a container named nx-cache:

• Blobs not accessed for 30 days will be moved to cold storage. If accessed again, they will be returned to hot storage.
• Blobs not accessed at all for 90 days will be automatically deleted.

Configure these rules to strike a balance between cost optimization and access efficiency:

{
  "rules": [
    {
      "enabled": true,
      "name": "Cleanup",
      "type": "Lifecycle",
      "definition": {
        "actions": {
          "baseBlob": {
            "enableAutoTierToHotFromCool": true,
            "tierToCool": {
              "daysAfterLastAccessTimeGreaterThan": 30
            },
            "delete": {
              "daysAfterLastAccessTimeGreaterThan": 90
            }
          }
        },
        "filters": {
          "blobTypes": ["blockBlob"],
          "prefixMatch": ["nx-cache"]
        }
      }
    }
  ]
}

More info on Configure a lifecycle management policy

All Custom Runners