Speed up your YAML build pipeline in Azure DevOps with the Cache task.
Sometimes the builds are taking a long time. Often the NuGet restore/npm install takes a couple of minutes to download all packages that you use in your solution. One way to speed up that process is to use the “new” Cache task in Azure DevOps.
It took me a long time to understand how you actually used this task (yes I know, I´m kind of solve…). But when I found the example in jraps20 build.yml(https://gist.github.com/jraps20/ea29de8b45493dcfa0a1ec5b0af82337) I finally understood how it works and how to do this.
So, what you could do is add 2 tasks. The first task is a PowerShell task creates a hash of the content in packages.config. That means that as long you don´t change and version of NuGet packages, and/or adding/removing NuGet packages, the hash will be the same. That will decide if the cached packages folder should be used or if it should download Nuget packages into the packages folder.
The second task is actual cache task that use the hash in the cachekey to figure out if there is any cached “packages” folder to reuse.
Note: The Cache task (Cache files between runs) is only supported when you are using YAML.
Before I implemented the caching my YAML file started like this:
And after implemented the caching tasks it looks like this:
You never really know how much this will fast up you build. But an example from my last project made the build pipeline become around 1.5 minutes faster. (If I have no packages.config changes of course). And of course, it depends on how many NuGet packages you use etc.
In our case in this project we use around 115 NuGet packages and the folder weight around 400Mb. It takes around 2 minutes (if lucky) to download these packages.
And with the cache task we managed to get the same task to run in 30 seconds.
I have not given you any example of how to do this with NPM packages. Here is an example of YAML that cache both NuGet and NPM packages.
- group: DXP-variables
- name: webProjectDirectory
- stage: Build
- job: DevInte
name: Azure Pipelines
buildPlatform: 'Any CPU'
displayName: 'Calculate and save packages.config hash'
# generates a hash of all packages.config and saves each on a single line on 'packagesconfighash.txt'
Get-FileHash -Algorithm MD5 -Path (Get-ChildItem packages.config -Recurse) >> packagesconfighash.txt
Write-Host "Hash File saved to $(System.DefaultWorkingDirectory)\packagesconfighash.txt"
- task: [email protected] # speed up builds by caching packages folder
key: nuget|1|$(Agent.OS)|$(Build.SourcesDirectory)\packagesconfighash.txt # hash map generated in previous step
cacheHitVar: 'nuget_cache_hit' # variable that can be checked to see if it was successful
displayName: Cache nuget packages
displayName: 'Use NuGet 5.*'
displayName: 'NuGet restore'
displayName: 'Calculate and save package.json hash'
# generates a hash of all package.json and saves each on a single line on 'packagejsonhash.txt'
Get-FileHash -Algorithm MD5 -Path (Get-ChildItem package.json -Recurse) >> packagejsonhash.txt
Write-Host "Hash File saved to $(System.DefaultWorkingDirectory)\packagejsonhash.txt"
- task: [email protected] # speed up builds by caching node_modules folder
key: nuget|1|$(Agent.OS)|$(Build.SourcesDirectory)\packagejsonhash.txt # hash map generated in previous step
cacheHitVar: 'npm_cache_hit' # variable that can be checked to see if it was successful
displayName: Cache npm packages
displayName: 'NPM Set progress false (speedup)'
customCommand: 'set progress=false'
displayName: 'npm install'
displayName: 'npm run build'
customCommand: 'run build'
I hope this helps!