Modules/businessdev.ALbuild.Containers/Public/Remove-BcSupersededArtifact.ps1
|
function Remove-BcSupersededArtifact { <# .SYNOPSIS Removes cached BC artifacts that a newer build of the same Major.Minor.Build has superseded. .DESCRIPTION With Select 'Latest' the resolved artifact is whatever Microsoft published most recently, and Microsoft publishes a new build within a minor very often - 33 builds existed for sandbox 28.4 in a single minor, the last few only hours apart. Every run therefore resolves a version the cache has never seen, downloads a fresh app+platform pair (3-5 GB) and leaves the previous one behind forever. The cache is not misbehaving; with 'Latest' it simply never gets a chance to hit. On an agent that only builds one product this fills the system drive within days, and the first symptom is not "disk full" but a misleading artifact-extraction failure. This removes the sibling versions that the kept one supersedes: same Major.Minor.Build (the first three version parts), strictly OLDER than -KeepVersion. A newer sibling is never touched, and neither is a different Major.Minor.Build - a pinned 28.3 stays untouched while 28.4 rolls forward. Removal takes the same locks Get-BcArtifact extracts under (Lock-ALbuildCacheEntry), so an entry another build is currently downloading into is skipped rather than deleted underneath it. .PARAMETER CacheFolder Artifact cache root, laid out <root>\<type>\<version>\<country|platform>. Both the host cache (ArtifactCacheFolder) and the container cache (BcArtifactCacheFolder) use this layout. .PARAMETER Type Artifact type folder to prune within, e.g. 'sandbox' or 'onprem'. .PARAMETER KeepVersion The version being kept - normally the one about to be downloaded. .PARAMETER TimeoutSeconds Lock wait per entry. Default 5. .PARAMETER RecentUseWindowHours Never remove a version that Get-BcArtifact handed out within this many hours. The extraction lock only guards writing, not reading: a container-engine compile reads first-party symbols straight out of the mounted artifact for the whole build, so on an agent running several jobs at once, pruning a version one job is still compiling against would break it with missing symbols. Get-BcArtifact stamps every version it hands out, and a stamped version is left for the next run instead. Steady state is still one version per Major.Minor.Build - the skip only defers it. Default 6 hours; 0 disables the guard. .EXAMPLE Remove-BcSupersededArtifact -CacheFolder 'C:\alb\artifacts' -Type 'sandbox' -KeepVersion '28.4.53241.53989' Removes 28.4.53241.53955 and older, keeps 28.3.* and anything newer. .OUTPUTS PSCustomObject per removed (or skipped) entry: Path, Version, Removed, Reason, BytesFreed. #> [CmdletBinding(SupportsShouldProcess, ConfirmImpact = 'Medium')] [OutputType([PSCustomObject])] param( [Parameter(Mandatory)] [ValidateNotNullOrEmpty()] [string] $CacheFolder, [Parameter(Mandatory)] [ValidateNotNullOrEmpty()] [string] $Type, [Parameter(Mandatory)] [ValidateNotNullOrEmpty()] [string] $KeepVersion, [int] $TimeoutSeconds = 5, [int] $RecentUseWindowHours = 6 ) $results = @() $keep = $null if (-not [version]::TryParse($KeepVersion, [ref] $keep)) { Write-ALbuildLog -Level Warning "Cannot prune superseded artifacts: '$KeepVersion' is not a version." return $results } # A 3-part prefix needs all three parts present; 'Latest' always resolves a 4-part build number. if ($keep.Build -lt 0) { Write-ALbuildLog "Skipping supersede prune: '$KeepVersion' has no build part to group by." return $results } $typeFolder = Join-Path -Path $CacheFolder -ChildPath $Type if (-not (Test-Path -LiteralPath $typeFolder)) { return $results } $prefix = '{0}.{1}.{2}' -f $keep.Major, $keep.Minor, $keep.Build $candidates = @(Get-ChildItem -LiteralPath $typeFolder -Directory -Force -ErrorAction SilentlyContinue | ForEach-Object { $parsed = $null if (-not [version]::TryParse($_.Name, [ref] $parsed)) { return } if ($parsed.Build -lt 0) { return } $itsPrefix = '{0}.{1}.{2}' -f $parsed.Major, $parsed.Minor, $parsed.Build if ($itsPrefix -ne $prefix) { return } if ($parsed -ge $keep) { return } [PSCustomObject]@{ Folder = $_; Version = $parsed } }) if ($candidates.Count -eq 0) { return $results } Write-ALbuildLog "Keeping only $prefix.$($keep.Revision) in '$typeFolder'; $($candidates.Count) superseded version(s) to remove." foreach ($candidate in ($candidates | Sort-Object Version)) { $path = $candidate.Folder.FullName # In use by a build that is still reading it? Leave it; the next run takes it. if ($RecentUseWindowHours -gt 0) { $lastUse = Get-BcArtifactLastUse -VersionFolder $path if ($lastUse -and $lastUse -gt (Get-Date).AddHours(-$RecentUseWindowHours)) { Write-ALbuildLog "Superseded artifact $($candidate.Version) was in use at $($lastUse.ToString('yyyy-MM-dd HH:mm')); leaving it for the next run." $results += [PSCustomObject]@{ Path = $path; Version = $candidate.Version; Removed = $false; Reason = 'RecentlyUsed'; BytesFreed = [long]0 } continue } } $size = [long]0 try { $size = [long](Get-ChildItem -LiteralPath $path -Recurse -File -Force -ErrorAction SilentlyContinue | Measure-Object -Property Length -Sum).Sum } catch { $size = [long]0 } if (-not $PSCmdlet.ShouldProcess($path, "Remove superseded artifact $($candidate.Version) ($([Math]::Round($size / 1GB, 2)) GB)")) { $results += [PSCustomObject]@{ Path = $path; Version = $candidate.Version; Removed = $false; Reason = 'WhatIf'; BytesFreed = [long]0 } continue } $locks = Lock-ALbuildCacheEntry -Path $path -TimeoutSeconds $TimeoutSeconds if (-not $locks.Acquired) { # Another build is extracting into it right now - leaving it is correct, the next run prunes it. Write-ALbuildLog -Level Warning "Superseded artifact '$path' is in use by another build (busy: $($locks.BusyPath)); leaving it in place." $results += [PSCustomObject]@{ Path = $path; Version = $candidate.Version; Removed = $false; Reason = 'InUse'; BytesFreed = [long]0 } continue } try { Remove-Item -LiteralPath $path -Recurse -Force -ErrorAction Stop Write-ALbuildLog -Level Success "Removed superseded artifact $($candidate.Version) ($([Math]::Round($size / 1GB, 2)) GB) from '$typeFolder'." $results += [PSCustomObject]@{ Path = $path; Version = $candidate.Version; Removed = $true; Reason = 'Superseded'; BytesFreed = $size } } catch { Write-ALbuildLog -Level Warning "Could not remove superseded artifact '$path': $($_.Exception.Message)" $results += [PSCustomObject]@{ Path = $path; Version = $candidate.Version; Removed = $false; Reason = "Error: $($_.Exception.Message)"; BytesFreed = [long]0 } } finally { Unlock-ALbuildCacheEntry -Locks $locks } } return $results } |