[DX-4553] Add docker build cache dance #1608
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| --- | ||
| "build-push-docker": minor | ||
| "ctf-build-image": minor | ||
| --- | ||
|
|
||
| Adds buildkit-cache-dance to properly, persistently cache docker build layers in CI |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,58 @@ | ||
| #!/bin/bash | ||
|
kalverra marked this conversation as resolved.
|
||
| set -e | ||
|
|
||
| SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" | ||
| VALIDATE_SCRIPT="${SCRIPT_DIR}/validate-cache-dance-config.sh" | ||
|
|
||
| echo "===== Testing build-push-docker cache dance configuration (Option 1) =====" | ||
|
|
||
| parse_val() { | ||
| local output="$1" | ||
| local key="$2" | ||
| echo "$output" | grep "^${key}=" | cut -d'=' -f2 | ||
| } | ||
|
|
||
| echo "Test 1: Default - no cache-dance and no cache-map" | ||
| RES=$("$VALIDATE_SCRIPT" "" "") | ||
| DANCE=$(parse_val "$RES" "cache-dance") | ||
| [ "$DANCE" = "false" ] || (echo "FAIL: expected cache-dance=false, got $DANCE" && exit 1) | ||
| echo "Test 1 passed." | ||
|
|
||
| echo "Test 2: Implicit enable - cache-map provided without explicit cache-dance flag" | ||
| VALID_MAP='{"go-mod-cache": "/go/pkg/mod"}' | ||
| RES=$("$VALIDATE_SCRIPT" "" "$VALID_MAP") | ||
| DANCE=$(parse_val "$RES" "cache-dance") | ||
| MAP=$(parse_val "$RES" "cache-map") | ||
| [ "$DANCE" = "true" ] || (echo "FAIL: expected cache-dance=true, got $DANCE" && exit 1) | ||
| [ "$MAP" = "$VALID_MAP" ] || (echo "FAIL: expected cache-map match" && exit 1) | ||
| echo "Test 2 passed." | ||
|
|
||
| echo "Test 3: Invalid JSON in cache-map (should fail)" | ||
| if "$VALIDATE_SCRIPT" "" "invalid-json"; then | ||
| echo "FAIL: Expected failure when cache-map is invalid JSON" | ||
| exit 1 | ||
| else | ||
| echo "Test 3 passed (failed as expected)." | ||
| fi | ||
|
|
||
| echo "Test 4: Explicit cache-dance=false overrides non-empty cache-map" | ||
| RES=$("$VALIDATE_SCRIPT" "false" "$VALID_MAP") | ||
| DANCE=$(parse_val "$RES" "cache-dance") | ||
| [ "$DANCE" = "false" ] || (echo "FAIL: expected cache-dance=false when explicitly disabled, got $DANCE" && exit 1) | ||
| echo "Test 4 passed." | ||
|
|
||
| echo "Test 5: Explicit cache-dance=true with missing cache-map (should fail)" | ||
| if "$VALIDATE_SCRIPT" "true" ""; then | ||
| echo "FAIL: Expected failure when cache-dance=true with missing cache-map" | ||
| exit 1 | ||
| else | ||
| echo "Test 5 passed (failed as expected)." | ||
| fi | ||
|
|
||
| echo "Test 6: Backward compatibility with CACHE_DANCE_CACHE_MAP env var" | ||
| RES=$(CACHE_DANCE_CACHE_MAP="$VALID_MAP" "$VALIDATE_SCRIPT" "" "") | ||
| DANCE=$(parse_val "$RES" "cache-dance") | ||
| [ "$DANCE" = "true" ] || (echo "FAIL: expected cache-dance=true from env var fallback, got $DANCE" && exit 1) | ||
| echo "Test 6 passed." | ||
|
|
||
| echo "All cache dance configuration tests completed successfully." | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| #!/bin/bash | ||
| set -e | ||
|
|
||
| # Validates cache-dance configuration inputs. | ||
| # Accepts positional arguments: | ||
| # $1: CACHE_DANCE ("true" | "false" | "") | ||
| # $2: CACHE_MAP (JSON string or path map) | ||
| # Or environment variables: CACHE_DANCE, CACHE_MAP, CACHE_DANCE_CACHE_MAP | ||
|
|
||
| EXPLICIT_DANCE="${1:-${CACHE_DANCE}}" | ||
| CACHE_MAP="${2:-${CACHE_MAP:-${CACHE_DANCE_CACHE_MAP}}}" | ||
|
|
||
| if [ -n "$CACHE_MAP" ]; then | ||
| if [ "$EXPLICIT_DANCE" = "false" ]; then | ||
| CACHE_DANCE="false" | ||
| else | ||
| CACHE_DANCE="true" | ||
| fi | ||
| else | ||
| if [ "$EXPLICIT_DANCE" = "true" ]; then | ||
| echo "::error::cache-map (or cache-dance-cache-map) input is required when cache-dance is true." | ||
| exit 1 | ||
| fi | ||
| CACHE_DANCE="false" | ||
| fi | ||
|
|
||
| if [ "$CACHE_DANCE" = "true" ]; then | ||
| if ! echo "$CACHE_MAP" | jq . >/dev/null 2>&1; then | ||
| echo "::error::cache-map (or cache-dance-cache-map) must be valid JSON." | ||
| exit 1 | ||
| fi | ||
| fi | ||
|
|
||
| echo "cache-dance=${CACHE_DANCE}" | ||
| echo "cache-map=${CACHE_MAP}" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| #!/bin/bash | ||
| set -e | ||
|
|
||
| # Applies Go dependency overrides | ||
| # Env inputs: GO_OVERRIDES | ||
|
|
||
| if [[ -z "${GO_OVERRIDES}" ]]; then | ||
| echo "No Go dependency overrides provided." | ||
| exit 0 | ||
| fi | ||
|
|
||
| echo "Applying Go dependency overrides..." | ||
| while IFS= read -r line || [[ -n "$line" ]]; do | ||
| [[ -z "$line" ]] && continue | ||
| if [[ "$line" != *"="* ]]; then | ||
| echo "::warning::Skipping malformed override line (missing '='): ${line}" | ||
| continue | ||
| fi | ||
|
|
||
| dep="${line%%=*}" | ||
| sha="${line#*=}" | ||
| [[ -z "$dep" || -z "$sha" ]] && continue | ||
|
|
||
| # Qualify module name if short name provided | ||
| if [[ "$dep" != *"/"* ]]; then | ||
| module="github.com/smartcontractkit/${dep}" | ||
| else | ||
| module="${dep}" | ||
| fi | ||
|
Contributor
There was a problem hiding this comment. Is there a use-case for supporting arbitrary overrides, if not I think limiting it to smartcontractkit deps makes the most sense to me. |
||
|
|
||
| echo "Replacing Go module: ${module} -> ${sha}" | ||
| if [[ "${DRY_RUN}" == "true" ]]; then | ||
| echo "[DRY RUN] go mod edit -replace ${module}=${sha}" | ||
| else | ||
| if [[ "$sha" == *"/"* ]]; then | ||
| go mod edit -replace "${module}=${sha}" | ||
| else | ||
| go mod edit -replace "${module}=${module}@${sha}" | ||
| fi | ||
| fi | ||
| done <<< "$GO_OVERRIDES" | ||
|
|
||
| if [[ "${DRY_RUN}" != "true" ]]; then | ||
| go mod tidy | ||
| fi | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These are "deprecated" but they were never released in the first place.