ADBM-3251: Implement decompression for pgbackrest repo-get

[dkovalev1]

Implement optional decompression for repo-get command

This commit introduces optional flag --decompress. If specified, pgBackRest automatically decompresses the file before writing to stdout. The compression algorithm is determined from the backup manifest (the same algorithm that was used when the backup was created). If decompression fails (e.g., the file is not compressed or the manifest entry is missing), an error is returned. Without this flag, the raw (possibly compressed) content is output.

Also, it is possible to override the compression algorithm specified in the backup manifest using compress-type option.

[dkovalev1]

The main path works fine for me:
spoiler

But I have problem when encryption is enabled:
spoiler

In SRS we have

The file is read from the repository storage. If encryption is enabled, it is automatically decrypted (transparently).

So, looks like it's the problem

yes, and it took to refactor the whole patch to fix it.

[dkovalev1]

It's still doesn't work for encrypted archive files =(
spoiler

It appears that to get a backup manifest, a different pass is needed, no the same as for archive info and the file. Made passwords processing more complicated.

Also, it appears that this case can not be reproduced in the unit tests, as it depends heavily on the logic from the backup command and archiving subsystem. I suggest to leave this case for manual testing.

[dkovalev1]

But I have problem with REPO:BACKUP and REPO:ARCHIVE in encrypted repo. =(

This is the case where stanza is really required. Added one more check.

Spoiler

rm -rf /home/denis/.data/pgbackrest-enc
mkdir -p /home/denis/.data/pgbackrest-enc

cat >/tmp/pgbackrest-enc.conf <<EOF
[global]
fork=GPDB
log-level-console=info
repo1-path=/home/denis/.data/pgbackrest-enc
repo1-cipher-type=aes-256-cbc
repo1-cipher-pass=test-passphrase
compress-type=gz
archive-check=n

[seg0]
pg1-path=/home/denis/arena/gg6/gpAux/gpdemo/datadirs/dbfast1/demoDataDir0/
pg1-port=6002
EOF

gpconfig -c archive_mode -v on
gpconfig -c archive_command --skipvalidation -v "'PGOPTIONS=\"-c gp_session_role=utility\" /home/denis/arena/pgbackrest/build/src/pgbackrest --config=/tmp/pgbackrest-enc.conf --stanza=seg%c archive-push %p'"
gpstop -afr

PGOPTIONS="-c gp_session_role=utility" \
/home/denis/arena/pgbackrest/build/src/pgbackrest --config=/tmp/pgbackrest-enc.conf --stanza=seg0 stanza-create

PGOPTIONS="-c gp_session_role=utility" \
/home/denis/arena/pgbackrest/build/src/pgbackrest --config=/tmp/pgbackrest-enc.conf --stanza=seg0 backup --type=full --no-archive-check

ls -1 /home/denis/.data/pgbackrest-enc/backup/seg0
# 20260713-191332F

export ENC_SET="20260713-191332F"

find /home/denis/.data/pgbackrest-enc/backup/seg0/$ENC_SET -type f -name '*.gz' | head -1
# /home/denis/.data/pgbackrest-enc/backup/seg0/20260713-191332F/pg_data/pg_hba.conf.gz

export FILE_NAME=pg_data/pg_hba.conf.gz

/home/denis/arena/pgbackrest/build/src/pgbackrest --config=/tmp/pgbackrest-enc.conf --stanza=seg0 \
repo-get "backup/seg0/$ENC_SET/$FILE_NAME" | gzip -cd | head -1
# 9.4

/home/denis/arena/pgbackrest/build/src/pgbackrest --config=/tmp/pgbackrest-enc.conf --stanza=seg0 --set=$ENC_SET \
repo-get --decompress "backup/seg0/$ENC_SET/$FILE_NAME"
# 9.4

/home/denis/arena/pgbackrest/build/src/pgbackrest --config=/tmp/pgbackrest-enc.conf --stanza=seg0 --set=$ENC_SET \
repo-get --decompress "<REPO:BACKUP>/$ENC_SET/$FILE_NAME"
# 9.4

/home/denis/arena/pgbackrest/build/src/pgbackrest --config=/tmp/pgbackrest-enc.conf --compress-type=gz \
repo-get --decompress "<REPO:BACKUP>/seg0/$ENC_SET/$FILE_NAME"

# error:
# 2026-07-13 19:53:29.623 P00  ERROR: [043]: stanza required

The same for archive command.

Spoiler

create extension gp_pitr;
select gp_create_restore_point('backup1');
select gp_switch_wal();

ls  /home/denis/.data/pgbackrest-enc/archive/seg0/9.4-1/0000000100000000
#000000010000000000000001-7d340d277852ffddd5831094608b7073a202c314.gz
#000000010000000000000002-e69f49ad5b5453c52083872b7bb117c6861e4720.gz

export ARCH_FILE="archive/seg0/9.4-1/0000000100000000/000000010000000000000001-7d340d277852ffddd5831094608b7073a202c314.gz"
export ARCH_FILE_ALIAS="<REPO:ARCHIVE>/seg0/9.4-1/0000000100000000/000000010000000000000001-7d340d277852ffddd5831094608b7073a202c314.gz"
export ARCH_FILE_ALIAS_2="<REPO:ARCHIVE>/9.4-1/0000000100000000/000000010000000000000001-7d340d277852ffddd5831094608b7073a202c314.gz"

/home/denis/arena/pgbackrest/build/src/pgbackrest --config=/tmp/pgbackrest-enc.conf --stanza=seg0 \
repo-get "$ARCH_FILE" | gzip -cd | head -3

# ~����l�Tj�M�
# ����J@v1�@v1 �)�2�  ����� ���
#                              @gp_pitr
# �1.17
# 
#       +0plpgsql

ls -1 /home/denis/.data/pgbackrest-enc/backup/seg0
# 20260713-191332F

export ENC_SET="20260713-191332F"

/home/denis/arena/pgbackrest/build/src/pgbackrest --config=/tmp/pgbackrest-enc.conf --stanza=seg0 --set="$ENC_SET" \
repo-get --decompress "$ARCH_FILE" | head -3

# ~����l�Tj�M�
# ����J@v1�@v1 �)�2�  ����� ���
#                              @gp_pitr
# �1.17
# 
#       +0plpgsql

/home/denis/arena/pgbackrest/build/src/pgbackrest --config=/tmp/pgbackrest-enc.conf --stanza=seg0 --set="$ENC_SET" \
repo-get --decompress "$ARCH_FILE_ALIAS_2" | head -3
# ~����l�Tj�M�
# ����J@v1�@v1 �)�2�  ����� ���
#                              @gp_pitr
# �1.17
# 
#       +0plpgsql

/home/denis/arena/pgbackrest/build/src/pgbackrest --config=/tmp/pgbackrest-enc.conf --compress-type=gz \
repo-get --decompress "$ARCH_FILE_ALIAS" | head -3

# error:
#
# 2026-07-13 19:57:21.276 P00  ERROR: [043]: stanza required

[hilltracer]

Do I need to specify something? My command is failing:

did you define alias with /9.4-1?

ARCH_FILE_ALIAS_2="<REPO:ARCHIVE>/9.4-1/...

[ShapatinAA]

did you define alias with /9.4-1?

No, I defined nothing.