HDFS-17948. Improve FedBalance DistCp option handling.

[qijiale76]

Description of PR

This PR improves how FedBalance builds the DistCp commands it submits.

Previously, FedBalance used a mostly fixed DistCp option set. This patch adds operator-facing controls for several DistCp settings:

• Add -skipAclPreserve to disable ACL preservation.
• Add hdfs.fedbalance.preserve.acl.enabled to control ACL preservation from configuration.
• Preserve ACLs only when both source and destination filesystems support ACLs.
• Add -preserveTimes to preserve file timestamps.
• Add -distcpStrategy to pass a DistCp copy strategy.
• Add -numListstatusThreads to pass DistCp listStatus thread count.
• Register -diffThreshold in the shared FedBalance CLI options.
• Serialize the new options in FedBalanceContext so recovered jobs use the same DistCp settings.
• Update FedBalance documentation and tests.

The default behavior remains conservative: timestamp preservation is disabled unless requested, DistCp strategy and listStatus thread count use DistCp defaults unless specified, and ACL preservation remains enabled only when ACLs are supported on both sides.

How was this patch tested?

Ran the related FedBalance unit tests.

This change has also been validated on a production cluster.

AI Tooling

Contains content generated by Codex and Claude Code.

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 35s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+0 🆗	markdownlint	0m 0s		markdownlint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	309m 34s		Maven dependency ordering for branch
+1 💚	mvninstall	56m 58s		trunk passed
+1 💚	compile	8m 35s		trunk passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚	compile	8m 42s		trunk passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚	checkstyle	3m 5s		trunk passed
+1 💚	mvnsite	1m 31s		trunk passed
+1 💚	javadoc	1m 15s		trunk passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚	javadoc	1m 13s		trunk passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
-1 ❌	spotbugs	0m 43s	/branch-spotbugs-hadoop-tools_hadoop-federation-balance-warnings.html	hadoop-tools/hadoop-federation-balance in trunk has 12 extant spotbugs warnings.
+1 💚	shadedclient	15m 42s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 16s		Maven dependency ordering for patch
+1 💚	mvninstall	0m 48s		the patch passed
+1 💚	compile	8m 6s		the patch passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚	javac	8m 6s		the patch passed
+1 💚	compile	8m 37s		the patch passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚	javac	8m 37s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	2m 59s		the patch passed
+1 💚	mvnsite	1m 30s		the patch passed
-1 ❌	javadoc	0m 34s	/results-javadoc-javadoc-hadoop-tools_hadoop-federation-balance-jdkUbuntu-21.0.11+10-1-24.04.2-Ubuntu.txt	hadoop-tools_hadoop-federation-balance-jdkUbuntu-21.0.11+10-1-24.04.2-Ubuntu with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu generated 6 new + 97 unchanged - 0 fixed = 103 total (was 97)
-1 ❌	javadoc	0m 35s	/results-javadoc-javadoc-hadoop-tools_hadoop-federation-balance-jdkUbuntu-17.0.19+10-1-24.04.2-Ubuntu.txt	hadoop-tools_hadoop-federation-balance-jdkUbuntu-17.0.19+10-1-24.04.2-Ubuntu with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu generated 6 new + 93 unchanged - 0 fixed = 99 total (was 93)
+1 💚	spotbugs	2m 23s		the patch passed
+1 💚	shadedclient	16m 40s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	1m 59s		hadoop-federation-balance in the patch passed.
+1 💚	unit	37m 58s		hadoop-hdfs-rbf in the patch passed.
+1 💚	asflicense	0m 44s		The patch does not generate ASF License warnings.
		500m 20s

Subsystem	Report/Notes
Docker	ClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8579/1/artifact/out/Dockerfile
GITHUB PR	#8579
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint markdownlint
uname	Linux bf9e176846ad 5.15.0-181-generic #191-Ubuntu SMP Fri May 22 19:09:02 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 53b09e0
Default Java	Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
Multi-JDK versions	/usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.11+10-1-24.04.2-Ubuntu /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8579/1/testReport/
Max. process+thread count	4150 (vs. ulimit of 10000)
modules	C: hadoop-tools/hadoop-federation-balance hadoop-hdfs-project/hadoop-hdfs-rbf U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8579/1/console
versions	git=2.43.0 maven=3.9.15 spotbugs=4.9.7
Powered by	Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 20s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+0 🆗	markdownlint	0m 0s		markdownlint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	1m 49s		Maven dependency ordering for branch
+1 💚	mvninstall	25m 55s		trunk passed
+1 💚	compile	8m 13s		trunk passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚	compile	8m 42s		trunk passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚	checkstyle	2m 54s		trunk passed
+1 💚	mvnsite	1m 27s		trunk passed
+1 💚	javadoc	1m 19s		trunk passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚	javadoc	1m 13s		trunk passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
-1 ❌	spotbugs	0m 45s	/branch-spotbugs-hadoop-tools_hadoop-federation-balance-warnings.html	hadoop-tools/hadoop-federation-balance in trunk has 12 extant spotbugs warnings.
+1 💚	shadedclient	16m 16s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 16s		Maven dependency ordering for patch
+1 💚	mvninstall	0m 43s		the patch passed
+1 💚	compile	8m 6s		the patch passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚	javac	8m 6s		the patch passed
+1 💚	compile	8m 45s		the patch passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚	javac	8m 45s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	2m 56s		the patch passed
+1 💚	mvnsite	1m 29s		the patch passed
+1 💚	javadoc	0m 33s		hadoop-tools_hadoop-federation-balance-jdkUbuntu-21.0.11+10-1-24.04.2-Ubuntu with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu generated 0 new + 88 unchanged - 9 fixed = 88 total (was 97)
+1 💚	javadoc	0m 41s		hadoop-hdfs-rbf in the patch passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu.
+1 💚	javadoc	0m 34s		hadoop-tools_hadoop-federation-balance-jdkUbuntu-17.0.19+10-1-24.04.2-Ubuntu with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu generated 0 new + 84 unchanged - 9 fixed = 84 total (was 93)
+1 💚	javadoc	0m 38s		hadoop-hdfs-rbf in the patch passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu.
+1 💚	spotbugs	2m 19s		the patch passed
+1 💚	shadedclient	16m 41s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	2m 14s		hadoop-federation-balance in the patch passed.
+1 💚	unit	37m 41s		hadoop-hdfs-rbf in the patch passed.
+1 💚	asflicense	0m 48s		The patch does not generate ASF License warnings.
		160m 55s

Subsystem	Report/Notes
Docker	ClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8579/2/artifact/out/Dockerfile
GITHUB PR	#8579
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint markdownlint
uname	Linux 96a9e1fddad0 5.15.0-181-generic #191-Ubuntu SMP Fri May 22 19:09:02 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / f3be380
Default Java	Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
Multi-JDK versions	/usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.11+10-1-24.04.2-Ubuntu /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8579/2/testReport/
Max. process+thread count	3691 (vs. ulimit of 10000)
modules	C: hadoop-tools/hadoop-federation-balance hadoop-hdfs-project/hadoop-hdfs-rbf U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8579/2/console
versions	git=2.43.0 maven=3.9.15 spotbugs=4.9.7
Powered by	Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.