Skip to content

HDFS-17967. Handle InvalidEncryptionKeyException in data-transfer paths - #8698

Open
JHSUYU wants to merge 1 commit into
apache:trunkfrom
JHSUYU:HDFS-17967
Open

HDFS-17967. Handle InvalidEncryptionKeyException in data-transfer paths#8698
JHSUYU wants to merge 1 commit into
apache:trunkfrom
JHSUYU:HDFS-17967

Conversation

@JHSUYU

@JHSUYU JHSUYU commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

JIRA: HDFS-17967

Summary

This is a follow-up of HDFS-17897 and HDFS-17899.

The following data-transfer connection paths still do not handle InvalidEncryptionKeyException:

  • DataXceiver.writeBlock() — mirror pipeline connection
  • DataXceiver.replaceBlock() — proxy copy connection
  • StripedBlockWriter — erasure coding target connection

With data-transfer encryption enabled, block-key rotation or transient RPC failures delaying key propagation can leave DataNodes temporarily out of sync. The SASL handshake
then fails with InvalidEncryptionKeyException.

Fix: Close the failed socket, clear the cached encryption key, and retry once. Retrying is safe because the exception occurs before the downstream writeBlock or copyBlock request is sent. A second failure is propagated.

Test

  • TestDataXceiverEncryptionKey#testWriteBlockRetriesInvalidEncryptionKeyToMirror
  • TestDataXceiverEncryptionKey#testReplaceBlockRetriesInvalidEncryptionKeyToProxy
  • TestStripedBlockWriterEncryptionKey#testClearEncryptionKeyOnRetry

@hadoop-yetus

Copy link
Copy Markdown

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 1m 2s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 43m 17s trunk passed
+1 💚 compile 1m 46s trunk passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚 compile 1m 46s trunk passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚 checkstyle 1m 48s trunk passed
+1 💚 mvnsite 1m 54s trunk passed
+1 💚 javadoc 1m 31s trunk passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚 javadoc 1m 29s trunk passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚 spotbugs 4m 7s trunk passed
+1 💚 shadedclient 31m 31s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 1m 21s the patch passed
+1 💚 compile 1m 15s the patch passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚 javac 1m 15s the patch passed
+1 💚 compile 1m 16s the patch passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚 javac 1m 16s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 1m 16s /results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 34 unchanged - 0 fixed = 35 total (was 34)
+1 💚 mvnsite 1m 26s the patch passed
+1 💚 javadoc 0m 58s the patch passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚 javadoc 1m 0s the patch passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚 spotbugs 3m 45s the patch passed
+1 💚 shadedclient 30m 34s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 275m 11s hadoop-hdfs in the patch passed.
+1 💚 asflicense 0m 48s The patch does not generate ASF License warnings.
407m 16s
Subsystem Report/Notes
Docker ClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8698/1/artifact/out/Dockerfile
GITHUB PR #8698
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux e441f3b8348b 5.15.0-181-generic #191-Ubuntu SMP Fri May 22 19:09:02 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 68adcd9
Default Java Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
Multi-JDK versions /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.11+10-1-24.04.2-Ubuntu /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8698/1/testReport/
Max. process+thread count 3542 (vs. ulimit of 10000)
modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8698/1/console
versions git=2.43.0 maven=3.9.15 spotbugs=4.9.7
Powered by Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants