Skip to content

YARN-11985: Placement rule does not allow to select Leaf Queue as Parent although Dynamic Queue Creation is turned on - #8686

Open
susheelgupta7 wants to merge 4 commits into
apache:trunkfrom
susheelgupta7:YARN-11985
Open

YARN-11985: Placement rule does not allow to select Leaf Queue as Parent although Dynamic Queue Creation is turned on#8686
susheelgupta7 wants to merge 4 commits into
apache:trunkfrom
susheelgupta7:YARN-11985

Conversation

@susheelgupta7

@susheelgupta7 susheelgupta7 commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Contains content generated by Cursor.

Description of PR

When creating a placement rule in the CS UI, the parent queue dropdown did not show queues that had dynamic queue creation enabled but no static child queues.
Now a queue is considered a parent if it either has child queues or has dynamic queue creation enabled.

How was this patch tested?

Tested manually on the cluster.

For code changes:

  • Does the title of this PR start with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • Object storage: Have the integration tests been executed and the endpoint
    declared according to the connector-specific documentation? Note: Automated CI
    testing doesn't cover all cases so manual testing with cloud storage is still
    required.
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

AI Tooling

If an AI tool was used:

@hadoop-yetus

Copy link
Copy Markdown

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 34s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ trunk Compile Tests _
+1 💚 mvninstall 41m 10s trunk passed
+1 💚 shadedclient 69m 46s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 1m 27s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 shadedclient 27m 26s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 asflicense 0m 46s The patch does not generate ASF License warnings.
101m 57s
Subsystem Report/Notes
Docker ClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8686/1/artifact/out/Dockerfile
GITHUB PR #8686
Optional Tests dupname asflicense shadedclient codespell detsecrets
uname Linux 08ead813ddc7 5.15.0-186-generic #196-Ubuntu SMP Sat Jun 20 16:09:34 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 8a94d52
Max. process+thread count 633 (vs. ulimit of 10000)
modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-capacity-scheduler-ui U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-capacity-scheduler-ui
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8686/1/console
versions git=2.43.0 maven=3.9.15
Powered by Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

@slfan1989 slfan1989 changed the title YARN-11985: Placement rule does not allow to select Leaf Queue as Par… YARN-11985: Placement rule does not allow to select Leaf Queue as Parent although Dynamic Queue Creation is turned on Aug 15, 2026

@slfan1989 slfan1989 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on YARN-11985. The overall approach makes sense and allows queues with Dynamic Queue Creation enabled to be selected as placement-rule parents even when they do not currently have child queues.

I found one issue where the live autoCreationEligibility takes precedence over a staged disable, which can leave an invalid queue in the parent dropdown. Please address the inline comment and add focused unit tests for live legacy/flexible eligibility, staged enable, and staged disable.

eligibility === AUTO_CREATION_PROPS.ELIGIBILITY_FLEXIBLE ||
eligibility === AUTO_CREATION_PROPS.ELIGIBILITY_LEGACY
) {
return true;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This early return prevents the staged configuration from taking effect. If the queue is currently flexible, but the user stages both auto-creation properties to false, the queue will still be included in the parent queue dropdown.

Could we read both values from getQueuePropertyValue first and let staged values take precedence, falling back to autoCreationEligibility only when neither property has a staged override?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @slfan1989 for the review. I’ve fixed the precedence issue.

@hadoop-yetus

Copy link
Copy Markdown

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 12m 53s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ trunk Compile Tests _
+1 💚 mvninstall 40m 48s trunk passed
+1 💚 shadedclient 69m 33s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 1m 28s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 shadedclient 27m 26s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 asflicense 0m 45s The patch does not generate ASF License warnings.
114m 3s
Subsystem Report/Notes
Docker ClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8686/2/artifact/out/Dockerfile
GITHUB PR #8686
Optional Tests dupname asflicense shadedclient codespell detsecrets
uname Linux 22ffbaf36e04 5.15.0-186-generic #196-Ubuntu SMP Sat Jun 20 16:09:34 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / a27c2f0
Max. process+thread count 613 (vs. ulimit of 10000)
modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-capacity-scheduler-ui U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-capacity-scheduler-ui
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8686/2/console
versions git=2.43.0 maven=3.9.15
Powered by Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

@K0K0V0K

K0K0V0K commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Thanks @susheelgupta7 for the fix!

FYI, the fix can be also validated with the built in dev server and mock data.
Before the change the root.marketing.test queue is not visible on the Add Placement Rule view, but after the change it is visible.

LGTM

@hadoop-yetus

Copy link
Copy Markdown

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 34s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ trunk Compile Tests _
+1 💚 mvninstall 41m 58s trunk passed
+1 💚 shadedclient 70m 36s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 1m 25s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 shadedclient 27m 24s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ asflicense 0m 45s /results-asflicense.txt The patch generated 1 ASF License warnings.
102m 31s
Subsystem Report/Notes
Docker ClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8686/3/artifact/out/Dockerfile
GITHUB PR #8686
Optional Tests dupname asflicense shadedclient codespell detsecrets
uname Linux ac1c53ffb1ad 5.15.0-186-generic #196-Ubuntu SMP Sat Jun 20 16:09:34 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / b8ff710
Max. process+thread count 639 (vs. ulimit of 10000)
modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-capacity-scheduler-ui U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-capacity-scheduler-ui
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8686/3/console
versions git=2.43.0 maven=3.9.15
Powered by Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus

Copy link
Copy Markdown

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 35s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+0 🆗 detsecrets 0m 1s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ trunk Compile Tests _
+1 💚 mvninstall 43m 55s trunk passed
+1 💚 shadedclient 72m 31s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 1m 26s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 shadedclient 27m 31s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ asflicense 0m 47s /results-asflicense.txt The patch generated 1 ASF License warnings.
104m 35s
Subsystem Report/Notes
Docker ClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8686/4/artifact/out/Dockerfile
GITHUB PR #8686
Optional Tests dupname asflicense shadedclient codespell detsecrets
uname Linux 3e6a6b971061 5.15.0-186-generic #196-Ubuntu SMP Sat Jun 20 16:09:34 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 7e09aa4
Max. process+thread count 613 (vs. ulimit of 10000)
modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-capacity-scheduler-ui U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-capacity-scheduler-ui
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8686/4/console
versions git=2.43.0 maven=3.9.15
Powered by Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus

Copy link
Copy Markdown

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 0s Docker mode activated.
-1 ❌ patch 0m 23s #8686 does not apply to trunk. Rebase required? Wrong Branch? See https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute for help.
Subsystem Report/Notes
GITHUB PR #8686
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8686/5/console
versions git=2.34.1
Powered by Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus

Copy link
Copy Markdown

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 34s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ trunk Compile Tests _
+1 💚 mvninstall 42m 36s trunk passed
+1 💚 shadedclient 71m 4s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 1m 26s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 shadedclient 27m 27s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ asflicense 0m 46s /results-asflicense.txt The patch generated 1 ASF License warnings.
103m 16s
Subsystem Report/Notes
Docker ClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8686/6/artifact/out/Dockerfile
GITHUB PR #8686
Optional Tests dupname asflicense shadedclient codespell detsecrets
uname Linux 188ec74c1e9b 5.15.0-186-generic #196-Ubuntu SMP Sat Jun 20 16:09:34 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 9d9ccaf
Max. process+thread count 610 (vs. ulimit of 10000)
modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-capacity-scheduler-ui U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-capacity-scheduler-ui
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8686/6/console
versions git=2.43.0 maven=3.9.15
Powered by Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

@susheelgupta7

Copy link
Copy Markdown
Contributor Author

@slfan1989 @brumi1024 The license warning is unrelated change causing the Yetus failure. Could you review this fix again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants