HADOOP-19971. Remove servlet and Jetty types from classes that do not need them - #8703
Open
joseluisll wants to merge 8 commits into
Open
HADOOP-19971. Remove servlet and Jetty types from classes that do not need them#8703joseluisll wants to merge 8 commits into
joseluisll wants to merge 8 commits into
Conversation
… module classpath
Several modules resolved more than one Jetty release, and more than one servlet
API, on a single classpath. Both combinations compile and then fail at run time,
on whichever code path reaches the wrong jar.
* Four modules used org.eclipse.jetty from their main sources while no pom in
their inheritance chain declared Jetty: hadoop-mapreduce-client-app,
hadoop-mapreduce-client-shuffle, hadoop-yarn-server-router and
hadoop-yarn-server-applicationhistoryservice. Each now declares the artifact
its main sources use, without a version, and jetty-http gains the
dependencyManagement entry it was missing. The Router declares jetty-webapp at
compile scope, since a test scope would override the compile-scoped copy it
inherits and take it off the Router's own runtime classpath.
* jersey-test-framework-provider-jetty carried jetty-continuation onto the test
classpath of around twenty modules at its own, older release. It is managed at
${jetty.version} rather than excluded, because Jersey needs the class:
Continuation is a final field on JettyHttpContainer's ResponseWriter,
initialised in the constructor.
* solr-core carried nine Jetty artifacts, and a servlet API, into
hadoop-yarn-applications-catalog-webapp's tests at a much older release. Those
tests reach Solr only through EmbeddedSolrServer, which runs no servlet
container, so none of it is needed. The exclusions are applied to
solr-test-framework as well, whose own path to solr-core would otherwise
reintroduce the same jars.
* hadoop-project excluded javax.servlet-api from jetty-server under the group
org.eclipse.jetty, which jetty-server has never used, so the exclusion matched
nothing and javax.servlet:javax.servlet-api stayed on the classpath beside the
managed jakarta.servlet:jakarta.servlet-api. Both publish the javax.servlet
packages, so which one a module compiled and ran against was decided by the
order of the jars, and 73 modules carried both. An exclusion naming the real
coordinates is added beside the existing one, and the same on
javax-websocket-server-impl, which reaches the API again through
websocket-servlet. hadoop-common now declares jakarta.servlet-api itself.
* hadoop-common declared jakarta.servlet.jsp-api, which reached some
eighty-five classpaths. Nothing in the tree uses JSP, so it and its managed
version are removed, together with the LICENSE-binary and NOTICE-binary
entries for the two artifacts that no longer ship.
One module keeps two servlet APIs, hadoop-yarn-server-timelineservice-hbase-tests,
where the second arrives with HBase's own test stack.
Co-Authored-By: Claude <noreply@anthropic.com>
jersey-test-framework-provider-jetty put jetty-continuation on the test classpath of around twenty modules at its own, older Jetty release. Managing that artifact pins the version but leaves Jersey's Jetty 9 container there. Switch to jersey-test-framework-provider-grizzly2 instead. It is the same Jersey release, it speaks real HTTP, and it puts no Jetty on a test classpath at all, so the jetty-continuation entry and its shade exclusion go with it. It brings no second servlet API: grizzly-http-servlet declares javax.servlet-api at provided scope, which is not transitive, and jersey-container-grizzly2-servlet declares the jakarta.servlet-api coordinate this tree already manages. LICENSE-binary gains the four Grizzly artifacts and the two jersey-container-grizzly2 ones, which reach hadoop-client-minicluster at compile scope. Co-Authored-By: Claude <noreply@anthropic.com>
joseluisll
changed the title
JobEndNotifier logged through org.eclipse.jetty.util.log.Log, Jetty's own logging facade, which is the only reason hadoop-mapreduce-client-app needed a Jetty dependency at all. Use SLF4J, as the rest of the tree does, and the declaration added earlier in this PR goes with it. The module now names no Jetty type anywhere, in main sources or tests. Raised by pan3793 in review. Co-Authored-By: Claude <noreply@anthropic.com>
…in javadoc. The import is unused: only a javadoc @link mentions the type. Spell that link out instead, and the class no longer references the servlet API at all. Co-Authored-By: Claude <noreply@anthropic.com>
…a PEM. CertificateUtil parses a PEM and has nothing to do with servlets. It used ServletException only to wrap the CertificateException it already catches. The new toRSAPublicKey reports that exception directly and names no servlet type. parseRSAPublicKey stays, deprecated, delegating to it and wrapping the exception back, so callers outside this tree keep compiling and keep seeing the same exception and message. It goes with the move to jakarta, where its signature has to change anyway. Co-Authored-By: Claude <noreply@anthropic.com>
…ider. SignerSecretProvider.init takes a ServletContext, so every implementation names a servlet type whether or not it uses one. Only ZKSignerSecretProvider really needs it, and what it needs is an attribute store, not a servlet: it shares a CuratorFramework as a context attribute. SecretProviderContext is that store, and initialize(Properties, SecretProviderContext, long) is the method providers should now override. The four providers here do, and none of them names a servlet type any more. Nothing breaks. init stays, deprecated, and stays the entry point callers use. It is now concrete: it adapts the ServletContext and calls initialize, so a provider overriding only initialize is initialized through it, while a provider overriding init is called directly, as before. Implementations live outside this tree - signer.secret.provider takes a classname - and keep compiling, linking and running untouched. The bridge dispatches virtually, so a provider extending RolloverSignerSecretProvider, overriding init and chaining to super still gets its rollover scheduler started. TestSignerSecretProviderCompatibility pins that and the rest of the contract. The adapter writes through to the real ServletContext, so the curator client is still a ServletContext attribute under the same name, as Configuration.md documents and as DelegationTokenAuthenticationFilter expects. A null ServletContext, which used to throw, now yields a store private to the provider. AuthenticationFilter and its tests are unchanged, which is the point. Co-Authored-By: Claude <noreply@anthropic.com>
It used org.eclipse.jetty.util.UrlEncoded to percent-encode a user name. java.net.URLEncoder does the same job from the JDK, so the class no longer needs Jetty. Co-Authored-By: Claude <noreply@anthropic.com>
ShuffleChannelHandler is a Netty handler. It imported org.eclipse.jetty.http.HttpHeader only to name the Connection and Keep-Alive headers. That was the module's last Jetty reference, so the jetty-http declaration HADOOP-19970 added to its pom goes too. The names are spelled as constants rather than taken from Netty's HttpHeaderNames and HttpHeaderValues, whose constants are lowercase AsciiStrings: using those would have sent "connection: keep-alive" where this handler has always sent "Connection: Keep-Alive". The shuffle response is read by other projects, so no byte of it changes here. Co-Authored-By: Claude <noreply@anthropic.com>
joseluisll
force-pushed
the
jetty-phase-b
branch
from
August 26, 2026 16:35
7f3cccc to
784374a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of PR
Stacked on #8699 (HADOOP-19970). The first two commits belong to that PR — review the last five.
Hadoop will eventually move from
javax.servlettojakarta.servlet. When it does, every class that mentions a servlet type has to be revisited, and every class that mentions a Jetty type has to be revisited again when the Jetty API changes underneath it.Several of those classes have no real connection to either. A component that generates signing secrets was handed the web server's context even though it only wanted somewhere to stash an object. A certificate parser reported a bad certificate as a web server error. A file-transfer handler built on Netty borrowed two header names from Jetty.
This clears those cases out, so the later migration is smaller and the classes concerned stop being part of it.
The MapReduce shuffle module stops depending on Jetty altogether, in code and in its build file. In hadoop-auth, the secret providers and the authentication token no longer mention the servlet API; they are initialised instead against a small two-method store that holds whatever needs sharing, which is all the one provider that used the servlet context was ever doing with it. One new class does mention the servlet API, and exists solely to keep the old path working. The YARN services client and the shuffle handler each replaced a small Jetty utility with the equivalent from the JDK or from Netty.
Nothing breaks. No public signature is removed or changed, and no dependency, version or setting changes. Code built against today's release keeps compiling and keeps running without being rebuilt.
SignerSecretProvider.initandCertificateUtil.parseRSAPublicKeystay, deprecated, and stay the entry points callers use; a cleaner alternative sits alongside each. Both deprecated methods go with the namespace change, where they have to change anyway.The shuffle protocol is untouched, byte for byte.
Deliberately out of scope: no namespace change, no Jetty upgrade, no Jersey change, no EE environments — the tree stays on Jetty 9.4, Jersey 2 and
javax.servletthroughout. The authentication handler classes still mention the servlet API, because they sit on the extension point HBase, Hive, Spark, Ozone and Knox build against; changing that is precisely the break this avoids, and it belongs with the namespace change.This works whether HADOOP-19912 goes through Jetty 12's ee8 environment or straight to ee10, and commits the project to neither.
How was this patch tested?
TestSignerSecretProviderCompatibilityis new and exists to prove the deprecated path still behaves — a provider written against the old entry point is still called, still receives the realServletContext, and a provider extendingRolloverSignerSecretProviderthat chains tosuperstill has its rollover scheduler started.TestCertificateUtilasserts the deprecated method still reports the exception it always did.AuthenticationFilter,TestAuthenticationFilterandTestZKSignerSecretProviderare unchanged from trunk, which is the clearest evidence the change is compatible.hadoop-auth: 52 tests, 0 failures.
TestShuffleChannelHandler: 8 tests, 0 failures.For code changes:
declared according to the connector-specific documentation? Note: Automated CI
testing doesn't cover all cases so manual testing with cloud storage is still
required.
LICENSE,LICENSE-binary,NOTICE-binaryfiles?No dependencies are added or removed; one is dropped from the shuffle module's pom.
AI Tooling
Contains content generated by Claude.
where is the name of the AI tool used.
https://www.apache.org/legal/generative-tooling.html