Skip to content

fix: close IO resources left open on abrupt or failing paths - #3532

Merged
pjfanning merged 1 commit into
apache:mainfrom
pjfanning:fix-io-resource-leaks
Sep 8, 2026
Merged

fix: close IO resources left open on abrupt or failing paths#3532
pjfanning merged 1 commit into
apache:mainfrom
pjfanning:fix-io-resource-leaks

Conversation

@pjfanning

Copy link
Copy Markdown
Member

Motivation

An audit of src/main for resource leaks turned up three places that open an IO resource and leave it open on at least one reachable path:

  • ResolvConfParser.parseFile never closes the Stream returned by Files.lines, so the fd on /etc/resolv.conf stays open until GC. This is the only use of Files.lines in the tree.
  • InputStreamSource.postStop fails the materialized value without closing the user-supplied InputStream. Every handler path closes it, but abrupt termination (materializer or actor system shutdown) skips the handlers, so the stream leaks. The sibling stages FileSource and OutputStreamGraphStage already close in postStop, so this one was the odd one out.
  • ArteryAeronUdpTransport.autoSelectPort leaks the DatagramChannel when bind throws.

Modification

Close the Files.lines stream in a finally. parseLines consumes the iterator eagerly, so an eager close is safe.

Close the input stream in InputStreamSource.postStop before failing the promise. The close is inlined rather than delegated to closeInputStream, because that helper routes failures through failStage, which is pointless on an already torn-down stage. A close failure is logged at debug and the promise still fails with AbruptStageTerminationException.

Wrap the bind in autoSelectPort in try/finally.

All three are method-body changes with no signature change.

Result

None of the three paths leaks a file descriptor.

Tests

  • sbt "stream-tests/testOnly org.apache.pekko.stream.io.InputStreamSourceSpec" — 9 succeeded, 0 failed. The new case "close the input stream on actor materializer shutdown" fails without the InputStreamSource change (8 succeeded, 1 failed) and passes with it.
  • sbt "actor/compile" "stream/compile" "remote/compile" — success
  • sbt "actor/mimaReportBinaryIssues" "stream/mimaReportBinaryIssues" "remote/mimaReportBinaryIssues" — success
  • Native scalafmt run on the four changed files.
  • No test for the ResolvConfParser and autoSelectPort fixes: neither changes observable behaviour, and asserting that an fd was released is not possible portably from Scala. Happy to add a Linux-guarded fd-count assertion if reviewers want one.

References

None - found by a resource-leak audit of src/main

Motivation:
Three places under src/main open an IO resource and leave it open on at
least one reachable path:

- ResolvConfParser.parseFile never closes the Stream returned by
  Files.lines, so the fd on /etc/resolv.conf stays open until GC. This is
  the only use of Files.lines in the tree.
- InputStreamSource.postStop fails the materialized value without closing
  the user-supplied InputStream. Every handler path closes it, but abrupt
  termination (materializer or actor system shutdown) skips the handlers,
  so the stream leaks. The sibling stages FileSource and
  OutputStreamGraphStage already close in postStop.
- ArteryAeronUdpTransport.autoSelectPort leaks the DatagramChannel when
  bind throws.

Modification:
Close the Files.lines stream in a finally; parseLines consumes the
iterator eagerly, so an eager close is safe.

Close the input stream in InputStreamSource.postStop before failing the
promise. The close is inlined rather than delegated to closeInputStream,
because that helper routes failures through failStage, which is pointless
on an already torn-down stage. A close failure is logged at debug and the
promise still fails with AbruptStageTerminationException.

Wrap the bind in autoSelectPort in try/finally.

Result:
None of the three paths leaks a file descriptor.

Tests:
- sbt "stream-tests/testOnly org.apache.pekko.stream.io.InputStreamSourceSpec" - 9 succeeded, 0 failed. The new case
  "close the input stream on actor materializer shutdown" fails without the InputStreamSource change (8 succeeded,
  1 failed) and passes with it.
- sbt "actor/compile" "stream/compile" "remote/compile" - success
- sbt "actor/mimaReportBinaryIssues" "stream/mimaReportBinaryIssues" "remote/mimaReportBinaryIssues" - success
- Native scalafmt run on the four changed files.
- No test for the ResolvConfParser and autoSelectPort fixes: neither changes observable behaviour, and asserting an
  fd was released is not possible portably from Scala.

References:
None - found by a resource-leak audit of src/main
Comment on lines 35 to +39
Try {
parseLines(Files.lines(file.toPath).iterator().asScala)
// the stream holds the file open until closed, and parseLines consumes it eagerly
val lines = Files.lines(file.toPath)
try parseLines(lines.iterator().asScala)
finally lines.close()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe use scala.util.Using?

    scala.util.Using(Files.lines(file.toPath)) { lines =>
      parseLines(lines.iterator().asScala)
    }

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current code base prefers Java try with resources probably because Scala Using is not supported directly by Scala 2.12. If we want to start using Scala Using, I think we should have a general PR to do that across existing call sites. Apparently, it is pretty efficient and shouldn't be significantly different performance wise to try with resources.

try {
socket.bind(new InetSocketAddress(hostname, 0))
socket.getLocalPort
} finally socket.close()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can also be used here:

  Using.resource(DatagramChannel.open().socket()) { socket =>
    socket.bind(new InetSocketAddress(hostname, 0))
    socket.getLocalPort
  }

@pjfanning
pjfanning merged commit 14dc8a6 into apache:main Sep 8, 2026
10 checks passed
@pjfanning
pjfanning deleted the fix-io-resource-leaks branch September 8, 2026 08:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants