Implement Workspace Diagnostic Scanning (Issue 31)

[Moataz-Aldawood]

Pull Request: Implement Workspace Diagnostic Scanning (Issue #31)

Overview

This PR introduces a much-requested feature: the ability to proactively scan an entire workspace for compilation errors and warnings without requiring the user to open every file manually. It fulfills Issue #31 and also addresses exclusion glob-patterns requested in Issue #19.

---

Changes & Features Introduced

1. Workspace Scanning Configuration:

   • netbeans.autoScanWorkspace: A new boolean setting (default: false) that allows users to automatically trigger a full workspace diagnostic scan whenever they load a project.
   • netbeans.scanExclude: A new array configuration (default: ["**/node_modules/**", "**/target/**", "**/build/**"]) allowing users to define glob patterns for directories that should be skipped during the scan to save resources.

2. New User Commands:

   • nbls.workspace.scan (Java: Scan Workspace for Diagnostics): Allows the user to manually trigger a background scan of all .java files in the workspace at any time. It includes a cancellable VS Code progress bar.
   • nbls.workspace.scan.clear (Java: Clear Workspace Diagnostics): A utility command to instantly clear all background scan results from the custom VS Code Problems panel collection.

3. Background Scanner Logic (extension.ts):

   • Implemented doWorkspaceScan which batch processes files by invoking the backend nbls.get.diagnostics command in small chunks. This prevents overloading the LSP server with hundreds of simultaneous parsing requests.
   • Added specific client-side filtering to gracefully drop massive ExceptionInInitializerError stack traces caused by upstream Lombok incompatibilities.

---

Fixes

Critical Bug Fix in TextDocumentServiceImpl.java
While implementing the scanner, we discovered a major bug in the computeDiagnostics method inside the backend language server. Previously, when the server computed errors and hints simultaneously, the resulting result list was being overwritten rather than aggregated. This caused standard compilation errors to randomly disappear or be completely overwritten by simple code hints.

• Fix: Refactored computeDiagnostics to use an ArrayList<> and .addAll() so that both ERRORS and HINTS are correctly merged and returned to the client in a single payload.

---

Challenges Encountered

Lombok Compatibility on master Branch:
During testing on the master branch, we encountered a significant challenge with Lombok throwing fatal java.lang.NoClassDefFoundError: Could not initialize class lombok.javac.Javac exceptions.

• Investigation: We discovered this is an upstream regression in the apache/netbeans master branch. The NetBeans team recently updated the internal nbjavac compiler to Java 22/23 and unfortunately dropped the EndPosTable compatibility patch (which existed in NetBeans 21) that allowed Lombok to function on newer JDKs.
• Resolution: Because this is an upstream compiler issue, we cannot fix it inside the VS Code extension wrapper. We implemented a temporary client-side filter in the scanner to prevent these massive, noisy stack traces from flooding the user's Problems panel, whilst continuing to report the valid Java cannot find symbol errors that result from the failed Lombok processing.

Native LSP Diagnostic Caching:
We encountered confusion regarding lingering diagnostics in the Problems panel after closing files. This is due to the native vscode-languageclient architecture where the backend server does not explicitly send empty arrays ([]) to clear diagnostics when files are closed. We solved this from the user's perspective by keeping our background scan results isolated in a separate, clearable custom bucket (projectDiagnosticCollection).

---

Recommendations for Upstream

1. Restore EndPosTable in nbjavac: We highly recommend filing a bug against the core apache/netbeans repository to reinstate the dummy EndPosTable patch in nbjavac for Java 22/23. Until this is fixed upstream, Lombok users running the latest NetBeans Language Server on modern JDKs will experience completely broken annotation processing.
2. Empty Diagnostic Arrays on didClose: The NetBeans Language Server should be updated to actively send publishDiagnostics(uri, []) when it receives a didClose notification for a file, ensuring that the VS Code Problems panel cleans up gracefully when a user closes a broken file.

---

PR approval and merge checklist:

1. Is this PR squashed?
2. Are author name / email address correct? Are co-authors correctly listed? Do the commit messages need updates?
3. Does the PR title and description still fit after the Nth iteration? Is the description sufficient to appear in the release notes?

[JaroslavTulach]

Why excluding Node.js modules? Is it common to have JavaScript in Java Maven or Gradle projects? Btw. there is https://bits.netbeans.org/dev/javadoc/org-netbeans-modules-queries/org/netbeans/api/queries/VisibilityQuery.html already implemented by NetBeans. There is also https://bits.netbeans.org/dev/javadoc/org-netbeans-modules-queries/org/netbeans/api/queries/SharabilityQuery.html ...

Ideally you want to scan only files that are tracked by version control system (e.g. SharabilityQuery) and/or visible (e.g. VisibilityQuery)

[Moataz-Aldawood]

Why excluding Node.js modules? Is it common to have JavaScript in Java Maven or Gradle projects?
Yes, it is actually incredibly common!
Many modern Java applications are built as "full-stack" applications within a single repository (a "monorepo" style).

Ideally you want to scan only files that are tracked by version control system (e.g. SharabilityQuery) and/or visible (e.g. VisibilityQuery)
Your idea is architecturally perfect for a pure NetBeans environment! But for a VS Code extension, relying on VS Code's findFiles is essentially doing the exact same thing (respecting git ignores and hidden files), just using VS Code's native, highly-optimized engine instead of the NetBeans Java engine.

why?

1. VS Code Already Does This Natively (and Faster)
   When we call vscode.workspace.findFiles('**/*.java') in our extension.ts, VS Code doesn't just blindly search the hard drive. It passes that request to its highly-optimized internal search engine (which is powered by Rust/ripgrep). By default, VS Code's findFiles automatically respects your .gitignore files (which maps to SharabilityQuery) and automatically respects your files.exclude settings (which maps to VisibilityQuery). It filters all of those out at the native OS level before the list of files ever reaches our extension.

2. Performance & Architecture
   If we wanted to use SharabilityQuery, we would have to move the file-discovery logic into the Java LSP backend. The backend would have to recursively crawl the disk in Java, check VisibilityQuery for every file, and then send a massive JSON array of thousands of file URIs back to the VS Code client over the LSP socket just so we could run our progress bar. By keeping file discovery on the client side, we leverage VS Code's lightning-fast native disk crawler and save the Java backend's memory exclusively for parsing code.

3. User Experience
   VS Code users expect extensions to respect their VS Code settings.json (like files.exclude). If we used NetBeans' internal visibility queries, a user might hide a folder in VS Code, but the NetBeans backend might still scan it because it wasn't hidden in the NetBeans project structure, leading to confusion.

[Moataz-Aldawood]

another option is to use the standard vscode files.exclude setting in .code-workspace or settings.json. but even this is not recommended. why? In VS Code, files.exclude strictly means "Hide this from the file explorer". If we tied the scanner to that setting, a user who simply wants to declutter their file tree (by hiding generated sources, for example) would accidentally break their Java diagnostics because the scanner would silently skip those hidden files.

So, my final recommendation is to create a dedicated exclusion setting as it already implemented.