Add HTTP proxy support for URL downloads

[rudimaulfurz]

Description

Add configurable HTTP forward-proxy support to WebDownloader.
NNTP/news-server connections stay completely unchanged and direct.

Some indexers do not allow NZB files to be downloaded and handed over to the download client. They only permit passing a URL, so the client has to follow the link (often via a local meta-search like NZBHydra2 that then redirects to the external indexer). The new proxy options make that flow work cleanly: local/Hydra URLs can be bypassed while the redirected external request goes through the HTTP proxy.

• Add UrlProxyHost, UrlProxyPort, UrlProxyUsername, UrlProxyPassword and UrlProxyBypass configuration options.
• Route HTTP URL downloads through the proxy using absolute-form request targets.
• Route HTTPS downloads through an HTTP CONNECT tunnel and start TLS against the original target host (correct SNI and certificate verification).
• Support HTTP Basic proxy authentication. Proxy-Authorization is only sent to the proxy, never forwarded to the origin after the tunnel is established.
• Re-evaluate proxy and bypass selection for every URL, including redirects.
• Bypass rules: exact hostnames, literal IP addresses, .domain suffixes, *.domain wildcards and IPv4 CIDR networks.
• Accept comma- or semicolon-separated bypass rules; surrounding whitespace is ignored.
• No local DNS resolution solely for bypass matching – external hostnames can still be resolved by the proxy.
• Validate the configured proxy port when a proxy host is set.
• Add configuration documentation and generated WebUI locale metadata for the new settings.

The proxy only applies to WebDownloader traffic (URL, RSS and NZB HTTP/HTTPS fetches). NNTP is not routed through it.

AI assistance

• This PR involved AI assistance (see AI Policy)

Tool: ChatGPT
Used for: design discussion, implementation help, test planning/automation, code review and commit preparation

Lib changes

None.

Testing

Testing during development / build and afterwards

• Linux build with existing test suite
• Windows test build
• Windows ctest
• Linux build with precompiled headers disabled
• cppcheck (upstream project configuration)
• HTTP WebDownloader request through the configured proxy
• HTTP request through a proxy with Basic authentication
• HTTPS request through an HTTP CONNECT tunnel
• HTTPS proxy authentication via CONNECT
• TLS SNI and certificate/hostname verification against the original destination (not the proxy)
• Proxy-Authorization is not forwarded to the tunneled origin
• Proxy-side DNS resolution (hostname that does not resolve locally)
• Downloaded response content check
• Proxy bypass for exact/literal IP target
• Bypassed target is contacted directly and does not appear in the proxy log
• Redirect from a bypassed/direct URL to a proxied external URL
• Proxy/bypass selection is re-evaluated after redirects
• Redirected HTTP request uses absolute-form proxy request URI
• Negative test with unreachable/invalid proxy port
• Direct vs. proxied public egress-IP check
• Real NZBHydra2 link/redirect flow through Privoxy to an external indexer
• NZB fetch through the external indexer succeeds
• NNTP/news-server traffic remains direct (nothing in the HTTP proxy log)
• Installation and startup of the final test build on QNAP TS-670 / QTS 4.3.6
• New proxy options are available in the WebUI/config template

QA Instructions / acceptance notes

• Configure UrlProxyHost and UrlProxyPort with a reachable HTTP forward proxy
• Leave UrlProxyHost empty → URL downloads still use the direct path
• Fetch a plain HTTP URL → request goes through the proxy
• Fetch an HTTPS URL → proxy receives a CONNECT host:port request
• HTTPS still validates the certificate against the original destination host
• TLS SNI contains the original destination hostname, not the proxy
• Configure UrlProxyUsername / UrlProxyPassword → HTTP Basic proxy authentication works
• Proxy-Authorization is only sent to the proxy, never to the HTTPS origin after the tunnel
• External hostname that is not resolvable locally still works (proxy-side DNS)
• Invalid or unreachable proxy port → URL download fails (no silent fallback to direct)
• Redirects continue to work through the proxy
• Proxy/bypass rules are re-evaluated after every redirect

For UrlProxyBypass

• Exact hostname bypass
• Exact literal IPv4 address bypass
• .example.com suffix rule
• *.example.com wildcard suffix rule
• IPv4 CIDR rule (e.g. 192.168.0.0/16)
• Comma-separated rules
• Semicolon-separated rules
• Surrounding spaces/tabs are ignored
• Bypassed target is contacted directly and does not appear in the proxy log
• Non-bypassed target still goes through the proxy
• Redirect from a bypassed local URL to an external URL switches from direct to proxied
• No IPv6 CIDR matching (IPv4 only)

NZBGet-specific acceptance

• RSS feeds load through the configured proxy
• NZB URL downloads load through the configured proxy
• NZBHydra2 local URL can be bypassed while the redirected external indexer URL goes through the proxy
• NZB is successfully imported after the redirect
• Start a Usenet download → NNTP connections do not appear in the HTTP proxy log
• NewsServer/NNTP traffic remains direct
• New settings are visible in the WebUI:
  • UrlProxyHost
  • UrlProxyPort
  • UrlProxyUsername
  • UrlProxyPassword
  • UrlProxyBypass
• Invalid proxy-port values are rejected by configuration validation
• Empty UrlProxyHost disables the feature cleanly

NZBHydra2 test scenario

NZBHydra2 must be set to send NZBGet a link (not the NZB contents) and use access type Redirect:

1. NZBGet requests the local Hydra /getnzb/... URL
2. Hydra answers with a redirect
3. NZBGet follows it to the external indexer

Hydra should advertise a reachable LAN URL (prefer IP over a local-only hostname), e.g. http://192.168.x.x:5076.
In NZBGet put that network into UrlProxyBypass, e.g. 192.168.0.0/16. The external indexer must not match the bypass list so the redirected request goes through the proxy.

Hydra’s own outbound proxy settings are independent. If Hydra itself uses a proxy, exclude local addresses there as well (proxyIgnoreLocal: true is typical).

Expected flow:

NZBGet
  → Hydra LAN URL          direct (UrlProxyBypass)
  → HTTP redirect
  → external indexer       via UrlProxyHost / UrlProxyPort

Proxy log should contain the external indexer request but not the local Hydra request.

Platform/build acceptance already covered

• Linux build and existing tests
• Windows test build
• Windows ctest
• Linux build without precompiled headers
• cppcheck
• QNAP TS-670 / QTS 4.3.6 installation and runtime smoke test

For final acceptance, the most important manual check is

On the final build, set a LAN/Hydra bypass rule and verify the local Hydra request is absent from the proxy log while the redirected external indexer request is present.