[Mosquitto] Add new integration for Eclipse Mosquitto broker logs

[smnschndr]

Proposed commit message

Adds a new mosquitto integration that collects the log of an Eclipse Mosquitto MQTT broker through the filestream input. One data stream, log. No dashboards in this initial release.

WHY: Mosquitto is a widely deployed open source MQTT broker, and its log is the only place where client connections, subscription grants, published topics and broker errors are visible. There is currently no Elastic integration for it.

WHAT — the timestamp handling is the substantive part of this PR.

Mosquitto prefixes each log line with a timestamp in one of two mutually exclusive formats, and which one you get depends on a single broker setting:

• log_timestamp_format unset (the default) writes Unix epoch seconds, always UTC.
• log_timestamp_format %Y-%m-%dT%H:%M:%S writes a strftime timestamp. This is what the Mosquitto documentation shows as its example, so it is common in the wild.

Both are handled by a single pipeline path rather than two branches: one grok splits the prefix from the body, and a small set of date processors resolves whichever variant matched. log_timestamp false is also handled — those documents are tagged mosquitto_missing_timestamp and keep the collection time rather than being dropped or erroring.

Two details worth flagging for review, both verified against the Mosquitto source rather than assumed:

• The formatted variant is rendered with localtime() and carries no UTC offset (src/logging.c get_time), while the epoch default is always UTC. A tz_offset policy option is therefore wired through to the date processor for the formatted case only.
• If an operator adds %z, glibc renders the offset without a colon (+0200). Elasticsearch's ISO8601 format accepts that form; there is a permanent test fixture line for both +0200 and +02:00.

Message bodies are parsed into ECS plus a mosquitto.log.* namespace, covering MQTT control packets (topic, QoS, message ID, payload size, dup/retain flags), connection lifecycle, subscription grants and denials, broker lifecycle and errors. The grok patterns were derived from the log__printf format strings in the Mosquitto source, not only from sample data, so they also cover the 1.6-era disconnect wording that omits the client address. Unrecognised lines are still indexed with their full text in message.

Mosquitto's file and stdout destinations write no severity marker at all, so log.level is derived from the wording of each message. This is documented in the README so the heuristic is not mistaken for a vendor-supplied field.

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.
• I have verified that any added dashboard complies with Kibana's Dashboard good practices — n/a, no dashboards in this release.

Author's Checklist

Open items the reviewer should weigh in on:

• The package icon is a placeholder. img/mosquitto-logo.svg is a neutral hand-drawn mark. The Eclipse Mosquitto logo was deliberately not used, to avoid shipping a trademarked asset without clearance. Guidance on sourcing the correct asset would be welcome.
• owner is set to elastic/integrations-triaging so the triage team can route this, rather than pre-empting the assignment. owner.type is community.
• vale was not run. It is not installed in the author's environment, so the docs were checked manually against the Elastic style rules instead. Worth a fresh pass in CI.
• Known limitation, documented in the README: the log_type unsubscribe line is a bare client id and topic separated by a space. That shape cannot be distinguished from ordinary two-word log messages, so it is indexed without a parsed topic. The accompanying UNSUBSCRIBE packet line is parsed normally.

How to test this PR locally

cd packages/mosquitto
elastic-package check
elastic-package test pipeline --data-streams log   # 6 tests
elastic-package test system --data-streams log     # 2 configs: ISO and epoch

The system test runs twice against the same broker log rendered in both timestamp formats, which is the clearest demonstration that the two paths converge.

Beyond the committed fixtures, the pipeline was run against a real 2591-line broker log captured from a Mosquitto 2.0.22 instance: zero pipeline errors, and only 9 lines (0.3%) produced no structured fields — all of them No will message specified., which carries nothing to extract. That log is not committed, as it is not synthetic.

To confirm the two timestamp variants really do converge, the generated expected outputs can be compared directly. Over the 39 log lines the two fixtures share, the resulting documents are identical except for event.original — same @timestamp, same parsed fields. The ISO fixture carries two extra lines beyond those 39, covering the %z offset forms that have no epoch equivalent, so the comparison is bounded to the common prefix:

cd packages/mosquitto/data_stream/log/_dev/test/pipeline
diff <(jq -S '[.expected[] | del(.event.original)][:39]' test-mosquitto-log-iso-sample.log-expected.json) \
     <(jq -S '[.expected[] | del(.event.original)][:39]' test-mosquitto-log-epoch-sample.log-expected.json)

Related issues

[elastic-vault-github-plugin-prod]

Reviewers

Buildkite won't run for external contributors automatically; you need to add a comment:

• /test : will kick off a build in Buildkite.

NOTE: https://github.com/elastic/integrations/blob/main/.buildkite/pull-requests.json contains all those details.

[smnschndr]

/test