MCD Agent

Changelog: control-plane/agent/CHANGELOG.md

MCD (MauticControlDaemon) is a host-level service that can run in two modes:

• standalone (without MCC)
• connected (managed by MCC over SSH and event callbacks)

Current capabilities

• CLI entrypoint
• Config loader (TOML)
• Mautic instance discovery from web roots
• Root-owned Docker runtime descriptor discovery from /etc/mcd/instances.d; descriptor-backed console operations are routed through an exact scoped docker exec, host-side SQL uses the descriptor's private host endpoint, and filesystem repair preserves the numeric container owner; host-native instances retain the original path. Descriptor removal is reconciled on the next inventory rescan.
• Two independent instance dimensions are recorded and evaluated for every operation: runtime (host or docker) and installation layout (zip or composer). The four matrix cells have separate MCC presentation and share one capability-derived allowlist; Docker never implies Composer and Composer never implies Docker.
• Docker descriptors may expose a host bind-mounted plugin path. MCD then inventories, installs, removes and fixes ownership for plugins through that exact path and still runs post-install console work through the container. Missing descriptor capabilities fail closed instead of falling back to <host_root>/plugins or www-data.
• Cross-host Docker migration is delegated to a root-owned executable under /usr/local/libexec/mcd-runtime-adapters. A target is eligible only when Docker is running, the exact named adapter is installed and the source image reference is available locally. Adapter/image validation precedes every destructive target cleanup.
• Instance uid is domain-based (from active nginx/apache vhost), fallback to root-based short id
• Mautic versions supported now: 4, 5, 6, 7
• Image provisioning uses the standard HTTPS path: after the HTTP vhost is created, Certbot must obtain the domain certificate, install the HTTPS vhost and pass the nginx validation/reload checks before deployment is considered successful. Do not use --no-certbot for normal or test instance creation; it is reserved only for an explicitly documented recovery case.
• DB settings extraction from Mautic local.php:
  • db_host
  • db_table_prefix
  • db_port
  • db_name
  • db_user
  • db_password
• Optional manual instance definitions for non-standard/container paths
• Remote direct host-level backups via sshfs + mydumper with state semaphores (last_success, last_status, history)
• Full host restore (files archive + myloader DB restore) from selected backup date/path
• Encrypted backup profile vault in local MCD SQLite (credentials stored encrypted, not plain)
• Lightweight critical host signal snapshot (mcd-cli signals) for MCC cache:
  • OOM kills (kernel)
  • MySQL/MariaDB critical patterns
  • php-fpm max_children pressure
  • web 5xx spikes from service journal
  • scheduler drift (tracked running vs real processes)
  • stuck PHP console workers
  • swap pressure level
• DB-driven task polling and command execution:
  • segment updates by id (mautic:segments:update -i <id>)
  • campaigns update/trigger by id
  • import execution on pending import queue
  • optional periodic contacts cleanup ({prefix}leads rows with empty email+phone fields)
• Segment whitelist policy during active campaigns
• Runtime concurrency controls for campaigns and segments; limits are worker ceilings, while automatic scheduler dispatch claims at most one new queued task per pass to avoid burst-starting all free workers at once
• Round-robin segment scheduling so all eligible segments are processed over time
• Priority/regular circles for both segments and campaigns with dynamic weights
• Queue-based throttling using DB queue metrics (message_queue for Mautic 5+)
• Segment scheduler modes:
  • id_weighted (per-id weighted circles)
  • classic_loop (full mautic:segments:update each daemon cycle)
• Cron replacement workers:
  • use [[jobs]] in config for interval-based independent tasks
  • examples: mautic:email:fetch every 900 sec and mautic:broadcasts:send every 60 sec
• Mautic 5/6/7 message-queue worker:
  • MCC stores a per-instance enabled flag and interval for mautic:messages:send; default is disabled and 3600 seconds;
  • existing direct or wrapper cron is commented and migrated on the first supporting agent run, while an existing [[jobs]] entry is imported when no canonical MCC setting exists;
  • after migration, generic jobs cannot bypass the per-instance checkbox;
  • Mautic 4 is not changed by this worker.
• Catalog-driven plugin operations:
  • MCC sends only operations declared for bundles installed on each instance;
  • MCD generically schedules typed mautic_console operations from runtime.plugin_operation_instance_settings;
  • catalog rules comment/remove matching legacy cron, migrate its cadence and preserve all values from a previous tile/runtime schema;
  • plugin titles, bundle IDs, commands, defaults and cron tokens live in the MCC plugin catalog, not in MCD scheduler branches.
• Scheduler model:
  • single daemon loop
  • DB/config refresh on poll_interval_sec
  • dispatcher refill on dispatch_interval_sec (keeps target parallelism over time, but starts automatic ring work one task at a time)
  • dependent segment chains share one worker lane; unrelated chains may still occupy other segment workers
  • campaign-pressure segment throttling is threshold-based: queued or short-running campaigns do not throttle segments by themselves; pressure starts when campaign_pressure_min_running_sec or campaign_pressure_min_running_count is reached
  • two circles for segments and campaigns (priority + regular) with separate parallel limits
  • spawn-and-release: daemon starts command and does not wait for completion
  • process status is tracked asynchronously by PID monitor
  • mini SQLite state DB keeps running/finished/failed/timeout task history with bounded retention
  • weight cache stored in SQLite (weight_cache), recalculated by weights_recalc_interval_sec and on active-id set change
  • state DB tables:
    • tasks (task execution history / running rows)
    • weight_cache (segment/campaign computed weights)
    • instances (local Mautic inventory + DB connection metadata)
• Mautic instance discovery is not executed every tick
• instance list is loaded from local inventory (SQLite) and can be refreshed on demand
• inventory and MCC-safe state include the instance runtime, container id, runtime paths/user, ZIP/Composer layout, declared capabilities, migration adapter and immutable image reference
• MCC push model:
  • periodic push to MCC (/api/v1/agent/state) every 5 minutes by default
  • apt state is refreshed at mcc.push_apt_state_interval_sec (default 120 sec) and also refreshed immediately when local APT/DPKG state changes
  • extra push on state change
  • extra push on alert signal changes
  • mutating CLI operations push immediately (for example service-profile apply, env ipv6 enable|disable)
  • push includes host config_state snapshot (schema_version, customized, sha256, full TOML) so MCC stores exact observed behavior
• MCD self-update model:
  • MCC returns build plan (test|approved|lts) via authenticated API.
  • MCD performs update locally (download/stage/atomic source switch/restart) and reports result back to MCC.
  • apply path does not run pip install; host update is source-switch only.
  • MCC limits concurrent update sessions (10 by default); extra nodes receive wait/retry signal.
  • MCD auto-cleans old self-update artifacts (/opt/mcd/var/updates archives + /opt/mcd/var/backup/mcd-src-preupdate-*) by retention policy.
    • default: keep last 3 archives and 3 preupdate backups, max age 30 days, cleanup once per day.
  • MCD keeps local config history (10 snapshots by default).
• MCC-driven dynamic service profiles:
  • service profile payload is stored on MCC and can be changed without MCD release rebuild.
  • MCD pulls and auto-applies host-specific profile by hardware plan (php-fpm, mysql, apt components) on the normal daemon loop by default.
  • manual fetch/apply remains available through mcd-cli service-profile.
• Transitional shared agent-state backend for all installations:
  • optional state.backend = "mysql_hybrid" stores outbound events + latest state snapshot in MySQL/MariaDB,
  • agent uses dedicated state DB (state.mysql_database, default mcd_state) and auto-creates it if missing,
  • local SQLite remains as minimal fallback queue when shared DB is unavailable,
  • keeps current scheduler/task runtime stable while moving state to DB-backed mode.

Profiles

Set in config:

• [profile]
• name = "custom|tiny|mini|midi|maxi|hiload|ultra|farm-tiny|farm-mini|farm-midi|farm-maxi|farm-hiload|farm-ultra"

Selection authority:

• a fresh installation whose profile is still passive starts in auto mode;
• on every MCD service start, auto chooses the lower capacity class reported by logical CPU and physical RAM: 1/<4 GiB=tiny, 2/4 GiB=mini, 4/8 GiB=midi, 8/16 GiB=maxi, 16+/32+ GiB=hiload, and 24+/96+ GiB=ultra;
• RAM class boundaries allow for the normal firmware/kernel reservation visible in Linux MemTotal (for example, nominal 4 GiB is commonly about 3.7 GiB);
• mcd-cli profile <name> and an MCC profile change persist manual mode, and hardware detection no longer changes that host;
• mcd-cli profile auto is the explicit way to return to hardware-managed selection; mcd-cli profile status displays both authority and recommendation;
• an active profile that predates this state marker is preserved as manual during upgrade, preventing surprise changes on existing hosts.

Preset rules:

• tiny: single ring, no throttle, no whitelists, segments 1, periodic full segment scan every 60s; campaigns use one worker with actual trigger-due campaigns first and rebuild-due campaigns second, newest-first published list.
• mini: single ring, no throttle, no whitelists, segments 4, campaign trigger 2, campaign rebuild 1, shared campaign cap 1.
• midi: dual ring, no throttle, whitelists enabled, priority size 10, parallel 3+1 for segments, updates, triggers.
• maxi: dual ring, throttle 200/5m, whitelists enabled, segments 5+1, triggers 3+1, rebuilds 2+1; during throttle only whitelist segments run in 1 stream.
• hiload: dual ring, throttle 200/5m, whitelists enabled, segments 6+2, triggers 4+2, rebuilds 3+1; during throttle only whitelist segments run in 2 streams and non-whitelist running segments are killed and queued to resume first after throttle ends.
• ultra: high-capacity dual ring for hosts with at least 24 CPUs and 96 GiB RAM; hardware-derived limits remain authoritative.
• farm-tiny through farm-ultra: manually selected high-density hardware line for many small, separate Mautic databases. MCC selects the class from actual CPU/RAM, permits up to one scheduler command per CPU within the RAM budget, caps each instance separately, and keeps one host slot available for campaign or import work.
• custom: uses explicit [runtime] values.

All active profiles use one elastic host budget. Segment rebuilds may borrow idle capacity but leave one emergency slot when the host has at least two slots; campaign and import work may claim that slot immediately. The fairness watchdog promotes any instance whose queued work has waited for five minutes.

Segment stale-priority rule (all non-passive profiles):

• segments with last_built_date older than 24h (or missing) are force-added to priority ring;
• this rule is independent from normal weight threshold/top-N ranking;
• if regular ring is empty, its slot is reused by priority ring automatically until regular items appear.

Per-instance whitelist entries may define an opt-in realtime subset. These IDs retain normal whitelist membership while also receiving dedicated executor capacity that regular priority work cannot consume. Segment entries accept ids, interval_sec and parallel; campaign entries accept ids, separate rebuild_interval_sec/trigger_interval_sec, and separate rebuild_parallel/trigger_parallel. Exact-task and campaign-root locks remain shared with normal dispatch, so realtime capacity cannot duplicate a native command already running for the same entity.

SQL segment auto-promotion:

• when segment_sql_ring_enabled and segment_sql_auto_enabled are true, MCD can rebuild SQL-safe segments directly in DB and remove them from native Mautic segment rings;
• repeated recent native segment failures/timeouts still promote SQL-safe segments into the regular SQL ring;
• SQL-safe page-hit segments are promoted into a dedicated long SQL ring after a successful native mautic:segments:update -i <id> takes at least segment_sql_auto_long_native_min_duration_sec seconds within segment_sql_auto_long_native_history_sec, even when the regular due-segment query would not select them;
• the long SQL ring runs before the regular SQL ring and is limited by segment_sql_long_ring_max_per_tick;
• direct SQL rebuilds update lead_lists_leads, lead_lists.last_built_date, build time metadata, and Mautic's segment count cache so the Mautic UI sees the segment as rebuilt.

Split Config

Recommended layout:

• entrypoint: /opt/mcd/etc/mcd.toml (small, package-safe)
• package defaults: /opt/mcd/src/etc/mcd-agent.system.example.toml
• package defaults: /opt/mcd/src/etc/mcd-agent.operator.example.toml
• host overrides: /opt/mcd/etc/mcd.local.toml

Entrypoint file uses:

• [include].files = ["/opt/mcd/src/etc/mcd-agent.system.example.toml", "/opt/mcd/src/etc/mcd-agent.operator.example.toml", "/opt/mcd/etc/mcd.local.toml"]

Merge and precedence:

1. include files are merged in listed order
2. values from entrypoint file override includes
3. profile baseline is applied
4. manually set [runtime] values override profile baseline

Why this layout:

• package update can safely replace /opt/mcd/src defaults;
• host custom behavior stays in /opt/mcd/etc/mcd.local.toml and is not overwritten by code update.

Commands

Production CLI (recommended):

• mcd-cli (no args -> interactive menu)
• mcd-cli health
• mcd-cli discover
• mcd-cli run
• mcd-cli run-once
• mcd-cli segments:update -i 5
• mcd-cli campaigns:trigger -i 83
• mcd-cli import
• mcd-cli plugins
• mcd-cli mautic-upgrade (interactive)
• mcd-cli mautic-upgrade check
• mcd-cli mautic-upgrade apply --mode zip --backup --yes
• mcd-cli backup profile-show --json
• cat backup-profile.json | mcd-cli backup profile-set --profile-json-stdin
• mcd-cli backup profile-set --profile-json-file /root/backup-profile.json

Source/dev equivalent (same command surface):

• python -m mcd_agent <same args as mcd-cli>
• python3 -m pip install -r requirements.txt (only when running from source tree)
• python -m mcd_agent interactive --config ./etc/mcd-agent.example.toml
  • interactive menu uses one active instance for operational actions
  • use Select Active Instance to switch target without restarting CLI
  • includes Cache menu:
  • Soft Clear (cache:clear)
  • Warmup (cache:warmup)
  • Hard Clear (delete var/cache/prod)
• mcd-cli wrapper notes:
  • plugin alias is supported for plugins
  • help aliases supported: mcd-cli /?, mcd-cli instances /?, mcd-cli plugins /?

Manual command behavior:

• In active profiles, exec and shorthand commands are scheduler-aware:
  • request is queued into local state DB and picked by daemon on next dispatch cycle;
  • launch is immediate relative to dispatch tick and can temporarily exceed ring slot formula by one manual task;
  • scheduler then holds new auto launches until total active tasks return to configured profile limits.
• If daemon does not pick queued request quickly, CLI cancels queue row and falls back to direct one-shot execution.
• python -m mcd_agent instances --config ./etc/mcd-agent.example.toml list
• python -m mcd_agent env ipv6 status
• python -m mcd_agent env ipv6 disable
• python -m mcd_agent env ipv6 enable
• python -m mcd_agent env policy show
• python -m mcd_agent env policy plan --policy-file ./policy.json --component all
• python -m mcd_agent signals --window-min 15 --json
• python -m mcd_agent self-update --config ./etc/mcd-agent.example.toml status --json
• python -m mcd_agent self-update --config ./etc/mcd-agent.example.toml check --json
• python -m mcd_agent self-update --config ./etc/mcd-agent.example.toml apply --yes
• python -m mcd_agent service-profile --config ./etc/mcd-agent.example.toml status --json
• python -m mcd_agent service-profile --config ./etc/mcd-agent.example.toml fetch --component php_fpm --json
• python -m mcd_agent service-profile --config ./etc/mcd-agent.example.toml apply --component php_fpm
• python -m mcd_agent service-profile --config ./etc/mcd-agent.example.toml fetch --component mysql --json
• python -m mcd_agent service-profile --config ./etc/mcd-agent.example.toml apply --component mysql
• python -m mcd_agent service-profile --config ./etc/mcd-agent.example.toml fetch --component apt --json
• python -m mcd_agent service-profile --config ./etc/mcd-agent.example.toml apply --component apt --dry-run
• python -m mcd_agent service-profile --config ./etc/mcd-agent.example.toml rescan --component apt
• python -m mcd_agent zabbix --config ./etc/mcd-agent.example.toml status --json
• python -m mcd_agent zabbix --config ./etc/mcd-agent.example.toml bootstrap-mysql-user

Notes:

• php_fpm apply includes FPM pool/opcache/redis tuning. Global managed 98-mcd-php.ini baseline is no longer used; legacy files are removed on apply if present.
• APT profile includes one-time Zabbix DB monitor bootstrap (zbx_monitor@127.0.0.1) with marker tracking and manual override via mcd-cli zabbix bootstrap-mysql-user --force.
• Zabbix monitors host services and hardware only. It does not receive per-instance Mautic version cache files; MCD keeps its internal, non-migrated state cache under /opt/mcd/generated/.
• APT profile includes modular one-time repo profiles with local markers (/opt/mcd/var/apt-repo-profiles.json):
  • db_repo_profile (auto-detect: MariaDB/Percona/MySQL families),
  • ondrej_php_profile,
  • nginx_official_stable_profile (official stable nginx.org repo, disables Ondrej nginx source),
  • ondrej_nginx_profile (legacy; disabled when official nginx profile is enabled). Automatic checks stop after successful apply/verify for the same MCC profile hash and re-run when MCC changes the profile; use service-profile rescan --component apt for manual recheck/fix.
• APT profile can also manage unattended-upgrades policy dynamically:
  • unattended_upgrade_mode=off|security|all,
  • unattended_upgrade_schedule_cron (host local cron),
  • unattended_upgrade_blacklist (excluded package patterns).
• Additional runtime protection keys for scheduler/pressure handling:
  • scheduler_reconcile_interval_sec
  • php_console_stuck_sec
  • host_pressure_pause_enabled
  • host_pressure_php_stuck_pause_threshold
  • host_pressure_swap_level_pause_threshold
• mcd-cli signals now also reports:
  • scheduler_state_drift
  • scheduler_duplicate_task_keys
  • php_console_stuck
  • swap_pressure_level
• Cluster replica freshness checks:
  • runtime.cluster_replica_freshness_enabled = true enables application-table freshness checks on async replica nodes.
  • runtime.cluster_replica_freshness_max_age_sec is the default allowed age for the newest row in checked tables.
  • runtime.cluster_replica_freshness_checks is a JSON list of checks with database, table, column, optional order_column and optional max_age_sec. Prefer order_column = "id" for large append-only metric tables so the check uses the newest row by index instead of a full MAX() scan.
• Cluster Galera route/source eligibility is stricter than a TCP or systemd health check. A Galera node is marked eligible only when it reports wsrep_cluster_status=Primary, wsrep_local_state_comment=Synced, wsrep_connected=ON and wsrep_ready=ON. Donor, joiner and non-Primary states are reported as blocked even when MySQL is reachable.
• python -m mcd_agent runtime-overrides --config ./etc/mcd-agent.example.toml show
• python -m mcd_agent runtime-overrides --config ./etc/mcd-agent.example.toml fetch --json
• python -m mcd_agent runtime-overrides --config ./etc/mcd-agent.example.toml push --json
• python -m mcd_agent runtime-overrides --config ./etc/mcd-agent.example.toml trigger
• python -m mcd_agent state-db --config ./etc/mcd-agent.example.toml status --json
• printf 'ROOT_DB_PASSWORD' | python -m mcd_agent state-db --config ./etc/mcd-agent.example.toml init --admin-user root --admin-password-stdin --admin-unix-socket /var/run/mysqld/mysqld.sock --json
• python -m mcd_agent maintenance --config ./etc/mcd-agent.example.toml status
• python -m mcd_agent maintenance --config ./etc/mcd-agent.example.toml on --kill-orphans --grace-sec 10
• python -m mcd_agent maintenance --config ./etc/mcd-agent.example.toml off
• python -m mcd_agent instances --config ./etc/mcd-agent.example.toml rescan
• python -m mcd_agent instances --config ./etc/mcd-agent.example.toml add --name m1 --root /var/www/m1 --console-path /var/www/m1/bin/console
• python -m mcd_agent instances --config ./etc/mcd-agent.example.toml remove --name m1
• python -m mcd_agent reload-config --config ./etc/mcd-agent.example.toml
• python -m mcd_agent time-check --config ./etc/mcd-agent.example.toml
• python -m mcd_agent profile --config ./etc/mcd-agent.example.toml status
• python -m mcd_agent profile --config ./etc/mcd-agent.example.toml tiny --yes
• python -m mcd_agent profile --config ./etc/mcd-agent.example.toml passive --yes
• python -m mcd_agent uninstall --yes
• python -m mcd_agent backup --config ./etc/mcd-agent.example.toml run
• python -m mcd_agent backup --config ./etc/mcd-agent.example.toml status --json
• python -m mcd_agent backup --config ./etc/mcd-agent.example.toml history --json
• python -m mcd_agent backup --config ./etc/mcd-agent.example.toml prune
• python -m mcd_agent backup --config ./etc/mcd-agent.example.toml restore --date 2026-03-01
• python -m mcd_agent backup --config ./etc/mcd-agent.example.toml profile-show
• cat backup-profile.json | python -m mcd_agent backup --config ./etc/mcd-agent.example.toml profile-set --profile-json-stdin
• python -m mcd_agent backup --config ./etc/mcd-agent.example.toml profile-set --profile-json-file ./backup-profile.json

Same operations via wrapper (mcd-cli):

• mcd-cli runtime-overrides show
• mcd-cli runtime-overrides fetch --json
• mcd-cli runtime-overrides push --json
• mcd-cli runtime-overrides trigger
• mcd-cli state-db status --json
• printf 'ROOT_DB_PASSWORD' | mcd-cli state-db init --admin-user root --admin-password-stdin --admin-unix-socket /var/run/mysqld/mysqld.sock --json
• mcd-cli maintenance status
• mcd-cli maintenance on --kill-orphans --grace-sec 10
• mcd-cli maintenance off
• mcd-cli instances rescan
• mcd-cli instances add --name m1 --root /var/www/m1 --console-path /var/www/m1/bin/console
• mcd-cli instances remove --name m1
• mcd-cli reload-config
• mcd-cli time-check
• mcd-cli profile status
• mcd-cli profile auto
• mcd-cli profile tiny --yes
• mcd-cli profile passive --yes
• mcd-cli uninstall --yes
• mcd-cli backup run
• mcd-cli backup status --json
• mcd-cli backup history --json
• mcd-cli backup prune
• mcd-cli backup restore --date 2026-03-01
• mcd-cli backup profile-show
• cat backup-profile.json | mcd-cli backup profile-set --profile-json-stdin
• mcd-cli backup profile-set --profile-json-file /root/backup-profile.json

Profile Model

• State is profile-based ([profile].name).
• profile=passive:
  • MCD runs in planning/statistics mode only (no Mautic task dispatch),
  • cron is expected to remain active.
• Non-passive profiles (tiny|mini|midi|maxi|hiload|ultra|farm-tiny|farm-mini|farm-midi|farm-maxi|farm-hiload|farm-ultra|custom) dispatch Mautic tasks.
• Automatic selection is persisted separately in /opt/mcd/var/profile-selection.json; it never overrides a manual selection.
• mcd-cli profile auto enables hardware-managed selection and immediately applies the current CPU/RAM recommendation.
• mcd-cli profile passive:
  • switches profile to passive,
  • restores cron from pre-active backups (or from MCD markers if backup missing),
  • restarts mcd.
• mcd-cli profile <tiny|mini|midi|maxi|hiload|ultra|farm-tiny|farm-mini|farm-midi|farm-maxi|farm-hiload|farm-ultra|custom>:
  • applies selected non-passive profile,
  • comments managed cron lines (segments:update, campaigns:update, campaigns:trigger, campaigns:rebuild, import),
  • always leaves mautic:emails:send under cron ownership; MCD does not migrate or schedule file-spool delivery,
  • restarts mcd.
• mcd-cli maintenance on|off|status:
  • temporary maintenance mode without profile change,
  • on pauses scheduler launches and can stop running Mautic console tasks,
  • off resumes scheduler launches only.

Cron replacement focus (phase now)

Replaced by daemon logic:

• mautic:segments:update (DB-selected segments only, with -i id)
• mautic:campaigns:rebuild and mautic:campaigns:trigger (only active/published campaigns from DB)
• mautic:import (runs when pending jobs appear)

Campaign command note:

• In MCD scheduler, mautic:campaigns:update is treated as a synonym of mautic:campaigns:rebuild.
• Only one pre-trigger campaign pass is scheduled (campaigns:rebuild) to avoid duplicate work.

Still left to cron for now:

• cache clear/warm
• maintenance cleanup
• SQL cleanup and other housekeeping tasks

Important:

• SQL selectors are configurable in [sql] for different Mautic schemas.
• Time in SQL should use daemon-provided UTC placeholders:
  • {now_utc} for point-in-time checks
  • {window_start_utc_24h} for 24h windows
  • this avoids dependency on MySQL/PHP server timezone settings.
• Console command templates are configurable in [commands] for different Mautic CLI variants.
• Config path detection:
  • Mautic 4: app/config/local.php
  • Mautic 5/6/7: config/local.php
• Mautic timezone:
  • parsed from local.php (default_timezone/timezone) and stored in instance inventory
  • used for quiet-window jobs (contacts cleanup) so daemon behavior follows instance timezone.
• Per-instance PHP runtime:
  • Mautic and PHP CLI configuration remain the source of truth for timezone and limits; MCD does not override PHP values at command launch.
  • mcd-cli instance-runtime status --json reports which nginx vhost files can be materialized.
  • mcd-cli instance-runtime apply generates per-instance PHP-FPM pools in /opt/mcd/generated/php/<version>/fpm/pools/.
  • MCD connects those pools through one FPM include file /etc/php/<version>/fpm/pool.d/99-mcd.conf containing include=/etc/php/<version>/fpm/pool.d/mcd/*.conf.
  • /etc/php/<version>/fpm/pool.d/mcd is a symlink to the generated pools directory; individual pool files are not scattered in /etc.
  • Matching nginx vhosts are rewritten from the shared socket (/run/php/php<version>-fpm.sock) to the instance socket (/run/php/php<version>-fpm-mcd-<slug>.sock).
  • MCD runs every bin/console command through the configured host PHP binary (normally /usr/bin/php). It never creates {instance}/.mcd, a PHP wrapper, or CLI -d overrides.
  • Global CLI conf.d/*.ini files are host-wide and are the only source for PHP CLI limits and timezone.
  • Apply validates php-fpm<version> -t and nginx -t before reload and restores snapshots on validation failure.
• Runtime execution user:
  • runtime.mautic_run_as_user (default www-data) is used for Mautic console commands.
• Filesystem permissions watchdog:
  • runtime.fs_permissions_guard_enabled enables periodic owner/mode guard for critical Mautic paths.
  • runtime.fs_permissions_guard_interval_sec controls per-instance check interval.
  • runtime.fs_permissions_guard_paths defines relative instance paths to enforce (var/cache, var/logs, var/spool, var/tmp, media/config paths).
  • runtime.fs_permissions_guard_fix_console_exec forces bin/console executable bit (chmod ug+x) and runtime owner.
  • runtime.fs_permissions_guard_console_relpath allows custom console location (default bin/console).
  • runtime.db_watchdog adds DB processlist watchdog policy (observe-first):
    • enabled, interval_sec, observe_only, processlist_limit, sample_limit
    • global_rules for shared defaults
    • host_rules for host-specific overrides (host patch overrides global rules by rule id)
    • telemetry is pushed to MCC in signals.totals and signals.details.db_watchdog_recent (no kill action while observe_only=true)
  • guard runs in all profiles, including passive (planning-only mode still keeps filesystem ownership healthy).
• Runtime tuning for large campaigns:
  • runtime.campaign_limit controls per-run trigger batch size.
  • runtime.campaign_limit = 0 (or off / unlimited via MCC runtime override) omits --campaign-limit, so one trigger run can process the whole campaign.
  • runtime.campaign_pressure_min_running_sec defaults to 120; a single running campaign must live at least this long before segment throttling is treated as campaign pressure.
  • runtime.campaign_pressure_min_running_count defaults to 2; this many simultaneous campaign workers trigger campaign pressure immediately. Set to 0 to disable the count rule.
  • Active MCD profiles run the campaign safety audit every 60 seconds. It evaluates published campaigns against Mautic's due-work conditions and runs mautic:campaigns:trigger -i ID only for eligible IDs; passive profiles keep external cron ownership.
  • When audit or due-work planning places a campaign in the priority ring but its priority worker count is zero, spill capacity dispatches that priority candidate before continuing an always-populated regular ring. This bounds detection-to-rebuild admission by the next scheduler pass while retaining host, instance, campaign and fairness limits.
  • on weak hosts start lower (e.g. 1000) so one long campaign does not block full daemon cycle for too long.
• Runtime tuning for catalog plugin operations:
  • canonical per-instance values live under runtime.plugin_operation_instance_settings;
  • scheduled operations default to disabled when neither old state nor a matching legacy cron exists;
  • explicit old enabled/disabled state and every other mapped field are canonicalized before cron migration; a cron enables the operation only when no older state exists.
• Self-update safety:
  • runtime.mcd_update_defer_during_campaigns = true prevents MCD self-update while campaign trigger/rebuild/update console jobs are running.
  • daemon auto-update keeps a short cooldown after campaign console activity to avoid restarting between batch passes.
• Retry and watchdog:
  • runtime.task_retry_max, runtime.task_retry_delay_sec control retries for concrete command execution.
  • runtime.task_retry_max semantics:
    • 1 = no retry (only initial attempt),
    • >1 = bounded retries (attempt cap),
    • 0 or negative = unlimited immediate retries (with task_retry_delay_sec pause).
  • global default: runtime.command_timeout_sec = 0 and runtime.worker_watchdog_sec = 0 (long-running tasks are not killed by timeout).
  • runtime.worker_stuck_policy = skip|restart and runtime.worker_stuck_restart_limit control reaction on stuck processes.
  • runtime.state_db_path sets SQLite process-state storage path (default /opt/mcd/var/mcd-state.db).
  • optional [state] section enables shared state backend:
    • backend = "sqlite|mysql_hybrid"
    • mysql_host/mysql_port/mysql_database/mysql_user/mysql_password
    • mysql_unix_socket (optional explicit socket path for local auth)
    • mysql_table_prefix, mysql_*_timeout_sec
    • mysql_snapshot_enabled
  • runtime hot-apply keys from MCC include state_backend and state_mysql_* (including state_mysql_unix_socket).
  • when host is local and password is empty, agent auto-detects common MySQL unix sockets for local auth.
  • in mysql_hybrid mode, agent attempts to create state DB automatically; on failure it keeps legacy SQLite behavior and reports init error to MCC (state_backend payload).
  • in mysql_hybrid mode, task/state runtime tables (tasks, manual_requests, weight_cache, runtime_sync) are primary in MySQL.
  • local SQLite stays as failover-only shadow (running/pending minimum) and is pruned after successful migration.
  • first successful MySQL bootstrap performs one-time SQLite -> MySQL migration for these runtime tables.
  • manual DB bootstrap is available via mcd-cli state-db init and is allowed for legacy mode when DB is missing or inaccessible.
  • bootstrap uses temporary admin credentials only for init, creates dedicated mcd_state runtime DB user, and persists only runtime credentials.
  • runtime.tasks_history_keep_days sets retention depth for non-running task rows in the live operational slice (default: 2 days).
  • runtime.tasks_history_max_rows sets hard cap for historical non-running rows in the live operational slice (default: 25000 rows).
  • runtime.tasks_archive_enabled, runtime.tasks_archive_dir, and runtime.tasks_archive_keep_days control compressed JSONL postmortem archive of task rows removed from live state (default: 14 days). Scheduling logic must not depend on archived rows.
  • runtime-sync snapshots are kept in backend runtime table runtime_sync:
  • local_runtime (runtime section from local config)
  • mcc_runtime (desired runtime payload fetched from MCC)
  • mcc_runtime_desired_state (revision, provenance, instance revisions, and last confirmed apply state)
  • active_runtime (last runtime apply metadata)

Runtime Sync (MCC <-> MCD)

• Canonical desired state:

  • MCC stores revisioned host state plus per-instance state in desired_runtime_states.
  • Instance records use immutable instance_uid, not a root path or the current host, and therefore follow an installation after a migration.
  • Legacy host JSON remains a compatibility mirror. MCC promotes legacy root/domain keyed instance maps to UID records on the next agent/API contact.
  • Each state row records source, origin host and agent version, modification time, revision, last applied revision, actual apply status, error, and the host that confirmed application.
  • MCD writes desired changes using optimistic revision matching. A disconnected or reinstalled agent with no matching base revision receives MCC state instead of overwriting newer confirmed state.
  • Static agent configuration continues to use MCC's existing desired config snapshot and startup recovery path; dynamic settings use the revisioned live protocol above. Together they let a reinstall recover its config at startup and apply post-start changes without a restart.

• Source of truth split:

  • desired runtime overrides are revisioned in MCC and mirrored to the legacy host table (runtime_overrides_json) for compatibility.
  • observed runtime overrides are pushed by MCD and stored separately in MCC (observed_runtime_overrides_json).

• Template runtime keys:

  • runtime.host_template=true marks host as template source.
  • runtime.template_autopromote_on_clone=true enables clone autopromote to new host identity in MCC when local hostname differs from configured [mcc].host_name.

• MCC -> MCD:

  • normal path: daemon polls MCC runtime endpoint.
  • immediate path: mcc_cli host-runtime set/unset triggers mcd-cli runtime-overrides trigger, daemon consumes trigger and pulls immediately.
  • a successful live apply is acknowledged back to MCC; no daemon restart is required.

• MCD -> MCC:

  • MCD state push contains runtime_overrides and extracted instance-scoped values keyed by instance_uid.
  • daemon also watches local mutable runtime section fingerprint and pushes to MCC immediately when it changes.
  • any mutating command that already does immediate state push updates observed runtime view in MCC without separate polling.
  • runtime.tasks_compact_* controls quiet-window compaction cadence (DELETE + optional VACUUM).
  • systemd service uses KillMode=control-group and TimeoutStopSec=15 so a restart cannot leave child Mautic commands orphaned while preserving the normal per-task process-group timeout behavior.

• Plugin interactive sync:

• MCD reads manifest.json from MCC plugin repo

  • shows status table (OK, UPDATE, MISSING, BROKEN) plus local-only rows (-)
  • table columns: installed version (from plugins/<Bundle>/Config/config.php) and server version (from manifest)
  • applies selected plugin operations, then runs cache:clear and mautic:plugin:install
  • after install/replace sets ownership to www-data:www-data for the updated bundle directory

• Backup module:

  • section [backup] in system config
  • host-level direct write to remote share via sshfs (no local dump staging)
  • opt-in instance backup transport [backup.storage].kind = "local" writes directly to an already-mounted absolute local_path; MCD does not manage mounts, encryption, credentials or storage lifecycle
  • local targets are root-owned, non-symlink, non-world-writable active mountpoints by default; unsafe broad paths, traversal and instance/target overlap fail closed
  • instance restore uses the generation marker to restore persistent files to the exact managed instance root and Myloader data to the exact tenant DB; missing targets or artifacts fail instead of returning a zero-work success
  • one run includes all discovered instance databases (with DB creds) + optional system files archive
  • remote layout: /<remote_root_dir>/<host_name>/<YYYY-MM-DD>/...
  • startup hygiene: stale /.incomplete-* directories from failed/aborted runs are cleaned automatically before a new backup starts
  • on success writes .mcd-backup.json marker in backup folder
  • local state semaphore per host in /opt/mcd/var/state/backup/host-<host>.json
  • state includes last_run_at, last_success_at, last_status, last_error, last_backup_path, and recent history
  • restore command supports:
    • latest backup auto-select (default)
    • restore by explicit --date YYYY-MM-DD
    • restore by explicit backup --path
  • backup profile credentials can be set without shell-history exposure:
    • backup profile-set --profile-json-stdin
    • backup profile-set --profile-json-file
  • where credentials/settings are stored:
    • authoritative runtime backup profile is in local MCD state DB (state_db_path) table backup_profile as encrypted payload (payload_enc);
    • this is why scheduler/backup can run even if some backup.* keys are absent in text config;
    • explicit stable backup sections are synchronized with mutable config (/opt/mcd/etc/mcd.toml):
      • DB -> config on backup profile-set / MCC-applied backup profile changes;
      • config -> DB by daemon periodic sync (for manual operator edits in config file).
  • secret refs/config keys:
    • [backup.secrets].key_path
    • [backup.storage].password_ref
    • [backup.mysql].password_ref
  • scheduler support in daemon ([backup.schedule]):
    • quiet-window execution
    • interval-based cadence
    • runs independently from Mautic task rings
    • global backup guard:
      • while backup lock is active, no new Mautic tasks are started (segments, campaigns, import, scheduled jobs)
      • pre-backup window (backup.schedule.pre_pause_sec, default 3600s) also blocks new task launches
      • already running tasks continue until completion; they are not killed by backup guard
      • dispatch resumes automatically when backup run finishes (success or failure)
  • default dump safety profile ([backup.mydumper]):
    • threads: 6
    • kill_long_queries=false
    • long_query_guard=0
    • process priority lowering enabled by default (ionice + nice)
    • transaction/lock flags are auto-selected:
      • --sync-thread-lock-mode=AUTO when supported by local mydumper
      • prefer --trx-tables, fallback to --trx-consistency-only on older versions

• Mautic upgrade:

  • mautic-upgrade check detects current version and suggests next target in chain
  • mautic-upgrade apply supports zip|composer|auto
  • optional --backup creates archive backup before upgrade
  • optional --with-system-upgrade runs php/nginx package-level migration steps
  • guarded Composer Mautic 6 to 7 upgrades require an explicit target and --allow-major; MCC runs a Storage Box instance backup first and starts the upgrade only after that backup succeeds
  • the 6 to 7 path is unavailable unless the active database is independently reported as MySQL 8.4+ or MariaDB 10.11+; MCD does not install or upgrade a database because a host-level database change can affect other instances
  • the Mautic 6 to 7 Composer path prepares Composer and Node 20, updates composer.json to the selected Mautic 7 target, runs Composer with dependencies, clears cache, finishes the Mautic updater, and reconciles known Doctrine migration metadata drift before the final migration check
  • --with-system-upgrade on the 6 to 7 flow installs PHP 8.4 packages, migrates custom PHP ini files such as 60-custom.ini and 90-redis-sessions.ini, rewrites nginx PHP-FPM socket references from 8.3 to 8.4, validates nginx, restarts services, and purges PHP 8.3 only after all other discovered host instances are Mautic 7-compatible
  • ZIP-to-Composer migration rewrites active root/www-data cron paths to the Composer root, disables the removed mautic:emails:send command on Mautic 5+, and fails with rollback if an active source-root cron entry remains
  • MCD repairs the Mautic 7 GrapesJS CKEditor GPL-license fallback on every inventory cycle, including passive hosts, and reapplies it after plugin or Mautic updates overwrite either the source or published builder artifact; the scoped builder URL revision also invalidates previously cached broken JS

• MCD self-version checks:

  • runtime.mcd_update_notify = true (default): show notice if MCC has newer MCD version
  • runtime.mcd_auto_update_enabled = false (default): auto-update disabled; notify-only
  • runtime.mcd_update_check_interval_sec: check interval
  • mcc.mcd_manifest_url (optional): explicit MCC manifest URL

Central Policy (Plan-Only in 0.4.0)

• MCD exposes host policy planning commands for centralized operations managed by MCC.
• Covered domains:
  • apt
  • iptables
  • database (MariaDB/MySQL)
  • php (php-fpm)
  • web (nginx/apache)
  • web.cloudflare_real_ip (nginx Cloudflare real-IP template task)
• Commands:
  • mcd-cli env policy show
  • mcd-cli env policy plan --policy-file <file>
  • mcd-cli env policy plan --policy-json '<json>' --component php
  • mcd-cli env policy plan --policy-json '<json>' --component web_cf_real_ip
• Safety:
  • this release does not apply policy changes on hosts;
  • output is execution plan only.

Validated runtime profile (current host, 2026-02-17)

• runtime.segment_mode = "classic_loop" (single full mautic:segments:update loop through MCD)
• Campaign workers enabled:
  • campaign_priority_parallel = 1
  • campaign_regular_parallel = 1
  • campaign_latest_priority_count = 2 (latest published campaigns are always in priority circle)
• Import worker enabled only on real pending imports:
  • enable_import_polling = true
  • import_poll_interval_sec = 30
• Cron-like minute tasks via independent [[jobs]] workers:
  • mautic:broadcasts:send (interval_sec = 60)
  • mautic:email:fetch (interval_sec = 900)
• Mautic 5/6/7 message queue via dedicated MCC/MCD settings:
  • message_queue_instance_settings[<instance>].enabled = false by default
  • message_queue_instance_settings[<instance>].interval_sec = 3600
• Daily cleanup via built-in contacts cleanup window:
  • enable_contacts_cleanup = true
  • contacts_cleanup_interval_sec = 86400
  • contacts_cleanup_quiet_hour = 2
  • contacts_cleanup_quiet_window_min = 60
• MCC-managed Clean Empty Contacts can run per instance inside a nightly window:
  • empty_leads_cleanup_instance_settings[<instance>].schedule_type = "nightly_window"
  • window_start = "22:00", window_end = "09:00"
  • batch_size = 50000
  • max_runs_per_window = 0 for no limit while the window is open
• MCC-managed Monitored Email Parser can replace mautic:email:fetch per instance:
  • monitored_email_parser_interval_sec = 900
  • monitored_email_parser_batch_size = 100 (agent caps at 5000)
  • monitored_email_parser_types = ["feedback_loop", "bounce", "unsubscribe"]
  • monitored_email_parser_delete_processed = true deletes mailbox messages only after a matching contact is found and email DNC is present or inserted
  • monitored_email_parser_whitelist = ["support@example.com"] skips exact internal emails and removes existing email-DNC rows for those contacts
• MCC-managed own-host mail uses bounce@<instance> as its return path and accepts only bounce@, fbl@ and abuse@ on the instance MX. Delivery reports and ARF complaints are parsed immediately into email DNC state for that instance; no IMAP mailbox credentials are required. External SMTP, Amazon SES and SendGrid profiles keep their Mautic return path unchanged, including an intentionally empty value.
• While at least one own-host domain is active, MCD maintains one exact, comment-tagged TCP/25 firewall rule through mcd-local-mail-firewall.service. Disabling the final own-host domain removes only that managed rule.

Benchmark notes (current host)

• Baseline standard command (scheduler paused):
  sudo -u www-data php /var/www/mautic/bin/console mautic:segments:update --batch-limit=1000
  • measured on 2026-02-17: elapsed=18.48s
• Previous measurement before latest fixes on same host: elapsed=22.84s
• MCD scheduler now launches independent task types in parallel (confirmed in one cycle): segment, campaign_update, campaign_trigger, job:broadcasts-send, job:messages-send.