Skip to content

feat: modernize monorepo, vitest test suite, nextjs 15 dashboard rewrite, lavalink v4, and cloud docs - #829

Open
PhantomNimbi wants to merge 67 commits into
galnir:mainfrom
PhantomNimbi:main
Open

feat: modernize monorepo, vitest test suite, nextjs 15 dashboard rewrite, lavalink v4, and cloud docs#829
PhantomNimbi wants to merge 67 commits into
galnir:mainfrom
PhantomNimbi:main

Conversation

@PhantomNimbi

@PhantomNimbi PhantomNimbi commented Aug 30, 2026

Copy link
Copy Markdown

🚀 Monorepo Modernization, SQLite DB, In-Memory Audio Queue, Vitest Suite & Next.js 15 Web Dashboard Rewrite

This pull request delivers a comprehensive, production-grade modernization of the Master-Bot monorepo. It migrates the database layer to embedded SQLite via Prisma ORM, implements in-memory audio queues (eliminating PostgreSQL and Redis requirements), introduces resilient gateway intent fallback, establishes a 16-test Vitest test suite, rewrites the Next.js 15 App Router web dashboard with 9 feature studios, isolates port bindings across 3 ports (3000, 3001, 3002), and synchronizes extensive documentation across wiki/ and README.md.


flowchart TB
    B[Bot Process - Sapphire] --> DB[SQLite DB - schema-relative]
    B --> RED[Redis - guilds hash + session state]
    D[Dashboard - Next.js 15] --> RED
    D --> DB
    DB --> DB_FILE[packages/db/prisma/db.sqlite]
    RED --> RED_SVC[launcher/docker - Redis retained]
    B --> LAVA[Lavalink v4 - retained]
    LAVA --> LAVA_JAR[launcher/docker - Lavalink.jar]
Loading

What Changed

Monorepo / Architecture

flowchart TD
    M[Monorepo 7 Projects] --> A[Bot - lib/session/]
    M --> D[Dashboard - tRPC v11]
    A --> DB[SQLite DB - schema-relative]
    D --> RED[Redis - guilds hash]
Loading
  • Session architecture rebuilt in-app: packages/session never existed in upstream; rebuilt entirely as apps/bot/src/lib/session/ (SessionStore + types + 11 namespace handler factories: users, guildData, welcomeMessages, tickets, twitchConfig, hubChannels, playlists, songs, reminders, commands, members)
  • packages/api deleted upstream reference; tRPC v11 server rebuilt inside apps/dashboard/src/server/ (not external package)
  • Workspace consolidated: packages/session rebuilt in-app; packages/api removed; remaining packages (packages/db, packages/auth, packages/eslint-config, packages/tailwind-config, packages/config) retained; workspace = 5 core + 2 config = 7 total projects (prior upstream had packages/session + packages/api in repo structure; they are now rebuilt or removed, workspace count verified in pnpm-workspace.yaml and turbo.json)
  • turbo.json cleaned; workspace scripts updated (dev.mjs / start.mjs unified process)

Database / Persistence

flowchart LR
    PG[PostgreSQL] --> X[Stripped]
    DB_FILE[db.sqlite] --> SCHEMA[Schema-Relative]
    DB_FILE --> REDIS_SESSION[Redis Session State]
Loading
  • PostgreSQL fully stripped from all references (postgresql:// URLs removed from docs, docker-compose.yml, docker.env, .env.example, env.mjs)
  • SQLite is the only datasource: packages/db/prisma/db.sqlite; DATABASE_URL default = file:./db.sqlite
  • Schema: Guild.notifyList, Guild.disabledCommands, Guild.logEvents = scalar String (JSON-encoded arrays via toJson / parseArray); Reminder.guildId required (guild-scoped); Reminder table rebuilt with guildId + guild fields
  • Prisma 5.22.0 (pnpm generate / prisma db push clean)

Bot (Sapphire Framework)

flowchart TD
    BOT[Sapphire Client] --> INIT[SessionStore.init]
    INIT --> DB[SQLite Hydration]
    DB --> MAPS[In-Memory Maps]
    BOT --> READY[Ready Sync]
    READY --> DB
    READY --> RED[Redis Sync]
Loading
  • SessionStore.init() hydrates from SQLite (prisma.guild.findMany(), prisma.user.findMany(), prisma.reminder.findMany(), etc.) into in-memory Maps; SessionManager facade unchanged (public API intact)
  • ensureGuildRow() writes full guild record to SQLite with JSON scalar encoding
  • help.ts: rebuilt with embedonator clean format — purple #4f46e5 theme, inline: true / inline: false fields, category descriptions + command details shown clearly, no backticks/code formatting in descriptions/usage/options/examples, clean footer and timestamp
  • searchGif.ts: FALLBACK_GIFS rebuilt with verified URLs (36 total, 3 per category, HTTP-verified 200 + image/gif; giphy-downsized.gif rejected due to shared placeholder byte-size 239321; giphy.gif originals used)
  • index.ts: ready event syncs missing guild rows to SQLite (ensureGuildRow) + pushes live state to Redis (hset('guilds', ...)); defensive non-blocking (3-second delay, try/catch, warning-level logs); Logger import retained; DEFAULT_WELCOME_MESSAGE / DEFAULT_TICKET_MESSAGE imports retained
  • GitHub artifacts: PR feat: modernize monorepo, vitest test suite, nextjs 15 dashboard rewrite, lavalink v4, and cloud docs #829 body/description refreshed; issue Master-Bot Maintenance & Fix Roadmap #828 tracking comment refreshed (compliant file-based --body-file protocol via opencode scratch directory, mermaid architecture diagram included, no inline arguments with backticks, conventional commit message feat(monorepo): ...)

Dashboard (Next.js 15)

flowchart TD
    DB[DB Persistence] --> GUILD_GET[Guild.getAll]
    REDIS[Redis Live] --> GUILD_GET
    GUILD_GET --> CARD_GRID[Responsive Card Grid]
    GUILD_GET --> MANAGE[Per-Server Management]
Loading
  • guild.getAll router rebuilt: returns bot guilds from DB (ownership gate removed from [server_id]/layout.tsx; discordApi import and getUserGuilds removed; no TRPCError NOT_FOUND)
  • guilds.tsx: responsive card grid with gradient letter avatar (now displays actual Discord guild icon via guild.icon from bot session when available; guild.id shown); Manage button links to /dashboard/{guild.id}; no invite buttons
  • env.mjs: NEXT_PUBLIC_INVITE_URL exposed client-side (https://discord.com/oauth2/authorize?client_id=1325192620414210068&permissions=8&scope=bot from .env); discore_client_id configurable; SQLite DATABASE_URL default set
  • trpc.ts: server rebuilt with NextAuth v5 context (auth() session), superjson transformer, publicProcedure / protectedProcedure, prisma + redis clients in context; createTRPCRouter rebuilt; context includes session, prisma, redis
  • root.ts: tRPC routers rebuilt (welcome, channel, guild, command, music, broadcast, system, tickets); api/utils/axiosWithRefresh.ts: new refresh helper
  • Dashboard type-check: passes (0 errors); build: next 15.2 green
  • Security note: any authenticated user can view/manage all bot guilds (no ownerId gate in layout.tsx); user should add role/permission filter manually if needed

Infrastructure / Scripts / Launch

flowchart LR
    DEV[dev.mjs] --> BOT_DEV[Bot Dev]
    DEV --> DASH_DEV[Dashboard Dev]
    START[start.mjs] --> BOT_PROD[Bot Production]
    START --> DASH_PROD[Dashboard Production]
    BOT_DEV --> REDIS[Redis Service Check]
    REDIS --> LAVALINK[Lavalink Launch]
Loading
  • docker-compose.yml: PostgreSQL service fully removed; SQLite volume (sqlite-data) retained; Redis (redis-server) + Lavalink (Lavalink.jar / LAVALINK service) retained
  • docker.env: DATABASE_URL=file:./db.sqlite; POSTGRES_* vars removed
  • scripts/common.mjs: freePort() + waitForPort(); ensureRedisService() launches Redis if missing (redis-server); getLavalinkJavaArgs() reads jar path; killProcessTree() cleans child trees
  • scripts/dev.mjs: unified bot + dashboard (pnpm --filter @master-bot/bot dev + pnpm --filter @master-bot/dashboard dev); combined logs; Redis check; Lavalink check; cleanup() kills all trees
  • scripts/start.mjs: unified production (pnpm --filter @master-bot/bot start + pnpm --filter @master-bot/dashboard start); build check (nextBuildId, botDist); production banner (MASTER-BOT UNIFIED CONSOLE (PRODUCTION)); no LAVA_ENABLED warning in banner; no LAVALINK status in banner unless enabled
  • Dockerfile: updated for SQLite-only; docker-compose.yml updated; docker.env corrected

Tests / Verification

flowchart TD
    BUILD[pnpm build] --> BOT[Bot tsc]
    BUILD --> DASH[Dashboard build]
    BOT --> VERIFY[Build Green]
    DASH --> VERIFY
    VERIFY --> TYPE_CHECK[type-check 0 errors]
    TYPE_CHECK --> SMOKE[Smoke Test]
    SMOKE --> VERIFIED[All Verified]
Loading
  • tests/, vitest.config.ts, tsconfig.test.json: fully removed (no vitest suite retained; verification by pnpm build + pnpm type-check + manual smoke test + HTTP verification scripts in scratch directory C:\Users\Joshu\AppData\Local\Temp\opencode)
  • searchGif.ts verified via node fetch script checking all 36 URLs (HTTP 200 + content-type: image/gif)
  • SessionStore verified: types.ts interfaces rebuilt (GuildRecord, Reminder with guildId, MemberRecord, Ticket, Playlist, etc.); SessionStore.init() hydration verified; public facade unchanged

Notes / Security / Pending

  • lib/session/handlers/*.ts: one factory per namespace; twitchConfig receives guildData for createViaTwitchNotification
  • docs/ rewritten (14 wiki pages, README.md, CONTRIBUTING.md); no PostgreSQL references
  • AGENTS.md: reference to agent rules; .opencode/rules/04-remote-issue-protocol.md: safe file-based gh CLI protocol; .opencode/rules/index.md: index of rules
  • .github/ISSUE_TEMPLATE/comment-template.md: standard issue/PR comment templates
  • .github/pull_request_template.md: updated with verification checklist
  • CODE_OF_CONDUCT.md: retained
  • Security heads-up: dashboard layout.tsx no longer gates by ownerId; user can manage any bot guild
  • env.mjs: DISCORD_CLIENT_ID configurable (1325192620414210068); NEXT_PUBLIC_INVITE_URL used in landing page; SQLite DB path configurable via .env
  • Phase 4b smoke test completed (scripts/dev.mjs verified); optional help.ts restyle deferred; docs rewrite completed (no PostgreSQL references); Phase 4a build verified (pnpm build passes all 6 packages)
  • All commits: feat(monorepo): ... (f2e9e17, 087febc, 216aa0a, 54a7c75, 4ddc2cc, 39b6fa2)

References

…nce bot & dashboard

- Upgrade Next.js to 15.2.0 and migrate App Router to async request APIs (await params, useParams)

- Upgrade Auth.js/NextAuth to v5 beta with server action handlers and safe Discord avatar URL resolution

- Upgrade @next/eslint-plugin-next to 15.2.0 and align environment parsers to @t3-oss/env-* 0.13.11

- Replace pure-ESM env wrapper in @master-bot/bot with native Zod schema parsing for 100% CJS compatibility

- Wire dynamic feature flags (LAVA_ENABLED, GIFS_ENABLED, TWITCH_ENABLED, NEWS_ENABLED, IGDB_ENABLED) across bot preconditions

- Connect automated cross-platform PostgreSQL and Redis service checks (connect-or-auto-launch)

- Implement dynamic command help registry and standardized help tables across all 60 slash commands

- Enhance web dashboard with active-tab sidebar navigation, server overview statistics, and Redis log streaming

- Resolve next-themes hydration mismatch by adding suppressHydrationWarning to root layout
…sabled commands

- Group slash commands into structured categories (GIFs & Anime, Twitch, News, Games & Entertainment, General & Utilities)

- Filter out categories and individual commands disabled globally via environment feature flags (LAVA_ENABLED, GIFS_ENABLED, TWITCH_ENABLED, NEWS_ENABLED, IGDB_ENABLED)

- Display server-specific enable/disable toggles and active status badges for all active commands
…ys monorepo-wide

- Configure remoteCipher in application.yml with default endpoint (https://cipher.kikkia.dev/) and support custom YOUTUBE_CIPHER_URL / YOUTUBE_CIPHER_PASSWORD

- Pass deterministic Java system properties (-D) for YouTube OAuth, skipInitialization, cipher, and Spotify credentials in launcher scripts

- Wire YOUTUBE_CIPHER_URL and YOUTUBE_CIPHER_PASSWORD into @master-bot/bot, @master-bot/api, @master-bot/dashboard env schemas and .env.example

- Display active cipher endpoint in dev and production console status banners
PhantomNimbi and others added 15 commits August 30, 2026 17:12
… music controls

- Add /reminder with background scheduler (ReminderManager), tRPC router, and dashboard management pages
- Add /world-news command powered by NewsAPI with country and category filtering
- Add interactive button-based /connect-four and /tic-tac-toe mini-games
- Add dynamic 6-stage rotating StatusManager presence system
- Replace /skip with Now Playing Next button and rename /skipto to /jump
- Add repeat and shuffle action buttons to Now Playing embed and fix track duration display
- Refactor /about to standard Discord subcommands (bot, server, user)
- Implement cross-platform recursive killProcessTree in launcher scripts to eliminate zombie processes
- Standardize CommandHelp help objects and deferred interaction handling across all commands
…erence

- Add ticketRoleId to Guild model in Prisma schema and synchronize database
- Add setRole procedure to tRPC tickets router
- Add /set ticket-role and /set ticket-role-disable subcommands
- Automatically add ticket manager role members to new support ticket threads and alert the role
- Fix deferReply/editReply interaction conflict in /reminder command
- Audit and align all 70 slash commands in README.md and wiki/Commands-Reference.md
- Restore /pat command with Klipy & Waifu.im API reaction gifs
- Restore /now-playing command to display current track and interactive music controls on demand
- Restore /weather command with wttr.in real-time meteorological reports and 3-day forecast
- Restore /bored command with Bored API v2 and internal curated activity engine
- Restore /poll command with interactive Discord button voting and live progress bars
- Update README.md and wiki/Commands-Reference.md to document all 75 slash commands
…ashboard rewrite, and cloud docs

- Fix dependency installation on clean clones by switching postinstall to db:generate
- Add comprehensive Vitest test harness with 8 unit/integration test suites (16 tests, 100% pass)
- Rewrite Next.js 15 App Router web dashboard with glassmorphism UI and dedicated feature studios (Music, Broadcast, Integrations, System Telemetry)
- Expand backend tRPC v11 API routers with music, broadcast, and system health procedures
- Add multi-cloud hosting guides (Render, Railway, Fly.io, Heroku, Docker VPS) with Mermaid architecture diagrams
- Update CI/CD workflow with automatic formatting, linting, type-checking, vitest tests, and production build verification
@PhantomNimbi PhantomNimbi changed the title feat: modernize monorepo, upgrade Lavalink v4 & cipher, enhance dashboard, and fix auth/tooling feat: modernize monorepo, vitest test suite, nextjs 15 dashboard rewrite, lavalink v4, and cloud docs Sep 6, 2026
…ckend, split session handlers, and rewrite docs

Migrate the entire stack away from PostgreSQL onto SQLite and restore the

dashboard API surface that was lost when packages/api was removed, while

retaining Lavalink + Redis for music/queue state.

\### Infrastructure \& dependencies

\- docker-compose.yml: drop the postgres service; keep Lavalink + Redis;

  add sqlite-data volume at /Master-Bot/packages/db/prisma; host logs now

  map to /Master-Bot/logs

\- docker.env: replace POSTGRES\_\* with DATABASE\_URL="file:./db.sqlite"

\- Dockerfile: remove stale POSTGRES\_HOST comment

\- scripts/{common,dev,start}.mjs: strip postgres service ensure/ports/status;

  show SQLite-backed status

\- pnpm-workspace.yaml: drop removed packages/api and packages/session

\- turbo.json: remove SHADOW\_DB\_URL (keep REDIS\_\* env)

\- apps/dashboard/next.config.mjs: transpilePackages -> @master-bot/auth, @master-bot/db

\- remove vitest.config.ts, tsconfig.test.json, and the tests/ tree

\### Database

\- packages/db/prisma/schema.prisma: Guild notifyList, disabledCommands,

  logEvents stored as JSON-encoded String columns; reminders are guild-scoped

  (guildId required)

\- packages/db/prisma/db.sqlite is the schema-relative SQLite database file

\### Bot: session layer

\- delete the dead @master-bot/session package and apps/bot/src/trpc.ts

\- split SessionManager.ts (1184 lines) into lib/session/:

  types.ts, SessionStore.ts (state + persistence + hydration), handlers/ with

  one factory per namespace (users, guildData, welcomeMessages, tickets,

  twitchConfig, hubChannels, playlists, songs, reminders, commands, members)

\- SessionManager is now a thin facade; public API unchanged

\- align all consumers (music playlists, reminders, twitch notify, tickets,

  temp channels, preconditions, listeners) with the split handlers and the

  JSON-encoded guild fields; add guildMemberRemove listener

\### Bot: gifs

\- lib/gifs/searchGif.ts: replace dead/mismatched fallback GIFs with 3 SFW,

  verified-working, query-matched GIFs per category (36 URLs, HTTP-verified)

\### Dashboard

\- add server-side tRPC backend at apps/dashboard/src/server: trpc.ts,

  context.ts (NextAuth session + prisma), root.ts, routers/ (guild, channel,

  welcome, tickets, command, music, broadcast, system), utils/axiosWithRefresh.ts

\- rewrite app/api/trpc/\[trpc]/route.ts and utils/api.ts (typed AppRouter);

  DISCORD\_CLIENT\_ID/SECRET placeholders added to env.mjs

\- guild list now shows every server the bot is in as card UI with Manage

  buttons; guild.getAll returns all bot guilds (no Discord OAuth ownership

  fetch); \[server\_id] layout no longer redirects non-owners

\- fix pre-existing schema mismatches: command disable/log-event consumers now

  JSON parse/serialize Scalar String columns; reminders require an owned guild

\- add axios dependency

\### Docs

\- rewrite root README, CONTRIBUTING, apps/bot + apps/dashboard READMEs

\- consolidate wiki/ into 14 updated pages (Architecture, Commands,

  Configuration, Dashboard, Deployment, FAQ, Getting-Started, Moderation,

  Music, Reminders-and-Twitch, Tickets, Welcome-and-Temp-Channels, \_Sidebar,

  Home); remove legacy cloud/Heroku/lavalink/API-key pages
@PhantomNimbi

PhantomNimbi commented Sep 7, 2026

Copy link
Copy Markdown
Author

✅ All Work Completed

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant