Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -64,3 +64,4 @@ yarn-debug.log*

# Worktrees
.worktrees/
/log/merge_duplicate_members/
20 changes: 20 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -90,5 +90,25 @@ test: ## Run the test suite in parallel
bundle exec rake parallel:setup
bundle exec parallel_rspec spec/ -n 3

detect_duplicate_members: ## Detect members duplicated by the codebar auth flow
DB_NAME=$(DUMP_DB) bundle exec rake member:duplicates:detect

fix_duplicate_members: ## Dry-run merge of duplicate members (set APPLY=1 to execute)
DB_NAME=$(DUMP_DB) bundle exec rake member:duplicates:fix

verify_duplicate_members: ## Verify no codebar-auth duplicate members remain
DB_NAME=$(DUMP_DB) bundle exec rake member:duplicates:verify

detect_duplicate_members_production: ## Detect duplicates on production DB directly
@read -p "Connect to PRODUCTION database? [y/N] " ans && [ "$$ans" = "y" ] || exit 1
@DB_URL=$$(heroku config:get DATABASE_URL --app=$(DUMP_APP)) bundle exec rake member:duplicates:detect

fix_duplicate_members_production: ## Dry-run merge on production DB (APPLY=1 executes)
@read -p "This MODIFIES the PRODUCTION database. Continue? [y/N] " ans && [ "$$ans" = "y" ] || exit 1
@DB_URL=$$(heroku config:get DATABASE_URL --app=$(DUMP_APP)) bundle exec rake member:duplicates:fix

verify_duplicate_members_production: ## Verify no duplicates remain on production DB
@DB_URL=$$(heroku config:get DATABASE_URL --app=$(DUMP_APP)) bundle exec rake member:duplicates:verify

check: ## Run setup checks
bundle exec rake setup:check
105 changes: 105 additions & 0 deletions docs/merge_duplicate_members.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
# Merge Duplicate Members

Temporary tool to detect and merge duplicate members created by the `/auth/codebar` GitHub sign-in flow (codebar/planner#2805).

## Background

When the codebar auth app was merged into planner on 2026-08-06, members with a GitHub account whose email differed from their `auth_services` record could not be matched automatically. The codebar auth flow created new accounts instead of linking to existing ones.

This tool finds those duplicates (by name, email, and auth UID heuristics) and merges their data into the original member.

## Prerequisites

A PostgreSQL dump of the production database, accessible as `codebar_production_dump`.

## Tasks

### Detect duplicates

```bash
make detect_duplicate_members
```

Dry run — lists all duplicate pairs found, with which detection strategies matched.

### Fix duplicates (dry run)

```bash
make fix_duplicate_members
```

Shows every step the merger would take, but wraps everything in a transaction that is rolled back. No data is modified.

### Fix duplicates (execute)

```bash
make fix_duplicate_members APPLY=1
```

Performs the merge for real inside a transaction.
A JSON log file is written to:

```
log/merge_duplicate_members/run_YYYYMMDDTHHMMSSZ.json
```

The path is printed after the run completes.

### Verify

```bash
make verify_duplicate_members
```

Re-detects duplicates and exits `0` when none remain, or `1` with the remaining pairs.

## Detection strategies

| Strategy | Description |
|----------|-------------|
| `name+surname` | Exact case-insensitive match on both fields |
| `email` | Exact case-insensitive match on email |
| `first-name+uid-surname` | First name matches; duplicate has no surname, but the codebar auth UID contains the original’s surname |
| `domain+local-part` | Non-generic domain; local parts overlap |

The tool also applies hard-coded manual overrides for edge cases the heuristics cannot detect.

## Safety properties

- **Dry run by default** — must pass `APPLY=1` to change data.
- **Idempotent** — re-running after a successful merge reports no duplicates.
- **Deactivates, not deletes** — duplicates are renamed to `duplicate.<id>.merged-into.<id>@codebar.io`, with all auth services and roles removed. Audit history is preserved in a `MemberNote`.
- **Logs every execution** — merge results written to a new per-run JSON file, even on failure. The log contains only opaque member IDs (`dup_id`, `orig_id`), strategies, and status. No names, emails, UIDs, or other PII. Safe to share or attach to PRs.

## Running in production

### Option A: local execution (no deploy required)

You can run the tool locally and connect directly to the Heroku production database using a connection string:

```bash
make detect_duplicate_members_production # lists duplicates from production
make fix_duplicate_members_production # dry-run against production
make fix_duplicate_members_production APPLY=1 # execute on production
make verify_duplicate_members_production # verify production
```

These targets fetch a fresh `DATABASE_URL` from Heroku each time and pass it via `DB_URL`. A confirmation prompt guards the detect and fix targets. A loud `WARNING: Connecting to REMOTE database …` message is printed when `DB_URL` points to a non-localhost host.

### Option B: run on Heroku dyno

Deploy the branch first, then run on Heroku:

```bash
heroku run rake member:duplicates:detect --app codebar-production
heroku run rake member:duplicates:fix APPLY=1 --app codebar-production
heroku run rake member:duplicates:verify --app codebar-production
```

## What to do if a false positive appears

Duplicate pairs can be added to the `MANUAL_OVERRIDES` array or excluded before execution. If in doubt, err on the side of not merging — the merge does not delete records, but it does permanently move associated data and deactivate the account.

---

*This is a temporary tool. Once all existing duplicates are resolved, it can be removed.*
Loading