Jump to content

Email/Architecture

From Stronghold Wiki
Revision as of 18:18, 5 July 2026 by Wikibot (talk | contribs) (Migrated from TODO.md email section)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

As of: 2026-07-05 — migrated from `TODO.md` (Email section) and cross-referenced with Services/Stalwart

See also: Services/Stalwart for the full mail server operational reference (admin API, ports, sieve rules, gotchas). This page covers the broader mail architecture goals and phase history.

Email (Stalwart Migration)

Architecture goal: Multi-user family & friends mail server

<syntaxhighlight lang="text"> Inbound: Internet → MXRoute (per-user spool) → getmail → Stalwart (per-user inbox) Outbound: User authenticates to Stalwart SMTP with own Authentik creds 

         → Stalwart relays via user's own MXRoute credentials (per-user relay routes)
         → 400 msg/hr global rate limit to MXRoute enforced by Stalwart queue throttle

Domains: craniumslows.com now. wilsoz.com → MXRoute once craniumslows validated. Users: Authentik LDAP → Stalwart. Custom attributes drive address provisioning. Spam: Server-wide Stalwart sieve → spam into each user's Junk Sieve: Per-user rules via ManageSieve (no rule affects another user) Webmail: SnappyMail at mymail.wilsoz.com External: Gmail/iCloud/SDF as SnappyMail additional accounts (direct IMAP, not getmail) </syntaxhighlight>

User provisioning model (Authentik custom LDAP attributes)

Each Authentik user gets three custom attributes:

  • mail-name — primary local part (e.g. joe)
  • mail-alias — secondary local part, optional (e.g. joe.smith)
  • default-domain — which domain is the primary SnappyMail identity

Groups control which domains are active:

  • craniumslows-email → @craniumslows.com addresses
  • wilsoz-email → @wilsoz.com addresses

Address matrix = {mail-name, mail-alias} × {domains from group memberships} All addresses on a user deliver to one Stalwart inbox. Each address gets its own MXRoute account + getmail .rc + per-user Stalwart relay route.

Address changes: old address becomes a forwarder to the new one. Inbox never moves.

Where the real mail data lives

Location What Size Action
/library/email/selfhosted/mail/wilsoz.com/marcus/Maildir/ Primary mailbox — Mailcow/Dovecot, ~100 nested folders ~8.9GB Import to Stalwart — in progress
/library/email/selfhosted/mail/wilsoz.com/marcus/Orig-Maildir/ Pre-Mailcow import (iRedMail/Gmail era) (subset) Verify not unique before discarding
/library/email/selfhosted/mail/craniumslows.com/marcus/Maildir/ craniumslows.com mailbox 152K Import to Stalwart
/var/mail/mnw Old iRedMail mbox (pre-Mailcow era) 37MB Import after primary migration
/opt/mailcow-dockerized/ Mailcow app stack only 49MB Remove after migration verified

Phase 3 (Mail Import) — COMPLETE ✓

  • [x] Import [email protected] — 137,335 messages from Mailcow Maildir, all folders preserved
  • [x] Import [email protected] — done (was tiny, only pre-Stalwart messages)
  • [x] Verify in SnappyMail — folder hierarchy confirmed, messages accessible
  • [ ] Import old iRedMail mbox — /var/mail/mnw (37MB) not yet done (low priority, very old mail)

Phase 4 (Getmail) — COMPLETE ✓ (one item remaining)

  • [x] Update delivery targets in .rc files → Stalwart LMTP localhost:24
  • [x] Create craniumslows-marcus.rc — fetches [email protected] from MXRoute, 439/439 delivered
  • [x] Keep delete = false — correctly set; flip to true only after validating in daily use
  • [x] Create systemd timer — getmail runs every 10 minutes via systemd
  • [x] Fastmail import COMPLETE — 12,226/12,226 delivered (2026-02-22 ~15:52)
  • [ ] ~~Configure SnappyMail additional accounts~~ — ARCHITECTURE CHANGED (see below): external accounts now use getmail, not SnappyMail connected accounts

Architecture decision (2026-02-22)

Marcus's external accounts (Gmail x2, iCloud, SDF) use getmail → Stalwart, NOT SnappyMail connected accounts. This gives a true unified inbox with sieve filtering. Phone/clients connect to one IMAP server.

  • [x] Add getmail config: [email protected] — sieve/gmail-thewilson.rc done. Import in progress: ~7,560/58,520 as of 2026-02-23 07:00 (~50 hrs remaining at current rate)
  • [x] Add getmail config: [email protected] — sieve/gmail-craniumslows.rc done. Will start after thewilson finishes.
  • [x] Add getmail config: [email protected] — sieve/icloud-marcusthewilson.rc done.
  • [x] Add getmail config: [email protected] — sieve/sdf-mnw.rc done. delete=false.
  • [x] Update systemd getmail.service — all 6 rc files active, daemon reloaded 2026-02-23.

Action needed — Phase 5 (Spam + Sieve)

  • [x] Fastmail import COMPLETE — 12,226/12,226 (done 2026-02-22)
  • [x] Sieve rules written — sieve/marcus.sieve ready. Decade-based numbering, all rules merged. See sieve/README.md.
  • [x] Mail classification script written — sieve/classify-inbox.py done.

[x] Audit actual Stalwart folder names — confirmed INBOX/XX- naming in place.

  • [x] Upload sieve/marcus.sieve — uploaded and ACTIVE via ManageSieve (confirmed 2026-02-23). Script: sieve/upload-sieve.py.
  • [x] Dedup script written — sieve/dedup.py. Dry-run confirmed 6,622 dupes. Run --execute to purge.
  • [ ] Run dedup + retroactive sort + empty .Junk — → plans/sieve-dedup.md (covers all three)
  • [ ] Enable server-wide spam filter — after sieve rules confirmed working in daily use

Action needed — Phase 6 (n8n provisioner + per-user relay)

  • [x] Create craniumslows-email and wilsoz-email groups in Authentik — done 2026-02-22; marcus in both
  • [x] Marcus's mailAddresses attribute set — {"mailAddresses": ["[email protected]"]} — enables dual-domain delivery in Stalwart
  • [x] Build n8n provisioner workflow — Matrix bot IYw0DmCuWEsiV2k9 (deployed 2026-03-01). !provision in #notifications room, walks through username/name/password/domain conversation. Adaptive polling: 15s during conversation, 5-min throttle when idle.
  • [x] Add custom LDAP attributes to Authentik users: mail-name, mail-alias, default-domain (future, when adding more users) — done 2026-05-10: attributes backfilled on marcus and now written by the provisioner for new users; LDAP property mappings + downstream consumers still deferred
  • [x] Configure global MXRoute rate limit — Stalwart queue throttle 400 msg/hr to blizzard.mxrouting.net. → plans/stalwart-helpers-fix.md (Phase 5) — done 2026-05-10: 400/1h limiter set 2026-05-10 via legacy prefix/values API
  • [x] Migrate marcus to per-user relay routes — replace shared craniumslows-mxroute with marcus-specific route — done 2026-05-10: marcus-craniumslows relay + per-sender rule added; shared craniumslows-mxroute kept as fallback

Action needed — [email protected] cleanup

  • [x] jkatz Authentik user — Confirmed deleted (pk:18 not found 2026-05-09). MXRoute account was never created. Consider done.

Action needed — Matrix deprovisioner (next session)

  • [x] Build Matrix deprovisioner — → plans/email-deprovisioner.md (depends on plans/stalwart-helpers-fix.md + plans/email-provisioner.md) — done 2026-05-10: deployed as jbGL9N1vXT8mB92V, end-to-end tested 2026-05-10

Action needed — Provisioner cleanup strategy

  • [x] Define orphan .rc file retention policy — done 2026-05-10: policy = move .rc into disabled/ with .deprov-YYYY-MM-DD suffix on deprovision; weekly cron (/etc/cron.d/getmail-disabled-purge) purges entries >30 days old. Legacy duplicates + test residue swept.

Action needed — Phase 7 (wilsoz.com → MXRoute)

  • [ ] Validate craniumslows.com in daily use (give it weeks)
  • [ ] Move wilsoz.com MX DNS → MXRoute
  • [ ] Replace fastmail.rc with wilsoz-marcus.rc (MXRoute fetch)
  • [ ] Update Stalwart wilsoz.com relay → MXRoute smarthost
  • [ ] Flip delete = true on all getmail configs once cutover verified

Minor

  • [ ] Investigate unexpected folders in SnappyMail MoveTo dropdown

Status (updated 2026-02-22)

Phases 1–4 complete. Stalwart + SnappyMail deployed, LDAP auth working with dual-domain delivery. Maildir import done (137k+ messages). Getmail running: craniumslows 439/439, Fastmail ~5.5k/12.2k in progress. craniumslows-email/wilsoz-email groups created in Authentik. Phase 5 (spam/sieve) and Phase 6 (n8n provisioner) are next. One manual step remaining: add 4 external accounts to SnappyMail UI.

Retrieved from "https://wiki.wilsoz.com/index.php?title=Email/Architecture&oldid=24"