Services/Stalwart
As of: 2026-07-05 — migrated from services/STALWART.md. Credential values redacted per Rule 0 (see OpenBao secret/stalwart) — the source repo file still has plaintext values and needs a separate scrub + rotation, flagged separately.
Stalwart — Mail Server (SMTP/IMAP/LDAP/Sieve/JMAP)
Complete setup & operations guide for Stalwart mail server with LDAP authentication, relay routing, and getmail integration.
Criticality: HIGH — Mail infrastructure. Losing Stalwart means loss of email receive/send capability. Unlike Authentik (SSO), Stalwart failure doesn't break other services, but email is critical infrastructure. Requires careful recovery.
Overview
Stalwart is a modern open-source mail server that provides:
- SMTP (port 25 inbound, 465/587 outbound) — message transport
- IMAP (port 143/993) — message retrieval with TLS
- POP3 (port 110/995) — legacy message retrieval
- Sieve (port 4190/ManageSieve) — mail filtering rules
- JMAP (port 443) — modern mail protocol (experimental)
- LDAP (LDAP passthrough to Authentik at port 389) — user authentication
- Web Admin (port 8181 internal HTTP API) — configuration and management
Current setup:
- Primary domain:
wilsoz.com— [email protected] is main account - Secondary domain:
craniumslows.com— accounts federated to wilsoz via getmail - Inbound mail sources: Fastmail, MXRoute, Gmail, iCloud, SDF (fetched via getmail)
- Outbound relay routes: Fastmail (wilsoz.com), MXRoute (craniumslows.com), Gmail, SDF (via proxy)
- Mail clients: eM Client (IMAP/SMTP), SnappyMail (web), Thunderbird (config compatible)
Current Version: Stalwart latest (updates automatically via Watchtower)
Access:
- Internal admin API:
http://localhost:8181/api - ManageSieve:
localhost:4190(STARTTLS required) - IMAP:
localhost:993(TLS) ormymail.wilsoz.com:993(external) - SMTP:
localhost:465ormymail.wilsoz.com:465(external)
Service Information
| Property | Value |
|---|---|
| Compose file | /library/stalwart-app/docker-compose.yml
|
| Container name | stalwart-mail
|
| Image | stalwartlabs/stalwart:latest
|
| Network mode | host (direct network access, not bridged)
|
| Restart policy | unless-stopped
|
| Data volume | /library/stalwart-app/data/ (contains /etc/, /queue/, /store/)
|
| TLS certificates | /etc/letsencrypt/live/mymail.wilsoz.com/ (mounted readonly)
|
| Admin credentials | admin:<REDACTED - see OpenBao secret/stalwart> (plaintext password, hashed in config.toml)
|
| Admin API port | 8181 (internal only) |
| SMTP port | 25 (inbound), 465 (outbound implicit TLS), 587 (submission STARTTLS) |
| IMAP ports | 143 (STARTTLS), 993 (implicit TLS) |
| POP3 ports | 110 (STARTTLS), 995 (implicit TLS) |
| Sieve port | 4190 (ManageSieve, STARTTLS then PLAIN auth) |
| JMAP port | 443 (via reverse proxy, not directly exposed) |
| LDAP passthrough | localhost:389 (to Authentik LDAP outpost)
|
Ports Summary
| Port | Protocol | Direction | Purpose | TLS | Notes |
|---|---|---|---|---|---|
| 25 | SMTP | Inbound | Receive mail from internet | None | MX record points here |
| 465 | SMTPS | Outbound | Send via Fastmail/MXRoute | Implicit | ISP allows only 465, blocks 25/587 |
| 587 | SMTP | Outbound | Send via some relays | STARTTLS | ISP blocks this, use 465 instead |
| 143 | IMAP | Internal | Retrieve mail (legacy) | STARTTLS | Plain AUTH disabled, must TLS first |
| 993 | IMAPS | Internal | Retrieve mail (standard) | Implicit | Recommended, supports admin impersonation |
| 110/995 | POP3/S | Internal | Legacy retrieval | — | Available but unused |
| 4190 | ManageSieve | Internal | Upload sieve rules | STARTTLS | Auth only advertised post-TLS |
| 8181 | HTTP API | Internal | Admin config/settings | None | Internal only, no TLS |
Initial Setup from Scratch
Prerequisites
- Docker and docker-compose installed
- LDAP access to Authentik LDAP outpost (must be running first)
- TLS certificate for
mymail.wilsoz.comat/etc/letsencrypt/live/mymail.wilsoz.com/ - DNS MX records configured to point to Stronghold
- Outbound SMTP to external relays configured (Fastmail, MXRoute, etc.)
1. Create Directory & Files
<syntaxhighlight lang="bash"> mkdir -p /library/stalwart-app/data cd /library/stalwart-app </syntaxhighlight>
2. Create docker-compose.yml
<syntaxhighlight lang="yaml"> version: "3.8"
services:
stalwart:
image: stalwartlabs/stalwart:latest
container_name: stalwart-mail
hostname: stronghold
network_mode: host
restart: unless-stopped
volumes:
- ./data:/opt/stalwart
- /etc/letsencrypt:/opt/letsencrypt:ro
env_file:
- .env
</syntaxhighlight>
3. Create .env File
<syntaxhighlight lang="bash">
- Admin credentials (plaintext; password gets hashed in config.toml)
STALWART_ADMIN_USER=admin STALWART_ADMIN_PASSWORD=<generate-strong-password-20-chars>
- LDAP (Authentik LDAP outpost)
LDAP_HOST=localhost LDAP_PORT=389 LDAP_BASE_DN="dc=ldap,dc=goauthentik,dc=io" LDAP_USER_BASE="ou=users,dc=ldap,dc=goauthentik,dc=io" LDAP_SERVICE_ACCOUNT="cn=stalwart-svc,ou=users,dc=ldap,dc=goauthentik,dc=io" LDAP_SERVICE_PASSWORD=<from-Authentik-stalwart-svc-user>
- Mail domains
PRIMARY_DOMAIN=wilsoz.com SECONDARY_DOMAIN=craniumslows.com
- TLS certificate (mounted from host)
TLS_CERT_PATH=/opt/letsencrypt/live/mymail.wilsoz.com/fullchain.pem TLS_KEY_PATH=/opt/letsencrypt/live/mymail.wilsoz.com/privkey.pem </syntaxhighlight>
4. Create config.toml (Bootstrap Configuration)
This is a complex TOML file. Start with a minimal bootstrap config containing only:
- Listener (SMTP, IMAP, POP3, Sieve ports)
- Storage (database and message store)
- TLS certificate paths
- Authentication (LDAP + fallback admin)
- Admin API
Key principle: All persistent settings (relay routes, routing strategy, rate limits) must be configured via the Admin HTTP API at port 8181, NOT in config.toml. The reason: GET /api/reload rewrites config.toml in flat key=value format, stripping all TOML sections.
<syntaxhighlight lang="toml">
- Minimal bootstrap config.toml
[server] hostname = "stronghold"
- Listeners
listeners.smtp bind = "0.0.0.0:25" protocol = "smtp"
listeners.smtp bind = "0.0.0.0:465" protocol = "smtp" tls.implicit = true
listeners.smtp bind = "0.0.0.0:587" protocol = "smtp" tls.starttls = true
listeners.imap bind = "0.0.0.0:143" protocol = "imap" tls.starttls = true
listeners.imap bind = "0.0.0.0:993" protocol = "imap" tls.implicit = true
listeners.pop3 bind = "0.0.0.0:110" protocol = "pop3" tls.starttls = true
listeners.pop3 bind = "0.0.0.0:995" protocol = "pop3" tls.implicit = true
listeners.managesieve bind = "0.0.0.0:4190" protocol = "sieve" tls.starttls = true
listeners.http bind = "127.0.0.1:8181" protocol = "http"
- TLS
[certificate.default] cert = "/opt/letsencrypt/live/mymail.wilsoz.com/fullchain.pem" private-key = "/opt/letsencrypt/live/mymail.wilsoz.com/privkey.pem"
- Storage (SQLite database + file-based message store)
[storage.data] type = "sqlite" path = "/opt/stalwart/store/data.sqlite"
[storage.fts] type = "sqlite" path = "/opt/stalwart/store/fts.sqlite"
[storage.directory] type = "sqlite" path = "/opt/stalwart/store/directory.sqlite"
[storage.lookup] type = "sqlite" path = "/opt/stalwart/store/lookup.sqlite"
[storage.blob] type = "fs" path = "/opt/stalwart/store/blobs"
[storage.queue] type = "fs" path = "/opt/stalwart/queue"
- Authentication
[auth] directory = "ldap"
directory.ldap name = "ldap" address = "localhost" port = 389 base-dn = "dc=ldap,dc=goauthentik,dc=io" user-base-dn = "ou=users,dc=ldap,dc=goauthentik,dc=io" bind-dn = "cn=stalwart-svc,ou=users,dc=ldap,dc=goauthentik,dc=io" bind-password = "<from OpenBao: secret/stalwart/ldap-bind-password>" # rotated 2026-05-18 filter = "(&(objectClass=posixAccount)(cn=?))" mail-attribute = "mail"
- Fallback admin (in case LDAP fails)
[authentication.fallback-admin] user = "admin" password = "$6$HFsr55uFqNc31Mw8$rLv/wtAzFCr0yNfY.../...hashed-password" # Will be generated on first startup
- Domains (minimal)
domains name = "wilsoz.com"
domains name = "craniumslows.com"
- Admin API
[server.admin] bind = "127.0.0.1" port = 8181
- Logging
[tracer.log] level = "info" </syntaxhighlight>
5. Start Stalwart
<syntaxhighlight lang="bash"> cd /library/stalwart-app docker compose up -d
- Wait for startup (~10 seconds)
sleep 10
- Check logs
docker logs stalwart-mail | tail -20
- Should see: "Stalwart Mail Server started" and port listeners binding
</syntaxhighlight>
6. Configure Via Admin API (port 8181)
After startup, all configuration happens via the HTTP API. See "Admin API" section below for how to:
- Create mail accounts (principals)
- Configure relay routes (outbound SMTP)
- Configure routing strategy (which route to use for each sender)
- Upload sieve rules
- Set spam filter thresholds
Example: Create a mail account
<syntaxhighlight lang="bash"> curl -u admin:<REDACTED - see OpenBao secret/stalwart> -X POST \
http://localhost:8181/api/principal \ -H "Content-Type: application/json" \ -d '{ "type": "individual", "name": "[email protected]", "emails": ["[email protected]"], "secrets": ["marcus-password"] }'
</syntaxhighlight>
7. Configure LDAP Sync
Stalwart can authenticate against Authentik LDAP, but users must exist. The simplest approach:
- Create users in Authentik → Directory → Users
- Add them to
wilsoz-emailorcraniumslows-emailgroups - Stalwart reads LDAP attributes:
mail,mailAddresses,memberOf
Test LDAP:
<syntaxhighlight lang="bash"> ldapsearch -x -H ldap://localhost:389 \
-D "cn=stalwart-svc,ou=users,dc=ldap,dc=goauthentik,dc=io" \ -w "<rotated 2026-05-18; see OpenBao secret/stalwart/ldap-bind-password>" \ -b "ou=users,dc=ldap,dc=goauthentik,dc=io" \ "cn=marcus" mail mailAddresses
</syntaxhighlight>
8. Set Up TLS Certificate Renewal
Stalwart uses certificates at /opt/letsencrypt/live/mymail.wilsoz.com/. When certificates renew (certbot, daily check):
- Create renewal hook to reload Stalwart:
<syntaxhighlight lang="bash"> sudo nano /etc/letsencrypt/renewal-hooks/deploy/stalwart-reload.sh </syntaxhighlight>
<syntaxhighlight lang="bash">
- !/bin/bash
- Reload Stalwart config after certificate renewal
docker exec stalwart-mail curl -u admin:<REDACTED - see OpenBao secret/stalwart> http://localhost:8181/api/reload echo "Stalwart reloaded on $(date)" >> /var/log/stalwart-reload.log </syntaxhighlight>
<syntaxhighlight lang="bash"> sudo chmod +x /etc/letsencrypt/renewal-hooks/deploy/stalwart-reload.sh </syntaxhighlight>
- Test renewal:
<syntaxhighlight lang="bash"> sudo certbot renew --dry-run </syntaxhighlight>
9. Configure VPS Reverse Proxy (if external access needed)
On VPS, forward mymail.wilsoz.com to Stronghold. Example nginx config:
<syntaxhighlight lang="nginx">
- IMAP/SMTP passthrough (not HTTP)
- This requires HAProxy or similar (not standard nginx)
- Alternatively, use Stronghold's direct port exposure + firewall rules
</syntaxhighlight>
Actually, for mail protocols (SMTP/IMAP), most setups expose ports directly. Configure firewall rules on stronghold to allow:
- Port 25 (inbound SMTP) — public
- Port 465/587 (outbound) — already open by ISP
- Port 993 (IMAP, TLS) — expose to VPS or public
Disaster Mode A: Configuration Corruption
Symptoms
- Can't send/receive mail
- LDAP authentication failing
- Sieve rules not executing
- Relay routes returning errors
- Database corrupted (SQLite integrity errors)
- Admin credentials lost or forgotten
Recovery Steps
Option 1: Reset Configuration (Keep Mail Data)
If only config is corrupted but mail data is intact:
<syntaxhighlight lang="bash"> cd /library/stalwart-app
- Stop Stalwart
docker compose down
- Backup old config
cp data/etc/config.toml data/etc/config.toml.corrupted-$(date +%Y%m%d)
- Copy clean bootstrap config (from Initial Setup step 4)
cp config.toml.bootstrap data/etc/config.toml
- Start fresh
docker compose up -d
- Reconfigure via API (routes, strategy, sieve, etc.)
- See Admin API section below
</syntaxhighlight>
Option 2: Reset Admin Password (Via Direct API)
If you can still reach the API but lost the admin password:
<syntaxhighlight lang="bash">
- Edit config.toml directly to set new password
- Hash password using Stalwart's built-in hash
docker exec stalwart-mail stalwartd hash-password "NewPassword123"
- Copy the hash output, edit config.toml:
[authentication.fallback-admin] user = "admin" password = "<paste-hash-here>"
- Restart
docker compose restart stalwart-mail </syntaxhighlight>
Option 3: Full Reset (Lose All Configuration + Users)
⚠️ This deletes everything except mail messages themselves.
<syntaxhighlight lang="bash"> cd /library/stalwart-app docker compose down
- Delete all config and database
rm -rf data/etc/* rm -rf data/store/*.sqlite mkdir -p data/etc data/store/blobs data/queue
- Copy bootstrap config
cp config.toml.bootstrap data/etc/config.toml
- Delete Docker volume (optional, if vol became corrupted)
- docker volume rm stalwart-mail_data
- Start from scratch
docker compose up -d
- Reconfigure everything:
- 1. Create relay routes
- 2. Create routing strategy
- 3. Create mail accounts
- 4. Upload sieve rules
- 5. Configure spam filters
</syntaxhighlight>
Disaster Mode B: Drive Failure / Full Recovery
If Backups Accessible
Stalwart data is backed up:
- PostgreSQL dumps (if using PG backend) — at Hetzner
/dumps/stalwart-*.sql.gz - Restic snapshots — at Hetzner
/data/stalwart/ - Timeshift snapshots — on
/raid1/timeshift/(if /var/lib/docker/volumes issue fixed)
Recovery Steps (Using Restic):
<syntaxhighlight lang="bash">
- 1. Deploy fresh Stalwart container (follow Initial Setup)
cd /library/stalwart-app docker compose up -d
- 2. Stop Stalwart
docker compose down
- 3. Restore mail data from Restic
cd /library/backup-stack docker compose up -d
docker exec restic-backup restic restore latest \
--target /tmp/restore \ --repo sftp:[email protected]:./backups \ --include "/data/stalwart"
- 4. Copy restored data to /library/stalwart-app
sudo cp -r /tmp/restore/data/stalwart/* /library/stalwart-app/data/ sudo chown -R 0:0 /library/stalwart-app/data/ # Stalwart runs as root in container
- 5. Start Stalwart
cd /library/stalwart-app docker compose up -d
- 6. Verify
docker logs stalwart-mail | grep -i "listener\|directory\|ldap" curl -u admin:<REDACTED - see OpenBao secret/stalwart> http://localhost:8181/api/principal | jq '.principals | length' </syntaxhighlight>
If Backups NOT Accessible
⚠️ Mail data will be permanently lost. Recovery options:
- Restore from getmail sources:
- Getmail configs at
/library/email/getmail/*.rcfetch from external sources - If Hetzner or tape backup includes getmail state files, can re-fetch mail
- Mail will only go back to getmail fetch dates (~7 days, depending on retention)
- Getmail configs at
- Request users to re-send important emails
- Restore from email client backups:
- If eM Client/Thunderbird cached mail, can export and reimport
- Full rebuild (minimal configuration, zero historical mail):
- Create new empty Stalwart instance
- Reconfigure routes, domains, users
- All historical mail is lost; only forward-going mail is received
Data Locations
Stalwart Data on Host
<syntaxhighlight lang="text"> /library/stalwart-app/ ├── docker-compose.yml (service definition) ├── .env (LDAP passwords, admin user) ├── config.toml.bootstrap (clean bootstrap config) └── data/
├── etc/config.toml (runtime config, auto-rewritten by /api/reload) ├── store/ │ ├── data.sqlite (principal directory, accounts, groups) │ ├── fts.sqlite (full-text search index) │ ├── directory.sqlite (LDAP cache) │ ├── lookup.sqlite (MX/DNS lookups) │ └── blobs/ (message attachments, large objects) └── queue/ (outbound mail queue)
</syntaxhighlight>
Content Storage
Messages themselves are stored differently based on storage backend:
- FileSystem (current):
/library/stalwart-app/data/store/blobs/— one file per message/attachment - Database: Could be SQLite or PostgreSQL
Backup Locations
- Hetzner Restic:
/data/stalwart/(daily snapshots)
Local Timeshift: /raid1/timeshift//library/stalwart-app/data/ (if /var/lib/docker/volumes included)
- Home directory: Config snippets, scripts at
/home/mnw/Developer/SystemResiliency/
What's Backed Up
✅ All mail messages (blobs/) ✅ Account directory and settings (data.sqlite) ✅ LDAP cache (directory.sqlite) ✅ Outbound queue (queue/) ✅ Config.toml (auto-rewritten but restored)
❌ NOT backed up by Timeshift: /var/lib/docker/volumes (includes SQLite WAL files — known issue)
✅ Backed up by Restic: All /library/stalwart-app content
Critical Gotchas
1. config.toml Rewriting Gotcha (CRITICAL)
Problem: When you call GET /api/reload, Stalwart rewrites config.toml in flat key=value format, stripping all TOML sections.
<syntaxhighlight lang="toml">
- Original config.toml
[server.admin] bind = "127.0.0.1" port = 8181
- After /api/reload
server.admin.bind = 127.0.0.1 server.admin.port = 8181 </syntaxhighlight>
Consequence: All relay routes, routing strategy, rate limits, spam settings created via API are LOST when rewriting happens.
Solution:
- Only store bootstrap/listener/storage/auth settings in config.toml
- All persistent settings (routes, strategy, spam) MUST live in the database via
/api/settingsAPI - Never call
/api/reloadunless you're OK with losing API-configured settings - If you accidentally called it, restore from backup immediately
2. Routes Live Under queue.route.* Namespace (CRITICAL)
Wrong: <syntaxhighlight lang="bash"> curl -s -u admin:<REDACTED> http://localhost:8181/api/settings/list?prefix=route.
- Returns 0 results — namespace is wrong
</syntaxhighlight>
Right: <syntaxhighlight lang="bash"> curl -s -u admin:<REDACTED> http://localhost:8181/api/settings/list?prefix=queue.route.
- Returns all routes
</syntaxhighlight>
Lesson: Always use full namespace path queue.route.NAME not route.NAME.
3. Settings API Format — Stalwart 0.15.4 Specific
assert_empty field is REQUIRED (and must be snake_case, NOT camelCase):
<syntaxhighlight lang="bash">
- WRONG — camelCase
-d '[{"type":"insert","prefix":"queue.route.fastmail","assertEmpty":false,...}]'
- Returns: 400 JSON deserialization failed
- RIGHT — snake_case
-d '[{"type":"insert","prefix":"queue.route.fastmail","assert_empty":false,...}]' </syntaxhighlight>
Quirk: The enum type values are camelCase (insert, delete, clear), but the struct fields use snake_case due to Serde serialization rules. This is version-specific to 0.15.4. Post-0.16.0 may change.
4. ISP Blocks Port 25 AND 587 Outbound — Port 465 Only
Fact: Your ISP (likely Comcast or similar) blocks:
- ✅ Inbound port 25 (you can receive)
- ❌ Outbound port 25 (can't relay)
- ❌ Outbound port 587 (can't relay, STARTTLS submission)
- ✅ Outbound port 465 (implicit TLS works)
Configuration: All outbound relay routes must use:
port = 465tls.implicit = trueprotocol = "smtp"
Example (Fastmail relay): <syntaxhighlight lang="bash"> curl -s -u admin:<REDACTED> http://localhost:8181/api/settings -H "Content-Type: application/json" \
-d '[{"type":"insert","prefix":"queue.route.fastmail-relay","assert_empty":false,"values":[
["type","relay"],
["address","smtp.fastmail.com"],
["port","465"],
["tls.implicit","true"],
["auth.username","[email protected]"],
["auth.secret","<fastmail-app-password>"]
]}]'
</syntaxhighlight>
5. SDF Relay Requires VPS Python Proxy (SDF CRAM-MD5 auth)
Problem: SDF mail server only supports CRAM-MD5 authentication, which Stalwart doesn't implement.
Solution: Python proxy on VPS (/opt/sdf-smtp-proxy.py, systemd: sdf-smtp-proxy.service)
- Listens on Tailscale IP
100.67.169.50:2587(no auth required) - Relays to
mx.sdf.org:587with CRAM-MD5 auth - Credentials stored in VPS
/opt/secrets.env
Stalwart config for SDF: <syntaxhighlight lang="bash">
- Set /etc/hosts mapping on stronghold
100.67.169.50 sdf-relay.internal
- Then add route
curl -s -u admin:<REDACTED> http://localhost:8181/api/settings -H "Content-Type: application/json" \
-d '[{"type":"insert","prefix":"queue.route.sdf-relay","assert_empty":false,"values":[
["type","relay"],
["address","sdf-relay.internal"],
["port","2587"],
["tls.implicit","false"],
["tls.starttls","true"]
]}]'
</syntaxhighlight>
Also, config.toml must set resolver.type = "system" (not "default") so it reads /etc/hosts:
<syntaxhighlight lang="toml"> [resolver] type = "system" # NOT "default" </syntaxhighlight>
6. Routing Strategy: Local Recipients FIRST (Rule 0000)
Problem: If rule 0000 doesn't check for local recipients, mail to [email protected] gets sent through an external relay and fails.
Correct strategy (rules 0000-0002):
<syntaxhighlight lang="bash"> curl -s -u admin:<REDACTED> http://localhost:8181/api/settings -H "Content-Type: application/json" \
-d '[{"type":"insert","prefix":"queue.strategy.route","assert_empty":false,"values":[
["0000.if","is_local_domain('\'\'\, rcpt_domain)"],
["0000.then","'\local'\"],
["0001.if","sender_domain == '\wilsoz.com'\"],
["0001.then","'\wilsoz-fastmail'\"],
["0002.else","'\mx'\"]
]}]'
</syntaxhighlight>
Lesson: ALWAYS put local-domain check as rule 0000 (executed first).
7. LDAP Authentication Requires STARTTLS on Port 143
Wrong: Try to use LOGIN auth on plain port 143
<syntaxhighlight lang="text">
imap.login('marcus', 'password') # Port 143
- Returns: AUTHENTICATIONFAILED
</syntaxhighlight>
Right: Use STARTTLS on port 143, or implicit TLS on port 993 <syntaxhighlight lang="text"> imap.starttls() # Upgrade to TLS imap.login('marcus', 'password') # Now LOGIN is available </syntaxhighlight>
This is a Stalwart security feature — LOGIN auth is only advertised after TLS negotiation (STARTTLS or implicit).
8. ManageSieve (Sieve Rules) Authentication Quirk
Problem: Python managesieve library calls login() before starttls(). After starttls(), capabilities change to include PLAIN. Library doesn't re-read capabilities, so login fails.
Solution: Use raw sockets (see "ManageSieve" section below in Operations).
9. Admin Impersonation via IMAP (May Not Work)
Syntax: admin*username@domain with admin password
<syntaxhighlight lang="text">
imap.login('admin*[email protected]', '<REDACTED - see OpenBao secret/stalwart>')
</syntaxhighlight>
Status: As of 2026-02-22, this returned AUTHENTICATIONFAILED. May need:
[imap] allow-plain-authconfig setting enabled- Or different impersonation mechanism
Workaround: Use LDAP directly or set user passwords and log in normally.
10. TLS Certificate Path Must Exist at Startup
Problem: If you mount /etc/letsencrypt:/opt/letsencrypt:ro but cert files don't exist, Stalwart refuses to start.
Solution: Before starting, verify: <syntaxhighlight lang="bash"> ls -l /etc/letsencrypt/live/mymail.wilsoz.com/
- fullchain.pem and privkey.pem must exist
- If they don't, run certbot
sudo certbot certonly -d mymail.wilsoz.com --dns-cloudflare </syntaxhighlight>
11. Outbound Rate Limiter Example (MXRoute 300/hr)
MXRoute has a rate limit of ~300 emails/hour. Configure in Stalwart:
<syntaxhighlight lang="bash"> curl -s -u admin:<REDACTED> http://localhost:8181/api/settings -H "Content-Type: application/json" \
-d '[{"type":"insert","prefix":"queue.limiter.outbound.mxroute","assert_empty":false,"values":[
["enable","true"],
["match","mx == '\blizzard.mxrouting.net'\"],
["key.0000","mx"],
["rate","300/1h"]
]}]'
</syntaxhighlight>
12. LDAP Password Storage (Plaintext in .env)
The LDAP_SERVICE_PASSWORD in .env is stored plaintext. It's the password for cn=stalwart-svc (service account), which:
- Has limited permissions (ldap-search group only)
- Doesn't have login access
- Is used only for directory queries
Store securely; if compromised, change in Authentik and update .env.
Verification Checklist
After recovery, verify:
- [ ] Container running:
docker ps | grep stalwart-mail - [ ] Listeners bound:
```bash docker logs stalwart-mail | grep -i "listening\|bound" # Should show SMTP:25, SMTP:465, IMAP:143, IMAP:993, etc. ```
- [ ] Admin API responding:
```bash curl -u admin:<REDACTED - see OpenBao secret/stalwart> http://localhost:8181/api/principal | jq '.principals | length' # Should return number of principals ```
- [ ] LDAP connected:
```bash docker logs stalwart-mail | grep -i "ldap\|directory" # Should show successful connections, not errors ```
- [ ] TLS certificate loaded:
```bash docker logs stalwart-mail | grep -i "certificate\|tls" # Should show cert paths loaded ```
- [ ] IMAP login works:
```bash
python3 -c "
import imaplib, ssl
ctx = ssl.create_default_context(); ctx.check_hostname=False; ctx.verify_mode=ssl.CERT_NONE
m = imaplib.IMAP4_SSL('localhost', 993)
status, data = m.login('marcus', '<password>')
print('IMAP login:', 'OK' if status == 'OK' else f'FAILED: {status}')
m.close()
"
```
- [ ] Relay routes configured:
```bash curl -s -u admin:<REDACTED - see OpenBao secret/stalwart> http://localhost:8181/api/settings/list?prefix=queue.route | jq '.settings | length' # Should return > 0 (number of configured routes) ```
- [ ] Routing strategy configured:
```bash curl -s -u admin:<REDACTED - see OpenBao secret/stalwart> http://localhost:8181/api/settings/list?prefix=queue.strategy | jq '.settings | length' # Should return > 0 (routing rules) ```
- [ ] Send test email:
```bash
# Use eM Client or swaks command-line tool
swaks --to [email protected] --from [email protected] --server localhost:465 --tls
# Should deliver without errors
```
- [ ] Receive test email:
- Send email from external account to [email protected]
- Check it appears in IMAP inbox
- [ ] Sieve rules executing:
```bash # If you have sieve rules, send test email and verify filtering works docker logs stalwart-mail | grep -i "sieve\|filter" ```
Dependencies
What Stalwart Depends On
- Authentik LDAP outpost — for user authentication (localhost:389)
- TLS certificate — from Let's Encrypt (/etc/letsencrypt)
- Network access to external relays:
- Fastmail: smtp.fastmail.com:465
- MXRoute: blizzard.mxrouting.net:465
- Gmail: smtp.gmail.com:465
- iCloud: smtp.mail.me.com:465
- SDF: sdf-relay.internal:2587 (via proxy)
- DNS resolution — for MX lookups and relay host resolution
- getmail — for inbound mail federation (craniumslows.com forwarding)
What Depends On Stalwart
- SnappyMail — web mail interface (frontend to Stalwart IMAP)
- eM Client — desktop mail client (uses IMAP/SMTP)
- getmail — mail federation from external accounts
- n8n workflows — email provisioning/deprovisioning
- Mail clients — Thunderbird, Outlook, etc.
Operations
Start/Stop/Restart
<syntaxhighlight lang="bash"> cd /library/stalwart-app
- Start
docker compose up -d
- Stop
docker compose down
- Restart
docker compose restart stalwart-mail
- Full restart (clears cache)
docker compose down && sleep 2 && docker compose up -d </syntaxhighlight>
View Logs
<syntaxhighlight lang="bash">
- Real-time logs
docker logs -f stalwart-mail
- Last 50 lines
docker logs stalwart-mail --tail=50
- Filter for errors
docker logs stalwart-mail 2>&1 | grep -i "error\|failed\|fatal"
- Filter for specific component
docker logs stalwart-mail | grep -i "ldap\|imap\|smtp\|queue" </syntaxhighlight>
Admin API Queries
<syntaxhighlight lang="bash"> ADMIN="admin:<REDACTED - see OpenBao secret/stalwart>" BASE="http://localhost:8181/api"
- List all principals (accounts, groups)
curl -s -u "$ADMIN" "$BASE/principal"
- Get specific principal
curl -s -u "$ADMIN" "$BASE/principal/[email protected]"
- Reload config from disk (WARNING: see gotcha #1)
curl -s -u "$ADMIN" "$BASE/reload"
- List routes
curl -s -u "$ADMIN" "$BASE/settings/list?prefix=queue.route"
- List strategy rules
curl -s -u "$ADMIN" "$BASE/settings/list?prefix=queue.strategy"
- List spam settings
curl -s -u "$ADMIN" "$BASE/settings/list?prefix=spam-filter"
- Count messages in queue
curl -s -u "$ADMIN" "$BASE/queue" | jq '.messages | length'
- View a specific message in queue
curl -s -u "$ADMIN" "$BASE/queue/<message-id>" </syntaxhighlight>
Manage Sieve Rules (ManageSieve port 4190)
<syntaxhighlight lang="bash">
- Upload sieve script (using helper script)
cd /home/mnw/Developer/SystemResiliency python3 sieve/upload-sieve.py
- Or use raw socket method (see Disaster Mode B section)
</syntaxhighlight>
Troubleshooting
Can't Log In via IMAP
Check LDAP: <syntaxhighlight lang="bash"> ldapsearch -x -H ldap://localhost:389 \
-D "cn=stalwart-svc,ou=users,dc=ldap,dc=goauthentik,dc=io" \ -w "<rotated 2026-05-18; see OpenBao secret/stalwart/ldap-bind-password>" \ -b "ou=users,dc=ldap,dc=goauthentik,dc=io" \ "cn=marcus" mail
</syntaxhighlight>
If LDAP works but IMAP login fails:
- Verify using port 993 (TLS) not 143 (plain)
- Or use STARTTLS on port 143
- Check if user is in
wilsoz-emailgroup (not required for LDAP, but may be required by Stalwart)
Admin impersonation not working:
- Try normal login instead:
imap.login('marcus', '<password>') - Impersonation may need config flag changes
Relay Routes Not Working
Debug: <syntaxhighlight lang="bash">
- Check routes exist
curl -s -u admin:<REDACTED> http://localhost:8181/api/settings/list?prefix=queue.route
- Check strategy rules
curl -s -u admin:<REDACTED> http://localhost:8181/api/settings/list?prefix=queue.strategy
- Check queue for stuck messages
curl -s -u admin:<REDACTED> http://localhost:8181/api/queue | jq '.messages[]'
- Check logs for relay errors
docker logs stalwart-mail | grep -i "relay\|route\|fastmail\|mxroute" </syntaxhighlight>
Common causes:
- Route not configured (blank response from API)
- Strategy rule missing or malformed (0000 not checking local domain)
- TLS cert not valid (relay host rejects connection)
- Rate limit exceeded (queue backs up, check MXRoute limit)
- Port 465 blocked (ISP still blocking? test external relay directly)
Database Corruption
Symptoms: "SQLite error" in logs, or can't connect to accounts
Fix: <syntaxhighlight lang="bash"> cd /library/stalwart-app docker compose down
- Backup corrupted database
cp data/store/data.sqlite data/store/data.sqlite.corrupted-$(date +%Y%m%d)
- Delete corrupted DBs (queue/messages preserved)
rm data/store/*.sqlite
- Start fresh (will recreate empty DBs)
docker compose up -d
- Reconfigure via API
</syntaxhighlight>
Outbound Queue Stuck
Check queue: <syntaxhighlight lang="bash"> curl -s -u admin:<REDACTED> http://localhost:8181/api/queue | jq '.messages | length'
- If > 100 messages stuck, investigate:
docker logs stalwart-mail | tail -100 | grep -i "reject\|error\|fail"
- Possible causes:
- - Rate limit hit (check MXRoute limit)
- - TLS cert issue with relay host
- - Firewall blocking outbound to relay
- - LDAP down (affects routing decision)
</syntaxhighlight>
Spam Filter Too Aggressive
Increase spam threshold: <syntaxhighlight lang="bash"> curl -s -u admin:<REDACTED> -X POST http://localhost:8181/api/settings \
-H "Content-Type: application/json" \
-d '[{"type":"insert","prefix":"spam-filter.score","assert_empty":false,"values":"spam","6.0"}]'
- Verify
curl -s -u admin:<REDACTED> http://localhost:8181/api/settings/list?prefix=spam-filter </syntaxhighlight>
Links & References
- Stalwart Docs: https://stalwartlabs.com/docs/
- Stalwart Admin API: http://localhost:8181/admin (Web UI)
- Stalwart Email Blog: https://stalwartlabs.com/blog/
- LDAP Documentation: https://ldapwiki.com/
- Sieve Mail Filters: https://tools.ietf.org/html/rfc5228
- Getmail Integration:
/home/mnw/Developer/SystemResiliency/services/STALWART.md(reference docs)
Related Services
- Authentik (auth):
/library/authentik-app/— LDAP provider, user directory - SnappyMail (webmail):
/library/snappymail-app/— web mail frontend - getmail (mail federation):
/library/email/getmail/— inbound mail syncing - Let's Encrypt (TLS):
/etc/letsencrypt/— certificates for TLS - VPS nginx (reverse proxy): Exposes mymail.wilsoz.com:993 externally
- Tailscale (VPN): Required for SDF relay via proxy