HanyanOS 部署手记

发表于 分类于

1. Introduction

Email is one of the hardest services to self-host. Port 25 blocking by cloud providers, IP reputation management, SPF/DKIM/DMARC configuration, and the inherent complexity of the SMTP/IMAP stack make it a minefield. For HanyanOS, running a full mail stack on an N100 behind NAT (192.168.1.18) with a single public IPv4 via AWS Lightsail added unique constraints.

This post documents the complete mail pipeline: Stalwart Mail Server (v0.16.5) for MTA/IMAP, SnappyMail as the webmail frontend, AWS SES as the outbound relay, and FRP tunnels bridging the VPS edge to the internal N100.

2. Architecture Overview

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
┌─ Internet ─────────────────────────────────────┐
│  MX: mail.chenyun.org → 52.220.247.252        │
└──────────────────────┬─────────────────────────┘
                       │
┌─ VPS (Lightsail SG) ───────────────────────────┐
│  Stream (443 SNI routing)      :25/:465/:993   │
│       │                              │          │
│  Nginx HTTPS :8443           FRPS tunnel gateway│
│       │                      7446/7447/7448     │
└───────┼──────────────────────────┬──────────────┘
        │                          │
┌─ FRP Tunnel ─────────────────────┘──────────────┐
│  VPS:7446  ←→  N100:25   (SMTP inbound)         │
│  VPS:7447  ←→  N100:465  (SMTP submission)      │
│  VPS:7448  ←→  N100:993  (IMAP over SSL)        │
│  VPS:7449  ←→  N100:8091 (SnappyMail webmail)   │
│  VPS:7450  ←→  N100:587  (SES relay outbound)   │
│  VPS:7451  ←→  N100:8443 (Stalwart admin)       │
└──────────────────────────────────────────────────┘
                        │
┌─ N100 (AiCore 192.168.1.18) ────────────────────┐
│                                                  │
│  ┌──────────┐    ┌────────────┐                  │
│  │ Stalwart │◄───│ SnappyMail │   Docker         │
│  │ :25/465  │    │ :8091      │   Compose        │
│  │ :993/587 │    │            │                  │
│  └────┬─────┘    └────────────┘                  │
│       │                                           │
│  ┌────▼─────┐                                    │
│  │ RocksDB  │  (embedded store)                  │
│  └──────────┘                                    │
│       │                                           │
│  ┌────▼─────┐                                    │
│  │ AWS SES  │  Outbound relay :587 (STARTTLS)    │
│  └──────────┘                                    │
└──────────────────────────────────────────────────┘

Key Design Decisions

Decision Rationale
Stalwart over Mailcow Single binary, no Postfix/Dovecot complexity, RocksDB-backed config, modern Rust codebase
SES relay outbound Lightsail blocks port 25 outbound — SES provides reputable SMTP relay with 62K free emails/month
FRP over Tailscale Zero client configuration; mobile devices connect directly to mail.chenyun.org without VPN
SnappyMail over RainLoop Lighter weight, actively maintained, simpler Docker deployment

3. Docker Compose Deployment

The entire mail stack runs in two containers orchestrated by a single docker-compose.yml:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
# ~/mail-server/docker-compose.yml
services:
  stalwart:
    image: stalwartlabs/stalwart:latest
    container_name: stalwart-mail
    restart: unless-stopped
    ports:
      - "25:25"
      - "465:465"
      - "587:587"
      - "143:143"
      - "993:993"
      - "127.0.0.1:8080:8080"
      - "4190:4190"
      - "127.0.0.1:8443:443"
    environment:
      - STALWART_HOSTNAME=mail.chenyun.org
      - STALWART_PUBLIC_URL=https://mail.chenyun.org
    volumes:
      - ./etc:/etc/stalwart
      - ./data:/var/lib/stalwart
      - /etc/ssl/chenyun:/etc/ssl/chenyun:ro

  snappymail:
    image: djmaze/snappymail:latest
    container_name: snappymail
    restart: unless-stopped
    ports:
      - "127.0.0.1:8091:8888"
    volumes:
      - ./snappy-data:/var/lib/snappymail
    environment:
      - TZ=Australia/Brisbane
    depends_on:
      - stalwart

Notable details:

  • Port 143 (IMAP plain) is exposed locally but blocked at UFW — only IMAPS (:993) reaches the internet
  • Port 4190 (Sieve manage sieve) is LAN-only
  • Ports 8080/8443 bound to 127.0.0.1 exclusively — admin panel not exposed externally
  • SSL certificates from /etc/ssl/chenyun/ are bind-mounted read-only

4. FRP Tunnel Configuration

The VPS runs frps as the tunnel gateway; the N100 runs frpc to establish persistent encrypted connections. The mail-specific tunnel entries:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
# /etc/frp/frpc.ini (mail section)
[mail-smtp]
type = tcp
local_ip = 127.0.0.1
local_port = 25
remote_port = 7446

[mail-submission]
type = tcp
local_ip = 127.0.0.1
local_port = 465
remote_port = 7447

[mail-imaps]
type = tcp
local_ip = 127.0.0.1
local_port = 993
remote_port = 7448

[mail-webmail]
type = tcp
local_ip = 127.0.0.1
local_port = 8091
remote_port = 7449

[mail-submission-587]
type = tcp
local_ip = 127.0.0.1
local_port = 587
remote_port = 7450

[stalwart-admin]
type = tcp
local_ip = 127.0.0.1
local_port = 8443
remote_port = 7451

On the VPS side, frps listens on the corresponding remote ports and the Nginx stream layer routes public :25, :465, and :993 directly to the FRP ports.

5. Outbound Delivery via AWS SES

AWS Lightsail blocks outbound port 25 by default. Deploying a traditional MTA-to-MTA outbound path was impossible. The solution: route all outbound email through AWS SES via port 587 (STARTTLS).

Configuration in Stalwart’s admin panel:

1
2
3
4
5
Outbound Relay:
  - Host: email-smtp.ap-southeast-1.amazonaws.com
  - Port: 587 (STARTTLS)
  - Auth: SMTP credentials (IAM user with ses:SendRawEmail)
  - Rate limit: 14 emails/second (SES default)

The SES sending limits (62,000 emails/month in the free tier) are more than sufficient for a personal mail server. SPF, DKIM, and DMARC records were added to the Route53 zone to maximize deliverability:

1
2
3
TXT  @      "v=spf1 include:amazonses.com ~all"
TXT  *._domainkey "v=DKIM1; p=<SES public key>"
TXT  _dmarc  "v=DMARC1; p=quarantine; rua=mailto:postmaster@chenyun.org"

6. Problems Encountered and Resolutions

6.1 AWS Port 25 Block (Critical)

Symptom: All outbound emails were silently dropped. No error in Stalwart logs, but recipients never received delivery.

Root Cause: Lightsail instances block outbound TCP 25 by default per AWS abuse policy.

Resolution: Opened an AWS support ticket requesting port 25 unblock, and simultaneously configured SES relay on port 587 as the primary outbound path. The SES route proved more reliable than direct MTA delivery and eliminated IP reputation concerns.

6.2 FRP Port Mapping Drift

Symptom: After multiple configuration iterations (docker-mailserver → Stalwart migration), the FRP port mappings diverged from the Nginx stream configuration on the VPS. Webmail worked, but IMAP connections timed out.

Resolution: Created a single source of truth — a fixed mapping table documented in HanyanOS/memory/infrastructure/network/README.md. Every FRP change must update the table before deployment.

6.3 Stalwart Route Name Collision

Symptom: After deleting an outbound route named “ses” and recreating it with the same name, Stalwart served stale cached configuration — the new route’s SMTP credentials were ignored.

Root Cause: Stalwart’s RocksDB-backed config cache retains route entries by name. Deleting and recreating with the same name does not invalidate the internal references.

Resolution: Created the new route as aws-ses instead of ses. After the name change, the configuration applied correctly.

6.4 Scroll Wheel Port Mutation

Symptom: During configuration, the submission port spontaneously changed from 587 to 586 in the admin UI.

Root Cause: Stalwart’s HTML5 admin interface uses number input fields. A mouse scroll hover over the port field while the browser window was unfocused caused an unintended increment.

Resolution: Always verify port values before saving. This is a UI ergonomics issue the Stalwart team should address.

6.5 Migration from docker-mailserver

Symptom: The previous mail stack (docker-mailserver + RainLoop) left stale Docker volumes, conflicting port bindings, and a defunct Docker network (email_default).

Resolution: Performed a complete teardown:

1
2
3
4
docker compose -f ~/mail/docker-compose.yml down -v
docker volume prune -f
docker network rm email_default
# Verify ports 25/143/465/587/993 are free

Then deployed the new stack from scratch. All old mail data was discarded (no migration needed — personal mail server with no critical historical data).

7. Security Posture

Layer Measure
Transport TLS 1.3 on all public ports (25/465/587/993)
Authentication SCRAM-SHA-256 for IMAP; LOGIN/PLAIN disabled
Firewall UFW on N100: only 25/465/587/993 allowed from VPS subnet
Rate Limiting Stalwart built-in: 50 connections/minute per IP
Anti-spam Stalwart’s RBL + SPF/DKIM/DMARC verification
Admin isolation Stalwart admin panel on 127.0.0.1:8443 only; no public route
Certificate Let’s Encrypt via acme.sh; auto-renewed every 60 days

8. Operational Notes

  • Status monitoring: Mail health is tracked as part of the HanyanOS system status dashboard, with automatic restart via Docker’s restart: unless-stopped policy
  • Backup: The ./data directory (RocksDB store) is included in the nightly HanyanOS backup rotation
  • Logging: Stalwart logs to Docker’s journald; log rotation is handled by the host’s logrotate for /var/lib/docker/containers/*/*.log
  • Performance: On N100, Stalwart consumes ~120MB RAM at idle, rising to ~250MB under load. CPU usage is negligible for personal email volumes (<100 messages/day)

9. Conclusion

Self-hosting email on an N100 behind NAT is entirely feasible with the right relay strategy. The combination of Stalwart (lightweight, Rust-based MTA/IMAP), AWS SES (reputable outbound relay), and FRP tunnels (NAT traversal) provides a reliable, low-maintenance mail infrastructure. The key lesson: never fight against cloud provider port blocking — use a relay service and redirect your engineering effort to monitoring and security hardening instead.

Next in series: WordPress + MySQL — Dockerized Blog Deployment on N100.