Quartz 4

design.prev

9 min read

SigenStor Control — Design

Single-user home-energy controller (Mick). k8s-hosted. Monitors + controls one Sigenergy SigenStor (battery + PV) using local Modbus TCP primary, Amber Electric for price/forecast, and a solar-forecast provider (Solcast preferred, Open-Meteo fallback).

Stack

  • Language: Go 1.22 (single static binary, low idle CPU, easy k8s deploy, mature modbus libs).
  • Modbus: github.com/whiteblock/go-modbus or github.com/grid-x/modbus (TCP, holding/input registers, FC6 write).
  • HTTP/JSON: stdlib net/http + encoding/json. No web framework for the API server; use chi for routing (small, stdlib-shaped).
  • DB: SQLite via modernc.org/sqlite (CGO-free). WAL mode. One file under /var/lib/sigenstor/sigenstor.db.
  • Time-series: keep last 30 days at 30-min resolution in DB tables; chart endpoints aggregate on the fly. No Prometheus — overkill for one site.
  • UI: server-rendered HTML via html/template + htmx + Alpine.js + Chart.js (CDN). One Go binary serves UI + JSON API. No SPA.
  • Auth: single user; HTTP Basic + bcrypt hash in env/secret. CSRF token on state-changing forms. Cookie session (signed).
  • Container: distroless gcr.io/distroless/static-debian12, multi-stage Dockerfile. Helm chart.
  • Logging: log/slog JSON to stdout. Metrics: Prometheus /metrics endpoint (process + custom). Traces: none.

Architecture (one process, three internal layers)

  • poller — fixed-tick goroutines: SigenStor every 5s, Amber every 60s, Solcast every 30min, price forecast refresh every 5min. All write to DB and an in-memory state struct.
  • controller — runs every 60s. Reads state + DB (schedules, mode, manual override, prices, forecast, SoC). Decides target charge/discharge kW and EMS mode register writes. Idempotent.
  • api — HTTP server: JSON /api/v1/... for programmatic access, HTML / for UI, /metrics for Prometheus.

Data model (SQLite, single file)

  • telemetry (ts INT PK, soc_pct REAL, bat_kw REAL, pv_kw REAL, load_kw REAL, grid_kw REAL, mode TEXT, online INT) — append-only, partitioned by retention job (drop >30d).
  • prices (ts INT PK, buy_c_per_kwh REAL, feed_in_c_per_kwh REAL, source TEXT, fetched_at INT) — 30-min slots, 24h+ horizon.
  • forecast_solar (ts INT PK, kwh REAL, source TEXT) — 30-min slots, 7d horizon.
  • forecast_price (ts INT PK, predicted_c_per_kwh REAL, interval_type TEXT) — from Amber.
  • schedules (id, name, days_mask, start_min, end_min, action TEXT, kw REAL, priority INT, enabled INT) — manual time-of-day rules.
  • mode_state (key TEXT PK, value TEXT) — current mode, manual override expiry, last decision reason, controller enabled flag.
  • events (ts, kind, payload TEXT) — mode changes, override set/clear, errors, Modbus write failures. 90d retention.
  • kv (key TEXT PK, value TEXT) — amber api token (encrypted at rest with key from k8s secret), solcast api key, modbus target, modbus unit id, modbus poll interval.

Sigenergy Modbus integration

  • Reference impl: Home Assistant sigenergy integration (https://github.com/ypengju/homeassistant-sigen). Reuse its register map verbatim (do not reinvent — verified against real firmware).
  • Transport: Modbus TCP on port 502 (or 5020/TCP — confirm during commissioning; default 502). Read holding registers for telemetry, write holding registers for control. Unit ID = 1 default.
  • Key registers (HA sigenergy integration maps; verify exact addresses at deploy):
    • Read: device master EMS status, SoC, battery power, PV power, load, grid power, energy totals.
    • Write: EMS mode (self-consume / forced charge / forced discharge / remote schedule), forced charge/discharge power limit, remote schedule enable/disable.
  • Auth: SigenStor local Modbus requires the installer/admin password for write operations (HA integration prompts). Store in k8s secret; pass to Modbus client on each write session (some firmware require re-auth per session).
  • Failure handling: on read error, mark last value stale, surface banner in UI. On write error, exponential backoff 1/2/4/8s, max 3 attempts, then raise event + revert to safe mode (self-consume). Never leave battery in forced-charge indefinitely.
  • Polling: 5s reads in a single goroutine; writes are serialized through a writeCh channel (one writer, FIFO) — avoids interleaved register updates.
  • Discovery: hardcode IP via k8s ConfigMap (sigenergy.local as env var SIGEN_HOST). Allow override from kv.

Amber Electric

  • Base URL: https://api.amber.com.au/v1.
  • Auth: bearer token from k8s secret. Endpoints:
    • GET /sites — list site IDs (resolve once, cache to kv).
    • GET /sites/{id}/prices/current — current 30-min interval + next interval.
    • GET /sites/{id}/prices/forecasts — predicted intervals (next 24h).
    • GET /sites/{id}/prices/next — paginated upcoming.
  • Store both buy (perKwh, c/kWh × 100) and feed-in (feedInPerKwh) at 30-min granularity. Use general price type (controlled load ignored — Mick has none relevant).
  • Refresh: current every 60s, forecast every 5min. Cache 5s in-memory to absorb burst UI reads.

Solar forecast

  • Primary: Solcast (free tier, 50 API calls/day, 30-min resolution, 7-day horizon). Use rooftop estimate if both sites configured.
  • Fallback: Open-Meteo (no key, 11-day horizon, less accurate for cloud).
  • Refresh: every 30min between 04:00–22:00 local, every 2h overnight. Cache to forecast_solar (UPSERT).
  • Use rolling sum of next 4h PV estimate vs current SoC to decide pre-charging headroom.

Controller logic (every 60s)

Inputs: current SoC, current prices, next-12h price curve, next-24h solar curve, schedules, manual override, controller-enabled flag, safety bounds (SoC min 10%, max 95%, max charge/discharge C-rate).

Decision order (first match wins, write once):

  1. Safety: if SoC ≤ 10% → force charge at max rate (capped) regardless of price.
  2. Safety: if SoC ≥ 95% → force discharge at capped rate.
  3. Manual override: if active and not expired → honor override target (charge kW, discharge kW, or self-consume). Expiry on mode_state.manual_override_until.
  4. Manual schedule: if a schedule row covers now AND enabled AND priority highest → execute (charge, discharge, or self-consume).
  5. Auto PRICE mode:
    • Compute next-12h average buy price vs current feed-in.
    • If cheap window starting in ≤60min AND SoC < 80% → charge at max.
    • If expensive window now AND SoC > 30% → discharge at capped kW (do not exceed house load + export cap).
    • Otherwise: self-consume (EMS default mode).
  6. Auto WEATHER mode: if tomorrow’s solar forecast < 2 kWh AND tonight’s prices low → pre-charge tonight; else self-consume.
  7. Auto USAGE mode: learn daily baseline load from last 14 days (kWh). Top-up to target SoC before predicted high-load evening window.
  8. Default: self-consume.

Always write a single coherent EMS-mode register update per tick. Log decision reason to events.

UI (htmx + Alpine + Chart.js)

Pages:

  • / Dashboard — current SoC gauge, PV/load/grid/battery kW tiles, current Amber buy + sell price + next interval change countdown, 24h price+SoC+PV chart, mode indicator, manual override buttons.
  • /schedules — CRUD list of manual time-of-day rules, day-of-week mask, action, kW, priority.
  • /automations — toggle PRICE / WEATHER / USAGE modes, threshold sliders (price spread, SoC bounds), enable/disable controller.
  • /history — 7/30-day charts: SoC, energy in/out, self-consumption ratio, $ saved estimate.
  • /settings — Modbus host, Amber token, Solcast key, manual override (set + expiry), backup/restore DB.
  • /events — paginated log of mode changes, errors, writes.

UX rules:

  • Big mode badge (AUTO / MANUAL / SCHEDULE / SAFETY) always visible.
  • One-click manual override: “Charge now 30 min”, “Discharge now 30 min”, “Self-consume 2 h”. Confirmation not required.
  • Banner on Modbus stale >60s, Amber token expired, forecast stale >2h.
  • All charts use 30-min resolution.

API (JSON)

  • GET /api/v1/state — current snapshot.
  • GET /api/v1/telemetry?from=&to=&bucket= — historic.
  • GET /api/v1/prices?from=&to=
  • GET /api/v1/forecast/solar?from=&to=
  • GET /api/v1/schedules / POST / PATCH /:id / DELETE /:id
  • POST /api/v1/mode — {mode: auto|manual|self_consume, action?: charge|discharge, kw?: float, until?: iso8601}
  • POST /api/v1/override — set manual override with expiry.
  • DELETE /api/v1/override — clear override.
  • GET /api/v1/events?from=&to=
  • GET /metrics — Prometheus.

All POST/DELETE require CSRF token + basic-auth session.

k8s deployment

  • Namespace: sigenstor.
  • Helm chart sigenstor-control (single Deployment, 1 replica, no HPA — single instance owns Modbus writes).
  • Resources: requests 50m / 64Mi, limits 500m / 256Mi (Go binary, idle ~30Mi).
  • Mounts: PVC sigenstor-data 1Gi RWO (sqlite file, logs optional).
  • Secrets: sigenstor-creds (amber token, solcast key, sigenergy modbus password, basic-auth htpasswd, sqlite-encryption key, session key).
  • ConfigMap: sigenstor-config (non-secret env: SIGEN_HOST, POLL_INTERVAL, DB_PATH, LOG_LEVEL, LISTEN_ADDR=:8080).
  • Service: ClusterIP on 8080. Ingress via existing nginx ingress (TLS via cert-manager, host sigenstor.home.<domain>).
  • Probes: liveness = GET /healthz (200 if DB pingable + last SigenStor poll <60s); readiness = same minus modbus check (allow degraded reads).
  • PDB: min 1, max 1.
  • Network: no egress restrictions to api.amber.com.au, api.solcast.com, api.open-meteo.com. Egress to SigenStor LAN IP only (NetworkPolicy egress to LAN CIDR).
  • Backup: nightly cronjob runs sqlite3 .backup /var/backups/sigenstor-$(date).db, retains 14, pushes to existing S3-compatible bucket via restic.

Security

  • All secrets via k8s secret, env-injected, never logged.
  • Modbus write password stored encrypted (AES-GCM with key from secret), decrypted in-memory only.
  • HTTPS only at ingress; HTTP→HTTPS redirect.
  • CSP: default-src ‘self’; connect-src ‘self’ api.amber.com.au api.solcast.com; chart.js + alpine.js vendored under /static (no CDN).
  • No external links from UI.

Observability

  • Prometheus scrape /metrics. Key series: sigenstor_soc_percent, sigenstor_battery_kw, sigenstor_pv_kw, sigenstor_grid_kw, sigenstor_amber_buy_cents, sigenstor_amber_feed_in_cents, sigenstor_controller_mode, sigenstor_modbus_errors_total, sigenstor_decision_duration_seconds.
  • Log lines structured JSON; INFO normal, WARN stale data / override, ERROR write failure / token expired.
  • Grafana dashboard: 4 panels (overview, prices, decisions, errors). Pre-built JSON committed.

Failure modes + safety

  • Modbus unreachable >120s → controller enters safe self-consume, banner alerts, no writes attempted.
  • Amber token 401 → cache last known prices, disable PRICE auto mode, banner alerts.
  • Solcast/Open-Meteo both fail >2h → disable WEATHER auto mode, PRICE+USAGE still work.
  • DB write failure → controller pauses, logs ERROR, banner alerts.
  • k8s pod restart → controller reads mode_state, re-asserts last mode to SigenStor on startup (idempotent).
  • Battery temperature/fault registers (read from HA register map) → if fault bit set, controller forces self-consume and alerts.
  • All writes idempotent; safe to restart at any time.

Testing strategy

  • Unit: controller decision tree with synthetic inputs (table-driven).
  • Integration: docker-compose stack with modbus-server (python pymodbus fake), fake amber/solcast HTTP servers. Run full controller tick against fixtures.
  • E2e manual: deploy to k8s, point at real SigenStor, verify dashboard + mode changes for one full diurnal cycle.
  • Regression: golden-file test for decision output given canned input.

Out of scope (v1)

  • Multi-site / multi-user.
  • VPP / grid services.
  • EV charger integration.
  • Home Assistant add-on packaging (HA is reference only).
  • Mobile app (UI is mobile-responsive).
  • MPPT / per-string PV telemetry (SigenStor exposes only totals).

Risks

  • Sigenergy firmware Modbus register changes between versions → pin firmware in docs, document any upgrade re-verification steps.
  • Amber rate limits (low, but cache + respect 429).
  • Solcast free tier 50 calls/day — current polling budget ~50/day fits.
  • SQLite single-writer — fine for one process; if scaled out later, switch to Postgres (schema portable).

Key files to be produced

  • cmd/sigenstor/main.go — wire everything.
  • internal/modbus/client.go — TCP client + register map (port of HA register constants).
  • internal/amber/client.go — REST client.
  • internal/solcast/client.go — REST client.
  • internal/controller/controller.go — decision tree.
  • internal/store/sqlite.go — DB layer.
  • internal/api/server.go — chi routes + htmx partials.
  • internal/api/templates/ — html templates.
  • internal/api/static/ — vendored alpine.js, chart.js, css.
  • deploy/Dockerfile, deploy/helm/... chart, deploy/grafana/dashboard.json.
  • README.md — register map source + commissioning steps.

Decisions summary (locked)

  • Local Modbus primary, no cloud mySigen dependency.
  • Go single binary, distroless.
  • SQLite single-file DB, no external DB.
  • htmx server-rendered UI, no SPA.
  • Single replica, no HPA.
  • PRICE / WEATHER / USAGE modes composable; safety overrides all.
  • Decision logged every tick for auditability.
  • Controller is idempotent and restart-safe.

Graph View