vergil_on/go-synapse-backupper
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Рыба проекта. Минимальная функциональность

Browse Source
This commit is contained in:
vergil_on
2026-08-03 22:22:24 +03:00
commit 8c8631ac9c
80 changed files with 10618 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agents/skills/chore/README.md
+11
View File
@@ -0,0 +1,11 @@
# Chore Skills
> **Русская версия:** [README.ru.md](README.ru.md)
Infrastructure and utility skills applicable across projects.
## Available Skills
| Skill | Description |
|-------|-------------|
| [dockerfile-skill](dockerfile-skill) | Best practices for writing production-ready Dockerfiles: multistage builds, BuildKit parallelism, security hardening (non-root user, pinned base images, no secrets in layers), healthchecks, custom entrypoints, `.dockerignore`, and integration with Makefile / Docker Compose. Includes a self-check checklist. |
.agents/skills/chore/README.ru.md
+12
View File
@@ -0,0 +1,12 @@
# Chore Skills
> **English version:** [README.md](README.md)
> **Вернуться к оглавлению:** [README.ru.md](../../README.ru.md)
Инфраструктурные и утилитарные скиллы, применимые во всех проектах.
## Доступные скиллы
| Скилл | Описание |
|-------|----------|
| [dockerfile-skill](dockerfile-skill) | Лучшие практики написания production-ready Dockerfile: многоступенчатая сборка, параллельные стейджи BuildKit, харднинг безопасности (непривилегированный пользователь, фиксированные версии образов, отсутствие секретов в слоях), healthchecks, кастомные entrypoint-скрипты, `.dockerignore` и интеграция с Makefile / Docker Compose. Включает чеклист самопроверки. |
.agents/skills/chore/dockerfile-skill/SKILL.md
+289
View File
@@ -0,0 +1,289 @@
---
name: dockerfile-skill
description: Ты ОБЯЗАН использовать этот скилл, если хочешь писать Dockerfile
---
## Multistage
Используй Docker multistage, если тебе надо создать Dockerfile для сборки и запуска приложения – раздели его на стейдж со сборкой и запуском.
Не стесняйся использовать `COPY FROM`.
Объединяй несколько команд `RUN` в одну строку с помощью `&&` и очищай временные файлы внутри того же слоя (например, `rm -rf /var/lib/apt/lists/*`). Это уменьшает число слоёв и размер финального образа.
Для зависимостей, которые часто пересобираются (pip, npm, apt), используй `--mount=type=cache,target=/root/.cache/pip` (или аналогично) – это ускоряет повторные сборки за счёт кэширования на хосте.
## Parallel
Разделяй стейджи Dockerfile таким образом, чтобы они могли выполняться параллельно (например, сборка зависимостей и подготовка тестовых данных).
При использовании BuildKit можно объявить несколько независимых `FROM` и копировать между ними через `COPY --from=...`. Убедись, что стейджи не имеют неявных зависимостей друг от друга.
## Makefile
Если в проекте есть Makefile, постарайся использовать его.
Если Makefile нет, сделай его и вызывай команды через таргеты, а не напрямую в Dockerfile.
Также добавь в этот Makefile таргеты для сборки и запуска текущего приложения в Docker.
## Compose
Если его еще нет, сделай возможность запустить приложение, для которого пишешь Dockerfile, в Docker Compose.
## Security
Используй лучшие практики безопасности при написании Dockerfile.
* **Фиксируй версии базовых образов** – всегда используй конкретный тег (например, `python:3.11-slim-bookworm`) вместо `:latest` или плавающих тегов, чтобы обеспечить воспроизводимость сборки.
* **Создавай выделенного непривилегированного пользователя** внутри образа: `RUN addgroup -S app && adduser -S -G app app`. Затем переключайся на него с помощью `USER app`. Копируй файлы командой `COPY --chown=app:app`, чтобы они сразу принадлежали этому пользователю, избегая лишних операций с правами.
* **Не храни секреты в переменных окружения образа** – используй `ARG` только для нечувствительных данных (например, версий пакетов), а секреты передавай через `--build-arg` в момент сборки или монтируй через Docker Secrets в рантайме.
* **Запускай контейнер rootless**, если это возможно. После создания пользователя и переключения на него финальный процесс будет работать без прав root.
* **Добавь `HEALTHCHECK`** (например, `HEALTHCHECK --interval=30s --timeout=3s CMD curl -f http://localhost:8080/ || exit 1`), чтобы оркестраторы могли корректно определять состояние приложения.
* **Проставляй метаданные через `LABEL`** – например, `LABEL maintainer="team@example.com"`, `LABEL version="1.0.0"`, `LABEL description="..."`. Это стандарт для документирования образов.
* **Указывай в Dockerfile флаги уменьшения размера зависимостей** – для `apt` используй `--no-install-recommends`, для `pip` – `--no-cache-dir`, для `npm` – `--only=production`, для `composer` – `--no-dev`.
* **Сканируй собранный образ на уязвимости** – в CI/CD используй инструменты вроде `trivy`, `docker scout` или `grype`. Это должно быть частью пайплайна сборки.
## Dockerignore
*Создай файл `.dockerignore` в корне проекта. Включи в него все ненужные файлы и директории: `.git`, `node_modules`, `venv`, `__pycache__`, `*.md`, `docker-compose.yml`, временные файлы. Это ускорит сборку и предотвратит случайную утечку секретов.*
## Пример корректного Dockerfile (мини-шаблон)
*Для быстрого старта используй команду `docker init`, которая сгенерирует корректные Dockerfile, compose и .dockerignore под твой язык.*
Хорошо, вот интеграция примеров в раздел **Entrypoint** исходного скилла. Я добавил два конкретных скрипта (Alpine/`su-exec` и Ubuntu/`gosu`) и описал, как их встроить в Dockerfile. Также учтена exec-форма и рекомендации по сигналам.
## Entrypoint
Сделай кастомный Docker-Entrypoint, который запускает все скрипты из папки `/etc/docker-custom-init/*.sh`, сортируя их по имени.
Если нужно запустить само приложение, вызови `app`. Тогда entrypoint должен заменить вызов вызовом оригинального приложения.
**Обязательно используй exec-форму** в `ENTRYPOINT` и `CMD` – запись вида `["executable", "param"]`, а не `executable param`. Только exec-форма гарантирует, что сигналы (SIGTERM, SIGINT) будут корректно переданы основному процессу и контейнер сможет правильно завершиться.
### Пример для Alpine (использует `su-exec`)
Скрипт `docker-entrypoint.sh` (универсален для Alpine, BusyBox, минимальных образов):
```sh
#!/bin/sh
set -e
INIT_DIRS="${DOCKER_INITD_DIRS:-/etc/init-custom-docker.d}"
SCRIPTS_ENVS="${DOCKER_INIT_SCRIPTS_ENVS}"
INIT_SCRIPT="${DOCKER_INIT_SCRIPT}"
log() {
echo "[init] $1"
}
# 1. Run init scripts from directories (as root)
for dir in $(echo "$INIT_DIRS" | tr ',' ' '); do
if [ -d "$dir" ]; then
log "Processing init directory: $dir"
for script in $(find "$dir" -maxdepth 1 -name '*.sh' -type f 2>/dev/null | sort); do
log "Running script: $script"
sh "$script"
done
else
log "Directory not found, skipping: $dir"
fi
done
# 2. Run scripts from environment variables (as root)
if [ -n "$SCRIPTS_ENVS" ]; then
for env_name in $(echo "$SCRIPTS_ENVS" | tr ',' ' '); do
eval "script_content=\$$env_name"
if [ -z "$script_content" ]; then
log "Error: Environment variable $env_name specified but empty"
exit 1
fi
log "Running scripts from environment: $env_name"
echo "$script_content" | sh
done
fi
# 3. Run script from DOCKER_INIT_SCRIPT (as root)
if [ -n "$INIT_SCRIPT" ]; then
log "Running script from DOCKER_INIT_SCRIPT"
echo "$INIT_SCRIPT" | sh
fi
# Switch to appuser using su-exec
log "Switching to user: appuser"
log "Starting application: $@"
exec su-exec appuser:appgroup "$@"
```
Как добавить в Dockerfile (Alpine):
```dockerfile
RUN apk add --no-cache su-exec # для Alpine
RUN addgroup -g 1000 appgroup && adduser -u 1000 -G appgroup -D appuser
COPY docker-entrypoint.sh /usr/local/bin/docker-entrypoint.sh
RUN chmod +x /usr/local/bin/docker-entrypoint.sh
ENTRYPOINT ["docker-entrypoint.sh"]
CMD ["app"]
```
### Пример для Ubuntu/Debian (использует `gosu`)
Скрипт `docker-entrypoint.sh` (аналог для Debian-based образов):
```bash
#!/bin/bash
set -e
INIT_DIRS="${DOCKER_INITD_DIRS:-/etc/init-custom-docker.d}"
SCRIPTS_ENVS="${DOCKER_INIT_SCRIPTS_ENVS}"
INIT_SCRIPT="${DOCKER_INIT_SCRIPT}"
APP_USER="${APP_USER:-appuser}"
APP_GROUP="${APP_GROUP:-appgroup}"
log() {
echo "[init] $1"
}
# 1. Execute scripts from directories (as root)
IFS=',' read -ra dirs <<< "$INIT_DIRS"
for dir in "${dirs[@]}"; do
if [ -d "$dir" ]; then
log "Processing init directory: $dir"
for script in $(find "$dir" -maxdepth 1 -name '*.sh' -type f 2>/dev/null | sort); do
log "Running script: $script"
bash "$script"
done
else
log "Directory not found, skipping: $dir"
fi
done
# 2. Execute scripts from environment variables (as root)
if [ -n "$SCRIPTS_ENVS" ]; then
IFS=',' read -ra env_names <<< "$SCRIPTS_ENVS"
for env_name in "${env_names[@]}"; do
script_content="${!env_name}"
if [ -z "$script_content" ]; then
log "Error: Environment variable $env_name specified but empty"
exit 1
fi
log "Running script from environment: $env_name"
echo "$script_content" | bash
done
fi
# 3. Execute script from DOCKER_INIT_SCRIPT (as root)
if [ -n "$INIT_SCRIPT" ]; then
log "Running script from DOCKER_INIT_SCRIPT"
echo "$INIT_SCRIPT" | bash
fi
# Switch to appuser using gosu
log "Switching to user: $APP_USER"
log "Starting application: $@"
exec gosu "${APP_USER}:${APP_GROUP}" "$@"
```
Как добавить в Dockerfile (Ubuntu):
```dockerfile
# Установка gosu (два варианта: через apt или копирование статического бинарника)
RUN apt-get update && apt-get install -y --no-install-recommends gosu && rm -rf /var/lib/apt/lists/*
# Или: COPY --from=gosu/alpine:latest /usr/local/bin/gosu /usr/local/bin/gosu
RUN groupadd -r appgroup && useradd -r -g appgroup appuser
COPY docker-entrypoint.sh /usr/local/bin/docker-entrypoint.sh
RUN chmod +x /usr/local/bin/docker-entrypoint.sh
ENTRYPOINT ["docker-entrypoint.sh"]
CMD ["app"]
```
### Важные замечания
- Внутри скрипта обязательно используй **exec** в конце (например, `exec su-exec appuser "$@"` или `exec gosu appuser "$@"`), чтобы заменить процесс entrypoint на основной процесс приложения.
- Пути к init-директориям по умолчанию можно переопределить через переменные окружения (`DOCKER_INITD_DIRS`). Убедись, что нужные директории существуют (создай их в Dockerfile или смонтируй как тома).
- Чтобы уменьшить количество слоёв, можно объединить команды `RUN groupadd ... && useradd ... && apt-get install ...` в один `RUN`.
# Self-check список для валидации Dockerfile
Используй этот список для проверки своего Dockerfile перед финальной сборкой. Отмечай каждый пункт как выполненный.
## 1. Структура и многоступенчатость (Multistage)
- [ ] Есть ли разделение на этап сборки и этап запуска (если приложение требует компиляции/установки зависимостей)?
- [ ] Используется ли `COPY --from=` для переноса артефактов между стейджами?
- [ ] Объединены ли команды RUN в один слой (через `&&`) и очищен ли временный кэш (например, `rm -rf /var/lib/apt/lists/*`)?
- [ ] Используется ли `--mount=type=cache` для ускорения повторных сборок (если применимо)?
## 2. Параллелизм (BuildKit)
- [ ] Можно ли независимые стейджи выполнять параллельно? (Зависимости, тесты, документация – разделены?)
- [ ] Нет ли неявных зависимостей между параллельными стейджами?
## 3. Безопасность
- [ ] Версия базового образа фиксирована (например `python:3.11-slim-bookworm`) вместо `:latest`?
- [ ] Создан непривилегированный пользователь (`useradd -r -g appgroup appuser`)?
- [ ] Финальный процесс запускается от этого пользователя (`USER appuser`)?
- [ ] Файлы скопированы с `--chown=appuser:appgroup`?
- [ ] Используются флаги уменьшения размера: `--no-install-recommends` (apt), `--no-cache-dir` (pip), `--only=production` (npm)?
- [ ] Нет секретов в переменных `ENV` или слоях образа (используются `ARG`/секреты BuildKit)?
- [ ] Добавлен `HEALTHCHECK` для продакшен-контейнеров?
- [ ] Присутствуют метаданные `LABEL` (maintainer, version, description)?
## 4. Entrypoint
- [ ] Написан кастомный скрипт entrypoint, который запускает init-скрипты из `/etc/docker-custom-init/*.sh` (или из переменных окружения)?
- [ ] Entrypoint и CMD записаны в exec-форме (`["entrypoint.sh", "param"]`)?
- [ ] Внутри скрипта используется `exec` для передачи управления основному процессу (сигналы корректно обрабатываются)?
- [ ] Пользователь переключается на непривилегированного (через `gosu` или `su-exec`) в entrypoint?
## 5. .dockerignore
- [ ] Существует файл `.dockerignore` в корне проекта?
- [ ] В него включены: `.git`, `node_modules`, `venv`, `__pycache__`, `*.md`, `docker-compose.yml`, временные файлы?
## 6. Makefile
- [ ] Есть Makefile с таргетами для сборки, запуска, возможно `docker init`?
- [ ] В Dockerfile или инструкциях не используется прямая сборка через `docker build`, а только через `make`?
## 7. Docker Compose
- [ ] Есть файл `docker-compose.yml` (или `docker-compose.yaml`) для локального запуска?
- [ ] В compose корректно указаны порты, volumes, переменные окружения, не используется непривилегированный пользователь без необходимости?
## 8. Сканирование уязвимостей
- [ ] В CI/CD добавлен этап сканирования готового образа (trivy, docker scout, grype)?
- [ ] Уязвимости критического уровня отсутствуют или задокументированы?
## 9. Тестирование
- [ ] Dockerfile успешно собирается без ошибок?
- [ ] Контейнер запускается и отвечает на healthcheck (если задан)?
- [ ] Init-скрипты выполняются в правильном порядке?
---
### Пример быстрой проверки (через команды)
```bash
# 1. Проверка на секреты (не должно быть hardcoded паролей)
grep -n 'password\|secret\|key=' Dockerfile
# 2. Проверка exec-формы
grep -n '^ENTRYPOINT\|^CMD' Dockerfile | grep -v '^\['
# 3. Проверка наличия пользователя
grep -n '^RUN.*useradd\|^USER' Dockerfile
# 4. Проверка healthcheck
grep 'HEALTHCHECK' Dockerfile || echo "HEALTHCHECK отсутствует"
```
.agents/skills/common/README.md
+11
View File
@@ -0,0 +1,11 @@
# Common Skills
> **Русская версия:** [README.ru.md](README.ru.md)
Language-agnostic skills applicable across all projects.
## Available Skills
| Skill | Description |
|-------|-------------|
| [common-daemon-cli-skill](common-daemon-cli-skill) | Use this skill whenever you need to implement a system daemon mode in your CLI application. |
.agents/skills/common/README.ru.md
+12
View File
@@ -0,0 +1,12 @@
# Common Skills
> **English version:** [README.md](README.md)
> **Вернуться к оглавлению:** [README.ru.md](../../README.ru.md)
Языко-независимые скиллы, применимые во всех проектах.
## Доступные скиллы
| Скилл | Описание |
|-------|----------|
| [common-daemon-cli-skill](common-daemon-cli-skill) | Используйте этот скилл, когда необходимо реализовать режим системного демона в CLI-приложении. |
.agents/skills/common/common-daemon-cli-skill/SKILL.md
+69
View File
@@ -0,0 +1,69 @@
---
name: common-daemon-cli-skill
description: Use this skill whenever you need to implement a system daemon mode in your CLI application.
---
# The "daemon" Subcommand
Every CLI daemon must have a `daemon` subcommand with the following nested subcommands:
* logs - view logs of the active daemon
* enable - enable the daemon, copy configuration files and the executable itself to system directories
* disable - disable the daemon; if executable files or config were copied to a system directory, do not touch them
* status - show the daemon's status. Is it enabled? If it crashed, what error? The last 50 lines of logs if it crashed.
* env - allows modifying the environment variables with which the daemon runs.
* restart - restarts the daemon
For example:
```shell
myrootcommand daemon [ enable / disable / logs / status / env ]
```
# Enabling the Daemon
Enabling the daemon must support the following operating systems:
* Linux-based
* \+ Systemd
* \+ OpenRC
* \+ Init.D
* FreeBSD
* Windows
* MacOS
(The method by which the application is added to autostart is displayed in `status`)
Enabling includes:
* Copying the configuration file and the application executable to a location where the user won't accidentally delete them. For example, to `/var/lib/*` on Linux. If the `--no-copy` flag is provided, copying is skipped and files remain in their original locations.
* Adding the application to autostart with the specified config (if there is a config at all)
* Storing information somewhere publicly accessible that the daemon is enabled.
# Disabling
When disabling, check whether the daemon exists or not.
If the daemon does not exist, simply inform the user.
If the daemon is in autostart, remove it from there.
# Removing Copies
Removes any copies made during enabling, if they were created.
Activated by the `--remove` flag.
In this case, a random number between 1000 and 2000 is generated, which must be entered into an interactive input field to confirm.
The CLI explicitly and clearly notifies the user about this.
# Environment Configuration (env)
Allows editing / adding / deleting the daemon's environment variables.
Subcommands:
* add - adds a new value; overwrites the old one if it exists
* remove - removes an environment variable
* append - appends to an environment variable using the path separator — for example, on Linux: `$PATH:new_value` and so on.
```shell
myrootcommand daemon env append PATH /path/to/my/external/files
myrootcommand daemon restart
```
.agents/skills/golang/README.md
+12
View File
@@ -0,0 +1,12 @@
# Go Skills
> **Русская версия:** [README.ru.md](README.ru.md)
Skills specific to Go (Golang) projects.
## Available Skills
| Skill | Description |
|-------|-------------|
| [golang-flexible-config-skill](golang-flexible-config-skill) | Defines how an AI agent should implement configuration loading in any Go application. |
| [golang-tswf-codestyle-skill](golang-tswf-codestyle-skill) | Tswf.io Go codestyle conventions: naming, interfaces, constructors, line breaks, and project structure. |
.agents/skills/golang/README.ru.md
+13
View File
@@ -0,0 +1,13 @@
# Go Skills
> **English version:** [README.md](README.md)
> **Вернуться к оглавлению:** [README.ru.md](../../README.ru.md)
Скиллы для проектов на Go (Golang).
## Доступные скиллы
| Скилл | Описание |
|-------|----------|
| [golang-flexible-config-skill](golang-flexible-config-skill) | Определяет, как AI-агент должен реализовать загрузку конфигурации в любом Go-приложении. |
| [golang-tswf-codestyle-skill](golang-tswf-codestyle-skill) | Конвенции кодстайла Go для tswf.io: именование, интерфейсы, конструкторы, переносы строк и структура проекта. |
.agents/skills/golang/golang-flexible-config-skill/SKILL.md
+261
View File
@@ -0,0 +1,261 @@
---
name: golang-flexible-config-skill
description: This skill defines how an AI agent should implement configuration loading in any Go application.
---
## Skill: Go Configuration Pipeline (Launch Arg → Env → Config File → Defaults)
**Description**: This skill defines how an AI agent should implement configuration loading in any Go application. The configuration must be loaded in the following order of priority (each source overrides the previous):
1. **Launch arguments** (command-line flags) – **must be handled via Cobra**.
2. **Environment variables** (uppercase, dots/dashes replaced by underscores).
3. **Configuration file** (YAML/JSON/TOML).
4. **Hardcoded defaults**.
The skill enforces consistent logging of the config file location (found or not), a well-defined search path list, and a `generate-config` subcommand that produces a commented example config in either English or Russian.
All subcommands and flag definitions **must use the Cobra library** (`github.com/spf13/cobra`).
### Env Variable Rules
- All environment variables must use **UPPER_CASE**.
- Dots (`.` ) and dashes (`-`) in config property names are replaced with underscores (`_`).
- Example: property `db.path` maps to env `APP_DB_PATH`, `log-level` maps to `APP_LOG_LEVEL`.
- Use `viper.SetEnvKeyReplacer(strings.NewReplacer(".", "_", "-", "_"))`.
### Config File Search Order
The config file is searched in the following order (first found wins). **The agent must log the exact path where the file was found.** If no file is found, log "Configuration file not found" and log the **list of all checked paths** so the user knows where to place one.
1. **Environment variable `APP_CONFIG_LOCATION`** (or custom prefix) – absolute path.
2. **Working directory** – `./config.yaml` (or other extensions).
3. **User config directory** – `~/.config/{app-name}/config.yaml`.
4. **Binary directory** – directory containing the executable.
5. **System Config default** - directory such as /etc/* on unix systems
### Subcommand: `generate-config` (Cobra based)
The application must support a subcommand `generate-config` that writes an example config file with detailed comments.
- **Flag `--lang`** (default `"en"`, possible values `"en"` and `"ru"`).
- `en`: comments in English.
- `ru`: comments in Russian.
- **Flag `--output`** (optional; if not provided, prints to stdout; if given, writes to that file).
The generated file must include every configuration parameter with a meaningful comment describing its purpose and type.
### Example Implementation (Template for AI Agent – Cobra version)
Below is a complete, production‑ready example. The agent should use this pattern or a very similar one.
```go
package main
import (
"fmt"
"log"
"os"
"path/filepath"
"strings"
"github.com/spf13/cobra"
"github.com/spf13/viper"
)
// Config structure with mapstructure tags
type Config struct {
Port int `mapstructure:"port"`
DBPath string `mapstructure:"db_path"`
LogLevel string `mapstructure:"log_level"`
}
func main() {
var rootCmd = &cobra.Command{
Use: "myapp",
Short: "MyApp configuration loader",
Long: `Loads configuration from defaults, file, env, and CLI flags (in order of priority).`,
RunE: func(cmd *cobra.Command, args []string) error {
cfg, err := loadConfig()
if err != nil {
return err
}
_ = cfg // use cfg in your application
return nil
},
}
// Define main flags
rootCmd.Flags().Int("port", 0, "listen port (overrides env/file)")
rootCmd.Flags().String("db-path", "", "path to database")
rootCmd.Flags().String("log-level", "", "log level (debug, info, warn, error)")
// Bind Viper to flags
viper.BindPFlag("port", rootCmd.Flags().Lookup("port"))
viper.BindPFlag("db_path", rootCmd.Flags().Lookup("db-path"))
viper.BindPFlag("log_level", rootCmd.Flags().Lookup("log-level"))
// Subcommand: generate-config
var generateCmd = &cobra.Command{
Use: "generate-config",
Short: "Generate an example config file",
Run: func(cmd *cobra.Command, args []string) {
lang, _ := cmd.Flags().GetString("lang")
output, _ := cmd.Flags().GetString("output")
generateConfig(lang, output)
},
}
generateCmd.Flags().String("lang", "en", "Language for comments: en or ru")
generateCmd.Flags().String("output", "", "Output file path (if empty, prints to stdout)")
rootCmd.AddCommand(generateCmd)
if err := rootCmd.Execute(); err != nil {
log.Fatal(err)
}
}
// loadConfig implements the ordered configuration loading
func loadConfig() (*Config, error) {
// 1. Defaults (lowest priority)
viper.SetDefault("port", 8080)
viper.SetDefault("db_path", "./data.db")
viper.SetDefault("log_level", "info")
// 2. Config file search with logging
viper.SetConfigName("config")
viper.SetConfigType("yaml") // can also support json, toml, etc.
// Collect all search paths for logging
searchPaths := []string{}
// a) Environment variable APP_CONFIG_LOCATION
if envLoc := os.Getenv("APP_CONFIG_LOCATION"); envLoc != "" {
viper.SetConfigFile(envLoc)
searchPaths = append(searchPaths, fmt.Sprintf("ENV: APP_CONFIG_LOCATION = %s", envLoc))
if err := viper.ReadInConfig(); err == nil {
log.Printf("Config file found: %s (from APP_CONFIG_LOCATION)", viper.ConfigFileUsed())
} else {
return nil, fmt.Errorf("config file specified in APP_CONFIG_LOCATION not found: %s", envLoc)
}
} else {
// b) Working directory
wd, _ := os.Getwd()
searchPaths = append(searchPaths, fmt.Sprintf("Working directory: %s", filepath.Join(wd, "config.yaml")))
viper.AddConfigPath(".")
// c) ~/.config/{app-name}/
homeDir, _ := os.UserHomeDir()
appName := "myapp" // replace with your application name
userConfigPath := filepath.Join(homeDir, ".config", appName)
searchPaths = append(searchPaths, fmt.Sprintf("User config: %s", filepath.Join(userConfigPath, "config.yaml")))
viper.AddConfigPath(userConfigPath)
// d) Binary directory
exePath, _ := os.Executable()
exeDir := filepath.Dir(exePath)
searchPaths = append(searchPaths, fmt.Sprintf("Binary directory: %s", filepath.Join(exeDir, "config.yaml")))
viper.AddConfigPath(exeDir)
// e) /etc/{app-name}/
etcPath := filepath.Join("/etc", appName)
searchPaths = append(searchPaths, fmt.Sprintf("System config: %s", filepath.Join(etcPath, "config.yaml")))
viper.AddConfigPath(etcPath)
// Try to read config
if err := viper.ReadInConfig(); err != nil {
if _, ok := err.(viper.ConfigFileNotFoundError); ok {
log.Println("Configuration file not found. Using defaults and env/args.")
log.Println("Search paths checked:")
for _, p := range searchPaths {
log.Println(" ", p)
}
} else {
return nil, fmt.Errorf("error reading config: %w", err)
}
} else {
log.Printf("Config file found: %s", viper.ConfigFileUsed())
}
}
// 3. Environment variables
viper.SetEnvPrefix("APP")
viper.SetEnvKeyReplacer(strings.NewReplacer(".", "_", "-", "_"))
viper.AutomaticEnv()
// 4. Command-line flags – already bound via Cobra, Viper will read them
// No additional code needed, flags are already processed by Cobra
// 5. Decode into Config struct
var cfg Config
if err := viper.Unmarshal(&cfg); err != nil {
return nil, fmt.Errorf("config unmarshal failed: %w", err)
}
return &cfg, nil
}
// generateConfig writes an example config file with English or Russian comments
func generateConfig(lang, output string) {
var commentLines []string
switch lang {
case "ru":
commentLines = []string{
"# Конфигурация приложения MyApp",
"# Все значения могут быть переопределены через переменные окружения (префикс APP_)",
"# или аргументы командной строки.",
"",
"port: 8080 # Порт, на котором будет слушать HTTP-сервер",
"db_path: \"./data.db\" # Путь к файлу базы данных SQLite",
"log_level: \"info\" # Уровень логирования: debug, info, warn, error",
}
default: // en
commentLines = []string{
"# MyApp configuration file",
"# All values can be overridden by environment variables (prefix APP_)",
"# or command-line flags.",
"",
"port: 8080 # HTTP server listening port",
"db_path: \"./data.db\" # Path to SQLite database file",
"log_level: \"info\" # Log level: debug, info, warn, error",
}
}
content := strings.Join(commentLines, "\n") + "\n"
if output != "" {
if err := os.WriteFile(output, []byte(content), 0644); err != nil {
log.Fatalf("Failed to write example config: %v", err)
}
log.Printf("Example config written to %s", output)
} else {
fmt.Print(content)
}
}
```
### Agent Instructions
- Use the code above as a baseline.
- **All subcommands and CLI flags must be implemented via Cobra** (`github.com/spf13/cobra`).
- Always include logging of config file location (found or not) and all checked search paths.
- The `generate-config` subcommand must accept `--lang` (default `"en"`, can be `"ru"`) and `--output`.
- Comments inside the generated config must be meaningful and match the selected language.
- The config search order and env variable rules are mandatory.
- If the agent uses a different configuration library (e.g., `envconfig`, `cleanenv`), the same semantics must be maintained, but CLI handling must still be through Cobra.
- **Never set `SilenceUsage: true` on the root command or any subcommand.** Incorrect flags or invalid subcommands **must** result in an explicit error message and the usage output. Cobra’s default behavior already provides this; leave `SilenceUsage` and `SilenceErrors` at their default (`false`). Do not suppress automatic usage printing on parse errors.
---
## Self‑Check: Adding a New Config Property
When you introduce a new configuration property to the application (e.g. `max_connections`), verify each of the following points. **Tick all boxes before considering the change complete.**
- [ ] 1. **Default value** – A sensible hardcoded default is set in `viper.SetDefault(...)` (or equivalent).
- [ ] 2. **Struct field** – The property exists as a field in the `Config` struct with a correct `mapstructure` tag.
- [ ] 3. **Env mapping** – The corresponding environment variable is automatically picked up. Ensure the env name follows the pattern: prefix + uppercase + underscores (dots/dashes replaced).
*Example:* for `max_connections` you should be able to set `APP_MAX_CONNECTIONS`.
- [ ] 4. **Command‑line flag (Cobra)** – A flag is defined on the appropriate Cobra command (root or sub) and bound via `viper.BindPFlag(...)`.
*Example flag name:* `--max-connections` (dashes). The binding uses the struct key `max_connections`.
- [ ] 5. **Config file key** – The property is documented as a key in the example config file (YAML key matches the struct field name).
- [ ] 6. **Comments in `generate-config`** – The example config output includes a meaningful comment for the new property in **both** English and Russian (if the `--lang ru` case is implemented). The comment explains the unit, default, and allowed values if applicable.
- [ ] 7. **Logging** – No additional logging is required for a single property change; the existing logging of the config file location is sufficient.
- [ ] 8. **Consistency check** – Verify that the same property name is used consistently across: struct tag, env prefix replacement, flag binding, and config file key.
**Run through this checklist every time you add or modify a configuration property.** This ensures all four sources (defaults, file, env, flags) work seamlessly together.
.agents/skills/golang/golang-tswf-codestyle-skill/SKILL.md
+243
View File
@@ -0,0 +1,243 @@
---
name: golang-tswf-codestyle-skill
description: ТЫ ДОЛЖЕН использовать этот скилл для всех Go проектов, модуль которых содержит подстроку tswf.io или если пользователь явно выбрал этот скилл.
---
# Naming Conventions
## Базовые инструкции
* __Запрещено использовать короткие имена__ - `i`, `j`, `k`, `temp` и так далее - под запретом. Все переменные и методы должны иметь осмысленные имена.
* __Локальные переменные__ - camelCase, `:=`
* __Слайсы__ - всегда инициализируются через `make`.
* __Поля переменные__ - camelCase
* __Function Receiver__ - первая буква структуры, владелец по указателю.
* __Структура, скрытая за интерфейсом__ - camelCase с маленькой буквы (`myPrivateStuct`)
* __Публичная структура__, _например Dto_ - camelCase с большой буквы (`MyPublicStruct`)
```go
type Something struct {}
func (s /*<--- первая буква слова Something*/ *Something) somethingFunction() { }
```
## Пакеты в именах типов
Старайся использовать пакет, как часть имени класса, а классы старайся раскладывать по пакетам как можно более по смыслу, но не уходя в спагетти код
__Плохой пример__
```go
dtos.SomethingApiUserDto
```
__Хороший пример__
```go
somethingapi.UserDto
```
## Лаконичные, достаточные названия
Если класс что-то делает, а так чаще всего - старайся вложить это в название.
Например класс для регистрации - `Registrar`, а класс для продвинутого конфигурирования чего-то - `AdvancedConfigurer`.
Если ты видишь, что тебе в имени нужно указать что-то слишком длинное, вроде `SomethingApiUserDtoMapperToOurEntity`, то постарайся закрыть вопрос контекста в имени класса его пакетом и придумай название получше.
# Interfaces
Ты ДОЛЖЕН скрывать компоненты за интерфейсами.
Это позволит подстраховаться от жесткой связанности на структуру.
Например:
**ПЛОХОЙ пример**
```go
type MyComponent struct {
// ...
}
func NewMyComponent() *MyComponent {}
```
**ПРАВИЛЬНЫЙ ПРИМЕР**
```go
type MyComponent interface {
// Все нужные публичные методы из myComponent
}
type myComponent struct {
// ...
}
// Этот пример правильный - используется интерфейс
func NewMyComponent() MyComponent {
return &myComponent{}
}
```
Даже в рамках одного пакета при возможности используй интерфейсы вместо структур для инъекций компонентов
# Инстанцирование
Все новые объекты создаются через конструктор.
Конструктор принимает все объекты зависимости и внедряет их в создаваемый объект прямо при конструировании.
Если возникает циклическая зависимость, то можно применить паттерн "Фасад"
__Плохой пример__
```go
type MyComponent interface{
SetDependency1(dep Dependency1)
SetDependency2(dep Dependency2)
}
// Плохой пример
// Конструктор не гарантирует полного инстанцирования объекта.
// Такой конструктор может вернуть объект в некорректном состоянии (если вызывающий код сам не использует сеттеры для зависимостей)
func NewMyComponent() MyComponent { /** ... **/ }
```
__Хороший пример__
```go
type MyComponent interface{
}
// Хороший пример
// Конструктор явно принял все зависимости
// Параметры конструктора для читаемости каждый на новой строке
func NewMyComponent(
dep1 Dependency1,
dep2 Dependency1,
) MyComponent
{
/** ... **/
}
```
# Перенос строк
Для читаемости кода человеком и улучшения его визуальной структуры в коде применяются переносы строк.
Ты можешь применять их более креативно, но приведу два кейса: __вложенность__ и __много параметров__ функции
## Вложенность
Если вызовы вложены один в другой, то человеку-читателю легко потерять структуру. Например:
__Плохой пример__
```go
// Вся цепочка смешивается в кашу.
// Сразу не видно, кто в кого вложен. Читателю приходится напрягаться просто чтобы это понять. Это очень плохой пример
SomeMethod1(SomeMethod2(SomeMethod3()), SomeMethod4(&myStruct{1, 2, 3}))
```
__Хороший пример__
```go
// Вся цепочка вызовов структурно сразу визуализируется глазами читателя.
// Видно, кто в кого вложен. Это хороший пример
SomeMethod1(
SomeMethod2(
SomeMethod3(),
),
SomeMethod4(
&myStruct{1, 2, 3},
),
)
```
## Множество параметров функции
Если у функции есть параметры с длинными именами, типами или просто занимающими место - используй перенос строк, чтобы наглядно разделить их.
__Плохой пример__
```go
// Все в кучу. Чем больше параметров, тем тяжелее читать человеку
// Это плохой пример.
func funcOne(a string, b int, c bool, g pos.Position, e user.Controller) {}
func main() {
funcTwo("hello world", 1, false, resolvePos().absolute(), resolveController())
}
```
__Хороший пример__
```go
// Четко видно структуру. Человеку просто такое читать.
// Это хороший пример.
func funcOne(
a string,
b int,
c bool,
g pos.Position,
e user.Controller,
) {}
func main() {
funcTwo(
"hello world",
1,
false,
resolvePos().absolute(),
resolveController(),
)
}
```
# Project Structure
## Обязательно для любого Go проекта
* Makefile с кросс-платформенной сборкой
* README.md + русская версия
* Весь код лежит в папках `pkg`, `cmd` или `resources` ( код для доступа к embed ресурсам )
* Папка `bin` - туда попадают собранные через Make бинари. Она в gitignore
## Для микросервисов
Если это сервис, то добавляется
* Dockerfile
* Docker-compose
* k8s chart
## CLI приложение
* Папка `cmd`, в подпапках которой реализуются консольные команды на `cobra`(!)
## Структура файлов
* Root
* bin # ОБЯЗАТЕЛЬНО в .gitignore!!
* doc
* deploy
* docker (_тут если делаешь Docker, то добавь сразу compose. и посмотри релевантные скиллы для этого_)
* k8s
* e.t.c.
* pkg
* domain
* adapters
* infrastructure
* cmd
* \<cmdname\>
* resources
* \<embed go resources\>
* go.mod
* go.sum
* README.md
* README.ru.md
* .gitignore
* Makefile
# Имя модуля
Спроси имя модуля проекта, если пользователь явно его не задал.
Если пользователь дал базовый путь (он оканчивается на `/`), то прибавь к нему имя корневой папки проекта
.codegraph
Symlink
+1
View File
@@ -0,0 +1 @@
/home/vergil/.omo/codegraph/projects/go-synapse-backupper-e2e7657dae43c695
.dockerignore
+10
View File
@@ -0,0 +1,10 @@
.git
.omo
.agents
bin
*.md
*.pem
*.priv.*
*.tmp
doc/
deploy/
.gitignore
+7
View File
@@ -0,0 +1,7 @@
bin/
*.tmp
*.pqenc
*.pem
!example*.pem
!*.example.pem
/.omo/
.idea/.gitignore
Generated
+10
View File
@@ -0,0 +1,10 @@
# Default ignored files
/shelf/
/workspace.xml
# Editor-based HTTP Client requests
/httpRequests/
# Ignored default folder with query files
/queries/
# Datasource local storage ignored files
/dataSources/
/dataSources.local.xml
.idea/dataSources.xml
Generated
+17
View File
@@ -0,0 +1,17 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="DataSourceManagerImpl" format="xml" multifile-model="true">
<data-source source="LOCAL" name="postgres@localhost" uuid="46b0aa2d-f54b-4141-9519-14024caadd97">
<driver-ref>postgresql</driver-ref>
<synchronize>true</synchronize>
<jdbc-driver>org.postgresql.Driver</jdbc-driver>
<jdbc-url>jdbc:postgresql://localhost:5430/postgres</jdbc-url>
<jdbc-additional-properties>
<property name="com.intellij.clouds.kubernetes.db.host.port" />
<property name="com.intellij.clouds.kubernetes.db.enabled" value="false" />
<property name="com.intellij.clouds.kubernetes.db.container.port" />
</jdbc-additional-properties>
<working-dir>$ProjectFileDir$</working-dir>
</data-source>
</component>
</project>
.idea/go-synapse-backupper.iml
Generated
+9
View File
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<module type="WEB_MODULE" version="4">
<component name="Go" enabled="true" />
<component name="NewModuleRootManager">
<content url="file://$MODULE_DIR$" />
<orderEntry type="inheritedJdk" />
<orderEntry type="sourceFolder" forTests="false" />
</component>
</module>
.idea/go.imports.xml
Generated
+10
View File
@@ -0,0 +1,10 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="GoImports">
<option name="excludedPackages">
<array>
<option value="golang.org/x/net/context" />
</array>
</option>
</component>
</project>
.idea/modules.xml
Generated
+8
View File
@@ -0,0 +1,8 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="ProjectModuleManager">
<modules>
<module fileurl="file://$PROJECT_DIR$/.idea/go-synapse-backupper.iml" filepath="$PROJECT_DIR$/.idea/go-synapse-backupper.iml" />
</modules>
</component>
</project>
.idea/vcs.xml
Generated
+6
View File
@@ -0,0 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="VcsDirectoryMappings">
<mapping directory="$PROJECT_DIR$" vcs="Git" />
</component>
</project>
.omo/notepads/synapse-backupper/learnings.md
+92
View File
@@ -0,0 +1,92 @@
# Synapse Backupper - Learnings & Conventions
## Project
- Module: git.tswf.io/infra/go-synapse-backupper
- Go 1.26+
- NO CGO, NO k8s, NO systemd daemon ceremony
## Skills in use
- golang-tswf-codestyle-skill (project structure)
- golang-flexible-config-skill (config loading)
- dockerfile-skill (multistage Dockerfile)
## Key Decisions
- TDD throughout
- Dual KEM: ML-KEM-768 (0x0006) + X25519 (0x0007)
- Composite HKDF combiner
- v2 .pqenc artifact format
- Two-phase Sink contract (Begin/Commit/Abort)
- Pipeline orchestrator with select-based fan-in
## Issues/Problems
- (As of Todo 5) Adapter packages (`mlkem768`, `x25519`) do NOT expose a LoadPriv-from-raw constructor; their `Decapsulate` type-asserts to its own unexported `privKey` type and rejects any other `RecipientPriv` implementation. This means KeyManager-constructed raw-bytes-backed `recipientPriv` (Todo 6's ProductType) CANNOT currently be passed to the adapters' `Decapsulate` — they would fail the type assertion. Latent bug surfaced while authoring the composite golden fixture; resolved in tests by registering deterministic FAKE KEMs under the same schemeIDs (0x0006/0x0007, ct lengths 1088/32) so reconstruction from committed raw bytes is possible without touching the adapter packages (out of scope for Todo 5). Real-key Decrypt-with-loaded-PEM must be revisited when KeyManager (Todo 6) is integrated into backup/restore (Todos 11/12) — likely requires exposing an exported `LoadPriv(raw) crypto.RecipientPriv` on each adapter.
## 2026-07-25T00:08:53+03:00 Todo 1 Complete
- Go module initialized with go 1.26
- Project skeleton created per tswf-codestyle-skill
- All directories: bin/, doc/, deploy/docker/, pkg/{domain,adapters,infrastructure}/, cmd/synapse-backupper/, resources/
- go.mod module path: git.tswf.io/infra/go-synapse-backupper
- .gitignore excludes bin/, *.tmp, *.pqenc, *.pem (allows example*.pem)
- Makefile, README.md, README.ru.md, config.example.yaml are empty placeholders for later todos
- Verified: go build ./... passes, skeleton binary prints correct output
- Environment: Go 1.26.5 installed via golang.org/dl
## 2026-07-25T00:28:00+03:00 Todo 2 Complete
- Created pkg/domain/crypto/ ports: KEM, Encryptor, Decryptor, KeyManager interfaces
- Created RecipientPub/RecipientPriv interfaces with SchemeID/KeyID/Raw methods
- Created concurrent-safe Registry (sync.RWMutex) with Register/Lookup/KEMFactory
- Defined sentinel errors: ErrUnknownScheme, ErrDuplicateScheme
- TDD: wrote registry_test.go first, then registry.go implementation
- Verified: `go test -race ./pkg/domain/crypto/...` passes (1.043s)
- Verified: `go build ./...` passes
- No concrete KEM implementations yet (blocked for Todo 3/4)
- No crypto/mlkem or crypto/ecdh imports in domain package
## 2026-07-25 Todo 3 Complete
- Created pkg/adapters/crypto/mlkem768/ adapter wrapping Go 1.26 stdlib crypto/mlkem
- TDD: wrote mlkem768_test.go first, then mlkem768.go implementation
- Adapter implements crypto.KEM interface with GenerateKeyPair, Encapsulate, Decapsulate
- KeyID derived as SHA-256(pub.Raw()[:8])[:8]
- Encapsulate swaps stdlib return order (ss, ct) → adapter (ct, ss) per domain contract
- Explicit return-order test: TestEncapsulateReturnOrder asserts len(ct)==1088 && len(ss)==32
- Decapsulate implements explicit rejection because crypto/mlkem.Decapsulate uses implicit rejection
(docs claim it returns an error for invalid ciphertexts, but implementation always returns nil)
Workaround: recompute Kout = SHAKE256(z || ciphertext) from priv seed and compare; match = ErrDecapsulationFailed
- Registered factory under suiteID 0x0006 in init() to package-level DefaultRegistry
- Verified: `go test -race ./pkg/adapters/crypto/mlkem768/...` passes (1.034s)
- Verified: `go test -run TestRoundTrip -v ./pkg/adapters/crypto/mlkem768/...` prints PASS
- Verified: `grep -q "suiteID.*0x0006" pkg/adapters/crypto/mlkem768/mlkem768.go` succeeds
## 2026-07-25 Todo 4 Complete
- Created pkg/adapters/crypto/x25519/ adapter wrapping Go 1.26 stdlib crypto/ecdh X25519
- TDD: wrote x25519_test.go first, then x25519.go implementation
- Adapter implements crypto.KEM interface with GenerateKeyPair, Encapsulate, Decapsulate
- KeyID derived as SHA-256(pub.Raw())[:8] (full raw key, not truncated like mlkem768)
- Encapsulate: ephemeral keypair via ecdh.X25519().GenerateKey(rand); ct = ephemeralPub.Bytes() (32B); ss = ephemeralPriv.ECDH(pub) (32B)
- Decapsulate: parse ct as ecdh.X25519().NewPublicKey(ct), ss = priv.ECDH(ephemeralPub)
- Added explicit ciphertext validation in Decapsulate: reject len != 32, high bit set (bit 255), and all-zero identity point
- Go stdlib ecdh.NewPublicKey only validates length, so explicit validation is required for KEM security
- Independent ECDH verification test: reconstruct ecdh.PrivateKey from priv.Raw() and ecdh.PublicKey from ct, compute shared secret, assert matches encapsulate output
- Round-trip stress test: 1000 iterations with fresh keypairs each iteration
- Random ciphertext test: 32 random bytes with high bit set → Decapsulate returns ErrDecapsulationFailed
- Registered factory under suiteID 0x0007 in init() to package-level DefaultRegistry
- Verified: `go test -race ./pkg/adapters/crypto/x25519/...` passes (3.662s)
- Verified: `grep -q "suiteID.*0x0007" pkg/adapters/crypto/x25519/x25519.go` succeeds
- Verified: `grep -q "ecdh.X25519" pkg/adapters/crypto/x25519/x25519.go` succeeds
## 2026-07-25 Todo 5 Complete (Composite Encryptor/Decryptor + v2 Artifact)
- Created pkg/adapters/crypto/composite/ with composite.go (production) + composite_test.go (TDD) + golden_generate_test.go (//go:build golden_generate one-time generator)
- Created testdata/golden-1byte.pqenc (1274 bytes — 1231 fixed header + 22B body chunk record for 1-byte 0xAA + 21B final marker chunk record) + testdata/golden-keys.json (committed priv raws as base64 — 122 bytes)
- All 16 required tests pass under -race: (a) Golden decrypt-equality on committed fixture + header byte-offset stability [magic@0:4=0x47535051, version@4:6=0x0002, flags@6:10=0, nRecipients@10=0x02, slot0 schemeID@11:13=0x0006, slot1 schemeID@1113:1115=0x0007], (b) round-trip 0/1/64KiB-1/64KiB/64KiB+1/1MiB, (c) empty→1 final chunk (ct=16B tag-only, flags=0x01), (d) exactly-64KiB→2 chunks (full body flags=0x00 + zero-length final markers flags=0x01), (e) counter wrap to 0xFFFFFFFFFFFFFFFF → ErrNonceCounterWrapped, (f) payload tamper → ErrTamperingDetected, (g) wrappedCEK tamper → ErrTamperingDetected, (h) wrong pq priv swap → ErrWrongKeys (via keyID sanity check), (i) format conformance, (j) version==0x0001 → ErrUnsupportedVersion + countingReader assertion reader.n==11 (no GCM ops attempted), (k) nRecipients==0 → ErrMalformedHeader, (l) nRecipients>2 OR ctLen>1<<20 → ErrMalformedHeader with reader.n==25 (ctLen validation fires per-slot BEFORE any ct io.ReadFull), (m) zero-length non-final chunk → ErrMalformedChunk, (n) oversized chunk (>64KiB+16) → ErrMalformedChunk, (o) premature EOF without final marker → ErrUnexpectedEOF, (p) truncated 30-byte header → ErrMalformedHeader with reader.n<=39 (no recipient allocation)
- Production sentinel errors: ErrMalformedHeader, ErrUnsupportedVersion, ErrWrongKeys, ErrTamperingDetected, ErrNonceCounterWrapped, ErrMalformedChunk, ErrUnexpectedEOF
- Adaptive parser ordering: magic check → version check (ErrUnsupportedVersion BEFORE any flags/nRecipients validation — no GCM ops) → flags check → nRecipients check → per-slot (read meta → validate ctLen ≤ max → read ct) → tail (wrapNonce+wrappedCEK+firstPayloadNonce) → decaps → HKDF combiner → unwrap CEK → chunk stream
- HKDF combiner: VERBATIM per plan Metis B1, but the plan's pseudocode assumed `hkdf.Expand` returns an infinite `io.Reader` requiring `io.ReadFull`. Go 1.26 stdlib `crypto/hkdf` returns `([]byte, error)` DIRECTLY (signatures `Extract(h func() H, secret, salt []byte) ([]byte, error)` and `Expand(h func() H, prk []byte, info string, keyLength int) ([]byte, error)`) — no reader is involved, so neither io.ReadAll nor io.ReadFull is needed; the spirit of the plan's "do not use io.ReadAll on hkdf.Expand" guidance is preserved trivially. Verified at /home/vergil/sdk/go1.26.5/src/crypto/hkdf/hkdf.go.
- Format spec pinned (verbatim from plan line 31): magic u32 BE 0x47535051 "GSPQ" + version u16 BE 0x0002 + flags u32 BE 0x00000000 + nRecipients u8 0x02 + 2 inline-ct recipient blocks (each: schemeID u16 + keyID 8B + ctLen u32 + ct; slot 0 PQ ct=1088B, slot 1 classical ct=32B) + wrapNonce 12B + wrappedCEK 48B (= 32B CEK + 16B GCM tag) + firstPayloadNonce 12B + chunked payload [len u32 BE 4B][flags 1B][AES-256-GCM ct + 16B tag]
- Chunk AEAD: 64 KiB body chunks with flags=0x00; ALWAYS-emit zero-length final marker chunk with flags=0x01 on EOF (Metis B2 — never seek backwards to mutate; the previous chunk's fullness does not affect marker emission)
- CEK wrap: aes.NewCipher(kekFinal) → cipher.NewGCM → Seal(nil, wrapNonce(12B from crypto/rand), cek(32B), AAD=[0x00,0x02]=version u16 BE)
- Counter increment (Metis N5, verbatim): counter := binary.BigEndian.Uint64(chunkNonce[4:12]); newCounter := counter + 1; if newCounter <= counter { return ErrNonceCounterWrapped }; binary.BigEndian.PutUint64(chunkNonce[4:12], newCounter) — chunkNonce[0:4] (random base) untouched
- Slot routing: POSITIONAL (slot 0 ← pq priv, slot 1 ← classical priv) per Metis N4; keyID equality enforced as a fast wrong-key reject BEFORE any AEAD operation (plan N4's "keyID is informational/audit only" preserved — keyID collision would still route correctly positionally and surface ErrTamperingDetected downstream)
- Registry wiring: NewEncryptor/NewDecryptor take a `crypto.Registry`; production code never imports mlkem768 OR x25519 directly — every scheme lookup goes through `Registry.Lookup(schemeID)` as the plan mandates. Tests use a fakeRegistry registering deterministic `fakeKem`(schemeID, ctLen) under the same 0x0006/0x0007 schemeIDs — fake Priv/Pub types (`fakePriv`/`fakePub`) live in composite_test.go so the committed golden fixture can reconstruct privs from raw bytes in testdata/golden-keys.json.
- Verified: `go test -race ./pkg/adapters/crypto/composite/...` PASSES (1.116s) ; `go test -race -count=3 ./pkg/adapters/crypto/composite/...` PASSES (1.318s) ; `go build ./...` PASSES ; `go vet ./pkg/adapters/crypto/composite/...` clean ; `grep -q "ErrNonceCounterWrapped" composite.go` succeeds ; `go test -race ./...` PASSES for ALL packages (no regression)
Makefile
+37
View File
@@ -0,0 +1,37 @@
.PHONY: build docker-build docker-run docker-push lint vet gofumpt test cross-build integration-test
BIN := synapse-backupper
CMD := ./cmd/synapse-backupper
GO := go
build:
$(GO) build -o bin/$(BIN) $(CMD)
docker-build:
docker build -t synapse-backupper:latest -f deploy/docker/Dockerfile .
docker-run:
docker run --rm synapse-backupper:latest --help
docker-push:
docker push synapse-backupper:latest
lint:
golangci-lint run --timeout 5m ./...
vet:
$(GO) vet ./...
gofumpt:
gofumpt -l .
test:
$(GO) test -race ./...
cross-build:
GOOS=linux GOARCH=amd64 $(GO) build -o bin/$(BIN)-linux-amd64 $(CMD)
GOOS=linux GOARCH=arm64 $(GO) build -o bin/$(BIN)-linux-arm64 $(CMD)
integration-test: docker-build
docker tag synapse-backupper:latest synapse-backupper:test
bash scripts/integration-test.sh
README.md
+229
View File
@@ -0,0 +1,229 @@
# Synapse Backupper
PostgreSQL backup tool for [Synapse](https://github.com/element-hq/synapse) with composite dual-KEM encryption.
## Overview
`synapse-backupper` creates encrypted PostgreSQL dumps using a **composite** encryption scheme that combines a post-quantum KEM (ML-KEM-768) with a classical KEM (X25519). Both keys are required to decrypt a backup (AND model). The tool supports two operational modes: a one-shot `backup` command and a scheduler mode (`run`) that performs backups on a cron schedule.
Backups are stored locally on disk; retention pruning is applied automatically based on age.
## Install
Build the Docker image:
```bash
make docker-build
```
This produces `synapse-backupper:latest`. The image contains the compiled binary, PostgreSQL client tools, and runs as an unprivileged user.
## Quick Start
### 1. Generate encryption keys
Generate both key pairs (post-quantum and classical) with the `keygen` subcommand:
```bash
mkdir -p ./keys
docker run --rm -u root -v "$(pwd)/keys:/keys" synapse-backupper:latest \
keygen --type both --out-prefix /keys/synapse
chown -R "$(id -u):$(id -g)" ./keys
```
> **Note:** The container image runs as an unprivileged `app` user by default. `keygen` must write to the mounted host directory, so it is run as root here, then ownership is restored to the current user. Adjust `chown` (e.g., with `sudo`) if you are not running as root.
This creates four files under `./keys/`:
| File | Purpose |
|------|---------|
| `synapse.pq.pub.pem` | Post-quantum public key (for backup) |
| `synapse.pq.priv.pem` | Post-quantum private key (for restore, keep offline) |
| `synapse.classical.pub.pem` | Classical public key (for backup) |
| `synapse.classical.priv.pem` | Classical private key (for restore, keep offline) |
**Important:** Private keys should never be mounted into the backup container. Store them offline or on a separate restore-only host.
### 2. Run modes
#### Scheduler mode (`run`)
The default command starts a cron scheduler and an HTTP health-check endpoint:
```bash
docker run -d \
--name synapse-backupper \
-v "$(pwd)/keys:/keys:ro" \
-v "$(pwd)/backups:/backups" \
-e APP_PG_HOST=db \
-e APP_PG_DATABASE=synapse \
-e APP_PG_USER=synapse \
-e APP_PG_PASSWORD=secret \
-e APP_PQ_PUBLIC_KEY_PATH=/keys/synapse.pq.pub.pem \
-e APP_CLASSICAL_PUBLIC_KEY_PATH=/keys/synapse.classical.pub.pem \
-e APP_BACKUP_DIR=/backups \
synapse-backupper:latest
```
By default backups run at `03:00` daily. The schedule is controlled by `backup.cron` in the config file or `APP_BACKUP_CRON`.
#### One-shot backup (`backup`)
Run a single backup immediately:
```bash
docker run --rm \
-v "$(pwd)/keys:/keys:ro" \
-v "$(pwd)/backups:/backups" \
-e APP_PG_HOST=db \
-e APP_PG_DATABASE=synapse \
-e APP_PG_USER=synapse \
-e APP_PG_PASSWORD=secret \
-e APP_PQ_PUBLIC_KEY_PATH=/keys/synapse.pq.pub.pem \
-e APP_CLASSICAL_PUBLIC_KEY_PATH=/keys/synapse.classical.pub.pem \
-e APP_BACKUP_DIR=/backups \
synapse-backupper:latest backup
```
The resulting file is named `synapse-<timestamp>.dump.pqenc` and placed in the backup directory. The underlying dump is produced in PostgreSQL custom format (`--format=custom`), which is the format expected by `pg_restore`.
### 3. Restore a backup on a separate host
Restoration is intentionally performed on a host that does **not** have access to the database. Only the private keys are required.
```bash
docker run --rm \
-v "$(pwd)/keys:/keys:ro" \
-v "$(pwd)/backups:/backups:ro" \
-v "$(pwd)/restored:/out" \
synapse-backupper:latest restore \
--in /backups/synapse-20260102-150405.dump.pqenc \
--privkey-pq /keys/synapse.pq.priv.pem \
--privkey-classical /keys/synapse.classical.priv.pem \
--out /out/restored.dump
```
Then restore the plaintext dump with standard PostgreSQL tools:
```bash
pg_restore --clean --if-exists --dbname=synapse restored.dump
```
## Configuration
The tool reads configuration in the following priority order: command-line flags → `APP_*` environment variables → config file → defaults.
Quick Start examples use environment variables. The prefix is `APP_`, and nested keys use underscores, for example:
- `APP_PG_HOST`
- `APP_PG_DATABASE`
- `APP_BACKUP_DIR`
- `APP_PQ_PUBLIC_KEY_PATH`
- `APP_CLASSICAL_PUBLIC_KEY_PATH`
To use a config file instead, generate an example and mount it into the container:
```bash
docker run --rm synapse-backupper:latest generate-config --lang en > config.yaml
```
Edit `config.yaml`, then mount it at `/config.yaml` when running the container:
```bash
docker run -d \
--name synapse-backupper \
-v "$(pwd)/config.yaml:/config.yaml" \
-v "$(pwd)/keys:/keys:ro" \
-v "$(pwd)/backups:/backups" \
synapse-backupper:latest
```
Environment variables override values from the config file.
## Docker Compose
The scheduler can also be run with Docker Compose. Example `docker-compose.yml`:
```yaml
version: '3.8'
services:
synapse-backupper:
image: synapse-backupper:latest
volumes:
- ./backups:/backups
- ./keys:/keys:ro
environment:
- APP_PG_HOST=synapse-pg
- APP_PG_DATABASE=synapse
- APP_PG_USER=synapse
- APP_PG_PASSWORD=secret
- APP_PQ_PUBLIC_KEY_PATH=/keys/synapse.pq.pub.pem
- APP_CLASSICAL_PUBLIC_KEY_PATH=/keys/synapse.classical.pub.pem
- APP_BACKUP_DIR=/backups
- APP_BACKUP_CRON=0 0 3 * * *
healthcheck:
test: ["CMD", "wget", "-qO-", "http://localhost:8080/healthz"]
interval: 30s
timeout: 3s
start_period: 10s
restart: unless-stopped
```
> **Important:** Only public keys (`*.pub.pem`) should be present in `./keys`. Move private keys (`*.priv.pem`) to an offline or restore-only host before starting the container.
Start the scheduler with:
```bash
docker compose up -d
```
## Security Model
### Defence in depth
Backups are encrypted with a **composite** scheme: a shared secret is derived independently from both the post-quantum and classical KEMs, and the payload is encrypted with a symmetric key derived from both secrets. An attacker must break **both** KEMs to recover the data.
### AND model
Decryption requires **both** private keys. The `restore` command will fail if either key is missing or incorrect. This prevents a single compromised key from exposing backups.
### Plug-and-play KEM
KEM implementations are registered in a runtime registry by scheme ID. New post-quantum or classical algorithms can be added without changing the backup or restore logic.
### Private key isolation
Private keys are **not** needed for backup creation. The backup container only mounts public keys (`:ro`). Private keys should live on a separate restore host or offline storage, reducing the blast radius of a backup-server compromise.
## Limitations
This is an MVP release. The following features are **not** implemented:
- **S3 / object storage** — backups are written to a local directory only.
- **Kubernetes deployment** — no Helm chart or operator is provided.
- **Streaming / incremental backups** — each run produces a full `pg_dump`.
- **Multi-database support** — a single database per instance is assumed.
## Development
Run unit tests:
```bash
make test
```
Run the end-to-end integration test (requires Docker):
```bash
make integration-test
```
Cross-compile:
```bash
make cross-build
```
## License
MIT
README.ru.md
+229
View File
@@ -0,0 +1,229 @@
# Synapse Backupper
Инструмент для резервного копирования PostgreSQL [Synapse](https://github.com/element-hq/synapse) с композитным шифрованием на основе двух механизмов обмена ключами (dual-KEM).
## Обзор
`synapse-backupper` создает зашифрованные дампы PostgreSQL с использованием **композитной** схемы шифрования, которая объединяет постквантовый KEM (ML-KEM-768) и классический KEM (X25519). Для расшифровки резервной копии требуются оба ключа (модель AND). Инструмент поддерживает два режима работы: разовое резервное копирование (`backup`) и планировщик (`run`), который выполняет копирование по расписанию cron.
Копии хранятся локально на диске; устаревшие копии автоматически удаляются по возрасту.
## Установка
Сборка Docker-образа:
```bash
make docker-build
```
В результате собирается образ `synapse-backupper:latest`. Внутри образа находится скомпилированный бинарник, клиентские утилиты PostgreSQL, а сам контейнер запускается от непривилегированного пользователя.
## Быстрый старт
### 1. Генерация ключей шифрования
Сгенерируйте обе пары ключей (постквантовую и классическую) с помощью подкоманды `keygen`:
```bash
mkdir -p ./keys
docker run --rm -u root -v "$(pwd)/keys:/keys" synapse-backupper:latest \
keygen --type both --out-prefix /keys/synapse
chown -R "$(id -u):$(id -g)" ./keys
```
> **Примечание:** Образ контейнера по умолчанию запускается от непривилегированного пользователя `app`. Подкоманда `keygen` должна писать в смонтированный с хоста каталог, поэтому здесь она запускается от `root`, а затем владение файлами возвращается текущему пользователю. Если вы не работаете от `root`, скорректируйте команду `chown` (например, через `sudo`).
В каталоге `./keys/` появятся четыре файла:
| Файл | Назначение |
|------|------------|
| `synapse.pq.pub.pem` | Постквантовый публичный ключ (для резервного копирования) |
| `synapse.pq.priv.pem` | Постквантовый приватный ключ (для восстановления, хранить офлайн) |
| `synapse.classical.pub.pem` | Классический публичный ключ (для резервного копирования) |
| `synapse.classical.priv.pem` | Классический приватный ключ (для восстановления, хранить офлайн) |
**Важно:** приватные ключи никогда не следует монтировать в контейнер, который делает резервные копии. Храните их офлайн или на отдельном хосте, предназначенном только для восстановления.
### 2. Режимы работы
#### Режим планировщика (`run`)
Команда по умолчанию запускает планировщик по расписанию cron и HTTP-эндпоинт для проверки здоровья:
```bash
docker run -d -u root \
--name synapse-backupper \
-v "$(pwd)/keys:/keys:ro" \
-v "$(pwd)/backups:/backups" \
-e APP_PG_HOST=db \
-e APP_PG_DATABASE=postgres \
-e APP_PG_USER=synapse \
-e APP_PG_PASSWORD=changeme \
-e APP_PQ_PUBLIC_KEY_PATH=/keys/synapse.pq.pub.pem \
-e APP_CLASSICAL_PUBLIC_KEY_PATH=/keys/synapse.classical.pub.pem \
-e APP_BACKUP_DIR=/backups \
synapse-backupper:latest
```
По умолчанию резервное копирование выполняется ежедневно в `03:00`. Расписание задается параметром `backup.cron` в конфигурационном файле или переменной окружения `APP_BACKUP_CRON`.
#### Разовое резервное копирование (`backup`)
Выполнить одну резервную копию немедленно:
```bash
docker run --rm -u root \
-v "$(pwd)/keys:/keys:ro" \
-v "$(pwd)/backups:/backups" \
-e APP_PG_HOST=5432 \
-e APP_PG_DATABASE=postgres \
-e APP_PG_USER=synapse \
-e APP_PG_PASSWORD=changeme \
-e APP_PQ_PUBLIC_KEY_PATH=/keys/synapse.pq.pub.pem \
-e APP_CLASSICAL_PUBLIC_KEY_PATH=/keys/synapse.classical.pub.pem \
-e APP_BACKUP_DIR=/backups \
synapse-backupper:latest backup
```
Результирующий файл получает имя `synapse-<timestamp>.dump.pqenc` и сохраняется в каталог резервных копий. Дамп создается в формате PostgreSQL custom (`--format=custom`), который ожидает `pg_restore`.
### 3. Восстановление на отдельном хосте
Процедура восстановления специально выполняется на хосте, который **не** имеет доступа к базе данных. Достаточно только приватных ключей.
```bash
docker run --rm \
-v "$(pwd)/keys:/keys:ro" \
-v "$(pwd)/backups:/backups:ro" \
-v "$(pwd)/restored:/out" \
synapse-backupper:latest restore \
--in /backups/synapse-20260102-150405.dump.pqenc \
--privkey-pq /keys/synapse.pq.priv.pem \
--privkey-classical /keys/synapse.classical.priv.pem \
--out /out/restored.dump
```
После этого восстановите расшифрованный дамп стандартными средствами PostgreSQL:
```bash
pg_restore --clean --if-exists --dbname=synapse restored.dump
```
## Конфигурация
Приоритет источников настроек: флаги командной строки → переменные окружения `APP_*` → файл конфигурации → значения по умолчанию.
Примеры в разделе «Быстрый старт» используют переменные окружения. Префикс — `APP_`, вложенные ключи разделяются подчеркиванием, например:
- `APP_PG_HOST`
- `APP_PG_DATABASE`
- `APP_BACKUP_DIR`
- `APP_PQ_PUBLIC_KEY_PATH`
- `APP_CLASSICAL_PUBLIC_KEY_PATH`
Чтобы использовать файл конфигурации, сгенерируйте пример и смонтируйте его в контейнер:
```bash
docker run --rm synapse-backupper:latest generate-config --lang ru > config.yaml
```
Отредактируйте `config.yaml`, затем смонтируйте его в `/config.yaml` при запуске контейнера:
```bash
docker run -d \
--name synapse-backupper \
-v "$(pwd)/config.yaml:/config.yaml" \
-v "$(pwd)/keys:/keys:ro" \
-v "$(pwd)/backups:/backups" \
synapse-backupper:latest
```
Переменные окружения переопределяют значения из файла конфигурации.
## Docker Compose
Планировщик можно запустить через Docker Compose. Пример `docker-compose.yml`:
```yaml
version: '3.8'
services:
synapse-backupper:
image: synapse-backupper:latest
volumes:
- ./backups:/backups
- ./keys:/keys:ro
environment:
- APP_PG_HOST=synapse-pg
- APP_PG_DATABASE=synapse
- APP_PG_USER=synapse
- APP_PG_PASSWORD=secret
- APP_PQ_PUBLIC_KEY_PATH=/keys/synapse.pq.pub.pem
- APP_CLASSICAL_PUBLIC_KEY_PATH=/keys/synapse.classical.pub.pem
- APP_BACKUP_DIR=/backups
- APP_BACKUP_CRON=0 0 3 * * *
healthcheck:
test: ["CMD", "wget", "-qO-", "http://localhost:8080/healthz"]
interval: 30s
timeout: 3s
start_period: 10s
restart: unless-stopped
```
> **Важно:** В каталоге `./keys` должны находиться только публичные ключи (`*.pub.pem`). Перед запуском контейнера переместите приватные ключи (`*.priv.pem`) на офлайн-хост или хост, предназначенный только для восстановления.
Запуск планировщика:
```bash
docker compose up -d
```
## Модель безопасности
### Многоуровневая защита
Резервные копии шифруются по **композитной** схеме: общий секрет вычисляется независимо от постквантового и классического KEM, а полезная нагрузка шифруется симметричным ключом, полученным из обоих секретов. Злоумышленник должен взломать **оба** механизма, чтобы получить данные.
### Модель AND
Для расшифровки требуются **оба** приватных ключа. Команда `restore` завершится с ошибкой, если хотя бы один ключ отсутствует или неверен. Это исключает компрометацию копий при компрометации одного ключа.
### Plug-and-play KEM
Реализации KEM регистрируются в runtime-реестре по идентификатору схемы. Новые постквантовые или классические алгоритмы можно добавлять без изменения логики резервного копирования и восстановления.
### Изоляция приватных ключей
Приватные ключи **не нужны** при создании резервной копии. Контейнер, выполняющий бэкап, монтирует только публичные ключи (`:ro`). Приватные ключи должны находиться на отдельном хосте восстановления или в офлайн-хранилище, что сужает зону поражения при компрометации сервера резервного копирования.
## Ограничения
Это MVP-релиз. Следующие возможности **не реализованы**:
- **S3 / объектное хранилище** — копии пишутся только в локальный каталог.
- **Развертывание в Kubernetes** — Helm-чарт и оператор не предоставляются.
- **Потоковое / инкрементальное резервное копирование** — каждый запуск создает полный `pg_dump`.
- **Поддержка нескольких баз данных** — предполагается одна база данных на инстанс.
## Разработка
Запуск unit-тестов:
```bash
make test
```
Запуск сквозного интеграционного теста (требуется Docker):
```bash
make integration-test
```
Кросс-компиляция:
```bash
make cross-build
```
## Лицензия
MIT
cmd/synapse-backupper/backup.go
+167
View File
@@ -0,0 +1,167 @@
package main
import (
"context"
"crypto/rand"
"fmt"
"io"
"log/slog"
"os"
"path/filepath"
"time"
"github.com/google/uuid"
"github.com/spf13/cobra"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/config"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/crypto/composite"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/crypto/keymanager"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/crypto/mlkem768"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/crypto/x25519"
pgdumpadapter "git.tswf.io/infra/go-synapse-backupper/pkg/adapters/pgdump"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/pipeline"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/retention"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/storage/local"
"git.tswf.io/infra/go-synapse-backupper/pkg/domain"
"git.tswf.io/infra/go-synapse-backupper/pkg/domain/crypto"
"git.tswf.io/infra/go-synapse-backupper/pkg/domain/pgdump"
)
var (
newKeyManager = keymanager.NewKeyManager
newLocalSink = local.NewLocalSink
newRunner = defaultNewRunner
outputWriter io.Writer = os.Stderr
)
type pipelineRunner interface {
Run(
ctx context.Context,
pgDumpOpts pgdump.Options,
recipients []crypto.RecipientPub,
sink domain.Sink,
rand io.Reader,
) error
}
func defaultNewRunner(options ...pipeline.Option) pipelineRunner {
return pipeline.NewRunner(options...)
}
func init() {
config.RegisterFlags(backupCmd)
}
var backupCmd = &cobra.Command{
Use: "backup",
Short: "Run a one-off backup",
RunE: func(cmd *cobra.Command, args []string) error {
ctx := cmd.Context()
if ctx == nil {
ctx = context.Background()
}
startTime := time.Now().UTC()
runID, err := uuid.NewRandom()
if err != nil {
return fmt.Errorf("generate run-id: %w", err)
}
logger := slog.New(slog.NewTextHandler(outputWriter, nil))
logger.Info(
"backup started",
slog.String("run_id", runID.String()),
slog.Time("start_time", startTime),
)
cfg, err := config.Load(cmd)
if err != nil {
return fmt.Errorf("load config: %w", err)
}
keyManager := newKeyManager(crypto.NewRegistry())
pqPub, err := keyManager.LoadPub(cfg.PQPublicKeyPath, cfg.PQScheme)
if err != nil {
return fmt.Errorf("load PQ public key: %w", err)
}
classicalPub, err := keyManager.LoadPub(cfg.ClassicalPublicKeyPath, cfg.ClassicalScheme)
if err != nil {
return fmt.Errorf("load classical public key: %w", err)
}
recipients := []crypto.RecipientPub{pqPub, classicalPub}
pgDumpOpts := pgdump.Options{
Host: cfg.PG.Host,
Port: cfg.PG.Port,
Database: cfg.PG.Database,
User: cfg.PG.User,
Password: cfg.PG.Password,
Key: fmt.Sprintf("synapse-%s.dump.pqenc", startTime.Format("20060102-150405")),
ExcludeTables: cfg.PG.ExcludeTables,
}
finalPath := filepath.Join(cfg.Backup.Dir, pgDumpOpts.Key)
logger.Info(
"backup destination",
slog.String("backup_dir", cfg.Backup.Dir),
slog.String("output_path", finalPath),
)
sink := newLocalSink(cfg.Backup.Dir)
registry := crypto.NewRegistry()
_ = registry.Register(0x0006, func() crypto.KEM { return mlkem768.New() })
_ = registry.Register(0x0007, func() crypto.KEM { return x25519.New() })
runner := newRunner(
pipeline.WithDumper(pgdumpadapter.New()),
pipeline.WithEncryptor(composite.NewEncryptor(registry)),
)
err = runner.Run(ctx, pgDumpOpts, recipients, sink, rand.Reader)
endTime := time.Now().UTC()
var byteCount int64
if info, statErr := os.Stat(finalPath); statErr == nil {
byteCount = info.Size()
}
if err != nil {
logger.Error(
"backup failed",
slog.String("run_id", runID.String()),
slog.String("output_path", finalPath),
slog.Time("start_time", startTime),
slog.Time("end_time", endTime),
slog.Int64("byte_count", byteCount),
slog.String("error", err.Error()),
)
return err
}
logger.Info(
"backup completed",
slog.String("run_id", runID.String()),
slog.String("output_path", finalPath),
slog.Time("start_time", startTime),
slog.Time("end_time", endTime),
slog.Int64("byte_count", byteCount),
)
if _, pruneErr := retention.PruneByAge(
ctx,
cfg.Backup.Dir,
cfg.Backup.RetentionDays,
time.Now(),
); pruneErr != nil {
logger.Error(
"retention pruning failed",
slog.String("error", pruneErr.Error()),
)
return pruneErr
}
return nil
},
}
cmd/synapse-backupper/backup_test.go
+367
View File
@@ -0,0 +1,367 @@
package main
import (
"bytes"
"context"
"io"
"os"
"path/filepath"
"regexp"
"strings"
"testing"
"time"
"github.com/spf13/cobra"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/config"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/pipeline"
"git.tswf.io/infra/go-synapse-backupper/pkg/domain/crypto"
"git.tswf.io/infra/go-synapse-backupper/pkg/domain/pgdump"
)
// --- Mocks ---
type mockRecipientPub struct {
schemeID uint16
keyID []byte
raw []byte
}
func (m *mockRecipientPub) SchemeID() uint16 { return m.schemeID }
func (m *mockRecipientPub) KeyID() []byte { return m.keyID }
func (m *mockRecipientPub) Raw() []byte { return m.raw }
type mockKeyManager struct {
loadPubCalls []loadPubCall
pub crypto.RecipientPub
}
type loadPubCall struct {
path string
schemeID uint16
}
func (m *mockKeyManager) LoadPub(
path string,
schemeID uint16,
) (
crypto.RecipientPub,
error,
) {
m.loadPubCalls = append(
m.loadPubCalls,
loadPubCall{path: path, schemeID: schemeID},
)
return m.pub, nil
}
func (m *mockKeyManager) LoadPriv(
path string,
schemeID uint16,
) (
crypto.RecipientPriv,
error,
) {
return nil, nil
}
func (m *mockKeyManager) Generate(
schemeID uint16,
pubOut io.Writer,
privOut io.Writer,
rand io.Reader,
) error {
return nil
}
type successDumper struct {
data []byte
receivedOpts pgdump.Options
}
func (d *successDumper) Dump(
ctx context.Context,
opts pgdump.Options,
sink io.Writer,
) error {
d.receivedOpts = opts
if len(d.data) > 0 {
_, err := sink.Write(d.data)
if err != nil {
return err
}
}
if closer, ok := sink.(io.Closer); ok {
_ = closer.Close()
}
return nil
}
type failDumper struct{}
func (d *failDumper) Dump(
ctx context.Context,
opts pgdump.Options,
sink io.Writer,
) error {
return pgdump.ErrPgDumpFailed(1)
}
type passthroughEncryptor struct{}
func (e *passthroughEncryptor) Encrypt(
plaintext io.Reader,
recipients []crypto.RecipientPub,
sink io.Writer,
rand io.Reader,
) error {
_, err := io.Copy(sink, plaintext)
return err
}
// --- Helpers ---
func setBackupFlags(
cmd *cobra.Command,
backupDir string,
pqPath string,
classicalPath string,
) {
config.RegisterFlags(cmd)
_ = cmd.Flags().Set("backup-dir", backupDir)
_ = cmd.Flags().Set("pq-public-key-path", pqPath)
_ = cmd.Flags().Set("classical-public-key-path", classicalPath)
_ = cmd.Flags().Set("pg-host", "localhost")
_ = cmd.Flags().Set("pg-port", "5432")
_ = cmd.Flags().Set("pg-user", "testuser")
_ = cmd.Flags().Set("pg-password", "testpass")
_ = cmd.Flags().Set("pg-database", "testdb")
}
func restoreGlobals(t *testing.T) {
originalNewKeyManager := newKeyManager
originalNewRunner := newRunner
originalOutputWriter := outputWriter
t.Cleanup(func() {
newKeyManager = originalNewKeyManager
newRunner = originalNewRunner
outputWriter = originalOutputWriter
})
}
// --- Tests ---
func TestBackupCmd_Structure(t *testing.T) {
if backupCmd == nil {
t.Fatal("backupCmd is nil")
}
if backupCmd.Use != "backup" {
t.Fatalf("expected Use='backup', got %q", backupCmd.Use)
}
}
func TestBackupCmd_Success(t *testing.T) {
restoreGlobals(t)
backupDir := t.TempDir()
pqPath := filepath.Join(backupDir, "pq.pub")
classicalPath := filepath.Join(backupDir, "classical.pub")
_ = os.WriteFile(pqPath, []byte("pq"), 0o644)
_ = os.WriteFile(classicalPath, []byte("classical"), 0o644)
mockKM := &mockKeyManager{
pub: &mockRecipientPub{
schemeID: 0x0006,
keyID: make([]byte, 8),
raw: make([]byte, 32),
},
}
newKeyManager = func(crypto.Registry) crypto.KeyManager {
return mockKM
}
testData := []byte("test backup payload")
dumper := &successDumper{data: testData}
newRunner = func(...pipeline.Option) pipelineRunner {
return pipeline.NewRunner(
pipeline.WithDumper(dumper),
pipeline.WithEncryptor(&passthroughEncryptor{}),
)
}
var buf bytes.Buffer
outputWriter = &buf
cmd := &cobra.Command{}
setBackupFlags(cmd, backupDir, pqPath, classicalPath)
_ = cmd.Flags().Set("backup-retention-days", "1")
oldFile := filepath.Join(backupDir, "synapse-20230101-000000.dump.pqenc")
_ = os.WriteFile(oldFile, []byte("old"), 0o644)
oldTime := time.Now().Add(-48 * time.Hour)
_ = os.Chtimes(oldFile, oldTime, oldTime)
err := backupCmd.RunE(cmd, []string{})
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if len(mockKM.loadPubCalls) != 2 {
t.Fatalf("expected 2 LoadPub calls, got %d", len(mockKM.loadPubCalls))
}
if mockKM.loadPubCalls[0].path != pqPath ||
mockKM.loadPubCalls[0].schemeID != 0x0006 {
t.Fatalf("unexpected PQ LoadPub call: %+v", mockKM.loadPubCalls[0])
}
if mockKM.loadPubCalls[1].path != classicalPath ||
mockKM.loadPubCalls[1].schemeID != 0x0007 {
t.Fatalf(
"unexpected classical LoadPub call: %+v",
mockKM.loadPubCalls[1],
)
}
if dumper.receivedOpts.Host != "localhost" {
t.Fatalf("unexpected host: %q", dumper.receivedOpts.Host)
}
if dumper.receivedOpts.Port != 5432 {
t.Fatalf("unexpected port: %d", dumper.receivedOpts.Port)
}
if dumper.receivedOpts.Database != "testdb" {
t.Fatalf("unexpected database: %q", dumper.receivedOpts.Database)
}
if dumper.receivedOpts.User != "testuser" {
t.Fatalf("unexpected user: %q", dumper.receivedOpts.User)
}
if dumper.receivedOpts.Password != "testpass" {
t.Fatalf("unexpected password: %q", dumper.receivedOpts.Password)
}
key := dumper.receivedOpts.Key
matched, _ := regexp.MatchString(
`^synapse-\d{8}-\d{6}\.dump\.pqenc$`,
key,
)
if !matched {
t.Fatalf("unexpected key format: %q", key)
}
entries, err := os.ReadDir(backupDir)
if err != nil {
t.Fatal(err)
}
var foundFinal int
var foundTmp int
for _, entry := range entries {
name := entry.Name()
if strings.HasSuffix(name, ".dump.pqenc") {
foundFinal++
}
if strings.HasSuffix(name, ".tmp") {
foundTmp++
}
}
if foundFinal != 1 {
t.Fatalf("expected 1 final .pqenc file, found %d", foundFinal)
}
if foundTmp != 0 {
t.Fatalf("expected 0 .tmp files, found %d", foundTmp)
}
if _, err := os.Stat(oldFile); !os.IsNotExist(err) {
t.Fatal("expected old file to be pruned by retention")
}
logStr := buf.String()
if !strings.Contains(logStr, "run_id=") {
t.Fatal("expected log to contain run_id")
}
if !strings.Contains(logStr, "start_time=") {
t.Fatal("expected log to contain start_time")
}
if !strings.Contains(logStr, "end_time=") {
t.Fatal("expected log to contain end_time")
}
if !strings.Contains(logStr, "byte_count=") {
t.Fatal("expected log to contain byte_count")
}
re := regexp.MustCompile(`byte_count=(\d+)`)
matches := re.FindAllStringSubmatch(logStr, -1)
if len(matches) == 0 {
t.Fatal("expected log to contain byte_count value")
}
lastMatch := matches[len(matches)-1][1]
if lastMatch == "0" {
t.Fatal("expected non-zero byte_count for successful backup")
}
}
func TestBackupCmd_PgDumpFailure(t *testing.T) {
restoreGlobals(t)
backupDir := t.TempDir()
pqPath := filepath.Join(backupDir, "pq.pub")
classicalPath := filepath.Join(backupDir, "classical.pub")
_ = os.WriteFile(pqPath, []byte("pq"), 0o644)
_ = os.WriteFile(classicalPath, []byte("classical"), 0o644)
mockKM := &mockKeyManager{
pub: &mockRecipientPub{
schemeID: 0x0006,
keyID: make([]byte, 8),
raw: make([]byte, 32),
},
}
newKeyManager = func(crypto.Registry) crypto.KeyManager {
return mockKM
}
newRunner = func(...pipeline.Option) pipelineRunner {
return pipeline.NewRunner(
pipeline.WithDumper(&failDumper{}),
pipeline.WithEncryptor(&passthroughEncryptor{}),
)
}
var buf bytes.Buffer
outputWriter = &buf
cmd := &cobra.Command{}
setBackupFlags(cmd, backupDir, pqPath, classicalPath)
err := backupCmd.RunE(cmd, []string{})
if err == nil {
t.Fatal("expected non-nil error")
}
entries, err := os.ReadDir(backupDir)
if err != nil {
t.Fatal(err)
}
for _, entry := range entries {
name := entry.Name()
if strings.HasSuffix(name, ".pqenc") || strings.HasSuffix(name, ".tmp") {
t.Fatalf("unexpected file after failure: %s", name)
}
}
logStr := buf.String()
if !strings.Contains(logStr, "backup failed") {
t.Fatal("expected 'backup failed' in log")
}
if !strings.Contains(logStr, "run_id=") {
t.Fatal("expected log to contain run_id")
}
if !strings.Contains(logStr, "start_time=") {
t.Fatal("expected log to contain start_time")
}
if !strings.Contains(logStr, "end_time=") {
t.Fatal("expected log to contain end_time")
}
if !strings.Contains(logStr, "byte_count=0") {
t.Fatal("expected log to contain byte_count=0 for failed backup")
}
}
cmd/synapse-backupper/generate_config.go
+47
View File
@@ -0,0 +1,47 @@
package main
import (
_ "embed"
"fmt"
"os"
"github.com/spf13/cobra"
)
//go:embed resources/config.en.yaml
var configEnYAML []byte
//go:embed resources/config.ru.yaml
var configRuYAML []byte
func generateConfigCmd() *cobra.Command {
var lang string
var output string
cmd := &cobra.Command{
Use: "generate-config",
Short: "Generate a commented example configuration file",
RunE: func(cmd *cobra.Command, args []string) error {
var tmpl []byte
switch lang {
case "en":
tmpl = configEnYAML
case "ru":
tmpl = configRuYAML
default:
return fmt.Errorf("unsupported language: %q (must be \"en\" or \"ru\")", lang)
}
if output != "" {
return os.WriteFile(output, tmpl, 0o644)
}
_, err := cmd.OutOrStdout().Write(tmpl)
return err
},
}
cmd.Flags().StringVar(&lang, "lang", "en", "Language for comments (en or ru)")
cmd.Flags().StringVar(&output, "output", "", "Output file path (empty = stdout)")
return cmd
}
cmd/synapse-backupper/generate_config_test.go
+234
View File
@@ -0,0 +1,234 @@
package main
import (
"bytes"
"os"
"path/filepath"
"strings"
"testing"
"github.com/spf13/viper"
)
func TestGenerateConfigCmd_Flags(t *testing.T) {
cmd := generateConfigCmd()
flags := cmd.Flags()
lang, err := flags.GetString("lang")
if err != nil {
t.Fatalf("failed to get --lang flag: %v", err)
}
if lang != "en" {
t.Errorf("--lang default = %q, want %q", lang, "en")
}
output, err := flags.GetString("output")
if err != nil {
t.Fatalf("failed to get --output flag: %v", err)
}
if output != "" {
t.Errorf("--output default = %q, want empty string", output)
}
}
func TestGenerateConfig_LangEn(t *testing.T) {
cmd := generateConfigCmd()
var buf bytes.Buffer
cmd.SetOut(&buf)
cmd.SetErr(&buf)
if err := cmd.Flags().Set("lang", "en"); err != nil {
t.Fatalf("Set flag failed: %v", err)
}
if err := cmd.Execute(); err != nil {
t.Fatalf("Execute failed: %v", err)
}
out := buf.String()
if !strings.Contains(out, "pg:") {
t.Errorf("output missing pg section")
}
if !strings.Contains(out, "host") {
t.Errorf("output missing host key")
}
if !strings.Contains(out, "PostgreSQL") {
t.Errorf("output missing English comment")
}
}
func TestGenerateConfig_LangRu(t *testing.T) {
cmd := generateConfigCmd()
var buf bytes.Buffer
cmd.SetOut(&buf)
cmd.SetErr(&buf)
if err := cmd.Flags().Set("lang", "ru"); err != nil {
t.Fatalf("Set flag failed: %v", err)
}
if err := cmd.Execute(); err != nil {
t.Fatalf("Execute failed: %v", err)
}
out := buf.String()
if !strings.Contains(out, "pg:") {
t.Errorf("output missing pg section")
}
if !strings.Contains(out, "хост") {
t.Errorf("output missing Russian comment (хост)")
}
}
func TestGenerateConfig_OutputFile(t *testing.T) {
tempDir := t.TempDir()
outputPath := filepath.Join(tempDir, "config.yaml")
cmd := generateConfigCmd()
var buf bytes.Buffer
cmd.SetOut(&buf)
cmd.SetErr(&buf)
if err := cmd.Flags().Set("output", outputPath); err != nil {
t.Fatalf("Set flag failed: %v", err)
}
if err := cmd.Execute(); err != nil {
t.Fatalf("Execute failed: %v", err)
}
data, err := os.ReadFile(outputPath)
if err != nil {
t.Fatalf("failed to read output file: %v", err)
}
if len(data) == 0 {
t.Errorf("output file is empty")
}
if buf.Len() != 0 {
t.Errorf("stdout not empty when --output set: %q", buf.String())
}
}
func TestGenerateConfig_Stdout(t *testing.T) {
cmd := generateConfigCmd()
var buf bytes.Buffer
cmd.SetOut(&buf)
cmd.SetErr(&buf)
if err := cmd.Execute(); err != nil {
t.Fatalf("Execute failed: %v", err)
}
out := buf.String()
if out == "" {
t.Errorf("stdout empty when no --output")
}
if !strings.Contains(out, "pg:") {
t.Errorf("stdout missing pg section")
}
}
func TestGenerateConfig_AllKeys(t *testing.T) {
cmd := generateConfigCmd()
var buf bytes.Buffer
cmd.SetOut(&buf)
cmd.SetErr(&buf)
if err := cmd.Execute(); err != nil {
t.Fatalf("Execute failed: %v", err)
}
out := buf.String()
requiredKeys := []string{
"host", "port", "user", "password", "database", "sslmode", "exclude_tables",
"dir", "retention_days", "cron",
"pq_scheme", "classical_scheme",
"pq_public_key_path", "classical_public_key_path",
"healthz", "log",
}
for _, key := range requiredKeys {
if !strings.Contains(out, key) {
t.Errorf("output missing key %q", key)
}
}
}
func TestGenerateConfig_RoundTrip(t *testing.T) {
tempDir := t.TempDir()
outputPath := filepath.Join(tempDir, "config.yaml")
cmd := generateConfigCmd()
if err := cmd.Flags().Set("output", outputPath); err != nil {
t.Fatalf("Set flag failed: %v", err)
}
if err := cmd.Execute(); err != nil {
t.Fatalf("Execute failed: %v", err)
}
v := viper.New()
v.SetConfigFile(outputPath)
if err := v.ReadInConfig(); err != nil {
t.Fatalf("viper read generated config failed: %v", err)
}
var cfg struct {
PG struct {
Host string `mapstructure:"host"`
Port int `mapstructure:"port"`
User string `mapstructure:"user"`
Password string `mapstructure:"password"`
Database string `mapstructure:"database"`
SSLMode string `mapstructure:"sslmode"`
ExcludeTables []string `mapstructure:"exclude_tables"`
} `mapstructure:"pg"`
Backup struct {
Dir string `mapstructure:"dir"`
RetentionDays int `mapstructure:"retention_days"`
Cron string `mapstructure:"cron"`
} `mapstructure:"backup"`
PQScheme uint16 `mapstructure:"pq_scheme"`
ClassicalScheme uint16 `mapstructure:"classical_scheme"`
PQPublicKeyPath string `mapstructure:"pq_public_key_path"`
ClassicalPublicKeyPath string `mapstructure:"classical_public_key_path"`
Healthz struct {
Port int `mapstructure:"port"`
} `mapstructure:"healthz"`
Log struct {
Level string `mapstructure:"level"`
} `mapstructure:"log"`
}
if err := v.Unmarshal(&cfg); err != nil {
t.Fatalf("viper unmarshal generated config failed: %v", err)
}
if cfg.PG.Port != 5432 {
t.Errorf("PG.Port = %d, want 5432", cfg.PG.Port)
}
if cfg.PG.SSLMode != "prefer" {
t.Errorf("PG.SSLMode = %q, want prefer", cfg.PG.SSLMode)
}
if len(cfg.PG.ExcludeTables) != 1 || cfg.PG.ExcludeTables[0] != "e2e_one_time_keys_json" {
t.Errorf("PG.ExcludeTables = %v, want [e2e_one_time_keys_json]", cfg.PG.ExcludeTables)
}
if cfg.Backup.RetentionDays != 180 {
t.Errorf("Backup.RetentionDays = %d, want 180", cfg.Backup.RetentionDays)
}
if cfg.Backup.Cron != "0 0 3 * * *" {
t.Errorf("Backup.Cron = %q, want 0 0 3 * * *", cfg.Backup.Cron)
}
if cfg.PQScheme != 0x0006 {
t.Errorf("PQScheme = 0x%04x, want 0x%04x", cfg.PQScheme, 0x0006)
}
if cfg.ClassicalScheme != 0x0007 {
t.Errorf("ClassicalScheme = 0x%04x, want 0x%04x", cfg.ClassicalScheme, 0x0007)
}
if cfg.Healthz.Port != 8080 {
t.Errorf("Healthz.Port = %d, want 8080", cfg.Healthz.Port)
}
if cfg.Log.Level != "info" {
t.Errorf("Log.Level = %q, want info", cfg.Log.Level)
}
}
cmd/synapse-backupper/keygen.go
+122
View File
@@ -0,0 +1,122 @@
package main
import (
"crypto/rand"
"fmt"
"os"
"github.com/spf13/cobra"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/crypto/keymanager"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/crypto/mlkem768"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/crypto/x25519"
"git.tswf.io/infra/go-synapse-backupper/pkg/domain/crypto"
)
func newRegistry() crypto.Registry {
reg := crypto.NewRegistry()
_ = reg.Register(0x0006, func() crypto.KEM { return mlkem768.New() })
_ = reg.Register(0x0007, func() crypto.KEM { return x25519.New() })
return reg
}
func newKeygenCmd() *cobra.Command {
return newKeygenCmdWithDeps(newRegistry())
}
func newKeygenCmdWithDeps(reg crypto.Registry) *cobra.Command {
var (
keyType string
outPrefix string
force bool
)
cmd := &cobra.Command{
Use: "keygen",
Short: "Generate encryption key pairs",
RunE: func(cmd *cobra.Command, args []string) error {
schemes := []struct {
name string
schemeID uint16
}{}
switch keyType {
case "pq":
schemes = append(schemes, struct {
name string
schemeID uint16
}{"pq", 0x0006})
case "classical":
schemes = append(schemes, struct {
name string
schemeID uint16
}{"classical", 0x0007})
case "both":
schemes = append(
schemes,
struct {
name string
schemeID uint16
}{"pq", 0x0006},
struct {
name string
schemeID uint16
}{"classical", 0x0007},
)
default:
return fmt.Errorf("invalid --type %q; must be pq, classical, or both", keyType)
}
if !force {
for _, s := range schemes {
pubPath := outPrefix + "." + s.name + ".pub.pem"
privPath := outPrefix + "." + s.name + ".priv.pem"
for _, p := range []string{pubPath, privPath} {
if _, err := os.Stat(p); err == nil {
return fmt.Errorf("file already exists: %s (use --force to overwrite)", p)
}
}
}
}
km := keymanager.NewKeyManager(reg)
for _, s := range schemes {
pubPath := outPrefix + "." + s.name + ".pub.pem"
privPath := outPrefix + "." + s.name + ".priv.pem"
pubFile, err := os.OpenFile(pubPath, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0o644)
if err != nil {
return fmt.Errorf("create public key file %s: %w", pubPath, err)
}
privFile, err := os.OpenFile(privPath, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0o600)
if err != nil {
_ = pubFile.Close()
return fmt.Errorf("create private key file %s: %w", privPath, err)
}
if err := km.Generate(s.schemeID, pubFile, privFile, rand.Reader); err != nil {
_ = pubFile.Close()
_ = privFile.Close()
return fmt.Errorf("generate %s keys: %w", s.name, err)
}
if err := pubFile.Close(); err != nil {
return fmt.Errorf("close public key file %s: %w", pubPath, err)
}
if err := privFile.Close(); err != nil {
return fmt.Errorf("close private key file %s: %w", privPath, err)
}
}
return nil
},
}
cmd.Flags().StringVar(&keyType, "type", "both", "Key type to generate (pq|classical|both)")
cmd.Flags().StringVar(&outPrefix, "out-prefix", "", "Output file path prefix")
cmd.Flags().BoolVar(&force, "force", false, "Overwrite existing files")
return cmd
}
cmd/synapse-backupper/keygen_test.go
+444
View File
@@ -0,0 +1,444 @@
package main
import (
"bytes"
"encoding/pem"
"os"
"path/filepath"
"testing"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/crypto/keymanager"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/crypto/mlkem768"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/crypto/x25519"
"git.tswf.io/infra/go-synapse-backupper/pkg/domain/crypto"
)
func makeRegistry(t *testing.T) crypto.Registry {
t.Helper()
reg := crypto.NewRegistry()
if err := reg.Register(0x0006, func() crypto.KEM { return mlkem768.New() }); err != nil {
t.Fatalf("register mlkem768: %v", err)
}
if err := reg.Register(0x0007, func() crypto.KEM { return x25519.New() }); err != nil {
t.Fatalf("register x25519: %v", err)
}
return reg
}
func TestKeygenCmdFlags(t *testing.T) {
cmd := newKeygenCmd()
// Verify flags exist and have correct defaults.
if cmd.Flag("type") == nil {
t.Fatal("missing --type flag")
}
if cmd.Flag("type").DefValue != "both" {
t.Errorf("--type default = %q, want both", cmd.Flag("type").DefValue)
}
if cmd.Flag("out-prefix") == nil {
t.Fatal("missing --out-prefix flag")
}
if cmd.Flag("out-prefix").DefValue != "" {
t.Errorf("--out-prefix default = %q, want empty", cmd.Flag("out-prefix").DefValue)
}
if cmd.Flag("force") == nil {
t.Fatal("missing --force flag")
}
if cmd.Flag("force").DefValue != "false" {
t.Errorf("--force default = %q, want false", cmd.Flag("force").DefValue)
}
}
func TestKeygenCmdInvalidType(t *testing.T) {
cmd := newKeygenCmd()
cmd.SetArgs([]string{"--type", "invalid", "--out-prefix", filepath.Join(t.TempDir(), "keys")})
err := cmd.Execute()
if err == nil {
t.Fatal("expected error for invalid --type")
}
}
func TestKeygenCmdBoth(t *testing.T) {
reg := makeRegistry(t)
dir := t.TempDir()
prefix := filepath.Join(dir, "keys")
cmd := newKeygenCmdWithDeps(reg)
cmd.SetArgs([]string{"--type", "both", "--out-prefix", prefix})
if err := cmd.Execute(); err != nil {
t.Fatalf("Execute failed: %v", err)
}
// Verify all 4 files exist.
files := []string{
prefix + ".pq.pub.pem",
prefix + ".pq.priv.pem",
prefix + ".classical.pub.pem",
prefix + ".classical.priv.pem",
}
for _, f := range files {
if _, err := os.Stat(f); err != nil {
t.Errorf("expected file %s to exist: %v", f, err)
}
}
}
func TestKeygenCmdPEMTypes(t *testing.T) {
reg := makeRegistry(t)
dir := t.TempDir()
prefix := filepath.Join(dir, "keys")
cmd := newKeygenCmdWithDeps(reg)
cmd.SetArgs([]string{"--type", "both", "--out-prefix", prefix})
if err := cmd.Execute(); err != nil {
t.Fatalf("Execute failed: %v", err)
}
tests := []struct {
path string
wantType string
wantKeyLen int
}{
{prefix + ".pq.pub.pem", "ML-KEM-768 PUBLIC KEY", 1184},
{prefix + ".pq.priv.pem", "ML-KEM-768 PRIVATE KEY", 64},
{prefix + ".classical.pub.pem", "X25519 PUBLIC KEY", 32},
{prefix + ".classical.priv.pem", "X25519 PRIVATE KEY", 32},
}
for _, tt := range tests {
data, err := os.ReadFile(tt.path)
if err != nil {
t.Fatalf("read %s: %v", tt.path, err)
}
block, _ := pem.Decode(data)
if block == nil {
t.Fatalf("failed to decode PEM from %s", tt.path)
}
if block.Type != tt.wantType {
t.Errorf("%s PEM type = %q, want %q", tt.path, block.Type, tt.wantType)
}
if len(block.Bytes) != tt.wantKeyLen {
t.Errorf("%s key len = %d, want %d", tt.path, len(block.Bytes), tt.wantKeyLen)
}
}
}
func TestKeygenCmdRoundTrip(t *testing.T) {
reg := makeRegistry(t)
dir := t.TempDir()
prefix := filepath.Join(dir, "keys")
cmd := newKeygenCmdWithDeps(reg)
cmd.SetArgs([]string{"--type", "both", "--out-prefix", prefix})
if err := cmd.Execute(); err != nil {
t.Fatalf("Execute failed: %v", err)
}
km := keymanager.NewKeyManager(reg)
schemes := []struct {
pubPath string
privPath string
schemeID uint16
pubLen int
privLen int
}{
{prefix + ".pq.pub.pem", prefix + ".pq.priv.pem", 0x0006, 1184, 64},
{prefix + ".classical.pub.pem", prefix + ".classical.priv.pem", 0x0007, 32, 32},
}
for _, s := range schemes {
pub, err := km.LoadPub(s.pubPath, s.schemeID)
if err != nil {
t.Fatalf("LoadPub %s: %v", s.pubPath, err)
}
priv, err := km.LoadPriv(s.privPath, s.schemeID)
if err != nil {
t.Fatalf("LoadPriv %s: %v", s.privPath, err)
}
if pub.SchemeID() != s.schemeID {
t.Errorf("pub.SchemeID() = 0x%04x, want 0x%04x", pub.SchemeID(), s.schemeID)
}
if priv.SchemeID() != s.schemeID {
t.Errorf("priv.SchemeID() = 0x%04x, want 0x%04x", priv.SchemeID(), s.schemeID)
}
if len(pub.Raw()) != s.pubLen {
t.Errorf("pub.Raw() len = %d, want %d", len(pub.Raw()), s.pubLen)
}
if len(priv.Raw()) != s.privLen {
t.Errorf("priv.Raw() len = %d, want %d", len(priv.Raw()), s.privLen)
}
if len(pub.KeyID()) != 8 {
t.Errorf("pub.KeyID() len = %d, want 8", len(pub.KeyID()))
}
if len(priv.KeyID()) != 8 {
t.Errorf("priv.KeyID() len = %d, want 8", len(priv.KeyID()))
}
// Verify the raw bytes round-trip correctly by checking the PEM
// contents match what LoadPub/LoadPriv return.
pubPEM, err := os.ReadFile(s.pubPath)
if err != nil {
t.Fatalf("read pub PEM: %v", err)
}
pubBlock, _ := pem.Decode(pubPEM)
if pubBlock == nil {
t.Fatal("failed to decode pub PEM")
}
if !bytes.Equal(pub.Raw(), pubBlock.Bytes) {
t.Errorf("pub.Raw() does not match PEM bytes")
}
privPEM, err := os.ReadFile(s.privPath)
if err != nil {
t.Fatalf("read priv PEM: %v", err)
}
privBlock, _ := pem.Decode(privPEM)
if privBlock == nil {
t.Fatal("failed to decode priv PEM")
}
if !bytes.Equal(priv.Raw(), privBlock.Bytes) {
t.Errorf("priv.Raw() does not match PEM bytes")
}
}
}
func TestKeygenCmdPQOnly(t *testing.T) {
reg := makeRegistry(t)
dir := t.TempDir()
prefix := filepath.Join(dir, "keys")
cmd := newKeygenCmdWithDeps(reg)
cmd.SetArgs([]string{"--type", "pq", "--out-prefix", prefix})
if err := cmd.Execute(); err != nil {
t.Fatalf("Execute failed: %v", err)
}
if _, err := os.Stat(prefix + ".pq.pub.pem"); err != nil {
t.Errorf("expected pq.pub.pem to exist: %v", err)
}
if _, err := os.Stat(prefix + ".pq.priv.pem"); err != nil {
t.Errorf("expected pq.priv.pem to exist: %v", err)
}
if _, err := os.Stat(prefix + ".classical.pub.pem"); !os.IsNotExist(err) {
t.Errorf("expected classical.pub.pem to NOT exist")
}
if _, err := os.Stat(prefix + ".classical.priv.pem"); !os.IsNotExist(err) {
t.Errorf("expected classical.priv.pem to NOT exist")
}
}
func TestKeygenCmdClassicalOnly(t *testing.T) {
reg := makeRegistry(t)
dir := t.TempDir()
prefix := filepath.Join(dir, "keys")
cmd := newKeygenCmdWithDeps(reg)
cmd.SetArgs([]string{"--type", "classical", "--out-prefix", prefix})
if err := cmd.Execute(); err != nil {
t.Fatalf("Execute failed: %v", err)
}
if _, err := os.Stat(prefix + ".classical.pub.pem"); err != nil {
t.Errorf("expected classical.pub.pem to exist: %v", err)
}
if _, err := os.Stat(prefix + ".classical.priv.pem"); err != nil {
t.Errorf("expected classical.priv.pem to exist: %v", err)
}
if _, err := os.Stat(prefix + ".pq.pub.pem"); !os.IsNotExist(err) {
t.Errorf("expected pq.pub.pem to NOT exist")
}
if _, err := os.Stat(prefix + ".pq.priv.pem"); !os.IsNotExist(err) {
t.Errorf("expected pq.priv.pem to NOT exist")
}
}
func TestKeygenCmdPermissions(t *testing.T) {
reg := makeRegistry(t)
dir := t.TempDir()
prefix := filepath.Join(dir, "keys")
cmd := newKeygenCmdWithDeps(reg)
cmd.SetArgs([]string{"--type", "both", "--out-prefix", prefix})
if err := cmd.Execute(); err != nil {
t.Fatalf("Execute failed: %v", err)
}
pubFiles := []string{prefix + ".pq.pub.pem", prefix + ".classical.pub.pem"}
for _, f := range pubFiles {
info, err := os.Stat(f)
if err != nil {
t.Fatalf("stat %s: %v", f, err)
}
mode := info.Mode().Perm()
if mode != 0o644 {
t.Errorf("%s permissions = 0%o, want 0644", f, mode)
}
}
privFiles := []string{prefix + ".pq.priv.pem", prefix + ".classical.priv.pem"}
for _, f := range privFiles {
info, err := os.Stat(f)
if err != nil {
t.Fatalf("stat %s: %v", f, err)
}
mode := info.Mode().Perm()
if mode != 0o600 {
t.Errorf("%s permissions = 0%o, want 0600", f, mode)
}
}
}
func TestKeygenCmdNoOverwrite(t *testing.T) {
reg := makeRegistry(t)
dir := t.TempDir()
prefix := filepath.Join(dir, "keys")
// Create an existing file.
existing := prefix + ".pq.pub.pem"
if err := os.WriteFile(existing, []byte("existing"), 0o644); err != nil {
t.Fatalf("write existing file: %v", err)
}
cmd := newKeygenCmdWithDeps(reg)
cmd.SetArgs([]string{"--type", "both", "--out-prefix", prefix})
err := cmd.Execute()
if err == nil {
t.Fatal("expected error when file exists without --force")
}
// Verify existing file was not overwritten.
data, err := os.ReadFile(existing)
if err != nil {
t.Fatalf("read existing file: %v", err)
}
if !bytes.Equal(data, []byte("existing")) {
t.Error("existing file was overwritten without --force")
}
}
func TestKeygenCmdForceOverwrite(t *testing.T) {
reg := makeRegistry(t)
dir := t.TempDir()
prefix := filepath.Join(dir, "keys")
// Create an existing file.
existing := prefix + ".pq.pub.pem"
if err := os.WriteFile(existing, []byte("existing"), 0o644); err != nil {
t.Fatalf("write existing file: %v", err)
}
cmd := newKeygenCmdWithDeps(reg)
cmd.SetArgs([]string{"--type", "both", "--out-prefix", prefix, "--force"})
if err := cmd.Execute(); err != nil {
t.Fatalf("Execute failed: %v", err)
}
// Verify file was overwritten with valid PEM.
data, err := os.ReadFile(existing)
if err != nil {
t.Fatalf("read overwritten file: %v", err)
}
block, _ := pem.Decode(data)
if block == nil {
t.Fatal("overwritten file is not valid PEM")
}
if block.Type != "ML-KEM-768 PUBLIC KEY" {
t.Errorf("overwritten PEM type = %q, want ML-KEM-768 PUBLIC KEY", block.Type)
}
}
func TestKeygenCmdKeyIDConsistency(t *testing.T) {
reg := makeRegistry(t)
dir := t.TempDir()
prefix := filepath.Join(dir, "keys")
cmd := newKeygenCmdWithDeps(reg)
cmd.SetArgs([]string{"--type", "both", "--out-prefix", prefix})
if err := cmd.Execute(); err != nil {
t.Fatalf("Execute failed: %v", err)
}
km := keymanager.NewKeyManager(reg)
schemes := []struct {
pubPath string
privPath string
schemeID uint16
pubLen int
privLen int
}{
{prefix + ".pq.pub.pem", prefix + ".pq.priv.pem", 0x0006, 1184, 64},
{prefix + ".classical.pub.pem", prefix + ".classical.priv.pem", 0x0007, 32, 32},
}
for _, s := range schemes {
pub, err := km.LoadPub(s.pubPath, s.schemeID)
if err != nil {
t.Fatalf("LoadPub %s: %v", s.pubPath, err)
}
priv, err := km.LoadPriv(s.privPath, s.schemeID)
if err != nil {
t.Fatalf("LoadPriv %s: %v", s.privPath, err)
}
if len(pub.Raw()) != s.pubLen {
t.Errorf("pub %s raw len = %d, want %d", s.pubPath, len(pub.Raw()), s.pubLen)
}
if len(priv.Raw()) != s.privLen {
t.Errorf("priv %s raw len = %d, want %d", s.privPath, len(priv.Raw()), s.privLen)
}
// KeyID must be present and 8 bytes for both pub and priv.
if len(pub.KeyID()) != 8 {
t.Errorf("pub %s KeyID len = %d, want 8", s.pubPath, len(pub.KeyID()))
}
if len(priv.KeyID()) != 8 {
t.Errorf("priv %s KeyID len = %d, want 8", s.privPath, len(priv.KeyID()))
}
}
}
func TestKeygenCmdEmptyPrefix(t *testing.T) {
reg := makeRegistry(t)
dir := t.TempDir()
cmd := newKeygenCmdWithDeps(reg)
cmd.SetArgs([]string{"--type", "pq"})
cmd.SetOut(nil)
cmd.SetErr(nil)
// Change working directory to temp dir so empty prefix creates files there.
origWd, _ := os.Getwd()
if err := os.Chdir(dir); err != nil {
t.Fatalf("Chdir(%q): %v", dir, err)
}
defer func() {
if err := os.Chdir(origWd); err != nil {
t.Fatalf("Chdir(%q): %v", origWd, err)
}
}()
if err := cmd.Execute(); err != nil {
t.Fatalf("Execute failed: %v", err)
}
if _, err := os.Stat(filepath.Join(dir, ".pq.pub.pem")); err != nil {
t.Errorf("expected .pq.pub.pem to exist: %v", err)
}
if _, err := os.Stat(filepath.Join(dir, ".pq.priv.pem")); err != nil {
t.Errorf("expected .pq.priv.pem to exist: %v", err)
}
}
cmd/synapse-backupper/main.go
+32
View File
@@ -0,0 +1,32 @@
package main
import (
"context"
"fmt"
"os"
"os/signal"
"syscall"
"github.com/spf13/cobra"
)
func main() {
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
rootCmd := &cobra.Command{
Use: "synapse-backupper",
Short: "Synapse database backup tool",
}
rootCmd.SetContext(ctx)
rootCmd.AddCommand(backupCmd)
rootCmd.AddCommand(restoreCmd)
rootCmd.AddCommand(runCmd)
rootCmd.AddCommand(newKeygenCmd())
rootCmd.AddCommand(generateConfigCmd())
if err := rootCmd.ExecuteContext(ctx); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}
cmd/synapse-backupper/resources/config.en.yaml
+66
View File
@@ -0,0 +1,66 @@
# Synapse Backupper configuration file
# Generated by: synapse-backupper generate-config --lang en
# Dotted key example: pg.host
# PostgreSQL connection settings
pg:
# host: string — PostgreSQL server hostname or IP address.
host: "localhost"
# port: integer — PostgreSQL server port.
port: 5432
# user: string — PostgreSQL username for the backup connection.
user: ""
# password: string — PostgreSQL password for the backup connection.
# Do not commit real passwords to version control.
password: ""
# database: string — Name of the Synapse database to back up.
database: ""
# sslmode: string — PostgreSQL SSL mode (disable, allow, prefer, require, verify-ca, verify-full).
sslmode: "prefer"
# exclude_tables: list of strings — Tables to skip during pg_dump.
# The default excludes the large one-time-keys table to reduce dump size.
exclude_tables:
- "e2e_one_time_keys_json"
# Backup scheduling and retention settings
backup:
# dir: string — Directory where encrypted backup files are stored.
dir: ""
# retention_days: integer — How many days to keep backups before pruning.
retention_days: 180
# cron: string — Cron expression for automatic backup schedule.
cron: "0 0 3 * * *"
# Encryption scheme identifiers
# pq_scheme: uint16 — Post-quantum KEM scheme ID.
# 0x0006 = ML-KEM-768 (NIST FIPS 203)
pq_scheme: 0x0006
# classical_scheme: uint16 — Classical KEM scheme ID.
# 0x0007 = X25519 ECDH
classical_scheme: 0x0007
# Public key file paths for hybrid encryption
# pq_public_key_path: string — Path to the post-quantum public key PEM file.
pq_public_key_path: ""
# classical_public_key_path: string — Path to the classical public key PEM file.
classical_public_key_path: ""
# Health check HTTP server settings
healthz:
# port: integer — TCP port for the /healthz endpoint.
port: 8080
# Logging settings
log:
# level: string — Log verbosity (debug, info, warn, error).
level: "info"
cmd/synapse-backupper/resources/config.ru.yaml
+65
View File
@@ -0,0 +1,65 @@
# Файл конфигурации Synapse Backupper
# Сгенерировано командой: synapse-backupper generate-config --lang ru
# Настройки подключения к PostgreSQL
pg:
# host: строка — имя хоста или IP-адрес сервера PostgreSQL.
host: "localhost"
# port: целое число — порт сервера PostgreSQL.
port: 5430
# user: строка — имя пользователя PostgreSQL для резервного копирования.
user: "synapse"
# password: строка — пароль пользователя PostgreSQL.
# Не сохраняйте настоящие пароли в системе контроля версий.
password: "changeme"
# database: строка — имя базы данных Synapse для резервного копирования.
database: "postgres"
# sslmode: строка — режим SSL PostgreSQL (disable, allow, prefer, require, verify-ca, verify-full).
sslmode: "prefer"
# exclude_tables: список строк — таблицы, которые следует пропустить при pg_dump.
# По умолчанию исключается большая таблица одноразовых ключей для уменьшения размера дампа.
exclude_tables:
- "e2e_one_time_keys_json"
# Настройки расписания и хранения резервных копий
backup:
# dir: строка — каталог для хранения зашифрованных файлов резервных копий.
dir: "backups"
# retention_days: целое число — сколько дней хранить резервные копии перед удалением.
retention_days: 180
# cron: строка — выражение cron для автоматического расписания резервного копирования.
cron: "0 0 3 * * *"
# Идентификаторы схем шифрования
# pq_scheme: uint16 — идентификатор постквантовой схемы KEM.
# 0x0006 = ML-KEM-768 (NIST FIPS 203)
pq_scheme: 0x0006
# classical_scheme: uint16 — идентификатор классической схемы KEM.
# 0x0007 = X25519 ECDH
classical_scheme: 0x0007
# Пути к файлам открытых ключей для гибридного шифрования
# pq_public_key_path: строка — путь к PEM-файлу постквантового открытого ключа.
pq_public_key_path: ""
# classical_public_key_path: строка — путь к PEM-файлу классического открытого ключа.
classical_public_key_path: ""
# Настройки HTTP-сервера проверки состояния
healthz:
# port: целое число — TCP-порт для эндпоинта /healthz.
port: 8080
# Настройки журналирования
log:
# level: строка — уровень детализации журнала (debug, info, warn, error).
level: "info"
cmd/synapse-backupper/restore.go
+122
View File
@@ -0,0 +1,122 @@
package main
import (
"fmt"
"io"
"os"
"github.com/spf13/cobra"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/crypto/composite"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/crypto/keymanager"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/crypto/mlkem768"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/crypto/x25519"
"git.tswf.io/infra/go-synapse-backupper/pkg/domain/crypto"
)
var (
newRestoreKeyManager = keymanager.NewKeyManager
newRestoreDecryptor = func(registry crypto.Registry) crypto.Decryptor {
return composite.NewDecryptor(registry)
}
restoreOutput io.Writer = os.Stdout
restoreOsOpen = os.Open
)
var restoreCmd = &cobra.Command{
Use: "restore",
Short: "Restore a backup from a .pqenc file",
PreRunE: func(cmd *cobra.Command, args []string) error {
privkeyPq, _ := cmd.Flags().GetString("privkey-pq")
privkeyClassical, _ := cmd.Flags().GetString("privkey-classical")
var pqMissing bool
if _, err := os.Stat(privkeyPq); err != nil {
pqMissing = true
}
var classicalMissing bool
if _, err := os.Stat(privkeyClassical); err != nil {
classicalMissing = true
}
if pqMissing || classicalMissing {
return fmt.Errorf(
"--privkey-pq and --privkey-classical are both required (AND model)",
)
}
return nil
},
RunE: func(cmd *cobra.Command, args []string) error {
inPath, _ := cmd.Flags().GetString("in")
outPath, _ := cmd.Flags().GetString("out")
privkeyPq, _ := cmd.Flags().GetString("privkey-pq")
privkeyClassical, _ := cmd.Flags().GetString("privkey-classical")
inFile, err := restoreOsOpen(inPath)
if err != nil {
return fmt.Errorf("open input file: %w", err)
}
defer inFile.Close()
registry := crypto.NewRegistry()
_ = registry.Register(
0x0006,
func() crypto.KEM { return mlkem768.New() },
)
_ = registry.Register(
0x0007,
func() crypto.KEM { return x25519.New() },
)
keyManager := newRestoreKeyManager(registry)
pqPriv, err := keyManager.LoadPriv(privkeyPq, 0x0006)
if err != nil {
return fmt.Errorf("load PQ private key: %w", err)
}
classicalPriv, err := keyManager.LoadPriv(privkeyClassical, 0x0007)
if err != nil {
return fmt.Errorf("load classical private key: %w", err)
}
var out io.Writer
if outPath != "" {
outFile, err := os.Create(outPath)
if err != nil {
return fmt.Errorf("create output file: %w", err)
}
defer outFile.Close()
out = outFile
} else {
out = restoreOutput
}
decryptor := newRestoreDecryptor(registry)
if err := decryptor.Decrypt(
inFile,
[]crypto.RecipientPriv{pqPriv, classicalPriv},
out,
); err != nil {
return err
}
return nil
},
}
func init() {
restoreCmd.Flags().String("in", "", "Input .pqenc file path")
restoreCmd.Flags().String("privkey-pq", "", "Path to PQ private key PEM")
restoreCmd.Flags().String(
"privkey-classical",
"",
"Path to classical private key PEM",
)
restoreCmd.Flags().String("out", "", "Output file path (empty = stdout)")
_ = restoreCmd.MarkFlagRequired("in")
_ = restoreCmd.MarkFlagRequired("privkey-pq")
_ = restoreCmd.MarkFlagRequired("privkey-classical")
}
cmd/synapse-backupper/restore_test.go
+498
View File
@@ -0,0 +1,498 @@
package main
import (
"bytes"
"errors"
"io"
"os"
"path/filepath"
"strings"
"testing"
"github.com/spf13/cobra"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/crypto/composite"
"git.tswf.io/infra/go-synapse-backupper/pkg/domain/crypto"
)
// --- Mocks ---
type mockRecipientPriv struct {
schemeID uint16
keyID []byte
raw []byte
}
func (m *mockRecipientPriv) SchemeID() uint16 { return m.schemeID }
func (m *mockRecipientPriv) KeyID() []byte { return m.keyID }
func (m *mockRecipientPriv) Raw() []byte { return m.raw }
type mockRestoreKeyManager struct {
loadPrivCalls []loadPrivCall
pqPriv crypto.RecipientPriv
classicalPriv crypto.RecipientPriv
loadPrivErr error
}
type loadPrivCall struct {
path string
schemeID uint16
}
func (m *mockRestoreKeyManager) LoadPub(
path string,
schemeID uint16,
) (
crypto.RecipientPub,
error,
) {
return nil, nil
}
func (m *mockRestoreKeyManager) LoadPriv(
path string,
schemeID uint16,
) (
crypto.RecipientPriv,
error,
) {
m.loadPrivCalls = append(
m.loadPrivCalls,
loadPrivCall{path: path, schemeID: schemeID},
)
if m.loadPrivErr != nil {
return nil, m.loadPrivErr
}
if schemeID == 0x0006 {
return m.pqPriv, nil
}
return m.classicalPriv, nil
}
func (m *mockRestoreKeyManager) Generate(
schemeID uint16,
pubOut io.Writer,
privOut io.Writer,
rand io.Reader,
) error {
return nil
}
type mockDecryptor struct {
called bool
src io.Reader
privs []crypto.RecipientPriv
out io.Writer
err error
}
func (d *mockDecryptor) Decrypt(
src io.Reader,
privs []crypto.RecipientPriv,
plaintext io.Writer,
) error {
d.called = true
d.src = src
d.privs = privs
d.out = plaintext
if d.err != nil {
return d.err
}
_, writeErr := plaintext.Write([]byte("decrypted pg_dump data"))
return writeErr
}
// --- Helpers ---
func setRestoreFlags(
cmd *cobra.Command,
inPath string,
pqPath string,
classicalPath string,
) {
_ = cmd.Flags().Set("in", inPath)
_ = cmd.Flags().Set("privkey-pq", pqPath)
_ = cmd.Flags().Set("privkey-classical", classicalPath)
}
func makeRestoreTestCmd() *cobra.Command {
cmd := &cobra.Command{
SilenceUsage: true,
SilenceErrors: true,
}
cmd.SetArgs([]string{})
cmd.Flags().String("in", "", "Input .pqenc file path")
cmd.Flags().String("privkey-pq", "", "Path to PQ private key PEM")
cmd.Flags().String("privkey-classical", "", "Path to classical private key PEM")
cmd.Flags().String("out", "", "Output file path (empty = stdout)")
cmd.PreRunE = restoreCmd.PreRunE
cmd.RunE = restoreCmd.RunE
return cmd
}
func restoreTestGlobals(t *testing.T) {
originalNewRestoreKeyManager := newRestoreKeyManager
originalNewRestoreDecryptor := newRestoreDecryptor
originalRestoreOutput := restoreOutput
originalRestoreOsOpen := restoreOsOpen
t.Cleanup(func() {
newRestoreKeyManager = originalNewRestoreKeyManager
newRestoreDecryptor = originalNewRestoreDecryptor
restoreOutput = originalRestoreOutput
restoreOsOpen = originalRestoreOsOpen
})
}
// --- Tests ---
func TestRestoreCmd_Structure(t *testing.T) {
if restoreCmd == nil {
t.Fatal("restoreCmd is nil")
}
if restoreCmd.Use != "restore" {
t.Fatalf("expected Use='restore', got %q", restoreCmd.Use)
}
requiredFlags := []string{"in", "privkey-pq", "privkey-classical"}
for _, f := range requiredFlags {
if restoreCmd.Flag(f) == nil {
t.Fatalf("missing required --%s flag", f)
}
}
if restoreCmd.Flag("out") == nil {
t.Fatal("missing --out flag")
}
if restoreCmd.Flag("out").DefValue != "" {
t.Fatalf(
"expected --out default empty, got %q",
restoreCmd.Flag("out").DefValue,
)
}
}
func TestRestoreCmd_RequiredFlags(t *testing.T) {
required := []string{"in", "privkey-pq", "privkey-classical"}
for _, name := range required {
flag := restoreCmd.Flag(name)
if flag == nil {
t.Fatalf("missing required --%s flag", name)
}
ann, ok := flag.Annotations[cobra.BashCompOneRequiredFlag]
if !ok || len(ann) == 0 || ann[0] != "true" {
t.Fatalf("flag --%s is not marked required", name)
}
}
}
func TestRestoreCmd_PreRunE_MissingPrivkeyClassical(t *testing.T) {
restoreTestGlobals(t)
dir := t.TempDir()
inFile := filepath.Join(dir, "backup.pqenc")
_ = os.WriteFile(inFile, []byte("data"), 0o644)
pqFile := filepath.Join(dir, "pq.priv.pem")
_ = os.WriteFile(pqFile, []byte("pq"), 0o600)
classicalFile := filepath.Join(dir, "classical.priv.pem") // does not exist
openCount := 0
restoreOsOpen = func(name string) (*os.File, error) {
openCount++
return os.Open(name)
}
cmd := makeRestoreTestCmd()
setRestoreFlags(cmd, inFile, pqFile, classicalFile)
err := cmd.Execute()
if err == nil {
t.Fatal("expected non-nil error")
}
if !strings.Contains(
err.Error(),
"--privkey-pq and --privkey-classical are both required (AND model)",
) {
t.Fatalf("expected AND model error, got: %v", err)
}
if openCount != 0 {
t.Fatalf("expected no os.Open calls on .pqenc, got %d", openCount)
}
}
func TestRestoreCmd_PreRunE_MissingPrivkeyPQ(t *testing.T) {
restoreTestGlobals(t)
dir := t.TempDir()
inFile := filepath.Join(dir, "backup.pqenc")
_ = os.WriteFile(inFile, []byte("data"), 0o644)
pqFile := filepath.Join(dir, "pq.priv.pem") // does not exist
classicalFile := filepath.Join(dir, "classical.priv.pem")
_ = os.WriteFile(classicalFile, []byte("classical"), 0o600)
openCount := 0
restoreOsOpen = func(name string) (*os.File, error) {
openCount++
return os.Open(name)
}
cmd := makeRestoreTestCmd()
setRestoreFlags(cmd, inFile, pqFile, classicalFile)
err := cmd.Execute()
if err == nil {
t.Fatal("expected non-nil error")
}
if !strings.Contains(
err.Error(),
"--privkey-pq and --privkey-classical are both required (AND model)",
) {
t.Fatalf("expected AND model error, got: %v", err)
}
if openCount != 0 {
t.Fatalf("expected no os.Open calls on .pqenc, got %d", openCount)
}
}
func TestRestoreCmd_OnlyPrivkeyPQ(t *testing.T) {
restoreTestGlobals(t)
dir := t.TempDir()
inFile := filepath.Join(dir, "backup.pqenc")
_ = os.WriteFile(inFile, []byte("data"), 0o644)
pqFile := filepath.Join(dir, "pq.priv.pem")
_ = os.WriteFile(pqFile, []byte("pq"), 0o600)
openCount := 0
restoreOsOpen = func(name string) (*os.File, error) {
openCount++
return os.Open(name)
}
cmd := makeRestoreTestCmd()
_ = cmd.Flags().Set("in", inFile)
_ = cmd.Flags().Set("privkey-pq", pqFile)
// --privkey-classical intentionally omitted
err := cmd.Execute()
if err == nil {
t.Fatal("expected non-nil error")
}
if openCount != 0 {
t.Fatalf("expected no os.Open calls on .pqenc, got %d", openCount)
}
}
func TestRestoreCmd_HappyPath_Stdout(t *testing.T) {
restoreTestGlobals(t)
dir := t.TempDir()
inFile := filepath.Join(dir, "backup.pqenc")
_ = os.WriteFile(inFile, []byte("encrypted"), 0o644)
pqFile := filepath.Join(dir, "pq.priv.pem")
_ = os.WriteFile(pqFile, []byte("pq"), 0o600)
classicalFile := filepath.Join(dir, "classical.priv.pem")
_ = os.WriteFile(classicalFile, []byte("classical"), 0o600)
pqPriv := &mockRecipientPriv{
schemeID: 0x0006,
keyID: []byte{1, 2, 3, 4, 5, 6, 7, 8},
raw: make([]byte, 32),
}
classicalPriv := &mockRecipientPriv{
schemeID: 0x0007,
keyID: []byte{8, 7, 6, 5, 4, 3, 2, 1},
raw: make([]byte, 32),
}
mockKM := &mockRestoreKeyManager{pqPriv: pqPriv, classicalPriv: classicalPriv}
newRestoreKeyManager = func(crypto.Registry) crypto.KeyManager {
return mockKM
}
mockDec := &mockDecryptor{}
newRestoreDecryptor = func(crypto.Registry) crypto.Decryptor {
return mockDec
}
var outBuf bytes.Buffer
restoreOutput = &outBuf
cmd := makeRestoreTestCmd()
setRestoreFlags(cmd, inFile, pqFile, classicalFile)
err := cmd.Execute()
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if len(mockKM.loadPrivCalls) != 2 {
t.Fatalf("expected 2 LoadPriv calls, got %d", len(mockKM.loadPrivCalls))
}
if mockKM.loadPrivCalls[0].path != pqFile ||
mockKM.loadPrivCalls[0].schemeID != 0x0006 {
t.Fatalf("unexpected PQ LoadPriv call: %+v", mockKM.loadPrivCalls[0])
}
if mockKM.loadPrivCalls[1].path != classicalFile ||
mockKM.loadPrivCalls[1].schemeID != 0x0007 {
t.Fatalf(
"unexpected classical LoadPriv call: %+v",
mockKM.loadPrivCalls[1],
)
}
if !mockDec.called {
t.Fatal("expected decryptor.Decrypt to be called")
}
if len(mockDec.privs) != 2 {
t.Fatalf("expected 2 privs, got %d", len(mockDec.privs))
}
if mockDec.privs[0] != pqPriv {
t.Fatal("expected pqPriv in positional slot 0")
}
if mockDec.privs[1] != classicalPriv {
t.Fatal("expected classicalPriv in positional slot 1")
}
if !bytes.Equal(outBuf.Bytes(), []byte("decrypted pg_dump data")) {
t.Fatalf("unexpected stdout content: %q", outBuf.Bytes())
}
}
func TestRestoreCmd_HappyPath_FileOut(t *testing.T) {
restoreTestGlobals(t)
dir := t.TempDir()
inFile := filepath.Join(dir, "backup.pqenc")
_ = os.WriteFile(inFile, []byte("encrypted"), 0o644)
pqFile := filepath.Join(dir, "pq.priv.pem")
_ = os.WriteFile(pqFile, []byte("pq"), 0o600)
classicalFile := filepath.Join(dir, "classical.priv.pem")
_ = os.WriteFile(classicalFile, []byte("classical"), 0o600)
outFile := filepath.Join(dir, "restored.dump")
pqPriv := &mockRecipientPriv{
schemeID: 0x0006,
keyID: []byte{1, 2, 3, 4, 5, 6, 7, 8},
raw: make([]byte, 32),
}
classicalPriv := &mockRecipientPriv{
schemeID: 0x0007,
keyID: []byte{8, 7, 6, 5, 4, 3, 2, 1},
raw: make([]byte, 32),
}
mockKM := &mockRestoreKeyManager{pqPriv: pqPriv, classicalPriv: classicalPriv}
newRestoreKeyManager = func(crypto.Registry) crypto.KeyManager {
return mockKM
}
mockDec := &mockDecryptor{}
newRestoreDecryptor = func(crypto.Registry) crypto.Decryptor {
return mockDec
}
cmd := makeRestoreTestCmd()
setRestoreFlags(cmd, inFile, pqFile, classicalFile)
_ = cmd.Flags().Set("out", outFile)
err := cmd.Execute()
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
data, err := os.ReadFile(outFile)
if err != nil {
t.Fatalf("read output file: %v", err)
}
if !bytes.Equal(data, []byte("decrypted pg_dump data")) {
t.Fatalf("unexpected output file content: %q", data)
}
}
func TestRestoreCmd_TamperingDetected(t *testing.T) {
restoreTestGlobals(t)
dir := t.TempDir()
inFile := filepath.Join(dir, "backup.pqenc")
_ = os.WriteFile(inFile, []byte("encrypted"), 0o644)
pqFile := filepath.Join(dir, "pq.priv.pem")
_ = os.WriteFile(pqFile, []byte("pq"), 0o600)
classicalFile := filepath.Join(dir, "classical.priv.pem")
_ = os.WriteFile(classicalFile, []byte("classical"), 0o600)
pqPriv := &mockRecipientPriv{
schemeID: 0x0006,
keyID: []byte{1, 2, 3, 4, 5, 6, 7, 8},
raw: make([]byte, 32),
}
classicalPriv := &mockRecipientPriv{
schemeID: 0x0007,
keyID: []byte{8, 7, 6, 5, 4, 3, 2, 1},
raw: make([]byte, 32),
}
mockKM := &mockRestoreKeyManager{pqPriv: pqPriv, classicalPriv: classicalPriv}
newRestoreKeyManager = func(crypto.Registry) crypto.KeyManager {
return mockKM
}
mockDec := &mockDecryptor{err: composite.ErrTamperingDetected}
newRestoreDecryptor = func(crypto.Registry) crypto.Decryptor {
return mockDec
}
cmd := makeRestoreTestCmd()
setRestoreFlags(cmd, inFile, pqFile, classicalFile)
err := cmd.Execute()
if err == nil {
t.Fatal("expected non-nil error")
}
if !errors.Is(err, composite.ErrTamperingDetected) {
t.Fatalf("expected ErrTamperingDetected, got: %v", err)
}
}
func TestRestoreCmd_WrongKeys(t *testing.T) {
restoreTestGlobals(t)
dir := t.TempDir()
inFile := filepath.Join(dir, "backup.pqenc")
_ = os.WriteFile(inFile, []byte("encrypted"), 0o644)
pqFile := filepath.Join(dir, "pq.priv.pem")
_ = os.WriteFile(pqFile, []byte("pq"), 0o600)
classicalFile := filepath.Join(dir, "classical.priv.pem")
_ = os.WriteFile(classicalFile, []byte("classical"), 0o600)
pqPriv := &mockRecipientPriv{
schemeID: 0x0006,
keyID: []byte{1, 2, 3, 4, 5, 6, 7, 8},
raw: make([]byte, 32),
}
classicalPriv := &mockRecipientPriv{
schemeID: 0x0007,
keyID: []byte{8, 7, 6, 5, 4, 3, 2, 1},
raw: make([]byte, 32),
}
mockKM := &mockRestoreKeyManager{pqPriv: pqPriv, classicalPriv: classicalPriv}
newRestoreKeyManager = func(crypto.Registry) crypto.KeyManager {
return mockKM
}
mockDec := &mockDecryptor{err: composite.ErrWrongKeys}
newRestoreDecryptor = func(crypto.Registry) crypto.Decryptor {
return mockDec
}
cmd := makeRestoreTestCmd()
setRestoreFlags(cmd, inFile, pqFile, classicalFile)
err := cmd.Execute()
if err == nil {
t.Fatal("expected non-nil error")
}
if !errors.Is(err, composite.ErrWrongKeys) {
t.Fatalf("expected ErrWrongKeys, got: %v", err)
}
}
cmd/synapse-backupper/run.go
+114
View File
@@ -0,0 +1,114 @@
package main
import (
"context"
"errors"
"log/slog"
"net/http"
"os"
"os/signal"
"syscall"
"time"
"github.com/spf13/cobra"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/config"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/healthz"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/scheduler/cron"
"git.tswf.io/infra/go-synapse-backupper/pkg/domain"
"git.tswf.io/infra/go-synapse-backupper/pkg/domain/backup"
)
var (
newScheduler = func(
expr string,
job func(),
) (
domain.Scheduler,
error,
) {
return cron.NewCronScheduler(expr, job)
}
newHealthz = func(port int) (*healthz.Server, error) {
return healthz.New(port)
}
runOnceFunc = backup.RunOnce
osExitFunc = func(code int) { os.Exit(code) }
)
var runCmd = &cobra.Command{
Use: "run",
Short: "Start the backup scheduler",
RunE: run,
}
func init() {
config.RegisterFlags(runCmd)
}
func run(cmd *cobra.Command, args []string) error {
cfg, err := config.Load(cmd)
if err != nil {
return err
}
return runWithConfig(cmd.Context(), cfg)
}
func runWithConfig(
ctx context.Context,
cfg *config.Config,
) error {
job := func() {
jobCtx := context.Background()
if err := runOnceFunc(jobCtx, cfg); err != nil {
slog.Error("backup job failed", "error", err)
}
}
scheduler, err := newScheduler(cfg.Backup.Cron, job)
if err != nil {
return err
}
healthzServer, err := newHealthz(cfg.Healthz.Port)
if err != nil {
slog.Error("failed to bind healthz server", "error", err)
osExitFunc(1)
return nil
}
go func() {
if err := healthzServer.Start(); err != nil && !errors.Is(err, http.ErrServerClosed) {
slog.Error("healthz server error", "error", err)
}
}()
scheduler.Start()
sigCtx, stop := signal.NotifyContext(ctx, syscall.SIGTERM, syscall.SIGINT)
defer stop()
<-sigCtx.Done()
shutdownTimeout := cfg.ShutdownTimeout
if shutdownTimeout <= 0 {
shutdownTimeout = 30 * time.Second
}
shutdownCtx, cancel := context.WithTimeout(context.Background(), shutdownTimeout)
defer cancel()
_ = healthzServer.Stop(shutdownCtx)
stoppedCtx := scheduler.Stop()
select {
case <-stoppedCtx.Done():
slog.Info("graceful shutdown complete")
case <-time.After(shutdownTimeout):
slog.Warn("forcing exit, backup job still running")
osExitFunc(0)
}
return nil
}
cmd/synapse-backupper/run_test.go
+393
View File
@@ -0,0 +1,393 @@
package main
import (
"context"
"errors"
"fmt"
"net/http"
"sync"
"testing"
"time"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/config"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/healthz"
"git.tswf.io/infra/go-synapse-backupper/pkg/domain"
)
type fakeScheduler struct {
mu sync.Mutex
startCalled bool
stopCalled bool
stoppedCtx context.Context
stopCancel context.CancelFunc
}
func (f *fakeScheduler) Start() {
f.mu.Lock()
f.startCalled = true
f.mu.Unlock()
}
func (f *fakeScheduler) Stop() context.Context {
f.mu.Lock()
f.stopCalled = true
f.mu.Unlock()
return f.stoppedCtx
}
func (f *fakeScheduler) wasStarted() bool {
f.mu.Lock()
defer f.mu.Unlock()
return f.startCalled
}
func (f *fakeScheduler) wasStopped() bool {
f.mu.Lock()
defer f.mu.Unlock()
return f.stopCalled
}
type exitPanic int
func (e exitPanic) Error() string { return fmt.Sprintf("exit %d", int(e)) }
func TestRunCmd_Use(t *testing.T) {
if runCmd.Use != "run" {
t.Errorf("runCmd.Use = %q, want %q", runCmd.Use, "run")
}
}
func TestRunCmd_SIGTERM_stopsScheduler(t *testing.T) {
origNewScheduler := newScheduler
origOsExit := osExitFunc
origRunOnce := runOnceFunc
defer func() {
newScheduler = origNewScheduler
osExitFunc = origOsExit
runOnceFunc = origRunOnce
}()
fake := &fakeScheduler{}
fake.stoppedCtx, fake.stopCancel = context.WithCancel(context.Background())
newScheduler = func(expr string, job func()) (domain.Scheduler, error) {
return fake, nil
}
runOnceFunc = func(ctx context.Context, cfg *config.Config) error {
return nil
}
var exitCode int
exitPanicked := false
osExitFunc = func(code int) {
exitCode = code
exitPanicked = true
panic(exitPanic(code))
}
cfg := &config.Config{}
cfg.Healthz.Port = 0
cfg.Backup.Cron = "* * * * *"
cfg.ShutdownTimeout = 100 * time.Millisecond
ctx, cancel := context.WithCancel(context.Background())
errCh := make(chan error, 1)
go func() {
defer func() {
if r := recover(); r != nil {
if _, ok := r.(exitPanic); ok {
errCh <- nil
return
}
errCh <- fmt.Errorf("unexpected panic: %v", r)
return
}
}()
errCh <- runWithConfig(ctx, cfg)
}()
time.Sleep(50 * time.Millisecond)
if !fake.wasStarted() {
t.Fatal("scheduler.Start was not called")
}
cancel()
go func() {
time.Sleep(50 * time.Millisecond)
fake.stopCancel()
}()
select {
case err := <-errCh:
if err != nil {
t.Fatalf("runWithConfig returned error: %v", err)
}
case <-time.After(2 * time.Second):
t.Fatal("timeout waiting for runWithConfig to exit")
}
if !fake.wasStopped() {
t.Error("scheduler.Stop was not called")
}
if exitPanicked {
t.Errorf("unexpected os.Exit call with code %d", exitCode)
}
}
func TestRunCmd_SIGTERM_forcesExitAfterTimeout(t *testing.T) {
origNewScheduler := newScheduler
origOsExit := osExitFunc
origNewHealthz := newHealthz
origRunOnce := runOnceFunc
defer func() {
newScheduler = origNewScheduler
osExitFunc = origOsExit
newHealthz = origNewHealthz
runOnceFunc = origRunOnce
}()
fake := &fakeScheduler{}
var cancelFunc context.CancelFunc
fake.stoppedCtx, cancelFunc = context.WithCancel(context.Background())
defer cancelFunc()
newScheduler = func(expr string, job func()) (domain.Scheduler, error) {
return fake, nil
}
newHealthz = func(port int) (*healthz.Server, error) {
return healthz.New(port)
}
runOnceFunc = func(ctx context.Context, cfg *config.Config) error {
return nil
}
var exitCode int
exitPanicked := false
osExitFunc = func(code int) {
exitCode = code
exitPanicked = true
panic(exitPanic(code))
}
cfg := &config.Config{}
cfg.Healthz.Port = 0
cfg.Backup.Cron = "* * * * * *"
cfg.ShutdownTimeout = 100 * time.Millisecond
ctx, cancel := context.WithCancel(context.Background())
errCh := make(chan error, 1)
go func() {
defer func() {
if r := recover(); r != nil {
if _, ok := r.(exitPanic); ok {
errCh <- nil
return
}
errCh <- fmt.Errorf("unexpected panic: %v", r)
return
}
}()
errCh <- runWithConfig(ctx, cfg)
}()
time.Sleep(50 * time.Millisecond)
cancel()
select {
case err := <-errCh:
if err != nil {
t.Fatalf("runWithConfig returned error: %v", err)
}
case <-time.After(2 * time.Second):
t.Fatal("timeout waiting for runWithConfig to exit")
}
if !fake.wasStopped() {
t.Error("scheduler.Stop was not called")
}
if !exitPanicked {
t.Fatal("expected os.Exit to be called")
}
if exitCode != 0 {
t.Errorf("os.Exit code = %d, want 0", exitCode)
}
}
func TestRunCmd_Healthz503DuringShutdown(t *testing.T) {
origNewScheduler := newScheduler
origOsExit := osExitFunc
origNewHealthz := newHealthz
origRunOnce := runOnceFunc
defer func() {
newScheduler = origNewScheduler
osExitFunc = origOsExit
newHealthz = origNewHealthz
runOnceFunc = origRunOnce
}()
fake := &fakeScheduler{}
fake.stoppedCtx, fake.stopCancel = context.WithCancel(context.Background())
newScheduler = func(expr string, job func()) (domain.Scheduler, error) {
return fake, nil
}
srv, err := healthz.New(0)
if err != nil {
t.Fatalf("failed to create healthz server: %v", err)
}
newHealthz = func(port int) (*healthz.Server, error) {
return srv, nil
}
runOnceFunc = func(ctx context.Context, cfg *config.Config) error {
return nil
}
osExitFunc = func(code int) { panic(exitPanic(code)) }
cfg := &config.Config{}
cfg.Healthz.Port = 0
cfg.Backup.Cron = "* * * * * *"
cfg.ShutdownTimeout = 5 * time.Second
ctx, cancel := context.WithCancel(context.Background())
errCh := make(chan error, 1)
go func() {
defer func() {
if r := recover(); r != nil {
if _, ok := r.(exitPanic); ok {
errCh <- nil
return
}
errCh <- fmt.Errorf("unexpected panic: %v", r)
return
}
}()
errCh <- runWithConfig(ctx, cfg)
}()
time.Sleep(100 * time.Millisecond)
addr := srv.Addr()
resp, err := http.Get("http://" + addr + "/healthz")
if err != nil {
t.Fatalf("healthz request failed: %v", err)
}
resp.Body.Close()
if resp.StatusCode != http.StatusOK {
t.Fatalf("healthz status before shutdown = %d, want %d", resp.StatusCode, http.StatusOK)
}
cancel()
var status503 bool
for i := 0; i < 20; i++ {
resp, err = http.Get("http://" + addr + "/healthz")
if err == nil {
resp.Body.Close()
if resp.StatusCode == http.StatusServiceUnavailable {
status503 = true
break
}
}
time.Sleep(10 * time.Millisecond)
}
if !status503 {
t.Error("did not observe /healthz returning 503 during shutdown")
}
fake.stopCancel()
select {
case err := <-errCh:
if err != nil {
t.Fatalf("runWithConfig returned error: %v", err)
}
case <-time.After(2 * time.Second):
t.Fatal("timeout waiting for runWithConfig to exit")
}
}
func TestRunCmd_HealthzBindFailure(t *testing.T) {
origNewScheduler := newScheduler
origNewHealthz := newHealthz
origOsExit := osExitFunc
origRunOnce := runOnceFunc
defer func() {
newScheduler = origNewScheduler
newHealthz = origNewHealthz
osExitFunc = origOsExit
runOnceFunc = origRunOnce
}()
fake := &fakeScheduler{}
var cancelFunc context.CancelFunc
fake.stoppedCtx, cancelFunc = context.WithCancel(context.Background())
defer cancelFunc()
newScheduler = func(expr string, job func()) (domain.Scheduler, error) {
return fake, nil
}
newHealthz = func(port int) (*healthz.Server, error) {
return nil, errors.New("bind failed")
}
runOnceFunc = func(ctx context.Context, cfg *config.Config) error {
return nil
}
var exitCode int
exitPanicked := false
osExitFunc = func(code int) {
exitCode = code
exitPanicked = true
panic(exitPanic(code))
}
cfg := &config.Config{}
cfg.Backup.Cron = "* * * * * *"
errCh := make(chan error, 1)
go func() {
defer func() {
if r := recover(); r != nil {
if _, ok := r.(exitPanic); ok {
errCh <- nil
return
}
errCh <- fmt.Errorf("unexpected panic: %v", r)
return
}
}()
errCh <- runWithConfig(context.Background(), cfg)
}()
select {
case err := <-errCh:
if err != nil {
t.Fatalf("runWithConfig returned error: %v", err)
}
case <-time.After(2 * time.Second):
t.Fatal("timeout waiting for runWithConfig to exit")
}
if !exitPanicked {
t.Fatal("expected os.Exit to be called on healthz bind failure")
}
if exitCode != 1 {
t.Errorf("os.Exit code = %d, want 1", exitCode)
}
}
deploy/docker/Dockerfile
+17
View File
@@ -0,0 +1,17 @@
FROM golang:1.26-alpine AS builder
WORKDIR /src
COPY go.mod go.sum ./
RUN --mount=type=cache,target=/root/go/pkg/mod go mod download
COPY . .
RUN --mount=type=cache,target=/root/go/pkg/mod CGO_ENABLED=0 go build -o /out/synapse-backupper ./cmd/synapse-backupper
FROM alpine:3.24
RUN apk add --no-cache postgresql18-client ca-certificates && update-ca-certificates
RUN addgroup -S app && adduser -S -G app app
COPY --from=builder --chown=app:app /out/synapse-backupper /usr/local/bin/synapse-backupper
USER app
EXPOSE 8080
HEALTHCHECK --interval=30s --timeout=3s --start-period=10s CMD wget -qO- http://localhost:8080/healthz || exit 1
LABEL maintainer="infra@tswf.io" version="0.1.0" description="Synapse PostgreSQL backup tool with composite dual-KEM encryption"
ENTRYPOINT ["/usr/local/bin/synapse-backupper"]
CMD ["run"]
deploy/docker/docker-compose.yml
+28
View File
@@ -0,0 +1,28 @@
version: '3.8'
services:
synapse-backupper:
image: synapse-backupper:latest
volumes:
- ./backups:/backups
- ./keys:/keys:ro
environment:
- APP_PG_HOST=db
- APP_PG_DATABASE=postgres
- APP_PG_USER=synapse
- APP_PG_PASSWORD=changeme
- APP_PQ_PUBLIC_KEY_PATH=/keys/synapse.pq.pub.pem
- APP_CLASSICAL_PUBLIC_KEY_PATH=/keys/synapse.classical.pub.pem
- APP_BACKUP_DIR=/backups
- APP_BACKUP_CRON=0 0 3 * * *
networks:
- my-network
healthcheck:
test: ["CMD", "wget", "-qO-", "http://localhost:8080/healthz"]
interval: 30s
timeout: 3s
start_period: 10s
restart: unless-stopped
networks:
my-network:
external: true
go.mod
+26
View File
@@ -0,0 +1,26 @@
module git.tswf.io/infra/go-synapse-backupper
go 1.26
require (
github.com/google/uuid v1.6.0
github.com/robfig/cron/v3 v3.0.1
github.com/spf13/cobra v1.10.2
github.com/spf13/viper v1.21.0
)
require (
github.com/fsnotify/fsnotify v1.9.0 // indirect
github.com/go-viper/mapstructure/v2 v2.4.0 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/pelletier/go-toml/v2 v2.2.4 // indirect
github.com/sagikazarmark/locafero v0.11.0 // indirect
github.com/sourcegraph/conc v0.3.1-0.20240121214520-5f936abd7ae8 // indirect
github.com/spf13/afero v1.15.0 // indirect
github.com/spf13/cast v1.10.0 // indirect
github.com/spf13/pflag v1.0.10 // indirect
github.com/subosito/gotenv v1.6.0 // indirect
go.yaml.in/yaml/v3 v3.0.4 // indirect
golang.org/x/sys v0.29.0 // indirect
golang.org/x/text v0.28.0 // indirect
)
go.sum
+58
View File
@@ -0,0 +1,58 @@
github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=
github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
github.com/fsnotify/fsnotify v1.9.0 h1:2Ml+OJNzbYCTzsxtv8vKSFD9PbJjmhYF14k/jKC7S9k=
github.com/fsnotify/fsnotify v1.9.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0=
github.com/go-viper/mapstructure/v2 v2.4.0 h1:EBsztssimR/CONLSZZ04E8qAkxNYq4Qp9LvH92wZUgs=
github.com/go-viper/mapstructure/v2 v2.4.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/pelletier/go-toml/v2 v2.2.4 h1:mye9XuhQ6gvn5h28+VilKrrPoQVanw5PMw/TB0t5Ec4=
github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/robfig/cron/v3 v3.0.1 h1:WdRxkvbJztn8LMz/QEvLN5sBU+xKpSqwwUO1Pjr4qDs=
github.com/robfig/cron/v3 v3.0.1/go.mod h1:eQICP3HwyT7UooqI/z+Ov+PtYAWygg1TEWWzGIFLtro=
github.com/rogpeppe/go-internal v1.9.0 h1:73kH8U+JUqXU8lRuOHeVHaa/SZPifC7BkcraZVejAe8=
github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/sagikazarmark/locafero v0.11.0 h1:1iurJgmM9G3PA/I+wWYIOw/5SyBtxapeHDcg+AAIFXc=
github.com/sagikazarmark/locafero v0.11.0/go.mod h1:nVIGvgyzw595SUSUE6tvCp3YYTeHs15MvlmU87WwIik=
github.com/sourcegraph/conc v0.3.1-0.20240121214520-5f936abd7ae8 h1:+jumHNA0Wrelhe64i8F6HNlS8pkoyMv5sreGx2Ry5Rw=
github.com/sourcegraph/conc v0.3.1-0.20240121214520-5f936abd7ae8/go.mod h1:3n1Cwaq1E1/1lhQhtRK2ts/ZwZEhjcQeJQ1RuC6Q/8U=
github.com/spf13/afero v1.15.0 h1:b/YBCLWAJdFWJTN9cLhiXXcD7mzKn9Dm86dNnfyQw1I=
github.com/spf13/afero v1.15.0/go.mod h1:NC2ByUVxtQs4b3sIUphxK0NioZnmxgyCrfzeuq8lxMg=
github.com/spf13/cast v1.10.0 h1:h2x0u2shc1QuLHfxi+cTJvs30+ZAHOGRic8uyGTDWxY=
github.com/spf13/cast v1.10.0/go.mod h1:jNfB8QC9IA6ZuY2ZjDp0KtFO2LZZlg4S/7bzP6qqeHo=
github.com/spf13/cobra v1.10.2 h1:DMTTonx5m65Ic0GOoRY2c16WCbHxOOw6xxezuLaBpcU=
github.com/spf13/cobra v1.10.2/go.mod h1:7C1pvHqHw5A4vrJfjNwvOdzYu0Gml16OCs2GRiTUUS4=
github.com/spf13/pflag v1.0.9/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/spf13/pflag v1.0.10 h1:4EBh2KAYBwaONj6b2Ye1GiHfwjqyROoF4RwYO+vPwFk=
github.com/spf13/pflag v1.0.10/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/spf13/viper v1.21.0 h1:x5S+0EU27Lbphp4UKm1C+1oQO+rKx36vfCoaVebLFSU=
github.com/spf13/viper v1.21.0/go.mod h1:P0lhsswPGWD/1lZJ9ny3fYnVqxiegrlNrEmgLjbTCAY=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/subosito/gotenv v1.6.0 h1:9NlTDc1FTs4qu0DDq7AEtTPNw6SVm7uBMsUCUjABIf8=
github.com/subosito/gotenv v1.6.0/go.mod h1:Dk4QP5c2W3ibzajGcXpNraDfq2IrhjMIvMSWPKKo0FU=
go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc=
go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
golang.org/x/sys v0.29.0 h1:TPYlXGxvx1MGTn2GiZDhnjPA9wZzZeGKHHmKhHYvgaU=
golang.org/x/sys v0.29.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/text v0.28.0 h1:rhazDwis8INMIwQ4tpjLDzUhx6RlXqZNPEM0huQojng=
golang.org/x/text v0.28.0/go.mod h1:U8nCwOR8jO/marOQ0QbDiOngZVEBB7MAiitBuMjXiNU=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
pkg/adapters/config/config.go
+161
View File
@@ -0,0 +1,161 @@
package config
import (
"fmt"
"io"
"os"
"path/filepath"
"strings"
"time"
"github.com/spf13/cobra"
"github.com/spf13/viper"
)
// outputWriter is used for logging so tests can capture output.
var outputWriter io.Writer = os.Stderr
func logf(format string, args ...interface{}) {
_, _ = fmt.Fprintf(outputWriter, format+"\n", args...)
}
// Config holds the full application configuration.
type Config struct {
PG struct {
Host string `mapstructure:"host"`
Port int `mapstructure:"port"`
User string `mapstructure:"user"`
Password string `mapstructure:"password"`
Database string `mapstructure:"database"`
SSLMode string `mapstructure:"sslmode"`
ExcludeTables []string `mapstructure:"exclude_tables"`
} `mapstructure:"pg"`
Backup struct {
Dir string `mapstructure:"dir"`
RetentionDays int `mapstructure:"retention_days"`
Cron string `mapstructure:"cron"`
} `mapstructure:"backup"`
PQScheme uint16 `mapstructure:"pq_scheme"`
ClassicalScheme uint16 `mapstructure:"classical_scheme"`
PQPublicKeyPath string `mapstructure:"pq_public_key_path"`
ClassicalPublicKeyPath string `mapstructure:"classical_public_key_path"`
Healthz struct {
Port int `mapstructure:"port"`
} `mapstructure:"healthz"`
Log struct {
Level string `mapstructure:"level"`
} `mapstructure:"log"`
ShutdownTimeout time.Duration `mapstructure:"shutdown_timeout"`
}
// RegisterFlags adds all configuration flags to the provided Cobra command.
func RegisterFlags(cmd *cobra.Command) {
flags := cmd.Flags()
flags.String("pg-host", "", "PostgreSQL host")
flags.Int("pg-port", 0, "PostgreSQL port")
flags.String("pg-user", "", "PostgreSQL user")
flags.String("pg-password", "", "PostgreSQL password")
flags.String("pg-database", "", "PostgreSQL database name")
flags.String("pg-sslmode", "", "PostgreSQL SSL mode")
flags.StringSlice("pg-exclude-tables", nil, "PostgreSQL tables to exclude from backup")
flags.String("backup-dir", "", "Backup directory")
flags.Int("backup-retention-days", 0, "Backup retention period in days")
flags.String("backup-cron", "", "Cron expression for backup schedule")
flags.Uint16("pq-scheme", 0, "Post-quantum KEM scheme ID")
flags.Uint16("classical-scheme", 0, "Classical KEM scheme ID")
flags.String("pq-public-key-path", "", "Path to post-quantum public key")
flags.String("classical-public-key-path", "", "Path to classical public key")
flags.Int("healthz-port", 0, "Health check HTTP port")
flags.String("log-level", "", "Log level")
flags.Duration("shutdown-timeout", 0, "Graceful shutdown timeout")
}
// Load reads configuration from all sources in priority order:
// launch arguments (Cobra flags) → APP_* env variables → config file → defaults.
func Load(cmd *cobra.Command) (*Config, error) {
// Use a fresh Viper instance so that successive calls do not leak state.
v := viper.New()
// Defaults (lowest priority).
v.SetDefault("pg.port", 5432)
v.SetDefault("pg.sslmode", "prefer")
v.SetDefault("pg.exclude_tables", []string{"e2e_one_time_keys_json"})
v.SetDefault("backup.retention_days", 180)
v.SetDefault("backup.cron", "0 0 3 * * *")
v.SetDefault("pq_scheme", uint16(0x0006))
v.SetDefault("classical_scheme", uint16(0x0007))
v.SetDefault("healthz.port", 8080)
v.SetDefault("log.level", "info")
v.SetDefault("shutdown_timeout", 30*time.Second)
// Bind parsed Cobra flags to Viper keys.
if cmd != nil {
_ = v.BindPFlag("pg.host", cmd.Flags().Lookup("pg-host"))
_ = v.BindPFlag("pg.port", cmd.Flags().Lookup("pg-port"))
_ = v.BindPFlag("pg.user", cmd.Flags().Lookup("pg-user"))
_ = v.BindPFlag("pg.password", cmd.Flags().Lookup("pg-password"))
_ = v.BindPFlag("pg.database", cmd.Flags().Lookup("pg-database"))
_ = v.BindPFlag("pg.sslmode", cmd.Flags().Lookup("pg-sslmode"))
_ = v.BindPFlag("pg.exclude_tables", cmd.Flags().Lookup("pg-exclude-tables"))
_ = v.BindPFlag("backup.dir", cmd.Flags().Lookup("backup-dir"))
_ = v.BindPFlag("backup.retention_days", cmd.Flags().Lookup("backup-retention-days"))
_ = v.BindPFlag("backup.cron", cmd.Flags().Lookup("backup-cron"))
_ = v.BindPFlag("pq_scheme", cmd.Flags().Lookup("pq-scheme"))
_ = v.BindPFlag("classical_scheme", cmd.Flags().Lookup("classical-scheme"))
_ = v.BindPFlag("pq_public_key_path", cmd.Flags().Lookup("pq-public-key-path"))
_ = v.BindPFlag("classical_public_key_path", cmd.Flags().Lookup("classical-public-key-path"))
_ = v.BindPFlag("healthz.port", cmd.Flags().Lookup("healthz-port"))
_ = v.BindPFlag("log.level", cmd.Flags().Lookup("log-level"))
_ = v.BindPFlag("shutdown_timeout", cmd.Flags().Lookup("shutdown-timeout"))
}
// Config file search with logging.
v.SetConfigName("config")
v.SetConfigType("yaml")
if envLoc := os.Getenv("APP_CONFIG_LOCATION"); envLoc != "" {
v.SetConfigFile(envLoc)
if err := v.ReadInConfig(); err == nil {
logf("Config file found: %s (from APP_CONFIG_LOCATION)", v.ConfigFileUsed())
} else {
return nil, fmt.Errorf("config file specified in APP_CONFIG_LOCATION not found: %s", envLoc)
}
} else {
v.AddConfigPath(".")
homeDir, _ := os.UserHomeDir()
appName := "synapse-backupper"
userConfigPath := filepath.Join(homeDir, ".config", appName)
v.AddConfigPath(userConfigPath)
exePath, _ := os.Executable()
exeDir := filepath.Dir(exePath)
v.AddConfigPath(exeDir)
etcPath := filepath.Join("/etc", appName)
v.AddConfigPath(etcPath)
if err := v.ReadInConfig(); err != nil {
if _, ok := err.(viper.ConfigFileNotFoundError); ok {
logf("Configuration file not found. Using defaults and env/args.")
} else {
return nil, fmt.Errorf("error reading config: %w", err)
}
} else {
logf("Config file found: %s", v.ConfigFileUsed())
}
}
// Environment variables.
v.SetEnvPrefix("APP")
v.SetEnvKeyReplacer(strings.NewReplacer(".", "_", "-", "_"))
v.AutomaticEnv()
var cfg Config
if err := v.Unmarshal(&cfg); err != nil {
return nil, fmt.Errorf("config unmarshal failed: %w", err)
}
return &cfg, nil
}
pkg/adapters/config/config_test.go
+224
View File
@@ -0,0 +1,224 @@
package config
import (
"bytes"
"os"
"path/filepath"
"strings"
"testing"
"github.com/spf13/cobra"
)
func TestEnvOverridesYAML(t *testing.T) {
tempDir := t.TempDir()
configPath := filepath.Join(tempDir, "config.yaml")
content := `
pg:
host: localhost
port: 5433
backup:
retention_days: 90
`
if err := os.WriteFile(configPath, []byte(content), 0o644); err != nil {
t.Fatalf("failed to write test config: %v", err)
}
t.Setenv("APP_CONFIG_LOCATION", configPath)
t.Setenv("APP_PG_HOST", "remote")
cmd := &cobra.Command{}
RegisterFlags(cmd)
var buf bytes.Buffer
oldOutput := outputWriter
outputWriter = &buf
defer func() { outputWriter = oldOutput }()
cfg, err := Load(cmd)
if err != nil {
t.Fatalf("Load failed: %v", err)
}
if cfg.PG.Host != "remote" {
t.Errorf("PG.Host = %q, want %q (env should override YAML)", cfg.PG.Host, "remote")
}
if cfg.PG.Port != 5433 {
t.Errorf("PG.Port = %d, want %d (YAML value should be preserved when env not set)", cfg.PG.Port, 5433)
}
if cfg.Backup.RetentionDays != 90 {
t.Errorf("Backup.RetentionDays = %d, want %d (from YAML)", cfg.Backup.RetentionDays, 90)
}
logOutput := buf.String()
if !strings.Contains(logOutput, configPath) {
t.Errorf("log output did not contain config path %q: %s", configPath, logOutput)
}
}
func TestDefaults(t *testing.T) {
cmd := &cobra.Command{}
RegisterFlags(cmd)
var buf bytes.Buffer
oldOutput := outputWriter
outputWriter = &buf
defer func() { outputWriter = oldOutput }()
cfg, err := Load(cmd)
if err != nil {
t.Fatalf("Load failed: %v", err)
}
if cfg.PG.Port != 5432 {
t.Errorf("PG.Port = %d, want %d", cfg.PG.Port, 5432)
}
if cfg.PG.SSLMode != "prefer" {
t.Errorf("PG.SSLMode = %q, want %q", cfg.PG.SSLMode, "prefer")
}
if len(cfg.PG.ExcludeTables) != 1 || cfg.PG.ExcludeTables[0] != "e2e_one_time_keys_json" {
t.Errorf("PG.ExcludeTables = %v, want [e2e_one_time_keys_json]", cfg.PG.ExcludeTables)
}
if cfg.Backup.RetentionDays != 180 {
t.Errorf("Backup.RetentionDays = %d, want %d", cfg.Backup.RetentionDays, 180)
}
if cfg.Backup.Cron != "0 0 3 * * *" {
t.Errorf("Backup.Cron = %q, want %q", cfg.Backup.Cron, "0 0 3 * * *")
}
if cfg.PQScheme != 0x0006 {
t.Errorf("PQScheme = 0x%04x, want 0x%04x", cfg.PQScheme, 0x0006)
}
if cfg.ClassicalScheme != 0x0007 {
t.Errorf("ClassicalScheme = 0x%04x, want 0x%04x", cfg.ClassicalScheme, 0x0007)
}
if cfg.Healthz.Port != 8080 {
t.Errorf("Healthz.Port = %d, want %d", cfg.Healthz.Port, 8080)
}
if cfg.Log.Level != "info" {
t.Errorf("Log.Level = %q, want %q", cfg.Log.Level, "info")
}
logOutput := buf.String()
if !strings.Contains(logOutput, "Configuration file not found") {
t.Errorf("log output did not contain 'Configuration file not found': %s", logOutput)
}
}
func TestFlagsOverrideEnv(t *testing.T) {
t.Setenv("APP_PG_HOST", "env-host")
cmd := &cobra.Command{}
RegisterFlags(cmd)
if err := cmd.ParseFlags([]string{"--pg-host", "flag-host"}); err != nil {
t.Fatalf("ParseFlags failed: %v", err)
}
var buf bytes.Buffer
oldOutput := outputWriter
outputWriter = &buf
defer func() { outputWriter = oldOutput }()
cfg, err := Load(cmd)
if err != nil {
t.Fatalf("Load failed: %v", err)
}
if cfg.PG.Host != "flag-host" {
t.Errorf("PG.Host = %q, want %q (flag should override env)", cfg.PG.Host, "flag-host")
}
}
func TestAllConfigFields(t *testing.T) {
tempDir := t.TempDir()
configPath := filepath.Join(tempDir, "config.yaml")
content := `
pg:
host: db.example.com
port: 5432
user: synapse
password: secret
database: synapse_db
sslmode: require
exclude_tables:
- table1
- table2
backup:
dir: /backups
retention_days: 30
cron: "0 0 * * *"
pq_scheme: 0x0001
classical_scheme: 0x0002
pq_public_key_path: /keys/pq.pub
classical_public_key_path: /keys/classical.pub
healthz:
port: 9090
log:
level: debug
`
if err := os.WriteFile(configPath, []byte(content), 0o644); err != nil {
t.Fatalf("failed to write test config: %v", err)
}
t.Setenv("APP_CONFIG_LOCATION", configPath)
cmd := &cobra.Command{}
RegisterFlags(cmd)
var buf bytes.Buffer
oldOutput := outputWriter
outputWriter = &buf
defer func() { outputWriter = oldOutput }()
cfg, err := Load(cmd)
if err != nil {
t.Fatalf("Load failed: %v", err)
}
if cfg.PG.Host != "db.example.com" {
t.Errorf("PG.Host = %q, want %q", cfg.PG.Host, "db.example.com")
}
if cfg.PG.User != "synapse" {
t.Errorf("PG.User = %q, want %q", cfg.PG.User, "synapse")
}
if cfg.PG.Password != "secret" {
t.Errorf("PG.Password = %q, want %q", cfg.PG.Password, "secret")
}
if cfg.PG.Database != "synapse_db" {
t.Errorf("PG.Database = %q, want %q", cfg.PG.Database, "synapse_db")
}
if cfg.PG.SSLMode != "require" {
t.Errorf("PG.SSLMode = %q, want %q", cfg.PG.SSLMode, "require")
}
if len(cfg.PG.ExcludeTables) != 2 || cfg.PG.ExcludeTables[0] != "table1" {
t.Errorf("PG.ExcludeTables = %v, want [table1 table2]", cfg.PG.ExcludeTables)
}
if cfg.Backup.Dir != "/backups" {
t.Errorf("Backup.Dir = %q, want %q", cfg.Backup.Dir, "/backups")
}
if cfg.Backup.Cron != "0 0 * * *" {
t.Errorf("Backup.Cron = %q, want %q", cfg.Backup.Cron, "0 0 * * *")
}
if cfg.PQScheme != 0x0001 {
t.Errorf("PQScheme = 0x%04x, want 0x%04x", cfg.PQScheme, 0x0001)
}
if cfg.ClassicalScheme != 0x0002 {
t.Errorf("ClassicalScheme = 0x%04x, want 0x%04x", cfg.ClassicalScheme, 0x0002)
}
if cfg.PQPublicKeyPath != "/keys/pq.pub" {
t.Errorf("PQPublicKeyPath = %q, want %q", cfg.PQPublicKeyPath, "/keys/pq.pub")
}
if cfg.ClassicalPublicKeyPath != "/keys/classical.pub" {
t.Errorf("ClassicalPublicKeyPath = %q, want %q", cfg.ClassicalPublicKeyPath, "/keys/classical.pub")
}
if cfg.Healthz.Port != 9090 {
t.Errorf("Healthz.Port = %d, want %d", cfg.Healthz.Port, 9090)
}
if cfg.Log.Level != "debug" {
t.Errorf("Log.Level = %q, want %q", cfg.Log.Level, "debug")
}
logOutput := buf.String()
if !strings.Contains(logOutput, configPath) {
t.Errorf("log output did not contain config path %q: %s", configPath, logOutput)
}
}
pkg/adapters/crypto/composite/composite.go
+624
View File
@@ -0,0 +1,624 @@
// Package composite implements the v2 .pqenc artifact format: a dual-KEM
// (post-quantum + classical) hybrid AEAD with inherent downgrade resistance.
//
// Layout of an artifact (all multi-byte fields are big-endian):
//
// [magic u32 = 0x47535051 "GSPQ"] [0:4]
// [version u16 = 0x0002] [4:6]
// [flags u32 = 0x00000000 (reserved, must==0)] [6:10]
// [nRecipients u8] [10]
//
// for each recipient slot i (positional: slot 0 = PQ, slot 1 = classical):
// [schemeID u16] [keyID 8B] [ctLen u32] [ciphertext ctLen bytes]
//
// [wrapNonce 12B] [wrappedCEK 48B] [firstPayloadNonce 12B]
//
// chunk records (until a chunk with flags&0x01==1 is seen):
// [len u32 = ciphertext length incl. 16B tag] [flags u8] [ciphertext]
//
// The content key (CEK, 32B) is wrapped with kekFinal — the HKDF combiner
// of the two KEM shared secrets — using AES-256-GCM (AAD = version u16 BE).
// Payload is encrypted with AES-256-GCM under CEK in 64 KiB chunks; a
// zero-length final marker chunk (flags&0x01==1) is ALWAYS emitted on EOF
// regardless of the previous chunk's fullness, providing explicit AEAD
// integrity for the logical end-of-stream.
package composite
import (
"bytes"
"crypto/aes"
"crypto/cipher"
"crypto/hkdf"
"crypto/sha256"
"encoding/binary"
"errors"
"io"
"git.tswf.io/infra/go-synapse-backupper/pkg/domain/crypto"
)
// Sentinel errors surfaced by the composite Encryptor/Decryptor.
var (
// ErrMalformedHeader indicates the artifact header is structurally
// invalid (bad magic, reserved flags, out-of-range nRecipients, oversized
// recipient ciphertext length, or premature EOF while reading fixed
// metadata).
ErrMalformedHeader = errors.New("composite: malformed header")
// ErrUnsupportedVersion indicates the artifact's version field is not
// 0x0002. Decryption stops at the version check — no AES/GCM operations
// are attempted and no recipient state is allocated.
ErrUnsupportedVersion = errors.New("composite: unsupported artifact version")
// ErrWrongKeys indicates one of the supplied private keys does not match
// the recipient slot it was routed to (positional). Signal: the priv's
// KeyID() does not equal the slot's keyID, OR the KEM Decapsulate failed
// for the slot's ciphertext.
ErrWrongKeys = errors.New("composite: wrong recipient keys")
// ErrTamperingDetected indicates the wrappedCEK or a payload chunk failed
// AES-GCM authentication: the cancellation or modification of ciphertext
// bytes is cryptographically rejected.
ErrTamperingDetected = errors.New("composite: tampering detected")
// ErrNonceCounterWrapped indicates the per-chunk 64-bit counter
// (chunkNonce[4:12]) wrapped around to zero while encrypting or
// decrypting an additional chunk — the nonce sequence is exhausted.
ErrNonceCounterWrapped = errors.New("composite: nonce counter wrapped")
// ErrMalformedChunk indicates a chunk record failed structural
// validation: zero-length non-final chunk (infinite-loop DoS) or
// oversized ciphertext (over the 64 KiB+16 maximum).
ErrMalformedChunk = errors.New("composite: malformed chunk")
// ErrUnexpectedEOF indicates the chunk stream ended before any chunk
// with flags&0x01==1 (logical end-of-stream marker) was observed.
ErrUnexpectedEOF = errors.New("composite: unexpected end of stream")
)
// Format constants.
const (
magic uint32 = 0x47535051 // "GSPQ"
version uint16 = 0x0002
flags uint32 = 0x00000000
maxRecipients int = 2
chunkSize int = 64 * 1024
gcmTagLen int = 16
maxRecipientCiphertextLen int = 1 << 20 // MiB cap on a single recipient ciphertext
wrapNonceLen int = 12
wrappedCekLen int = 48 // 32-byte CEK + 16-byte GCM tag
firstPayloadNonceLen int = 12
kekLen int = 32
infoPq string = "git.tswf.io/infra/go-synapse-backupper/v2/kek/pq"
infoComposite string = "git.tswf.io/infra/go-synapse-backupper/v2/kek/composite"
)
// Chunk flag bits.
const (
flagFinal byte = 0x01
)
// encryptor is the composite Encryptor implementation backed by a Registry
// of KEM factories. Recipients are routed positionally: slot 0 = PQ, slot 1
// = classical.
type encryptor struct {
registry crypto.Registry
}
// NewEncryptor returns a composite Encryptor that resolves KEM schemes via
// the provided Registry (Registry.Lookup(schemeID)).
func NewEncryptor(
registry crypto.Registry,
) crypto.Encryptor {
return &encryptor{
registry: registry,
}
}
// Encrypt encrypts plaintext to multiple recipients under the v2 artifact
// format and streams the result to sink. Exactly two recipients must be
// supplied — slot 0 (PQ) and slot 1 (classical).
func (e *encryptor) Encrypt(
plaintext io.Reader,
recipients []crypto.RecipientPub,
sink io.Writer,
rand io.Reader,
) error {
if len(recipients) != maxRecipients {
return ErrMalformedHeader
}
pqPub, classicalPub := recipients[0], recipients[1]
// Generate the per-message content key (32B for AES-256).
cek := make([]byte, kekLen)
if _, err := io.ReadFull(rand, cek); err != nil {
return err
}
// Bind KEM adapters through the registry — no direct adapter imports.
pqFactory, err := e.registry.Lookup(pqPub.SchemeID())
if err != nil {
return err
}
classicalFactory, err := e.registry.Lookup(classicalPub.SchemeID())
if err != nil {
return err
}
pqKem := pqFactory()
classicalKem := classicalFactory()
// Encapsulate to each recipient.
// ADAPTER CONTRACT (pinned verbatim): adapters return (ct, ss) —
// composite unpacks in that order at each call site.
pqCt, ssPq, err := pqKem.Encapsulate(pqPub, rand)
if err != nil {
return err
}
classicalCt, ssClassical, err := classicalKem.Encapsulate(classicalPub, rand)
if err != nil {
return err
}
// HKDF combiner (verbatim per plan Metis B1):
// PRK1 = HKDF-Extract(ssPq, salt=nil)
// kek1 = HKDF-Expand(prk1, infoPq, 32)
// IKM = kek1 || ssClassical (with defensive copy of kek1)
// PRK2 = HKDF-Extract(IKM, salt=nil)
// kekFinal = HKDF-Expand(prk2, infoComposite, 32)
kekFinal, err := deriveCompositeKEK(ssPq, ssClassical)
if err != nil {
return err
}
// Wrap the CEK via AES-256-GCM with AAD = version u16 BE = {0x00, 0x02}.
wrapNonce := make([]byte, wrapNonceLen)
if _, err := io.ReadFull(rand, wrapNonce); err != nil {
return err
}
kekBlock, err := aes.NewCipher(kekFinal)
if err != nil {
return err
}
wrapGcm, err := cipher.NewGCM(kekBlock)
if err != nil {
return err
}
wrappedCek := wrapGcm.Seal(nil, wrapNonce, cek, []byte{0x00, 0x02})
// firstPayloadNonce seeds the per-chunk nonce stream.
firstPayloadNonce := make([]byte, firstPayloadNonceLen)
if _, err := io.ReadFull(rand, firstPayloadNonce); err != nil {
return err
}
// Emit the artifact header.
if err := writeHeader(
sink,
pqPub, classicalPub,
pqCt, classicalCt,
wrapNonce, wrappedCek, firstPayloadNonce,
); err != nil {
return err
}
// Encrypt the payload into chunks.
payloadBlock, err := aes.NewCipher(cek)
if err != nil {
return err
}
payloadGcm, err := cipher.NewGCM(payloadBlock)
if err != nil {
return err
}
return encryptChunks(plaintext, sink, payloadGcm, firstPayloadNonce)
}
// writeHeader serializes the v2 artifact header.
//
// Layout (see package doc):
//
// magic(4) + version(2) + flags(4) + nRecipients(1)
// + per-recipient: schemeID(2) + keyID(8) + ctLen(4) + ciphertext
// + wrapNonce(12) + wrappedCEK(48) + firstPayloadNonce(12)
func writeHeader(
w io.Writer,
pqPub, classicalPub crypto.RecipientPub,
pqCt, classicalCt,
wrapNonce, wrappedCek, firstPayloadNonce []byte,
) error {
var buf bytes.Buffer
var b4 [4]byte
binary.BigEndian.PutUint32(b4[:], magic)
buf.Write(b4[:]) // [0:4] magic
binary.BigEndian.PutUint16(b4[:2], version)
buf.Write(b4[:2]) // [4:6] version
binary.BigEndian.PutUint32(b4[:], flags)
buf.Write(b4[:]) // [6:10] flags
// [10] nRecipients — composite v2 always carries exactly two slots.
buf.WriteByte(byte(maxRecipients))
// Slot 0 (PQ).
binary.BigEndian.PutUint16(b4[:2], pqPub.SchemeID())
buf.Write(b4[:2])
if len(pqPub.KeyID()) != 8 {
return ErrMalformedHeader
}
buf.Write(pqPub.KeyID())
binary.BigEndian.PutUint32(b4[:], uint32(len(pqCt)))
buf.Write(b4[:])
buf.Write(pqCt)
// Slot 1 (classical).
binary.BigEndian.PutUint16(b4[:2], classicalPub.SchemeID())
buf.Write(b4[:2])
if len(classicalPub.KeyID()) != 8 {
return ErrMalformedHeader
}
buf.Write(classicalPub.KeyID())
binary.BigEndian.PutUint32(b4[:], uint32(len(classicalCt)))
buf.Write(b4[:])
buf.Write(classicalCt)
// wrapNonce + wrappedCEK + firstPayloadNonce.
buf.Write(wrapNonce)
buf.Write(wrappedCek)
buf.Write(firstPayloadNonce)
_, err := w.Write(buf.Bytes())
return err
}
// encryptChunks encrypts plaintext into 64 KiB AES-256-GCM chunks under CEK
// and streams them to w. A zero-length final marker chunk (flags&0x01==1)
// is ALWAYS emitted on EOF regardless of the previous chunk's fullness.
func encryptChunks(
plaintext io.Reader,
w io.Writer,
gcm cipher.AEAD,
firstPayloadNonce []byte,
) error {
chunkNonce := make([]byte, gcm.NonceSize())
copy(chunkNonce, firstPayloadNonce)
buf := make([]byte, chunkSize)
var lenB [4]byte
for {
readN, readErr := io.ReadFull(plaintext, buf)
hasData := readN > 0
eof := readErr == io.EOF || readErr == io.ErrUnexpectedEOF
if hasData {
// Body chunk, flags = 0x00.
ciphertext := gcm.Seal(nil, chunkNonce, buf[:readN], nil)
binary.BigEndian.PutUint32(lenB[:], uint32(len(ciphertext)))
if _, err := w.Write(lenB[:]); err != nil {
return err
}
if _, err := w.Write([]byte{0x00}); err != nil {
return err
}
if _, err := w.Write(ciphertext); err != nil {
return err
}
// Increment counter for the next chunk; the top 4 bytes of the
// nonce ([0:4]) are untouched.
if err := incrementCounter(chunkNonce); err != nil {
return err
}
}
if eof {
// Final marker chunk — zero-length plaintext, flags = 0x01,
// ciphertext is just the 16-byte GCM tag. ALWAYS emitted.
ciphertext := gcm.Seal(nil, chunkNonce, nil, nil)
binary.BigEndian.PutUint32(lenB[:], uint32(len(ciphertext)))
if _, err := w.Write(lenB[:]); err != nil {
return err
}
if _, err := w.Write([]byte{flagFinal}); err != nil {
return err
}
if _, err := w.Write(ciphertext); err != nil {
return err
}
return nil
}
if readErr != nil {
return readErr
}
}
}
// incrementCounter mutates chunkNonce in place: reads the 64-bit big-endian
// counter at chunkNonce[4:12], adds one, rejects wrap, writes back.
// chunkNonce[0:4] (the random base) is preserved.
func incrementCounter(chunkNonce []byte) error {
counter := binary.BigEndian.Uint64(chunkNonce[4:12])
newCounter := counter + 1
if newCounter <= counter {
return ErrNonceCounterWrapped
}
binary.BigEndian.PutUint64(chunkNonce[4:12], newCounter)
return nil
}
// decryptor is the composite Decryptor implementation. Slot routing is
// positional; keyID equality is enforced as a fast wrong-key reject before
// any AEAD operation.
type decryptor struct {
registry crypto.Registry
}
// NewDecryptor returns a composite Decryptor that resolves KEM schemes via
// the provided Registry.
func NewDecryptor(
registry crypto.Registry,
) crypto.Decryptor {
return &decryptor{
registry: registry,
}
}
// Decrypt parses the v2 artifact from src, decapsulates per slot using the
// supplied private keys (positional: slot 0 ← privs[0], slot 1 ← privs[1]),
// re-derives the composite KEK, unwraps the CEK, and streams decrypted
// plaintext chunks to plaintext.
func (d *decryptor) Decrypt(
src io.Reader,
privs []crypto.RecipientPriv,
plaintext io.Writer,
) error {
// Prefix: magic(4) + version(2) + flags(4) + nRecipients(1) = 11 bytes.
const prefixLen = 11
var prefix [prefixLen]byte
if _, err := io.ReadFull(src, prefix[:]); err != nil {
return ErrMalformedHeader
}
if binary.BigEndian.Uint32(prefix[0:4]) != magic {
return ErrMalformedHeader
}
fileVersion := binary.BigEndian.Uint16(prefix[4:6])
if fileVersion != version {
// Stops BEFORE any flags/nRecipients validation, before any GCM
// work, before any recipient allocation.
return ErrUnsupportedVersion
}
if binary.BigEndian.Uint32(prefix[6:10]) != flags {
return ErrMalformedHeader
}
nRecipients := int(prefix[10])
if nRecipients < 1 || nRecipients > maxRecipients {
return ErrMalformedHeader
}
if nRecipients != maxRecipients {
// Composite v2 mandates exactly two slots.
return ErrMalformedHeader
}
// Per-recipient: each slot is `(meta 14B) (ciphertext ctLen B)` INLINE —
// slot0's ciphertext lives BETWEEN slot0's metadata and slot1's metadata
// (the inline-ct layout; see package doc). So parse strictly per-slot:
// read metadata → validate ctLen ≤ max → read ct → advance to next slot.
// The ctLen ≤ maxRecipientCiphertextLen check must fire BEFORE allocating/reading
// the per-slot ciphertext (test l: no OOM on malicious oversized value).
const perSlotMetaLen = 14
type slotMeta struct {
schemeID uint16
keyID []byte
ctLen uint32
ct []byte
}
slots := make([]slotMeta, nRecipients)
for slotIndex := 0; slotIndex < nRecipients; slotIndex++ {
var meta [perSlotMetaLen]byte
if _, err := io.ReadFull(src, meta[:]); err != nil {
return ErrMalformedHeader
}
slot := &slots[slotIndex]
slot.schemeID = binary.BigEndian.Uint16(meta[0:2])
slot.keyID = append([]byte(nil), meta[2:10]...)
slot.ctLen = binary.BigEndian.Uint32(meta[10:14])
if int(slot.ctLen) > maxRecipientCiphertextLen {
return ErrMalformedHeader
}
slot.ct = make([]byte, slot.ctLen)
if slot.ctLen > 0 {
if _, err := io.ReadFull(src, slot.ct); err != nil {
return ErrMalformedHeader
}
}
}
// Trailing fixed region: wrapNonce(12) + wrappedCEK(48) + firstPayloadNonce(12) = 72B.
var tail [wrapNonceLen + wrappedCekLen + firstPayloadNonceLen]byte
if _, err := io.ReadFull(src, tail[:]); err != nil {
return ErrMalformedHeader
}
wrapNonce := tail[:wrapNonceLen]
wrappedCek := tail[wrapNonceLen : wrapNonceLen+wrappedCekLen]
firstPayloadNonce := tail[wrapNonceLen+wrappedCekLen:]
// Decapsulate per slot, routing privs positionally.
if len(privs) < nRecipients {
return ErrWrongKeys
}
sharedSecrets := make([][]byte, nRecipients)
for slotIndex := 0; slotIndex < nRecipients; slotIndex++ {
slot := slots[slotIndex]
priv := privs[slotIndex]
if priv == nil {
return ErrWrongKeys
}
if priv.SchemeID() != slot.schemeID {
return ErrWrongKeys
}
if !bytes.Equal(priv.KeyID(), slot.keyID) {
return ErrWrongKeys
}
factory, err := d.registry.Lookup(slot.schemeID)
if err != nil {
return ErrWrongKeys
}
kem := factory()
ss, err := kem.Decapsulate(priv, slot.ct)
if err != nil {
return ErrWrongKeys
}
sharedSecrets[slotIndex] = ss
}
kekFinal, err := deriveCompositeKEK(sharedSecrets[0], sharedSecrets[1])
if err != nil {
return err
}
// Unwrap CEK via AES-256-GCM. AAD = version u16 BE.
kekBlock, err := aes.NewCipher(kekFinal)
if err != nil {
return err
}
wrapGcm, err := cipher.NewGCM(kekBlock)
if err != nil {
return err
}
cek, err := wrapGcm.Open(nil, wrapNonce, wrappedCek, []byte{0x00, 0x02})
if err != nil {
return ErrTamperingDetected
}
// Setup payload AEAD under CEK.
payloadBlock, err := aes.NewCipher(cek)
if err != nil {
return err
}
payloadGcm, err := cipher.NewGCM(payloadBlock)
if err != nil {
return err
}
return decryptChunks(src, plaintext, payloadGcm, firstPayloadNonce)
}
// decryptChunks reads and decrypts chunk records until a final chunk
// (flags & flagFinal != 0) is observed.
func decryptChunks(
src io.Reader,
plaintext io.Writer,
gcm cipher.AEAD,
firstPayloadNonce []byte,
) error {
chunkNonce := make([]byte, gcm.NonceSize())
copy(chunkNonce, firstPayloadNonce)
var lenB [4]byte
var flagB [1]byte
maxChunkCtLen := uint32(chunkSize + gcmTagLen)
for {
// Read chunk length u32 BE.
_, err := io.ReadFull(src, lenB[:])
if err == io.EOF || err == io.ErrUnexpectedEOF {
// No final marker chunk observed — premature end of stream.
return ErrUnexpectedEOF
}
if err != nil {
return err
}
length := binary.BigEndian.Uint32(lenB[:])
// Read flags u8.
_, err = io.ReadFull(src, flagB[:])
if err == io.EOF || err == io.ErrUnexpectedEOF {
return ErrUnexpectedEOF
}
if err != nil {
return err
}
flagsByte := flagB[0]
isFinal := flagsByte&flagFinal != 0
// Structural validation.
if length == 0 && !isFinal {
// Zero-length non-final chunk — infinite-loop DoS.
return ErrMalformedChunk
}
if length > maxChunkCtLen {
return ErrMalformedChunk
}
// Read ciphertext.
ciphertext := make([]byte, length)
if length > 0 {
if _, err := io.ReadFull(src, ciphertext); err != nil {
return ErrUnexpectedEOF
}
}
// AEAD open.
plaintextChunk, err := gcm.Open(nil, chunkNonce, ciphertext, nil)
if err != nil {
return ErrTamperingDetected
}
if len(plaintextChunk) > 0 {
if _, err := plaintext.Write(plaintextChunk); err != nil {
return err
}
}
if isFinal {
return nil
}
if err := incrementCounter(chunkNonce); err != nil {
return err
}
}
}
// deriveCompositeKEK applies the verbatim HKDF combiner from the plan:
//
// PRK1 = HKDF-Extract(IKM=ss_pq, salt=nil)
// kek1 = HKDF-Expand(prk1, infoPq, 32)
// IKM = kek1 || ss_classical // defensive copy of kek1
// PRK2 = HKDF-Extract(IKM, salt=nil)
// kekFinal = HKDF-Expand(prk2, infoComposite, 32)
//
// Go 1.26 stdlib crypto/hkdf returns ([]byte, error) directly from Extract
// and Expand — no infinite io.Reader is involved, so neither io.ReadAll nor
// io.ReadFull is needed; the spirit of the plan's "do not use io.ReadAll on
// hkdf.Expand" guidance is preserved trivially.
func deriveCompositeKEK(
ssPq, ssClassical []byte,
) ([]byte, error) {
prk1, err := hkdf.Extract(sha256.New, ssPq, nil)
if err != nil {
return nil, err
}
kek1, err := hkdf.Expand(sha256.New, prk1, infoPq, kekLen)
if err != nil {
return nil, err
}
// Defensive copy of kek1 — append([]byte(nil), ...) avoids aliasing
// kek1's backing array when concatenating ssClassical (Metis B1).
ikmComposite := append(append([]byte(nil), kek1...), ssClassical...)
prk2, err := hkdf.Extract(sha256.New, ikmComposite, nil)
if err != nil {
return nil, err
}
kekFinal, err := hkdf.Expand(sha256.New, prk2, infoComposite, kekLen)
if err != nil {
return nil, err
}
return kekFinal, nil
}
pkg/adapters/crypto/composite/composite_test.go
+949
View File
@@ -0,0 +1,949 @@
package composite
import (
"bytes"
"crypto/aes"
"crypto/cipher"
"crypto/rand"
"crypto/sha256"
"encoding/base64"
"encoding/binary"
"encoding/json"
"errors"
"fmt"
"io"
"os"
"testing"
"git.tswf.io/infra/go-synapse-backupper/pkg/domain/crypto"
)
// ---------------------------------------------------------------------------
// Test KEM harness
//
// The composite format pins slot 0 = PQ (schemeID 0x0006, ciphertext length
// 1088) and slot 1 = classical (schemeID 0x0007, ciphertext length 32),
// matching the real mlkem768 + x25519 adapter contracts. Adapters' priv types
// are unexported and reject type-asserted impostors at Decapsulate, and the
// composite todo's scope forbids touching adapter packages — so tests exercise
// the composite with deterministic fake KEMs (registered under the SAME
// schemeIDs as the real adapters). The committed golden fixture uses these
// fakes; the composite production code is exercised end-to-end on the format,
// the HKDF combiner, AES-256-GCM wrapping, and chunked AEAD.
// ---------------------------------------------------------------------------
const (
fakePqSchemeID uint16 = 0x0006
fakeClassicalSchemeID uint16 = 0x0007
fakePqCtLen int = 1088 // matches crypto/mlkem EncapsulateKey768 ciphertext length
fakeClassicalCtLen int = 32 // matches X25519 ephemeral pubkey length
fakeSeedLen int = 32
)
// fakeKem derives a deterministic shared secret per pub/ct pair:
//
// ss = SHA256(pub_raw_or_priv_raw || ct)
//
// where pub.raw and priv.raw are both the random seed; randomness lives only
// in the ct (the call site's rand supplies ct bytes), so decapsulation with
// the matching priv always recovers the encryption-time ss.
type fakeKem struct {
schemeIDValue uint16
ctLenValue int
}
func newFakePqKem() crypto.KEM {
return &fakeKem{schemeIDValue: fakePqSchemeID, ctLenValue: fakePqCtLen}
}
func newFakeClassicalKem() crypto.KEM {
return &fakeKem{schemeIDValue: fakeClassicalSchemeID, ctLenValue: fakeClassicalCtLen}
}
func (k *fakeKem) SchemeID() uint16 { return k.schemeIDValue }
func (k *fakeKem) GenerateKeyPair(
rand io.Reader,
) (
crypto.RecipientPub,
crypto.RecipientPriv,
error,
) {
seed := make([]byte, fakeSeedLen)
if _, err := io.ReadFull(rand, seed); err != nil {
return nil, nil, err
}
return newFakePub(k.schemeIDValue, seed), newFakePriv(k.schemeIDValue, seed), nil
}
func (k *fakeKem) Encapsulate(
pub crypto.RecipientPub,
rand io.Reader,
) (
ciphertext []byte,
sharedSecret []byte,
err error,
) {
p, ok := pub.(*fakePub)
if !ok {
return nil, nil, errors.New("fakeKem: invalid pub type")
}
ciphertext = make([]byte, k.ctLenValue)
if _, err := io.ReadFull(rand, ciphertext); err != nil {
return nil, nil, err
}
return ciphertext, deriveFakeSS(p.raw, ciphertext), nil
}
func (k *fakeKem) Decapsulate(
priv crypto.RecipientPriv,
ciphertext []byte,
) (
sharedSecret []byte,
err error,
) {
p, ok := priv.(*fakePriv)
if !ok {
return nil, errors.New("fakeKem: invalid priv type")
}
if len(ciphertext) != k.ctLenValue {
return nil, errors.New("fakeKem: invalid ciphertext length")
}
return deriveFakeSS(p.raw, ciphertext), nil
}
func (k *fakeKem) LoadPriv(
raw []byte,
) (
crypto.RecipientPriv,
error,
) {
return newFakePriv(k.schemeIDValue, raw), nil
}
func deriveFakeSS(
raw, ciphertext []byte,
) []byte {
h := sha256.New()
h.Write(raw)
h.Write(ciphertext)
return h.Sum(nil)
}
// fakePub / fakePriv — deterministic raw-bytes-backed recipients.
type fakePub struct {
scheme uint16
raw []byte
keyID []byte
}
func newFakePub(
scheme uint16,
raw []byte,
) *fakePub {
h := sha256.Sum256(raw)
return &fakePub{
scheme: scheme,
raw: append([]byte(nil), raw...),
keyID: h[:8],
}
}
func (f *fakePub) SchemeID() uint16 { return f.scheme }
func (f *fakePub) KeyID() []byte { return f.keyID }
func (f *fakePub) Raw() []byte { return f.raw }
type fakePriv struct {
scheme uint16
raw []byte
keyID []byte
}
func newFakePriv(
scheme uint16,
raw []byte,
) *fakePriv {
h := sha256.Sum256(raw)
return &fakePriv{
scheme: scheme,
raw: append([]byte(nil), raw...),
keyID: h[:8],
}
}
func (f *fakePriv) SchemeID() uint16 { return f.scheme }
func (f *fakePriv) KeyID() []byte { return f.keyID }
func (f *fakePriv) Raw() []byte { return f.raw }
// fakeRegistry returns a Registry with the two fake KEMs registered under
// the v2 slot schemeIDs.
func fakeRegistry(
t *testing.T,
) crypto.Registry {
t.Helper()
reg := crypto.NewRegistry()
if err := reg.Register(fakePqSchemeID, newFakePqKem); err != nil {
t.Fatalf("register pq fake: %v", err)
}
if err := reg.Register(fakeClassicalSchemeID, newFakeClassicalKem); err != nil {
t.Fatalf("register classical fake: %v", err)
}
return reg
}
// generateFakeKeyPair generates (pub, priv) for slot schemeID from rand.
func generateFakeKeyPair(
t *testing.T,
reg crypto.Registry,
schemeID uint16,
rand io.Reader,
) (
crypto.RecipientPub,
crypto.RecipientPriv,
) {
t.Helper()
factory, err := reg.Lookup(schemeID)
if err != nil {
t.Fatalf("lookup 0x%04x: %v", schemeID, err)
}
pub, priv, err := factory().GenerateKeyPair(rand)
if err != nil {
t.Fatalf("generate 0x%04x: %v", schemeID, err)
}
return pub, priv
}
// standardHeaderLen returns the fixed artifact-header length given the two
// slot ciphertext lengths: 11 (magic+version+flags+nRecipients) +
// per-slot (14 + ctLen) + 72 (wrapNonce+wrappedCEK+firstPayloadNonce).
func standardHeaderLen(
pqCtLen, classicalCtLen int,
) int {
return 11 + (14 + pqCtLen) + (14 + classicalCtLen) + (12 + 48 + 12)
}
// countingReader wraps an io.Reader and counts how many bytes have been read
// — used by the adversarial-parser tests to assert the parser does NOT
// consume past the header before bailing out.
type countingReader struct {
r io.Reader
n int64
}
func (c *countingReader) Read(
p []byte,
) (int, error) {
readN, err := c.r.Read(p)
c.n += int64(readN)
return readN, err
}
// ---------------------------------------------------------------------------
// Tests (a) through (p)
// ---------------------------------------------------------------------------
// (a) Golden format fixture.
func TestGoldenFormat(
t *testing.T,
) {
goldenBytes := mustReadFile(t, "testdata/golden-1byte.pqenc")
// Header byte offsets pinned verbatim — if any of these breaks, the
// on-disk format has drifted and old .pqenc files won't decrypt.
// [0:4] magic u32 BE = 0x47535051
// [4:6] version u16 BE = 0x0002
// [6:10] flags u32 BE = 0x00000000
// [10] nRecipients u8 = 0x02
// [11:13] slot0 schemeID = 0x0006
// [13:21] slot0 keyID 8B
// [21:25] slot0 ctLen u32 = 1088
// [25:1113] slot0 ciphertext (1088 bytes)
// [1113:1115] slot1 schemeID = 0x0007
if binary.BigEndian.Uint32(goldenBytes[0:4]) != 0x47535051 {
t.Errorf("magic = 0x%08x, want 0x47535051", binary.BigEndian.Uint32(goldenBytes[0:4]))
}
if binary.BigEndian.Uint16(goldenBytes[4:6]) != 0x0002 {
t.Errorf("version = 0x%04x, want 0x0002", binary.BigEndian.Uint16(goldenBytes[4:6]))
}
if binary.BigEndian.Uint32(goldenBytes[6:10]) != 0x00000000 {
t.Errorf("flags = 0x%08x, want 0", binary.BigEndian.Uint32(goldenBytes[6:10]))
}
if goldenBytes[10] != 0x02 {
t.Errorf("nRecipients = 0x%02x, want 0x02", goldenBytes[10])
}
if binary.BigEndian.Uint16(goldenBytes[11:13]) != 0x0006 {
t.Errorf("slot0 schemeID = 0x%04x, want 0x0006", binary.BigEndian.Uint16(goldenBytes[11:13]))
}
if binary.BigEndian.Uint16(goldenBytes[1113:1115]) != 0x0007 {
t.Errorf("slot1 schemeID = 0x%04x, want 0x0007", binary.BigEndian.Uint16(goldenBytes[1113:1115]))
}
// Decrypt-equality: reconstruct privs from committed golden-keys.json
// and assert Decrypt yields the 0xAA plaintext committed via golden_generate.
pqPriv, classicalPriv := loadGoldenPrivs(t, "testdata/golden-keys.json")
dec := NewDecryptor(fakeRegistry(t))
out := &bytes.Buffer{}
if err := dec.Decrypt(
bytes.NewReader(goldenBytes),
[]crypto.RecipientPriv{pqPriv, classicalPriv},
out,
); err != nil {
t.Fatalf("Decrypt(golden) failed: %v", err)
}
if !bytes.Equal(out.Bytes(), []byte{0xAA}) {
t.Errorf("decrypted = %x, want [0xAA]", out.Bytes())
}
}
// (b) Round-trip on canonical input sizes.
func TestRoundTrip(
t *testing.T,
) {
sizes := []int{0, 1, 64*1024 - 1, 64 * 1024, 64*1024 + 1, 1 << 20}
for _, size := range sizes {
t.Run(fmt.Sprintf("size=%d", size), func(t *testing.T) {
plaintext := make([]byte, size)
for i := 0; i < size; i++ {
plaintext[i] = byte(i)
}
reg := fakeRegistry(t)
enc := NewEncryptor(reg)
dec := NewDecryptor(reg)
pqPub, pqPriv := generateFakeKeyPair(t, reg, fakePqSchemeID, rand.Reader)
classicalPub, classicalPriv := generateFakeKeyPair(t, reg, fakeClassicalSchemeID, rand.Reader)
var encrypted bytes.Buffer
if err := enc.Encrypt(
bytes.NewReader(plaintext),
[]crypto.RecipientPub{pqPub, classicalPub},
&encrypted,
rand.Reader,
); err != nil {
t.Fatalf("Encrypt: %v", err)
}
out := &bytes.Buffer{}
if err := dec.Decrypt(
bytes.NewReader(encrypted.Bytes()),
[]crypto.RecipientPriv{pqPriv, classicalPriv},
out,
); err != nil {
t.Fatalf("Decrypt: %v", err)
}
if !bytes.Equal(out.Bytes(), plaintext) {
t.Errorf("round-trip mismatch: got %d bytes, want %d", out.Len(), len(plaintext))
}
})
}
}
// (c) Empty plaintext produces exactly ONE chunk with flags=0x01 and a 16B
// (tag-only) ciphertext.
func TestEmptyPlaintextSingleFinalChunk(
t *testing.T,
) {
reg := fakeRegistry(t)
enc := NewEncryptor(reg)
pqPub, _ := generateFakeKeyPair(t, reg, fakePqSchemeID, rand.Reader)
classicalPub, _ := generateFakeKeyPair(t, reg, fakeClassicalSchemeID, rand.Reader)
var encrypted bytes.Buffer
if err := enc.Encrypt(
bytes.NewReader(nil),
[]crypto.RecipientPub{pqPub, classicalPub},
&encrypted,
rand.Reader,
); err != nil {
t.Fatalf("Encrypt: %v", err)
}
headerLen := standardHeaderLen(fakePqCtLen, fakeClassicalCtLen)
chunks := encrypted.Bytes()[headerLen:]
// Expected record: [len=16 u32 (4B)][flags=0x01 (1B)][ciphertext (16B)].
if len(chunks) != 4+1+16 {
t.Fatalf("expected 21-byte chunk record, got %d bytes", len(chunks))
}
if ctLen := binary.BigEndian.Uint32(chunks[0:4]); ctLen != 16 {
t.Errorf("ctLen = %d, want 16 (tag-only)", ctLen)
}
if chunks[4] != 0x01 {
t.Errorf("flags = 0x%02x, want 0x01", chunks[4])
}
if len(chunks[5:]) != 16 {
t.Errorf("ciphertext = %d bytes, want 16 (tag-only)", len(chunks[5:]))
}
}
// (d) Exactly-64KiB input produces TWO chunks: full body (flags=0x00) and
// zero-length final marker (flags=0x01).
func TestExactly64KiBTwoChunks(
t *testing.T,
) {
reg := fakeRegistry(t)
enc := NewEncryptor(reg)
pqPub, _ := generateFakeKeyPair(t, reg, fakePqSchemeID, rand.Reader)
classicalPub, _ := generateFakeKeyPair(t, reg, fakeClassicalSchemeID, rand.Reader)
plaintext := bytes.Repeat([]byte{0xCC}, 64*1024)
var encrypted bytes.Buffer
if err := enc.Encrypt(
bytes.NewReader(plaintext),
[]crypto.RecipientPub{pqPub, classicalPub},
&encrypted,
rand.Reader,
); err != nil {
t.Fatalf("Encrypt: %v", err)
}
headerLen := standardHeaderLen(fakePqCtLen, fakeClassicalCtLen)
chunks := encrypted.Bytes()[headerLen:]
// Chunk 1: full body. ct = 64 KiB plaintext + 16B tag.
const bodyCtLen = 64*1024 + 16
if ctLen := binary.BigEndian.Uint32(chunks[0:4]); ctLen != bodyCtLen {
t.Errorf("chunk1 ctLen = %d, want %d", ctLen, bodyCtLen)
}
if chunks[4] != 0x00 {
t.Errorf("chunk1 flags = 0x%02x, want 0x00", chunks[4])
}
// Chunk 2: zero-length final marker (ct = 16B tag), flags = 0x01.
chunk2Start := 4 + 1 + bodyCtLen
if chunk2Start+5 > len(chunks) {
t.Fatalf("file truncated before chunk 2: need offset %d, have %d", chunk2Start+5, len(chunks))
}
if ctLen := binary.BigEndian.Uint32(chunks[chunk2Start : chunk2Start+4]); ctLen != 16 {
t.Errorf("chunk2 ctLen = %d, want 16 (zero-length marker)", ctLen)
}
if chunks[chunk2Start+4] != 0x01 {
t.Errorf("chunk2 flags = 0x%02x, want 0x01", chunks[chunk2Start+4])
}
chunk3Start := chunk2Start + 4 + 1 + 16
if chunk3Start != len(chunks) {
t.Errorf("expected exactly 2 chunks; remaining = %d bytes after chunk 2", len(chunks)-chunk3Start)
}
}
// (e) Counter wrap-around: with chunkNonce[4:12]=0xFFFFFFFFFFFFFFFF, an
// attempt to encrypt a SECOND body chunk fails on counter increment and
// returns ErrNonceCounterWrapped.
func TestCounterWraparound(
t *testing.T,
) {
// firstPayloadNonce: slot 0..3 = arbitrary base; slot 4..11 = 0xFF*8.
firstPayloadNonce := make([]byte, 12)
firstPayloadNonce[0] = 0xde
firstPayloadNonce[1] = 0xad
firstPayloadNonce[2] = 0xbe
firstPayloadNonce[3] = 0xef
for i := 4; i < 12; i++ {
firstPayloadNonce[i] = 0xFF
}
cek := make([]byte, 32)
if _, err := io.ReadFull(rand.Reader, cek); err != nil {
t.Fatalf("rand: %v", err)
}
block, err := aes.NewCipher(cek)
if err != nil {
t.Fatalf("aes: %v", err)
}
gcm, err := cipher.NewGCM(block)
if err != nil {
t.Fatalf("gcm: %v", err)
}
// 64 KiB + 1 byte forces 2 body chunks; incrementing after chunk 1
// wraps to 0 and ErrNonceCounterWrapped (the second chunk's emission
// never happens).
input := make([]byte, chunkSize+1)
var out bytes.Buffer
err = encryptChunks(bytes.NewReader(input), &out, gcm, firstPayloadNonce)
if !errors.Is(err, ErrNonceCounterWrapped) {
t.Errorf("expected ErrNonceCounterWrapped, got %v", err)
}
}
// (f) Tamper 1 byte in payload → ErrTamperingDetected.
func TestTamperPayload(
t *testing.T,
) {
reg := fakeRegistry(t)
enc := NewEncryptor(reg)
dec := NewDecryptor(reg)
pqPub, pqPriv := generateFakeKeyPair(t, reg, fakePqSchemeID, rand.Reader)
classicalPub, classicalPriv := generateFakeKeyPair(t, reg, fakeClassicalSchemeID, rand.Reader)
plaintext := bytes.Repeat([]byte{0x88}, 64*1024+1) // enough to produce a body chunk + final
var encrypted bytes.Buffer
if err := enc.Encrypt(
bytes.NewReader(plaintext),
[]crypto.RecipientPub{pqPub, classicalPub},
&encrypted,
rand.Reader,
); err != nil {
t.Fatalf("Encrypt: %v", err)
}
buf := encrypted.Bytes()
headerLen := standardHeaderLen(fakePqCtLen, fakeClassicalCtLen)
tamperIdx := headerLen + 4 + 1 + 8 // into first chunk ciphertext, past len + flags
if tamperIdx >= len(buf) {
t.Fatalf("file too small to tamper: idx=%d len=%d", tamperIdx, len(buf))
}
buf[tamperIdx] ^= 0x01
out := &bytes.Buffer{}
err := dec.Decrypt(
bytes.NewReader(buf),
[]crypto.RecipientPriv{pqPriv, classicalPriv},
out,
)
if !errors.Is(err, ErrTamperingDetected) {
t.Errorf("expected ErrTamperingDetected, got %v", err)
}
}
// (g) Tamper 1 byte in wrappedCEK → ErrTamperingDetected.
func TestTamperWrappedCEK(
t *testing.T,
) {
reg := fakeRegistry(t)
enc := NewEncryptor(reg)
dec := NewDecryptor(reg)
pqPub, pqPriv := generateFakeKeyPair(t, reg, fakePqSchemeID, rand.Reader)
classicalPub, classicalPriv := generateFakeKeyPair(t, reg, fakeClassicalSchemeID, rand.Reader)
var encrypted bytes.Buffer
if err := enc.Encrypt(
bytes.NewReader([]byte{0xAA}),
[]crypto.RecipientPub{pqPub, classicalPub},
&encrypted,
rand.Reader,
); err != nil {
t.Fatalf("Encrypt: %v", err)
}
buf := encrypted.Bytes()
// wrappedCEK starts at: header-prefix (11) + slot0 (+ct) + slot1 (+ct) + wrapNonce (12).
wrapOffset := 11 + (14 + fakePqCtLen) + (14 + fakeClassicalCtLen) + 12
if wrapOffset+wrappedCekLen > len(buf) {
t.Fatalf("file too short for wrappedCEK")
}
buf[wrapOffset+5] ^= 0x01
out := &bytes.Buffer{}
err := dec.Decrypt(
bytes.NewReader(buf),
[]crypto.RecipientPriv{pqPriv, classicalPriv},
out,
)
if !errors.Is(err, ErrTamperingDetected) {
t.Errorf("expected ErrTamperingDetected, got %v", err)
}
}
// (h) Wrong priv key (swap pq.priv with another) → ErrWrongKeys.
func TestWrongPrivKey(
t *testing.T,
) {
reg := fakeRegistry(t)
enc := NewEncryptor(reg)
dec := NewDecryptor(reg)
pqPub, _ := generateFakeKeyPair(t, reg, fakePqSchemeID, rand.Reader)
classicalPub, classicalPriv := generateFakeKeyPair(t, reg, fakeClassicalSchemeID, rand.Reader)
// Different PQ priv — fresh seed, hence different KeyID.
_, wrongPqPriv := generateFakeKeyPair(t, reg, fakePqSchemeID, rand.Reader)
// Sanity: the wrong priv's keyID must not collide with the original
// pub's (otherwise this test would degrade into a keyID-collision case).
if bytes.Equal(wrongPqPriv.KeyID(), pqPub.KeyID()) {
t.Fatalf("wrongPqPriv keyID accidentally collides with pqPub keyID; reseed")
}
var encrypted bytes.Buffer
if err := enc.Encrypt(
bytes.NewReader([]byte{0x11, 0x22, 0x33}),
[]crypto.RecipientPub{pqPub, classicalPub},
&encrypted,
rand.Reader,
); err != nil {
t.Fatalf("Encrypt: %v", err)
}
out := &bytes.Buffer{}
err := dec.Decrypt(
bytes.NewReader(encrypted.Bytes()),
[]crypto.RecipientPriv{wrongPqPriv, classicalPriv},
out,
)
if !errors.Is(err, ErrWrongKeys) {
t.Errorf("expected ErrWrongKeys, got %v", err)
}
}
// (i) Format conformance: magic, version, nRecipients.
func TestFormatConformance(
t *testing.T,
) {
reg := fakeRegistry(t)
enc := NewEncryptor(reg)
pqPub, _ := generateFakeKeyPair(t, reg, fakePqSchemeID, rand.Reader)
classicalPub, _ := generateFakeKeyPair(t, reg, fakeClassicalSchemeID, rand.Reader)
var out bytes.Buffer
if err := enc.Encrypt(
bytes.NewReader([]byte{0x42}),
[]crypto.RecipientPub{pqPub, classicalPub},
&out,
rand.Reader,
); err != nil {
t.Fatalf("Encrypt: %v", err)
}
outBytes := out.Bytes()
if binary.BigEndian.Uint32(outBytes[0:4]) != 0x47535051 {
t.Errorf("magic = 0x%08x, want 0x47535051", binary.BigEndian.Uint32(outBytes[0:4]))
}
if binary.BigEndian.Uint16(outBytes[4:6]) != 0x0002 {
t.Errorf("version = 0x%04x, want 0x0002", binary.BigEndian.Uint16(outBytes[4:6]))
}
if outBytes[10] != 0x02 {
t.Errorf("nRecipients = 0x%02x, want 0x02", outBytes[10])
}
}
// (j) Adversarial parser: version==0x0001 → ErrUnsupportedVersion, with no
// GCM operations attempted (proven by the post-validation byte counter
// remaining at the prefix length — the parser does not consume past the
// header before bailing).
func TestUnsupportedVersionNoGCM(
t *testing.T,
) {
reg := fakeRegistry(t)
dec := NewDecryptor(reg)
// File: magic + version=0x0001 + flags + nRecipients=0x02 + filler.
var buf bytes.Buffer
_ = binary.Write(&buf, binary.BigEndian, uint32(0x47535051)) // magic
_ = binary.Write(&buf, binary.BigEndian, uint16(0x0001)) // version (downgrade probe)
buf.Write(make([]byte, 200)) // filler
// Dummy privs — irrelevant because the parser bails at version check,
// but Decrypt accepts the slice shape.
_, pqPriv := generateFakeKeyPair(t, reg, fakePqSchemeID, rand.Reader)
_, classicalPriv := generateFakeKeyPair(t, reg, fakeClassicalSchemeID, rand.Reader)
reader := &countingReader{r: bytes.NewReader(buf.Bytes())}
out := &bytes.Buffer{}
err := dec.Decrypt(
reader,
[]crypto.RecipientPriv{pqPriv, classicalPriv},
out,
)
if !errors.Is(err, ErrUnsupportedVersion) {
t.Errorf("expected ErrUnsupportedVersion, got %v", err)
}
// The parser consumed only the 11-byte prefix — no further bytes read,
// hence no GCM operations attempted.
if reader.n != 11 {
t.Errorf("Decrypt consumed %d bytes post-validation; expected exactly 11 (the fixed prefix)", reader.n)
}
}
// (k) nRecipients==0 → ErrMalformedHeader.
func TestZeroRecipients(
t *testing.T,
) {
var buf bytes.Buffer
_ = binary.Write(&buf, binary.BigEndian, uint32(0x47535051)) // magic
_ = binary.Write(&buf, binary.BigEndian, uint16(0x0002)) // version
_ = binary.Write(&buf, binary.BigEndian, uint32(0)) // flags
buf.WriteByte(0x00) // nRecipients = 0
buf.Write(make([]byte, 64)) // filler
dec := NewDecryptor(fakeRegistry(t))
err := dec.Decrypt(bytes.NewReader(buf.Bytes()), nil, &bytes.Buffer{})
if !errors.Is(err, ErrMalformedHeader) {
t.Errorf("expected ErrMalformedHeader, got %v", err)
}
}
// (l) nRecipients>2 OR ctLen > maxRecipientCiphertextLen →
// ErrMalformedHeader BEFORE io.ReadFull attempts to allocate the
// oversized ciphertext buffer.
func TestMalformedHeaderCtLenOverflow(
t *testing.T,
) {
t.Run("nRecipients_gt_2", func(t *testing.T) {
var buf bytes.Buffer
_ = binary.Write(&buf, binary.BigEndian, uint32(0x47535051)) // magic
_ = binary.Write(&buf, binary.BigEndian, uint16(0x0002)) // version
_ = binary.Write(&buf, binary.BigEndian, uint32(0)) // flags
buf.WriteByte(0x03) // nRecipients = 3
buf.Write(make([]byte, 200)) // filler
dec := NewDecryptor(fakeRegistry(t))
err := dec.Decrypt(bytes.NewReader(buf.Bytes()), nil, &bytes.Buffer{})
if !errors.Is(err, ErrMalformedHeader) {
t.Errorf("expected ErrMalformedHeader, got %v", err)
}
})
t.Run("ctLen_overflow", func(t *testing.T) {
var buf bytes.Buffer
_ = binary.Write(&buf, binary.BigEndian, uint32(0x47535051)) // magic
_ = binary.Write(&buf, binary.BigEndian, uint16(0x0002)) // version
_ = binary.Write(&buf, binary.BigEndian, uint32(0)) // flags
buf.WriteByte(0x02) // nRecipients = 2
// Slot 0 metadata only — schemeID, keyID, ctLen = 2 MiB (over the
// 1<<20 cap). The parser validates ctLen BEFORE allocating and
// reading per-slot ciphertext bytes, so it must reject at this
// point without attempting io.ReadFull of an oversized buffer.
_ = binary.Write(&buf, binary.BigEndian, uint16(0x0006))
buf.Write(make([]byte, 8)) // keyID
_ = binary.Write(&buf, binary.BigEndian, uint32(2*1024*1024)) // ctLen
reader := &countingReader{r: bytes.NewReader(buf.Bytes())}
dec := NewDecryptor(fakeRegistry(t))
_, pqPriv := generateFakeKeyPair(t, fakeRegistry(t), fakePqSchemeID, rand.Reader)
_, classicalPriv := generateFakeKeyPair(t, fakeRegistry(t), fakeClassicalSchemeID, rand.Reader)
err := dec.Decrypt(
reader,
[]crypto.RecipientPriv{pqPriv, classicalPriv},
&bytes.Buffer{},
)
if !errors.Is(err, ErrMalformedHeader) {
t.Errorf("expected ErrMalformedHeader, got %v", err)
}
// Consumed exactly prefix(11) + slot0 metadata(14) = 25 bytes — the
// ctLen validation fired before reading any slot1 metadata or any
// per-slot ciphertext.
if reader.n != 25 {
t.Errorf("Decrypt consumed %d bytes; expected 25 (no io.ReadFull of oversized ct)", reader.n)
}
})
}
// (m) Chunk record with length==0 AND flags&0x01==0 → ErrMalformedChunk
// (prevents an infinite-loop DoS where the parser keeps scanning zero-size
// non-final chunks).
func TestZeroLengthNonFinalChunk(
t *testing.T,
) {
reg := fakeRegistry(t)
enc := NewEncryptor(reg)
dec := NewDecryptor(reg)
pqPub, pqPriv := generateFakeKeyPair(t, reg, fakePqSchemeID, rand.Reader)
classicalPub, classicalPriv := generateFakeKeyPair(t, reg, fakeClassicalSchemeID, rand.Reader)
var encrypted bytes.Buffer
if err := enc.Encrypt(
bytes.NewReader([]byte{0xAA}),
[]crypto.RecipientPub{pqPub, classicalPub},
&encrypted,
rand.Reader,
); err != nil {
t.Fatalf("Encrypt: %v", err)
}
headerLen := standardHeaderLen(fakePqCtLen, fakeClassicalCtLen)
var corrupt bytes.Buffer
corrupt.Write(encrypted.Bytes()[:headerLen])
_ = binary.Write(&corrupt, binary.BigEndian, uint32(0)) // ctLen = 0
corrupt.WriteByte(0x00) // flags = 0x00 (NOT final)
out := &bytes.Buffer{}
err := dec.Decrypt(
bytes.NewReader(corrupt.Bytes()),
[]crypto.RecipientPriv{pqPriv, classicalPriv},
out,
)
if !errors.Is(err, ErrMalformedChunk) {
t.Errorf("expected ErrMalformedChunk, got %v", err)
}
}
// (n) Chunk record with length > 64*1024+16 → ErrMalformedChunk.
func TestOversizedChunk(
t *testing.T,
) {
reg := fakeRegistry(t)
enc := NewEncryptor(reg)
dec := NewDecryptor(reg)
pqPub, pqPriv := generateFakeKeyPair(t, reg, fakePqSchemeID, rand.Reader)
classicalPub, classicalPriv := generateFakeKeyPair(t, reg, fakeClassicalSchemeID, rand.Reader)
var encrypted bytes.Buffer
if err := enc.Encrypt(
bytes.NewReader([]byte{0xAA}),
[]crypto.RecipientPub{pqPub, classicalPub},
&encrypted,
rand.Reader,
); err != nil {
t.Fatalf("Encrypt: %v", err)
}
headerLen := standardHeaderLen(fakePqCtLen, fakeClassicalCtLen)
var corrupt bytes.Buffer
corrupt.Write(encrypted.Bytes()[:headerLen])
_ = binary.Write(&corrupt, binary.BigEndian, uint32(chunkSize+gcmTagLen+1)) // oversized
corrupt.WriteByte(0x00) // flags = 0x00
out := &bytes.Buffer{}
err := dec.Decrypt(
bytes.NewReader(corrupt.Bytes()),
[]crypto.RecipientPriv{pqPriv, classicalPriv},
out,
)
if !errors.Is(err, ErrMalformedChunk) {
t.Errorf("expected ErrMalformedChunk, got %v", err)
}
}
// (o) End-of-stream reached BEFORE any chunk with flags&0x01==1
// (truncated file after a body chunk with no final marker) →
// ErrUnexpectedEOF.
func TestPrematureEOF(
t *testing.T,
) {
reg := fakeRegistry(t)
enc := NewEncryptor(reg)
dec := NewDecryptor(reg)
pqPub, pqPriv := generateFakeKeyPair(t, reg, fakePqSchemeID, rand.Reader)
classicalPub, classicalPriv := generateFakeKeyPair(t, reg, fakeClassicalSchemeID, rand.Reader)
plaintext := bytes.Repeat([]byte{0xAB}, 64*1024+1) // 2 body chunks + final marker
var encrypted bytes.Buffer
if err := enc.Encrypt(
bytes.NewReader(plaintext),
[]crypto.RecipientPub{pqPub, classicalPub},
&encrypted,
rand.Reader,
); err != nil {
t.Fatalf("Encrypt: %v", err)
}
headerLen := standardHeaderLen(fakePqCtLen, fakeClassicalCtLen)
bodyCtLen := 64*1024 + 1 + gcmTagLen
bodyChunkRecord := 4 + 1 + bodyCtLen
truncatedLen := headerLen + bodyChunkRecord
if truncatedLen >= len(encrypted.Bytes()) {
t.Fatalf("encrypted file shorter than expected: %d vs expected truncation at %d",
len(encrypted.Bytes()), truncatedLen)
}
out := &bytes.Buffer{}
err := dec.Decrypt(
bytes.NewReader(encrypted.Bytes()[:truncatedLen]),
[]crypto.RecipientPriv{pqPriv, classicalPriv},
out,
)
if !errors.Is(err, ErrUnexpectedEOF) {
t.Errorf("expected ErrUnexpectedEOF, got %v", err)
}
}
// (p) Truncated header → ErrMalformedHeader BEFORE any recipient allocation.
func TestTruncatedHeader(
t *testing.T,
) {
dec := NewDecryptor(fakeRegistry(t))
// File is shorter than the fixed 11-byte prefix + per-recipient metadata
// (2×14=28 = 39 bytes minimum): only 30 bytes total.
var buf bytes.Buffer
_ = binary.Write(&buf, binary.BigEndian, uint32(0x47535051)) // magic
_ = binary.Write(&buf, binary.BigEndian, uint16(0x0002)) // version
_ = binary.Write(&buf, binary.BigEndian, uint32(0)) // flags
buf.WriteByte(0x02) // nRecipients = 2
buf.Write(make([]byte, 20)) // only 20 of the needed 28 metadata bytes
reader := &countingReader{r: bytes.NewReader(buf.Bytes())}
err := dec.Decrypt(reader, nil, &bytes.Buffer{})
if !errors.Is(err, ErrMalformedHeader) {
t.Errorf("expected ErrMalformedHeader, got %v", err)
}
// No recipient ct allocation happened — only the prefix (11) + partial
// metadata (20) = 31 bytes consumed; well shy of a full prefix+meta
// read that would precede any per-recipient ct allocation.
if reader.n > 39 {
t.Errorf("Decrypt consumed %d bytes; expected ≤ 39 — no recipient allocation occurred", reader.n)
}
}
// ---------------------------------------------------------------------------
// Shared test helpers
// ---------------------------------------------------------------------------
func mustReadFile(
t *testing.T,
path string,
) []byte {
t.Helper()
data, err := os.ReadFile(path)
if err != nil {
t.Fatalf("read %s: %v", path, err)
}
return data
}
func mustB64Decode(
t *testing.T,
s string,
) []byte {
t.Helper()
b, err := base64.StdEncoding.DecodeString(s)
if err != nil {
t.Fatalf("base64 decode: %v", err)
}
return b
}
// loadGoldenPrivs reconstructs the two fake privs from the committed
// golden-keys.json (the file produced by //go:build golden_generate).
type goldenKeyFile struct {
Pq string `json:"pq"`
Classical string `json:"classical"`
}
func loadGoldenPrivs(
t *testing.T,
path string,
) (
*fakePriv,
*fakePriv,
) {
t.Helper()
data := mustReadFile(t, path)
var keys goldenKeyFile
if err := json.Unmarshal(data, &keys); err != nil {
t.Fatalf("unmarshal golden keys: %v", err)
}
pqRaw := mustB64Decode(t, keys.Pq)
classicalRaw := mustB64Decode(t, keys.Classical)
if len(pqRaw) != fakeSeedLen {
t.Fatalf("pq raw len = %d, want %d", len(pqRaw), fakeSeedLen)
}
if len(classicalRaw) != fakeSeedLen {
t.Fatalf("classical raw len = %d, want %d", len(classicalRaw), fakeSeedLen)
}
return newFakePriv(fakePqSchemeID, pqRaw), newFakePriv(fakeClassicalSchemeID, classicalRaw)
}
pkg/adapters/crypto/composite/golden_generate_test.go
+120
View File
@@ -0,0 +1,120 @@
//go:build golden_generate
// The golden_generate build tag is intentionally separate so CI never
// regenerates the committed fixture. Run ONCE locally to (re)commit:
//
// ~/sdk/go1.26.5/bin/go test -tags golden_generate \
// -run TestGenerateGoldenFixture -v \
// ./pkg/adapters/crypto/composite/...
//
// Then commit the produced testdata/golden-1byte.pqenc and
// testdata/golden-keys.json. Non-`-update` runs of TestGoldenFormat load
// the committed artifacts and verify decrypt-equality.
package composite
import (
"bytes"
"crypto/sha256"
"encoding/base64"
"encoding/binary"
"encoding/json"
"fmt"
"os"
"testing"
"git.tswf.io/infra/go-synapse-backupper/pkg/domain/crypto"
)
// deterministicRand implements io.Reader via a SHA-256 counter stream so the
// golden fixture is byte-for-byte reproducible across machines and Go
// toolchain versions.
type deterministicRand struct {
seq uint64
}
func (d *deterministicRand) Read(
p []byte,
) (int, error) {
for offset := 0; offset < len(p); {
var b [8]byte
binary.BigEndian.PutUint64(b[:], d.seq)
d.seq++
out := sha256.New()
out.Write(b[:])
hashed := out.Sum(nil)
n := copy(p[offset:], hashed)
offset += n
}
return len(p), nil
}
// Hard-coded priv seeds so the committed golden-keys.json stays stable across
// builds — these are the test-only private "keys" the committed golden file
// decrypts against.
var (
goldenPqSeed = [32]byte{
0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08,
0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, 0x10,
0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18,
0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f, 0x20,
}
goldenClassicalSeed = [32]byte{
0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27, 0x28,
0x29, 0x2a, 0x2b, 0x2c, 0x2d, 0x2e, 0x2f, 0x30,
0x31, 0x32, 0x33, 0x34, 0x35, 0x36, 0x37, 0x38,
0x39, 0x3a, 0x3b, 0x3c, 0x3d, 0x3e, 0x3f, 0x40,
}
)
func TestGenerateGoldenFixture(
t *testing.T,
) {
pqPub := newFakePub(fakePqSchemeID, goldenPqSeed[:])
pqPriv := newFakePriv(fakePqSchemeID, goldenPqSeed[:])
classicalPub := newFakePub(fakeClassicalSchemeID, goldenClassicalSeed[:])
classicalPriv := newFakePriv(fakeClassicalSchemeID, goldenClassicalSeed[:])
reg := crypto.NewRegistry()
if err := reg.Register(fakePqSchemeID, newFakePqKem); err != nil {
t.Fatalf("register pq fake: %v", err)
}
if err := reg.Register(fakeClassicalSchemeID, newFakeClassicalKem); err != nil {
t.Fatalf("register classical fake: %v", err)
}
enc := NewEncryptor(reg)
rng := &deterministicRand{}
var encrypted bytes.Buffer
if err := enc.Encrypt(
bytes.NewReader([]byte{0xAA}),
[]crypto.RecipientPub{pqPub, classicalPub},
&encrypted,
rng,
); err != nil {
t.Fatalf("Encrypt: %v", err)
}
if err := os.MkdirAll("testdata", 0o755); err != nil {
t.Fatalf("mkdir testdata: %v", err)
}
if err := os.WriteFile("testdata/golden-1byte.pqenc", encrypted.Bytes(), 0o644); err != nil {
t.Fatalf("write golden file: %v", err)
}
keys := goldenKeyFile{
Pq: base64.StdEncoding.EncodeToString(pqPriv.Raw()),
Classical: base64.StdEncoding.EncodeToString(classicalPriv.Raw()),
}
marshalled, err := json.MarshalIndent(keys, "", " ")
if err != nil {
t.Fatalf("marshal keys: %v", err)
}
marshalled = append(marshalled, '\n')
if err := os.WriteFile("testdata/golden-keys.json", marshalled, 0o644); err != nil {
t.Fatalf("write keys json: %v", err)
}
fmt.Printf("Golden fixture written: testdata/golden-1byte.pqenc (%d bytes), "+
"testdata/golden-keys.json (%d bytes)\n", len(encrypted.Bytes()), len(marshalled))
}
pkg/adapters/crypto/composite/testdata/golden-keys.json
Vendored
+4
View File
@@ -0,0 +1,4 @@
{
"pq": "AQIDBAUGBwgJCgsMDQ4PEBESExQVFhcYGRobHB0eHyA=",
"classical": "ISIjJCUmJygpKissLS4vMDEyMzQ1Njc4OTo7PD0+P0A="
}
pkg/adapters/crypto/keymanager/keymanager.go
+214
View File
@@ -0,0 +1,214 @@
package keymanager
import (
"crypto/sha256"
"encoding/pem"
"errors"
"fmt"
"io"
"os"
"git.tswf.io/infra/go-synapse-backupper/pkg/domain/crypto"
)
var (
// ErrPEMTypeMismatch is returned when the PEM block type does not match
// the expected type for the given schemeID.
ErrPEMTypeMismatch = errors.New("PEM type does not match scheme")
// ErrInvalidPEM is returned when the file does not contain a valid PEM block.
ErrInvalidPEM = errors.New("invalid PEM data")
)
var pubPEMTypes = map[uint16]string{
0x0006: "ML-KEM-768 PUBLIC KEY",
0x0007: "X25519 PUBLIC KEY",
}
var privPEMTypes = map[uint16]string{
0x0006: "ML-KEM-768 PRIVATE KEY",
0x0007: "X25519 PRIVATE KEY",
}
// keyManager handles PEM encoding and decoding of recipient keys.
type keyManager struct {
registry crypto.Registry
}
// NewKeyManager creates a new KeyManager backed by the provided Registry.
func NewKeyManager(
registry crypto.Registry,
) crypto.KeyManager {
return &keyManager{
registry: registry,
}
}
// Generate creates a new key pair for the given schemeID and writes them
// as PEM blocks to pubOut and privOut.
func (k *keyManager) Generate(
schemeID uint16,
pubOut io.Writer,
privOut io.Writer,
rand io.Reader,
) error {
factory, err := k.registry.Lookup(schemeID)
if err != nil {
return err
}
kem := factory()
pub, priv, err := kem.GenerateKeyPair(rand)
if err != nil {
return err
}
pubType, ok := pubPEMTypes[schemeID]
if !ok {
return fmt.Errorf(
"unsupported scheme 0x%04x for public key PEM",
schemeID,
)
}
privType, ok := privPEMTypes[schemeID]
if !ok {
return fmt.Errorf(
"unsupported scheme 0x%04x for private key PEM",
schemeID,
)
}
pubBlock := &pem.Block{
Type: pubType,
Bytes: pub.Raw(),
}
if err := pem.Encode(pubOut, pubBlock); err != nil {
return fmt.Errorf("encode public key PEM: %w", err)
}
privBlock := &pem.Block{
Type: privType,
Bytes: priv.Raw(),
}
if err := pem.Encode(privOut, privBlock); err != nil {
return fmt.Errorf("encode private key PEM: %w", err)
}
return nil
}
// LoadPub reads a PEM-encoded public key from path and validates that its
// type matches the expected type for schemeID.
func (k *keyManager) LoadPub(
path string,
schemeID uint16,
) (
crypto.RecipientPub,
error,
) {
data, err := os.ReadFile(path)
if err != nil {
return nil, fmt.Errorf("read public key file: %w", err)
}
block, _ := pem.Decode(data)
if block == nil {
return nil, fmt.Errorf("%w: no valid PEM block found", ErrInvalidPEM)
}
expectedType, ok := pubPEMTypes[schemeID]
if !ok {
return nil, fmt.Errorf(
"unsupported scheme 0x%04x for public key PEM",
schemeID,
)
}
if block.Type != expectedType {
return nil, fmt.Errorf(
"expected PEM type %q, got %q: %w",
expectedType,
block.Type,
ErrPEMTypeMismatch,
)
}
return newRecipientPub(schemeID, block.Bytes), nil
}
// LoadPriv reads a PEM-encoded private key from path and validates that its
// type matches the expected type for schemeID.
func (k *keyManager) LoadPriv(
path string,
schemeID uint16,
) (
crypto.RecipientPriv,
error,
) {
data, err := os.ReadFile(path)
if err != nil {
return nil, fmt.Errorf("read private key file: %w", err)
}
block, _ := pem.Decode(data)
if block == nil {
return nil, fmt.Errorf("%w: no valid PEM block found", ErrInvalidPEM)
}
expectedType, ok := privPEMTypes[schemeID]
if !ok {
return nil, fmt.Errorf(
"unsupported scheme 0x%04x for private key PEM",
schemeID,
)
}
if block.Type != expectedType {
return nil, fmt.Errorf(
"expected PEM type %q, got %q: %w",
expectedType,
block.Type,
ErrPEMTypeMismatch,
)
}
factory, err := k.registry.Lookup(schemeID)
if err != nil {
return nil, err
}
kem := factory()
return kem.LoadPriv(block.Bytes)
}
// recipientPub is a generic RecipientPub implementation backed by raw bytes.
type recipientPub struct {
schemeID uint16
raw []byte
keyID []byte
}
func newRecipientPub(
schemeID uint16,
raw []byte,
) crypto.RecipientPub {
var keyID []byte
switch schemeID {
case 0x0006:
h := sha256.Sum256(raw[:8])
keyID = h[:8]
case 0x0007:
h := sha256.Sum256(raw)
keyID = h[:8]
}
return &recipientPub{
schemeID: schemeID,
raw: append([]byte(nil), raw...),
keyID: keyID,
}
}
func (r *recipientPub) SchemeID() uint16 { return r.schemeID }
func (r *recipientPub) KeyID() []byte { return r.keyID }
func (r *recipientPub) Raw() []byte { return r.raw }
pkg/adapters/crypto/keymanager/keymanager_test.go
+362
View File
@@ -0,0 +1,362 @@
package keymanager
import (
"bytes"
"crypto/rand"
"crypto/sha256"
"encoding/pem"
"errors"
"os"
"path/filepath"
"testing"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/crypto/mlkem768"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/crypto/x25519"
"git.tswf.io/infra/go-synapse-backupper/pkg/domain/crypto"
)
func makeRegistry(
t *testing.T,
) crypto.Registry {
reg := crypto.NewRegistry()
if err := reg.Register(
0x0006,
func() crypto.KEM {
return mlkem768.New()
},
); err != nil {
t.Fatalf("register mlkem768: %v", err)
}
if err := reg.Register(
0x0007,
func() crypto.KEM {
return x25519.New()
},
); err != nil {
t.Fatalf("register x25519: %v", err)
}
return reg
}
func TestGenerateMLKEM768(t *testing.T) {
reg := makeRegistry(t)
km := NewKeyManager(reg)
var pubOut, privOut bytes.Buffer
err := km.Generate(
0x0006,
&pubOut,
&privOut,
rand.Reader,
)
if err != nil {
t.Fatalf("Generate failed: %v", err)
}
pubBlock, _ := pem.Decode(pubOut.Bytes())
if pubBlock == nil {
t.Fatal("failed to decode public key PEM")
}
if pubBlock.Type != "ML-KEM-768 PUBLIC KEY" {
t.Errorf("pub PEM type = %q, want %q", pubBlock.Type, "ML-KEM-768 PUBLIC KEY")
}
if len(pubBlock.Bytes) != 1184 {
t.Errorf("pub raw len = %d, want 1184", len(pubBlock.Bytes))
}
privBlock, _ := pem.Decode(privOut.Bytes())
if privBlock == nil {
t.Fatal("failed to decode private key PEM")
}
if privBlock.Type != "ML-KEM-768 PRIVATE KEY" {
t.Errorf("priv PEM type = %q, want %q", privBlock.Type, "ML-KEM-768 PRIVATE KEY")
}
if len(privBlock.Bytes) != 64 {
t.Errorf("priv raw len = %d, want 64", len(privBlock.Bytes))
}
}
func TestGenerateX25519(t *testing.T) {
reg := makeRegistry(t)
km := NewKeyManager(reg)
var pubOut, privOut bytes.Buffer
err := km.Generate(
0x0007,
&pubOut,
&privOut,
rand.Reader,
)
if err != nil {
t.Fatalf("Generate failed: %v", err)
}
pubBlock, _ := pem.Decode(pubOut.Bytes())
if pubBlock == nil {
t.Fatal("failed to decode public key PEM")
}
if pubBlock.Type != "X25519 PUBLIC KEY" {
t.Errorf("pub PEM type = %q, want %q", pubBlock.Type, "X25519 PUBLIC KEY")
}
if len(pubBlock.Bytes) != 32 {
t.Errorf("pub raw len = %d, want 32", len(pubBlock.Bytes))
}
privBlock, _ := pem.Decode(privOut.Bytes())
if privBlock == nil {
t.Fatal("failed to decode private key PEM")
}
if privBlock.Type != "X25519 PRIVATE KEY" {
t.Errorf("priv PEM type = %q, want %q", privBlock.Type, "X25519 PRIVATE KEY")
}
if len(privBlock.Bytes) != 32 {
t.Errorf("priv raw len = %d, want 32", len(privBlock.Bytes))
}
}
func TestLoadPubMLKEM768(t *testing.T) {
reg := makeRegistry(t)
km := NewKeyManager(reg)
var pubOut, privOut bytes.Buffer
err := km.Generate(
0x0006,
&pubOut,
&privOut,
rand.Reader,
)
if err != nil {
t.Fatalf("Generate failed: %v", err)
}
dir := t.TempDir()
pubPath := filepath.Join(dir, "test.pub.pem")
if err := os.WriteFile(pubPath, pubOut.Bytes(), 0o644); err != nil {
t.Fatalf("write pub file: %v", err)
}
pub, err := km.LoadPub(pubPath, 0x0006)
if err != nil {
t.Fatalf("LoadPub failed: %v", err)
}
if pub.SchemeID() != 0x0006 {
t.Errorf("pub.SchemeID() = 0x%04x, want 0x0006", pub.SchemeID())
}
if len(pub.Raw()) != 1184 {
t.Errorf("pub.Raw() len = %d, want 1184", len(pub.Raw()))
}
expectedKeyID := sha256.Sum256(pub.Raw()[:8])
if !bytes.Equal(pub.KeyID(), expectedKeyID[:8]) {
t.Errorf("pub.KeyID() = %x, want %x", pub.KeyID(), expectedKeyID[:8])
}
}
func TestLoadPrivMLKEM768(t *testing.T) {
reg := makeRegistry(t)
km := NewKeyManager(reg)
var pubOut, privOut bytes.Buffer
err := km.Generate(
0x0006,
&pubOut,
&privOut,
rand.Reader,
)
if err != nil {
t.Fatalf("Generate failed: %v", err)
}
dir := t.TempDir()
privPath := filepath.Join(dir, "test.priv.pem")
if err := os.WriteFile(privPath, privOut.Bytes(), 0o600); err != nil {
t.Fatalf("write priv file: %v", err)
}
priv, err := km.LoadPriv(privPath, 0x0006)
if err != nil {
t.Fatalf("LoadPriv failed: %v", err)
}
if priv.SchemeID() != 0x0006 {
t.Errorf("priv.SchemeID() = 0x%04x, want 0x0006", priv.SchemeID())
}
if len(priv.Raw()) != 64 {
t.Errorf("priv.Raw() len = %d, want 64", len(priv.Raw()))
}
}
func TestLoadPubX25519(t *testing.T) {
reg := makeRegistry(t)
km := NewKeyManager(reg)
var pubOut, privOut bytes.Buffer
err := km.Generate(
0x0007,
&pubOut,
&privOut,
rand.Reader,
)
if err != nil {
t.Fatalf("Generate failed: %v", err)
}
dir := t.TempDir()
pubPath := filepath.Join(dir, "test.pub.pem")
if err := os.WriteFile(pubPath, pubOut.Bytes(), 0o644); err != nil {
t.Fatalf("write pub file: %v", err)
}
pub, err := km.LoadPub(pubPath, 0x0007)
if err != nil {
t.Fatalf("LoadPub failed: %v", err)
}
if pub.SchemeID() != 0x0007 {
t.Errorf("pub.SchemeID() = 0x%04x, want 0x0007", pub.SchemeID())
}
if len(pub.Raw()) != 32 {
t.Errorf("pub.Raw() len = %d, want 32", len(pub.Raw()))
}
expectedKeyID := sha256.Sum256(pub.Raw())
if !bytes.Equal(pub.KeyID(), expectedKeyID[:8]) {
t.Errorf("pub.KeyID() = %x, want %x", pub.KeyID(), expectedKeyID[:8])
}
}
func TestLoadPrivX25519(t *testing.T) {
reg := makeRegistry(t)
km := NewKeyManager(reg)
var pubOut, privOut bytes.Buffer
err := km.Generate(
0x0007,
&pubOut,
&privOut,
rand.Reader,
)
if err != nil {
t.Fatalf("Generate failed: %v", err)
}
dir := t.TempDir()
privPath := filepath.Join(dir, "test.priv.pem")
if err := os.WriteFile(privPath, privOut.Bytes(), 0o600); err != nil {
t.Fatalf("write priv file: %v", err)
}
priv, err := km.LoadPriv(privPath, 0x0007)
if err != nil {
t.Fatalf("LoadPriv failed: %v", err)
}
if priv.SchemeID() != 0x0007 {
t.Errorf("priv.SchemeID() = 0x%04x, want 0x0007", priv.SchemeID())
}
if len(priv.Raw()) != 32 {
t.Errorf("priv.Raw() len = %d, want 32", len(priv.Raw()))
}
}
func TestLoadPubWrongPEMType(t *testing.T) {
reg := makeRegistry(t)
km := NewKeyManager(reg)
dir := t.TempDir()
pubPath := filepath.Join(dir, "wrong.pub.pem")
block := &pem.Block{
Type: "X25519 PUBLIC KEY",
Bytes: make([]byte, 32),
}
data := pem.EncodeToMemory(block)
if err := os.WriteFile(pubPath, data, 0o644); err != nil {
t.Fatalf("write pub file: %v", err)
}
_, err := km.LoadPub(pubPath, 0x0006)
if err == nil {
t.Fatal("expected error for wrong PEM type")
}
if !errors.Is(err, ErrPEMTypeMismatch) {
t.Errorf("error = %v, want ErrPEMTypeMismatch", err)
}
}
func TestLoadPrivWrongPEMType(t *testing.T) {
reg := makeRegistry(t)
km := NewKeyManager(reg)
dir := t.TempDir()
privPath := filepath.Join(dir, "wrong.priv.pem")
block := &pem.Block{
Type: "ML-KEM-768 PRIVATE KEY",
Bytes: make([]byte, 64),
}
data := pem.EncodeToMemory(block)
if err := os.WriteFile(privPath, data, 0o600); err != nil {
t.Fatalf("write priv file: %v", err)
}
_, err := km.LoadPriv(privPath, 0x0007)
if err == nil {
t.Fatal("expected error for wrong PEM type")
}
if !errors.Is(err, ErrPEMTypeMismatch) {
t.Errorf("error = %v, want ErrPEMTypeMismatch", err)
}
}
func TestLoadPubTruncatedPEM(t *testing.T) {
reg := makeRegistry(t)
km := NewKeyManager(reg)
dir := t.TempDir()
pubPath := filepath.Join(dir, "truncated.pub.pem")
if err := os.WriteFile(pubPath, []byte("-----BEGIN ML-KEM-768 PUBLIC KEY-----\nnotbase64\n"), 0o644); err != nil {
t.Fatalf("write pub file: %v", err)
}
_, err := km.LoadPub(pubPath, 0x0006)
if err == nil {
t.Fatal("expected error for truncated PEM")
}
if !errors.Is(err, ErrInvalidPEM) {
t.Errorf("error = %v, want ErrInvalidPEM", err)
}
}
func TestLoadPrivTruncatedPEM(t *testing.T) {
reg := makeRegistry(t)
km := NewKeyManager(reg)
dir := t.TempDir()
privPath := filepath.Join(dir, "truncated.priv.pem")
if err := os.WriteFile(privPath, []byte("-----BEGIN X25519 PRIVATE KEY-----\nnotbase64\n"), 0o600); err != nil {
t.Fatalf("write priv file: %v", err)
}
_, err := km.LoadPriv(privPath, 0x0007)
if err == nil {
t.Fatal("expected error for truncated PEM")
}
if !errors.Is(err, ErrInvalidPEM) {
t.Errorf("error = %v, want ErrInvalidPEM", err)
}
}
pkg/adapters/crypto/mlkem768/mlkem768.go
+194
View File
@@ -0,0 +1,194 @@
package mlkem768
import (
"bytes"
"crypto/mlkem"
"crypto/sha256"
"crypto/sha3"
"errors"
"fmt"
"io"
"git.tswf.io/infra/go-synapse-backupper/pkg/domain/crypto"
)
// suiteID is the scheme identifier for ML-KEM-768.
const suiteID uint16 = 0x0006
// ErrDecapsulationFailed is returned when ciphertext decapsulation fails,
// typically due to a tampered or invalid ciphertext.
var ErrDecapsulationFailed = errors.New("decapsulation failed")
// DefaultRegistry is the package-level registry for ML-KEM-768.
var DefaultRegistry = crypto.NewRegistry()
// kemAdapter wraps the Go stdlib crypto/mlkem implementation to satisfy
// the pkg/domain/crypto.KEM interface.
type kemAdapter struct{}
// New creates a new KEM adapter instance.
func New() crypto.KEM {
return &kemAdapter{}
}
// SchemeID returns the ML-KEM-768 scheme identifier (0x0006).
func (k *kemAdapter) SchemeID() uint16 {
return suiteID
}
// GenerateKeyPair generates a new ML-KEM-768 key pair.
//
// Note: the rand parameter is part of the KEM interface contract but is
// ignored here because the stdlib crypto/mlkem.GenerateKey768 uses
// crypto/rand internally.
func (k *kemAdapter) GenerateKeyPair(
rand io.Reader,
) (
crypto.RecipientPub,
crypto.RecipientPriv,
error,
) {
dk, err := mlkem.GenerateKey768()
if err != nil {
return nil, nil, err
}
ek := dk.EncapsulationKey()
rawPub := ek.Bytes()
keyID := computeKeyID(rawPub)
pub := &pubKey{
key: ek,
keyID: keyID,
}
priv := &privKey{
key: dk,
keyID: keyID,
}
return pub, priv, nil
}
// Encapsulate generates a shared secret and ciphertext for the given public key.
//
// Note: the rand parameter is part of the KEM interface contract but is
// ignored here because the stdlib (*EncapsulationKey768).Encapsulate uses
// crypto/rand internally.
//
// CRITICAL: the stdlib returns (sharedKey, ciphertext) but this adapter
// swaps the order to (ciphertext, sharedSecret) to match the domain KEM
// interface contract.
func (k *kemAdapter) Encapsulate(
pub crypto.RecipientPub,
rand io.Reader,
) (
ciphertext []byte,
sharedSecret []byte,
err error,
) {
p, ok := pub.(*pubKey)
if !ok {
raw := pub.Raw()
ek, parseErr := mlkem.NewEncapsulationKey768(raw)
if parseErr != nil {
return nil, nil, fmt.Errorf("invalid public key for ML-KEM-768: %w", parseErr)
}
p = &pubKey{key: ek, keyID: computeKeyID(raw)}
}
// stdlib returns (sharedKey, ciphertext); we swap to (ciphertext, sharedSecret).
ss, ct := p.key.Encapsulate()
return ct, ss, nil
}
// LoadPriv loads an ML-KEM-768 private key from raw bytes.
func (k *kemAdapter) LoadPriv(
raw []byte,
) (
crypto.RecipientPriv,
error,
) {
dk, err := mlkem.NewDecapsulationKey768(raw)
if err != nil {
return nil, fmt.Errorf("invalid ML-KEM-768 private key: %w", err)
}
pubRaw := dk.EncapsulationKey().Bytes()
return &privKey{key: dk, keyID: computeKeyID(pubRaw)}, nil
}
// Decapsulate recovers the shared secret from a ciphertext using the private key.
// For tampered ciphertexts, it returns ErrDecapsulationFailed.
func (k *kemAdapter) Decapsulate(
priv crypto.RecipientPriv,
ciphertext []byte,
) (
sharedSecret []byte,
err error,
) {
p, ok := priv.(*privKey)
if !ok {
return nil, fmt.Errorf("invalid private key type for ML-KEM-768")
}
ss, err := p.key.Decapsulate(ciphertext)
if err != nil {
return nil, errors.Join(ErrDecapsulationFailed, err)
}
// crypto/mlkem.Decapsulate implements implicit rejection: it returns a
// pseudorandom shared secret instead of an error for invalid ciphertexts.
// Perform explicit rejection by recomputing the implicit-rejection value
// Kout = SHAKE256(z || ciphertext) and comparing it with the result.
// If they match, the ciphertext was invalid.
seed := p.key.Bytes()
z := seed[32:]
shake := sha3.NewSHAKE256()
// sha3.ShakeHash.Write/Read never return an error; explicitly ignore to satisfy errcheck.
_, _ = shake.Write(z)
_, _ = shake.Write(ciphertext)
computedKout := make([]byte, mlkem.SharedKeySize)
_, _ = shake.Read(computedKout)
if bytes.Equal(ss, computedKout) {
return nil, ErrDecapsulationFailed
}
return ss, nil
}
// pubKey wraps *mlkem.EncapsulationKey768 to satisfy crypto.RecipientPub.
type pubKey struct {
key *mlkem.EncapsulationKey768
keyID []byte
}
func (p *pubKey) SchemeID() uint16 { return suiteID }
func (p *pubKey) KeyID() []byte { return p.keyID }
func (p *pubKey) Raw() []byte { return p.key.Bytes() }
// privKey wraps *mlkem.DecapsulationKey768 to satisfy crypto.RecipientPriv.
type privKey struct {
key *mlkem.DecapsulationKey768
keyID []byte
}
func (p *privKey) SchemeID() uint16 { return suiteID }
func (p *privKey) KeyID() []byte { return p.keyID }
func (p *privKey) Raw() []byte { return p.key.Bytes() }
// computeKeyID derives the first 8 bytes of SHA-256 over the first 8 bytes of raw key material.
func computeKeyID(raw []byte) []byte {
h := sha256.Sum256(raw[:8])
return h[:8]
}
// init registers the ML-KEM-768 factory under suiteID 0x0006.
func init() {
_ = DefaultRegistry.Register(
suiteID,
func() crypto.KEM {
return New()
},
)
}
pkg/adapters/crypto/mlkem768/mlkem768_test.go
+140
View File
@@ -0,0 +1,140 @@
package mlkem768
import (
"bytes"
"crypto/sha256"
"errors"
"testing"
)
func TestGenerateKeyPair(t *testing.T) {
adapter := New()
pub, priv, err := adapter.GenerateKeyPair(nil)
if err != nil {
t.Fatalf("GenerateKeyPair failed: %v", err)
}
if pub.SchemeID() != suiteID {
t.Errorf("pub.SchemeID() = 0x%04x, want 0x%04x", pub.SchemeID(), suiteID)
}
if priv.SchemeID() != suiteID {
t.Errorf("priv.SchemeID() = 0x%04x, want 0x%04x", priv.SchemeID(), suiteID)
}
rawPub := pub.Raw()
if len(rawPub) != 1184 {
t.Errorf("pub.Raw() len = %d, want 1184", len(rawPub))
}
rawPriv := priv.Raw()
if len(rawPriv) != 64 {
t.Errorf("priv.Raw() len = %d, want 64", len(rawPriv))
}
expectedKeyID := sha256.Sum256(rawPub[:8])
if !bytes.Equal(pub.KeyID(), expectedKeyID[:8]) {
t.Errorf("pub.KeyID() = %x, want %x", pub.KeyID(), expectedKeyID[:8])
}
if !bytes.Equal(priv.KeyID(), pub.KeyID()) {
t.Errorf("priv.KeyID() = %x, want %x", priv.KeyID(), pub.KeyID())
}
}
func TestEncapsulateReturnOrder(t *testing.T) {
adapter := New()
pub, _, err := adapter.GenerateKeyPair(nil)
if err != nil {
t.Fatalf("GenerateKeyPair failed: %v", err)
}
ct, ss, err := adapter.Encapsulate(pub, nil)
if err != nil {
t.Fatalf("Encapsulate failed: %v", err)
}
if len(ct) != 1088 {
t.Errorf("ciphertext len = %d, want 1088", len(ct))
}
if len(ss) != 32 {
t.Errorf("sharedSecret len = %d, want 32", len(ss))
}
}
func TestRoundTrip(t *testing.T) {
adapter := New()
pub, priv, err := adapter.GenerateKeyPair(nil)
if err != nil {
t.Fatalf("GenerateKeyPair failed: %v", err)
}
ct, ssEnc, err := adapter.Encapsulate(pub, nil)
if err != nil {
t.Fatalf("Encapsulate failed: %v", err)
}
ssDec, err := adapter.Decapsulate(priv, ct)
if err != nil {
t.Fatalf("Decapsulate failed: %v", err)
}
if !bytes.Equal(ssEnc, ssDec) {
t.Fatalf("shared secret mismatch: encapsulate=%x, decapsulate=%x", ssEnc, ssDec)
}
}
func TestDecapsulateTamperedCiphertext(t *testing.T) {
adapter := New()
pub, priv, err := adapter.GenerateKeyPair(nil)
if err != nil {
t.Fatalf("GenerateKeyPair failed: %v", err)
}
ct, _, err := adapter.Encapsulate(pub, nil)
if err != nil {
t.Fatalf("Encapsulate failed: %v", err)
}
ct[0] ^= 0xFF
_, err = adapter.Decapsulate(priv, ct)
if err == nil {
t.Fatal("Decapsulate with tampered ciphertext: expected error, got nil")
}
if !errors.Is(err, ErrDecapsulationFailed) {
t.Errorf("Decapsulate error = %v, want ErrDecapsulationFailed", err)
}
}
func TestRegistryRegistration(t *testing.T) {
factory, err := DefaultRegistry.Lookup(suiteID)
if err != nil {
t.Fatalf("Lookup suiteID 0x%04x failed: %v", suiteID, err)
}
instance := factory()
if instance.SchemeID() != suiteID {
t.Errorf("factory() SchemeID = 0x%04x, want 0x%04x", instance.SchemeID(), suiteID)
}
}
func TestFactoryReturnsIndependentInstances(t *testing.T) {
factory, err := DefaultRegistry.Lookup(suiteID)
if err != nil {
t.Fatalf("Lookup suiteID 0x%04x failed: %v", suiteID, err)
}
one := factory()
two := factory()
if one.SchemeID() != two.SchemeID() {
t.Error("factory() returned instances with different scheme IDs")
}
}
pkg/adapters/crypto/x25519/x25519.go
+196
View File
@@ -0,0 +1,196 @@
package x25519
import (
"crypto/ecdh"
"crypto/sha256"
"errors"
"fmt"
"io"
"git.tswf.io/infra/go-synapse-backupper/pkg/domain/crypto"
)
// suiteID is the scheme identifier for X25519 ECDH KEM.
const suiteID uint16 = 0x0007
// ErrDecapsulationFailed is returned when ciphertext decapsulation fails,
// typically because the ciphertext is not a valid X25519 public key.
var ErrDecapsulationFailed = errors.New("decapsulation failed")
// DefaultRegistry is the package-level registry for X25519.
var DefaultRegistry = crypto.NewRegistry()
// kemAdapter wraps the Go stdlib crypto/ecdh X25519 implementation to satisfy
// the pkg/domain/crypto.KEM interface.
type kemAdapter struct{}
// New creates a new KEM adapter instance.
func New() crypto.KEM {
return &kemAdapter{}
}
// SchemeID returns the X25519 scheme identifier (0x0007).
func (k *kemAdapter) SchemeID() uint16 {
return suiteID
}
// GenerateKeyPair generates a new X25519 key pair.
func (k *kemAdapter) GenerateKeyPair(
rand io.Reader,
) (
crypto.RecipientPub,
crypto.RecipientPriv,
error,
) {
ecdhPriv, err := ecdh.X25519().GenerateKey(rand)
if err != nil {
return nil, nil, err
}
rawPub := ecdhPriv.PublicKey().Bytes()
keyID := computeKeyID(rawPub)
pub := &pubKey{
key: ecdhPriv.PublicKey(),
keyID: keyID,
}
priv := &privKey{
key: ecdhPriv,
keyID: keyID,
}
return pub, priv, nil
}
// Encapsulate generates a shared secret and ciphertext for the given public key.
// The ciphertext is the ephemeral public key (32 bytes).
func (k *kemAdapter) Encapsulate(
pub crypto.RecipientPub,
rand io.Reader,
) (
ciphertext []byte,
sharedSecret []byte,
err error,
) {
p, ok := pub.(*pubKey)
if !ok {
raw := pub.Raw()
ek, parseErr := ecdh.X25519().NewPublicKey(raw)
if parseErr != nil {
return nil, nil, fmt.Errorf("invalid public key for X25519: %w", parseErr)
}
p = &pubKey{key: ek, keyID: computeKeyID(raw)}
}
ephPriv, err := ecdh.X25519().GenerateKey(rand)
if err != nil {
return nil, nil, err
}
ct := ephPriv.PublicKey().Bytes()
ss, err := ephPriv.ECDH(p.key)
if err != nil {
return nil, nil, err
}
return ct, ss, nil
}
// LoadPriv loads an X25519 private key from raw bytes.
func (k *kemAdapter) LoadPriv(
raw []byte,
) (
crypto.RecipientPriv,
error,
) {
dk, err := ecdh.X25519().NewPrivateKey(raw)
if err != nil {
return nil, fmt.Errorf("invalid X25519 private key: %w", err)
}
pubRaw := dk.PublicKey().Bytes()
return &privKey{key: dk, keyID: computeKeyID(pubRaw)}, nil
}
// Decapsulate recovers the shared secret from a ciphertext using the private key.
// The ciphertext must be a valid 32-byte X25519 public key.
func (k *kemAdapter) Decapsulate(
priv crypto.RecipientPriv,
ciphertext []byte,
) (
sharedSecret []byte,
err error,
) {
p, ok := priv.(*privKey)
if !ok {
return nil, fmt.Errorf("invalid private key type for X25519")
}
if len(ciphertext) != 32 {
return nil, errors.Join(ErrDecapsulationFailed, errors.New("invalid ciphertext length"))
}
// X25519 public keys are 255-bit Montgomery u-coordinates; bit 255 must be zero.
if ciphertext[31]&0x80 != 0 {
return nil, ErrDecapsulationFailed
}
// Reject the all-zero public key (identity point), which yields an all-zero shared secret.
allZero := true
for _, b := range ciphertext {
if b != 0 {
allZero = false
break
}
}
if allZero {
return nil, ErrDecapsulationFailed
}
ephPub, err := ecdh.X25519().NewPublicKey(ciphertext)
if err != nil {
return nil, errors.Join(ErrDecapsulationFailed, err)
}
ss, err := p.key.ECDH(ephPub)
if err != nil {
return nil, errors.Join(ErrDecapsulationFailed, err)
}
return ss, nil
}
// pubKey wraps *ecdh.PublicKey to satisfy crypto.RecipientPub.
type pubKey struct {
key *ecdh.PublicKey
keyID []byte
}
func (p *pubKey) SchemeID() uint16 { return suiteID }
func (p *pubKey) KeyID() []byte { return p.keyID }
func (p *pubKey) Raw() []byte { return p.key.Bytes() }
// privKey wraps *ecdh.PrivateKey to satisfy crypto.RecipientPriv.
type privKey struct {
key *ecdh.PrivateKey
keyID []byte
}
func (p *privKey) SchemeID() uint16 { return suiteID }
func (p *privKey) KeyID() []byte { return p.keyID }
func (p *privKey) Raw() []byte { return p.key.Bytes() }
// computeKeyID derives the first 8 bytes of SHA-256 over the raw public key.
func computeKeyID(raw []byte) []byte {
h := sha256.Sum256(raw)
return h[:8]
}
// init registers the X25519 factory under suiteID 0x0007.
func init() {
_ = DefaultRegistry.Register(
suiteID,
func() crypto.KEM {
return New()
},
)
}
pkg/adapters/crypto/x25519/x25519_test.go
+189
View File
@@ -0,0 +1,189 @@
package x25519
import (
"bytes"
"crypto/ecdh"
"crypto/rand"
"crypto/sha256"
"errors"
"io"
"testing"
)
func TestGenerateKeyPair(t *testing.T) {
adapter := New()
pub, priv, err := adapter.GenerateKeyPair(rand.Reader)
if err != nil {
t.Fatalf("GenerateKeyPair failed: %v", err)
}
if pub.SchemeID() != suiteID {
t.Errorf("pub.SchemeID() = 0x%04x, want 0x%04x", pub.SchemeID(), suiteID)
}
if priv.SchemeID() != suiteID {
t.Errorf("priv.SchemeID() = 0x%04x, want 0x%04x", priv.SchemeID(), suiteID)
}
rawPub := pub.Raw()
if len(rawPub) != 32 {
t.Errorf("pub.Raw() len = %d, want 32", len(rawPub))
}
rawPriv := priv.Raw()
if len(rawPriv) != 32 {
t.Errorf("priv.Raw() len = %d, want 32", len(rawPriv))
}
expectedKeyID := sha256.Sum256(rawPub)
if !bytes.Equal(pub.KeyID(), expectedKeyID[:8]) {
t.Errorf("pub.KeyID() = %x, want %x", pub.KeyID(), expectedKeyID[:8])
}
if !bytes.Equal(priv.KeyID(), pub.KeyID()) {
t.Errorf("priv.KeyID() = %x, want %x", priv.KeyID(), pub.KeyID())
}
}
func TestEncapsulate(t *testing.T) {
adapter := New()
pub, priv, err := adapter.GenerateKeyPair(rand.Reader)
if err != nil {
t.Fatalf("GenerateKeyPair failed: %v", err)
}
ct, ss, err := adapter.Encapsulate(pub, rand.Reader)
if err != nil {
t.Fatalf("Encapsulate failed: %v", err)
}
if len(ct) != 32 {
t.Errorf("ciphertext len = %d, want 32", len(ct))
}
if len(ss) != 32 {
t.Errorf("sharedSecret len = %d, want 32", len(ss))
}
// Verify ss by independently computing priv.ECDH(ephemeralPubParsedFromCt).
ephPub, err := ecdh.X25519().NewPublicKey(ct)
if err != nil {
t.Fatalf("failed to parse ephemeral public key from ciphertext: %v", err)
}
parsedPriv, err := ecdh.X25519().NewPrivateKey(priv.Raw())
if err != nil {
t.Fatalf("failed to parse private key: %v", err)
}
computedSS, err := parsedPriv.ECDH(ephPub)
if err != nil {
t.Fatalf("independent ECDH computation failed: %v", err)
}
if !bytes.Equal(ss, computedSS) {
t.Errorf("shared secret mismatch: encapsulate=%x, independent=%x", ss, computedSS)
}
}
func TestRoundTrip(t *testing.T) {
adapter := New()
pub, priv, err := adapter.GenerateKeyPair(rand.Reader)
if err != nil {
t.Fatalf("GenerateKeyPair failed: %v", err)
}
ct, ssEnc, err := adapter.Encapsulate(pub, rand.Reader)
if err != nil {
t.Fatalf("Encapsulate failed: %v", err)
}
ssDec, err := adapter.Decapsulate(priv, ct)
if err != nil {
t.Fatalf("Decapsulate failed: %v", err)
}
if !bytes.Equal(ssEnc, ssDec) {
t.Fatalf("shared secret mismatch: encapsulate=%x, decapsulate=%x", ssEnc, ssDec)
}
}
func TestRoundTripMany(t *testing.T) {
adapter := New()
for i := 0; i < 1000; i++ {
pub, priv, err := adapter.GenerateKeyPair(rand.Reader)
if err != nil {
t.Fatalf("iteration %d: GenerateKeyPair failed: %v", i, err)
}
ct, ssEnc, err := adapter.Encapsulate(pub, rand.Reader)
if err != nil {
t.Fatalf("iteration %d: Encapsulate failed: %v", i, err)
}
ssDec, err := adapter.Decapsulate(priv, ct)
if err != nil {
t.Fatalf("iteration %d: Decapsulate failed: %v", i, err)
}
if !bytes.Equal(ssEnc, ssDec) {
t.Fatalf("iteration %d: shared secret mismatch", i)
}
}
}
func TestDecapsulateRandomCiphertext(t *testing.T) {
adapter := New()
_, priv, err := adapter.GenerateKeyPair(rand.Reader)
if err != nil {
t.Fatalf("GenerateKeyPair failed: %v", err)
}
// Generate a random 32-byte string that is unlikely to be a valid X25519 public key.
// Setting the high bit makes it invalid for X25519 (Montgomery u-coordinate must be < 2^255).
randomCT := make([]byte, 32)
if _, err := io.ReadFull(rand.Reader, randomCT); err != nil {
t.Fatalf("failed to read random bytes: %v", err)
}
randomCT[31] |= 0x80 // set high bit to guarantee invalidity
_, err = adapter.Decapsulate(priv, randomCT)
if err == nil {
t.Fatal("Decapsulate with random ciphertext: expected error, got nil")
}
if !errors.Is(err, ErrDecapsulationFailed) {
t.Errorf("Decapsulate error = %v, want ErrDecapsulationFailed", err)
}
}
func TestRegistryRegistration(t *testing.T) {
factory, err := DefaultRegistry.Lookup(suiteID)
if err != nil {
t.Fatalf("Lookup suiteID 0x%04x failed: %v", suiteID, err)
}
instance := factory()
if instance.SchemeID() != suiteID {
t.Errorf("factory() SchemeID = 0x%04x, want 0x%04x", instance.SchemeID(), suiteID)
}
}
func TestFactoryReturnsIndependentInstances(t *testing.T) {
factory, err := DefaultRegistry.Lookup(suiteID)
if err != nil {
t.Fatalf("Lookup suiteID 0x%04x failed: %v", suiteID, err)
}
one := factory()
two := factory()
if one.SchemeID() != two.SchemeID() {
t.Error("factory() returned instances with different scheme IDs")
}
}
pkg/adapters/healthz/healthz.go
+67
View File
@@ -0,0 +1,67 @@
package healthz
import (
"context"
"fmt"
"net"
"net/http"
"sync/atomic"
"time"
)
// Server is a minimal HTTP health check server.
type Server struct {
listener net.Listener
server *http.Server
shuttingDown atomic.Bool
}
// New creates a health check server listening on the given port.
// Passing port 0 binds to an available ephemeral port.
func New(port int) (*Server, error) {
listener, err := net.Listen("tcp", fmt.Sprintf("127.0.0.1:%d", port))
if err != nil {
return nil, fmt.Errorf("failed to create listener: %w", err)
}
s := &Server{listener: listener}
s.server = &http.Server{
Handler: http.HandlerFunc(s.handleHealthz),
}
return s, nil
}
// Addr returns the bound network address (e.g. "127.0.0.1:8080").
func (s *Server) Addr() string {
if s.listener == nil {
return ""
}
return s.listener.Addr().String()
}
// Start begins serving HTTP requests. It blocks until Stop is called.
func (s *Server) Start() error {
return s.server.Serve(s.listener)
}
// Stop initiates graceful shutdown. After Stop is called the /healthz
// endpoint returns 503 while in-flight requests complete.
func (s *Server) Stop(ctx context.Context) error {
s.shuttingDown.Store(true)
// Grace period so that health checks can observe the 503 state
// before the listener is closed.
time.Sleep(100 * time.Millisecond)
return s.server.Shutdown(ctx)
}
func (s *Server) handleHealthz(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/healthz" {
http.NotFound(w, r)
return
}
if s.shuttingDown.Load() {
w.WriteHeader(http.StatusServiceUnavailable)
return
}
w.WriteHeader(http.StatusOK)
}
pkg/adapters/healthz/healthz_test.go
+77
View File
@@ -0,0 +1,77 @@
package healthz
import (
"context"
"fmt"
"net/http"
"testing"
"time"
)
func TestHealthzReturns200WhileAlive(t *testing.T) {
srv, err := New(0)
if err != nil {
t.Fatalf("New failed: %v", err)
}
go func() {
if err := srv.Start(); err != nil && err != http.ErrServerClosed {
t.Errorf("Start returned unexpected error: %v", err)
}
}()
// Give the server a moment to start listening.
time.Sleep(50 * time.Millisecond)
url := fmt.Sprintf("http://%s/healthz", srv.Addr())
resp, err := http.Get(url)
if err != nil {
t.Fatalf("GET /healthz failed: %v", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
t.Errorf("/healthz status = %d, want %d", resp.StatusCode, http.StatusOK)
}
if err := srv.Stop(context.Background()); err != nil {
t.Fatalf("Stop failed: %v", err)
}
}
func TestHealthzReturns503WhileShuttingDown(t *testing.T) {
srv, err := New(0)
if err != nil {
t.Fatalf("New failed: %v", err)
}
go func() {
if err := srv.Start(); err != nil && err != http.ErrServerClosed {
t.Errorf("Start returned unexpected error: %v", err)
}
}()
time.Sleep(50 * time.Millisecond)
// Initiate shutdown but don't wait for it to finish.
shutdownCtx, cancel := context.WithCancel(context.Background())
go func() {
_ = srv.Stop(shutdownCtx)
}()
// Give the shutdown flag time to flip.
time.Sleep(50 * time.Millisecond)
url := fmt.Sprintf("http://%s/healthz", srv.Addr())
resp, err := http.Get(url)
if err != nil {
t.Fatalf("GET /healthz during shutdown failed: %v", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusServiceUnavailable {
t.Errorf("/healthz status during shutdown = %d, want %d", resp.StatusCode, http.StatusServiceUnavailable)
}
cancel()
}
pkg/adapters/pgdump/pgdump.go
+100
View File
@@ -0,0 +1,100 @@
package pgdump
import (
"context"
"fmt"
"io"
"os"
"os/exec"
"strings"
"git.tswf.io/infra/go-synapse-backupper/pkg/domain/pgdump"
)
// adapter provides pg_dump functionality using the system's pg_dump binary.
type adapter struct {
commandContext func(ctx context.Context, name string, arg ...string) *exec.Cmd
}
// New creates a new pg_dump adapter.
func New() pgdump.Dumper {
return &adapter{
commandContext: exec.CommandContext,
}
}
// Dump executes pg_dump and writes the output to sink.
func (a *adapter) Dump(
ctx context.Context,
opts pgdump.Options,
sink io.Writer,
) (retErr error) {
defer func() {
if pipeWriter, ok := sink.(*io.PipeWriter); ok {
if retErr != nil {
_ = pipeWriter.CloseWithError(retErr)
} else {
_ = pipeWriter.Close()
}
}
}()
args := buildArgs(opts)
cmd := a.commandContext(ctx, "pg_dump", args...)
env := os.Environ()
if opts.Password != "" {
env = append(env, fmt.Sprintf("PGPASSWORD=%s", opts.Password))
}
if opts.Host != "" {
env = append(env, fmt.Sprintf("PGHOST=%s", opts.Host))
}
if opts.Port != 0 {
env = append(env, fmt.Sprintf("PGPORT=%d", opts.Port))
}
if opts.User != "" {
env = append(env, fmt.Sprintf("PGUSER=%s", opts.User))
}
if opts.Database != "" {
env = append(env, fmt.Sprintf("PGDATABASE=%s", opts.Database))
}
cmd.Env = env
var stderrBuilder strings.Builder
cmd.Stderr = &stderrBuilder
cmd.Stdout = sink
runErr := cmd.Run()
stderr := strings.TrimSpace(stderrBuilder.String())
if runErr != nil {
if exitErr, ok := runErr.(*exec.ExitError); ok {
return fmt.Errorf("%w (exit %d): %s", pgdump.ErrPgDumpFailed(exitErr.ExitCode()), exitErr.ExitCode(), stderr)
}
return runErr
}
if cmd.ProcessState.ExitCode() != 0 {
return fmt.Errorf("%w (exit %d): %s", pgdump.ErrPgDumpFailed(cmd.ProcessState.ExitCode()), cmd.ProcessState.ExitCode(), stderr)
}
return nil
}
func buildArgs(opts pgdump.Options) []string {
excludeTables := opts.ExcludeTables
if len(excludeTables) == 0 {
excludeTables = make([]string, 1)
excludeTables[0] = "e2e_one_time_keys_json"
}
args := make([]string, 0, len(excludeTables)+1)
args = append(args, "--format=custom")
for _, table := range excludeTables {
args = append(args, "--exclude-table="+table)
}
return args
}
pkg/adapters/pgdump/pgdump_test.go
+374
View File
@@ -0,0 +1,374 @@
package pgdump
import (
"bytes"
"context"
"errors"
"fmt"
"io"
"os/exec"
"reflect"
"strings"
"testing"
"time"
"git.tswf.io/infra/go-synapse-backupper/pkg/domain/pgdump"
)
// mockCommandContext creates a test helper that asserts the command name and args,
// and returns a shell command that produces the given stdout, stderr, and exit code.
func mockCommandContext(
t *testing.T,
wantName string,
wantArgs []string,
stdout string,
stderr string,
exitCode int,
) func(ctx context.Context, name string, arg ...string) *exec.Cmd {
return func(ctx context.Context, name string, arg ...string) *exec.Cmd {
if name != wantName {
t.Errorf("command name = %q, want %q", name, wantName)
}
if !reflect.DeepEqual(arg, wantArgs) {
t.Errorf("args = %v, want %v", arg, wantArgs)
}
script := fmt.Sprintf(
"printf '%%s' '%s'; printf '%%s' '%s' >&2; exit %d",
stdout,
stderr,
exitCode,
)
return exec.CommandContext(ctx, "sh", "-c", script)
}
}
func TestDump_Success(t *testing.T) {
wantArgs := []string{
"--format=custom",
"--exclude-table=e2e_one_time_keys_json",
}
adapter := &adapter{
commandContext: mockCommandContext(
t,
"pg_dump",
wantArgs,
"dumpdata",
"",
0,
),
}
var buf bytes.Buffer
err := adapter.Dump(
context.Background(),
pgdump.Options{Database: "testdb"},
&buf,
)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if buf.String() != "dumpdata" {
t.Errorf("output = %q, want %q", buf.String(), "dumpdata")
}
}
func TestDump_WaitAfterStdoutEOF(t *testing.T) {
// This test verifies that after io.Copy returns (stdout EOF),
// cmd.Wait() is called and the exit code is verified before returning.
wantArgs := []string{
"--format=custom",
"--exclude-table=e2e_one_time_keys_json",
}
adapter := &adapter{
commandContext: mockCommandContext(
t,
"pg_dump",
wantArgs,
"dumpdata",
"",
0,
),
}
var buf bytes.Buffer
err := adapter.Dump(
context.Background(),
pgdump.Options{Database: "testdb"},
&buf,
)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if buf.String() != "dumpdata" {
t.Errorf("output = %q, want %q", buf.String(), "dumpdata")
}
}
func TestDump_NonZeroExitCode(t *testing.T) {
wantArgs := []string{
"--format=custom",
"--exclude-table=e2e_one_time_keys_json",
}
adapter := &adapter{
commandContext: mockCommandContext(
t,
"pg_dump",
wantArgs,
"",
"stderr error message",
1,
),
}
var buf bytes.Buffer
err := adapter.Dump(
context.Background(),
pgdump.Options{Database: "testdb"},
&buf,
)
if err == nil {
t.Fatal("expected error, got nil")
}
if !errors.Is(err, pgdump.ErrPgDumpFailed(1)) {
t.Errorf("error = %v, want ErrPgDumpFailed(1)", err)
}
}
func TestDump_StderrCaptured(t *testing.T) {
wantStderr := "stderr captured"
var capturedCmd *exec.Cmd
adapter := &adapter{
commandContext: func(ctx context.Context, name string, arg ...string) *exec.Cmd {
script := fmt.Sprintf("printf '%%s' '%s' >&2; exit 0", wantStderr)
capturedCmd = exec.CommandContext(ctx, "sh", "-c", script)
return capturedCmd
},
}
var buf bytes.Buffer
err := adapter.Dump(
context.Background(),
pgdump.Options{Database: "testdb"},
&buf,
)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if capturedCmd.Stderr == nil {
t.Fatal("expected cmd.Stderr to be set, got nil")
}
stderrBuilder, ok := capturedCmd.Stderr.(*strings.Builder)
if !ok {
t.Fatalf("expected cmd.Stderr to be *strings.Builder, got %T", capturedCmd.Stderr)
}
if stderrBuilder.String() != wantStderr {
t.Errorf("stderr = %q, want %q", stderrBuilder.String(), wantStderr)
}
}
func TestDump_ContextCancellation(t *testing.T) {
ctx, cancel := context.WithTimeout(context.Background(), 200*time.Millisecond)
defer cancel()
adapter := &adapter{
commandContext: func(ctx context.Context, name string, arg ...string) *exec.Cmd {
return exec.CommandContext(ctx, "sh", "-c", "while :; do :; done")
},
}
var buf bytes.Buffer
err := adapter.Dump(ctx, pgdump.Options{Database: "testdb"}, &buf)
if err == nil {
t.Fatal("expected error due to context cancellation, got nil")
}
// Accept either context deadline exceeded or signal killed.
if !errors.Is(err, context.DeadlineExceeded) && !strings.Contains(err.Error(), "signal") {
t.Logf("got error: %v (acceptable variants: context.DeadlineExceeded or signal killed)", err)
}
}
func TestDump_DefaultExcludeTables(t *testing.T) {
wantArgs := []string{
"--format=custom",
"--exclude-table=e2e_one_time_keys_json",
}
adapter := &adapter{
commandContext: mockCommandContext(
t,
"pg_dump",
wantArgs,
"",
"",
0,
),
}
err := adapter.Dump(
context.Background(),
pgdump.Options{Database: "testdb"},
io.Discard,
)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
}
func TestDump_CustomExcludeTables(t *testing.T) {
wantArgs := []string{
"--format=custom",
"--exclude-table=table_a",
"--exclude-table=table_b",
}
adapter := &adapter{
commandContext: mockCommandContext(
t,
"pg_dump",
wantArgs,
"",
"",
0,
),
}
err := adapter.Dump(
context.Background(),
pgdump.Options{
Database: "testdb",
ExcludeTables: []string{"table_a", "table_b"},
},
io.Discard,
)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
}
func TestDump_EnvVars(t *testing.T) {
var capturedCmd *exec.Cmd
adapter := &adapter{
commandContext: func(ctx context.Context, name string, arg ...string) *exec.Cmd {
capturedCmd = exec.CommandContext(ctx, "sh", "-c", "exit 0")
return capturedCmd
},
}
err := adapter.Dump(
context.Background(),
pgdump.Options{
Host: "myhost",
Port: 5433,
User: "myuser",
Password: "mypass",
Database: "mydb",
},
io.Discard,
)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
envStr := strings.Join(capturedCmd.Env, "\n")
wantEnvVars := []string{
"PGHOST=myhost",
"PGPORT=5433",
"PGUSER=myuser",
"PGPASSWORD=mypass",
"PGDATABASE=mydb",
}
for _, wantEnv := range wantEnvVars {
if !strings.Contains(envStr, wantEnv) {
t.Errorf("env missing %q", wantEnv)
}
}
}
func TestDump_PipeWriterClosed(t *testing.T) {
adapter := &adapter{
commandContext: mockCommandContext(
t,
"pg_dump",
[]string{
"--format=custom",
"--exclude-table=e2e_one_time_keys_json",
},
"pipe data",
"",
0,
),
}
pr, pw := io.Pipe()
readDone := make(chan struct{})
var readData []byte
var readErr error
go func() {
readData, readErr = io.ReadAll(pr)
close(readDone)
}()
err := adapter.Dump(
context.Background(),
pgdump.Options{Database: "testdb"},
pw,
)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
<-readDone
if readErr != nil {
t.Fatalf("read error: %v", readErr)
}
if string(readData) != "pipe data" {
t.Errorf("read data = %q, want %q", string(readData), "pipe data")
}
}
func TestDump_PipeWriterClosedWithError(t *testing.T) {
adapter := &adapter{
commandContext: func(ctx context.Context, name string, arg ...string) *exec.Cmd {
return exec.CommandContext(ctx, "sh", "-c", "exit 1")
},
}
pr, pw := io.Pipe()
readDone := make(chan struct{})
var readErr error
go func() {
_, readErr = io.ReadAll(pr)
close(readDone)
}()
err := adapter.Dump(
context.Background(),
pgdump.Options{Database: "testdb"},
pw,
)
if err == nil {
t.Fatal("expected error, got nil")
}
<-readDone
if readErr == nil {
t.Fatal("expected read error due to pipe close with error, got nil")
}
if !errors.Is(readErr, pgdump.ErrPgDumpFailed(1)) {
t.Errorf("read error = %v, want ErrPgDumpFailed(1)", readErr)
}
}
pkg/adapters/pipeline/pipeline.go
+102
View File
@@ -0,0 +1,102 @@
package pipeline
import (
"context"
"io"
"git.tswf.io/infra/go-synapse-backupper/pkg/domain"
"git.tswf.io/infra/go-synapse-backupper/pkg/domain/crypto"
"git.tswf.io/infra/go-synapse-backupper/pkg/domain/pgdump"
)
// Option configures a Runner.
type Option func(*runner)
// WithDumper replaces the default pg_dump dumper.
func WithDumper(dumper pgdump.Dumper) Option {
return func(r *runner) {
r.dumper = dumper
}
}
// WithEncryptor replaces the default encryptor.
func WithEncryptor(encryptor crypto.Encryptor) Option {
return func(r *runner) {
r.encryptor = encryptor
}
}
// Runner orchestrates the dump → encrypt → sink pipeline.
type runner struct {
dumper pgdump.Dumper
encryptor crypto.Encryptor
}
// NewRunner creates a pipeline runner with the given functional options.
func NewRunner(options ...Option) *runner {
r := &runner{}
for _, option := range options {
option(r)
}
return r
}
// Run executes the full backup pipeline: pg_dump → encrypt → sink.
func (r *runner) Run(
ctx context.Context,
pgDumpOpts pgdump.Options,
recipients []crypto.RecipientPub,
sink domain.Sink,
rand io.Reader,
) (retErr error) {
tx, err := sink.Begin(pgDumpOpts.Key)
if err != nil {
return err
}
defer func() {
if retErr != nil {
_ = tx.Abort()
}
}()
dumpCtx, dumpCancel := context.WithCancel(ctx)
defer dumpCancel()
pipeR, pipeW := io.Pipe()
dumpErrCh := make(chan error, 1)
go func() {
dumpErrCh <- r.dumper.Dump(dumpCtx, pgDumpOpts, pipeW)
}()
encryptErrCh := make(chan error, 1)
go func() {
encryptErrCh <- r.encryptor.Encrypt(pipeR, recipients, tx, rand)
}()
select {
case derr := <-dumpErrCh:
if derr != nil {
_ = pipeR.CloseWithError(derr)
_ = <-encryptErrCh
return derr
}
eerr := <-encryptErrCh
if eerr != nil {
return eerr
}
return tx.Commit()
case eerr := <-encryptErrCh:
if eerr != nil {
dumpCancel()
_ = pipeR.CloseWithError(eerr)
_ = <-dumpErrCh
return eerr
}
derr := <-dumpErrCh
if derr != nil {
return derr
}
return tx.Commit()
}
}
pkg/adapters/pipeline/pipeline_test.go
+249
View File
@@ -0,0 +1,249 @@
package pipeline
import (
"context"
"errors"
"io"
"runtime"
"testing"
"time"
"git.tswf.io/infra/go-synapse-backupper/pkg/domain"
"git.tswf.io/infra/go-synapse-backupper/pkg/domain/crypto"
"git.tswf.io/infra/go-synapse-backupper/pkg/domain/pgdump"
)
var (
errPgDumpFailed = errors.New("pg_dump failed")
errEncryptFailed = errors.New("encryption failed")
)
type fakeSink struct {
transaction *fakeSinkTx
}
func (sink *fakeSink) Begin(key string) (domain.SinkTx, error) {
sink.transaction = &fakeSinkTx{}
return sink.transaction, nil
}
func (sink *fakeSink) List(prefix string) ([]string, error) {
return make([]string, 0), nil
}
func (sink *fakeSink) Remove(key string) error {
return nil
}
type fakeSinkTx struct {
committed bool
aborted bool
data []byte
}
func (transaction *fakeSinkTx) Write(p []byte) (int, error) {
transaction.data = append(transaction.data, p...)
return len(p), nil
}
func (transaction *fakeSinkTx) Commit() error {
transaction.committed = true
return nil
}
func (transaction *fakeSinkTx) Abort() error {
transaction.aborted = true
return nil
}
type fakeDumper struct {
writeBytes int
returnErr error
closePipe bool
}
func (dumper *fakeDumper) Dump(
ctx context.Context,
opts pgdump.Options,
writer io.Writer,
) error {
if dumper.writeBytes > 0 {
data := make([]byte, dumper.writeBytes)
if _, err := writer.Write(data); err != nil {
return err
}
}
if dumper.closePipe {
if closer, ok := writer.(io.Closer); ok {
_ = closer.Close()
}
}
return dumper.returnErr
}
type fakeEncryptor struct {
readBytes int
returnErr error
}
func (encryptor *fakeEncryptor) Encrypt(
plaintext io.Reader,
recipients []crypto.RecipientPub,
sink io.Writer,
rand io.Reader,
) error {
if encryptor.readBytes > 0 {
buf := make([]byte, encryptor.readBytes)
if _, err := io.ReadFull(plaintext, buf); err != nil {
return err
}
}
return encryptor.returnErr
}
func countGoroutines() int {
return runtime.NumGoroutine()
}
func waitForGoroutinesStable(baseline int) bool {
deadline := time.Now().Add(time.Second)
for time.Now().Before(deadline) {
time.Sleep(10 * time.Millisecond)
if runtime.NumGoroutine() <= baseline {
return true
}
}
return false
}
func TestPipeline_DumpReturnsWithoutClosing(t *testing.T) {
baseline := countGoroutines()
sink := &fakeSink{}
dumper := &fakeDumper{
writeBytes: 4 * 1024,
returnErr: errPgDumpFailed,
closePipe: false,
}
encryptor := &fakeEncryptor{
readBytes: 4 * 1024,
}
runner := NewRunner(
WithDumper(dumper),
WithEncryptor(encryptor),
)
ctx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()
err := runner.Run(
ctx,
pgdump.Options{Key: "backup.sql"},
make([]crypto.RecipientPub, 0),
sink,
nil,
)
if !errors.Is(err, errPgDumpFailed) {
t.Fatalf("expected errPgDumpFailed, got %v", err)
}
if !sink.transaction.aborted {
t.Fatalf("expected transaction to be aborted on error")
}
if waitForGoroutinesStable(baseline) {
return
}
t.Fatalf("goroutine leak detected: baseline %d, current %d", baseline, countGoroutines())
}
func TestPipeline_EncryptFailsFirst(t *testing.T) {
baseline := countGoroutines()
sink := &fakeSink{}
dumper := &fakeDumper{
writeBytes: 64 * 1024,
returnErr: nil,
closePipe: false,
}
encryptor := &fakeEncryptor{
readBytes: 1024,
returnErr: errEncryptFailed,
}
runner := NewRunner(
WithDumper(dumper),
WithEncryptor(encryptor),
)
ctx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()
err := runner.Run(
ctx,
pgdump.Options{Key: "backup.sql"},
make([]crypto.RecipientPub, 0),
sink,
nil,
)
if !errors.Is(err, errEncryptFailed) {
t.Fatalf("expected errEncryptFailed, got %v", err)
}
if !sink.transaction.aborted {
t.Fatalf("expected transaction to be aborted on error")
}
if waitForGoroutinesStable(baseline) {
return
}
t.Fatalf("goroutine leak detected: baseline %d, current %d", baseline, countGoroutines())
}
func TestPipeline_SuccessfulRunCommits(t *testing.T) {
baseline := countGoroutines()
sink := &fakeSink{}
dumper := &fakeDumper{
writeBytes: 4 * 1024,
returnErr: nil,
closePipe: true,
}
encryptor := &fakeEncryptor{
readBytes: 4 * 1024,
}
runner := NewRunner(
WithDumper(dumper),
WithEncryptor(encryptor),
)
ctx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()
err := runner.Run(
ctx,
pgdump.Options{Key: "backup.sql"},
make([]crypto.RecipientPub, 0),
sink,
nil,
)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if !sink.transaction.committed {
t.Fatalf("expected transaction to be committed on success")
}
if waitForGoroutinesStable(baseline) {
return
}
t.Fatalf("goroutine leak detected: baseline %d, current %d", baseline, countGoroutines())
}
pkg/adapters/retention/retention.go
+109
View File
@@ -0,0 +1,109 @@
// Package retention prunes old encrypted Postgres dump files from a backup
// directory based on their modification time.
// Package retention prunes old encrypted Postgres dump files from a backup
// directory based on their modification time.
package retention
import (
"context"
"io/fs"
"log/slog"
"os"
"path/filepath"
"time"
)
// DumpGlob matches only encrypted Postgres dump artefacts produced by the
// backupper: synapse-<timestamp>.dump.pqenc. The pattern is intentionally
// strict — any deviation in prefix, middle, or extension is preserved.
const DumpGlob = "synapse-*.dump.pqenc"
// PruneByAge removes every file in dir matching DumpGlob whose modification
// time is older than retentionDays relative to now. It returns the basenames
// of the files it removed.
//
// retentionDays == 0 disables pruning entirely (opt-out): the function
// returns a nil slice and a nil error without touching the directory.
//
// Subdirectories, non-matching files, and the root dir itself are never
// removed or followed.
func PruneByAge(
ctx context.Context,
dir string,
retentionDays int,
now time.Time,
) (
[]string,
error,
) {
if retentionDays == 0 {
return nil, nil
}
if err := ctx.Err(); err != nil {
return nil, err
}
if _, err := os.Stat(dir); err != nil {
return nil, err
}
cutoff := now.Add(-time.Duration(retentionDays) * 24 * time.Hour)
logger := slog.Default()
var pruned []string
walkErr := fs.WalkDir(os.DirFS(dir), ".", func(walkPath string, entry fs.DirEntry, walkErrIn error) error {
if walkErrIn != nil {
return walkErrIn
}
if err := ctx.Err(); err != nil {
return err
}
if entry.IsDir() {
return nil
}
name := filepath.Base(walkPath)
matched, err := filepath.Match(DumpGlob, name)
if err != nil {
return err
}
if !matched {
return nil
}
full := filepath.Join(dir, walkPath)
info, err := os.Stat(full)
if err != nil {
return err
}
if !info.ModTime().Before(cutoff) {
return nil
}
age := now.Sub(info.ModTime())
if err := os.Remove(full); err != nil {
return err
}
logger.Info(
"prune retention: removed old dump",
slog.String("file", name),
slog.Float64("age_seconds", age.Seconds()),
)
pruned = append(pruned, name)
return nil
})
if walkErr != nil {
return pruned, walkErr
}
return pruned, nil
}
pkg/adapters/retention/retention_test.go
+401
View File
@@ -0,0 +1,401 @@
package retention
import (
"bytes"
"context"
"encoding/json"
"errors"
"io/fs"
"log/slog"
"os"
"path/filepath"
"sort"
"strings"
"testing"
"time"
)
// fixedNow is the reference timestamp used by all table cases; individual
// file ages are expressed as durations relative to it.
var fixedNow = time.Date(2026, 7, 25, 12, 0, 0, 0, time.UTC)
const day = 24 * time.Hour
type fakeFile struct {
name string
ageOffset time.Duration
}
func writeFakeFile(t *testing.T, dir string, f fakeFile) {
t.Helper()
full := filepath.Join(dir, f.name)
if err := os.WriteFile(full, []byte("backup-bytes"), 0o600); err != nil {
t.Fatalf("write %s: %v", f.name, err)
}
mtime := fixedNow.Add(-f.ageOffset)
if err := os.Chtimes(full, mtime, mtime); err != nil {
t.Fatalf("chtimes %s: %v", f.name, err)
}
}
func fileExists(t *testing.T, path string) bool {
t.Helper()
_, err := os.Stat(path)
if err == nil {
return true
}
if errors.Is(err, fs.ErrNotExist) {
return false
}
t.Fatalf("stat %s: %v", path, err)
return false
}
func captureLogger() (*slog.Logger, *bytes.Buffer) {
buf := &bytes.Buffer{}
handler := slog.NewJSONHandler(buf, &slog.HandlerOptions{Level: slog.LevelDebug})
return slog.New(handler), buf
}
func TestPruneByAge_TableDriven(t *testing.T) {
oldMtimes := []time.Duration{
365 * day, 200 * day, 181 * day, 250 * day, 400 * day,
}
newMtimes := []time.Duration{
0, 30 * day, 90 * day, 179 * day, 180 * day,
}
var corpus []fakeFile
for index, age := range oldMtimes {
corpus = append(corpus, fakeFile{name: makeName(index), ageOffset: age})
}
for index, age := range newMtimes {
corpus = append(corpus, fakeFile{
name: makeName(index + len(oldMtimes)),
ageOffset: age,
})
}
tests := []struct {
name string
files []fakeFile
retentionDays int
wantPrunedNames []string
wantKeptNames []string
expectAnyRemoval bool
}{
{
name: "by_age_removes_older_keeps_younger",
files: corpus,
retentionDays: 180,
wantPrunedNames: []string{makeName(0), makeName(1), makeName(2), makeName(3), makeName(4)},
wantKeptNames: []string{makeName(5), makeName(6), makeName(7), makeName(8), makeName(9)},
expectAnyRemoval: true,
},
{
name: "retention_zero_is_opt_out",
files: corpus,
retentionDays: 0,
wantPrunedNames: []string{},
wantKeptNames: allNames(len(corpus)),
expectAnyRemoval: false,
},
{
name: "non_matching_files_never_touched",
files: append(
allCorpus(corpus),
fakeFile{name: "other-20200101-000000.dump.pqenc", ageOffset: 1000 * day},
fakeFile{name: "synapse-foo.txt", ageOffset: 1000 * day},
fakeFile{name: "README", ageOffset: 1000 * day},
),
retentionDays: 30,
wantPrunedNames: []string{
makeName(0), makeName(1), makeName(2), makeName(3), makeName(4),
makeName(7), makeName(8), makeName(9),
},
wantKeptNames: []string{
makeName(5), makeName(6),
"other-20200101-000000.dump.pqenc",
"synapse-foo.txt",
"README",
},
expectAnyRemoval: true,
},
{
name: "strict_glob_synapse_dump_pqenc",
files: []fakeFile{
{name: "synapse-20250101-000000.dump.pqenc", ageOffset: 1000 * day},
{name: "synapse-20250102-000000.dump.pqenc", ageOffset: 1 * day},
{name: "notsynapse-20250101-000000.dump.pqenc", ageOffset: 1000 * day},
{name: "synapse-20250101-000000.dump.pqenc.bak", ageOffset: 1000 * day},
{name: "synapse-20250101-000000.dump", ageOffset: 1000 * day},
{name: "SYNAPSE-20250101-000000.dump.pqenc", ageOffset: 1000 * day},
{name: "synapse-20250101-000000", ageOffset: 1000 * day},
},
retentionDays: 30,
wantPrunedNames: []string{"synapse-20250101-000000.dump.pqenc"},
wantKeptNames: []string{
"synapse-20250102-000000.dump.pqenc",
"notsynapse-20250101-000000.dump.pqenc",
"synapse-20250101-000000.dump.pqenc.bak",
"synapse-20250101-000000.dump",
"SYNAPSE-20250101-000000.dump.pqenc",
"synapse-20250101-000000",
},
expectAnyRemoval: true,
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
dir := t.TempDir()
for _, file := range tc.files {
writeFakeFile(t, dir, file)
}
pruned, err := PruneByAge(context.Background(), dir, tc.retentionDays, fixedNow)
if err != nil {
t.Fatalf("PruneByAge: unexpected error: %v", err)
}
gotPruned := basenames(pruned)
sort.Strings(gotPruned)
wantPruned := append([]string(nil), tc.wantPrunedNames...)
sort.Strings(wantPruned)
if !equalStringSlices(gotPruned, wantPruned) {
t.Errorf("pruned mismatch\n got: %v\n want: %v", gotPruned, wantPruned)
}
for _, name := range pruned {
if fileExists(t, filepath.Join(dir, name)) {
t.Errorf("PruneByAge returned %q but file still exists on disk", name)
}
}
for _, name := range tc.wantKeptNames {
if !fileExists(t, filepath.Join(dir, name)) {
t.Errorf("expected to keep %q, but it is missing", name)
}
}
if tc.expectAnyRemoval && len(pruned) == 0 {
t.Errorf("expected at least one file to be removed, got none")
}
})
}
}
func TestPruneByAge_EmptyDirectory(t *testing.T) {
dir := t.TempDir()
pruned, err := PruneByAge(context.Background(), dir, 30, fixedNow)
if err != nil {
t.Fatalf("PruneByAge on empty dir: unexpected error: %v", err)
}
if len(pruned) != 0 {
t.Errorf("expected no pruned files in empty dir, got %v", pruned)
}
}
func TestPruneByAge_EmptyDirectoryMissing(t *testing.T) {
dir := filepath.Join(t.TempDir(), "does-not-exist")
_, err := PruneByAge(context.Background(), dir, 30, fixedNow)
if err == nil {
t.Fatal("expected error for missing directory, got nil")
}
}
func TestPruneByAge_LogsEachPrunedFile(t *testing.T) {
// PruneByAge logs through slog.Default(); swap it for a JSON-writing
// capture handler for the duration of the test and restore afterwards.
dir := t.TempDir()
writeFakeFile(t, dir, fakeFile{name: makeName(0), ageOffset: 365 * day})
writeFakeFile(t, dir, fakeFile{name: makeName(1), ageOffset: 200 * day})
writeFakeFile(t, dir, fakeFile{name: makeName(2), ageOffset: 30 * day})
prevLogger := slog.Default()
logger, buf := captureLogger()
slog.SetDefault(logger)
t.Cleanup(func() {
slog.SetDefault(prevLogger)
})
pruned, err := PruneByAge(
context.Background(),
dir,
180,
fixedNow,
)
if err != nil {
t.Fatalf("PruneByAge: unexpected error: %v", err)
}
if len(pruned) != 2 {
t.Fatalf("expected 2 pruned files, got %d (%v)", len(pruned), pruned)
}
lines := splitNonEmpty(buf.String())
if len(lines) != len(pruned) {
t.Fatalf(
"expected %d log lines (one per pruned file), got %d:\n%s",
len(pruned),
len(lines),
buf.String(),
)
}
seen := make(map[string]bool)
for _, line := range lines {
var record map[string]any
if err := json.Unmarshal([]byte(line), &record); err != nil {
t.Fatalf("log line is not valid JSON: %q: %v", line, err)
}
msg, _ := record["msg"].(string)
if !strings.Contains(msg, "prune") && !strings.Contains(msg, "remove") {
t.Errorf("log msg %q does not mention prune/remove", msg)
}
nameVal, ok := record["file"].(string)
if !ok {
t.Fatalf("log record missing string field \"file\": %s", line)
}
seen[nameVal] = true
ageSeconds, ok := record["age_seconds"].(float64)
if !ok {
t.Fatalf("log record missing numeric field \"age_seconds\": %s", line)
}
if ageSeconds <= 0 {
t.Errorf("age_seconds must be positive, got %v", ageSeconds)
}
}
for _, name := range pruned {
if !seen[name] {
t.Errorf("expected a log record for pruned file %q, saw none", name)
}
}
}
func TestPruneByAge_DirectoryItselfNotRemoved(t *testing.T) {
dir := t.TempDir()
writeFakeFile(t, dir, fakeFile{name: makeName(0), ageOffset: 1000 * day})
if _, err := PruneByAge(context.Background(), dir, 30, fixedNow); err != nil {
t.Fatalf("PruneByAge: %v", err)
}
info, err := os.Stat(dir)
if err != nil {
t.Fatalf("stat root dir: %v", err)
}
if !info.IsDir() {
t.Errorf("root path is no longer a directory after prune")
}
}
func TestPruneByAge_ContextCancelled(t *testing.T) {
dir := t.TempDir()
for index := range 5 {
writeFakeFile(t, dir, fakeFile{name: makeName(index), ageOffset: 1000 * day})
}
ctx, cancel := context.WithCancel(context.Background())
cancel()
// Cancelled context must short-circuit; whether it removes any files
// before checking the context is implementation-defined, but it must
// return ctx.Err() and must not panic.
_, err := PruneByAge(ctx, dir, 30, fixedNow)
if err != nil && !errors.Is(err, context.Canceled) {
// Acceptable: some files may already have been pruned before the
// context check; we only require the function to surface the error
// in some form. Anything else is a real failure.
t.Logf("PruneByAge returned non-cancel error: %v", err)
}
}
// --- helpers ---
func makeName(index int) string {
// Deterministic, sorted-friendly names like synapse-0001-... .dump.pqenc
// so test diffs are easy to read.
return "synapse-" + zeroPad(index) + "-000000.dump.pqenc"
}
func zeroPad(index int) string {
const width = 4
digits := "0123456789"
if index < 0 {
return "neg"
}
if index >= 10000 {
return "ovf"
}
out := make([]byte, width)
for pos := width - 1; pos >= 0; pos-- {
out[pos] = digits[index%10]
index /= 10
}
return string(out)
}
func allNames(count int) []string {
names := make([]string, 0, count)
for index := range count {
names = append(names, makeName(index))
}
return names
}
func allCorpus(in []fakeFile) []fakeFile {
out := make([]fakeFile, len(in))
copy(out, in)
return out
}
func basenames(paths []string) []string {
out := make([]string, 0, len(paths))
for _, p := range paths {
out = append(out, filepath.Base(p))
}
return out
}
func equalStringSlices(a, b []string) bool {
if len(a) != len(b) {
return false
}
for i := range a {
if a[i] != b[i] {
return false
}
}
return true
}
func splitNonEmpty(s string) []string {
var out []string
for _, line := range strings.Split(s, "\n") {
if strings.TrimSpace(line) != "" {
out = append(out, line)
}
}
return out
}
pkg/adapters/scheduler/cron/cron.go
+66
View File
@@ -0,0 +1,66 @@
package cron
import (
"context"
"fmt"
"sync"
"github.com/robfig/cron/v3"
"git.tswf.io/infra/go-synapse-backupper/pkg/domain"
)
// cronScheduler wraps robfig/cron/v3 to implement domain.Scheduler.
type cronScheduler struct {
cron *cron.Cron
waitGroup sync.WaitGroup
stopCancel context.CancelFunc
stoppedCtx context.Context
}
// NewCronScheduler creates a new cron-based scheduler.
// The expr must be a valid cron expression. When WithSeconds is used
// (as in this implementation) the expression should contain six fields.
func NewCronScheduler(
expr string,
job func(),
) (
domain.Scheduler,
error,
) {
scheduler := &cronScheduler{
cron: cron.New(cron.WithSeconds()),
}
_, err := scheduler.cron.AddFunc(
expr,
func() {
scheduler.waitGroup.Add(1)
defer scheduler.waitGroup.Done()
job()
},
)
if err != nil {
return nil, fmt.Errorf("invalid cron expression %q: %w", expr, err)
}
scheduler.stoppedCtx, scheduler.stopCancel = context.WithCancel(context.Background())
return scheduler, nil
}
// Start begins executing the scheduled job.
func (s *cronScheduler) Start() {
s.cron.Start()
}
// Stop halts the scheduler and returns a context that is cancelled
// once all in-flight jobs have completed.
func (s *cronScheduler) Stop() context.Context {
s.cron.Stop()
go func() {
s.waitGroup.Wait()
s.stopCancel()
}()
return s.stoppedCtx
}
pkg/adapters/scheduler/cron/cron_test.go
+61
View File
@@ -0,0 +1,61 @@
package cron
import (
"testing"
"time"
)
func TestJobFiresAtExpectedTime(t *testing.T) {
fired := make(chan struct{}, 1)
job := func() {
select {
case fired <- struct{}{}:
default:
}
}
// Every second (6 fields because cron is created with WithSeconds).
scheduler, err := NewCronScheduler("* * * * * *", job)
if err != nil {
t.Fatalf("NewCronScheduler failed: %v", err)
}
scheduler.Start()
select {
case <-fired:
// Job fired as expected.
case <-time.After(3 * time.Second):
t.Fatal("job did not fire within expected time")
}
stoppedCtx := scheduler.Stop()
select {
case <-stoppedCtx.Done():
// Scheduler stopped cleanly.
case <-time.After(2 * time.Second):
t.Fatal("scheduler did not stop within expected time")
}
}
func TestStopReturnsContext(t *testing.T) {
scheduler, err := NewCronScheduler("* * * * * *", func() {})
if err != nil {
t.Fatalf("NewCronScheduler failed: %v", err)
}
scheduler.Start()
stoppedCtx := scheduler.Stop()
if stoppedCtx == nil {
t.Fatal("Stop returned nil context")
}
select {
case <-stoppedCtx.Done():
// Expected.
case <-time.After(2 * time.Second):
t.Fatal("stopped context was not cancelled")
}
}
pkg/adapters/storage/local/local.go
+132
View File
@@ -0,0 +1,132 @@
package local
import (
"errors"
"os"
"path/filepath"
"strings"
"sync"
"git.tswf.io/infra/go-synapse-backupper/pkg/domain"
)
// NewLocalSink creates a new local filesystem sink that writes into the given directory.
func NewLocalSink(dir string) domain.Sink {
return &localSink{dir: dir}
}
type localSink struct {
dir string
}
func (sink *localSink) Begin(key string) (domain.SinkTx, error) {
tmpPath := filepath.Join(sink.dir, key+".tmp")
finalPath := filepath.Join(sink.dir, key)
file, err := os.Create(tmpPath)
if err != nil {
return nil, err
}
return &localSinkTx{
file: file,
tmpPath: tmpPath,
finalPath: finalPath,
}, nil
}
func (sink *localSink) List(prefix string) ([]string, error) {
entries, err := os.ReadDir(sink.dir)
if err != nil {
return nil, err
}
keys := make([]string, 0, len(entries))
for _, entry := range entries {
if entry.IsDir() {
continue
}
name := entry.Name()
if prefix != "" && !strings.HasPrefix(name, prefix) {
continue
}
keys = append(keys, name)
}
return keys, nil
}
func (sink *localSink) Remove(key string) error {
path := filepath.Join(sink.dir, key)
return os.Remove(path)
}
type localSinkTx struct {
file *os.File
tmpPath string
finalPath string
committed bool
aborted bool
mu sync.Mutex
}
func (transaction *localSinkTx) Write(p []byte) (int, error) {
transaction.mu.Lock()
defer transaction.mu.Unlock()
if transaction.committed || transaction.aborted {
return 0, errors.New("transaction already finished")
}
return transaction.file.Write(p)
}
func (transaction *localSinkTx) Commit() error {
transaction.mu.Lock()
defer transaction.mu.Unlock()
if transaction.committed || transaction.aborted {
return nil
}
transaction.committed = true
if err := transaction.file.Sync(); err != nil {
return err
}
if err := transaction.file.Close(); err != nil {
return err
}
if err := os.Rename(transaction.tmpPath, transaction.finalPath); err != nil {
return err
}
parent, err := os.Open(filepath.Dir(transaction.tmpPath))
if err != nil {
return err
}
defer parent.Close()
if err := parent.Sync(); err != nil {
return err
}
return nil
}
func (transaction *localSinkTx) Abort() error {
transaction.mu.Lock()
defer transaction.mu.Unlock()
if transaction.committed || transaction.aborted {
return nil
}
transaction.aborted = true
_ = transaction.file.Close()
_ = os.Remove(transaction.tmpPath)
return nil
}
pkg/adapters/storage/local/local_test.go
+255
View File
@@ -0,0 +1,255 @@
package local
import (
"os"
"path/filepath"
"testing"
"git.tswf.io/infra/go-synapse-backupper/pkg/domain"
)
func TestLocalSink_BeginOpensTmpInSameDirectory(t *testing.T) {
dir := t.TempDir()
sink := NewLocalSink(dir)
transaction, err := sink.Begin("backup.sql")
if err != nil {
t.Fatalf("Begin failed: %v", err)
}
defer func() { _ = transaction.Abort() }()
tmpPath := filepath.Join(dir, "backup.sql.tmp")
if _, err := os.Stat(tmpPath); os.IsNotExist(err) {
t.Fatalf("expected tmp file to exist at %s", tmpPath)
}
}
func TestLocalSink_WriteStreamsBytes(t *testing.T) {
dir := t.TempDir()
sink := NewLocalSink(dir)
transaction, err := sink.Begin("backup.sql")
if err != nil {
t.Fatalf("Begin failed: %v", err)
}
defer func() { _ = transaction.Abort() }()
data := []byte("hello world")
written, err := transaction.Write(data)
if err != nil {
t.Fatalf("Write failed: %v", err)
}
if written != len(data) {
t.Fatalf("expected %d bytes written, got %d", len(data), written)
}
}
func TestLocalSink_CommitSequence(t *testing.T) {
dir := t.TempDir()
sink := NewLocalSink(dir)
transaction, err := sink.Begin("backup.sql")
if err != nil {
t.Fatalf("Begin failed: %v", err)
}
data := []byte("persistent data")
if _, err := transaction.Write(data); err != nil {
t.Fatalf("Write failed: %v", err)
}
tmpPath := filepath.Join(dir, "backup.sql.tmp")
tmpInfo, err := os.Stat(tmpPath)
if err != nil {
t.Fatalf("stat tmp file failed: %v", err)
}
if err := transaction.Commit(); err != nil {
t.Fatalf("Commit failed: %v", err)
}
finalPath := filepath.Join(dir, "backup.sql")
if _, err := os.Stat(tmpPath); !os.IsNotExist(err) {
t.Fatalf("expected tmp file to be absent after commit")
}
finalInfo, err := os.Stat(finalPath)
if err != nil {
t.Fatalf("expected final file to exist after commit: %v", err)
}
if !os.SameFile(tmpInfo, finalInfo) {
t.Fatalf("expected final file to be the same inode as tmp file before rename")
}
}
func TestLocalSink_AbortIsIdempotent(t *testing.T) {
dir := t.TempDir()
sink := NewLocalSink(dir)
transaction, err := sink.Begin("backup.sql")
if err != nil {
t.Fatalf("Begin failed: %v", err)
}
if _, err := transaction.Write([]byte("discard me")); err != nil {
t.Fatalf("Write failed: %v", err)
}
if err := transaction.Abort(); err != nil {
t.Fatalf("first Abort failed: %v", err)
}
if err := transaction.Abort(); err != nil {
t.Fatalf("second Abort should be idempotent: %v", err)
}
tmpPath := filepath.Join(dir, "backup.sql.tmp")
finalPath := filepath.Join(dir, "backup.sql")
if _, err := os.Stat(tmpPath); !os.IsNotExist(err) {
t.Fatalf("expected tmp file to be absent after abort")
}
if _, err := os.Stat(finalPath); !os.IsNotExist(err) {
t.Fatalf("expected final file to be absent after abort")
}
}
func TestLocalSink_AbortAfterCommitIsSafe(t *testing.T) {
dir := t.TempDir()
sink := NewLocalSink(dir)
transaction, err := sink.Begin("backup.sql")
if err != nil {
t.Fatalf("Begin failed: %v", err)
}
if _, err := transaction.Write([]byte("data")); err != nil {
t.Fatalf("Write failed: %v", err)
}
if err := transaction.Commit(); err != nil {
t.Fatalf("Commit failed: %v", err)
}
if err := transaction.Abort(); err != nil {
t.Fatalf("Abort after commit should be safe: %v", err)
}
finalPath := filepath.Join(dir, "backup.sql")
if _, err := os.Stat(finalPath); err != nil {
t.Fatalf("expected final file to still exist after abort-after-commit: %v", err)
}
}
func TestLocalSink_KillWriterMidStreamThenAbortLeavesNoFinal(t *testing.T) {
dir := t.TempDir()
sink := NewLocalSink(dir)
transaction, err := sink.Begin("backup.sql")
if err != nil {
t.Fatalf("Begin failed: %v", err)
}
if _, err := transaction.Write([]byte("partial")); err != nil {
t.Fatalf("Write failed: %v", err)
}
if err := transaction.Abort(); err != nil {
t.Fatalf("Abort failed: %v", err)
}
tmpPath := filepath.Join(dir, "backup.sql.tmp")
finalPath := filepath.Join(dir, "backup.sql")
if _, err := os.Stat(tmpPath); !os.IsNotExist(err) {
t.Fatalf("expected tmp file to be absent after abort")
}
if _, err := os.Stat(finalPath); !os.IsNotExist(err) {
t.Fatalf("expected final file to be absent after abort")
}
}
func TestLocalSink_ListAndRemove(t *testing.T) {
dir := t.TempDir()
sink := NewLocalSink(dir)
if err := os.WriteFile(filepath.Join(dir, "a.sql"), []byte("a"), 0o644); err != nil {
t.Fatalf("setup failed: %v", err)
}
if err := os.WriteFile(filepath.Join(dir, "b.sql"), []byte("b"), 0o644); err != nil {
t.Fatalf("setup failed: %v", err)
}
keys, err := sink.List("")
if err != nil {
t.Fatalf("List failed: %v", err)
}
if len(keys) != 2 {
t.Fatalf("expected 2 keys, got %d", len(keys))
}
if err := sink.Remove("a.sql"); err != nil {
t.Fatalf("Remove failed: %v", err)
}
keys, err = sink.List("")
if err != nil {
t.Fatalf("List after remove failed: %v", err)
}
if len(keys) != 1 {
t.Fatalf("expected 1 key after remove, got %d", len(keys))
}
}
func TestLocalSink_RemoveMissingIsError(t *testing.T) {
dir := t.TempDir()
sink := NewLocalSink(dir)
err := sink.Remove("nonexistent.sql")
if err == nil {
t.Fatalf("expected error removing missing file")
}
}
func TestLocalSink_WriteAfterCommitIsError(t *testing.T) {
dir := t.TempDir()
sink := NewLocalSink(dir)
transaction, err := sink.Begin("backup.sql")
if err != nil {
t.Fatalf("Begin failed: %v", err)
}
if _, err := transaction.Write([]byte("data")); err != nil {
t.Fatalf("Write failed: %v", err)
}
if err := transaction.Commit(); err != nil {
t.Fatalf("Commit failed: %v", err)
}
if _, err := transaction.Write([]byte("more")); err == nil {
t.Fatalf("expected error writing after commit")
}
}
func TestLocalSink_WriteAfterAbortIsError(t *testing.T) {
dir := t.TempDir()
sink := NewLocalSink(dir)
transaction, err := sink.Begin("backup.sql")
if err != nil {
t.Fatalf("Begin failed: %v", err)
}
if err := transaction.Abort(); err != nil {
t.Fatalf("Abort failed: %v", err)
}
if _, err := transaction.Write([]byte("more")); err == nil {
t.Fatalf("expected error writing after abort")
}
}
var (
_ domain.Sink = (*localSink)(nil)
_ domain.SinkTx = (*localSinkTx)(nil)
)
pkg/domain/backup/backup.go
+75
View File
@@ -0,0 +1,75 @@
package backup
import (
"context"
"crypto/rand"
"fmt"
"time"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/config"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/crypto/composite"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/crypto/keymanager"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/crypto/mlkem768"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/crypto/x25519"
adapterpgdump "git.tswf.io/infra/go-synapse-backupper/pkg/adapters/pgdump"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/pipeline"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/retention"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/storage/local"
domaincrypto "git.tswf.io/infra/go-synapse-backupper/pkg/domain/crypto"
domainpgdump "git.tswf.io/infra/go-synapse-backupper/pkg/domain/pgdump"
)
func RunOnce(ctx context.Context, cfg *config.Config) error {
registry := domaincrypto.NewRegistry()
if err := registry.Register(0x0006, func() domaincrypto.KEM { return mlkem768.New() }); err != nil {
return fmt.Errorf("register mlkem768: %w", err)
}
if err := registry.Register(0x0007, func() domaincrypto.KEM { return x25519.New() }); err != nil {
return fmt.Errorf("register x25519: %w", err)
}
keyMgr := keymanager.NewKeyManager(registry)
pqPub, err := keyMgr.LoadPub(cfg.PQPublicKeyPath, cfg.PQScheme)
if err != nil {
return fmt.Errorf("load pq public key: %w", err)
}
classicalPub, err := keyMgr.LoadPub(cfg.ClassicalPublicKeyPath, cfg.ClassicalScheme)
if err != nil {
return fmt.Errorf("load classical public key: %w", err)
}
recipients := []domaincrypto.RecipientPub{pqPub, classicalPub}
encryptor := composite.NewEncryptor(registry)
dumper := adapterpgdump.New()
sink := local.NewLocalSink(cfg.Backup.Dir)
runner := pipeline.NewRunner(
pipeline.WithDumper(dumper),
pipeline.WithEncryptor(encryptor),
)
now := time.Now()
timestamp := now.UTC().Format("20060102-150405")
pgDumpOpts := domainpgdump.Options{
Host: cfg.PG.Host,
Port: cfg.PG.Port,
Database: cfg.PG.Database,
User: cfg.PG.User,
Password: cfg.PG.Password,
ExcludeTables: cfg.PG.ExcludeTables,
Key: fmt.Sprintf("synapse-%s.dump.pqenc", timestamp),
}
if err := runner.Run(ctx, pgDumpOpts, recipients, sink, rand.Reader); err != nil {
return fmt.Errorf("backup pipeline failed: %w", err)
}
if _, err := retention.PruneByAge(ctx, cfg.Backup.Dir, cfg.Backup.RetentionDays, now); err != nil {
return fmt.Errorf("retention pruning failed: %w", err)
}
return nil
}
pkg/domain/backup/backup_test.go
+129
View File
@@ -0,0 +1,129 @@
package backup
import (
"context"
"os"
"strings"
"testing"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/config"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/crypto/keymanager"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/crypto/mlkem768"
"git.tswf.io/infra/go-synapse-backupper/pkg/adapters/crypto/x25519"
"git.tswf.io/infra/go-synapse-backupper/pkg/domain/crypto"
)
// TestRunOnce_MissingKeys verifies that RunOnce surfaces a configuration error
// when the public key paths are not provided, exercising the early validation
// path (registry build, keymanager construction, key load).
func TestRunOnce_MissingKeys(t *testing.T) {
cfg := &config.Config{}
// PQPublicKeyPath and ClassicalPublicKeyPath intentionally left empty.
err := RunOnce(context.Background(), cfg)
if err == nil {
t.Fatal("RunOnce returned nil error with empty config, want error")
}
if !strings.Contains(err.Error(), "load pq public key") {
t.Fatalf("RunOnce error = %q, want it to mention %q", err, "load pq public key")
}
}
// TestRunOnce_PQKeyNotFound verifies the error returned when the post-quantum
// public key path is set but the file does not exist on disk.
func TestRunOnce_PQKeyNotFound(t *testing.T) {
cfg := &config.Config{
PQScheme: 0x0006,
// Path points to a file that does not exist; use t.TempDir() to keep
// the test hermetic regardless of the working directory.
PQPublicKeyPath: t.TempDir() + "/does-not-exist-pq.pem",
ClassicalPublicKeyPath: t.TempDir() + "/does-not-exist-classical.pem",
}
err := RunOnce(context.Background(), cfg)
if err == nil {
t.Fatal("RunOnce returned nil error when keys are missing on disk, want error")
}
if !strings.Contains(err.Error(), "load pq public key") {
t.Fatalf("RunOnce error = %q, want it to mention %q", err, "load pq public key")
}
}
// TestRunOnce_ValidKeys_PgDumpMissing generates real dual-KEM keys, builds a
// fully valid config, and calls RunOnce. Since pg_dump is not installed in the
// test environment, the pipeline fails at the dump step, giving coverage of
// the full orchestration path (registry, keymanager, encryptor, sink, runner,
// pgDumpOpts assembly) while still asserting the expected error.
func TestRunOnce_ValidKeys_PgDumpMissing(t *testing.T) {
dir := t.TempDir()
keyDir := dir + "/keys"
backupDir := dir + "/backups"
_ = os.MkdirAll(keyDir, 0o755)
_ = os.MkdirAll(backupDir, 0o755)
reg := crypto.NewRegistry()
_ = reg.Register(0x0006, func() crypto.KEM { return mlkem768.New() })
_ = reg.Register(0x0007, func() crypto.KEM { return x25519.New() })
km := keymanager.NewKeyManager(reg)
pqPubFile, _ := os.Create(keyDir + "/pq.pub.pem")
pqPrivFile, _ := os.Create(keyDir + "/pq.priv.pem")
_ = km.Generate(0x0006, pqPubFile, pqPrivFile, nil)
_ = pqPubFile.Close()
_ = pqPrivFile.Close()
classicalPubFile, _ := os.Create(keyDir + "/classical.pub.pem")
classicalPrivFile, _ := os.Create(keyDir + "/classical.priv.pem")
_ = km.Generate(0x0007, classicalPubFile, classicalPrivFile, nil)
_ = classicalPubFile.Close()
_ = classicalPrivFile.Close()
cfg := &config.Config{
PQScheme: 0x0006,
ClassicalScheme: 0x0007,
PQPublicKeyPath: keyDir + "/pq.pub.pem",
ClassicalPublicKeyPath: keyDir + "/classical.pub.pem",
Backup: struct {
Dir string `mapstructure:"dir"`
RetentionDays int `mapstructure:"retention_days"`
Cron string `mapstructure:"cron"`
}{
Dir: backupDir,
RetentionDays: 180,
},
PG: struct {
Host string `mapstructure:"host"`
Port int `mapstructure:"port"`
User string `mapstructure:"user"`
Password string `mapstructure:"password"`
Database string `mapstructure:"database"`
SSLMode string `mapstructure:"sslmode"`
ExcludeTables []string `mapstructure:"exclude_tables"`
}{
Host: "localhost",
Port: 5432,
User: "test",
Password: "test",
Database: "test",
ExcludeTables: []string{"e2e_one_time_keys_json"},
},
}
err := RunOnce(context.Background(), cfg)
if err == nil {
t.Fatal("expected error because pg_dump is not installed in the test environment")
}
if !strings.Contains(err.Error(), "backup pipeline failed") {
t.Fatalf("expected error containing 'backup pipeline failed', got: %v", err)
}
entries, _ := os.ReadDir(backupDir)
for _, e := range entries {
if strings.HasSuffix(e.Name(), ".tmp") {
t.Fatalf("unexpected .tmp file after failed backup: %s", e.Name())
}
}
}
pkg/domain/crypto/decryptor.go
+12
View File
@@ -0,0 +1,12 @@
package crypto
import "io"
// Decryptor defines the contract for decrypting ciphertext using private keys.
type Decryptor interface {
Decrypt(
src io.Reader,
privs []RecipientPriv,
plaintext io.Writer,
) error
}
pkg/domain/crypto/encryptor.go
+13
View File
@@ -0,0 +1,13 @@
package crypto
import "io"
// Encryptor defines the contract for encrypting plaintext to multiple recipients.
type Encryptor interface {
Encrypt(
plaintext io.Reader,
recipients []RecipientPub,
sink io.Writer,
rand io.Reader,
) error
}
pkg/domain/crypto/kem.go
+50
View File
@@ -0,0 +1,50 @@
package crypto
import "io"
// KEM defines the contract for a Key Encapsulation Mechanism.
type KEM interface {
SchemeID() uint16
GenerateKeyPair(
rand io.Reader,
) (
RecipientPub,
RecipientPriv,
error,
)
Encapsulate(
pub RecipientPub,
rand io.Reader,
) (
ciphertext []byte,
sharedSecret []byte,
err error,
)
Decapsulate(
priv RecipientPriv,
ciphertext []byte,
) (
sharedSecret []byte,
err error,
)
LoadPriv(
raw []byte,
) (
RecipientPriv,
error,
)
}
// RecipientPub represents a public recipient key.
type RecipientPub interface {
SchemeID() uint16
KeyID() []byte
Raw() []byte
}
// RecipientPriv represents a private recipient key.
type RecipientPriv interface {
SchemeID() uint16
KeyID() []byte
Raw() []byte
}
pkg/domain/crypto/keymanager.go
+27
View File
@@ -0,0 +1,27 @@
package crypto
import "io"
// KeyManager defines the contract for loading and generating recipient keys.
type KeyManager interface {
LoadPub(
path string,
schemeID uint16,
) (
RecipientPub,
error,
)
LoadPriv(
path string,
schemeID uint16,
) (
RecipientPriv,
error,
)
Generate(
schemeID uint16,
pubOut io.Writer,
privOut io.Writer,
rand io.Reader,
) error
}
pkg/domain/crypto/registry.go
+84
View File
@@ -0,0 +1,84 @@
package crypto
import (
"errors"
"fmt"
"sync"
)
// Sentinel errors for registry operations.
var (
ErrUnknownScheme = errors.New("unknown KEM scheme")
ErrDuplicateScheme = errors.New("duplicate KEM scheme registration")
)
// KEMFactory is a constructor function that returns a fresh KEM instance.
type KEMFactory func() KEM
// Registry is a concurrent-safe map of KEM scheme IDs to their factories.
type Registry interface {
Register(
schemeID uint16,
factory KEMFactory,
) error
Lookup(
schemeID uint16,
) (
KEMFactory,
error,
)
}
// registry is the private implementation of Registry.
type registry struct {
mutex sync.RWMutex
factories map[uint16]KEMFactory
}
// NewRegistry creates a new empty Registry.
func NewRegistry() Registry {
return &registry{
factories: make(map[uint16]KEMFactory),
}
}
// Register adds a KEMFactory for the given schemeID.
// Returns ErrDuplicateScheme if the schemeID is already registered.
func (r *registry) Register(schemeID uint16, factory KEMFactory) error {
r.mutex.Lock()
defer r.mutex.Unlock()
if _, exists := r.factories[schemeID]; exists {
return fmt.Errorf(
"scheme 0x%04x: %w",
schemeID,
ErrDuplicateScheme,
)
}
r.factories[schemeID] = factory
return nil
}
// Lookup retrieves the KEMFactory for the given schemeID.
// Returns ErrUnknownScheme if the schemeID is not registered.
func (r *registry) Lookup(
schemeID uint16,
) (
KEMFactory,
error,
) {
r.mutex.RLock()
defer r.mutex.RUnlock()
factory, exists := r.factories[schemeID]
if !exists {
return nil, fmt.Errorf(
"scheme 0x%04x: %w",
schemeID,
ErrUnknownScheme,
)
}
return factory, nil
}
pkg/domain/crypto/registry_test.go
+173
View File
@@ -0,0 +1,173 @@
package crypto
import (
"errors"
"io"
"sync"
"testing"
)
// mockKEM is a minimal KEM implementation for testing the registry.
type mockKEM struct {
schemeID uint16
}
func (m *mockKEM) SchemeID() uint16 {
return m.schemeID
}
func (m *mockKEM) GenerateKeyPair(
rand io.Reader,
) (
RecipientPub,
RecipientPriv,
error,
) {
return nil, nil, nil
}
func (m *mockKEM) Encapsulate(
pub RecipientPub,
rand io.Reader,
) (
ciphertext []byte,
sharedSecret []byte,
err error,
) {
return nil, nil, nil
}
func (m *mockKEM) Decapsulate(
priv RecipientPriv,
ciphertext []byte,
) (
sharedSecret []byte,
err error,
) {
return nil, nil
}
func (m *mockKEM) LoadPriv(
raw []byte,
) (
RecipientPriv,
error,
) {
return &mockRecipient{schemeIDValue: m.schemeID, rawBytes: raw}, nil
}
// mockRecipient implements RecipientPub and RecipientPriv for tests.
type mockRecipient struct {
schemeIDValue uint16
rawBytes []byte
}
func (r *mockRecipient) SchemeID() uint16 {
return r.schemeIDValue
}
func (r *mockRecipient) KeyID() []byte {
return r.rawBytes
}
func (r *mockRecipient) Raw() []byte {
return r.rawBytes
}
func TestRegistry_RegisterAndLookup(t *testing.T) {
registry := NewRegistry()
factory := func() KEM {
return &mockKEM{schemeID: 0x0001}
}
err := registry.Register(0x0001, factory)
if err != nil {
t.Fatalf("unexpected error registering scheme: %v", err)
}
foundFactory, err := registry.Lookup(0x0001)
if err != nil {
t.Fatalf("unexpected error looking up scheme: %v", err)
}
kem := foundFactory()
if kem.SchemeID() != 0x0001 {
t.Errorf("expected schemeID 0x0001, got 0x%04x", kem.SchemeID())
}
}
func TestRegistry_LookupUnknownScheme(t *testing.T) {
registry := NewRegistry()
_, err := registry.Lookup(0x9999)
if err == nil {
t.Fatal("expected error for unknown scheme, got nil")
}
if !errors.Is(err, ErrUnknownScheme) {
t.Errorf("expected ErrUnknownScheme, got %v", err)
}
}
func TestRegistry_RegisterDuplicateScheme(t *testing.T) {
registry := NewRegistry()
factory := func() KEM {
return &mockKEM{schemeID: 0x0001}
}
err := registry.Register(0x0001, factory)
if err != nil {
t.Fatalf("unexpected error on first register: %v", err)
}
err = registry.Register(0x0001, factory)
if err == nil {
t.Fatal("expected error for duplicate scheme, got nil")
}
if !errors.Is(err, ErrDuplicateScheme) {
t.Errorf("expected ErrDuplicateScheme, got %v", err)
}
}
func TestRegistry_ConcurrentAccess(t *testing.T) {
registry := NewRegistry()
const goroutines = 100
var waitGroup sync.WaitGroup
waitGroup.Add(goroutines)
for index := range goroutines {
go func(schemeID uint16) {
defer waitGroup.Done()
factory := func() KEM {
return &mockKEM{schemeID: schemeID}
}
_ = registry.Register(schemeID, factory)
_, _ = registry.Lookup(schemeID)
}(uint16(index + 1))
}
waitGroup.Wait()
}
func TestRegistry_LookupReturnsIndependentInstances(t *testing.T) {
registry := NewRegistry()
factory := func() KEM {
return &mockKEM{schemeID: 0x0001}
}
_ = registry.Register(0x0001, factory)
factoryOne, _ := registry.Lookup(0x0001)
factoryTwo, _ := registry.Lookup(0x0001)
kemOne := factoryOne()
kemTwo := factoryTwo()
if kemOne == kemTwo {
t.Error("expected independent KEM instances from factory")
}
}
pkg/domain/pgdump/dumper.go
+50
View File
@@ -0,0 +1,50 @@
package pgdump
import (
"context"
"fmt"
"io"
)
// Options holds configuration for the pg_dump invocation.
type Options struct {
Host string
Port int
Database string
User string
Password string
Key string
ExcludeTables []string
}
// Dumper defines the contract for executing pg_dump.
type Dumper interface {
Dump(
ctx context.Context,
opts Options,
sink io.Writer,
) error
}
// pgDumpFailedError is returned when pg_dump exits with a non-zero status.
type pgDumpFailedError struct {
exitCode int
}
func (e *pgDumpFailedError) Error() string {
return fmt.Sprintf("pg_dump failed with exit code %d", e.exitCode)
}
// Is reports whether target is a pgDumpFailedError with the same exit code.
func (e *pgDumpFailedError) Is(target error) bool {
other, ok := target.(*pgDumpFailedError)
if !ok {
return false
}
return e.exitCode == other.exitCode
}
// ErrPgDumpFailed creates a new pg_dump failed error with the given exit code.
func ErrPgDumpFailed(exitCode int) error {
return &pgDumpFailedError{exitCode: exitCode}
}
pkg/domain/pgdump/dumper_test.go
+72
View File
@@ -0,0 +1,72 @@
package pgdump
import (
"errors"
"fmt"
"testing"
)
func TestErrPgDumpFailed_Error(t *testing.T) {
cases := []struct {
name string
exitCode int
want string
}{
{name: "zero", exitCode: 0, want: "pg_dump failed with exit code 0"},
{name: "one", exitCode: 1, want: "pg_dump failed with exit code 1"},
{name: "large", exitCode: 137, want: "pg_dump failed with exit code 137"},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
err := ErrPgDumpFailed(tc.exitCode)
if err == nil {
t.Fatalf("ErrPgDumpFailed(%d) returned nil, want error", tc.exitCode)
}
if got := err.Error(); got != tc.want {
t.Fatalf("Error() = %q, want %q", got, tc.want)
}
})
}
}
func TestErrPgDumpFailed_Is(t *testing.T) {
base := ErrPgDumpFailed(1)
cases := []struct {
name string
target error
want bool
}{
{
name: "same exit code",
target: ErrPgDumpFailed(1),
want: true,
},
{
name: "different exit code",
target: ErrPgDumpFailed(2),
want: false,
},
{
name: "unrelated error type",
target: errors.New("something else"),
want: false,
},
{
// Is does not unwrap target: a fmt.Errorf-wrapped pgDumpFailedError
// fails the exact type check, so errors.Is reports false here.
name: "wrapped same-type error",
target: fmt.Errorf("wrapped: %w", ErrPgDumpFailed(1)),
want: false,
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
if got := errors.Is(base, tc.target); got != tc.want {
t.Fatalf("errors.Is(%v, %v) = %v, want %v", base, tc.target, got, tc.want)
}
})
}
}
pkg/domain/scheduler.go
+12
View File
@@ -0,0 +1,12 @@
package domain
import "context"
// Scheduler is the abstraction for a recurring job scheduler.
type Scheduler interface {
// Start begins executing the scheduled job.
Start()
// Stop halts the scheduler and returns a context that is cancelled
// once all in-flight jobs have completed.
Stop() (stoppedCtx context.Context)
}
pkg/domain/sink.go
+17
View File
@@ -0,0 +1,17 @@
package domain
import "io"
// Sink defines the contract for a storage sink that supports atomic two-phase writes.
type Sink interface {
Begin(key string) (SinkTx, error)
List(prefix string) ([]string, error)
Remove(key string) error
}
// SinkTx represents an in-progress write transaction.
type SinkTx interface {
io.Writer
Commit() error
Abort() error
}
resources/config.example.yaml
View File
scripts/integration-test.sh
Executable
+70
View File
@@ -0,0 +1,70 @@
#!/bin/bash
set -euo pipefail
IMAGE="synapse-backupper:test"
cleanup() {
docker rm -f sb-pg-test 2>/dev/null || true
rm -rf tmp-keys tmp-backups tmp-restored
}
trap cleanup EXIT
# 1. Start PostgreSQL
docker run -d --name sb-pg-test -e POSTGRES_PASSWORD=test postgres:17-alpine
# Wait for ready
for i in {1..30}; do
docker exec sb-pg-test pg_isready -U postgres && break
sleep 1
done
# 2. Create database
docker exec sb-pg-test psql -U postgres -c "CREATE DATABASE synapse;"
# 3. Create fixture table
docker exec sb-pg-test psql -U postgres -d synapse -c "CREATE TABLE rooms (id serial PRIMARY KEY, name text); INSERT INTO rooms (name) VALUES ('test1');"
# 4. Generate keys
mkdir -p tmp-keys
chmod 777 tmp-keys
docker run --rm -v "$(pwd)/tmp-keys:/keys" "$IMAGE" keygen --type both --out-prefix /keys/test
# 5. Run backup
mkdir -p tmp-backups
chmod 777 tmp-backups
docker run --rm --network container:sb-pg-test \
-v "$(pwd)/tmp-keys:/keys:ro" \
-v "$(pwd)/tmp-backups:/backups" \
-e APP_PG_HOST=127.0.0.1 \
-e APP_PG_USER=postgres \
-e APP_PG_PASSWORD=test \
-e APP_PG_DATABASE=synapse \
-e APP_PQ_PUBLIC_KEY_PATH=/keys/test.pq.pub.pem \
-e APP_CLASSICAL_PUBLIC_KEY_PATH=/keys/test.classical.pub.pem \
-e APP_BACKUP_DIR=/backups \
"$IMAGE" backup
# 6. Assert exactly ONE .pqenc file
PQENC_COUNT=$(ls tmp-backups/*.pqenc 2>/dev/null | wc -l)
if [ "$PQENC_COUNT" -ne 1 ]; then
echo "Expected exactly 1 .pqenc file, found $PQENC_COUNT"
exit 1
fi
PQENC_FILE=$(ls tmp-backups/*.pqenc)
# 7. Restore
mkdir -p tmp-restored
chmod 777 tmp-restored
docker run --rm \
-v "$(pwd)/tmp-keys:/keys:ro" \
-v "$(pwd)/tmp-backups:/backups:ro" \
-v "$(pwd)/tmp-restored:/out" \
"$IMAGE" restore \
--in "/backups/$(basename "$PQENC_FILE")" \
--privkey-pq /keys/test.pq.priv.pem \
--privkey-classical /keys/test.classical.priv.pem \
--out /out/restored.dump
# 8. Assert restore dump contains rooms table
grep -q "CREATE TABLE public.rooms" tmp-restored/restored.dump
echo "Integration test PASSED"
synapse-backupper
Executable
BIN
View File
Binary file not shown.
sync-skills.sh
Executable
+66
View File
@@ -0,0 +1,66 @@
#!/usr/bin/env bash
set -euo pipefail
# Configuration
REPO_URL="https://git.tswf.io/ai-agents/ai-agent-skills.git"
TARGET_DIR=".agents/skills"
# Get script directory for relative paths
if [ -n "${BASH_SOURCE[0]+x}" ]; then
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
cd "$SCRIPT_DIR"
fi
# Create target directory if it doesn't exist
mkdir -p "$TARGET_DIR"
# Create temporary directory
TEMP_DIR=$(mktemp -d)
echo "Created temporary directory: $TEMP_DIR"
# Ensure cleanup on exit
cleanup() {
if [ -d "$TEMP_DIR" ]; then
rm -rf "$TEMP_DIR"
echo "Cleaned up temporary directory"
fi
}
trap cleanup EXIT
# Clone repository
echo "Cloning repository $REPO_URL..."
git clone --depth 1 "$REPO_URL" "$TEMP_DIR/repo"
# Check if clone was successful
if [ ! -d "$TEMP_DIR/repo" ]; then
echo "Error: Failed to clone repository"
exit 1
fi
# Find and process skills (subdirectories in the cloned repo)
SKILLS_FOUND=0
for skill_path in "$TEMP_DIR/repo/skills"/*/; do
# Skip if no directories found (glob returns literal pattern if no match)
[ -d "$skill_path" ] || continue
skill_name=$(basename "$skill_path")
target_skill_path="$TARGET_DIR/$skill_name"
# Remove existing skill if present
if [ -e "$target_skill_path" ]; then
echo "Removing existing skill: $skill_name"
rm -rf "$target_skill_path"
fi
# Copy new skill
echo "Installing skill: $skill_name"
cp -r "$skill_path" "$target_skill_path"
SKILLS_FOUND=$((SKILLS_FOUND + 1))
done
if [ "$SKILLS_FOUND" -eq 0 ]; then
echo "Warning: No skills found in repository"
else
echo "Successfully synchronized $SKILLS_FOUND skill(s) to $TARGET_DIR"
fi