Compare commits
4
Commits
8ccb8b7c15
...
master
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
237529f7db | ||
|
|
d241174855 | ||
|
|
f25f5df62d | ||
|
|
e965b0943d |
@@ -18,3 +18,8 @@ backups/
|
|||||||
.DS_Store
|
.DS_Store
|
||||||
Thumbs.db
|
Thumbs.db
|
||||||
*.swp
|
*.swp
|
||||||
|
|
||||||
|
# New architecture build outputs
|
||||||
|
/dist/
|
||||||
|
coverage.out
|
||||||
|
node_modules/
|
||||||
|
|||||||
+103
@@ -0,0 +1,103 @@
|
|||||||
|
# 跨设备接续入口
|
||||||
|
|
||||||
|
更新时间:2026-09-25。此文件用于在没有原聊天记录、没有原设备缓存的情况下接续开发。
|
||||||
|
这是开发交接,不是生产安装说明。新设备先阅读此文件,不要执行根 README 下方的旧部署脚本。
|
||||||
|
|
||||||
|
## 1. 获取正确分支
|
||||||
|
|
||||||
|
新架构已从 `codex/new-deployment-architecture` 合并到 `master`,用户已手动推送源码,
|
||||||
|
并授权提交、推送最新交接文档。**后续开发和跨设备接续统一使用 `master`。**
|
||||||
|
先为新设备配置仓库 SSH 权限,并从可信渠道核对 Git 服务器主机指纹;不要禁用主机校验。
|
||||||
|
|
||||||
|
新设备克隆:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
git clone --branch master --single-branch ssh://git@git.joywaygames.cn:2222/joywaygamess/server-deploy.git
|
||||||
|
cd server-deploy
|
||||||
|
git branch --show-current
|
||||||
|
git log -1 --oneline
|
||||||
|
git status --short
|
||||||
|
```
|
||||||
|
|
||||||
|
已有本地仓库:先检查未提交更改,不要 reset、clean 或自动 stash;获取远端分支后再切换。
|
||||||
|
|
||||||
|
```sh
|
||||||
|
git status --short
|
||||||
|
git fetch origin
|
||||||
|
git switch master
|
||||||
|
git pull --ff-only
|
||||||
|
```
|
||||||
|
|
||||||
|
若此前用 `--single-branch` 只克隆了工作分支,先运行
|
||||||
|
`git fetch origin refs/heads/master:refs/remotes/origin/master` 获取 master;
|
||||||
|
本地尚无 master 时,用 `git switch -c master --no-track origin/master` 创建分支,
|
||||||
|
再用 `git pull --ff-only origin master` 显式同步(避免旧单分支 fetch 配置影响跟踪关系)。
|
||||||
|
普通完整克隆可使用 `git switch --track origin/master`。
|
||||||
|
用 `git log -1 --oneline` 核对交接提交;旧工作分支只作历史参考。
|
||||||
|
有冲突或远端分叉时停止核对,不强制覆盖。跨设备轮换前,显式提交并推送本轮需要带走的文件。
|
||||||
|
聊天记录、本机临时文件和未提交修改不会随 Git 同步。
|
||||||
|
|
||||||
|
## 2. 阅读顺序与权威性
|
||||||
|
|
||||||
|
1. 本文件:当前停止点、待确认项与安全边界。
|
||||||
|
2. [实施状态](docs/implementation-status.md):已完成工作包和实际验证证据。
|
||||||
|
3. [完整方案](docs/2026-09-25-complete-optimization-proposal.md):已批准的总体架构。
|
||||||
|
4. [开发环境](docs/development.md):新设备安装前提、构建与测试。
|
||||||
|
5. [CLI 协议](protocol/README.md)及所改模块的 `internal/<模块>/README.md`:接口与限制。
|
||||||
|
6. 需要远程工作时阅读[服务器初检](docs/2026-09-25-target-server-preflight.md),并重新核对现场。
|
||||||
|
|
||||||
|
2026-09-24 的 Caddy/结构审查文档是历史讨论,不能覆盖已选定的 Traefik 方案。
|
||||||
|
`docs/superpowers/plans/` 是各阶段计划,旧复选框不代表当前完成度;以实施状态和实际代码/测试为准。
|
||||||
|
仓库不依赖某一 AI 工具的会话、记忆或个人技能目录才能开发。
|
||||||
|
|
||||||
|
## 3. 已确认架构与交付形式
|
||||||
|
|
||||||
|
- 为新服务器建设,不做旧服务器的数据迁移或旧脚本兼容;保留原有数据是最高约束。
|
||||||
|
- 应用可独立部署,数据库随应用隔离;公共环境为 Docker Compose、Traefik 等。
|
||||||
|
- 本地管理产品:React/TypeScript 页面 + Node 本地服务 + SQLite,经 SSH 调用远端 Go `deployctl`。
|
||||||
|
- 用户界面由浏览器打开;本地 API 只监听回环并需会话、Host/Origin、CSRF 防护。
|
||||||
|
- 不是已选定的 Electron/Tauri 桌面安装 App,也不是公网管理网站。
|
||||||
|
- 便携目标:Windows/macOS 解压启动,运行时随包交付,不要求最终用户安装开发工具。
|
||||||
|
启动器、端口分配、数据目录、退出行为、签名和更新机制尚未实现/验收。
|
||||||
|
- Apple 风格 UI;面板关闭不应影响已部署应用,已提交远端任务最终由 systemd 独立监督。
|
||||||
|
后一项仍是未完成的设计目标。
|
||||||
|
- `claude-dev-stack` 不纳入新体系;旧脚本暂留对照,不得作为新执行器的后端。
|
||||||
|
|
||||||
|
## 4. 当前完成与停止点
|
||||||
|
|
||||||
|
已完成八个基础工作包:离线计划、状态/锁、应用包完整性、受限 Compose 策略、本机预检、
|
||||||
|
dpkg 清单/版本锁、Docker 仓库签名认证、实际五个 Docker deb 的字节校验。
|
||||||
|
|
||||||
|
`cmd/deployctl` 和 `internal/` 为当前实现。CLI 没有可用的安装、部署、升级、恢复写入口;
|
||||||
|
`executable` 始终为 false。状态库尚未接入执行器。
|
||||||
|
`apps/ops-panel/` 尚不存在;本地服务、UI、SSH 管理层、便携打包尚未交付。
|
||||||
|
真实 deb 测试通过不等于完整依赖解析或安装验收。
|
||||||
|
|
||||||
|
下一工作包:Ubuntu 依赖源认证与隔离的完整 APT 依赖事务预演。
|
||||||
|
最近提出的细化方案是:原生 APT 求解器、可信元数据/包状态快照、独立配置/缓存、
|
||||||
|
不继承宿主钩子、只模拟;删除/降级/修改保留包或现场漂移时拒绝,输出完整变更清单。
|
||||||
|
**该 APT 适配器细化方案尚待用户确认,尚未实现;本次跨设备交接不是对它的批准。**
|
||||||
|
接续时先复跑基础测试,再请用户确认此边界,随后编写实施计划和失败测试。
|
||||||
|
|
||||||
|
## 5. 服务器与凭据边界
|
||||||
|
|
||||||
|
- 指定新服务器及已核对指纹见服务器初检文档;该文档是历史观察,不是当前现场保证。
|
||||||
|
- 截至本次交接,只完成该新服务器 SSH 只读初检;新执行器没有在那里安装/部署应用。
|
||||||
|
- 新架构阶段没有操作旧服务器;不要复用旧升级任务的授权去迁移或恢复数据。
|
||||||
|
- 新设备需要自行配置私钥或 SSH Agent、仓库访问权限和可信 known_hosts;均不通过 Git 交付。
|
||||||
|
- 私钥、有效密码、token、真实 `.env`、个人 SSH 配置、备份和数据库不能新增到提交中。
|
||||||
|
`.gitignore` 不会自动移除历史已跟踪的敏感文件;旧配置不得作为新部署输入。
|
||||||
|
历史凭据轮换和 Git 历史清理需另行授权,不在本次交接范围。
|
||||||
|
- `claude-dev-stack/.env` 的删除已按用户要求提交(`f25f5df`),并包含在本地 master 中,不恢复它。
|
||||||
|
不以合并为由擅自清理其余旧目录;Git 历史仍可能含旧配置,不要使用或复制其中凭据。
|
||||||
|
- 远端写入前重新核对主机身份、数据/服务、容量和影响,并取得具体操作授权。
|
||||||
|
不做整机升级、不重置防火墙、不删除卷/数据库、不把备份恢复覆盖到原数据。
|
||||||
|
|
||||||
|
## 6. 可直接交给接续助手的说明
|
||||||
|
|
||||||
|
> 请先阅读 HANDOFF.md、docs/development.md、docs/implementation-status.md 和已批准的完整方案。
|
||||||
|
> 检查实际分支和 git status,保留用户修改,在当前目录接续,不依赖旧聊天或其他设备的绝对路径。
|
||||||
|
> 先构建并运行本机测试,区分历史证据与本次结果。当前停在第八工作包之后,APT 隔离细化方案待确认。
|
||||||
|
> 不执行旧部署脚本,不改服务器数据,不把校验结果当成安装授权;提交/推送按本轮用户要求处理。
|
||||||
|
|
||||||
|
后续每个工作包完成时更新本文件的停止点及实施状态;涉及协议、工具链或交付方式变化时同步开发指南。
|
||||||
@@ -1,5 +1,14 @@
|
|||||||
# Docker 服务部署集合
|
# Docker 服务部署集合
|
||||||
|
|
||||||
|
> **换设备继续开发:先读 [HANDOFF.md](HANDOFF.md),按 [开发指南](docs/development.md) 准备环境。**
|
||||||
|
> 新架构已合并到 `master`;其他设备请拉取 `master` 并按交接说明继续开发。
|
||||||
|
|
||||||
|
> 新服务器架构正在重构:选定 **Traefik + 本地管理面板 + Go deployctl**。
|
||||||
|
> 当前新执行器支持离线计划、应用包校验、受限 Compose 策略、本机预检、环境安装提案及 Docker 仓库签名/deb 文件校验,尚不能部署、升级或恢复应用。
|
||||||
|
> 参见 [完整方案](docs/2026-09-25-complete-optimization-proposal.md)、
|
||||||
|
> [实施状态](docs/implementation-status.md) 和 [CLI 使用说明](protocol/README.md)。
|
||||||
|
> 下方是旧部署实现的历史使用说明,不是新架构的安装指南;旧脚本不会成为新执行后端。
|
||||||
|
|
||||||
一套完整的自托管服务部署方案,适用于阿里云等国内服务器环境。
|
一套完整的自托管服务部署方案,适用于阿里云等国内服务器环境。
|
||||||
|
|
||||||
## 服务列表
|
## 服务列表
|
||||||
|
|||||||
@@ -1,43 +0,0 @@
|
|||||||
# =============================================================
|
|
||||||
# Claude Dev Stack 配置文件
|
|
||||||
# 复制为 .env 并按需填写
|
|
||||||
# =============================================================
|
|
||||||
|
|
||||||
# ── Claude / API 配置 ────────────────────────────────────────
|
|
||||||
# 推荐:灵眸 AI(国内直连,无需代理)https://docs.lmuai.com/docs/tools/claude-code
|
|
||||||
ANTHROPIC_AUTH_TOKEN=sk-ccaf8169bce7c735dfa1f021ec326b3904842dfd50e1ae817c3417017e0d8ee0
|
|
||||||
ANTHROPIC_BASE_URL=https://api.lmuai.com
|
|
||||||
|
|
||||||
# 备选:Anthropic 官方 API Key(海外直连)
|
|
||||||
# ANTHROPIC_API_KEY=
|
|
||||||
# ANTHROPIC_BASE_URL=https://api.anthropic.com
|
|
||||||
|
|
||||||
# 备选:DeepSeek 兼容接口
|
|
||||||
# ANTHROPIC_API_KEY=
|
|
||||||
# ANTHROPIC_BASE_URL=https://api.deepseek.com/anthropic
|
|
||||||
|
|
||||||
# 默认使用的模型
|
|
||||||
# 灵眸可选:claude-opus-4-7 | claude-sonnet-4-6 | claude-sonnet-4-5 | claude-haiku-4-5
|
|
||||||
# DeepSeek可选:deepseek-v4-pro
|
|
||||||
CLAUDE_MODEL=claude-sonnet-4-6
|
|
||||||
|
|
||||||
# ── WSL2 ────────────────────────────────────────────────────
|
|
||||||
# WSL2 发行版名称(wsl --list 查看已安装发行版)
|
|
||||||
WSL_DISTRO=Ubuntu
|
|
||||||
|
|
||||||
# 设为 true 可跳过 WSL2 安装步骤(已安装时使用)
|
|
||||||
SKIP_WSL_INSTALL=false
|
|
||||||
|
|
||||||
# ── Unity MCP ───────────────────────────────────────────────
|
|
||||||
# unity-mcp-server 自动克隆至 WSL2 ~/.mcp-servers/unity-mcp-server/
|
|
||||||
# Unity Plugin 需手动通过 Package Manager 安装:
|
|
||||||
# https://github.com/AnkleBreaker-Studio/unity-mcp-plugin.git
|
|
||||||
# (无需额外配置)
|
|
||||||
|
|
||||||
# ── Rust / Token Killer ──────────────────────────────────────
|
|
||||||
# (暂无需配置,预留扩展用)
|
|
||||||
# CARGO_REGISTRY_MIRROR=https://rsproxy.cn/
|
|
||||||
|
|
||||||
# ── Docker 镜像加速(可选)──────────────────────────────────
|
|
||||||
# 若 WSL2 内需要 Docker,可配置国内加速镜像(逗号分隔)
|
|
||||||
# DOCKER_REGISTRY_MIRRORS=https://docker.m.daocloud.io,https://hub-mirror.c.163.com
|
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"os"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"server-deploy/internal/cli"
|
||||||
|
)
|
||||||
|
|
||||||
|
func main() {
|
||||||
|
os.Exit(cli.Run(os.Args[1:], os.Stdin, os.Stdout, os.Stderr, time.Now))
|
||||||
|
}
|
||||||
@@ -0,0 +1,294 @@
|
|||||||
|
# 部署架构审查与管理工具接入建议
|
||||||
|
|
||||||
|
日期:2026-09-24。状态:历史审查与讨论记录,尚未实施。用户已明确以全新服务器为目标,不需要渐进改造或存量兼容;本文的源码发现仍供参考,涉及保留旧结构、接管与过渡层的建议已撤回。当前候选架构以 [全新部署架构](2026-09-24-greenfield-architecture.md) 为准。
|
||||||
|
|
||||||
|
## 1. 范围与结论
|
||||||
|
|
||||||
|
本次检查本仓库全部顶层目录、7 份 Compose、共享基础脚本、各工具部署/备份/卸载入口、Gitea/Vaultwarden 升级逻辑、Nginx 模板与已有测试目录。参考项目为 `G:/Works/YouleGames/tools/docker-ops-panel`,检查其文档、连接与身份核验、任务执行、服务清单、会话校验和打包结构。
|
||||||
|
|
||||||
|
这是本地源码与架构审查,没有连接生产服务器,没有执行部署、恢复、清理、安装或迁移;不把历史会话的线上结果当作当前生产状态。没有读取实际 `.env` 内容;在普通源码中发现硬编码凭据,本报告不收录其值。
|
||||||
|
|
||||||
|
结论:目前应用运行单元已基本分离,主要问题是部署生命周期仍耦合主机环境,缺少稳定的实例身份、操作协议和恢复边界。推荐保留“一工具一独立部署单元”,先规范脚本,再建立可选的本地 SSH 管理面板。无需合并成一份总 Compose,也无需为此引入 Kubernetes。
|
||||||
|
|
||||||
|
## 2. 当前目录与运行单元
|
||||||
|
|
||||||
|
- `base/`:Docker、Nginx、Certbot、防火墙、系统初始化及共享 Shell 函数;同时承担安装程序和公共库两种职责。
|
||||||
|
- `gitea/`:独立 Compose,应用 + 私有 MySQL 8.4;含部署、备份、升级、卸载及 Nginx 模板。
|
||||||
|
- `joplin/`:独立 Compose,应用 + 私有 PostgreSQL;含部署、备份、卸载及 Nginx 模板。
|
||||||
|
- `vaultwarden/`:独立 Compose,应用持久数据;含部署、备份、升级、卸载及 Nginx 模板。
|
||||||
|
- `certd/`、`siyuan/`、`portainer/`:各自独立 Compose、数据目录和运维脚本。
|
||||||
|
- `rustdesk/`:独立 Compose,hbbs + hbbr 组成一个工具,共享该工具的数据目录,包含公网 TCP/UDP 和回环 WebSocket 入口。
|
||||||
|
- `vps-xray/`:独立 systemd 部署,不使用 Compose;另含内核网络参数、防火墙、定时重启及客户端配置生成。应支持不同主机部署。
|
||||||
|
- `claude/`:开发机 CLI 配置脚本,不是 Linux 服务器应用栈。
|
||||||
|
- `claude-dev-stack/`:用户已明确废弃,目录与其中唯一的 `.env` 已从工作区删除,不纳入工具目录、管理界面、适配器或部署计划;历史版本仍保留该文件。
|
||||||
|
- `docs/`:已有 Xray 设计与测试计划。
|
||||||
|
- `.claude/`:本地辅助目录,不作为发布单元。
|
||||||
|
|
||||||
|
当前 Web 路径为:宿主 Nginx → 各应用的回环映射端口。MySQL/PostgreSQL 没有发布宿主端口。应用之间未发现业务依赖。Gitea SSH、RustDesk TCP/UDP 为公网直达例外。Certd 是独立证书管理应用,当前脚本中的站点证书仍由 Certbot 处理,不能认为 Certd 已接管所有证书。
|
||||||
|
|
||||||
|
## 3. 主要发现及证据
|
||||||
|
|
||||||
|
### P0:发布内容中存在凭据边界问题
|
||||||
|
|
||||||
|
初次审查时 `git ls-files` 确认 `claude-dev-stack/.env` 已被跟踪,随后按用户要求从工作区删除并排除管理范围;`claude/setup-claude-deepseek.sh:14` 和 PowerShell 对应脚本存在硬编码 API 凭据。根 `.gitignore` 中 Xray 生成配置的忽略规则被注释,多个节点信息文件也被跟踪。
|
||||||
|
|
||||||
|
新增面板的上传/打包必须使用发布文件白名单,不能递归上传整个工作区。需要另行确认并轮换已进入版本历史的有效凭据,将实际环境文件移出跟踪;仅添加 ignore 不会移除已经跟踪的文件。当前仅按用户要求删除废弃的 claude-dev-stack,不重写历史或轮换密钥。
|
||||||
|
|
||||||
|
### P1:应用部署会影响整台主机
|
||||||
|
|
||||||
|
`certd/deploy.sh:145` 等应用入口会调用 `init_system`;`base/setup.sh:72` 执行整机 `apt-get upgrade`。应用配置校验在这些步骤之后。Gitea 在自己的脚本里重复了同一套初始化逻辑。
|
||||||
|
|
||||||
|
`base/setup.sh:157` 在已有 Docker 配置缺少 registry-mirrors 字段时覆盖整个 daemon.json,随后重启 Docker。应用部署不应隐式执行这些操作。
|
||||||
|
|
||||||
|
建议拆分主机初始化、主机维护和应用生命周期;应用只检查基础能力。公共库被 source 时不得隐式安装或修改系统。
|
||||||
|
|
||||||
|
### P1:脚本位置依赖阻碍单独分发
|
||||||
|
|
||||||
|
Certd、Joplin、Vaultwarden、SiYuan、Portainer、RustDesk 的 deploy.sh 均要求 `../base/setup.sh` 存在,即使服务器已安装所需环境也无法跳过。它们运行时独立,部署包却不独立。
|
||||||
|
|
||||||
|
共享源代码可以保留,但单工具发布包应带版本固定的最小运维库,或使用显式安装并验证版本的公共运行器;不能依赖相邻仓库目录。建议先采用发布时打包最小库,减少服务端额外依赖。
|
||||||
|
|
||||||
|
### P1:同主机多实例隔离不完整
|
||||||
|
|
||||||
|
7 份 Compose 都使用固定 container_name;大部分宿主端口和 Nginx 上游固定。Vaultwarden 虽支持 VAULTWARDEN_PORT,Nginx 模板仍固定 8080。Gitea 升级检测支持 GITEA_HTTP_PORT,但 Compose 和 Nginx 仍固定 3000。
|
||||||
|
|
||||||
|
仅设置 Compose 项目名还不够,容器名、端口、数据路径、Nginx 站点名称和备份范围必须同时隔离。项目身份不能只靠当前文件夹名推断。
|
||||||
|
|
||||||
|
现有实例应读取 Docker Compose 标签与挂载建立绑定,不应直接删除 container_name 或改项目名后重建;这可能创建第二套容器或误连数据。
|
||||||
|
|
||||||
|
### P1:共享入口和证书缺少事务与归属管理
|
||||||
|
|
||||||
|
`base/setup.sh:342` 起先把站点替换成临时 HTTP 配置,再申请证书;失败后没有恢复旧站点。`deploy_nginx_conf` 覆盖目标文件后才测试,失败时磁盘上可能留着无效配置。
|
||||||
|
|
||||||
|
各工具卸载入口还会删除证书并可能移除共享 Certbot cron。应由独立入口模块持有站点和证书引用,应用卸载只释放自己的绑定;续期调度由环境模块负责。
|
||||||
|
|
||||||
|
建议保持宿主 Nginx,使用主机级锁,备份旧配置、生成候选、验证、切换、reload、探测,失败时还原。每张证书只有一个明确的管理者,Certbot 与 Certd 不同时自动修改同一证书。
|
||||||
|
|
||||||
|
### P1:备份/恢复契约不足以支撑通用面板
|
||||||
|
|
||||||
|
`certd/backup.sh`、`portainer/backup.sh`、`joplin/backup.sh` 等把配置打包错误通过 `|| true` 忽略,随后仍输出成功。多个脚本仅按目录年龄清理备份,缺少成功清单、引用保护与明确的自有文件边界。运行中的数据库/文件目录归档也不能统一视为一致快照。
|
||||||
|
|
||||||
|
Gitea 的备份有单独 flock,但 upgrade.sh 没有使用同一把实例锁,不能阻止备份和升级交叠。统一面板锁也必须覆盖 CLI 入口。
|
||||||
|
|
||||||
|
恢复应声明类型:配置恢复、镜像回退、数据库恢复、完整快照恢复。数据库迁移不能套用参考面板的通用镜像回退。备份目录需要 manifest、完成标记、文件摘要、版本、数据位置及一致性模式;被回滚点引用的备份禁止自动清理。
|
||||||
|
|
||||||
|
### P1:现有升级脚本不应直接视为经过全面验证的执行后端
|
||||||
|
|
||||||
|
`gitea/upgrade.sh:617` 附近的 wait_healthy 仍接受任意非 000 HTTP 响应,包括 500。自动和手动回滚忽略 stop 失败后继续恢复数据库,存在写入方未停就恢复数据的风险。自动回滚还可能在服务未就绪后输出“已回滚”。
|
||||||
|
|
||||||
|
恢复文件为合并解压,数据库恢复保留新版本额外表;这不等于精确还原。全量备份回灌也会覆盖备份内的 Git 引用文件,不能声称升级后的新提交天然保留。
|
||||||
|
|
||||||
|
此前“跨小版本必有 schema 版本增长”“表行数不减少即数据完好”“遗留表普遍无害”都不宜作为所有版本的通用保证。应绑定目标容器、实际数据库、目标版本兼容要求和具体恢复演练;通过有限场景只能证明对应场景。
|
||||||
|
|
||||||
|
### P2:非 Docker 工具及端口资源需要单独建模
|
||||||
|
|
||||||
|
Xray 默认 XRAY_PORT=443(`vps-xray/deploy.sh:411`),与同一地址上的 Nginx TLS 入口存在绑定冲突的可能;它还执行 sysctl 和防火墙修改。不能把所有工具都抽象成 docker compose up。
|
||||||
|
|
||||||
|
面板应支持 Compose 与 systemd 两类执行器;Xray 先支持状态及明确的维护动作。主机网络调优必须作为单独环境计划展示,安装前检查真实监听地址、协议和端口。
|
||||||
|
|
||||||
|
### P2:版本、能力、文档存在漂移
|
||||||
|
|
||||||
|
多个镜像使用 latest/lts/小版本浮动标签;仓库 Gitea 默认仍为 1.25。根 README 未覆盖 Xray、CLI 工具和残留目录,列出不存在的 gitea/migrate.sh,并把所有端口描述为回环,遗漏实际公网例外。
|
||||||
|
|
||||||
|
只有 Gitea/Vaultwarden 有专门 upgrade.sh,其余工具不能仅因存在 deploy.sh 就在面板上显示“安全升级/回滚”。面板应按适配器实际能力显示操作,解析并记录具体镜像 digest。
|
||||||
|
|
||||||
|
## 4. 推荐目标架构
|
||||||
|
|
||||||
|
采用三层结构,管理界面是可选控制入口:
|
||||||
|
|
||||||
|
1. **主机环境**:Docker/Compose、宿主 Nginx、证书续期、防火墙。单独检查、安装、维护,主机级变更说明影响范围。
|
||||||
|
2. **独立应用实例**:每实例独立 Compose 项目/或 systemd 单元、配置、数据、备份与任务记录。应用私有数据库留在实例内。Gitea 不依赖 Joplin、Certd 或 Portainer。
|
||||||
|
3. **本地管理面板与 CLI**:共用操作协议和执行器。面板停止后,应用继续工作,已提交的远程任务可以继续并被重新核对。
|
||||||
|
|
||||||
|
应用实例目标由 hostId + appId + instanceId + 规范化部署路径 + 实际项目/单元身份组成。所有写计划绑定目标、配置/编排摘要、明确版本、影响资源和恢复能力;执行前在锁内复核。
|
||||||
|
|
||||||
|
独立部署应同时满足:只选择一个工具即可准备它自己的依赖;不要求安装其他应用或面板;部署/升级不修改其他应用;独立停启、备份、恢复、卸载;同一台主机可部署第二实例;环境能力缺失时给出独立环境准备计划。
|
||||||
|
|
||||||
|
数据库可以在高级配置中外置,但默认不合并共享 MySQL/PostgreSQL;避免将升级、故障和恢复范围扩大到多个工具。
|
||||||
|
|
||||||
|
## 5. 管理工具参考项目的取舍
|
||||||
|
|
||||||
|
可借鉴的机制:
|
||||||
|
|
||||||
|
- React/Vite + Node 本地服务、Windows/macOS 便携启动与打包。
|
||||||
|
- SSH 私钥/Agent、SHA256 指纹确认,传输层明确区分断线和任务失败。
|
||||||
|
- 本地 Host/Origin/会话/CSRF 校验。
|
||||||
|
- 目标绑定与过期预览、配置漂移检查、受控参数、秘密遮蔽。
|
||||||
|
- 远端任务目录、退出码、日志、进程状态、断线后的状态核对。
|
||||||
|
- 主机环境操作与应用生命周期分离。
|
||||||
|
|
||||||
|
需要重建的业务层:
|
||||||
|
|
||||||
|
- `src/contracts.ts:1` 固定了 game-docker 的服务列表。
|
||||||
|
- `src/connections.ts:388` 会拒绝不存在 game-docker 服务的目录,不能直接连接本仓库的 Gitea/Joplin 部署。
|
||||||
|
- 部署目录、环境字段、微信/QQ、发布文件清单与业务站点路由均为专用逻辑。
|
||||||
|
- 现有档案偏向“一个连接目标绑定一个项目”;这里需要“一台主机登记多个独立实例”。
|
||||||
|
|
||||||
|
因此建议在本仓库新增 `tools/ops-panel/`,复用经抽取和测试的机制,应用适配器独立定义。不要直接搬整个参考项目再堆条件判断。参考项目自己也将真实 Linux Docker 验收列为未完成,不能把其本地测试当作生产保证。
|
||||||
|
|
||||||
|
Portainer 保持可选:它提供容器层管理,本工具负责仓库应用生命周期、备份与恢复计划。直接在 Portainer 修改编排可能造成漂移,面板须检测并要求重新识别。docker.sock 的 `:ro` 挂载不能当作 Docker API 只读权限。
|
||||||
|
|
||||||
|
## 6. 候选目录与交付方式
|
||||||
|
|
||||||
|
第一阶段保留现有应用目录和线上路径,新增职责明确的目录:
|
||||||
|
|
||||||
|
```text
|
||||||
|
server-deploy/
|
||||||
|
base/ # 兼容旧入口;逐步成为显式主机环境命令
|
||||||
|
lib/ # 可版本化、无隐式副作用的通用运维函数
|
||||||
|
contracts/ # 应用清单、计划、实例、任务、备份 schema
|
||||||
|
tools/ops-panel/ # 可独立启动的本地管理工具
|
||||||
|
gitea/ # 原路径保留,逐个添加应用清单与适配入口
|
||||||
|
joplin/
|
||||||
|
vaultwarden/
|
||||||
|
certd/ siyuan/ portainer/ rustdesk/
|
||||||
|
vps-xray/ # systemd 适配器
|
||||||
|
claude/ # 开发机配置工具,不默认纳入服务器部署
|
||||||
|
tests/ # 协议、脚本故障、真实隔离验收
|
||||||
|
docs/
|
||||||
|
```
|
||||||
|
|
||||||
|
应用清单声明:标识/版本、运行器、Compose 服务角色、环境要求、端口及入口、秘密字段、数据挂载、健康探测、升级与恢复策略、支持的动作。清单不是任意命令执行接口,执行器仍使用受控命令和参数校验。
|
||||||
|
|
||||||
|
远端先保持 `/opt/gitea` 等既有目录;已有数据路径按真实挂载登记。新实例可采用 `/opt/selfhost/instances/<实例>/`、`/var/lib/selfhost/<实例>/` 和 `/var/backups/selfhost/<实例>/`,但不自动移动旧目录。单工具发布包只包含自身清单、Compose、脚本/模板和所需版本的通用库,避免依赖整仓库。
|
||||||
|
|
||||||
|
## 7. 方案比较与实施顺序
|
||||||
|
|
||||||
|
**推荐:独立应用 + 共享环境 + 本地 SSH 面板。** 最接近当前结构,改造可以逐工具进行,日常运维不依赖服务器额外常驻控制服务。
|
||||||
|
|
||||||
|
**备选:独立应用 + 服务器 Web 面板。** 适合多人协作,但增加集中凭据、用户权限、审计、HTTPS 和面板自身可用性的维护成本。
|
||||||
|
|
||||||
|
**不推荐:全服务合并 Compose,通过 profiles 选择。** 能集中启动,却容易重新引入共享环境、项目级操作和恢复范围耦合,不符合本次强调的独立部署目标。
|
||||||
|
|
||||||
|
建议实施顺序:
|
||||||
|
|
||||||
|
1. 凭据与发布边界清理;明确环境与应用责任;登记所有已有实例,不重建容器。
|
||||||
|
2. 固化实例/任务/备份协议;统一实例锁及主机环境锁,应用配置校验先于所有修改。
|
||||||
|
3. 选择无外部数据库的 SiYuan 做独立部署试点,验证端口参数、反代、数据保留、多实例及失败恢复。
|
||||||
|
4. 建立面板只读版:多主机、多实例、状态、日志、配置脱敏、备份目录和任务核对。
|
||||||
|
5. 接入经过验收的部署/配置更新/备份动作;Gitea、Vaultwarden、Joplin 各自定义迁移与恢复策略。
|
||||||
|
6. 单独接入主机环境维护、Xray;按需扩展 Certd 与 Portainer。
|
||||||
|
|
||||||
|
升级应选择明确目标版本;预览可以查询新版本,但确认后的执行不能重新解析 latest。取消选择工具只表示本次不操作,不能转换为卸载。卸载默认保留数据,销毁数据是独立动作。
|
||||||
|
|
||||||
|
## 8. 验收边界
|
||||||
|
|
||||||
|
- 只拿单个工具发布包,在准备好的环境中完成部署;同机启动第二实例且端口、数据、路由不交叉。
|
||||||
|
- 面板或 SSH 断开后,远程任务可以核对,不自动重发有副作用的步骤。
|
||||||
|
- 部署单个工具时,其他工具容器 ID、挂载、配置与运行状态不发生非预期变化。
|
||||||
|
- backup/upgrade/restore/uninstall 共用锁;公共环境维护与应用变更互斥。
|
||||||
|
- 镜像拉取失败、磁盘不足、数据库未就绪、配置漂移、Nginx 校验失败,均有明确阶段和退出状态。
|
||||||
|
- 停机失败时禁止恢复数据库;备份失败禁止继续迁移;恢复失败明确进入需人工检查,不输出成功。
|
||||||
|
- 恢复既验证归档也实际恢复数据库/仓库,并核验应用功能;数据行数只是辅助证据。
|
||||||
|
- 保留现有数据路径、镜像和回滚资料;接管只建立基线,执行迁移另有计划。
|
||||||
|
|
||||||
|
## 9. UI 方向(用户已确认)
|
||||||
|
|
||||||
|
管理工具采用 Apple 风格,参考 macOS 原生工具的信息组织与交互节奏。此项是视觉与交互要求,不意味着必须改用 Swift、Electron 或其他桌面框架,也不等同于已选择本地或服务端部署形态。
|
||||||
|
|
||||||
|
- **整体视觉**:中性背景、克制的蓝色强调、清晰字体层级、充足留白、适度圆角与轻分隔。避免大面积渐变、过度毛玻璃和满屏同质卡片。
|
||||||
|
- **主框架**:侧边栏负责导航;顶部持续显示当前服务器、环境与应用实例;内容区按概览、应用、环境、任务和备份组织。切换目标后清除旧预览,避免操作到上一台服务器。
|
||||||
|
- **应用列表**:优先用紧凑列表展示名称、版本、状态和入口;详情页提供配置、日志、备份与升级等能力,未支持的动作不伪装成可用功能。
|
||||||
|
- **操作流程**:部署按“选择工具 → 配置 → 预检与变更预览 → 执行结果”推进。配置采用分组表单,长任务使用明确的阶段和可展开日志,保留任务恢复入口。
|
||||||
|
- **生产操作**:预览明确展示目标、影响的实例、预计停机及数据恢复范围;危险动作使用独立且具体的确认。颜色辅助提示,不能代替文字说明。
|
||||||
|
- **可访问性**:键盘焦点可见、文本对比度充足、状态不只依靠颜色;以系统字体栈适配 Windows/macOS,不分发 Apple 专有字体资源。
|
||||||
|
|
||||||
|
下一步设计应先明确页面结构与核心流程,再制作少量关键页面原型;当前尚未实现 UI。
|
||||||
|
|
||||||
|
## 10. 官方参考
|
||||||
|
|
||||||
|
- Compose 项目命名与隔离:https://docs.docker.com/compose/how-tos/project-name/
|
||||||
|
- Compose 默认网络:https://docs.docker.com/compose/how-tos/networking/
|
||||||
|
- 服务定义与 container_name:https://docs.docker.com/reference/compose-file/services/
|
||||||
|
- Docker daemon 权限:https://docs.docker.com/engine/security/
|
||||||
|
- SSH 保护 Docker 访问:https://docs.docker.com/engine/security/protect-access/
|
||||||
|
|
||||||
|
待确认的产品选择:管理工具是本地 SSH 面板还是服务器 Web 平台。本文优先推荐前者;选择后再形成实施规格与任务拆分。
|
||||||
|
|
||||||
|
## 11. 第二轮评估:收敛为可实施的结构
|
||||||
|
|
||||||
|
### 11.1 判断与纠偏
|
||||||
|
|
||||||
|
第一版对独立应用、共享环境、渐进接管的方向判断成立,但仍不足以作为直接开发的架构规格:执行责任不明确,容易让 Node、CLI、Shell 各自实现一遍操作;对多实例、外部数据库、双入口提供者等扩展的优先级偏高;UI 罗列功能而没有明确使用层级。
|
||||||
|
|
||||||
|
现代化的衡量标准是可复现、声明清楚、可观测、失败可判定、模块可独立测试,而不是增加框架、微服务或自动发现组件。本项目适合模块化单体管理工具 + 独立应用发布包 + 明确的环境模块。
|
||||||
|
|
||||||
|
### 11.2 独立部署的实际边界
|
||||||
|
|
||||||
|
首版必须保证每个工具能单独安装、停启、升级和备份,不依赖其他业务应用及面板在线。共享 Docker、入口和系统资源意味着仍有共同故障域,不能承诺同主机上的绝对故障隔离。
|
||||||
|
|
||||||
|
同主机多实例作为架构约束预留(独立身份、参数化资源),先在试点应用验证;不要求首版所有工具完成多实例 UI 和复杂迁移。外部数据库作为未来扩展,不在首版同时支持所有组合。
|
||||||
|
|
||||||
|
Gitea + 私有 MySQL、Joplin + 私有 PostgreSQL、RustDesk 两组件分别是一个部署单元。应用镜像更新与其数据库引擎升级是不同操作,不能因全栈 pull 顺带升级数据库。
|
||||||
|
|
||||||
|
应用包只声明所需环境与入口,不负责无条件安装 Nginx、签发证书或重启 Docker。入口模式至少在模型中区分“受管入口”“外部入口”“仅内网”,实际支持范围由适配器声明;应用自身要求公网 HTTPS 时仍需阻止不满足要求的配置。
|
||||||
|
|
||||||
|
### 11.3 入口方案与部署合理性
|
||||||
|
|
||||||
|
当前改造基线继续使用宿主 Nginx + Certbot:保留既有证书、站点和路径,集中到环境模块管理,避免与应用迁移同时更换网关。每台主机的监听地址和端口必须有明确归属,应用只申请自己的路由。
|
||||||
|
|
||||||
|
Caddy 的自动 HTTPS 可以减少新服务器的证书管理代码,是合理的后续选择;但它不自动解决既有 Nginx 路由、特殊长连接、证书接管和端口迁移。因此首版不同时实现两套网关,不强迫存量服务切换。入口操作接口应保持小而具体:检查、规划站点、应用站点、验证、移除本站点绑定。
|
||||||
|
|
||||||
|
容器化网关也可行,但不能把所有数据库加入公共代理网络。若未来改用容器网关,需单独设计前端网络与应用私有网络。当前回环端口 + 宿主代理在此规模下足够清楚。
|
||||||
|
|
||||||
|
### 11.4 管理工具采用模块化单体
|
||||||
|
|
||||||
|
候选仍为 React + TypeScript 前端及一个 Node 本地服务;技术版本在实施时锁定。首版不引入微服务、独立消息队列、常驻远端 Agent、插件市场或 Electron/Tauri 外壳。
|
||||||
|
|
||||||
|
面板内部按职责组织,而不是持续扩大一个 operations.ts:
|
||||||
|
|
||||||
|
```text
|
||||||
|
tools/ops-panel/src/
|
||||||
|
domain/ # Host、AppDefinition、Instance、Plan、Operation、Backup
|
||||||
|
application/ # 发现、接管、预览、执行、核对等用例
|
||||||
|
infrastructure/ # SSH/SFTP、存储、应用目录读取
|
||||||
|
http/ # 本地会话、校验、API、任务事件
|
||||||
|
web/
|
||||||
|
app/ # 路由、当前目标、整体布局
|
||||||
|
features/ # 服务器、应用实例、任务、设置
|
||||||
|
ui/ # 基础组件与设计 token
|
||||||
|
```
|
||||||
|
|
||||||
|
依赖方向为 UI/API → 用例 → 领域模型;用例通过小接口使用 SSH 和存储实现。领域模型不依赖 React、HTTP 或 SSH。不要为每个目录强建独立 npm 包;出现第二个真实消费者时再提取公共包。
|
||||||
|
|
||||||
|
主机档案与应用实例分开:一台主机有多个实例;应用定义描述模板和能力,实例记录具体路径、配置引用和现场身份。Plan 表示有时效的目标变更;Operation 表示一次执行;Backup 表示可验证的恢复资料。不要把这些全部塞进一个不断扩展的 ServerProfile。
|
||||||
|
|
||||||
|
### 11.5 单一执行入口,避免三套实现
|
||||||
|
|
||||||
|
前端负责输入、预览和展示;本地服务负责连接、调用用例与核对任务;远端版本化运行器负责现场检查、实例锁、执行步骤和结果记录。应用适配器提供真正有差异的备份、迁移、健康验证及恢复步骤。
|
||||||
|
|
||||||
|
GUI 与服务器上的 CLI 都调用同一个远端执行入口。旧 deploy.sh/backup.sh/upgrade.sh 在逐项验收后才改成兼容入口;不新增一套 TypeScript Shell 字符串流程与原脚本并行维护。
|
||||||
|
|
||||||
|
目标流程:发现/检查 → 生成计划 → 展示变更 → 确认 → 锁内重验 → 执行 → 验证 → 记录结果。远端运行器不成为常驻服务;每个任务保留固定版本代码、计划标识、实例身份、阶段日志和退出状态,SSH 断线后继续运行或进入明确的待核对状态。
|
||||||
|
|
||||||
|
任务结果必须区分:成功、失败且已恢复、失败需处理、结果待核对。不能将网络断开直接标记为失败或自动重试,也不能把“发出恢复命令”当作“已恢复”。中断恢复指核对现场和受控续做,不代表每个 Shell 步骤都能从中间续跑。
|
||||||
|
|
||||||
|
首版同主机写任务串行、跨主机可并行,读取不取写锁;CLI 使用相同锁约定。后续证明有必要时才细分实例并行锁与环境排他锁,避免过早设计复杂调度器。手工操作及 Portainer 不会自动遵守本工具的锁,因此执行前仍要核对配置与现场漂移。
|
||||||
|
|
||||||
|
### 11.6 单一事实来源与分发
|
||||||
|
|
||||||
|
应用目录内的清单是该工具字段、能力和运行入口的唯一声明来源;Compose 是容器编排来源。面板不再手写第二套服务列表或镜像默认值。旧实例以实际挂载与项目标签建立记录,不由新模板覆盖现场。
|
||||||
|
|
||||||
|
发布时产出独立、版本化的应用包,包含本应用定义、Compose/模板、适配步骤与固定版本的最小运行器。源代码共享,发布包自包含;更新管理面板不隐式更新服务器应用包。
|
||||||
|
|
||||||
|
版本需区分面板版本、应用包版本、运行器协议版本和上游镜像 digest。计划锁定具体包与镜像;包摘要只证明内容一致,若要防范供应链替换还需要可信发布来源或签名,不能将普通 checksum 当作来源认证。
|
||||||
|
|
||||||
|
本地保存主机档案和任务索引,远端任务结果是执行核对依据。首版可以用经串行化和原子写入的 JSON 文件存储小规模元数据;不为几台服务器强加本地数据库。通过 Store 接口隔离,确有查询、规模或事务需求时再换 SQLite。秘密不进入任务摘要、前端持久存储或普通日志。
|
||||||
|
|
||||||
|
### 11.7 Apple 风格落实到信息架构
|
||||||
|
|
||||||
|
全局导航保持简洁:服务器、任务、设置。进入服务器后显示其应用实例与独立的运行环境页;“添加应用”进入工具目录。实例详情再提供概览、配置、日志、备份与版本历史,避免把每个动作升级为全局导航项。
|
||||||
|
|
||||||
|
常用操作贴近当前对象:重启在实例工具栏,升级在版本信息附近,恢复从具体备份进入。预览使用稳定的页面或侧栏,只对真正需要确认的影响做确认,不层层叠弹窗。任务状态与阶段、日志分开,真实可计算时才展示百分比。
|
||||||
|
|
||||||
|
视觉采用中性色、统一字级和间距、小面积状态色、系统字体、清楚的选中与焦点状态;适量材质不妨碍日志阅读。Apple 风格是一致性、层级和反馈,不是简单地增加圆角或玻璃背景。离线显示最后更新时间与“状态未确认”,不把历史绿色状态当作当前健康。
|
||||||
|
|
||||||
|
### 11.8 缩小首版并调整交付方式
|
||||||
|
|
||||||
|
第一条闭环采用一个应用:应用包 → 独立 CLI → 面板识别 → 部署预览 → 执行 → 健康验证 → 备份/恢复演练。完成后接入第二个带数据库的应用,验证抽象是否成立,再批量扩展。
|
||||||
|
|
||||||
|
这比先重写全部脚本、再开发一个大型面板更容易发现协议设计问题。共享环境准备作为单独入口随试点一起提供,但 Docker 版本升级、证书提供者切换、所有工具多实例和外部数据库组合不同时纳入首版。
|
||||||
|
|
||||||
|
验收应覆盖:只分发单工具包、另一应用持续正常、断线后任务核对、停机失败拒绝恢复、备份失败阻止迁移、失败结果真实、已有实例原路径接管。UI 原型与领域接口可以先设计,但本节仍是评估后的建议,未开始应用代码实现。
|
||||||
|
|
||||||
|
补充官方参考:Caddy 自动 HTTPS https://caddyserver.com/docs/automatic-https 。其能力说明不构成对当前生产网关切换的建议。
|
||||||
@@ -0,0 +1,155 @@
|
|||||||
|
# 全新服务器部署与管理工具架构
|
||||||
|
|
||||||
|
日期:2026-09-24。状态:候选设计,待评审,不代表实现完成。
|
||||||
|
|
||||||
|
## 1. 已确认的约束
|
||||||
|
|
||||||
|
- 为全新服务器设计;不保留旧部署目录、脚本接口或迁移兼容层。
|
||||||
|
- 工具可以独立部署,公共环境可以共享;管理面板不是应用运行依赖。
|
||||||
|
- 优先考虑职责清晰、设计一致、可复现及数据保护。
|
||||||
|
- UI 采用 Apple 风格。
|
||||||
|
- claude-dev-stack 已废弃,不进入新设计。
|
||||||
|
|
||||||
|
本文取代旧审查报告中以渐进兼容为前提的建议。完整架构统一设计;工程实施仍需分模块验证,不等于维持两代部署体系。
|
||||||
|
|
||||||
|
## 2. 总体选择
|
||||||
|
|
||||||
|
推荐本地 React/TypeScript 管理面板 + Node 本地 API,通过 SSH 调用 Linux 上的单一运维执行器。执行器建议用 Go 编译为独立 CLI 二进制,以避免远端 Node 依赖和大规模 Shell 字符串拼接;代价是仓库需维护 TypeScript 与 Go 两种语言及对应测试。
|
||||||
|
|
||||||
|
公共环境包括 Docker Engine/Compose、宿主 Caddy、systemd 与必要的主机准备。Caddy 负责反向代理和自动 HTTPS;默认不再安装 Nginx + Certbot,不再由应用写续期 cron。DNS、证书签发可达性与云安全组仍需实际校验。
|
||||||
|
|
||||||
|
应用采用各自独立的 Compose 项目。Xray 采用 systemd 适配器,保持适合其运行方式的部署。没有总 Compose,也没有跨业务应用共享数据库。
|
||||||
|
|
||||||
|
本地面板形态是推荐选择,用户尚未明确指定运行位置;若选服务器 Web 平台,认证、凭据存储与多用户边界需要另行设计,但应用包和执行器协议保持相同。
|
||||||
|
|
||||||
|
## 3. 服务器布局与职责
|
||||||
|
|
||||||
|
```text
|
||||||
|
/usr/local/bin/deployctl 唯一 CLI 入口
|
||||||
|
/opt/server-deploy/packages/<id>/<ver>/ 只读、版本化应用包与校验信息
|
||||||
|
/etc/server-deploy/ 主机配置、实例声明、秘密引用
|
||||||
|
/var/lib/server-deploy/ 实例登记、任务状态、应用持久数据
|
||||||
|
/var/backups/server-deploy/<instance>/ 备份集与恢复清单
|
||||||
|
/etc/caddy/ 受管网关配置
|
||||||
|
```
|
||||||
|
|
||||||
|
具体子路径由实例 ID 生成并校验,不允许用户输入直接成为任意写入/删除目标。应用包与可变数据分离;更新包不覆盖数据目录。秘密文件使用严格文件权限并从备份与日志展示中脱敏。
|
||||||
|
|
||||||
|
Gitea + MySQL、Joplin + PostgreSQL、RustDesk hbbs + hbbr 分别作为一个应用单元。它们的数据库/配套组件有明确角色,应用镜像升级不默认更新数据库引擎。
|
||||||
|
|
||||||
|
Compose 项目名由实例 ID 固定生成,不使用固定 container_name。同一工具的不同实例拥有不同配置、挂载、端口、项目网络和备份目录;多实例是统一模型的一部分,不依靠复制脚本修改名称。
|
||||||
|
|
||||||
|
## 4. 入口与网络
|
||||||
|
|
||||||
|
选择宿主 systemd 管理的 Caddy,HTTP 应用只向宿主回环地址发布端口,Caddy 转发到对应端口。该方案使网关不依赖 Docker socket,也不需要将不同应用的数据库放进公共网络。
|
||||||
|
|
||||||
|
端口由执行器分配、登记并在执行前检查占用,不能每次启动随机变化。应用对外 URL、Compose 映射和网关上游都来自同一份实例配置。计划阶段的可用端口检查不消除抢占竞争,绑定失败仍必须作为可恢复错误处理。
|
||||||
|
|
||||||
|
每个应用的数据库只连接本项目网络,不发布宿主端口。Compose 项目隔离不是对同一 Docker daemon 的强安全隔离;有高权限 Docker 访问的主体仍可跨项目操作。
|
||||||
|
|
||||||
|
Gitea SSH、RustDesk TCP/UDP、Xray 是协议直达入口,单独声明监听地址、端口、协议及防火墙要求。Xray 与 HTTPS 网关不能同时绑定同一地址的 TCP 443;部署规划应拒绝冲突,或明确使用不同端口/IP/主机。
|
||||||
|
|
||||||
|
网关变更由唯一入口模块管理:收集已登记路由 → 生成候选配置 → 校验 → 切换并 reload → 访问验证。失败保留旧配置及诊断。证书状态和持久数据独立保存;删除一个应用不得删除其他应用的证书或共享网关。
|
||||||
|
|
||||||
|
Certd 保留为可选的独立工具,用于额外证书工作流;不自动管理 Caddy 已负责的证书。Portainer 同样可选,属于高权限容器管理入口,不是本工具的依赖。
|
||||||
|
|
||||||
|
## 5. 应用包与单一事实来源
|
||||||
|
|
||||||
|
每个应用包包含:
|
||||||
|
|
||||||
|
- 版本化清单:参数类型、秘密标记、组件角色、入口与数据位置、支持动作。
|
||||||
|
- Compose 模板或 systemd 模板。
|
||||||
|
- 版本锁:明确的上游版本和平台对应镜像 digest。
|
||||||
|
- 应用特有的备份、迁移与验证策略声明。
|
||||||
|
- 协议版本、文件清单和完整性摘要。
|
||||||
|
|
||||||
|
声明以可校验的数据表达,不能退化为任意远程命令字段。复杂恢复由内置且经过测试的应用适配器处理;不为简单需求创造通用工作流编程语言。
|
||||||
|
|
||||||
|
面板表单和能力显示读取应用清单,不手写第二份服务列表。应用包发布与面板发布解耦;执行器在预检中校验应用包协议兼容性。校验和验证内容一致性,可信分发还需可信来源/发布签名。
|
||||||
|
|
||||||
|
只分发一个应用包及兼容的 deployctl,就应能在准备好的主机上通过 CLI 部署该工具。公共环境初始化是独立命令,应用部署只检查环境或生成明确的环境准备计划,不隐式整机升级。
|
||||||
|
|
||||||
|
## 6. 唯一运维执行器
|
||||||
|
|
||||||
|
deployctl 同时服务本机 CLI 与 SSH 调用,承担现场检查、规划、互斥锁、执行、健康验证及结果记录。面板后端不再生成另一套安装/回滚脚本;GUI 与 CLI 的业务操作经过同一执行路径。
|
||||||
|
|
||||||
|
协议使用版本化 JSON 请求/响应及阶段事件,应用核心不依赖终端彩色输出。参数作为独立进程参数传递,避免经过 shell 二次解释;日志统一脱敏。高权限操作只在明确的主机维护或应用执行路径发生。
|
||||||
|
|
||||||
|
远程任务通过 systemd 管理的任务单元脱离 SSH 会话,执行固定版本程序。持久记录 operation ID、计划摘要、目标身份、运行阶段及退出状态。机器重启后先核对现场,不假设数据库迁移可以自动从任意位置续跑。
|
||||||
|
|
||||||
|
操作顺序为:inspect → plan → approve → apply → verify。apply 在锁内重验计划条件;过期、配置变化、版本变化时要求重新规划。同一主机写任务先统一串行,跨主机可并行,读取独立执行;后续若需要并发,可在保持接口不变的前提下细分锁范围。
|
||||||
|
|
||||||
|
结果明确区分成功、失败已恢复、失败需处理、状态待核对。恢复数据库前必须确认写入方已停止。镜像回退、配置恢复、数据恢复是不同动作,不承诺统一“回滚一切”。
|
||||||
|
|
||||||
|
备份包含完成标志、源实例、版本、内容摘要、一致性模式与恢复要求;归档通过校验不等于恢复演练通过。恢复覆盖升级后数据的风险必须出现在计划中。被恢复点引用的备份禁止自动清理。
|
||||||
|
|
||||||
|
## 7. 管理工具结构
|
||||||
|
|
||||||
|
管理工具为一个模块化单体,前后端随一个便携产品发布:
|
||||||
|
|
||||||
|
```text
|
||||||
|
apps/ops-panel/
|
||||||
|
src/domain/ 主机、定义、实例、计划、任务、备份
|
||||||
|
src/application/ 主机连接、应用目录、计划确认、任务核对
|
||||||
|
src/infrastructure/ SSH/SFTP、应用包仓库、存储
|
||||||
|
src/http/ 本地会话、API、任务事件
|
||||||
|
src/web/app/ 布局、路由与当前目标
|
||||||
|
src/web/features/ 服务器、实例、任务、设置
|
||||||
|
src/web/ui/ 组件和设计 token
|
||||||
|
```
|
||||||
|
|
||||||
|
前端不直接执行命令;HTTP 层不实现部署逻辑;连接模块不识别具体 Gitea 字段;领域模型不依赖 React 或 SSH。远端执行器是操作结果的权威来源,本地保存连接档案、用户偏好和任务索引。
|
||||||
|
|
||||||
|
本地持久化采用 SQLite,适合关联主机/实例/计划/任务、执行记录查询及事务写入;它是嵌入式文件,不引入额外数据库服务。具体驱动需结合便携 Node 运行时验证。SSH 优先使用 Agent/本地私钥引用,数据库中不保存私钥内容。
|
||||||
|
|
||||||
|
本地服务只监听回环地址,保留 Host/Origin、会话及 CSRF 校验。面板关闭不影响应用和远程任务;重新打开后按任务 ID 核对结果。
|
||||||
|
|
||||||
|
## 8. 新仓库目录
|
||||||
|
|
||||||
|
```text
|
||||||
|
server-deploy/
|
||||||
|
apps/ops-panel/ 本地管理产品
|
||||||
|
cmd/deployctl/ Go CLI 入口
|
||||||
|
internal/ 执行器内部模块
|
||||||
|
planner/
|
||||||
|
executor/
|
||||||
|
state/
|
||||||
|
runtime/ compose、systemd、gateway 适配
|
||||||
|
applications/ 必要的应用专有策略
|
||||||
|
catalog/ gitea、joplin、vaultwarden 等版本化应用包源
|
||||||
|
platform/ Docker、Caddy、主机初始化与维护定义
|
||||||
|
protocol/ 请求、计划、事件及清单的 schema
|
||||||
|
scripts/ 构建、发布、打包;不承载另一套运维业务
|
||||||
|
tests/ 协议一致性、隔离端到端、故障与恢复验收
|
||||||
|
docs/ 架构决策、操作说明和应用恢复边界
|
||||||
|
```
|
||||||
|
|
||||||
|
catalog 放部署对象,apps 放管理产品,platform 放公共环境,执行逻辑集中在 deployctl。旧顶层工具目录不作为兼容结构保留;实际重组在设计批准后的实施中完成,不能把设计文档新增解释为已完成重构。
|
||||||
|
|
||||||
|
## 9. Apple 风格与信息结构
|
||||||
|
|
||||||
|
主导航:服务器、任务、设置。服务器详情下分应用实例与运行环境;添加应用打开应用目录。实例详情提供概览、配置、日志、备份和版本历史,操作位于具体对象附近。
|
||||||
|
|
||||||
|
采用清晰留白、统一字级/圆角/间距、克制的状态色、系统字体、键盘操作、深浅色主题和可访问焦点。日志区域强调密度与可读性,不套用大面积玻璃材质。任务阶段与日志分离;离线状态显示最后核对时间。
|
||||||
|
|
||||||
|
部署以明确目标和差异预览为中心;默认值来自应用包,高级项逐步展开。需要数据恢复或停机的操作显示具体范围,不用重复弹窗代替设计清晰的操作流程。
|
||||||
|
|
||||||
|
## 10. 设计质量的验收标准
|
||||||
|
|
||||||
|
- 新增常规 Compose 应用主要增加 catalog 定义,而非修改 UI/连接/任务核心。
|
||||||
|
- GUI 与 CLI 对同一请求产生相同现场计划和执行结果。
|
||||||
|
- 应用 A 升级不变更应用 B 的配置、数据库、数据或镜像。
|
||||||
|
- 同工具多实例的项目名、端口、数据和入口从配置生成,无手工改源码。
|
||||||
|
- 面板失联、SSH 中断、磁盘不足、镜像不可用、停机失败和恢复失败均有可核对结果。
|
||||||
|
- Caddy、数据库及应用数据有明确所有者与备份边界。
|
||||||
|
- 模板、协议、执行器和界面有自动测试;真实隔离服务器覆盖部署、升级、备份及恢复。
|
||||||
|
|
||||||
|
共享主机仍有共同故障域。本方案提供生命周期和资源归属隔离,不声称达到多租户强隔离或多节点高可用。
|
||||||
|
|
||||||
|
## 官方依据
|
||||||
|
|
||||||
|
- Caddy 自动 HTTPS:https://caddyserver.com/docs/automatic-https
|
||||||
|
- Compose 项目身份:https://docs.docker.com/compose/how-tos/project-name/
|
||||||
|
- systemd transient services:https://www.freedesktop.org/software/systemd/man/latest/systemd-run.html
|
||||||
|
|
||||||
|
待评审:本地面板运行形态、Go 单二进制执行器以及统一 Caddy 入口均为本方案的明确推荐,尚未视为用户逐项批准的实现选型。
|
||||||
@@ -0,0 +1,299 @@
|
|||||||
|
# 新服务器部署与管理工具:完整优化方案
|
||||||
|
|
||||||
|
日期:2026-09-25。状态:用户已批准开始实施;选定 Traefik、本地面板、Go 执行器。不代表已实现或经过服务器验收。
|
||||||
|
|
||||||
|
## 1. 范围与决策状态
|
||||||
|
|
||||||
|
已确认:为新服务器设计;不做旧服务器迁移或旧脚本兼容;每个工具独立部署;公共环境可共享;Apple 风格界面;claude-dev-stack 不进入新体系。
|
||||||
|
|
||||||
|
实施默认仅在本地开发和隔离测试环境进行,不连接生产服务器,不自动提交代码。之后用户已指定新服务器并授权只读初检,记录见 target-server-preflight 文档;这不代表已授权任意写操作。2026-09-25 用户要求跨设备拉取接续,本次据此整理并提交/推送新架构交接成果。此前已删除的 claude-dev-stack/.env 保持现状。
|
||||||
|
|
||||||
|
用户在完整方案后明确要求“按照方案,开始”,因此以本方案的 Traefik + 本地面板 + Go 执行器作为实施基线。2026-09-24 的 Caddy 方案保留为历史讨论,不与本方案同时实施。
|
||||||
|
|
||||||
|
推荐组合:本地 React/TypeScript 面板 + Node API + SQLite;SSH 调用 Go deployctl;远端 Docker Compose + Traefik + systemd;应用包独立分发。首版面向单管理员、可信主机,不承诺多租户强隔离或集群高可用。
|
||||||
|
|
||||||
|
## 2. 现有项目的结构性问题
|
||||||
|
|
||||||
|
依据 2026-09-24 源码审查及本轮目录核对:
|
||||||
|
|
||||||
|
- base 混合函数库和主机安装程序;应用部署可触发整机升级、Docker 配置覆盖与重启。
|
||||||
|
- 应用运行单元基本独立,但脚本依赖相邻 base 目录;固定容器名和端口妨碍多实例。
|
||||||
|
- 网关、证书、定时任务由多个应用脚本各自维护,资源归属不清。
|
||||||
|
- 部分备份失败被忽略,一致性、完成标记、恢复演练与保留策略不足。
|
||||||
|
- Gitea 的健康判断、停止写入后恢复、回滚结果判断等问题,不能作为新执行后端继续继承。
|
||||||
|
- 版本浮动、文档漂移、实际秘密进入源码或历史,影响可复现与发布安全。
|
||||||
|
|
||||||
|
旧脚本只作为功能和故障案例来源,不包装成新管理工具的执行后端。完整重构允许按模块开发验证,但不建立长期双轨兼容系统。
|
||||||
|
|
||||||
|
## 3. 四个明确的职责层
|
||||||
|
|
||||||
|
### 3.1 控制界面
|
||||||
|
|
||||||
|
本地面板负责主机连接、应用选择、参数输入、差异展示、任务观察。面板停止不影响应用,也不终止已经提交的远端任务。
|
||||||
|
|
||||||
|
### 3.2 运维执行器
|
||||||
|
|
||||||
|
deployctl 是唯一业务执行入口:检查、计划、应用、备份、恢复、升级、卸载、任务核对。CLI 与面板走相同路径,不各写一套脚本。
|
||||||
|
|
||||||
|
建议 Go:远端独立二进制,无需安装 Node;进程调用、文件操作和状态机更易测试。代价是维护 Go 与 TypeScript 两套构建链,不以减少语言数量牺牲远端可靠性。
|
||||||
|
|
||||||
|
### 3.3 公共环境
|
||||||
|
|
||||||
|
Docker Engine/Compose、Traefik、受限 Docker API 代理、systemd 任务与计时器、必要防火墙规则。环境初始化和维护是独立操作,必须展示整机影响。
|
||||||
|
|
||||||
|
### 3.4 应用实例
|
||||||
|
|
||||||
|
各实例独立编排、配置、数据、备份、版本记录。私有数据库属于应用,不提升为跨应用共享环境数据库。
|
||||||
|
|
||||||
|
## 4. 网关、网络与证书
|
||||||
|
|
||||||
|
### 4.1 Traefik 候选的取舍
|
||||||
|
|
||||||
|
应用路由从包内声明生成 Compose 标签;Traefik 使用 Docker provider 动态发现。部署工具校验域名归属和标签,但不再另写一份同域名的文件路由。普通容器 HTTP 服务不发布宿主端口。
|
||||||
|
|
||||||
|
采用此方案的理由是应用包可携带完整入口声明,而非 Traefik 比 Caddy 更现代。若希望网关不访问 Docker API,则采用此前宿主 Caddy + 回环上游方案更简单。首版只实现其中一种,不做多网关切换平台。
|
||||||
|
|
||||||
|
Traefik 是独立环境 Compose 项目,静态配置变化或自身升级作为主机级维护;动态应用路由不要求重启整个网关。主机和网关故障仍可能影响全部应用 HTTP 入口。
|
||||||
|
|
||||||
|
### 4.2 首版网络边界
|
||||||
|
|
||||||
|
- 一张受控共享入口网络,仅连接 Traefik 与需要公开 HTTP 的应用组件。
|
||||||
|
- 每个应用另有私有后端网络,数据库只连接本应用后端网络、不发布宿主端口。
|
||||||
|
- 一张专用 Docker API 管理网络,仅连接 Traefik 与 socket proxy,不发布宿主端口。
|
||||||
|
- Traefik 设置 exposedByDefault=false,并过滤受管应用标签;显式指定上游端口和入口网络。
|
||||||
|
- 路由、服务名称包含实例 ID,域名绑定有唯一性校验,避免跨应用覆盖。
|
||||||
|
|
||||||
|
共享入口网络上的应用可以相互连接:这是首版同一管理员、同一信任域的明确取舍,不是强隔离。若未来运行不可信应用,应使用独立主机或专门设计的隔离网络,不把 Compose 项目名当安全边界。
|
||||||
|
|
||||||
|
socket proxy 必须按实际需要限制 Docker API 路径和方法,验收确认拒绝容器创建、exec、删除等写操作。只读挂载 socket 不是 API 授权。允许读取的容器元数据仍可能泄露秘密,因此凭据尽量使用文件挂载而非标签或环境变量。
|
||||||
|
|
||||||
|
代理本身仍持有高权限 socket,必须固定版本、保护网络、审查供应链;不能描述成彻底消除了 Docker 权限风险。
|
||||||
|
|
||||||
|
### 4.3 证书所有权
|
||||||
|
|
||||||
|
Traefik 内置 ACME 管理入口证书:自动申请、续期和加载。默认普通子域名证书,用户预先配置 A/AAAA;部署前检查解析及验证端口,不默认索取 DNS API 密钥。
|
||||||
|
|
||||||
|
泛域名或不能使用公网端口验证时,单独启用 DNS challenge;凭据采用支持的最小权限策略。DNS 验证 TXT 与网站 A/AAAA 解析是两回事。
|
||||||
|
|
||||||
|
acme.json 放在独立持久目录、严格权限保护、加密备份;不进入 Git/日志/普通导出。单实例管理该存储,不能靠多个 Traefik 共享该文件实现高可用。
|
||||||
|
|
||||||
|
Certd 可单独部署处理额外证书用途,但不同时管理 Traefik 的证书。应用删除只移除其路由,不随意编辑共享 acme.json。
|
||||||
|
|
||||||
|
网关管理接口默认不对公网开放,通过 SSH 隧道访问。证书监控检查实际 TLS 端点而不仅是本地文件时间。
|
||||||
|
|
||||||
|
### 4.4 非 HTTP 入口
|
||||||
|
|
||||||
|
Gitea SSH、RustDesk TCP/UDP、Xray 独立声明端口和防火墙要求,不强制套进 HTTP 代理。Xray 保持 systemd,默认单独主机或非冲突端口;同一 IP 的 TCP 443 冲突必须拒绝,不默认设计复杂协议复用。
|
||||||
|
|
||||||
|
## 5. 应用包与实例模型
|
||||||
|
|
||||||
|
AppDefinition 表示应用种类,PackageRelease 表示部署包版本,Instance 表示某主机上的实际安装,不能混为一个“服务”。
|
||||||
|
|
||||||
|
应用包包含 manifest、参数 schema、Compose/systemd 模板、上游版本锁、健康规则、数据位置、一致性与升级策略声明、文件摘要及可信发布信息。
|
||||||
|
|
||||||
|
规则:
|
||||||
|
|
||||||
|
- 不使用固定 container_name;Compose 项目名、资源名和路径由稳定实例 ID 生成。
|
||||||
|
- 包版本、程序版本、数据库版本、镜像 digest 分别记录;发布时确认目标 CPU 架构支持。
|
||||||
|
- 禁止 latest 作为可复现部署依据;升级前解析并锁定具体目标。
|
||||||
|
- 数据目录不在包目录内;应用程序更新不覆盖持久数据。
|
||||||
|
- UI 字段和可用动作从 schema/能力声明生成,不维护第二套应用清单。
|
||||||
|
- 常规应用新增包即可接入;复杂备份或迁移需要内置适配器及测试,不允许包声明任意 root Shell 工作流。
|
||||||
|
- 包只从受信来源发布;摘要是完整性检查,不是身份认证。
|
||||||
|
- 面板、执行器、包分别发布,显式校验协议兼容,不隐式互相升级。
|
||||||
|
|
||||||
|
准备好公共环境后,独立应用包 + deployctl 即可通过 CLI 部署,不依赖整个仓库或面板。
|
||||||
|
|
||||||
|
## 6. 现有工具的目标归属
|
||||||
|
|
||||||
|
- Gitea:应用 + 私有 MySQL 一个部署单元;数据库大版本升级单列维护计划。
|
||||||
|
- Joplin:应用 + 私有 PostgreSQL 一个部署单元。
|
||||||
|
- Vaultwarden:独立应用,数据库类型在包中明确;SQLite 不可直接在线复制当作可靠备份。
|
||||||
|
- SiYuan:独立应用,数据目录与客户端写入的一致性需要专门验证。
|
||||||
|
- RustDesk:hbbs/hbbr 同一应用单元,保存身份密钥,验证直连/中继链路。
|
||||||
|
- Certd:可选证书工具,不是公共入口证书的第二管理者。
|
||||||
|
- Portainer:可选高权限管理入口;其人工操作会造成漂移,不能与本工具并发修改同一实例。
|
||||||
|
- Xray:systemd 应用适配器;内核调优、整机网络修改属于环境维护。
|
||||||
|
- claude:开发机辅助脚本,归类到 extras/workstation,不混进服务器应用目录。
|
||||||
|
- claude-dev-stack:排除,不恢复、不保留适配器。
|
||||||
|
|
||||||
|
支持部署不自动等于支持安全升级与恢复;每项能力完成验收后才在面板开放。
|
||||||
|
|
||||||
|
## 7. 操作协议与任务状态
|
||||||
|
|
||||||
|
核心流程:inspect → plan → 确认 → 锁内复核 → apply → verify。
|
||||||
|
|
||||||
|
计划绑定主机身份、实例 ID、当前状态摘要、目标包/digest、端口和域名、停机与数据影响、备份要求、失败处置、有效期和计划哈希。只读检查不修改配置或秘密。
|
||||||
|
|
||||||
|
apply 拒绝过期或发生漂移的计划。operation ID / 幂等键在远端去重,SSH 断开后先查询原任务,不自动重做升级或恢复。
|
||||||
|
|
||||||
|
任务由 systemd 服务运行,不依附 SSH 会话;记录固定执行器版本、事件序号、阶段检查点和最终结果。重启后先核对现场,禁止从任意数据库迁移步骤盲目续跑。
|
||||||
|
|
||||||
|
首版同一主机写操作串行,跨主机可并行。自动备份、CLI 和 UI 使用同一把锁。外部人工操作无法被锁约束,必须做现场复核和漂移提示。
|
||||||
|
|
||||||
|
状态至少包括排队、执行中、成功、失败已恢复、失败需处理、结果待核对、已取消。取消是受控请求,仅在安全检查点停止;不能在恢复数据库时强杀进程然后宣告“取消成功”。
|
||||||
|
|
||||||
|
SSH 启动命令使用固定入口,JSON 经 stdin/受保护文件传递,不把用户参数拼进远程 shell。子进程使用参数数组;目录校验包括归属、规范化路径和符号链接逃逸。
|
||||||
|
|
||||||
|
远端写状态是权威,本地 SQLite 是连接档案、任务索引与视图缓存。协议含版本号,日志统一脱敏,不记录秘密请求体。
|
||||||
|
|
||||||
|
## 8. 数据保护、备份和恢复
|
||||||
|
|
||||||
|
### 8.1 备份契约
|
||||||
|
|
||||||
|
先确认源实例和数据范围、可用空间与恢复所需版本;按适配器冻结全部写入方,或使用明确可证明的一致性机制。
|
||||||
|
|
||||||
|
- Gitea:统一考虑数据库、Git 仓库、LFS、附件、配置与秘密;Git SSH/后台任务同样是写入方。仅做数据库 dump 不算完整备份。
|
||||||
|
- Joplin:数据库及配置;如支持额外外部存储,必须纳入范围。
|
||||||
|
- Vaultwarden:SQLite 使用受支持备份方式或停机备份,同时覆盖附件等文件。
|
||||||
|
- SiYuan、RustDesk、Certd、Portainer:按各自数据格式设计停写和导出策略,不能套一个在线 tar。
|
||||||
|
|
||||||
|
临时输出 → 检查导出退出码和内容 → 生成 manifest/摘要 → 原子标记完成。失败备份不可选为恢复点,不触发成功备份清理。
|
||||||
|
|
||||||
|
manifest 记录源身份、时间、包与应用/数据库版本、数据范围、一致性方式、工具版本和文件摘要。区分“生成成功”“完整性检查通过”“实际恢复验证通过”。
|
||||||
|
|
||||||
|
### 8.2 保留与异地
|
||||||
|
|
||||||
|
建议初始策略:每日一次、保留 7 个日备份和 4 个周备份;按数据规模、变更频率与停机窗口调整。每日一次意味着潜在丢失接近一天的数据,不等于零数据损失。
|
||||||
|
|
||||||
|
至少有一份加密异机/对象存储副本;本机备份不防磁盘损坏和主机丢失。备份目标和凭据由用户配置,禁止自动采购或上传到未指定外部服务。
|
||||||
|
|
||||||
|
在确认可用替代恢复点前不得清除最后可恢复备份;升级恢复点、正在恢复或被引用备份禁止自动清理。加密密钥应有独立安全副本,否则备份可能无法恢复。
|
||||||
|
|
||||||
|
定时任务运行在远端,不依赖面板常开。远端通知使用用户配置的通道;没有通道时不能声称面板关闭仍会主动提醒。
|
||||||
|
|
||||||
|
### 8.3 恢复契约
|
||||||
|
|
||||||
|
默认先恢复到隔离实例验证,避免直接覆盖当前数据。原地恢复必须明确展示会覆盖备份时间之后的新增数据,并生成当前状态救援备份。
|
||||||
|
|
||||||
|
恢复前确认全部写入方确已停止;停止失败立即中止。完整恢复进入干净目标,禁止用合并解压假装精确还原。校验权限、秘密、版本兼容后启动并执行功能验证。
|
||||||
|
|
||||||
|
恢复后的邮件、Webhook、定时任务和公网入口在演练环境默认禁用,防止恢复测试触发真实外部操作。
|
||||||
|
|
||||||
|
归档摘要通过只能证明文件未变化;数据库行数不能证明业务完整。RTO 通过实际恢复计时测量后给出,不提前承诺。
|
||||||
|
|
||||||
|
## 9. 升级、回退与卸载
|
||||||
|
|
||||||
|
升级前检查受支持升级路径和迁移要求,下载并验证目标包/镜像,预估空间与停机时间,再备份、迁移、启动、验证。不能先停业务才发现镜像不可下载。
|
||||||
|
|
||||||
|
健康判定包含进程/容器状态、应用就绪、预期 HTTP 状态与必要功能探测;500 不是健康,200 登录页面也不等于所有功能正常。
|
||||||
|
|
||||||
|
Gitea 验收需覆盖登录、读取仓库、clone、受控测试仓库 push、附件/LFS(启用时)及重启后数据。其他应用按自身业务定义探测,不复用单一 HTTP 判断。
|
||||||
|
|
||||||
|
镜像回退、配置还原、数据恢复是三个独立能力。发生不可逆数据库迁移后不能自动降级旧镜像;自动补偿仅适用于适配器明确支持的步骤。无法证明可恢复时保持诊断并请求人工处理,不输出“已回滚成功”。
|
||||||
|
|
||||||
|
默认卸载只停止并移除实例运行资源与自有路由,保留数据和备份。清除数据是独立高风险动作,要求目标实例确认、路径归属验证及备份检查;不删除共享网关、其他实例、宿主证书或全局卷。
|
||||||
|
|
||||||
|
## 10. 管理工具架构与 UI
|
||||||
|
|
||||||
|
模块化单体,不拆微服务。React/TypeScript + Vite 前端,Node 本地 API,SQLite 元数据;具体运行时/依赖版本在实施时选定并锁定,完成 Windows/macOS 便携包验证。
|
||||||
|
|
||||||
|
交付形式为本地服务配合浏览器页面,不是 Electron/Tauri 桌面安装 App,也不是公网管理网站。
|
||||||
|
便携目标为解压后启动、随包携带运行时;启动器、端口/数据目录和退出行为尚待实现,不能当成现有能力。
|
||||||
|
|
||||||
|
模块边界:
|
||||||
|
|
||||||
|
- domain:Host、AppDefinition、PackageRelease、Instance、Plan、Operation、Backup、RouteBinding。
|
||||||
|
- application:连接、计划确认、任务核对、备份查询等用例,不执行任意 shell。
|
||||||
|
- infrastructure:SSH/SFTP、SQLite、包仓库、凭据引用。
|
||||||
|
- http:会话、验证、API、事件传输,不包含部署策略。
|
||||||
|
- web:页面、表单、任务状态和设计系统。
|
||||||
|
|
||||||
|
SSH 主机指纹首次信任需确认,变化阻止连接;优先 Agent 或密钥文件引用,不保存私钥内容。API 只监听回环,校验 Host/Origin、会话与 CSRF;本地服务也不是无认证裸 API。
|
||||||
|
|
||||||
|
参考 docker-ops-panel 的连接验证、会话保护、远程任务追踪和便携分发方式,不复制其业务固定服务列表、单项目连接模型或巨型操作模块。
|
||||||
|
|
||||||
|
Apple 风格信息架构:全局服务器/任务/设置;服务器内应用/环境/入口与证书;实例内概览/配置/日志/备份/版本。添加应用是目录与参数流程,而不是额外复杂全局导航。
|
||||||
|
|
||||||
|
视觉以系统字体、留白、克制色彩、统一间距和圆角、浅深色、键盘焦点为主;日志使用高可读密度,不做满屏玻璃效果。不可计算的任务不伪造百分比;离线显示最后核对时间。
|
||||||
|
|
||||||
|
部署和恢复展示真实目标、差异、停机与数据影响。危险确认只用于真正有风险的动作;不靠频繁弹窗代替清楚的权限和资源归属。
|
||||||
|
|
||||||
|
## 11. 源码与服务器目录
|
||||||
|
|
||||||
|
源码:
|
||||||
|
|
||||||
|
```text
|
||||||
|
apps/ops-panel/ 本地管理产品
|
||||||
|
cmd/deployctl/ Go CLI
|
||||||
|
internal/
|
||||||
|
planner/ 计划、前提与差异
|
||||||
|
executor/ 状态机、锁、补偿
|
||||||
|
state/ 远端登记、事件与恢复核对
|
||||||
|
runtime/ Compose、systemd、网关适配
|
||||||
|
applications/ 应用专用备份、升级和验证
|
||||||
|
security/ 路径、秘密、包验证
|
||||||
|
catalog/<app>/ 清单、模板、版本锁与应用测试资料
|
||||||
|
platform/ 环境定义和模板
|
||||||
|
protocol/ JSON schema 和协议兼容测试
|
||||||
|
extras/workstation/ 非服务器辅助工具
|
||||||
|
scripts/ 构建发布;不放第二套运维逻辑
|
||||||
|
tests/ 集成、故障注入、恢复与端到端测试
|
||||||
|
docs/ 架构决策、操作手册、恢复手册
|
||||||
|
```
|
||||||
|
|
||||||
|
远端:
|
||||||
|
|
||||||
|
```text
|
||||||
|
/usr/local/bin/deployctl
|
||||||
|
/opt/server-deploy/packages/<app>/<version>/
|
||||||
|
/etc/server-deploy/instances/<id>/
|
||||||
|
/etc/server-deploy/secrets/<id>/
|
||||||
|
/var/lib/server-deploy/instances/<id>/
|
||||||
|
/var/lib/server-deploy/operations/<id>/
|
||||||
|
/var/lib/server-deploy/platform/traefik/
|
||||||
|
/var/backups/server-deploy/<instance>/<backup>/
|
||||||
|
```
|
||||||
|
|
||||||
|
配置、包、秘密、数据、备份、日志分别管理权限和保留策略。不把整个仓库上传到服务器。应用进程不应可修改执行器、包信任配置或操作记录。
|
||||||
|
|
||||||
|
## 12. 主机安全与供应链
|
||||||
|
|
||||||
|
首版只承诺一套经过端到端验收的 Linux LTS 系统和 CPU 架构,其他组合通过兼容矩阵扩展,不声称支持所有发行版。
|
||||||
|
|
||||||
|
bootstrap 是经过审查的显式管理员操作,安装可信、固定版本的执行器和环境。不把任意用户可写路径的程序加入无限 sudo;若后续实现受限运维账户,需要验证包、路径和模板不能绕过权限边界。
|
||||||
|
|
||||||
|
整机升级、Docker 重启、sysctl 和 SSH/防火墙修改单独计划。防火墙操作保留现有 SSH 通路;验证云安全组与 Docker 实际端口暴露,不能只看主机防火墙配置。
|
||||||
|
|
||||||
|
秘密不进 Git/前端存储/标签/日志;历史已泄露的有效凭据需要轮换,删除文件不等于消除泄露。密钥轮换和历史清理另行授权执行。
|
||||||
|
|
||||||
|
发布采用白名单打包、依赖锁定、构建校验、秘密扫描、受信签名或分发通道。容器尽可能降权、限制资源并配置日志轮转;具体只读根目录/能力限制按应用实测,不盲目统一启用。
|
||||||
|
|
||||||
|
## 13. 实施工作包与交付顺序
|
||||||
|
|
||||||
|
这是完整新架构的工程顺序,不是存量渐进迁移:
|
||||||
|
|
||||||
|
1. 冻结三项关键选择,确定协议、清单、实例和备份契约;补全威胁模型及验收用例。
|
||||||
|
2. 构建 deployctl、任务存储、锁、计划复核和可信分发;验证断线、重启和幂等。
|
||||||
|
3. 实现公共环境及选定网关;验证网络权限、解析、证书续期与入口隔离。
|
||||||
|
4. 用 Gitea 完成 CLI 部署—备份—升级—隔离恢复闭环,用 Joplin 验证第二种数据库适配。
|
||||||
|
5. 建立本地面板与 Apple 风格设计系统,将已验证操作接入,不把 UI 作为执行逻辑试验场。
|
||||||
|
6. 接入其余应用和 Xray,能力按实际验收逐项开放。
|
||||||
|
7. 完成便携打包、灾难恢复、文档和全新服务器安装验收;删除旧部署实现并切换根 README,不保留旧兼容入口。
|
||||||
|
|
||||||
|
每个工作包可单独测试和审查,协议约束贯穿始终。不是先把所有旧脚本复制进新目录再统一包装。
|
||||||
|
|
||||||
|
## 14. 发布验收门槛
|
||||||
|
|
||||||
|
- 单个应用包在干净主机上可独立安装;不依赖面板或相邻仓库目录。
|
||||||
|
- 同应用两个实例并存,名称、端口、数据、域名、备份不冲突。
|
||||||
|
- 应用 A 升级不改应用 B 数据和编排;共同环境维护明确报告共享影响。
|
||||||
|
- 无公网数据库端口、Docker API 或未保护的网关管理接口;socket proxy 写请求确实被拒绝。
|
||||||
|
- ACME 测试环境验证申请/续期路径,实际 TLS 握手验证加载结果;不靠频繁生产签发做测试。
|
||||||
|
- 注入磁盘不足、备份失败、停止失败、镜像拉取失败、数据库迁移失败、恢复失败,结果真实且可核对。
|
||||||
|
- SSH 中断/面板关闭/主机重启后不会重复执行升级,任务能重新关联或明确标记待处理。
|
||||||
|
- 每种声明支持恢复的应用都有真实恢复演练记录,检查业务数据和身份密钥,不只检查归档。
|
||||||
|
- 默认卸载保留数据,清理只涉及明确自有目标;操作记录和安全测试覆盖路径逃逸。
|
||||||
|
- 本地界面访问控制、SSH 指纹、日志脱敏、包完整性与来源验证有自动测试。
|
||||||
|
- 前端单元/E2E、协议一致性、Go 单元/集成、隔离 Linux 主机验收全部通过,并区分未测项。
|
||||||
|
|
||||||
|
不能承诺“绝不丢数据”或“所有升级自动回滚”。可以承诺的工程目标是:危险动作有明确边界、恢复点真实可用、失败不伪装成功、每项能力有可复核验收证据。
|
||||||
|
|
||||||
|
## 15. 官方参考
|
||||||
|
|
||||||
|
- [Traefik Docker provider 与安全边界](https://doc.traefik.io/traefik/reference/install-configuration/providers/docker/)
|
||||||
|
- [Traefik ACME 与自动续期](https://doc.traefik.io/traefik/reference/install-configuration/tls/certificate-resolvers/acme/)
|
||||||
|
- [Compose 项目身份](https://docs.docker.com/compose/how-tos/project-name/)
|
||||||
|
- [Caddy 候选的自动 HTTPS 能力](https://caddyserver.com/docs/automatic-https)
|
||||||
|
|
||||||
|
这些文档支持产品能力说明;本项目的目录、权限、流程和测试门槛是设计建议,不是已经完成的实现。
|
||||||
@@ -0,0 +1,35 @@
|
|||||||
|
# 新服务器只读初检
|
||||||
|
|
||||||
|
目标:`root@121.199.168.54`。用户明确指定此服务器,并核对了控制台主机指纹。
|
||||||
|
本记录是初步现场观察,不是部署验收,不保证主机上没有其他数据。
|
||||||
|
|
||||||
|
## 身份与范围
|
||||||
|
|
||||||
|
- ED25519:`SHA256:7sEaEMqm0ZqfIRkTIQ/7a3wgMfo9mS0JcDBWWmDvBf8`。
|
||||||
|
- 已在本机 SSH known_hosts 记录信任;后续连接使用严格主机校验。
|
||||||
|
- 使用已有密钥登录成功,未读取或输出私钥。
|
||||||
|
- 远端只执行系统、服务、端口、软件包和目录元数据查询。
|
||||||
|
- 未安装软件、上传执行器、修改防火墙、重启服务、迁移或删除数据。
|
||||||
|
|
||||||
|
## 实际观察
|
||||||
|
|
||||||
|
- Ubuntu 26.04.1 LTS,Linux 7.0.0-31-generic,x86_64。
|
||||||
|
- 2 个逻辑 CPU,内存 3622 MiB,无 swap。
|
||||||
|
- 根分区 ext4,约 79 GiB,总可用约 73 GiB,inode 使用约 3%。
|
||||||
|
- systemd 状态 running,没有 failed 单元;NTP 已同步,Asia/Shanghai 时区。
|
||||||
|
- 监听包含 SSH 22、本机 DNS、DHCP 与时间服务;80/443 未监听。
|
||||||
|
- 没有找到 Docker 命令;dpkg 未发现 docker-ce、docker.io、containerd.io、
|
||||||
|
containerd、docker-compose-plugin、nginx 或 certbot 软件包。
|
||||||
|
- UFW inactive;nft 查询未输出规则。云安全组不属于这些主机查询的覆盖范围。
|
||||||
|
- /opt、/srv 存在,未发现一级子目录;常见 Docker、containerd、server-deploy、
|
||||||
|
Nginx、Let's Encrypt 数据/配置根目录未发现。未扫描全盘或读取应用文件。
|
||||||
|
|
||||||
|
## 部署前仍需完成
|
||||||
|
|
||||||
|
1. 校验 Ubuntu 26.04 的 Docker 官方支持及安装来源,固定并验证版本。
|
||||||
|
2. 完成 Compose 安全策略、执行器运行前检查、环境安装和失败处理实现。
|
||||||
|
3. 确定首批应用、域名、ACME 联系邮箱及备份目标。
|
||||||
|
4. 核对域名解析和云安全组;主机未监听不等于公网可达。
|
||||||
|
5. 正式写操作前重新检查现场,展示并确认环境安装影响。
|
||||||
|
|
||||||
|
本次没有对旧服务器 121.43.180.59 执行任何操作。
|
||||||
@@ -0,0 +1,99 @@
|
|||||||
|
# 新设备开发与验证
|
||||||
|
|
||||||
|
先看根目录 [HANDOFF.md](../HANDOFF.md)。以下命令在仓库根目录执行,不依赖原开发者的磁盘路径。
|
||||||
|
这里只构建/测试新执行器,不调用旧部署脚本,不连接服务器,不安装 Docker。
|
||||||
|
|
||||||
|
## 环境前提
|
||||||
|
|
||||||
|
- Git 和 SSH 客户端:克隆工作分支并访问仓库。服务器 SSH 权限与仓库权限是两件事。
|
||||||
|
- Go:`go.mod` 要求 1.26.0 或更新版本;本次基线实际使用 Go 1.27.1。
|
||||||
|
- Node.js:仅用于无第三方依赖的 CLI 冒烟测试;本次 Windows 基线为 v24.19.0。
|
||||||
|
尚无面板 package.json,不需要 `npm install`。
|
||||||
|
- Windows:PowerShell;Linux:Bash。Linux 专有行为使用 Ubuntu/WSL 验证。
|
||||||
|
- Linux `go test -race` 需要可用的 C 编译器/CGO;检查 `go env CGO_ENABLED` 和 `cc --version`。
|
||||||
|
- 在线仓库测试另外需要 curl、GnuPG 的 `/usr/bin/gpg` 与 `/usr/bin/gpgv`、Python 3、awk、tar 和常规核心工具。
|
||||||
|
|
||||||
|
从组织认可的软件渠道或 [Go 官方下载](https://go.dev/dl/) / [Node 官方下载](https://nodejs.org/en/download)
|
||||||
|
获取工具链;核对下载摘要。不要复制旧机器临时目录里的未知二进制。
|
||||||
|
上述版本是实测基线,不表示所有更新版本已验收。macOS 便携产品尚未验证;完整 Linux 专有测试需 Linux。
|
||||||
|
|
||||||
|
## Windows:基础离线检查
|
||||||
|
|
||||||
|
确认 `go` 和 `node` 已在 PATH。新设备无需沿用原设备的 GOCACHE、临时 Go 目录或磁盘盘符。
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
go version
|
||||||
|
node --version
|
||||||
|
go test ./... -count=1
|
||||||
|
if ($LASTEXITCODE -ne 0) { throw 'Go tests failed' }
|
||||||
|
go vet ./...
|
||||||
|
if ($LASTEXITCODE -ne 0) { throw 'Go vet failed' }
|
||||||
|
go build -o dist/deployctl.exe ./cmd/deployctl
|
||||||
|
if ($LASTEXITCODE -ne 0) { throw 'Build failed' }
|
||||||
|
node --test tests/cli-smoke.test.mjs
|
||||||
|
if ($LASTEXITCODE -ne 0) { throw 'CLI smoke tests failed' }
|
||||||
|
./dist/deployctl.exe version
|
||||||
|
Get-Content -Raw protocol/examples/plan-request.json | ./dist/deployctl.exe plan
|
||||||
|
```
|
||||||
|
|
||||||
|
Windows 可能因权限限制跳过符号链接测试;Linux 必须补验。仓库 `.gitattributes` 已将 Shell 脚本固定为 LF,
|
||||||
|
不要把脚本转为 CRLF。Git 换行提示不等于测试失败;实际错误须单独排查。
|
||||||
|
|
||||||
|
## Linux/WSL:基础离线检查
|
||||||
|
|
||||||
|
```bash
|
||||||
|
set -euo pipefail
|
||||||
|
go version
|
||||||
|
node --version
|
||||||
|
go test ./... -count=1
|
||||||
|
go vet ./...
|
||||||
|
go test -race ./... -count=1
|
||||||
|
go build -o dist/deployctl ./cmd/deployctl
|
||||||
|
node --test tests/cli-smoke.test.mjs
|
||||||
|
./dist/deployctl version
|
||||||
|
./dist/deployctl plan < protocol/examples/plan-request.json
|
||||||
|
./dist/deployctl preflight
|
||||||
|
```
|
||||||
|
|
||||||
|
`preflight` 报告运行它的本机/WSL,不是云服务器。它可以报告“不支持/有阻塞项”,不代表测试失败,
|
||||||
|
也不得为消除阻塞项擅自修改机器。WSL 临时目录可能随实例生命周期消失,不能作为跨会话依赖。
|
||||||
|
|
||||||
|
另有 `bash scripts/verify-linux.sh /绝对路径/已验证的Go.linux-amd64.tar.gz`:
|
||||||
|
会解包到新建临时目录、使用独立缓存执行 Go 全量测试/vet/race/构建及 CLI 只读检查。
|
||||||
|
它不下载 Go,不自动校验调用者提供的 Go 压缩包,不运行 Node 冒烟;调用前验证官方压缩包摘要,
|
||||||
|
Node 冒烟按上面命令另跑。该脚本按 amd64 测试流程提供,其他架构没有同等实测证据。
|
||||||
|
|
||||||
|
## 可选在线测试:不要默认运行
|
||||||
|
|
||||||
|
先完成离线检查。以下测试只在本地下载公开测试资料,不使用 SSH、不运行 APT、不解包或安装 deb。
|
||||||
|
需访问 Docker 官方仓库;网络、上游签名/密钥或元数据有效期变化都可能使测试失败,不能绕过校验。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
set -euo pipefail
|
||||||
|
go build -o dist/deployctl ./cmd/deployctl
|
||||||
|
# 只验证官方仓库签名、索引及篡改拒绝
|
||||||
|
bash scripts/probe-docker-repository.sh "$PWD/dist/deployctl"
|
||||||
|
# 另外下载约 90 MB(随测试版本变化)的五个 deb,验证并故意损坏其中一个
|
||||||
|
DEPLOYCTL_ONLINE_ARTIFACT_PROBE=1 bash scripts/probe-docker-repository.sh "$PWD/dist/deployctl"
|
||||||
|
```
|
||||||
|
|
||||||
|
需要连同 Go 全量 Linux 检查及真实签名单元测试一起运行时:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
DEPLOYCTL_ONLINE_REPOSITORY_PROBE=1 DEPLOYCTL_ONLINE_ARTIFACT_PROBE=1 \
|
||||||
|
bash scripts/verify-linux.sh /绝对路径/已验证的Go.linux-amd64.tar.gz
|
||||||
|
```
|
||||||
|
|
||||||
|
命令末尾的临时目录仅用于该次诊断。负向测试会故意篡改资料,不能当成安装输入或备份。
|
||||||
|
日志中出现预期的 `artifact verification failed` / `repository verification failed` 后,
|
||||||
|
应跟随篡改已拒绝的提示且整个脚本退出码为 0;不能只看一条成功输出。
|
||||||
|
未提供真实元数据夹具时,`TestStagedSignatureIntegration` 会跳过,这是明确的在线覆盖缺口,
|
||||||
|
不是已完成在线验收。
|
||||||
|
|
||||||
|
## 接续纪律
|
||||||
|
|
||||||
|
- 当前没有可运行的面板/便携包;开发依赖与未来用户免安装的产品依赖不要混淆。
|
||||||
|
- 先失败测试、再实现;用真实 CLI 和隔离 Linux 环境覆盖涉及的边界。
|
||||||
|
- 未通过验证、不支持或未检查的能力须保持不可执行;不要临时开放 `apply` 以便演示。
|
||||||
|
- 每次交接更新 HANDOFF 和实施状态,记录运行环境、命令、结果、跳过项及待确认事项。
|
||||||
|
- `dist/`、下载包、缓存、日志、SSH 密钥与真实配置不纳入源码提交。
|
||||||
@@ -0,0 +1,220 @@
|
|||||||
|
# 新架构实施状态
|
||||||
|
|
||||||
|
更新:2026-09-25。这是工程状态,不是生产部署验收报告。
|
||||||
|
|
||||||
|
跨设备接续从 [HANDOFF.md](../HANDOFF.md) 开始;工具链准备及可复现命令见[开发指南](development.md)。
|
||||||
|
本文件各工作包的测试结论属于相应阶段的历史证据,新设备仍需复跑。
|
||||||
|
|
||||||
|
## 已固定的选择
|
||||||
|
|
||||||
|
Traefik、本地 React/Node 管理面板、Go deployctl。当前目录开发,新架构已从
|
||||||
|
`codex/new-deployment-architecture` 合并到 `master`。用户已手动推送源码,
|
||||||
|
并授权将最新交接文档提交、推送到 master;后续接续统一使用 master。
|
||||||
|
以后的提交或推送依照用户当轮要求,不自动进行。
|
||||||
|
|
||||||
|
## 第一工作包:只读执行器基础
|
||||||
|
|
||||||
|
已实现:
|
||||||
|
|
||||||
|
- 独立 Go module,标准库实现,无第三方 Go 依赖。
|
||||||
|
- `version`、`plan`、`verify-plan` CLI;JSON stdin/stdout 协议。
|
||||||
|
- 严格输入校验:字段大小写、未知/重复/缺失字段、null、类型、体积限制。
|
||||||
|
- 实例标识、DNS 名称和 SHA-256 摘要校验。
|
||||||
|
- 实例项目名/数据路径推导、15 分钟有效期、内容绑定和状态漂移拒绝。
|
||||||
|
- 所有输出标记为不可执行的离线预览;未提供任何部署写命令。
|
||||||
|
- Go 单元测试以及真实二进制 Node 冒烟测试。
|
||||||
|
|
||||||
|
此阶段输入中的主机身份和状态摘要由调用方提供,不代表远程探测。
|
||||||
|
单镜像摘要只是预览基础字段,正式应用包必须完整绑定所有组件镜像。
|
||||||
|
摘要不是签名;只读校验通过不是写操作授权;路径拼接不是安全文件系统访问实现。
|
||||||
|
|
||||||
|
## 验证记录
|
||||||
|
|
||||||
|
- 在 Windows 使用临时 Go 1.27.1 官方工具链(下载 SHA-256 校验通过)。
|
||||||
|
- `go test ./... -count=1` 通过。
|
||||||
|
- `go vet ./...` 通过。
|
||||||
|
- Windows amd64 二进制构建、运行预览、校验、拒绝 apply 通过。
|
||||||
|
- `node --test tests/cli-smoke.test.mjs` 通过;确认测试工作目录无新增文件。
|
||||||
|
- 独立审查未发现 Critical/Important 问题;发现时间格式接受范围过宽,已通过先失败后通过的回归测试收紧为 UTC 秒级格式。
|
||||||
|
- Linux amd64 / CGO_ENABLED=0 交叉编译通过;未在 Linux 执行。
|
||||||
|
- 构建产物位于忽略目录 dist,不纳入源码发布。
|
||||||
|
|
||||||
|
## 第二工作包:任务状态与互斥基础
|
||||||
|
|
||||||
|
已实现 internal/state 内部模块,尚未接入 CLI 写操作:
|
||||||
|
|
||||||
|
- Linux flock / Windows LockFileEx 主机目录互斥,支持真实进程终止后释放锁。
|
||||||
|
- 操作 ID 与计划摘要绑定;重复请求返回原记录,不能改绑计划。
|
||||||
|
- 修订号比较确保只有一个调用方能领取任务;未决任务阻止其他写任务。
|
||||||
|
- 状态机区分成功、失败已恢复、需处理、未知及取消;终态不可重跑。
|
||||||
|
- 有界、带完整性摘要的快照;私有临时文件、同步、替换及 Linux 目录同步。
|
||||||
|
- 写入失败后冻结会话,必须重新打开并核对现场;不伪装保存成功。
|
||||||
|
- 初始化标记避免已有快照丢失后静默重置;新任务入库预留后续状态空间。
|
||||||
|
- 不自动清理幂等历史,不自动重跑中断任务;尚无事件审计日志和任务监督器。
|
||||||
|
|
||||||
|
验证使用临时目录,不涉及应用数据。Windows 测试通过;两项符号链接用例在
|
||||||
|
Windows 因创建链接权限不足跳过,在 Ubuntu WSL 中实际通过。Linux 全套测试、
|
||||||
|
状态模块竞态检测、go vet 和真实 CLI 运行已完成。进程终止测试不等于断电测试。
|
||||||
|
独立审查指出接近容量上限时缺少终态空间,已用 113,357 条历史记录复现并修复。
|
||||||
|
|
||||||
|
## 第三工作包:应用包校验与本机只读检查
|
||||||
|
|
||||||
|
已实现:
|
||||||
|
|
||||||
|
- 共享严格 JSON 解码,递归校验组件/文件数组,拒绝未知字段、null 和非法 UTF-8。
|
||||||
|
- 应用包清单绑定所有组件镜像摘要、平台、入口和文件内容摘要。
|
||||||
|
- 校验调用方提供的清单摘要,精确检查文件清单;拒绝缺失、多余、越界路径、符号链接和特殊文件。
|
||||||
|
- 清单、单文件、总内容和文件数量均设上限;返回已验证字节,供后续适配器使用。
|
||||||
|
- `verify-package` 仅输出清单和结果,不输出文件正文,不授予执行权限。
|
||||||
|
- `inspect` 只读取本机平台和 systemd/Docker 客户端文件状态,不启动进程或连接 Docker。
|
||||||
|
- 未检查的条件明确列出,`deploymentReady` 始终为 false。
|
||||||
|
|
||||||
|
调用方必须通过可信渠道获取期望摘要,并保证暂存目录及其父目录可信、校验期间不被修改。
|
||||||
|
内容摘要匹配不是发布者身份认证;本批不解析 Compose 语义,也不证明镜像能运行。
|
||||||
|
本机文件存在不等于服务健康,尚未检测 Docker daemon、Compose 版本、网络或磁盘容量。
|
||||||
|
|
||||||
|
本批验证:Windows 全套测试、go vet、构建及两项真实 CLI 冒烟测试通过;Ubuntu WSL
|
||||||
|
全套测试、go vet、全模块竞态检测、构建和真实 plan/inspect 运行通过。
|
||||||
|
Linux 实际执行了符号链接和 FIFO 拒绝测试。第一次 Linux 竞态构建曾报标准库 fmt
|
||||||
|
导入异常;待并行开发结束后以全新工具链目录和缓存重跑通过,异常未再复现,根因未确认。
|
||||||
|
独立审查发现默认 Docker 路径无效或不可读时漏查备用路径,已先复现三个失败用例再修复;
|
||||||
|
复审无遗留发现,修复后 Windows/Linux 全套检查再次通过。
|
||||||
|
|
||||||
|
## 第四工作包:受限 Compose 策略基础
|
||||||
|
|
||||||
|
- 新增 `check-package`:先校验完整应用包,再检查内存中的入口字节,不重新打开文件。
|
||||||
|
- `isolated-compose-v1` 仅接受严格 JSON;字段白名单拒绝未覆盖功能。
|
||||||
|
- 服务与清单组件/镜像精确匹配;要求显式非 root UID:GID、只读根文件系统、
|
||||||
|
丢弃全部 capabilities 和 no-new-privileges。
|
||||||
|
- 首版仅允许 backend 内部网络及普通项目命名卷;拒绝宿主挂载、外部资源、
|
||||||
|
跨服务共享卷、重叠挂载、系统目录挂载、Docker socket、主机网络、额外服务和构建/钩子。
|
||||||
|
- 扩展共享 JSON 解码器,递归检查动态服务字典的未知字段、缺失字段和 null。
|
||||||
|
- 包摘要与策略名在结果中明确返回,`executable` 和发布者认证始终为 false。
|
||||||
|
|
||||||
|
这是策略基础,不是完整 Compose 适配器。环境/秘密注入、Traefik 路由、健康规则、
|
||||||
|
应用权限例外、卷初始化权限和资源上限尚未实现;当前策略无法直接覆盖 Gitea 等完整应用。
|
||||||
|
它不验证镜像内部路径/默认行为,不证明 Docker 资源归属,也不替代实际部署/恢复测试。
|
||||||
|
精确契约见 [策略说明](../internal/composepolicy/README.md)。本批未连接或修改云服务器。
|
||||||
|
|
||||||
|
验证:先观察策略缺失、字典递归校验缺失和 CLI 命令缺失导致测试失败,再实现通过。
|
||||||
|
Windows 全套 Go 测试、go vet、构建和 3 项真实 CLI 冒烟测试通过;Ubuntu WSL
|
||||||
|
全套测试、go vet、全模块竞态检测、构建及现有只读 CLI 检查通过。
|
||||||
|
独立审查未发现本批范围内的问题,并独立重跑 composepolicy/wire/cli 测试通过。
|
||||||
|
尚未用真实 Docker/Compose 加载或运行策略示例,不能视为容器运行验收。
|
||||||
|
|
||||||
|
## 第五工作包:本机预检与环境安装提案
|
||||||
|
|
||||||
|
- 新增只读 `preflight`,输出实际观察、阻塞项、候选步骤和影响;不是可执行安装计划。
|
||||||
|
- 在 Linux 读取有限大小的 os-release、有效 UID 和 /var/lib 所在文件系统可用空间。
|
||||||
|
- 对常见运行时/数据/安装源路径作保守存在性检查,链接或不可读路径不视作干净环境。
|
||||||
|
- 仅在支持的 Ubuntu 版本/代号组合、root、systemd、磁盘初筛及资源检查通过时显示候选步骤。
|
||||||
|
- 明确指出软件包安装可能启动 Docker 并影响主机网络规则;不生成或执行 Shell 命令。
|
||||||
|
- 版本锁、软件包清单、源信任、主机身份和网络检查未完成,任何提案始终不可执行。
|
||||||
|
|
||||||
|
目前只是现场观察和安装提案基础,尚未实现完整运行前检查、APT 事务计划或环境安装。
|
||||||
|
5 GiB 是安装初筛阈值,不是应用/备份容量保证。没有扫描所有安装源或非标准数据路径。
|
||||||
|
本批未上传或在云服务器执行新二进制;CLI 只报告它实际运行的本机环境。
|
||||||
|
详细边界见 [预检说明](../internal/preflight/README.md)。
|
||||||
|
|
||||||
|
验证:Windows 全套测试、go vet、构建和 4 项真实 CLI 冒烟测试通过;Ubuntu WSL
|
||||||
|
全套测试、go vet、竞态检测、构建和实际 preflight 输出通过。Linux 用临时目录验证了
|
||||||
|
真实 statfs 查询和悬空符号链接拒绝作为“空环境”的行为,未涉及应用数据。
|
||||||
|
独立静态审查未发现本批范围内的问题;未完成真实软件包安装验收。
|
||||||
|
|
||||||
|
## 第六工作包:dpkg 清单、版本锁与安装事务草案
|
||||||
|
|
||||||
|
- 只读检查 dpkg 状态及更新日志目录,报告 Docker/容器运行时相关包和状态文件摘要。
|
||||||
|
- 相关包无论已安装、残留配置或部分安装均阻止全新安装候选;读取失败不视为未安装。
|
||||||
|
- 新增 `plan-environment`,输入明确的 Docker 五包版本锁,现场信息由本机收集。
|
||||||
|
- 限制官方源地址、发行版、架构、文件路径、版本、大小和 SHA-256;Engine/CLI 版本须一致。
|
||||||
|
- 草案绑定锁文件与现场观察摘要,列明请求包、影响和阻塞项;不执行下载、apt 或配置写入。
|
||||||
|
|
||||||
|
本批实现的是源地址约束和调用方版本锁校验,不是仓库签名认证。仍未验证签名 Release、
|
||||||
|
Packages 到 deb 的摘要链、元数据时效、实际文件或完整依赖事务;始终返回不可执行且源未认证。
|
||||||
|
未从在线仓库选择真实安装版本;没有对云服务器执行安装、升级或数据改动。
|
||||||
|
|
||||||
|
验证:Windows 全套 Go 测试、go vet、构建和 5 项真实 CLI 冒烟测试通过;Ubuntu WSL
|
||||||
|
全套测试、go vet、竞态检测、构建及真实 dpkg 清单读取通过。独立审查指出两项测试
|
||||||
|
被提前校验掩盖,已修正输入并通过临时停用对应校验的反向测试证明能捕获错误;
|
||||||
|
恢复生产校验后全套复验通过,复审无遗留发现。未执行真实 APT 安装事务或仓库签名验收。
|
||||||
|
|
||||||
|
## 第七工作包:官方仓库签名与元数据摘要链
|
||||||
|
|
||||||
|
- 新增 `verify-repository`:在 Linux 验证暂存的 Docker 公钥、Release 签名和 Packages 摘要。
|
||||||
|
- 公钥字节摘要及主指纹固定在代码中;使用隔离临时目录调用系统 GnuPG,不修改 APT 或个人密钥环。
|
||||||
|
- 校验发行版、架构、stable 组件、元数据日期与有效期,再按五个明确版本生成绑定 Release 的版本锁。
|
||||||
|
- 文件大小、类型、重复字段/记录、无效路径及未知签名状态均受限制;错误不回显原始输入。
|
||||||
|
- 认证输出不是执行授权;实际 deb 文件尚未下载验证,`packageBytesVerified`、`executable` 仍为 false。
|
||||||
|
|
||||||
|
真实 Docker resolute/amd64 元数据在本地 WSL 验证通过,篡改 Release 和 Packages 的端到端
|
||||||
|
CLI 测试均被拒绝。验证所选版本仅作测试样本,不代表最终安装版本。没有访问或修改云服务器,
|
||||||
|
没有安装软件、变更防火墙或接触应用数据。GnuPG 只写入本次创建的临时目录。
|
||||||
|
该校验依赖可信系统时钟、GnuPG 和无并发写入的可信暂存目录;30 天时效窗口不是持久防回退账本。
|
||||||
|
详细契约见 [仓库验证说明](../internal/aptrepo/README.md)。
|
||||||
|
|
||||||
|
验证:Windows 全套 Go 测试、go vet、构建及 6 项真实 CLI 冒烟测试通过;Linux 全套测试、
|
||||||
|
go vet、竞态检测、构建与官方在线元数据验证通过。独立审查发现并修复了输出缓冲的
|
||||||
|
io.Copy 大小限制绕过,失败回归用例已转绿;篡改测试改为合法语法,并增加交叉验证以防
|
||||||
|
其他解析错误掩盖签名/摘要检查。测试用 Python 已隔离环境,实测 PYTHONOPTIMIZE=1
|
||||||
|
不能禁用断言。没有完成实际 deb 文件验证或依赖事务预演。
|
||||||
|
|
||||||
|
## 第八工作包:实际 deb 文件完整性验证
|
||||||
|
|
||||||
|
- 新增 `verify-artifacts`:每次重新认证仓库元数据,再验证对应五个真实 deb 文件。
|
||||||
|
- 暂存目录必须精确包含这五个文件;拒绝缺失、多余、符号链接、特殊文件及大小/摘要不符。
|
||||||
|
- 流式计算 SHA-256,不将大型 deb 整体载入内存,不解包、不执行包内安装脚本。
|
||||||
|
- 任一文件失败时整个请求失败;只有全部通过才返回 `packageBytesVerified: true`。
|
||||||
|
- `executable` 保持 false;验证结果是当次观察,不是以后可复用的安装授权。
|
||||||
|
|
||||||
|
本地 Ubuntu WSL 下载并验证了五个官方 resolute/amd64 deb 测试样本(约 90 MB);
|
||||||
|
同长度单字节篡改测试被拒绝。没有安装任何下载包,没有调用 APT,没有连接或修改云服务器。
|
||||||
|
测试样本暂存于本地临时目录,其中一个被测试故意修改,不可拿去安装。
|
||||||
|
|
||||||
|
验证:Windows 全套测试、go vet、构建与 7 项 CLI 冒烟通过;Linux 全套测试、go vet、
|
||||||
|
竞态检测、构建及真实 deb 下载/签名/摘要/篡改测试通过。独立审查指出的临时辅助文件权限
|
||||||
|
边界和协议测试被其他错误掩盖的问题已修复,复审无遗留发现。协议测试通过临时换用宽松
|
||||||
|
解码器确认能够失败,恢复严格解码后复验通过。以上是本批范围内的验证,不是安装验收。
|
||||||
|
|
||||||
|
本批完成的是 Docker 五包的字节校验,不是完整安装事务预演。Ubuntu 依赖源的认证、
|
||||||
|
全部依赖的版本/摘要锁定,以及隔离 APT 配置/包状态的预演仍未交付。
|
||||||
|
APT 会读取多层配置,后续适配器须隔离配置与状态,不直接复用宿主默认配置;
|
||||||
|
参考 [APT 配置加载顺序](https://manpages.debian.org/trixie/apt/apt.conf.5.en.html)。
|
||||||
|
|
||||||
|
## 跨设备开发交接(2026-09-25)
|
||||||
|
|
||||||
|
用户要求补充记录并实现其他设备拉取接续。本次将此前八个工作包的源码、测试、协议及文档
|
||||||
|
纳入 `codex/new-deployment-architecture` 分支交付,不合并到 master;交接入口为根目录 HANDOFF.md。
|
||||||
|
开发指南记录工具链基线、Windows/Linux 命令、可选在线测试和凭据边界;补充了本地服务+浏览器
|
||||||
|
的便携交付目标,并明确 APT 隔离细化方案仍待确认。原有旧 `.env` 本地删除不纳入本次提交。
|
||||||
|
|
||||||
|
对 Git 暂存快照导出的新架构干净源码副本复验:Windows 全量 Go 测试、vet、构建、7 项 CLI
|
||||||
|
冒烟及示例运行通过;Linux 全量测试、vet、race、构建及 CLI 只读检查通过。Windows 最初在
|
||||||
|
创建构建目录时遇到本地沙箱权限拒绝,授权重跑同一副本后通过,未修改代码或放宽目录权限。
|
||||||
|
这是同一设备上的隔离源码副本/双环境验证,不是另一台物理设备或 macOS 的验收。
|
||||||
|
文档相对链接已检查。本轮未重复在线下载;其通过证据保留在第七、八工作包,新设备可按指南重跑。
|
||||||
|
|
||||||
|
## 合并状态(2026-09-25)
|
||||||
|
|
||||||
|
用户随后要求提交所有剩余项目改动,旧 `claude-dev-stack/.env` 删除已记录在 `f25f5df`。
|
||||||
|
之后按明确指令快进合并回本地 master;最初用户选择“仅合并到本地 master”,助手未执行推送。
|
||||||
|
前述“不合并 master / .env 删除未纳入交付”描述的是上一阶段的历史范围,不再是当前本地状态。
|
||||||
|
随后用户确认已手动推送,并明确要求最新修改也提交、推送。只读远端核对确认源码已到达
|
||||||
|
origin/master(f25f5df);本次同步剩余文档修订,统一接续分支为 master,不再要求切换旧工作分支。
|
||||||
|
|
||||||
|
## 尚未交付
|
||||||
|
|
||||||
|
1. 远程现场 inspect、持久化状态与执行器集成、systemd 任务监督及重启现场核对。
|
||||||
|
2. 应用包可信分发、发布者认证、完整 Compose 安全策略和运行适配器,以及应用包与计划的完整绑定。
|
||||||
|
3. Traefik 环境安装、受限 Docker API 代理、网络与 ACME 实测。
|
||||||
|
4. Gitea/Joplin 和其余工具的部署、升级、备份与隔离恢复。
|
||||||
|
5. 本地面板、SSH、SQLite 与 Apple 风格交互界面。
|
||||||
|
6. Windows/macOS 便携产品、Linux 故障注入与生产发布验收。
|
||||||
|
|
||||||
|
前三个工作包未连接服务器。用户随后指定新服务器 `121.199.168.54` 并确认主机指纹,
|
||||||
|
现已完成 SSH 只读初检,详见 [现场记录](2026-09-25-target-server-preflight.md)。
|
||||||
|
未安装环境、迁移或清除任何应用数据,未操作旧服务器,未删除旧部署实现。
|
||||||
|
旧脚本暂留作开发对照,不作为新执行器后端;新架构完成验收后按总方案移除。
|
||||||
|
|
||||||
|
下一工作包是 Ubuntu 依赖源认证与隔离的完整依赖事务预演,之后才接入安装执行,
|
||||||
|
再逐步接入受控配置/秘密、入口路由和任务执行器。
|
||||||
|
完整产品尚未完成,不可将当前 CLI 用作部署工具。
|
||||||
@@ -0,0 +1,62 @@
|
|||||||
|
# Operation State Implementation Plan
|
||||||
|
|
||||||
|
> **For agentic workers:** Use superpowers:executing-plans with test-driven-development. No production access or automatic commits.
|
||||||
|
|
||||||
|
**Goal:** Persist operation identity and status under an exclusive OS lock, refusing duplicate execution and ambiguous state.
|
||||||
|
|
||||||
|
**Architecture:** An internal state package acquires a nonblocking per-directory host lock and returns a session. Every mutation saves a complete bounded snapshot using write/sync/rename; the session becomes unusable on persistence errors. The directory must be a pre-existing trusted local directory, not a network share.
|
||||||
|
|
||||||
|
**Tech Stack:** Go standard library, Linux flock / Windows LockFileEx, os.Root.
|
||||||
|
|
||||||
|
**Spec:** docs/2026-09-25-complete-optimization-proposal.md sections 7, 8, 12, 14.
|
||||||
|
|
||||||
|
## Constraints
|
||||||
|
|
||||||
|
- Current directory and codex/new-deployment-architecture branch; preserve prior edits.
|
||||||
|
- No CLI write endpoint or Docker/application operation is enabled by this package.
|
||||||
|
- No automatic deletion of lock files, stale-task replay or corrupted-state reset.
|
||||||
|
- Lock-file inode must remain stable; close releases the OS lock.
|
||||||
|
- Metadata integrity is not authorization, and terminal success must be verified by the future executor.
|
||||||
|
- Linux is the production target; Windows provides development tests. Power-loss durability on Windows is not claimed.
|
||||||
|
|
||||||
|
## Task 1: Exclusive host sessions
|
||||||
|
|
||||||
|
Files: internal/state/store.go, lock_linux.go, lock_windows.go, lock_unsupported.go, store_test.go.
|
||||||
|
|
||||||
|
Interface: `Acquire(directory, hostID string) (*Session, error)`; `Session.Close() error`.
|
||||||
|
|
||||||
|
- [x] Write tests: second session returns ErrBusy, closed session rejects use, independent directories do not block each other, invalid IDs fail, killed child releases OS lock.
|
||||||
|
- [x] Run `go test ./internal/state`; observe missing implementation failure.
|
||||||
|
- [x] Implement nonblocking OS lock with private regular file inside os.Root; refuse unsupported systems and symlink state files.
|
||||||
|
- [x] Run tests using real temp directories and subprocesses, not mocked locks.
|
||||||
|
|
||||||
|
```go
|
||||||
|
second, err := Acquire(dir, "host-one")
|
||||||
|
if second != nil || !errors.Is(err, ErrBusy) { t.Fatal("host lock bypassed") }
|
||||||
|
```
|
||||||
|
|
||||||
|
## Task 2: Persistent idempotency and state transitions
|
||||||
|
|
||||||
|
Files: internal/state/operation.go, snapshot.go, operation_test.go.
|
||||||
|
|
||||||
|
Interfaces: `Begin(id, planHash string) (Operation, bool, error)`; `Get(id string) (Operation, error)`; `Advance(id string, revision uint64, next Status) (Operation, error)`.
|
||||||
|
|
||||||
|
- [x] Tests: duplicate ID/hash returns original with created=false; same ID/different hash conflicts; reopened state persists; unrelated new operation blocked by unresolved prior operation; stale revision and running-to-running rejected; terminal results immutable.
|
||||||
|
- [x] Tests: corrupt/truncated/oversized/host-mismatched snapshots rejected; write failure poisons session and never returns success; surviving running state is not silently reset.
|
||||||
|
- [x] Implement canonical checksummed snapshot, bounded read, unique temporary file, file sync, atomic rename and Linux directory sync. Any persistence failure requires closing/reopening and checking the actual state.
|
||||||
|
- [x] Run full tests and real killed-child recovery test. Abrupt process death is not a power-loss durability test.
|
||||||
|
|
||||||
|
```go
|
||||||
|
again, created, err := session.Begin("op-one", hash)
|
||||||
|
if err != nil || created || again.Revision != 1 { t.Fatal("duplicate submission changed state") }
|
||||||
|
```
|
||||||
|
|
||||||
|
## Task 3: Verification and documentation
|
||||||
|
|
||||||
|
- [x] Run gofmt, `go test ./... -count=1`, `go vet ./...`, native CLI smoke tests and Linux cross-build including state tests.
|
||||||
|
- [x] Independent read-only code review; repair actionable findings with regression tests.
|
||||||
|
- [x] Update docs/implementation-status.md and internal/state/README.md with lifecycle, trusted-root requirements, residual risks, and remaining systemd/executor work.
|
||||||
|
|
||||||
|
Additional evidence: Ubuntu WSL full suite and `go test -race ./internal/state` passed.
|
||||||
|
Windows symlink tests skipped for unavailable privilege; both passed on Linux.
|
||||||
|
Regression tests cover missing initialized snapshots and 113,357-record capacity boundary.
|
||||||
@@ -0,0 +1,56 @@
|
|||||||
|
# Package Verification and Inspection Implementation Plan
|
||||||
|
|
||||||
|
> **For agentic workers:** Execute with test-driven-development; package worker and local inspection work have disjoint write scopes. No commits, production connections, or deployment writes.
|
||||||
|
|
||||||
|
**Goal:** Add read-only package verification and a factual local prerequisite report to deployctl.
|
||||||
|
|
||||||
|
**Architecture:** A shared bounded strict JSON decoder supports nested typed arrays. An immutable package verifier returns verified manifest and file bytes, never executes templates. A local probe reports OS/architecture and tool/runtime presence, with unperformed checks explicitly marked unknown.
|
||||||
|
|
||||||
|
**Tech Stack:** Go standard library. Existing local Windows and Ubuntu WSL test runtimes.
|
||||||
|
|
||||||
|
**Spec:** docs/2026-09-25-complete-optimization-proposal.md sections 5, 7, 12, 14.
|
||||||
|
|
||||||
|
## Boundaries and integration
|
||||||
|
|
||||||
|
- Parent owns internal/wire, internal/inspect, internal/cli, docs and CLI tests.
|
||||||
|
- Package worker owns only internal/appbundle; requirements are in package-verifier-brief.md beside this plan.
|
||||||
|
- Package code consumes `wire.Decode(reader io.Reader, target any, limit int64) error`.
|
||||||
|
- CLI consumes `appbundle.Verify(directory, expectedDigest string) (Verified, error)`; Verified has Manifest, Files map[string][]byte, Digest string. CLI never prints file bodies.
|
||||||
|
- Local inspect is not full host acceptance; Docker daemon, Compose, firewall, DNS, permissions and port availability remain explicitly unverified.
|
||||||
|
- Expected digest must come from a trusted channel. Matching it does not establish publisher identity on its own.
|
||||||
|
|
||||||
|
## Task 1: Shared strict wire decoder
|
||||||
|
|
||||||
|
- [x] Move strict JSON decoder from internal/cli to internal/wire, keeping all CLI tests passing.
|
||||||
|
- [x] Add failing tests for nested array unknown/missing/case-alias fields and null elements, duplicate keys and input limit.
|
||||||
|
- [x] Extend recursive validation through typed slices and reject invalid UTF-8. Preserve canonical UTC timestamp policy.
|
||||||
|
- [x] Test the API `wire.Decode(strings.NewReader(input), &value, 1024)` using real JSON.
|
||||||
|
|
||||||
|
## Task 2: Application package verifier
|
||||||
|
|
||||||
|
- [x] Worker implements bounded manifest validation, pinned multi-component images, exact file inventory, SHA-256 content checks and path/symlink rejection using tests first.
|
||||||
|
- [x] Parent reviews written code and runs all tests, including Linux symlink cases.
|
||||||
|
- [x] Independent review checks all file access and integrity boundaries.
|
||||||
|
|
||||||
|
## Task 3: Read-only inspection and CLI
|
||||||
|
|
||||||
|
- [x] internal/inspect provides `Collect() Report` and a testable filesystem probe. Report includes OS, architecture, Linux support, systemd runtime presence and Docker client presence, with explicit unchecked prerequisites.
|
||||||
|
- [x] Missing or inaccessible files produce missing/unknown observations, never false success. No shell commands, network, credentials or environment changes.
|
||||||
|
- [x] CLI `inspect` emits observations with deploymentReady=false.
|
||||||
|
- [x] CLI `verify-package` accepts `{directory, expectedDigest}`, emits manifest, digest, file count and executable=false; rejects invalid input with redacted diagnostics.
|
||||||
|
- [x] Add actual CLI tests for new commands and unsupported writes.
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
- [x] Windows go test/vet, CLI smoke; Linux full tests, race check and build.
|
||||||
|
- [x] Record separate implemented, unverified and not-yet-supported capabilities.
|
||||||
|
|
||||||
|
## Execution notes
|
||||||
|
|
||||||
|
Task 1 + Task 2 share only the Decode interface; Task 2 must not modify it. Task 3 consumes their exports without exposing package bytes. Directory paths are accepted only for read-only local verification. Files remain subject to trusted staging ownership; future execution must use the verified snapshot rather than reopening mutable package files.
|
||||||
|
|
||||||
|
Independent review found an alternate Docker path omitted when the primary path
|
||||||
|
was invalid or inaccessible. Three failing regression cases reproduced it before
|
||||||
|
the fix. Fallback now prefers presence, then uncertainty, invalidity and absence.
|
||||||
|
Focused re-review found no remaining issues. Final Windows and Linux checks passed;
|
||||||
|
see implementation-status.md for the initial non-reproduced Linux compiler error.
|
||||||
@@ -0,0 +1,77 @@
|
|||||||
|
# Runner Foundation Implementation Plan
|
||||||
|
|
||||||
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox syntax for tracking.
|
||||||
|
|
||||||
|
**Goal:** Deliver a runnable, read-only deployctl foundation that strictly validates deployment requests and produces expiring, state-bound plans without executing legacy scripts.
|
||||||
|
|
||||||
|
**Architecture:** Go standard-library CLI consumes bounded JSON over stdin. Domain validation and planning are separate from CLI transport. No deployment writes are enabled before the executor and application recovery contracts pass their own acceptance tests.
|
||||||
|
|
||||||
|
**Tech Stack:** Go, standard testing package, JSON protocol v1.
|
||||||
|
|
||||||
|
**Spec:** docs/2026-09-25-complete-optimization-proposal.md
|
||||||
|
|
||||||
|
## Global Constraints
|
||||||
|
|
||||||
|
- Traefik + local React/Node panel + Go runner are approved choices.
|
||||||
|
- Do not connect to production or modify existing application data.
|
||||||
|
- Preserve current uncommitted documents and claude-dev-stack deletion.
|
||||||
|
- No implicit system upgrade, arbitrary shell field, mutable image tag or fake successful apply.
|
||||||
|
- This plan is the first independently testable work package, not the full product.
|
||||||
|
- No commit or push without user request.
|
||||||
|
|
||||||
|
## Delivery sequence after this package
|
||||||
|
|
||||||
|
Remote state/locking/idempotency → trusted packages and Compose runtime → Traefik environment → Gitea backup/upgrade/restore → Joplin verification → local panel → other applications → isolated Linux acceptance and replacement of legacy source. These require separate implementation plans; they are not represented as completed by this package.
|
||||||
|
|
||||||
|
## Task 1: Validated deployment intent
|
||||||
|
|
||||||
|
Files: go.mod; internal/planner/intent.go; internal/planner/intent_test.go.
|
||||||
|
|
||||||
|
Interface: Intent.Validate() error. Intent includes protocolVersion, hostId, instanceId, appId, packageDigest, imageDigest, domain, observedStateDigest.
|
||||||
|
|
||||||
|
- [x] Write table-driven tests: valid request succeeds; traversal IDs, missing fields, unsupported protocol, floating images, invalid domains and invalid hashes fail.
|
||||||
|
- [x] Run `go test ./internal/planner` and observe the missing implementation failure.
|
||||||
|
- [x] Implement strict lower-case IDs, sha256 digest validation and ASCII DNS names; do not normalize invalid input silently.
|
||||||
|
- [x] Run tests and `go vet ./...`.
|
||||||
|
|
||||||
|
Example boundary assertion:
|
||||||
|
|
||||||
|
```go
|
||||||
|
request.InstanceID = "../gitea"
|
||||||
|
if request.Validate() == nil { t.Fatal("accepted path traversal") }
|
||||||
|
```
|
||||||
|
|
||||||
|
## Task 2: Expiring, state-bound plan
|
||||||
|
|
||||||
|
Files: internal/planner/plan.go; internal/planner/plan_test.go.
|
||||||
|
|
||||||
|
Interfaces: Build(Intent, time.Time) (Plan, error); Plan.Verify(Intent, time.Time) error.
|
||||||
|
|
||||||
|
- [x] Test repeatable hashes at fixed time, separate instance identities, expiry boundary, altered intent/hash and observed-state drift rejection.
|
||||||
|
- [x] Observe failing tests before implementation.
|
||||||
|
- [x] Implement SHA-256 over typed JSON with an empty hash field, 15-minute TTL, deterministic project/data paths and verification of all derived fields.
|
||||||
|
- [x] Run `go test ./internal/planner`.
|
||||||
|
|
||||||
|
```go
|
||||||
|
if err := plan.Verify(intent, plan.ExpiresAt); err == nil { t.Fatal("accepted expired plan") }
|
||||||
|
```
|
||||||
|
|
||||||
|
Plan hash is an integrity binding, NOT authentication or approval. Supplied host/state are unverified offline input; no write operation may trust them without future remote inspection.
|
||||||
|
|
||||||
|
## Task 3: Read-only CLI and contract
|
||||||
|
|
||||||
|
Files: cmd/deployctl/main.go; internal/cli/run.go; internal/cli/run_test.go; protocol/README.md; protocol/examples/plan-request.json.
|
||||||
|
|
||||||
|
Interface: cli.Run(args []string, in io.Reader, out, diagnostics io.Writer, now func() time.Time) int.
|
||||||
|
|
||||||
|
- [x] Tests exercise actual JSON input/output: plan and verify success, unknown fields, duplicate fields, trailing objects, oversized input, unsupported commands including apply, and no input echo on errors.
|
||||||
|
- [x] Observe missing CLI behavior failure.
|
||||||
|
- [x] Implement `version`, `plan`, `verify-plan` only; strict size-bounded decoder; generic diagnostics; nonzero exit on refusal. `apply` stays unavailable.
|
||||||
|
- [x] Build and execute the real CLI with sample stdin; verify output parses and `apply` fails.
|
||||||
|
- [x] Document exact wire format, offline limitation, commands and remaining stages.
|
||||||
|
|
||||||
|
## Verification and handoff
|
||||||
|
|
||||||
|
- [x] gofmt; go test ./... -count=1; go vet ./...; build Windows and Linux binaries.
|
||||||
|
- [x] Confirm no external dependencies and no production connection/process execution in CLI.
|
||||||
|
- [x] Record actual verification results and remaining work in docs/implementation-status.md.
|
||||||
@@ -0,0 +1,19 @@
|
|||||||
|
# Package verifier worker requirements
|
||||||
|
|
||||||
|
Implement ONLY internal/appbundle/*.go and optional internal/appbundle/README.md in G:/Works/server-deploy. No commits, no other files, no subagents, no production access. Use apply_patch, tests first. Parent handles CLI, shared decoder, environment inspection and integration review.
|
||||||
|
|
||||||
|
API: `Verify(directory, expectedDigest string) (Verified, error)`.
|
||||||
|
Verified: `Manifest Manifest`, `Files map[string][]byte`, `Digest string`.
|
||||||
|
Manifest JSON fields (all required, no extras): protocolVersion (1), appId (lowercase ID), version (numeric x.y.z), runtime (compose), entrypoint (relative path to listed file), platforms ([]string of linux/amd64 or linux/arm64, unique, nonempty), components ([]Component), files ([]File).
|
||||||
|
Component fields: name (lowercase ID), image (lowercase repository@sha256:64hex; no tag, port, whitespace or shell syntax in initial subset). Components 1..32, names unique; repository components use lowercase ASCII alnum with separators dot/underscore/hyphen, slash between components, no empty segments.
|
||||||
|
File fields: path (portable relative slash path), digest (sha256:64 lowercase hex). Files 1..128, unique; manifest.json itself excluded from files. No slash root, drive, backslash, colon, dot/dotdot segments, dotfiles, empty segments, trailing spaces/dots, Windows device-name segments (including extensions), or case-colliding names. Allowed path segment alphabet lowercase ASCII a-z0-9 underscore hyphen dot, first character alphanumeric. Max path 240 bytes, segment 100. ID rules consistent with planner: first a-z, remaining a-z0-9-, max48.
|
||||||
|
|
||||||
|
Manifest file name manifest.json; maximum 1MiB. ExpectedDigest pins SHA-256 of the exact raw manifest bytes (manifest pins every file). Validate digest before parsing. Use shared `server-deploy/internal/wire.Decode(io.Reader, target, int64 limit)` being implemented by parent; it rejects duplicate/unknown/missing/case-alias fields, nulls including slice elements, trailing data and oversized input.
|
||||||
|
|
||||||
|
Read only regular files under os.Root, reject symlinks including intermediate directory links, file size max4MiB, total payload <=16MiB. Exact inventory: reject any unlisted files, secrets, symlinks, special files or unnecessary directories; allow only parent directories of declared files plus manifest.json. Bound enumeration to declared inventory rather than unbounded walk; reject extras immediately. No extraction, writes, processes, network or arbitrary command execution. Require absolute directory; staging root is caller-selected trusted directory (ancestors trusted), don't claim hostile concurrent mutation is fully prevented. Return verified bytes for later consumers; do not reopen files to execute anything.
|
||||||
|
|
||||||
|
Entrypoint must be listed. DO NOT claim Compose semantics safe/validated: contents are authenticated as bytes only. Signature trust store and executable policy come later; expected hash is a caller trust anchor, not publisher authentication. Do not hardcode live app image hashes or fetch versions.
|
||||||
|
|
||||||
|
Tests with real temp directories: valid two-component package, wrong manifest digest, changed/missing/extra file, nested valid file, symlink/intermediate symlink (skip only missing Windows privilege, Linux tested by parent), traversal, duplicate components/file paths, unpinned image, unsupported protocol/runtime/platform, missing/unknown/null component fields, oversized payload, entrypoint missing. Fixture hashes may be calculated from known test bytes, but test expected acceptance/rejection independently.
|
||||||
|
|
||||||
|
Toolchain: C:/Users/Joywayer/AppData/Local/Temp/server-deploy-go-d67bef26e1ee49718e2b62311429f729/go/bin/go.exe. Set GOCACHE to a dedicated temp path if needed. Parent creates wire shortly; don't change it to unblock. Report test results and changed paths, plus any blockers/concerns, without declaring full deployment readiness.
|
||||||
@@ -0,0 +1,36 @@
|
|||||||
|
# Local package verifier
|
||||||
|
|
||||||
|
`Verify(directory, expectedDigest)` accepts an absolute staging directory and a
|
||||||
|
canonical `sha256:` digest of its exact `manifest.json` bytes. It returns the
|
||||||
|
validated manifest, matching payload bytes keyed by relative path, and manifest
|
||||||
|
digest. On failure it returns a zero `Verified` and fixed diagnostic text without
|
||||||
|
embedding file contents, decoder errors, or filesystem paths.
|
||||||
|
|
||||||
|
The manifest is limited to 1 MiB, each payload to 4 MiB, and all payloads together
|
||||||
|
to 16 MiB. Metadata checks precede opening regular files; actual reads also have
|
||||||
|
a limit plus one overflow-detection byte and must match the observed size.
|
||||||
|
The manifest digest is checked before shared `wire.Decode` parses it.
|
||||||
|
|
||||||
|
Inventory is derived from at most 128 declared files and their parent directories.
|
||||||
|
Each directory is enumerated one entry at a time; an unexpected entry fails
|
||||||
|
immediately. Symlinks, special files, empty/unnecessary directories, and unlisted
|
||||||
|
files are rejected. Lowercase portable paths prevent case collisions. Files and
|
||||||
|
subdirectories are opened relative to `os.Root`, with identity checks around open.
|
||||||
|
|
||||||
|
The caller must select a trusted staging directory with trusted ancestors and
|
||||||
|
prevent concurrent mutation. These checks do not provide a transactional snapshot
|
||||||
|
or complete protection against hostile concurrent filesystem changes. Consumers
|
||||||
|
should use the returned bytes rather than reopen package files.
|
||||||
|
|
||||||
|
This authenticates bytes against a caller-provided trust anchor. It does not
|
||||||
|
authenticate a publisher, validate Compose semantics, or authorize execution.
|
||||||
|
Signature trust stores and executable policy are outside this package.
|
||||||
|
|
||||||
|
Tests use local temporary directories. Symlink cases skip only Windows error 1314
|
||||||
|
(missing symlink privilege). Linux-specific FIFO cases verify rejection without
|
||||||
|
opening the FIFO; run tests with a timeout, for example:
|
||||||
|
|
||||||
|
```text
|
||||||
|
go test ./internal/appbundle -count=1 -timeout=30s
|
||||||
|
go vet ./internal/appbundle
|
||||||
|
```
|
||||||
@@ -0,0 +1,48 @@
|
|||||||
|
package appbundle
|
||||||
|
|
||||||
|
import (
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Check lexical rejection independently of missing-file rejection. Invalid paths
|
||||||
|
// cannot always be materialized on Windows, so a missing file is not sufficient
|
||||||
|
// evidence that the manifest validator enforces portable paths.
|
||||||
|
func TestPortablePathValidation(t *testing.T) {
|
||||||
|
for _, path := range []string{
|
||||||
|
"", "../escape", "a/../b", "a/./b", "/root", "c:/file", `a\b`,
|
||||||
|
"a//b", "a/", ".env", "a/.secret", "a/secret ", "a/secret.",
|
||||||
|
"con", "prn.txt", "aux.txt", "nul", "a/com1.log", "lpt9",
|
||||||
|
"UPPER.txt", "a:stream", "café", strings.Repeat("a", 101),
|
||||||
|
strings.Repeat("a/", 120) + "b",
|
||||||
|
} {
|
||||||
|
if validPath(path) {
|
||||||
|
t.Errorf("accepted invalid path %q", path)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, path := range []string{
|
||||||
|
"compose.yaml", "dir-a/1_config.json", "com0.txt", "lpt10.txt",
|
||||||
|
strings.Repeat("a", 100), strings.Repeat("a", 100) + "/" + strings.Repeat("b", 100) + "/" + strings.Repeat("c", 38),
|
||||||
|
} {
|
||||||
|
if !validPath(path) {
|
||||||
|
t.Errorf("rejected valid path %q", path)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestImageDigestAndRepositoryValidation(t *testing.T) {
|
||||||
|
for _, image := range []string{
|
||||||
|
"api@sha256:" + strings.Repeat("a", 63), "api@sha256:" + strings.Repeat("A", 64),
|
||||||
|
"api@sha256:" + strings.Repeat("a", 65), "api@sha512:" + strings.Repeat("a", 64),
|
||||||
|
"api:tag@sha256:" + strings.Repeat("a", 64), "api@sha256:" + strings.Repeat("a", 64) + "@extra",
|
||||||
|
} {
|
||||||
|
if validImage(image) {
|
||||||
|
t.Errorf("accepted invalid image %q", image)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, repo := range []string{"a", "registry.example/team/api", "team_name/app-v2.0"} {
|
||||||
|
if !validImage(repo + "@sha256:" + strings.Repeat("a", 64)) {
|
||||||
|
t.Errorf("rejected valid repository %q", repo)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,312 @@
|
|||||||
|
// Package appbundle authenticates a bounded local bundle as bytes.
|
||||||
|
package appbundle
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"crypto/sha256"
|
||||||
|
"encoding/hex"
|
||||||
|
"errors"
|
||||||
|
"io"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"regexp"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"server-deploy/internal/wire"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
manifestLimit int64 = 1 << 20
|
||||||
|
fileLimit int64 = 4 << 20
|
||||||
|
payloadLimit int64 = 16 << 20
|
||||||
|
)
|
||||||
|
|
||||||
|
var (
|
||||||
|
idPattern = regexp.MustCompile(`^[a-z][a-z0-9-]{0,47}$`)
|
||||||
|
versionPattern = regexp.MustCompile(`^[0-9]+\.[0-9]+\.[0-9]+$`)
|
||||||
|
digestPattern = regexp.MustCompile(`^sha256:[0-9a-f]{64}$`)
|
||||||
|
repositoryPart = regexp.MustCompile(`^[a-z0-9]+(?:[._-]+[a-z0-9]+)*$`)
|
||||||
|
pathPart = regexp.MustCompile(`^[a-z0-9][a-z0-9_.-]*$`)
|
||||||
|
)
|
||||||
|
|
||||||
|
type Manifest struct {
|
||||||
|
ProtocolVersion int `json:"protocolVersion"`
|
||||||
|
AppID string `json:"appId"`
|
||||||
|
Version string `json:"version"`
|
||||||
|
Runtime string `json:"runtime"`
|
||||||
|
Entrypoint string `json:"entrypoint"`
|
||||||
|
Platforms []string `json:"platforms"`
|
||||||
|
Components []Component `json:"components"`
|
||||||
|
Files []File `json:"files"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type Component struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
Image string `json:"image"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type File struct {
|
||||||
|
Path string `json:"path"`
|
||||||
|
Digest string `json:"digest"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type Verified struct {
|
||||||
|
Manifest Manifest
|
||||||
|
Files map[string][]byte
|
||||||
|
Digest string
|
||||||
|
}
|
||||||
|
|
||||||
|
// Verify authenticates the exact manifest and listed payload bytes. The expected
|
||||||
|
// digest is a caller trust anchor, not publisher authentication. Compose syntax
|
||||||
|
// and execution safety are not evaluated. The caller must use a trusted staging
|
||||||
|
// directory with trusted ancestors and prevent concurrent mutation; os.Root and
|
||||||
|
// identity checks do not provide a transaction against hostile concurrent writers.
|
||||||
|
func Verify(directory, expectedDigest string) (Verified, error) {
|
||||||
|
if !filepath.IsAbs(directory) {
|
||||||
|
return Verified{}, errors.New("bundle directory must be absolute")
|
||||||
|
}
|
||||||
|
// A trailing separator can make Lstat follow a directory symlink on Unix.
|
||||||
|
// Normalize it before checking the caller-selected root itself.
|
||||||
|
directory = filepath.Clean(directory)
|
||||||
|
if !digestPattern.MatchString(expectedDigest) {
|
||||||
|
return Verified{}, errors.New("invalid expected manifest digest")
|
||||||
|
}
|
||||||
|
info, err := os.Lstat(directory)
|
||||||
|
if err != nil || !info.IsDir() || info.Mode()&os.ModeSymlink != 0 {
|
||||||
|
return Verified{}, errors.New("invalid bundle directory")
|
||||||
|
}
|
||||||
|
root, err := os.OpenRoot(directory)
|
||||||
|
if err != nil {
|
||||||
|
return Verified{}, errors.New("cannot open bundle directory")
|
||||||
|
}
|
||||||
|
defer root.Close()
|
||||||
|
openedInfo, err := root.Stat(".")
|
||||||
|
if err != nil || !os.SameFile(info, openedInfo) {
|
||||||
|
return Verified{}, errors.New("bundle directory changed")
|
||||||
|
}
|
||||||
|
raw, err := readRegular(root, "manifest.json", manifestLimit)
|
||||||
|
if err != nil {
|
||||||
|
return Verified{}, errors.New("cannot read bounded regular manifest")
|
||||||
|
}
|
||||||
|
if hash(raw) != expectedDigest {
|
||||||
|
return Verified{}, errors.New("manifest digest mismatch")
|
||||||
|
}
|
||||||
|
var manifest Manifest
|
||||||
|
if err := wire.Decode(bytes.NewReader(raw), &manifest, manifestLimit); err != nil {
|
||||||
|
// Never propagate decoder errors: some include offending input values.
|
||||||
|
return Verified{}, errors.New("invalid manifest JSON")
|
||||||
|
}
|
||||||
|
tree, err := validate(manifest)
|
||||||
|
if err != nil {
|
||||||
|
return Verified{}, err
|
||||||
|
}
|
||||||
|
files := make(map[string][]byte, len(manifest.Files))
|
||||||
|
remaining := payloadLimit
|
||||||
|
if err := readInventory(root, tree, "", files, &remaining); err != nil {
|
||||||
|
return Verified{}, err
|
||||||
|
}
|
||||||
|
return Verified{Manifest: manifest, Files: files, Digest: expectedDigest}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func hash(data []byte) string {
|
||||||
|
sum := sha256.Sum256(data)
|
||||||
|
return "sha256:" + hex.EncodeToString(sum[:])
|
||||||
|
}
|
||||||
|
|
||||||
|
// readRegular caps both the pre-open size and actual bytes read. The one extra
|
||||||
|
// byte detects growth or oversize without an unbounded read, even after Stat.
|
||||||
|
// Names are single validated path segments relative to an already-open Root.
|
||||||
|
func readRegular(root *os.Root, name string, limit int64) ([]byte, error) {
|
||||||
|
info, err := root.Lstat(name)
|
||||||
|
if err != nil || !info.Mode().IsRegular() || info.Size() < 0 || info.Size() > limit {
|
||||||
|
return nil, errors.New("invalid file type or size")
|
||||||
|
}
|
||||||
|
f, err := root.Open(name)
|
||||||
|
if err != nil {
|
||||||
|
return nil, errors.New("cannot open file")
|
||||||
|
}
|
||||||
|
defer f.Close()
|
||||||
|
openedInfo, err := f.Stat()
|
||||||
|
if err != nil || !openedInfo.Mode().IsRegular() || !os.SameFile(info, openedInfo) || openedInfo.Size() != info.Size() {
|
||||||
|
return nil, errors.New("file changed before read")
|
||||||
|
}
|
||||||
|
data, err := io.ReadAll(io.LimitReader(f, limit+1))
|
||||||
|
if err != nil || int64(len(data)) > limit || int64(len(data)) != openedInfo.Size() {
|
||||||
|
return nil, errors.New("file read exceeds bounds or changed")
|
||||||
|
}
|
||||||
|
return data, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
type inventory struct {
|
||||||
|
children map[string]*inventory
|
||||||
|
digest string
|
||||||
|
}
|
||||||
|
|
||||||
|
func validate(m Manifest) (*inventory, error) {
|
||||||
|
if m.ProtocolVersion != 1 || m.Runtime != "compose" || !idPattern.MatchString(m.AppID) || !versionPattern.MatchString(m.Version) {
|
||||||
|
return nil, errors.New("unsupported or invalid manifest identity")
|
||||||
|
}
|
||||||
|
if len(m.Platforms) < 1 || len(m.Platforms) > 2 {
|
||||||
|
return nil, errors.New("invalid platforms")
|
||||||
|
}
|
||||||
|
platforms := make(map[string]bool)
|
||||||
|
for _, platform := range m.Platforms {
|
||||||
|
if (platform != "linux/amd64" && platform != "linux/arm64") || platforms[platform] {
|
||||||
|
return nil, errors.New("invalid platforms")
|
||||||
|
}
|
||||||
|
platforms[platform] = true
|
||||||
|
}
|
||||||
|
if len(m.Components) < 1 || len(m.Components) > 32 {
|
||||||
|
return nil, errors.New("invalid component count")
|
||||||
|
}
|
||||||
|
names := make(map[string]bool)
|
||||||
|
for _, c := range m.Components {
|
||||||
|
if !idPattern.MatchString(c.Name) || names[c.Name] || !validImage(c.Image) {
|
||||||
|
return nil, errors.New("invalid component")
|
||||||
|
}
|
||||||
|
names[c.Name] = true
|
||||||
|
}
|
||||||
|
if len(m.Files) < 1 || len(m.Files) > 128 {
|
||||||
|
return nil, errors.New("invalid file count")
|
||||||
|
}
|
||||||
|
tree := &inventory{children: map[string]*inventory{"manifest.json": {}}}
|
||||||
|
paths := make(map[string]bool)
|
||||||
|
for _, file := range m.Files {
|
||||||
|
if !validPath(file.Path) || file.Path == "manifest.json" || !digestPattern.MatchString(file.Digest) || paths[file.Path] {
|
||||||
|
return nil, errors.New("invalid file declaration")
|
||||||
|
}
|
||||||
|
paths[file.Path] = true
|
||||||
|
node := tree
|
||||||
|
parts := strings.Split(file.Path, "/")
|
||||||
|
for i, part := range parts {
|
||||||
|
next, exists := node.children[part]
|
||||||
|
if i == len(parts)-1 {
|
||||||
|
if exists {
|
||||||
|
return nil, errors.New("conflicting file paths")
|
||||||
|
}
|
||||||
|
node.children[part] = &inventory{digest: file.Digest}
|
||||||
|
} else {
|
||||||
|
if exists && next.children == nil {
|
||||||
|
return nil, errors.New("conflicting file paths")
|
||||||
|
}
|
||||||
|
if !exists {
|
||||||
|
next = &inventory{children: make(map[string]*inventory)}
|
||||||
|
node.children[part] = next
|
||||||
|
}
|
||||||
|
node = next
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !paths[m.Entrypoint] {
|
||||||
|
return nil, errors.New("entrypoint must be a listed file")
|
||||||
|
}
|
||||||
|
return tree, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func validImage(image string) bool {
|
||||||
|
repo, pin, found := strings.Cut(image, "@")
|
||||||
|
if !found || !digestPattern.MatchString(pin) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
for _, part := range strings.Split(repo, "/") {
|
||||||
|
if !repositoryPart.MatchString(part) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
func validPath(path string) bool {
|
||||||
|
if len(path) == 0 || len(path) > 240 {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
for _, part := range strings.Split(path, "/") {
|
||||||
|
if len(part) > 100 || !pathPart.MatchString(part) || strings.HasSuffix(part, ".") {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
base, _, _ := strings.Cut(part, ".")
|
||||||
|
switch base {
|
||||||
|
case "con", "prn", "aux", "nul", "conin$", "conout$":
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
if len(base) == 4 && (strings.HasPrefix(base, "com") || strings.HasPrefix(base, "lpt")) && base[3] >= '1' && base[3] <= '9' {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
// readInventory enumerates only declared directories, one entry at a time. Each
|
||||||
|
// directory consumes at most its declared child count plus one entry; extras
|
||||||
|
// fail immediately, with no unbounded ReadDir or recursive filesystem walk.
|
||||||
|
func readInventory(root *os.Root, node *inventory, prefix string, files map[string][]byte, remaining *int64) error {
|
||||||
|
dir, err := root.Open(".")
|
||||||
|
if err != nil {
|
||||||
|
return errors.New("cannot enumerate bundle")
|
||||||
|
}
|
||||||
|
defer dir.Close()
|
||||||
|
seen := make(map[string]bool, len(node.children))
|
||||||
|
for {
|
||||||
|
entries, err := dir.ReadDir(1)
|
||||||
|
if err != nil && err != io.EOF {
|
||||||
|
return errors.New("cannot enumerate bundle")
|
||||||
|
}
|
||||||
|
if len(entries) == 0 {
|
||||||
|
if err == io.EOF {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
return errors.New("invalid directory enumeration")
|
||||||
|
}
|
||||||
|
entry := entries[0]
|
||||||
|
name := entry.Name()
|
||||||
|
child, exists := node.children[name]
|
||||||
|
if !exists || seen[name] {
|
||||||
|
return errors.New("unexpected bundle entry")
|
||||||
|
}
|
||||||
|
seen[name] = true
|
||||||
|
info, statErr := root.Lstat(name)
|
||||||
|
if statErr != nil || info.Mode()&os.ModeSymlink != 0 {
|
||||||
|
return errors.New("invalid bundle entry")
|
||||||
|
}
|
||||||
|
if child.children != nil {
|
||||||
|
if !info.IsDir() {
|
||||||
|
return errors.New("expected bundle directory")
|
||||||
|
}
|
||||||
|
sub, openErr := root.OpenRoot(name)
|
||||||
|
if openErr != nil {
|
||||||
|
return errors.New("cannot open bundle subdirectory")
|
||||||
|
}
|
||||||
|
openedInfo, statErr := sub.Stat(".")
|
||||||
|
if statErr != nil || !os.SameFile(info, openedInfo) {
|
||||||
|
sub.Close()
|
||||||
|
return errors.New("bundle subdirectory changed")
|
||||||
|
}
|
||||||
|
err := readInventory(sub, child, prefix+name+"/", files, remaining)
|
||||||
|
sub.Close()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if !info.Mode().IsRegular() {
|
||||||
|
return errors.New("expected regular bundle file")
|
||||||
|
}
|
||||||
|
if prefix == "" && name == "manifest.json" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
limit := min(fileLimit, *remaining)
|
||||||
|
data, readErr := readRegular(root, name, limit)
|
||||||
|
if readErr != nil {
|
||||||
|
return errors.New("cannot read bounded regular payload")
|
||||||
|
}
|
||||||
|
if hash(data) != child.digest {
|
||||||
|
return errors.New("payload digest mismatch")
|
||||||
|
}
|
||||||
|
*remaining -= int64(len(data))
|
||||||
|
files[prefix+name] = data
|
||||||
|
}
|
||||||
|
if len(seen) != len(node.children) {
|
||||||
|
return errors.New("missing bundle entry")
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,29 @@
|
|||||||
|
package appbundle_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"syscall"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestRejectFIFOWithoutOpening(t *testing.T) {
|
||||||
|
for _, name := range []string{"manifest.json", "config.txt", "extra"} {
|
||||||
|
t.Run(name, func(t *testing.T) {
|
||||||
|
files := basicFiles()
|
||||||
|
dir, pin := packageDir(t, manifest(files), files)
|
||||||
|
path := filepath.Join(dir, name)
|
||||||
|
if name != "extra" {
|
||||||
|
if err := os.Remove(path); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := syscall.Mkfifo(path, 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
// Opening a FIFO for reading would block. Verification must reject its
|
||||||
|
// metadata before opening it; the parent Linux run uses a test timeout.
|
||||||
|
rejected(t, dir, pin)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,408 @@
|
|||||||
|
package appbundle_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"crypto/sha256"
|
||||||
|
"encoding/hex"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"runtime"
|
||||||
|
"strings"
|
||||||
|
"syscall"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"server-deploy/internal/appbundle"
|
||||||
|
)
|
||||||
|
|
||||||
|
func digest(data []byte) string {
|
||||||
|
sum := sha256.Sum256(data)
|
||||||
|
return "sha256:" + hex.EncodeToString(sum[:])
|
||||||
|
}
|
||||||
|
|
||||||
|
func manifest(files map[string][]byte) map[string]any {
|
||||||
|
entries := []any{}
|
||||||
|
for path, data := range files {
|
||||||
|
entries = append(entries, map[string]any{"path": path, "digest": digest(data)})
|
||||||
|
}
|
||||||
|
return map[string]any{
|
||||||
|
"protocolVersion": 1, "appId": "demo-app", "version": "1.2.3",
|
||||||
|
"runtime": "compose", "entrypoint": "compose.yaml",
|
||||||
|
"platforms": []any{"linux/amd64", "linux/arm64"},
|
||||||
|
"components": []any{
|
||||||
|
map[string]any{"name": "api", "image": "registry.example/team/api@sha256:" + strings.Repeat("a", 64)},
|
||||||
|
map[string]any{"name": "db", "image": "db@sha256:" + strings.Repeat("b", 64)},
|
||||||
|
},
|
||||||
|
"files": entries,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func basicFiles() map[string][]byte {
|
||||||
|
return map[string][]byte{"compose.yaml": []byte("services: {}\n"), "config.txt": []byte("known fixture\n")}
|
||||||
|
}
|
||||||
|
|
||||||
|
func write(t *testing.T, dir, path string, data []byte) {
|
||||||
|
t.Helper()
|
||||||
|
full := filepath.Join(dir, filepath.FromSlash(path))
|
||||||
|
if err := os.MkdirAll(filepath.Dir(full), 0700); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(full, data, 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func packageDir(t *testing.T, m map[string]any, files map[string][]byte) (string, string) {
|
||||||
|
t.Helper()
|
||||||
|
dir := t.TempDir()
|
||||||
|
for path, data := range files {
|
||||||
|
write(t, dir, path, data)
|
||||||
|
}
|
||||||
|
return dir, saveManifest(t, dir, m)
|
||||||
|
}
|
||||||
|
|
||||||
|
func saveManifest(t *testing.T, dir string, m map[string]any) string {
|
||||||
|
t.Helper()
|
||||||
|
raw, err := json.Marshal(m)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
write(t, dir, "manifest.json", raw)
|
||||||
|
return digest(raw)
|
||||||
|
}
|
||||||
|
|
||||||
|
func rejected(t *testing.T, dir, pin string) error {
|
||||||
|
t.Helper()
|
||||||
|
v, err := appbundle.Verify(dir, pin)
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("accepted invalid package")
|
||||||
|
}
|
||||||
|
if v.Digest != "" || v.Files != nil || v.Manifest.AppID != "" {
|
||||||
|
t.Fatal("returned partial verified data on error")
|
||||||
|
}
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestVerifyReturnsAuthenticatedBytes(t *testing.T) {
|
||||||
|
files := basicFiles()
|
||||||
|
files["nested/config/data.txt"] = []byte("nested bytes")
|
||||||
|
dir, pin := packageDir(t, manifest(files), files)
|
||||||
|
v, err := appbundle.Verify(dir, pin)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if v.Digest != pin || v.Manifest.AppID != "demo-app" || len(v.Manifest.Components) != 2 || len(v.Files) != 3 {
|
||||||
|
t.Fatalf("incorrect verified result: digest=%q files=%d", v.Digest, len(v.Files))
|
||||||
|
}
|
||||||
|
for path, data := range files {
|
||||||
|
if !bytes.Equal(v.Files[path], data) {
|
||||||
|
t.Fatalf("incorrect bytes for %s", path)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
write(t, dir, "compose.yaml", []byte("later mutation"))
|
||||||
|
if !bytes.Equal(v.Files["compose.yaml"], []byte("services: {}\n")) {
|
||||||
|
t.Fatal("verified bytes changed after disk mutation")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestManifestPinAndParseBoundary(t *testing.T) {
|
||||||
|
for _, tc := range []struct{ name, raw, pin string }{
|
||||||
|
{"wrong pin", "not json", "sha256:" + strings.Repeat("0", 64)},
|
||||||
|
{"empty pin", "{}", ""},
|
||||||
|
{"bare pin", "{}", strings.Repeat("a", 64)},
|
||||||
|
{"uppercase pin", "{}", "sha256:" + strings.Repeat("A", 64)},
|
||||||
|
{"malformed authenticated manifest", "{SECRET_CONTENT", "auto"},
|
||||||
|
{"oversized manifest", strings.Repeat(" ", (1<<20)+1), "auto"},
|
||||||
|
} {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
write(t, dir, "manifest.json", []byte(tc.raw))
|
||||||
|
pin := tc.pin
|
||||||
|
if pin == "auto" {
|
||||||
|
pin = digest([]byte(tc.raw))
|
||||||
|
}
|
||||||
|
err := rejected(t, dir, pin)
|
||||||
|
if strings.Contains(err.Error(), "SECRET_CONTENT") {
|
||||||
|
t.Fatal("manifest bytes leaked in error")
|
||||||
|
}
|
||||||
|
if tc.name == "wrong pin" && !strings.Contains(strings.ToLower(err.Error()), "digest") {
|
||||||
|
t.Fatal("digest must be checked before JSON parsing")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestManifestValidation(t *testing.T) {
|
||||||
|
cases := []struct {
|
||||||
|
name string
|
||||||
|
change func(map[string]any)
|
||||||
|
}{
|
||||||
|
{"protocol", func(m map[string]any) { m["protocolVersion"] = 2 }},
|
||||||
|
{"runtime", func(m map[string]any) { m["runtime"] = "shell" }},
|
||||||
|
{"app id", func(m map[string]any) { m["appId"] = "Demo" }},
|
||||||
|
{"long id", func(m map[string]any) { m["appId"] = strings.Repeat("a", 49) }},
|
||||||
|
{"version", func(m map[string]any) { m["version"] = "1.2.3-beta" }},
|
||||||
|
{"empty platforms", func(m map[string]any) { m["platforms"] = []any{} }},
|
||||||
|
{"platform", func(m map[string]any) { m["platforms"] = []any{"windows/amd64"} }},
|
||||||
|
{"duplicate platform", func(m map[string]any) { m["platforms"] = []any{"linux/amd64", "linux/amd64"} }},
|
||||||
|
{"null platform", func(m map[string]any) { m["platforms"] = []any{nil} }},
|
||||||
|
{"empty components", func(m map[string]any) { m["components"] = []any{} }},
|
||||||
|
{"duplicate component", func(m map[string]any) { c := m["components"].([]any); c[1].(map[string]any)["name"] = "api" }},
|
||||||
|
{"invalid component name", func(m map[string]any) { m["components"].([]any)[0].(map[string]any)["name"] = "1api" }},
|
||||||
|
{"missing component field", func(m map[string]any) { delete(m["components"].([]any)[0].(map[string]any), "image") }},
|
||||||
|
{"unknown component field", func(m map[string]any) { m["components"].([]any)[0].(map[string]any)["secret"] = "SECRET_CONTENT" }},
|
||||||
|
{"null component field", func(m map[string]any) { m["components"].([]any)[0].(map[string]any)["image"] = nil }},
|
||||||
|
{"null component", func(m map[string]any) { m["components"] = []any{nil} }},
|
||||||
|
{"too many components", func(m map[string]any) {
|
||||||
|
c := []any{}
|
||||||
|
for i := 0; i < 33; i++ {
|
||||||
|
c = append(c, map[string]any{"name": "c-" + strings.Repeat("a", i), "image": "api@sha256:" + strings.Repeat("a", 64)})
|
||||||
|
}
|
||||||
|
m["components"] = c
|
||||||
|
}},
|
||||||
|
{"empty files", func(m map[string]any) { m["files"] = []any{} }},
|
||||||
|
{"duplicate file", func(m map[string]any) { f := m["files"].([]any); m["files"] = append(f, f[0]) }},
|
||||||
|
{"null file", func(m map[string]any) { m["files"] = []any{nil} }},
|
||||||
|
{"bad file digest", func(m map[string]any) {
|
||||||
|
m["files"].([]any)[0].(map[string]any)["digest"] = "sha256:" + strings.Repeat("A", 64)
|
||||||
|
}},
|
||||||
|
{"entrypoint absent", func(m map[string]any) { m["entrypoint"] = "missing.yaml" }},
|
||||||
|
{"null files", func(m map[string]any) { m["files"] = nil }},
|
||||||
|
{"missing top field", func(m map[string]any) { delete(m, "appId") }},
|
||||||
|
{"unknown top field", func(m map[string]any) { m["secret"] = "SECRET_CONTENT" }},
|
||||||
|
{"case alias", func(m map[string]any) { m["AppId"] = m["appId"]; delete(m, "appId") }},
|
||||||
|
}
|
||||||
|
for _, tc := range cases {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
files := basicFiles()
|
||||||
|
m := manifest(files)
|
||||||
|
tc.change(m)
|
||||||
|
dir, pin := packageDir(t, m, files)
|
||||||
|
err := rejected(t, dir, pin)
|
||||||
|
if strings.Contains(err.Error(), "SECRET_CONTENT") {
|
||||||
|
t.Fatal("manifest content leaked")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRejectImageSubsetViolations(t *testing.T) {
|
||||||
|
for _, repo := range []string{"api:latest", "api", "host:5000/api", "UPPER/api", "a//b", "/api", "api/", "a/$b", "a;b", "a b", ".api", "api."} {
|
||||||
|
t.Run(repo, func(t *testing.T) {
|
||||||
|
files := basicFiles()
|
||||||
|
m := manifest(files)
|
||||||
|
img := repo + "@sha256:" + strings.Repeat("a", 64)
|
||||||
|
if repo == "api:latest" || repo == "api" {
|
||||||
|
img = repo
|
||||||
|
}
|
||||||
|
m["components"].([]any)[0].(map[string]any)["image"] = img
|
||||||
|
dir, pin := packageDir(t, m, files)
|
||||||
|
rejected(t, dir, pin)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRejectNonportablePaths(t *testing.T) {
|
||||||
|
for _, path := range []string{"../escape", "a/../b", "a/./b", "/root", "c:/file", `a\b`, "a//b", "a/", ".env", "a/.secret", "a/secret ", "a/secret.", "con", "aux.txt", "a/com1.log", "lpt9", "CLOCK$", "UPPER.txt", "a:stream", "manifest.json", strings.Repeat("a", 101), strings.Repeat("a/", 120) + "b"} {
|
||||||
|
t.Run(path, func(t *testing.T) {
|
||||||
|
files := basicFiles()
|
||||||
|
m := manifest(files)
|
||||||
|
m["files"] = append(m["files"].([]any), map[string]any{"path": path, "digest": digest(nil)})
|
||||||
|
dir, pin := packageDir(t, m, files)
|
||||||
|
rejected(t, dir, pin)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExactInventory(t *testing.T) {
|
||||||
|
for _, mode := range []string{"changed", "missing", "extra", "secret", "directory", "nested extra", "file directory conflict", "case collision"} {
|
||||||
|
t.Run(mode, func(t *testing.T) {
|
||||||
|
files := basicFiles()
|
||||||
|
files["nested/file.txt"] = []byte("nested")
|
||||||
|
m := manifest(files)
|
||||||
|
dir, pin := packageDir(t, m, files)
|
||||||
|
switch mode {
|
||||||
|
case "changed":
|
||||||
|
write(t, dir, "config.txt", []byte("SECRET_CONTENT"))
|
||||||
|
case "missing":
|
||||||
|
if err := os.Remove(filepath.Join(dir, "config.txt")); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
case "extra":
|
||||||
|
write(t, dir, "extra.txt", []byte("SECRET_CONTENT"))
|
||||||
|
case "secret":
|
||||||
|
write(t, dir, ".env", []byte("SECRET_CONTENT"))
|
||||||
|
case "directory":
|
||||||
|
if err := os.Mkdir(filepath.Join(dir, "unused"), 0700); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
case "nested extra":
|
||||||
|
write(t, dir, "nested/extra.txt", []byte("SECRET_CONTENT"))
|
||||||
|
case "file directory conflict":
|
||||||
|
m["files"] = append(m["files"].([]any), map[string]any{"path": "config.txt/child", "digest": digest(nil)})
|
||||||
|
pin = saveManifest(t, dir, m)
|
||||||
|
case "case collision":
|
||||||
|
m["files"] = append(m["files"].([]any), map[string]any{"path": "CONFIG.txt", "digest": digest(nil)})
|
||||||
|
pin = saveManifest(t, dir, m)
|
||||||
|
}
|
||||||
|
err := rejected(t, dir, pin)
|
||||||
|
if strings.Contains(err.Error(), "SECRET_CONTENT") {
|
||||||
|
t.Fatal("file contents leaked")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPayloadBounds(t *testing.T) {
|
||||||
|
for _, tc := range []struct {
|
||||||
|
name string
|
||||||
|
sizes []int
|
||||||
|
valid bool
|
||||||
|
}{
|
||||||
|
{"file at limit", []int{4 << 20}, true},
|
||||||
|
{"file over limit", []int{(4 << 20) + 1}, false},
|
||||||
|
{"total at limit", []int{4 << 20, 4 << 20, 4 << 20, 4 << 20}, true},
|
||||||
|
{"total over limit", []int{4 << 20, 4 << 20, 4 << 20, 4 << 20, 1}, false},
|
||||||
|
} {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
files := map[string][]byte{}
|
||||||
|
for i, size := range tc.sizes {
|
||||||
|
path := string(rune('a'+i)) + ".txt"
|
||||||
|
if i == 0 {
|
||||||
|
path = "compose.yaml"
|
||||||
|
}
|
||||||
|
files[path] = bytes.Repeat([]byte("x"), size)
|
||||||
|
}
|
||||||
|
dir, pin := packageDir(t, manifest(files), files)
|
||||||
|
if tc.valid {
|
||||||
|
if _, err := appbundle.Verify(dir, pin); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
rejected(t, dir, pin)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func symlink(t *testing.T, target, link string) {
|
||||||
|
t.Helper()
|
||||||
|
if err := os.Symlink(target, link); err != nil {
|
||||||
|
if runtime.GOOS == "windows" && errors.Is(err, syscall.Errno(1314)) {
|
||||||
|
t.Skip("Windows symlink privilege unavailable")
|
||||||
|
}
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRejectSymlinks(t *testing.T) {
|
||||||
|
for _, mode := range []string{"file", "manifest", "intermediate", "extra", "root", "root trailing separator"} {
|
||||||
|
t.Run(mode, func(t *testing.T) {
|
||||||
|
files := basicFiles()
|
||||||
|
dir, pin := packageDir(t, manifest(files), files)
|
||||||
|
switch mode {
|
||||||
|
case "file", "manifest":
|
||||||
|
name := "config.txt"
|
||||||
|
if mode == "manifest" {
|
||||||
|
name = "manifest.json"
|
||||||
|
}
|
||||||
|
path := filepath.Join(dir, name)
|
||||||
|
target := filepath.Join(t.TempDir(), "target")
|
||||||
|
if err := os.Rename(path, target); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
symlink(t, target, path)
|
||||||
|
case "intermediate":
|
||||||
|
outside := t.TempDir()
|
||||||
|
write(t, outside, "data.txt", []byte("outside"))
|
||||||
|
symlink(t, outside, filepath.Join(dir, "nested"))
|
||||||
|
m := manifest(files)
|
||||||
|
m["files"] = append(m["files"].([]any), map[string]any{"path": "nested/data.txt", "digest": digest([]byte("outside"))})
|
||||||
|
pin = saveManifest(t, dir, m)
|
||||||
|
case "extra":
|
||||||
|
symlink(t, t.TempDir(), filepath.Join(dir, "extra"))
|
||||||
|
case "root", "root trailing separator":
|
||||||
|
link := filepath.Join(t.TempDir(), "bundle")
|
||||||
|
symlink(t, dir, link)
|
||||||
|
dir = link
|
||||||
|
if mode == "root trailing separator" {
|
||||||
|
dir += string(os.PathSeparator)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
rejected(t, dir, pin)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAbsoluteDirectoryRequired(t *testing.T) { rejected(t, ".", "sha256:"+strings.Repeat("a", 64)) }
|
||||||
|
|
||||||
|
func TestRejectTrailingAndDuplicateJSON(t *testing.T) {
|
||||||
|
for _, suffix := range []string{" {}", ",\"appId\":\"other\"}"} {
|
||||||
|
t.Run(suffix, func(t *testing.T) {
|
||||||
|
files := basicFiles()
|
||||||
|
dir, _ := packageDir(t, manifest(files), files)
|
||||||
|
raw, err := os.ReadFile(filepath.Join(dir, "manifest.json"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if strings.HasPrefix(suffix, ",") {
|
||||||
|
raw = raw[:len(raw)-1]
|
||||||
|
}
|
||||||
|
raw = append(raw, []byte(suffix)...)
|
||||||
|
write(t, dir, "manifest.json", raw)
|
||||||
|
rejected(t, dir, digest(raw))
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestManifestExactByteLimit(t *testing.T) {
|
||||||
|
files := basicFiles()
|
||||||
|
dir, _ := packageDir(t, manifest(files), files)
|
||||||
|
raw, err := os.ReadFile(filepath.Join(dir, "manifest.json"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
originalPin := digest(raw)
|
||||||
|
raw = append(raw, bytes.Repeat([]byte(" "), (1<<20)-len(raw))...)
|
||||||
|
write(t, dir, "manifest.json", raw)
|
||||||
|
rejected(t, dir, originalPin) // Whitespace must change the exact-byte pin.
|
||||||
|
if _, err := appbundle.Verify(dir, digest(raw)); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInventoryCountBoundaries(t *testing.T) {
|
||||||
|
for _, n := range []int{128, 129} {
|
||||||
|
t.Run(fmt.Sprint(n), func(t *testing.T) {
|
||||||
|
files := map[string][]byte{"compose.yaml": {}}
|
||||||
|
for i := 1; i < n; i++ {
|
||||||
|
files[fmt.Sprintf("file-%d.txt", i)] = []byte{}
|
||||||
|
}
|
||||||
|
dir, pin := packageDir(t, manifest(files), files)
|
||||||
|
if n == 128 {
|
||||||
|
if _, err := appbundle.Verify(dir, pin); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
rejected(t, dir, pin)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestComponentCountBoundary(t *testing.T) {
|
||||||
|
files := basicFiles()
|
||||||
|
m := manifest(files)
|
||||||
|
components := []any{}
|
||||||
|
for i := 0; i < 32; i++ {
|
||||||
|
components = append(components, map[string]any{"name": fmt.Sprintf("component-%d", i), "image": "api@sha256:" + strings.Repeat("a", 64)})
|
||||||
|
}
|
||||||
|
m["components"] = components
|
||||||
|
m["appId"] = strings.Repeat("a", 48)
|
||||||
|
dir, pin := packageDir(t, m, files)
|
||||||
|
if _, err := appbundle.Verify(dir, pin); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,90 @@
|
|||||||
|
# Staged Docker repository authentication
|
||||||
|
|
||||||
|
`deployctl verify-repository` accepts strict JSON with `directory` (absolute trusted
|
||||||
|
staging directory), `suite`, `architecture`, and `versions` (exact version strings
|
||||||
|
for docker-ce, docker-ce-cli, containerd.io, docker-buildx-plugin and
|
||||||
|
docker-compose-plugin). It does not select a latest version or resolve dependencies.
|
||||||
|
|
||||||
|
The directory must contain `docker.asc`, `Release`, `Release.gpg` and uncompressed
|
||||||
|
`Packages`. The caller obtains these from the Docker Ubuntu repository. No runtime
|
||||||
|
network request is performed by this command. Limits are 1 MiB, 1 MiB, 64 KiB and
|
||||||
|
16 MiB respectively. Files must be nonempty regular files; symlinks are rejected.
|
||||||
|
Trusted ancestors and absence of concurrent writers are required. This is not an
|
||||||
|
atomic filesystem snapshot against hostile writers. Additional staging files are
|
||||||
|
ignored, never executed.
|
||||||
|
|
||||||
|
## Trust and verification
|
||||||
|
|
||||||
|
1. Exact armored key SHA-256 is pinned in source. Initial trust was bootstrapped
|
||||||
|
from `https://download.docker.com/linux/ubuntu/gpg`, not supplied by the request.
|
||||||
|
Primary fingerprint: `9DC858229FC7DD38854AE2D88D81803C0EBFCD88`.
|
||||||
|
Rotation or formatting changes require a reviewed source update; fail closed.
|
||||||
|
2. Linux `/usr/bin/gpg` dearmors into a new private temporary directory; `/usr/bin/gpgv`
|
||||||
|
verifies the detached signature against the copied Release bytes, with that
|
||||||
|
keyring and isolated homedir. No shell, ambient GnuPG config or personal keyring.
|
||||||
|
Each child has a 15-second timeout and bounded output. Errors do not expose raw
|
||||||
|
GnuPG diagnostics or metadata. Non-Linux or missing tools fail closed.
|
||||||
|
3. Successful process exit and a single accepted primary-fingerprint signature
|
||||||
|
are required; weak digests, expired/revoked/bad signatures and unknown status
|
||||||
|
types fail closed. SHA-256/384/512 are accepted.
|
||||||
|
4. Authenticated Release must describe Docker CE and the requested supported Ubuntu
|
||||||
|
suite, architecture and stable component. Date must be within 30 days and no
|
||||||
|
more than 10 minutes in the future; Valid-Until is enforced when present.
|
||||||
|
5. The exact uncompressed stable Packages entry's SHA-256 and byte size must match.
|
||||||
|
Five explicitly requested versions are resolved to authenticated index records;
|
||||||
|
the derived lock is checked by installplan's source/path/version constraints.
|
||||||
|
|
||||||
|
The machine clock, OS and installed GnuPG are trusted. This is not a persistent
|
||||||
|
anti-rollback ledger: an older authentic Release inside the freshness window can
|
||||||
|
pass, and a missing Valid-Until is governed by the local 30-day policy. No online
|
||||||
|
key revocation lookup is performed. Key pin maintenance is an operator responsibility.
|
||||||
|
|
||||||
|
## Result boundaries
|
||||||
|
|
||||||
|
`repositoryAuthenticated: true` attests to these metadata bytes at `verifiedAt`.
|
||||||
|
For `verify-repository`, `packageBytesVerified: false` and `executable: false` remain explicit: no deb bytes,
|
||||||
|
Ubuntu dependency repository, dependency closure, installation scripts, system
|
||||||
|
compatibility or service behavior have been verified. The result is not a signed
|
||||||
|
capability. `plan-environment` still treats a supplied lock as untrusted and does
|
||||||
|
not accept a caller's authentication claim to clear its trust blockers.
|
||||||
|
|
||||||
|
Signature scratch data is removed on normal return; a killed process can leave its own
|
||||||
|
private temporary directory. Deployment writes remain disabled. `writesEnabled`
|
||||||
|
in `version` refers to deployment writes, not verification scratch files.
|
||||||
|
|
||||||
|
`scripts/probe-docker-repository.sh /absolute/path/to/deployctl` is an optional
|
||||||
|
local online integration check. It downloads public metadata over HTTPS into a
|
||||||
|
new temporary directory, tests real authentication and rejects tampering. Versions
|
||||||
|
selected by this test are fixtures, not recommended installation versions. It
|
||||||
|
does not install packages, alter APT, use SSH or touch application data.
|
||||||
|
|
||||||
|
## Verify actual deb bytes
|
||||||
|
|
||||||
|
`verify-artifacts` requires the same request fields as `verify-repository`, plus
|
||||||
|
`artifactDirectory`: an absolute trusted directory containing exactly the five deb
|
||||||
|
files named by the basename of each authenticated `Filename`. No other entries,
|
||||||
|
subdirectories, symlinks or special files are accepted. The two directories must
|
||||||
|
not be concurrently modified. Repository signatures and metadata are verified
|
||||||
|
anew in the same call; the command accepts neither a supplied lock nor trust flags.
|
||||||
|
|
||||||
|
Each file is streamed through SHA-256 with a 64 KiB copy buffer and its signed
|
||||||
|
index size as the read bound (plus one overflow-detection byte). The existing
|
||||||
|
lock policy caps each file at 512 MiB and requires exactly five files. A last-file
|
||||||
|
failure rejects the entire result. No partial successful report is emitted.
|
||||||
|
The verifier never unpacks a deb or executes its contents. It does not change
|
||||||
|
the staged files. Metadata identity checks supplement, not replace, the trusted
|
||||||
|
directory/no concurrent writer requirement.
|
||||||
|
|
||||||
|
Only `verify-artifacts` may set `packageBytesVerified: true`; `executable` stays
|
||||||
|
false. This verifies the selected five Docker package bytes, not the complete
|
||||||
|
Ubuntu dependency closure, package-internal safety or installation compatibility.
|
||||||
|
It is a point-in-time observation: do not reuse this JSON to authorize later
|
||||||
|
execution of paths that may have changed. A future installer must recheck or
|
||||||
|
own immutable verified snapshots under its operation lock.
|
||||||
|
|
||||||
|
Set `DEPLOYCTL_ONLINE_ARTIFACT_PROBE=1` along with
|
||||||
|
`DEPLOYCTL_ONLINE_REPOSITORY_PROBE=1` when running `scripts/verify-linux.sh` to
|
||||||
|
also download the five real public deb fixtures into a new local temporary
|
||||||
|
directory. The optional probe verifies all files, then changes one byte without
|
||||||
|
changing length and asserts rejection. It retains test files for inspection;
|
||||||
|
they are not installed and one is intentionally damaged by the negative test.
|
||||||
@@ -0,0 +1,117 @@
|
|||||||
|
package aptrepo
|
||||||
|
|
||||||
|
import (
|
||||||
|
"crypto/sha256"
|
||||||
|
"encoding/hex"
|
||||||
|
"errors"
|
||||||
|
"io"
|
||||||
|
"os"
|
||||||
|
"path"
|
||||||
|
"path/filepath"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"server-deploy/internal/installplan"
|
||||||
|
)
|
||||||
|
|
||||||
|
// VerifyArtifacts authenticates repository metadata anew before checking the
|
||||||
|
// selected five deb files. It never accepts a caller-asserted authenticated lock.
|
||||||
|
// The staging directories and ancestors must be trusted and not concurrently
|
||||||
|
// modified. This is observation evidence, not a capability to later execute paths.
|
||||||
|
func VerifyArtifacts(metadataDirectory, artifactDirectory, suite, arch string, versions map[string]string, now time.Time) (Result, error) {
|
||||||
|
result, err := Verify(metadataDirectory, suite, arch, versions, now)
|
||||||
|
if err != nil {
|
||||||
|
return Result{}, err
|
||||||
|
}
|
||||||
|
if err := verifyArtifacts(artifactDirectory, result.Lock); err != nil {
|
||||||
|
return Result{}, err
|
||||||
|
}
|
||||||
|
result.PackageBytesVerified = true
|
||||||
|
return result, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func verifyArtifacts(directory string, lock installplan.Lock) error {
|
||||||
|
reject := errors.New("invalid staged Docker artifacts")
|
||||||
|
if _, err := installplan.Validate(lock, lock.Suite, lock.Architecture); err != nil {
|
||||||
|
return reject
|
||||||
|
}
|
||||||
|
if !filepath.IsAbs(directory) {
|
||||||
|
return reject
|
||||||
|
}
|
||||||
|
directory = filepath.Clean(directory)
|
||||||
|
before, err := os.Lstat(directory)
|
||||||
|
if err != nil || !before.IsDir() || before.Mode()&os.ModeSymlink != 0 {
|
||||||
|
return reject
|
||||||
|
}
|
||||||
|
root, err := os.OpenRoot(directory)
|
||||||
|
if err != nil {
|
||||||
|
return reject
|
||||||
|
}
|
||||||
|
defer root.Close()
|
||||||
|
opened, err := root.Stat(".")
|
||||||
|
if err != nil || !os.SameFile(before, opened) {
|
||||||
|
return reject
|
||||||
|
}
|
||||||
|
expected := make(map[string]installplan.Package, len(lock.Packages))
|
||||||
|
for _, p := range lock.Packages {
|
||||||
|
expected[path.Base(p.Filename)] = p
|
||||||
|
}
|
||||||
|
dir, err := root.Open(".")
|
||||||
|
if err != nil {
|
||||||
|
return reject
|
||||||
|
}
|
||||||
|
defer dir.Close()
|
||||||
|
// Enumerate at most the expected count plus one, never an unbounded directory.
|
||||||
|
seen := make(map[string]bool)
|
||||||
|
for {
|
||||||
|
entries, err := dir.ReadDir(1)
|
||||||
|
if err != nil && err != io.EOF {
|
||||||
|
return reject
|
||||||
|
}
|
||||||
|
if len(entries) == 0 {
|
||||||
|
if err == io.EOF {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
return reject
|
||||||
|
}
|
||||||
|
name := entries[0].Name()
|
||||||
|
p, ok := expected[name]
|
||||||
|
if !ok || seen[name] {
|
||||||
|
return reject
|
||||||
|
}
|
||||||
|
seen[name] = true
|
||||||
|
if err := verifyArtifact(root, name, p); err != nil {
|
||||||
|
return reject
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(seen) != len(expected) {
|
||||||
|
return reject
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func verifyArtifact(root *os.Root, name string, p installplan.Package) error {
|
||||||
|
reject := errors.New("artifact size or digest mismatch")
|
||||||
|
before, err := root.Lstat(name)
|
||||||
|
if err != nil || !before.Mode().IsRegular() || before.Size() != int64(p.Size) {
|
||||||
|
return reject
|
||||||
|
}
|
||||||
|
file, err := root.Open(name)
|
||||||
|
if err != nil {
|
||||||
|
return reject
|
||||||
|
}
|
||||||
|
defer file.Close()
|
||||||
|
opened, err := file.Stat()
|
||||||
|
if err != nil || !opened.Mode().IsRegular() || !os.SameFile(before, opened) || opened.Size() != before.Size() {
|
||||||
|
return reject
|
||||||
|
}
|
||||||
|
digest := sha256.New()
|
||||||
|
n, err := io.CopyBuffer(digest, io.LimitReader(file, int64(p.Size)+1), make([]byte, 64<<10))
|
||||||
|
if err != nil || n != int64(p.Size) || "sha256:"+hex.EncodeToString(digest.Sum(nil)) != p.Digest {
|
||||||
|
return reject
|
||||||
|
}
|
||||||
|
after, err := file.Stat()
|
||||||
|
if err != nil || after.Size() != opened.Size() || !after.ModTime().Equal(opened.ModTime()) {
|
||||||
|
return reject
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,44 @@
|
|||||||
|
package aptrepo
|
||||||
|
|
||||||
|
import (
|
||||||
|
"os"
|
||||||
|
"path"
|
||||||
|
"path/filepath"
|
||||||
|
"syscall"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestArtifactRejectsSymlinksAndFIFO(t *testing.T) {
|
||||||
|
for _, kind := range []string{"root-link", "file-link", "fifo"} {
|
||||||
|
t.Run(kind, func(t *testing.T) {
|
||||||
|
dir, lock := artifactFixture(t)
|
||||||
|
filename := filepath.Join(dir, path.Base(lock.Packages[0].Filename))
|
||||||
|
switch kind {
|
||||||
|
case "root-link":
|
||||||
|
link := filepath.Join(t.TempDir(), "link")
|
||||||
|
if err := os.Symlink(dir, link); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
dir = link + "/"
|
||||||
|
case "file-link":
|
||||||
|
target := filepath.Join(t.TempDir(), "target")
|
||||||
|
if err := os.Rename(filename, target); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.Symlink(target, filename); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
case "fifo":
|
||||||
|
if err := os.Remove(filename); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := syscall.Mkfifo(filename, 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := verifyArtifacts(dir, lock); err == nil {
|
||||||
|
t.Fatal("unsafe file accepted")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,99 @@
|
|||||||
|
package aptrepo
|
||||||
|
|
||||||
|
import (
|
||||||
|
"crypto/sha256"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"path"
|
||||||
|
"path/filepath"
|
||||||
|
"reflect"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"server-deploy/internal/installplan"
|
||||||
|
)
|
||||||
|
|
||||||
|
func artifactFixture(t *testing.T) (string, installplan.Lock) {
|
||||||
|
t.Helper()
|
||||||
|
dir := t.TempDir()
|
||||||
|
index := fixtureIndex()
|
||||||
|
lock, err := Resolve(fixtureRelease(index), index, "resolute", "amd64", fixturePins(), fixtureNow)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
for i := range lock.Packages {
|
||||||
|
p := &lock.Packages[i]
|
||||||
|
data := []byte("artifact fixture " + p.Name)
|
||||||
|
p.Size = uint64(len(data))
|
||||||
|
p.Digest = fmt.Sprintf("sha256:%x", sha256.Sum256(data))
|
||||||
|
if err := os.WriteFile(filepath.Join(dir, path.Base(p.Filename)), data, 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return dir, lock
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestArtifactBytesAndIdentity(t *testing.T) {
|
||||||
|
dir, lock := artifactFixture(t)
|
||||||
|
before := lock
|
||||||
|
before.Packages = append([]installplan.Package(nil), lock.Packages...)
|
||||||
|
if err := verifyArtifacts(dir, lock); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if !reflect.DeepEqual(before, lock) {
|
||||||
|
t.Fatal("lock mutated")
|
||||||
|
}
|
||||||
|
for _, p := range lock.Packages {
|
||||||
|
data, err := os.ReadFile(filepath.Join(dir, path.Base(p.Filename)))
|
||||||
|
if err != nil || string(data) != "artifact fixture "+p.Name {
|
||||||
|
t.Fatal("artifact modified")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestArtifactRejectsTamperWithoutPartialSuccess(t *testing.T) {
|
||||||
|
for _, kind := range []string{"same-size", "truncated", "extra-byte", "missing", "directory", "bad-lock", "relative-root", "extra-file"} {
|
||||||
|
t.Run(kind, func(t *testing.T) {
|
||||||
|
dir, lock := artifactFixture(t)
|
||||||
|
p := lock.Packages[4] // The last artifact must fail the whole set.
|
||||||
|
filename := filepath.Join(dir, path.Base(p.Filename))
|
||||||
|
switch kind {
|
||||||
|
case "same-size":
|
||||||
|
data := []byte("artifact fixture " + p.Name)
|
||||||
|
data[0] = 'X'
|
||||||
|
if err := os.WriteFile(filename, data, 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
case "truncated":
|
||||||
|
if err := os.Truncate(filename, int64(p.Size)-1); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
case "extra-byte":
|
||||||
|
if err := os.Truncate(filename, int64(p.Size)+1); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
case "missing":
|
||||||
|
if err := os.Remove(filename); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
case "directory":
|
||||||
|
if err := os.Remove(filename); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.Mkdir(filename, 0700); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
case "bad-lock":
|
||||||
|
lock.Packages[0].Filename = "../../secret.deb"
|
||||||
|
case "relative-root":
|
||||||
|
dir = "relative"
|
||||||
|
case "extra-file":
|
||||||
|
if err := os.WriteFile(filepath.Join(dir, "unexpected.deb"), []byte("x"), 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := verifyArtifacts(dir, lock); err == nil {
|
||||||
|
t.Fatal("invalid artifacts accepted")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,232 @@
|
|||||||
|
package aptrepo
|
||||||
|
|
||||||
|
import (
|
||||||
|
"crypto/sha256"
|
||||||
|
"encoding/hex"
|
||||||
|
"errors"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
"unicode"
|
||||||
|
"unicode/utf8"
|
||||||
|
|
||||||
|
"server-deploy/internal/installplan"
|
||||||
|
)
|
||||||
|
|
||||||
|
var metadataError = errors.New("invalid Docker repository metadata")
|
||||||
|
|
||||||
|
var pinnedPackageNames = [...]string{"docker-ce", "docker-ce-cli", "containerd.io", "docker-buildx-plugin", "docker-compose-plugin"}
|
||||||
|
|
||||||
|
// Resolve parses Release bytes already authenticated by the caller and binds
|
||||||
|
// explicitly pinned packages to that Release. It does not verify signatures.
|
||||||
|
func Resolve(release, index []byte, suite, arch string, versions map[string]string, now time.Time) (installplan.Lock, error) {
|
||||||
|
if len(release) > 1<<20 || len(index) > 16<<20 || len(versions) != len(pinnedPackageNames) {
|
||||||
|
return installplan.Lock{}, metadataError
|
||||||
|
}
|
||||||
|
for _, name := range pinnedPackageNames {
|
||||||
|
if versions[name] == "" {
|
||||||
|
return installplan.Lock{}, metadataError
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
var fields map[string]string
|
||||||
|
if err := parseControl(release, func(stanza map[string]string) error {
|
||||||
|
if fields != nil {
|
||||||
|
return metadataError
|
||||||
|
}
|
||||||
|
fields = stanza
|
||||||
|
return nil
|
||||||
|
}); err != nil {
|
||||||
|
return installplan.Lock{}, metadataError
|
||||||
|
}
|
||||||
|
if fields["suite"] != suite || fields["origin"] != "Docker" || fields["label"] != "Docker CE" || !hasWord(fields["architectures"], arch) || !hasWord(fields["components"], "stable") {
|
||||||
|
return installplan.Lock{}, metadataError
|
||||||
|
}
|
||||||
|
date, err := time.Parse(time.RFC1123Z, fields["date"])
|
||||||
|
if err != nil || date.After(now.Add(10*time.Minute)) || date.Before(now.Add(-30*24*time.Hour)) {
|
||||||
|
return installplan.Lock{}, metadataError
|
||||||
|
}
|
||||||
|
if value, ok := fields["valid-until"]; ok {
|
||||||
|
expiry, err := time.Parse(time.RFC1123Z, value)
|
||||||
|
if err != nil || !expiry.After(now) || expiry.Before(date) {
|
||||||
|
return installplan.Lock{}, metadataError
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Bind the exact uncompressed index bytes before interpreting any records.
|
||||||
|
indexSum := sha256.Sum256(index)
|
||||||
|
expectedPath := "stable/binary-" + arch + "/Packages"
|
||||||
|
seenPaths := make(map[string]bool)
|
||||||
|
matched := false
|
||||||
|
for _, line := range strings.Split(fields["sha256"], "\n") {
|
||||||
|
if strings.TrimSpace(line) == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
entry := strings.Fields(line)
|
||||||
|
if len(entry) != 3 || !validSHA256(entry[0]) || seenPaths[entry[2]] {
|
||||||
|
return installplan.Lock{}, metadataError
|
||||||
|
}
|
||||||
|
seenPaths[entry[2]] = true
|
||||||
|
size, err := decimalSize(entry[1])
|
||||||
|
if err != nil {
|
||||||
|
return installplan.Lock{}, metadataError
|
||||||
|
}
|
||||||
|
if entry[2] == expectedPath {
|
||||||
|
if size != uint64(len(index)) || entry[0] != hex.EncodeToString(indexSum[:]) {
|
||||||
|
return installplan.Lock{}, metadataError
|
||||||
|
}
|
||||||
|
matched = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !matched {
|
||||||
|
return installplan.Lock{}, metadataError
|
||||||
|
}
|
||||||
|
|
||||||
|
selected := make(map[string]installplan.Package)
|
||||||
|
seenRecords := make(map[[3]string]bool)
|
||||||
|
if err := parseControl(index, func(record map[string]string) error {
|
||||||
|
identity := [3]string{record["package"], record["version"], record["architecture"]}
|
||||||
|
if identity[0] != "" && identity[1] != "" && identity[2] != "" {
|
||||||
|
if seenRecords[identity] {
|
||||||
|
return metadataError
|
||||||
|
}
|
||||||
|
seenRecords[identity] = true
|
||||||
|
}
|
||||||
|
version, target := versions[identity[0]]
|
||||||
|
if !target {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
for _, field := range []string{"package", "version", "architecture", "filename", "size", "sha256"} {
|
||||||
|
if record[field] == "" || strings.Contains(record[field], "\n") {
|
||||||
|
return metadataError
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if identity[1] != version || identity[2] != arch {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
size, err := decimalSize(record["size"])
|
||||||
|
if err != nil {
|
||||||
|
return metadataError
|
||||||
|
}
|
||||||
|
selected[identity[0]] = installplan.Package{
|
||||||
|
Name: identity[0], Version: version, Filename: record["filename"],
|
||||||
|
Size: size, Digest: "sha256:" + record["sha256"],
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}); err != nil {
|
||||||
|
return installplan.Lock{}, metadataError
|
||||||
|
}
|
||||||
|
|
||||||
|
releaseSum := sha256.Sum256(release)
|
||||||
|
lock := installplan.Lock{
|
||||||
|
ProtocolVersion: 1, Repository: "https://download.docker.com/linux/ubuntu",
|
||||||
|
Suite: suite, Architecture: arch, ReleaseDigest: "sha256:" + hex.EncodeToString(releaseSum[:]),
|
||||||
|
}
|
||||||
|
for _, name := range pinnedPackageNames {
|
||||||
|
p, ok := selected[name]
|
||||||
|
if !ok {
|
||||||
|
return installplan.Lock{}, metadataError
|
||||||
|
}
|
||||||
|
lock.Packages = append(lock.Packages, p)
|
||||||
|
}
|
||||||
|
if _, err := installplan.Validate(lock, suite, arch); err != nil {
|
||||||
|
return installplan.Lock{}, metadataError
|
||||||
|
}
|
||||||
|
return lock, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// parseControl preserves continuation boundaries and rejects duplicate fields
|
||||||
|
// before invoking visit. Processing one stanza at a time bounds retained values.
|
||||||
|
func parseControl(raw []byte, visit func(map[string]string) error) error {
|
||||||
|
if !utf8.Valid(raw) {
|
||||||
|
return metadataError
|
||||||
|
}
|
||||||
|
text := strings.ReplaceAll(string(raw), "\r\n", "\n")
|
||||||
|
for _, r := range text {
|
||||||
|
if unicode.IsControl(r) && r != '\n' && r != '\t' {
|
||||||
|
return metadataError
|
||||||
|
}
|
||||||
|
}
|
||||||
|
fields := make(map[string]*strings.Builder)
|
||||||
|
current := ""
|
||||||
|
flush := func() error {
|
||||||
|
if len(fields) == 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
stanza := make(map[string]string, len(fields))
|
||||||
|
for name, value := range fields {
|
||||||
|
stanza[name] = value.String()
|
||||||
|
}
|
||||||
|
if err := visit(stanza); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
fields = make(map[string]*strings.Builder)
|
||||||
|
current = ""
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
for line := range strings.SplitSeq(text, "\n") {
|
||||||
|
if line == "" {
|
||||||
|
if err := flush(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if line[0] == ' ' || line[0] == '\t' {
|
||||||
|
if current == "" {
|
||||||
|
return metadataError
|
||||||
|
}
|
||||||
|
fields[current].WriteByte('\n')
|
||||||
|
fields[current].WriteString(strings.Trim(line, " \t"))
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
name, value, ok := strings.Cut(line, ":")
|
||||||
|
if !ok || name == "" {
|
||||||
|
return metadataError
|
||||||
|
}
|
||||||
|
for _, r := range name {
|
||||||
|
if !(r >= 'a' && r <= 'z' || r >= 'A' && r <= 'Z' || r >= '0' && r <= '9' || r == '-') {
|
||||||
|
return metadataError
|
||||||
|
}
|
||||||
|
}
|
||||||
|
current = strings.ToLower(name)
|
||||||
|
if _, exists := fields[current]; exists {
|
||||||
|
return metadataError
|
||||||
|
}
|
||||||
|
fields[current] = &strings.Builder{}
|
||||||
|
fields[current].WriteString(strings.Trim(value, " \t"))
|
||||||
|
}
|
||||||
|
return flush()
|
||||||
|
}
|
||||||
|
|
||||||
|
func hasWord(value, word string) bool {
|
||||||
|
for _, item := range strings.Fields(value) {
|
||||||
|
if item == word {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
func decimalSize(value string) (uint64, error) {
|
||||||
|
for _, r := range value {
|
||||||
|
if r < '0' || r > '9' {
|
||||||
|
return 0, metadataError
|
||||||
|
}
|
||||||
|
}
|
||||||
|
size, err := strconv.ParseUint(value, 10, 64)
|
||||||
|
if err != nil {
|
||||||
|
return 0, metadataError
|
||||||
|
}
|
||||||
|
return size, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func validSHA256(value string) bool {
|
||||||
|
if len(value) != 64 {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
for _, r := range value {
|
||||||
|
if !(r >= '0' && r <= '9' || r >= 'a' && r <= 'f') {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
@@ -0,0 +1,327 @@
|
|||||||
|
package aptrepo
|
||||||
|
|
||||||
|
import (
|
||||||
|
"crypto/sha256"
|
||||||
|
"fmt"
|
||||||
|
"reflect"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"server-deploy/internal/installplan"
|
||||||
|
)
|
||||||
|
|
||||||
|
var fixtureNow = time.Date(2026, 9, 25, 6, 38, 50, 0, time.UTC)
|
||||||
|
|
||||||
|
func fixturePins() map[string]string {
|
||||||
|
return map[string]string{
|
||||||
|
"docker-ce": "5:29.1.0-1~ubuntu.26.04~resolute",
|
||||||
|
"docker-ce-cli": "5:29.1.0-1~ubuntu.26.04~resolute",
|
||||||
|
"containerd.io": "2.1.5-1~ubuntu.26.04~resolute",
|
||||||
|
"docker-buildx-plugin": "0.30.1-1~ubuntu.26.04~resolute",
|
||||||
|
"docker-compose-plugin": "2.40.3-1~ubuntu.26.04~resolute",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Inline Debian control fixtures retain Docker's Release field layout (notably
|
||||||
|
// no Codename) and epoch-free pool filenames. Artifact hashes are test data;
|
||||||
|
// authentication of Release is outside Resolve's contract.
|
||||||
|
func fixtureRecord(name, version, arch string) string {
|
||||||
|
fileVersion := version
|
||||||
|
if _, after, ok := strings.Cut(version, ":"); ok {
|
||||||
|
fileVersion = after
|
||||||
|
}
|
||||||
|
return fmt.Sprintf("Package: %s\nVersion: %s\nArchitecture: %s\nMaintainer: Docker <support@docker.com>\nFilename: dists/resolute/pool/stable/%s/%s_%s_%s.deb\nSize: 12345\nSHA256: %s\nDescription: Docker package\n continuation with a colon: allowed\n .\n another paragraph\n\n", name, version, arch, arch, name, fileVersion, arch, strings.Repeat("a", 64))
|
||||||
|
}
|
||||||
|
|
||||||
|
func fixtureIndex() []byte {
|
||||||
|
pins := fixturePins()
|
||||||
|
var index strings.Builder
|
||||||
|
for _, name := range []string{"docker-ce", "docker-ce-cli", "containerd.io", "docker-buildx-plugin", "docker-compose-plugin"} {
|
||||||
|
index.WriteString(fixtureRecord(name, pins[name], "amd64"))
|
||||||
|
}
|
||||||
|
return []byte(index.String())
|
||||||
|
}
|
||||||
|
|
||||||
|
func fixtureRelease(index []byte) []byte {
|
||||||
|
return []byte(fmt.Sprintf("Architectures: amd64 arm64 armhf s390x ppc64el\nComponents: stable edge test nightly\nDate: Thu, 24 Sep 2026 06:38:50 +0000\nLabel: Docker CE\nOrigin: Docker\nSuite: resolute\nSHA256:\n %x %d stable/binary-amd64/Packages\n", sha256.Sum256(index), len(index)))
|
||||||
|
}
|
||||||
|
|
||||||
|
func replace(raw []byte, old, new string) []byte {
|
||||||
|
return []byte(strings.Replace(string(raw), old, new, 1))
|
||||||
|
}
|
||||||
|
|
||||||
|
func assertRejected(t *testing.T, release, index []byte, suite, arch string, pins map[string]string, now time.Time) {
|
||||||
|
t.Helper()
|
||||||
|
lock, err := Resolve(release, index, suite, arch, pins, now)
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("invalid metadata accepted")
|
||||||
|
}
|
||||||
|
if !reflect.DeepEqual(lock, installplan.Lock{}) {
|
||||||
|
t.Fatal("failure returned a partial lock")
|
||||||
|
}
|
||||||
|
// Rejection messages must not disclose any untrusted metadata or pins.
|
||||||
|
if strings.Contains(err.Error(), "PRIVATE-MARKER") {
|
||||||
|
t.Fatal("error echoed metadata")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResolveDockerMetadataChain(t *testing.T) {
|
||||||
|
index := fixtureIndex()
|
||||||
|
release := fixtureRelease(index)
|
||||||
|
lock, err := Resolve(release, index, "resolute", "amd64", fixturePins(), fixtureNow)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if lock.ProtocolVersion != 1 || lock.Repository != "https://download.docker.com/linux/ubuntu" || lock.Suite != "resolute" || lock.Architecture != "amd64" || lock.ReleaseDigest != fmt.Sprintf("sha256:%x", sha256.Sum256(release)) {
|
||||||
|
t.Fatalf("incorrect release binding: %+v", lock)
|
||||||
|
}
|
||||||
|
if len(lock.Packages) != 5 {
|
||||||
|
t.Fatal("incorrect package count")
|
||||||
|
}
|
||||||
|
want := installplan.Package{Name: "docker-ce", Version: "5:29.1.0-1~ubuntu.26.04~resolute", Filename: "dists/resolute/pool/stable/amd64/docker-ce_29.1.0-1~ubuntu.26.04~resolute_amd64.deb", Digest: "sha256:" + strings.Repeat("a", 64), Size: 12345}
|
||||||
|
if lock.Packages[0] != want {
|
||||||
|
t.Fatalf("wrong selected package: %+v", lock.Packages[0])
|
||||||
|
}
|
||||||
|
for _, p := range lock.Packages {
|
||||||
|
if p.Version != fixturePins()[p.Name] {
|
||||||
|
t.Fatal("pin was not preserved")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if _, err := installplan.Validate(lock, "resolute", "amd64"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResolveCompatibleControlFormatting(t *testing.T) {
|
||||||
|
for _, mode := range []string{"mixed case", "CRLF", "no final newline", "other versions and architectures", "arm64", "valid expiry", "future boundary", "age boundary"} {
|
||||||
|
t.Run(mode, func(t *testing.T) {
|
||||||
|
index, arch, now := fixtureIndex(), "amd64", fixtureNow
|
||||||
|
switch mode {
|
||||||
|
case "mixed case":
|
||||||
|
index = []byte(strings.ReplaceAll(string(index), "Package:", "pAcKaGe:"))
|
||||||
|
case "CRLF":
|
||||||
|
index = []byte(strings.ReplaceAll(string(index), "\n", "\r\n"))
|
||||||
|
case "no final newline":
|
||||||
|
index = []byte(strings.TrimRight(string(index), "\n"))
|
||||||
|
case "other versions and architectures":
|
||||||
|
index = append(index, fixtureRecord("docker-ce", "5:99.0-1", "amd64")...)
|
||||||
|
index = append(index, fixtureRecord("docker-ce", fixturePins()["docker-ce"], "arm64")...)
|
||||||
|
case "arm64":
|
||||||
|
arch = "arm64"
|
||||||
|
index = []byte(strings.ReplaceAll(string(index), "amd64", arch))
|
||||||
|
case "future boundary":
|
||||||
|
now = fixtureNow.Add(-24*time.Hour - 10*time.Minute)
|
||||||
|
case "age boundary":
|
||||||
|
now = fixtureNow.Add(29 * 24 * time.Hour)
|
||||||
|
}
|
||||||
|
release := fixtureRelease(index)
|
||||||
|
switch mode {
|
||||||
|
case "mixed case":
|
||||||
|
release = replace(release, "SHA256:", "sHa256:")
|
||||||
|
release = replace(release, "Suite:", "sUiTe:")
|
||||||
|
case "CRLF":
|
||||||
|
release = []byte(strings.ReplaceAll(string(release), "\n", "\r\n"))
|
||||||
|
case "no final newline":
|
||||||
|
release = []byte(strings.TrimRight(string(release), "\n"))
|
||||||
|
case "arm64":
|
||||||
|
release = replace(release, "binary-amd64/Packages", "binary-arm64/Packages")
|
||||||
|
case "valid expiry":
|
||||||
|
release = append(release, "Valid-Until: Sat, 26 Sep 2026 06:38:50 +0000\n"...)
|
||||||
|
}
|
||||||
|
lock, err := Resolve(release, index, "resolute", arch, fixturePins(), now)
|
||||||
|
if err != nil || len(lock.Packages) != 5 || lock.Architecture != arch {
|
||||||
|
t.Fatalf("compatible metadata rejected: %v", err)
|
||||||
|
}
|
||||||
|
if lock.ReleaseDigest != fmt.Sprintf("sha256:%x", sha256.Sum256(release)) {
|
||||||
|
t.Fatal("digest did not bind original bytes")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResolveRejectsReleaseMetadata(t *testing.T) {
|
||||||
|
index := fixtureIndex()
|
||||||
|
for name, mutate := range map[string]func([]byte) []byte{
|
||||||
|
"suite": func(r []byte) []byte { return replace(r, "Suite: resolute", "Suite: noble") },
|
||||||
|
"codename cannot replace suite": func(r []byte) []byte { return replace(r, "Suite:", "Codename:") },
|
||||||
|
"origin": func(r []byte) []byte { return replace(r, "Origin: Docker", "Origin: PRIVATE-MARKER") },
|
||||||
|
"label": func(r []byte) []byte { return replace(r, "Label: Docker CE", "Label: Other") },
|
||||||
|
"arch token": func(r []byte) []byte { return replace(r, "amd64 arm64", "xamd64 arm64") },
|
||||||
|
"component token": func(r []byte) []byte { return replace(r, "stable edge", "unstable edge") },
|
||||||
|
"invalid date": func(r []byte) []byte { return replace(r, "Thu, 24 Sep 2026 06:38:50 +0000", "PRIVATE-MARKER") },
|
||||||
|
"future date": func(r []byte) []byte {
|
||||||
|
return replace(r, "Thu, 24 Sep 2026 06:38:50 +0000", "Fri, 25 Sep 2026 06:48:51 +0000")
|
||||||
|
},
|
||||||
|
"stale date": func(r []byte) []byte {
|
||||||
|
return replace(r, "Thu, 24 Sep 2026 06:38:50 +0000", "Wed, 26 Aug 2026 06:38:49 +0000")
|
||||||
|
},
|
||||||
|
"expired": func(r []byte) []byte { return append(r, "Valid-Until: Fri, 25 Sep 2026 06:38:50 +0000\n"...) },
|
||||||
|
"invalid expiry": func(r []byte) []byte { return append(r, "Valid-Until: PRIVATE-MARKER\n"...) },
|
||||||
|
"duplicate case insensitive field": func(r []byte) []byte { return append(r, "oRiGiN: Docker\n"...) },
|
||||||
|
"duplicate unrelated field": func(r []byte) []byte { return append(r, "X-Info: one\nx-info: two\n"...) },
|
||||||
|
"second stanza": func(r []byte) []byte { return append(r, "\nSuite: resolute\n"...) },
|
||||||
|
"MD5 only": func(r []byte) []byte { return replace(r, "SHA256:", "MD5Sum:") },
|
||||||
|
"SHA1 only": func(r []byte) []byte { return replace(r, "SHA256:", "SHA1:") },
|
||||||
|
"compressed only": func(r []byte) []byte { return replace(r, "/Packages", "/Packages.gz") },
|
||||||
|
"path prefix": func(r []byte) []byte { return replace(r, "stable/binary", "./stable/binary") },
|
||||||
|
"checksum arch": func(r []byte) []byte { return replace(r, "binary-amd64", "binary-arm64") },
|
||||||
|
"checksum size": func(r []byte) []byte {
|
||||||
|
return replace(r, fmt.Sprintf(" %d ", len(index)), fmt.Sprintf(" %d ", len(index)+1))
|
||||||
|
},
|
||||||
|
"checksum negative size": func(r []byte) []byte { return replace(r, fmt.Sprintf(" %d ", len(index)), " -1 ") },
|
||||||
|
"checksum extra column": func(r []byte) []byte { return replace(r, "/Packages\n", "/Packages extra\n") },
|
||||||
|
"checksum invalid digest": func(r []byte) []byte {
|
||||||
|
return replace(r, fmt.Sprintf("%x", sha256.Sum256(index)), strings.Repeat("g", 64))
|
||||||
|
},
|
||||||
|
"duplicate checksum entry": func(r []byte) []byte {
|
||||||
|
return append(r, fmt.Sprintf(" %x %d stable/binary-amd64/Packages\n", sha256.Sum256(index), len(index))...)
|
||||||
|
},
|
||||||
|
"conflicting checksum entry": func(r []byte) []byte {
|
||||||
|
return append(r, fmt.Sprintf(" %s %d stable/binary-amd64/Packages\n", strings.Repeat("b", 64), len(index))...)
|
||||||
|
},
|
||||||
|
"duplicate other checksum entry": func(r []byte) []byte {
|
||||||
|
return append(r, strings.Repeat(" "+strings.Repeat("a", 64)+" 1 other/Packages\n", 2)...)
|
||||||
|
},
|
||||||
|
"orphan continuation": func(r []byte) []byte { return append([]byte(" orphan\n"), r...) },
|
||||||
|
"invalid field name": func(r []byte) []byte { return append(r, "Bad Field: value\n"...) },
|
||||||
|
} {
|
||||||
|
t.Run(name, func(t *testing.T) {
|
||||||
|
assertRejected(t, mutate(fixtureRelease(index)), index, "resolute", "amd64", fixturePins(), fixtureNow)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
for _, field := range []string{"Architectures", "Components", "Date", "Label", "Origin", "Suite"} {
|
||||||
|
t.Run("missing "+field, func(t *testing.T) {
|
||||||
|
r := fixtureRelease(index)
|
||||||
|
lines := strings.Split(string(r), "\n")
|
||||||
|
for i, line := range lines {
|
||||||
|
if strings.HasPrefix(line, field+":") {
|
||||||
|
lines = append(lines[:i], lines[i+1:]...)
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
assertRejected(t, []byte(strings.Join(lines, "\n")), index, "resolute", "amd64", fixturePins(), fixtureNow)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
// Both times are in the future relative to now, but expiry precedes Date.
|
||||||
|
r := append(fixtureRelease(index), "Valid-Until: Thu, 24 Sep 2026 06:37:50 +0000\n"...)
|
||||||
|
assertRejected(t, r, index, "resolute", "amd64", fixturePins(), fixtureNow.Add(-24*time.Hour-2*time.Minute))
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResolveRejectsTamperedIndex(t *testing.T) {
|
||||||
|
index := fixtureIndex()
|
||||||
|
r := fixtureRelease(index)
|
||||||
|
index = replace(index, "Size: 12345", "Size: 12346") // Same byte size, different digest.
|
||||||
|
assertRejected(t, r, index, "resolute", "amd64", fixturePins(), fixtureNow)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResolveRejectsPackageAmbiguityAndInvalidLock(t *testing.T) {
|
||||||
|
for name, mutate := range map[string]func([]byte) []byte{
|
||||||
|
"duplicate field": func(p []byte) []byte {
|
||||||
|
return replace(p, "Package: docker-ce\n", "Package: docker-ce\npAcKaGe: docker-ce\n")
|
||||||
|
},
|
||||||
|
"duplicate unrelated field": func(p []byte) []byte { return replace(p, "Description:", "X-Info: one\nx-info: two\nDescription:") },
|
||||||
|
"duplicate record": func(p []byte) []byte {
|
||||||
|
return append(p, fixtureRecord("docker-ce", fixturePins()["docker-ce"], "amd64")...)
|
||||||
|
},
|
||||||
|
"conflicting record": func(p []byte) []byte {
|
||||||
|
return append(p, strings.Replace(fixtureRecord("docker-ce", fixturePins()["docker-ce"], "amd64"), "Size: 12345", "Size: 999", 1)...)
|
||||||
|
},
|
||||||
|
"missing record": func(p []byte) []byte { return []byte(strings.SplitN(string(p), "\n\n", 2)[1]) },
|
||||||
|
"wrong architecture": func(p []byte) []byte { return replace(p, "Architecture: amd64", "Architecture: all") },
|
||||||
|
"wrong version": func(p []byte) []byte { return replace(p, "Version: 5:29.1.0", "Version: 5:29.2.0") },
|
||||||
|
"unsafe filename": func(p []byte) []byte { return replace(p, "Filename: dists/resolute", "Filename: ../PRIVATE-MARKER") },
|
||||||
|
"wrong filename version": func(p []byte) []byte { return replace(p, "docker-ce_29.1.0", "docker-ce_29.2.0") },
|
||||||
|
"epoch filename": func(p []byte) []byte { return replace(p, "docker-ce_29.1.0", "docker-ce_5:29.1.0") },
|
||||||
|
"wrong filename suite": func(p []byte) []byte { return replace(p, "Filename: dists/resolute", "Filename: dists/noble") },
|
||||||
|
"zero size": func(p []byte) []byte { return replace(p, "Size: 12345", "Size: 0") },
|
||||||
|
"negative size": func(p []byte) []byte { return replace(p, "Size: 12345", "Size: -1") },
|
||||||
|
"overflow size": func(p []byte) []byte { return replace(p, "Size: 12345", "Size: 18446744073709551616") },
|
||||||
|
"excess package size": func(p []byte) []byte { return replace(p, "Size: 12345", "Size: 536870913") },
|
||||||
|
"bad digest": func(p []byte) []byte { return replace(p, "SHA256: "+strings.Repeat("a", 64), "SHA256: PRIVATE-MARKER") },
|
||||||
|
"folded required field": func(p []byte) []byte { return replace(p, "Size: 12345", "Size: 12345\n 6") },
|
||||||
|
"invalid syntax": func(p []byte) []byte { return append(p, "PRIVATE-MARKER\n"...) },
|
||||||
|
} {
|
||||||
|
t.Run(name, func(t *testing.T) {
|
||||||
|
index := mutate(fixtureIndex())
|
||||||
|
assertRejected(t, fixtureRelease(index), index, "resolute", "amd64", fixturePins(), fixtureNow)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
for _, field := range []string{"Package", "Version", "Architecture", "Filename", "Size", "SHA256"} {
|
||||||
|
t.Run("missing "+field, func(t *testing.T) {
|
||||||
|
index := replace(fixtureIndex(), field+":", "X-Removed:")
|
||||||
|
assertRejected(t, fixtureRelease(index), index, "resolute", "amd64", fixturePins(), fixtureNow)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResolveRequiresExactExplicitPins(t *testing.T) {
|
||||||
|
for _, mode := range []string{"nil", "missing", "extra", "empty", "latest", "engine mismatch", "invalid version"} {
|
||||||
|
t.Run(mode, func(t *testing.T) {
|
||||||
|
pins := fixturePins()
|
||||||
|
switch mode {
|
||||||
|
case "nil":
|
||||||
|
pins = nil
|
||||||
|
case "missing":
|
||||||
|
delete(pins, "containerd.io")
|
||||||
|
case "extra":
|
||||||
|
pins["unexpected"] = "1.0"
|
||||||
|
case "empty":
|
||||||
|
pins["containerd.io"] = ""
|
||||||
|
case "latest":
|
||||||
|
pins["containerd.io"] = "latest"
|
||||||
|
case "engine mismatch":
|
||||||
|
pins["docker-ce-cli"] = "5:29.2.0-1~ubuntu.26.04~resolute"
|
||||||
|
case "invalid version":
|
||||||
|
pins["containerd.io"] = "1;PRIVATE-MARKER"
|
||||||
|
}
|
||||||
|
// Make invalid versions available too: Validate, rather than a missing
|
||||||
|
// match, must enforce version syntax and engine/CLI equality.
|
||||||
|
index := fixtureIndex()
|
||||||
|
for _, name := range []string{"containerd.io", "docker-ce-cli"} {
|
||||||
|
if v := pins[name]; v != "" && v != fixturePins()[name] {
|
||||||
|
index = replace(index, fixtureRecord(name, fixturePins()[name], "amd64"), fixtureRecord(name, v, "amd64"))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
assertRejected(t, fixtureRelease(index), index, "resolute", "amd64", pins, fixtureNow)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResolveRejectsInvalidText(t *testing.T) {
|
||||||
|
for _, invalid := range []string{"\x00", "\x01", "\x1b", "\x7f", "\u0085", "\r", "\xff"} {
|
||||||
|
t.Run(fmt.Sprintf("%x", invalid), func(t *testing.T) {
|
||||||
|
index := fixtureIndex()
|
||||||
|
r := append(fixtureRelease(index), "X-Info: PRIVATE-MARKER"+invalid+"suffix\n"...)
|
||||||
|
assertRejected(t, r, index, "resolute", "amd64", fixturePins(), fixtureNow)
|
||||||
|
index = append(index, "Package: unrelated\nDescription: PRIVATE-MARKER"+invalid+"suffix\n"...)
|
||||||
|
assertRejected(t, fixtureRelease(index), index, "resolute", "amd64", fixturePins(), fixtureNow)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResolveExactByteLimits(t *testing.T) {
|
||||||
|
for _, target := range []string{"release", "index"} {
|
||||||
|
for _, excess := range []int{0, 1} {
|
||||||
|
t.Run(fmt.Sprintf("%s+%d", target, excess), func(t *testing.T) {
|
||||||
|
index := fixtureIndex()
|
||||||
|
if target == "index" {
|
||||||
|
index = append(index, "Package: unrelated\nDescription: "...)
|
||||||
|
index = append(index, strings.Repeat("x", (16<<20)+excess-len(index)-1)...)
|
||||||
|
index = append(index, '\n')
|
||||||
|
}
|
||||||
|
release := fixtureRelease(index)
|
||||||
|
if target == "release" {
|
||||||
|
release = append(release, "X-Padding: "...)
|
||||||
|
release = append(release, strings.Repeat("x", (1<<20)+excess-len(release)-1)...)
|
||||||
|
release = append(release, '\n')
|
||||||
|
}
|
||||||
|
if excess != 0 {
|
||||||
|
assertRejected(t, release, index, "resolute", "amd64", fixturePins(), fixtureNow)
|
||||||
|
} else if _, err := Resolve(release, index, "resolute", "amd64", fixturePins(), fixtureNow); err != nil {
|
||||||
|
t.Fatalf("exact limit rejected: %v", err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,169 @@
|
|||||||
|
package aptrepo
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"crypto/sha256"
|
||||||
|
"encoding/hex"
|
||||||
|
"errors"
|
||||||
|
"io"
|
||||||
|
"os"
|
||||||
|
"os/exec"
|
||||||
|
"path/filepath"
|
||||||
|
"runtime"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Bootstrapped from Docker's official HTTPS key endpoint. Rotation requires a
|
||||||
|
// reviewed code update, never a caller-supplied key or fingerprint override.
|
||||||
|
const dockerFingerprint = "9DC858229FC7DD38854AE2D88D81803C0EBFCD88"
|
||||||
|
const dockerKeySHA256 = "1500c1f56fa9e26b9b8f42452a553675796ade0807cdce11975eb98170b3a570"
|
||||||
|
|
||||||
|
func readStaging(directory string) (map[string][]byte, error) {
|
||||||
|
fail := errors.New("invalid repository staging files")
|
||||||
|
if !filepath.IsAbs(directory) {
|
||||||
|
return nil, fail
|
||||||
|
}
|
||||||
|
directory = filepath.Clean(directory)
|
||||||
|
info, err := os.Lstat(directory)
|
||||||
|
if err != nil || !info.IsDir() || info.Mode()&os.ModeSymlink != 0 {
|
||||||
|
return nil, fail
|
||||||
|
}
|
||||||
|
root, err := os.OpenRoot(directory)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail
|
||||||
|
}
|
||||||
|
defer root.Close()
|
||||||
|
opened, err := root.Stat(".")
|
||||||
|
if err != nil || !os.SameFile(info, opened) {
|
||||||
|
return nil, fail
|
||||||
|
}
|
||||||
|
result := make(map[string][]byte)
|
||||||
|
for name, limit := range map[string]int64{"docker.asc": 1 << 20, "Release": 1 << 20, "Release.gpg": 65536, "Packages": 16 << 20} {
|
||||||
|
data, err := readFile(root, name, limit)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail
|
||||||
|
}
|
||||||
|
result[name] = data
|
||||||
|
}
|
||||||
|
return result, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func readFile(root *os.Root, name string, limit int64) ([]byte, error) {
|
||||||
|
fail := errors.New("invalid metadata file")
|
||||||
|
before, err := root.Lstat(name)
|
||||||
|
if err != nil || !before.Mode().IsRegular() || before.Size() <= 0 || before.Size() > limit {
|
||||||
|
return nil, fail
|
||||||
|
}
|
||||||
|
f, err := root.Open(name)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail
|
||||||
|
}
|
||||||
|
defer f.Close()
|
||||||
|
opened, err := f.Stat()
|
||||||
|
if err != nil || !opened.Mode().IsRegular() || !os.SameFile(before, opened) || before.Size() != opened.Size() {
|
||||||
|
return nil, fail
|
||||||
|
}
|
||||||
|
data, err := io.ReadAll(io.LimitReader(f, limit+1))
|
||||||
|
if err != nil || int64(len(data)) != opened.Size() || int64(len(data)) > limit {
|
||||||
|
return nil, fail
|
||||||
|
}
|
||||||
|
return data, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func authenticate(files map[string][]byte, now time.Time) error {
|
||||||
|
sum := sha256.Sum256(files["docker.asc"])
|
||||||
|
if hex.EncodeToString(sum[:]) != dockerKeySHA256 {
|
||||||
|
return errors.New("repository trust anchor mismatch")
|
||||||
|
}
|
||||||
|
if runtime.GOOS != "linux" {
|
||||||
|
return errors.New("repository authentication requires Linux GnuPG")
|
||||||
|
}
|
||||||
|
// All untrusted bytes are copied to a private snapshot. No caller path reaches
|
||||||
|
// a subprocess, and neither system nor personal keyrings are consulted.
|
||||||
|
dir, err := os.MkdirTemp("", "deployctl-signature-")
|
||||||
|
if err != nil {
|
||||||
|
return errors.New("cannot create signature workspace")
|
||||||
|
}
|
||||||
|
defer os.RemoveAll(dir) // Only our own freshly allocated directory.
|
||||||
|
for _, name := range []string{"docker.asc", "Release", "Release.gpg"} {
|
||||||
|
if err := os.WriteFile(filepath.Join(dir, name), files[name], 0600); err != nil {
|
||||||
|
return errors.New("cannot snapshot repository metadata")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if _, err := runGPG(dir, "/usr/bin/gpg", "--batch", "--no-options", "--homedir", dir, "--dearmor", "--output", filepath.Join(dir, "docker.gpg"), filepath.Join(dir, "docker.asc")); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
status, err := runGPG(dir, "/usr/bin/gpgv", "--homedir", dir, "--keyring", filepath.Join(dir, "docker.gpg"), "--status-fd", "1", filepath.Join(dir, "Release.gpg"), filepath.Join(dir, "Release"))
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return checkStatus(status, now)
|
||||||
|
}
|
||||||
|
|
||||||
|
type cappedOutput struct{ buffer bytes.Buffer }
|
||||||
|
|
||||||
|
func (b *cappedOutput) Len() int { return b.buffer.Len() }
|
||||||
|
func (b *cappedOutput) Bytes() []byte { return b.buffer.Bytes() }
|
||||||
|
|
||||||
|
func (b *cappedOutput) Write(p []byte) (int, error) {
|
||||||
|
if len(p) > 65536-b.Len() {
|
||||||
|
return 0, errors.New("signature output exceeds limit")
|
||||||
|
}
|
||||||
|
return b.buffer.Write(p)
|
||||||
|
}
|
||||||
|
|
||||||
|
func runGPG(dir, program string, args ...string) ([]byte, error) {
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
cmd := exec.CommandContext(ctx, program, args...)
|
||||||
|
cmd.Dir = dir
|
||||||
|
cmd.Env = []string{"LC_ALL=C", "LANG=C", "HOME=" + dir, "GNUPGHOME=" + dir, "PATH=/usr/bin:/bin"}
|
||||||
|
var output, diagnostics cappedOutput
|
||||||
|
cmd.Stdout = &output
|
||||||
|
cmd.Stderr = &diagnostics
|
||||||
|
cmd.WaitDelay = time.Second
|
||||||
|
if err := cmd.Run(); err != nil {
|
||||||
|
return nil, errors.New("repository signature tool failed")
|
||||||
|
}
|
||||||
|
return output.Bytes(), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func checkStatus(status []byte, now time.Time) error {
|
||||||
|
fail := errors.New("repository signature rejected")
|
||||||
|
valid, good := 0, 0
|
||||||
|
for _, line := range strings.Split(string(status), "\n") {
|
||||||
|
if line == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
f := strings.Fields(line)
|
||||||
|
if len(f) < 2 || f[0] != "[GNUPG:]" {
|
||||||
|
return fail
|
||||||
|
}
|
||||||
|
switch f[1] {
|
||||||
|
case "NEWSIG", "KEY_CONSIDERED", "SIG_ID":
|
||||||
|
case "GOODSIG":
|
||||||
|
good++
|
||||||
|
case "VALIDSIG":
|
||||||
|
// fingerprint, date, timestamp, expiry, version, reserved, public-key
|
||||||
|
// algorithm, digest algorithm, signature class, primary fingerprint.
|
||||||
|
if len(f) != 12 || f[11] != dockerFingerprint || (f[9] != "8" && f[9] != "9" && f[9] != "10") || f[10] != "00" {
|
||||||
|
return fail
|
||||||
|
}
|
||||||
|
issued, e1 := strconv.ParseInt(f[4], 10, 64)
|
||||||
|
expiry, e2 := strconv.ParseInt(f[5], 10, 64)
|
||||||
|
if e1 != nil || e2 != nil || issued <= 0 || expiry < 0 || issued > now.Add(10*time.Minute).Unix() || (expiry != 0 && expiry <= now.Unix()) {
|
||||||
|
return fail
|
||||||
|
}
|
||||||
|
valid++
|
||||||
|
default:
|
||||||
|
return fail // Includes expired/revoked/bad/unknown signatures.
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if valid != 1 || good != 1 {
|
||||||
|
return fail
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,105 @@
|
|||||||
|
package aptrepo
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"encoding/json"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"server-deploy/internal/installplan"
|
||||||
|
"syscall"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestStagingRejectsLinksAndFIFO(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
for _, name := range []string{"docker.asc", "Release", "Release.gpg", "Packages"} {
|
||||||
|
if err := os.WriteFile(filepath.Join(dir, name), []byte("metadata"), 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
link := filepath.Join(t.TempDir(), "link")
|
||||||
|
if err := os.Symlink(dir, link); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := readStaging(link + "/"); err == nil {
|
||||||
|
t.Fatal("root symlink accepted")
|
||||||
|
}
|
||||||
|
if err := os.Remove(filepath.Join(dir, "Release")); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.Symlink(filepath.Join(dir, "Packages"), filepath.Join(dir, "Release")); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := readStaging(dir); err == nil {
|
||||||
|
t.Fatal("file symlink accepted")
|
||||||
|
}
|
||||||
|
if err := os.Remove(filepath.Join(dir, "Release")); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := syscall.Mkfifo(filepath.Join(dir, "Release"), 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := readStaging(dir); err == nil {
|
||||||
|
t.Fatal("FIFO accepted")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGPGFailureRedactsDiagnostics(t *testing.T) {
|
||||||
|
if _, err := runGPG(t.TempDir(), "/nonexistent/signature-tool-secret"); err == nil || err.Error() != "repository signature tool failed" {
|
||||||
|
t.Fatal("unredacted or absent error", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestOutputBound(t *testing.T) {
|
||||||
|
var b cappedOutput
|
||||||
|
if _, err := b.Write(make([]byte, 65536)); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := b.Write([]byte{1}); err == nil {
|
||||||
|
t.Fatal("output limit missing")
|
||||||
|
}
|
||||||
|
if b.Len() != 65536 {
|
||||||
|
t.Fatal("buffer grew beyond limit")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestStagedSignatureIntegration(t *testing.T) {
|
||||||
|
// Explicit opt-in public fixture, downloaded by the metadata-only probe.
|
||||||
|
dir := os.Getenv("DEPLOYCTL_REPOSITORY_FIXTURE")
|
||||||
|
if dir == "" {
|
||||||
|
t.Skip("public signed fixture not supplied; run repository probe for real signature evidence")
|
||||||
|
}
|
||||||
|
files, err := readStaging(dir)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := authenticate(files, time.Now()); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
// Result comes from the successful real CLI invocation. Recheck the exact
|
||||||
|
// fixture versions before making a semantically valid signature mutation.
|
||||||
|
var response struct {
|
||||||
|
Lock installplan.Lock `json:"lock"`
|
||||||
|
}
|
||||||
|
raw, err := os.ReadFile(filepath.Join(dir, "result.json"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err = json.Unmarshal(raw, &response); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
versions := make(map[string]string)
|
||||||
|
for _, p := range response.Lock.Packages {
|
||||||
|
versions[p.Name] = p.Version
|
||||||
|
}
|
||||||
|
// Unknown Release extension remains valid metadata; only its signature
|
||||||
|
// should reject it. This catches a bypass hidden by syntax failures.
|
||||||
|
files["Release"] = append(bytes.TrimRight(files["Release"], "\n"), []byte("\nX-Verification-Probe: changed\n")...)
|
||||||
|
if _, err := Resolve(files["Release"], files["Packages"], response.Lock.Suite, response.Lock.Architecture, versions, time.Now()); err != nil {
|
||||||
|
t.Fatal("mutation masked by metadata rejection", err)
|
||||||
|
}
|
||||||
|
if err := authenticate(files, time.Now()); err == nil {
|
||||||
|
t.Fatal("tampered signature accepted")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,77 @@
|
|||||||
|
package aptrepo
|
||||||
|
|
||||||
|
import (
|
||||||
|
"io"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestOutputCopyBound(t *testing.T) {
|
||||||
|
var b cappedOutput
|
||||||
|
_, err := io.Copy(&b, io.LimitReader(strings.NewReader(strings.Repeat("x", 65537)), 65537))
|
||||||
|
if err == nil || b.Len() > 65536 {
|
||||||
|
t.Fatal("io.Copy bypassed output limit", b.Len(), err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestWellFormedPackageTamperNeedsRebinding(t *testing.T) {
|
||||||
|
index := fixtureIndex()
|
||||||
|
release := fixtureRelease(index)
|
||||||
|
changed := []byte(strings.Replace(string(index), "SHA256: a", "SHA256: b", 1))
|
||||||
|
if _, err := Resolve(release, changed, "resolute", "amd64", fixturePins(), fixtureNow); err == nil {
|
||||||
|
t.Fatal("unbound index accepted")
|
||||||
|
}
|
||||||
|
if _, err := Resolve(fixtureRelease(changed), changed, "resolute", "amd64", fixturePins(), fixtureNow); err != nil {
|
||||||
|
t.Fatal("tamper test masked by parser rejection", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const goodStatus = "[GNUPG:] NEWSIG\n[GNUPG:] GOODSIG 7EA0A9C3F273FCD8 Docker\n[GNUPG:] VALIDSIG D3306A018370199E527AE7997EA0A9C3F273FCD8 2026-09-24 1790231932 0 4 0 1 10 00 9DC858229FC7DD38854AE2D88D81803C0EBFCD88\n"
|
||||||
|
|
||||||
|
func TestSignatureStatus(t *testing.T) {
|
||||||
|
now := time.Date(2026, 9, 25, 0, 0, 0, 0, time.UTC)
|
||||||
|
if err := checkStatus([]byte(goodStatus), now); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
for _, s := range []string{"", "[GNUPG:] GOODSIG key name\n", goodStatus + goodStatus,
|
||||||
|
strings.ReplaceAll(goodStatus, dockerFingerprint, strings.Repeat("A", 40)),
|
||||||
|
strings.Replace(goodStatus, " 10 00 ", " 2 00 ", 1),
|
||||||
|
strings.Replace(goodStatus, "1790231932 0", "1990231932 0", 1),
|
||||||
|
strings.Replace(goodStatus, "1790231932 0", "1790231932 1790231933", 1),
|
||||||
|
goodStatus + "[GNUPG:] EXPKEYSIG bad\n", goodStatus + "[GNUPG:] BADSIG bad\n",
|
||||||
|
} {
|
||||||
|
if checkStatus([]byte(s), now) == nil {
|
||||||
|
t.Errorf("accepted invalid signature status: %q", s)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestReadStaging(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
for _, name := range []string{"docker.asc", "Release", "Release.gpg", "Packages"} {
|
||||||
|
if err := os.WriteFile(filepath.Join(dir, name), []byte("bytes"), 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if _, err := readStaging(dir); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := readStaging("relative"); err == nil {
|
||||||
|
t.Fatal("relative directory accepted")
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(filepath.Join(dir, "Release.gpg"), make([]byte, 65537), 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := readStaging(dir); err == nil {
|
||||||
|
t.Fatal("oversize signature accepted")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestWrongKeyFailsBeforeExternalProcess(t *testing.T) {
|
||||||
|
if err := authenticate(map[string][]byte{"docker.asc": []byte("untrusted")}, time.Now()); err == nil {
|
||||||
|
t.Fatal("untrusted key accepted")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,34 @@
|
|||||||
|
// Package aptrepo authenticates staged Docker APT metadata, never installs it.
|
||||||
|
package aptrepo
|
||||||
|
|
||||||
|
import (
|
||||||
|
"server-deploy/internal/installplan"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
type Result struct {
|
||||||
|
ProtocolVersion int `json:"protocolVersion"`
|
||||||
|
RepositoryAuthenticated bool `json:"repositoryAuthenticated"`
|
||||||
|
PackageBytesVerified bool `json:"packageBytesVerified"`
|
||||||
|
Executable bool `json:"executable"`
|
||||||
|
VerifiedAt time.Time `json:"verifiedAt"`
|
||||||
|
PrimaryFingerprint string `json:"primaryFingerprint"`
|
||||||
|
Lock installplan.Lock `json:"lock"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Verify requires a trusted staging directory and trusted ancestors, with no
|
||||||
|
// concurrent writers. Returned JSON is evidence, not an execution capability.
|
||||||
|
func Verify(directory, suite, arch string, versions map[string]string, now time.Time) (Result, error) {
|
||||||
|
files, err := readStaging(directory)
|
||||||
|
if err != nil {
|
||||||
|
return Result{}, err
|
||||||
|
}
|
||||||
|
if err := authenticate(files, now); err != nil {
|
||||||
|
return Result{}, err
|
||||||
|
}
|
||||||
|
lock, err := Resolve(files["Release"], files["Packages"], suite, arch, versions, now)
|
||||||
|
if err != nil {
|
||||||
|
return Result{}, err
|
||||||
|
}
|
||||||
|
return Result{ProtocolVersion: 1, RepositoryAuthenticated: true, VerifiedAt: now.UTC().Truncate(time.Second), PrimaryFingerprint: dockerFingerprint, Lock: lock}, nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,43 @@
|
|||||||
|
package cli
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"server-deploy/internal/installplan"
|
||||||
|
"server-deploy/internal/preflight"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestEnvironmentPlanCLI(t *testing.T) {
|
||||||
|
r := preflight.Collect()
|
||||||
|
suite, arch := "resolute", "amd64"
|
||||||
|
if r.Runtime.OS == "linux" {
|
||||||
|
suite = r.Distribution.Codename
|
||||||
|
arch = r.Runtime.Architecture
|
||||||
|
}
|
||||||
|
if suite != "resolute" && suite != "noble" && suite != "jammy" {
|
||||||
|
t.Skip("local Linux distribution outside initial lock policy")
|
||||||
|
}
|
||||||
|
l := installplan.Lock{ProtocolVersion: 1, Repository: "https://download.docker.com/linux/ubuntu", Suite: suite, Architecture: arch, ReleaseDigest: "sha256:" + strings.Repeat("a", 64), Packages: []installplan.Package{}}
|
||||||
|
for _, name := range []string{"docker-ce", "docker-ce-cli", "containerd.io", "docker-buildx-plugin", "docker-compose-plugin"} {
|
||||||
|
l.Packages = append(l.Packages, installplan.Package{Name: name, Version: "1.2.3-1", Filename: "dists/" + suite + "/pool/stable/" + arch + "/" + name + "_1.2.3-1_" + arch + ".deb", Digest: "sha256:" + strings.Repeat("b", 64), Size: 123})
|
||||||
|
}
|
||||||
|
raw, _ := json.Marshal(struct {
|
||||||
|
Lock installplan.Lock `json:"lock"`
|
||||||
|
}{l})
|
||||||
|
code, out, diagnostics := run([]string{"plan-environment"}, string(raw))
|
||||||
|
var result struct {
|
||||||
|
Mode string `json:"mode"`
|
||||||
|
Draft installplan.Draft `json:"draft"`
|
||||||
|
}
|
||||||
|
if code != 0 || json.Unmarshal([]byte(out), &result) != nil || result.Mode != "local-environment-draft" || result.Draft.Executable || result.Draft.RepositoryAuthenticated || len(result.Draft.RequestedPackages) != 5 || len(result.Draft.Blockers) == 0 {
|
||||||
|
t.Fatalf("bad draft %s %s", out, diagnostics)
|
||||||
|
}
|
||||||
|
for _, bad := range []string{`{}`, strings.Replace(string(raw), `"1.2.3-1"`, `"secret;reboot"`, 1)} {
|
||||||
|
code, out, diagnostics := run([]string{"plan-environment"}, bad)
|
||||||
|
if code == 0 || out != "" || strings.Contains(diagnostics, "secret") {
|
||||||
|
t.Fatal("invalid request accepted or leaked")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,8 @@
|
|||||||
|
package cli
|
||||||
|
|
||||||
|
import (
|
||||||
|
"io"
|
||||||
|
"server-deploy/internal/wire"
|
||||||
|
)
|
||||||
|
|
||||||
|
func decodeStrict(in io.Reader, target any) error { return wire.Decode(in, target, 1<<20) }
|
||||||
@@ -0,0 +1,117 @@
|
|||||||
|
package cli
|
||||||
|
|
||||||
|
import (
|
||||||
|
"crypto/sha256"
|
||||||
|
"encoding/hex"
|
||||||
|
"encoding/json"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func packageFixture(t *testing.T) (string, string) {
|
||||||
|
return packageFixturePayload(t, []byte("services: {}\n"))
|
||||||
|
}
|
||||||
|
|
||||||
|
func packageFixturePayload(t *testing.T, payload []byte) (string, string) {
|
||||||
|
t.Helper()
|
||||||
|
dir := t.TempDir()
|
||||||
|
digest := func(data []byte) string { sum := sha256.Sum256(data); return "sha256:" + hex.EncodeToString(sum[:]) }
|
||||||
|
manifest := map[string]any{
|
||||||
|
"protocolVersion": 1, "appId": "example", "version": "1.0.0", "runtime": "compose", "entrypoint": "compose.yaml",
|
||||||
|
"platforms": []string{"linux/amd64"},
|
||||||
|
"components": []map[string]string{{"name": "server", "image": "example/server@sha256:" + strings.Repeat("a", 64)}},
|
||||||
|
"files": []map[string]string{{"path": "compose.yaml", "digest": digest(payload)}},
|
||||||
|
}
|
||||||
|
raw, err := json.Marshal(manifest)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(filepath.Join(dir, "manifest.json"), raw, 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(filepath.Join(dir, "compose.yaml"), payload, 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return dir, digest(raw)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCheckPackagePolicyCLI(t *testing.T) {
|
||||||
|
payload := `{"services":{"server":{"image":"example/server@sha256:` + strings.Repeat("a", 64) + `","user":"1000:1000","read_only":true,"cap_drop":["ALL"],"security_opt":["no-new-privileges:true"],"restart":"no","networks":["backend"],"volumes":[]}},"networks":{"backend":{"internal":true}},"volumes":{}}`
|
||||||
|
for _, tc := range []struct {
|
||||||
|
payload string
|
||||||
|
accepted bool
|
||||||
|
}{
|
||||||
|
{payload, true},
|
||||||
|
{strings.Replace(payload, `"read_only":true`, `"read_only":false`, 1), false},
|
||||||
|
{`services: {}`, false},
|
||||||
|
} {
|
||||||
|
dir, digest := packageFixturePayload(t, []byte(tc.payload))
|
||||||
|
input, _ := json.Marshal(map[string]string{"directory": dir, "expectedDigest": digest})
|
||||||
|
code, out, diagnostics := run([]string{"check-package"}, string(input))
|
||||||
|
if tc.accepted {
|
||||||
|
var result struct {
|
||||||
|
PolicyPassed bool `json:"policyPassed"`
|
||||||
|
Executable bool `json:"executable"`
|
||||||
|
Profile string `json:"profile"`
|
||||||
|
Digest string `json:"digest"`
|
||||||
|
}
|
||||||
|
if code != 0 || json.Unmarshal([]byte(out), &result) != nil || !result.PolicyPassed || result.Executable || result.Profile == "" || result.Digest != digest {
|
||||||
|
t.Fatalf("bad check result: %s %s", out, diagnostics)
|
||||||
|
}
|
||||||
|
if strings.Contains(out, "1000:1000") {
|
||||||
|
t.Fatal("payload leaked")
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(filepath.Join(dir, "compose.yaml"), []byte(tc.payload+" "), 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if code, _, _ := run([]string{"check-package"}, string(input)); code == 0 {
|
||||||
|
t.Fatal("policy bypassed integrity check")
|
||||||
|
}
|
||||||
|
} else if code == 0 || out != "" {
|
||||||
|
t.Fatal("unsafe package accepted")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestVerifyPackageCLI(t *testing.T) {
|
||||||
|
dir, digest := packageFixture(t)
|
||||||
|
input, _ := json.Marshal(map[string]string{"directory": dir, "expectedDigest": digest})
|
||||||
|
code, out, diagnostics := run([]string{"verify-package"}, string(input))
|
||||||
|
if code != 0 || diagnostics != "" {
|
||||||
|
t.Fatalf("package verification failed: %s", diagnostics)
|
||||||
|
}
|
||||||
|
var response struct {
|
||||||
|
Verified bool `json:"verified"`
|
||||||
|
Executable bool `json:"executable"`
|
||||||
|
PublisherAuthenticated bool `json:"publisherAuthenticated"`
|
||||||
|
FileCount int `json:"fileCount"`
|
||||||
|
Digest string `json:"digest"`
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal([]byte(out), &response); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if !response.Verified || response.Executable || response.PublisherAuthenticated || response.FileCount != 1 || response.Digest != digest {
|
||||||
|
t.Fatalf("misleading verification: %s", out)
|
||||||
|
}
|
||||||
|
if strings.Contains(out, "services: {}") {
|
||||||
|
t.Fatal("file contents leaked")
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(filepath.Join(dir, "compose.yaml"), []byte("secret tampering"), 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
code, out, diagnostics = run([]string{"verify-package"}, string(input))
|
||||||
|
if code == 0 || out != "" || strings.Contains(diagnostics, "secret tampering") {
|
||||||
|
t.Fatal("tampering accepted or leaked")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestVerifyPackageInvalidRequests(t *testing.T) {
|
||||||
|
for _, input := range []string{`{}`, `{"directory":".","expectedDigest":"latest"}`, `{"directory":"secret","expectedDigest":"bad","password":"do-not-echo"}`} {
|
||||||
|
code, out, diagnostics := run([]string{"verify-package"}, input)
|
||||||
|
if code == 0 || out != "" || strings.Contains(diagnostics, "do-not-echo") {
|
||||||
|
t.Fatal("invalid package request accepted or leaked")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,21 @@
|
|||||||
|
package cli
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestPreflightDoesNotAuthorizeInstallation(t *testing.T) {
|
||||||
|
code, out, diagnostics := run([]string{"preflight"}, "")
|
||||||
|
var response struct {
|
||||||
|
ProtocolVersion int `json:"protocolVersion"`
|
||||||
|
Mode string `json:"mode"`
|
||||||
|
Proposal struct {
|
||||||
|
Executable bool `json:"executable"`
|
||||||
|
Blockers []string `json:"blockers"`
|
||||||
|
} `json:"proposal"`
|
||||||
|
}
|
||||||
|
if code != 0 || json.Unmarshal([]byte(out), &response) != nil || response.ProtocolVersion != 1 || response.Mode != "local-environment-proposal" || response.Proposal.Executable || len(response.Proposal.Blockers) == 0 {
|
||||||
|
t.Fatalf("unsafe report: %s %s", out, diagnostics)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,37 @@
|
|||||||
|
package cli
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestRepositoryRejectsUntrustedRequests(t *testing.T) {
|
||||||
|
for _, input := range []string{`{}`, `{"directory":"secret-relative","suite":"resolute","architecture":"amd64","versions":{}}`,
|
||||||
|
`{"directory":"secret-relative","suite":"resolute","architecture":"amd64","versions":{},"repositoryAuthenticated":true}`} {
|
||||||
|
var out, diagnostics bytes.Buffer
|
||||||
|
if Run([]string{"verify-repository"}, strings.NewReader(input), &out, &diagnostics, time.Now) == 0 {
|
||||||
|
t.Fatal("accepted invalid request")
|
||||||
|
}
|
||||||
|
if out.Len() != 0 || strings.Contains(diagnostics.String(), "secret-relative") {
|
||||||
|
t.Fatal("invalid response leaked input")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestArtifactCommandRejectsCallerTrust(t *testing.T) {
|
||||||
|
for _, field := range []string{`"packageBytesVerified":true`, `"repositoryAuthenticated":true`, `"lock":{}`} {
|
||||||
|
var out, diagnostics bytes.Buffer
|
||||||
|
input := `{"directory":"secret-relative","artifactDirectory":"secret-artifacts","suite":"resolute","architecture":"amd64","versions":{},` + field + `}`
|
||||||
|
if Run([]string{"verify-artifacts"}, strings.NewReader(input), &out, &diagnostics, time.Now) == 0 {
|
||||||
|
t.Fatal("caller trust accepted")
|
||||||
|
}
|
||||||
|
if out.Len() != 0 || strings.Contains(diagnostics.String(), "secret") {
|
||||||
|
t.Fatal("input leaked")
|
||||||
|
}
|
||||||
|
if diagnostics.String() != "invalid artifact verification request\n" {
|
||||||
|
t.Fatal("request was not rejected at the protocol boundary", diagnostics.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,167 @@
|
|||||||
|
// Package cli exposes read-only protocol endpoints, not a shell wrapper.
|
||||||
|
package cli
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"server-deploy/internal/appbundle"
|
||||||
|
"server-deploy/internal/aptrepo"
|
||||||
|
"server-deploy/internal/composepolicy"
|
||||||
|
"server-deploy/internal/inspect"
|
||||||
|
"server-deploy/internal/installplan"
|
||||||
|
"server-deploy/internal/planner"
|
||||||
|
"server-deploy/internal/preflight"
|
||||||
|
)
|
||||||
|
|
||||||
|
func Run(args []string, in io.Reader, out, diagnostics io.Writer, now func() time.Time) int {
|
||||||
|
fail := func(message string) int { fmt.Fprintln(diagnostics, message); return 1 }
|
||||||
|
if len(args) != 1 {
|
||||||
|
return fail("usage: deployctl version | inspect | preflight | plan-environment | verify-repository | verify-artifacts | plan | verify-plan | verify-package | check-package")
|
||||||
|
}
|
||||||
|
var response any
|
||||||
|
switch args[0] {
|
||||||
|
case "verify-artifacts":
|
||||||
|
var request struct {
|
||||||
|
Directory string `json:"directory"`
|
||||||
|
ArtifactDirectory string `json:"artifactDirectory"`
|
||||||
|
Suite string `json:"suite"`
|
||||||
|
Architecture string `json:"architecture"`
|
||||||
|
Versions map[string]string `json:"versions"`
|
||||||
|
}
|
||||||
|
if decodeStrict(in, &request) != nil {
|
||||||
|
return fail("invalid artifact verification request")
|
||||||
|
}
|
||||||
|
verified, err := aptrepo.VerifyArtifacts(request.Directory, request.ArtifactDirectory, request.Suite, request.Architecture, request.Versions, now())
|
||||||
|
if err != nil {
|
||||||
|
return fail("artifact verification failed")
|
||||||
|
}
|
||||||
|
response = verified
|
||||||
|
case "verify-repository":
|
||||||
|
var request struct {
|
||||||
|
Directory string `json:"directory"`
|
||||||
|
Suite string `json:"suite"`
|
||||||
|
Architecture string `json:"architecture"`
|
||||||
|
Versions map[string]string `json:"versions"`
|
||||||
|
}
|
||||||
|
if decodeStrict(in, &request) != nil {
|
||||||
|
return fail("invalid repository verification request")
|
||||||
|
}
|
||||||
|
verified, err := aptrepo.Verify(request.Directory, request.Suite, request.Architecture, request.Versions, now())
|
||||||
|
if err != nil {
|
||||||
|
return fail("repository verification failed")
|
||||||
|
}
|
||||||
|
response = verified
|
||||||
|
case "plan-environment":
|
||||||
|
var request struct {
|
||||||
|
Lock installplan.Lock `json:"lock"`
|
||||||
|
}
|
||||||
|
if decodeStrict(in, &request) != nil {
|
||||||
|
return fail("invalid environment lock request")
|
||||||
|
}
|
||||||
|
report := preflight.Collect()
|
||||||
|
draft, err := installplan.Build(report, request.Lock)
|
||||||
|
if err != nil {
|
||||||
|
return fail("environment lock rejected")
|
||||||
|
}
|
||||||
|
response = struct {
|
||||||
|
ProtocolVersion int `json:"protocolVersion"`
|
||||||
|
Mode string `json:"mode"`
|
||||||
|
ObservedAt time.Time `json:"observedAt"`
|
||||||
|
Report preflight.Report `json:"report"`
|
||||||
|
Draft installplan.Draft `json:"draft"`
|
||||||
|
}{1, "local-environment-draft", now().UTC().Truncate(time.Second), report, draft}
|
||||||
|
case "preflight":
|
||||||
|
report := preflight.Collect()
|
||||||
|
response = struct {
|
||||||
|
ProtocolVersion int `json:"protocolVersion"`
|
||||||
|
Mode string `json:"mode"`
|
||||||
|
ObservedAt time.Time `json:"observedAt"`
|
||||||
|
Report preflight.Report `json:"report"`
|
||||||
|
Proposal preflight.Proposal `json:"proposal"`
|
||||||
|
}{1, "local-environment-proposal", now().UTC().Truncate(time.Second), report, preflight.Plan(report)}
|
||||||
|
case "verify-package", "check-package":
|
||||||
|
var request struct {
|
||||||
|
Directory string `json:"directory"`
|
||||||
|
ExpectedDigest string `json:"expectedDigest"`
|
||||||
|
}
|
||||||
|
if err := decodeStrict(in, &request); err != nil {
|
||||||
|
return fail("invalid package verification request")
|
||||||
|
}
|
||||||
|
verified, err := appbundle.Verify(request.Directory, request.ExpectedDigest)
|
||||||
|
if err != nil {
|
||||||
|
return fail("package verification failed: invalid manifest, inventory or digest")
|
||||||
|
}
|
||||||
|
if args[0] == "check-package" {
|
||||||
|
if composepolicy.Check(verified.Manifest, verified.Files[verified.Manifest.Entrypoint]) != nil {
|
||||||
|
return fail("package rejected by restricted Compose policy")
|
||||||
|
}
|
||||||
|
response = struct {
|
||||||
|
ProtocolVersion int `json:"protocolVersion"`
|
||||||
|
PolicyPassed bool `json:"policyPassed"`
|
||||||
|
Profile string `json:"profile"`
|
||||||
|
Digest string `json:"digest"`
|
||||||
|
Executable bool `json:"executable"`
|
||||||
|
PublisherAuthenticated bool `json:"publisherAuthenticated"`
|
||||||
|
}{1, true, composepolicy.Profile, verified.Digest, false, false}
|
||||||
|
break
|
||||||
|
}
|
||||||
|
response = struct {
|
||||||
|
ProtocolVersion int `json:"protocolVersion"`
|
||||||
|
Verified bool `json:"verified"`
|
||||||
|
Executable bool `json:"executable"`
|
||||||
|
PublisherAuthenticated bool `json:"publisherAuthenticated"`
|
||||||
|
Digest string `json:"digest"`
|
||||||
|
FileCount int `json:"fileCount"`
|
||||||
|
Manifest appbundle.Manifest `json:"manifest"`
|
||||||
|
}{1, true, false, false, verified.Digest, len(verified.Files), verified.Manifest}
|
||||||
|
case "inspect":
|
||||||
|
response = inspect.Collect()
|
||||||
|
case "version":
|
||||||
|
response = struct {
|
||||||
|
Version string `json:"version"`
|
||||||
|
ProtocolVersion int `json:"protocolVersion"`
|
||||||
|
WritesEnabled bool `json:"writesEnabled"`
|
||||||
|
}{"0.1.0-dev", planner.ProtocolVersion, false}
|
||||||
|
case "plan":
|
||||||
|
var intent planner.Intent
|
||||||
|
if err := decodeStrict(in, &intent); err != nil {
|
||||||
|
return fail("invalid request: expected strict protocol JSON (maximum 1 MiB)")
|
||||||
|
}
|
||||||
|
plan, err := planner.Build(intent, now())
|
||||||
|
if err != nil {
|
||||||
|
return fail("invalid deployment intent")
|
||||||
|
}
|
||||||
|
response = struct {
|
||||||
|
ProtocolVersion int `json:"protocolVersion"`
|
||||||
|
Mode string `json:"mode"`
|
||||||
|
Executable bool `json:"executable"`
|
||||||
|
Plan planner.Plan `json:"plan"`
|
||||||
|
}{planner.ProtocolVersion, "offline-preview", false, plan}
|
||||||
|
case "verify-plan":
|
||||||
|
var request struct {
|
||||||
|
Plan planner.Plan `json:"plan"`
|
||||||
|
Current planner.Intent `json:"current"`
|
||||||
|
}
|
||||||
|
if err := decodeStrict(in, &request); err != nil {
|
||||||
|
return fail("invalid verification request")
|
||||||
|
}
|
||||||
|
if err := request.Plan.Verify(request.Current, now()); err != nil {
|
||||||
|
return fail("plan rejected: expired, changed or invalid")
|
||||||
|
}
|
||||||
|
response = struct {
|
||||||
|
ProtocolVersion int `json:"protocolVersion"`
|
||||||
|
Mode string `json:"mode"`
|
||||||
|
Valid bool `json:"valid"`
|
||||||
|
Executable bool `json:"executable"`
|
||||||
|
}{planner.ProtocolVersion, "offline-preview", true, false}
|
||||||
|
default:
|
||||||
|
return fail("unsupported command; deployment writes are not enabled")
|
||||||
|
}
|
||||||
|
if err := json.NewEncoder(out).Encode(response); err != nil {
|
||||||
|
return fail("cannot write response")
|
||||||
|
}
|
||||||
|
return 0
|
||||||
|
}
|
||||||
@@ -0,0 +1,187 @@
|
|||||||
|
package cli
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"server-deploy/internal/planner"
|
||||||
|
)
|
||||||
|
|
||||||
|
func fixture() string {
|
||||||
|
return `{"protocolVersion":1,"hostId":"host-one","instanceId":"git-one","appId":"gitea","packageDigest":"sha256:` + strings.Repeat("a", 64) + `","imageDigest":"sha256:` + strings.Repeat("b", 64) + `","domain":"git.example.com","observedStateDigest":"sha256:` + strings.Repeat("c", 64) + `"}`
|
||||||
|
}
|
||||||
|
|
||||||
|
func run(args []string, input string) (int, string, string) {
|
||||||
|
var out, diagnostic bytes.Buffer
|
||||||
|
code := Run(args, strings.NewReader(input), &out, &diagnostic, func() time.Time { return time.Date(2026, 9, 25, 12, 0, 0, 0, time.UTC) })
|
||||||
|
return code, out.String(), diagnostic.String()
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPlanAndVerify(t *testing.T) {
|
||||||
|
code, output, diagnostic := run([]string{"plan"}, fixture())
|
||||||
|
if code != 0 || diagnostic != "" {
|
||||||
|
t.Fatalf("plan failed: %d %s", code, diagnostic)
|
||||||
|
}
|
||||||
|
var response struct {
|
||||||
|
ProtocolVersion int `json:"protocolVersion"`
|
||||||
|
Mode string `json:"mode"`
|
||||||
|
Executable bool `json:"executable"`
|
||||||
|
Plan planner.Plan `json:"plan"`
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal([]byte(output), &response); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if response.Mode != "offline-preview" || response.Executable || response.ProtocolVersion != 1 || response.Plan.ProjectName != "sd-git-one" {
|
||||||
|
t.Fatalf("misleading plan: %s", output)
|
||||||
|
}
|
||||||
|
request, _ := json.Marshal(struct {
|
||||||
|
Plan planner.Plan `json:"plan"`
|
||||||
|
Current planner.Intent `json:"current"`
|
||||||
|
}{response.Plan, response.Plan.Intent})
|
||||||
|
code, output, diagnostic = run([]string{"verify-plan"}, string(request))
|
||||||
|
if code != 0 || diagnostic != "" || !strings.Contains(output, `"valid":true`) {
|
||||||
|
t.Fatalf("verify failed: %d %s %s", code, output, diagnostic)
|
||||||
|
}
|
||||||
|
response.Plan.Hash = "tampered"
|
||||||
|
request, _ = json.Marshal(struct {
|
||||||
|
Plan planner.Plan `json:"plan"`
|
||||||
|
Current planner.Intent `json:"current"`
|
||||||
|
}{response.Plan, response.Plan.Intent})
|
||||||
|
code, _, _ = run([]string{"verify-plan"}, string(request))
|
||||||
|
if code == 0 {
|
||||||
|
t.Fatal("accepted tampered plan")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRejectsAmbiguousAndOversizedInput(t *testing.T) {
|
||||||
|
cases := []string{
|
||||||
|
``, `{}`, `null`, `[]`,
|
||||||
|
strings.TrimSuffix(fixture(), "}") + `,"password":"do-not-echo"}`,
|
||||||
|
strings.TrimSuffix(fixture(), "}") + `,"hostId":"other"}`,
|
||||||
|
strings.TrimSuffix(fixture(), "}") + `,"HostId":"other"}`,
|
||||||
|
strings.Replace(fixture(), `"hostId"`, `"HostId"`, 1),
|
||||||
|
strings.Replace(fixture(), `"protocolVersion":1`, `"protocolVersion":null`, 1),
|
||||||
|
fixture() + ` {}`, fixture() + ` trailing`,
|
||||||
|
strings.Repeat(" ", 1024*1024) + fixture(),
|
||||||
|
}
|
||||||
|
for _, input := range cases {
|
||||||
|
code, out, diagnostic := run([]string{"plan"}, input)
|
||||||
|
if code == 0 || out != "" || diagnostic == "" {
|
||||||
|
t.Fatalf("invalid input accepted: code=%d", code)
|
||||||
|
}
|
||||||
|
if strings.Contains(diagnostic, "do-not-echo") {
|
||||||
|
t.Fatal("secret leaked")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRejectsWriteCommandsAndUnexpectedArguments(t *testing.T) {
|
||||||
|
for _, args := range [][]string{nil, {"apply"}, {"upgrade"}, {"restore"}, {"plan", "extra"}, {"version", "extra"}} {
|
||||||
|
code, out, _ := run(args, fixture())
|
||||||
|
if code == 0 || out != "" {
|
||||||
|
t.Fatalf("accepted command %v", args)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestVersion(t *testing.T) {
|
||||||
|
code, out, diagnostic := run([]string{"version"}, "")
|
||||||
|
if code != 0 || diagnostic != "" || !json.Valid([]byte(out)) {
|
||||||
|
t.Fatal("version failed")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInspectIsReadOnlyAndExplicitlyIncomplete(t *testing.T) {
|
||||||
|
code, out, diagnostics := run([]string{"inspect"}, "")
|
||||||
|
if code != 0 || diagnostics != "" {
|
||||||
|
t.Fatalf("inspect failed: %s", diagnostics)
|
||||||
|
}
|
||||||
|
var report struct {
|
||||||
|
ProtocolVersion int `json:"protocolVersion"`
|
||||||
|
OS string `json:"os"`
|
||||||
|
DeploymentReady bool `json:"deploymentReady"`
|
||||||
|
DockerDaemon string `json:"dockerDaemon"`
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal([]byte(out), &report); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if report.ProtocolVersion != 1 || report.OS == "" || report.DeploymentReady || report.DockerDaemon != "not_checked" {
|
||||||
|
t.Fatalf("misleading report: %s", out)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestNestedVerificationRejectsDuplicateAndNullFields(t *testing.T) {
|
||||||
|
_, output, _ := run([]string{"plan"}, fixture())
|
||||||
|
var response struct {
|
||||||
|
Plan json.RawMessage `json:"plan"`
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal([]byte(output), &response); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
valid := `{"plan":` + string(response.Plan) + `,"current":` + fixture() + `}`
|
||||||
|
for _, input := range []string{
|
||||||
|
strings.Replace(valid, `"projectName":"sd-git-one"`, `"projectName":"sd-git-one","projectName":"sd-git-one"`, 1),
|
||||||
|
strings.Replace(valid, `"createdAt":"2026-09-25T12:00:00Z"`, `"createdAt":null`, 1),
|
||||||
|
strings.Replace(valid, `"intent":`, `"Intent":`, 1),
|
||||||
|
} {
|
||||||
|
code, out, _ := run([]string{"verify-plan"}, input)
|
||||||
|
if code == 0 || out != "" {
|
||||||
|
t.Fatal("accepted ambiguous nested request")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestVerificationRequiresCanonicalUTCTimestamps(t *testing.T) {
|
||||||
|
_, output, _ := run([]string{"plan"}, fixture())
|
||||||
|
var response struct {
|
||||||
|
Plan json.RawMessage `json:"plan"`
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal([]byte(output), &response); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
valid := `{"plan":` + string(response.Plan) + `,"current":` + fixture() + `}`
|
||||||
|
for _, timestamp := range []string{
|
||||||
|
"2026-09-25T12:00:00,000Z",
|
||||||
|
"2026-09-25T12:00:00.000Z",
|
||||||
|
"2026-09-25T12:00:00+00:00",
|
||||||
|
"2026-09-26T12:00:00+24:00",
|
||||||
|
"2026-09-25T13:00:00+00:60",
|
||||||
|
} {
|
||||||
|
t.Run(timestamp, func(t *testing.T) {
|
||||||
|
input := strings.Replace(valid, "2026-09-25T12:00:00Z", timestamp, 1)
|
||||||
|
code, out, _ := run([]string{"verify-plan"}, input)
|
||||||
|
if code == 0 || out != "" {
|
||||||
|
t.Fatal("accepted noncanonical timestamp")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
type brokenReader struct{}
|
||||||
|
|
||||||
|
func (brokenReader) Read([]byte) (int, error) { return 0, errors.New("secret reader error") }
|
||||||
|
|
||||||
|
type brokenWriter struct{}
|
||||||
|
|
||||||
|
func (brokenWriter) Write([]byte) (int, error) { return 0, errors.New("secret writer error") }
|
||||||
|
|
||||||
|
func TestIOErrorsFailWithoutLeakingDetails(t *testing.T) {
|
||||||
|
var out, diagnostics bytes.Buffer
|
||||||
|
if Run([]string{"plan"}, brokenReader{}, &out, &diagnostics, time.Now) == 0 {
|
||||||
|
t.Fatal("ignored read error")
|
||||||
|
}
|
||||||
|
if out.Len() != 0 || strings.Contains(diagnostics.String(), "secret") {
|
||||||
|
t.Fatal("leaked input error")
|
||||||
|
}
|
||||||
|
diagnostics.Reset()
|
||||||
|
if Run([]string{"version"}, strings.NewReader(""), brokenWriter{}, &diagnostics, time.Now) == 0 {
|
||||||
|
t.Fatal("ignored write error")
|
||||||
|
}
|
||||||
|
if strings.Contains(diagnostics.String(), "secret") {
|
||||||
|
t.Fatal("leaked output error")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,78 @@
|
|||||||
|
# Restricted Compose policy
|
||||||
|
|
||||||
|
`Check(manifest, entrypointBytes)` is a pure, offline policy check. The CLI
|
||||||
|
`check-package` first calls `appbundle.Verify`, then passes the authenticated
|
||||||
|
entrypoint snapshot here. It never reopens the entrypoint, runs Compose, reads
|
||||||
|
`.env`, resolves templates, or contacts a daemon. Direct internal callers must
|
||||||
|
perform the same integrity/trust check first.
|
||||||
|
|
||||||
|
Profile: `isolated-compose-v1`. This is an initial restricted backend profile,
|
||||||
|
not the finished application's Compose contract. It deliberately rejects public
|
||||||
|
routing, secrets/config injection, environment settings, healthchecks, dependency
|
||||||
|
ordering and application-specific privilege exceptions until adapters exist.
|
||||||
|
Existing YAML packages can still pass `verify-package`; that does not mean they
|
||||||
|
pass `check-package`. Only JSON entrypoint content is accepted by this policy.
|
||||||
|
|
||||||
|
## Exact shape
|
||||||
|
|
||||||
|
All fields below are mandatory, all unlisted fields are rejected:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"services": {
|
||||||
|
"api": {
|
||||||
|
"image": "example/api@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
||||||
|
"user": "1000:1000",
|
||||||
|
"read_only": true,
|
||||||
|
"cap_drop": ["ALL"],
|
||||||
|
"security_opt": ["no-new-privileges:true"],
|
||||||
|
"restart": "no",
|
||||||
|
"networks": ["backend"],
|
||||||
|
"volumes": [
|
||||||
|
{"type": "volume", "source": "data", "target": "/data", "read_only": false}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"networks": {"backend": {"internal": true}},
|
||||||
|
"volumes": {"data": {}}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
The example digest is synthetic, not an installable release.
|
||||||
|
|
||||||
|
- 4 MiB input limit. Shared strict decoder rejects duplicate, case-alias,
|
||||||
|
unknown, missing and null fields, including typed map values.
|
||||||
|
- 1–32 services, exactly matching manifest component names and pinned images.
|
||||||
|
- UID and GID must be canonical positive uint32 decimals, excluding 4294967295.
|
||||||
|
No root, account-name lookup, interpolation or inherited user defaults.
|
||||||
|
- Restart must be `no` or `unless-stopped`; root filesystem must be read-only,
|
||||||
|
all capabilities dropped and privilege escalation disabled.
|
||||||
|
- Exactly one network: `backend`, with `internal: true`. No default network,
|
||||||
|
external network, host networking, published port or arbitrary router label.
|
||||||
|
- At most 128 plain named volumes; every declaration must be mounted exactly
|
||||||
|
once. Empty volume maps/lists are permitted for stateless services. No external
|
||||||
|
names, drivers, driver options, bind mounts or cross-service sharing.
|
||||||
|
- Mount paths must be absolute canonical ASCII paths, at most 240 bytes, with
|
||||||
|
no root, overlapping mount or system-tree mount. Denied trees: /proc, /sys,
|
||||||
|
/dev, /etc, /run, /var/run, /bin, /sbin, /usr, /lib, /lib64 (including ancestors).
|
||||||
|
- No command overrides, hooks, build, include, extends, profile, socket access,
|
||||||
|
devices or arbitrary privilege additions. Unknown future keys also fail closed.
|
||||||
|
|
||||||
|
## What passing does not prove
|
||||||
|
|
||||||
|
This check does not authenticate a publisher, validate image contents or mount
|
||||||
|
destinations inside an image, provision usable volume ownership, reserve resource
|
||||||
|
names, verify engine/Compose compatibility, limit resource consumption, prove
|
||||||
|
application readiness, or provide backup/restore. Image defaults and existing
|
||||||
|
Docker resources must still be validated by future adapters and preflight.
|
||||||
|
|
||||||
|
Future execution must bind an explicit instance project name, verify resource
|
||||||
|
ownership under the host lock, and use the exact checked snapshot without extra
|
||||||
|
Compose files, ambient overrides or subsequent interpolation. An integrity hash
|
||||||
|
and this restricted policy are not authorization to run a deployment.
|
||||||
|
|
||||||
|
References checked during implementation:
|
||||||
|
[Compose services](https://docs.docker.com/reference/compose-file/services/),
|
||||||
|
[Compose networks](https://docs.docker.com/reference/compose-file/networks/),
|
||||||
|
[Compose config](https://docs.docker.com/reference/cli/docker/compose/config/).
|
||||||
|
No real Docker/Compose execution has been validated in this batch.
|
||||||
@@ -0,0 +1,135 @@
|
|||||||
|
// Package composepolicy validates a deliberately restricted, offline Compose
|
||||||
|
// profile. Passing this policy never authorizes execution or proves image safety.
|
||||||
|
package composepolicy
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"errors"
|
||||||
|
"path"
|
||||||
|
"regexp"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"server-deploy/internal/appbundle"
|
||||||
|
"server-deploy/internal/wire"
|
||||||
|
)
|
||||||
|
|
||||||
|
const Profile = "isolated-compose-v1"
|
||||||
|
|
||||||
|
type document struct {
|
||||||
|
Services map[string]service `json:"services"`
|
||||||
|
Networks map[string]network `json:"networks"`
|
||||||
|
Volumes map[string]struct{} `json:"volumes"`
|
||||||
|
}
|
||||||
|
type network struct {
|
||||||
|
Internal bool `json:"internal"`
|
||||||
|
}
|
||||||
|
type service struct {
|
||||||
|
Image string `json:"image"`
|
||||||
|
User string `json:"user"`
|
||||||
|
ReadOnly bool `json:"read_only"`
|
||||||
|
CapDrop []string `json:"cap_drop"`
|
||||||
|
SecurityOpt []string `json:"security_opt"`
|
||||||
|
Restart string `json:"restart"`
|
||||||
|
Networks []string `json:"networks"`
|
||||||
|
Volumes []mount `json:"volumes"`
|
||||||
|
}
|
||||||
|
type mount struct {
|
||||||
|
Type string `json:"type"`
|
||||||
|
Source string `json:"source"`
|
||||||
|
Target string `json:"target"`
|
||||||
|
ReadOnly bool `json:"read_only"`
|
||||||
|
}
|
||||||
|
|
||||||
|
var identifier = regexp.MustCompile(`^[a-z][a-z0-9-]{0,47}$`)
|
||||||
|
var pinnedImage = regexp.MustCompile(`^[a-z0-9][a-z0-9._/-]*@sha256:[0-9a-f]{64}$`)
|
||||||
|
var targetPath = regexp.MustCompile(`^/[a-zA-Z0-9_./-]+$`)
|
||||||
|
|
||||||
|
// Check consumes only the authenticated entrypoint bytes and manifest supplied
|
||||||
|
// by appbundle.Verify. It neither reads files nor renders/interpolates templates.
|
||||||
|
// All fields in the profile are mandatory; unknown Compose features fail closed.
|
||||||
|
func Check(manifest appbundle.Manifest, data []byte) error {
|
||||||
|
reject := errors.New("Compose document rejected by restricted policy")
|
||||||
|
var d document
|
||||||
|
if wire.Decode(bytes.NewReader(data), &d, 4<<20) != nil {
|
||||||
|
return reject
|
||||||
|
}
|
||||||
|
if len(manifest.Components) < 1 || len(manifest.Components) > 32 || len(d.Services) != len(manifest.Components) {
|
||||||
|
return reject
|
||||||
|
}
|
||||||
|
if len(d.Networks) != 1 || !d.Networks["backend"].Internal || len(d.Volumes) > 128 {
|
||||||
|
return reject
|
||||||
|
}
|
||||||
|
images := make(map[string]string, len(manifest.Components))
|
||||||
|
for _, c := range manifest.Components {
|
||||||
|
if !identifier.MatchString(c.Name) || !pinnedImage.MatchString(c.Image) || images[c.Name] != "" {
|
||||||
|
return reject
|
||||||
|
}
|
||||||
|
images[c.Name] = c.Image
|
||||||
|
}
|
||||||
|
for name := range d.Volumes {
|
||||||
|
if !identifier.MatchString(name) {
|
||||||
|
return reject
|
||||||
|
}
|
||||||
|
}
|
||||||
|
used := make(map[string]bool)
|
||||||
|
for name, s := range d.Services {
|
||||||
|
if images[name] == "" || s.Image != images[name] || !nonRootUser(s.User) || !s.ReadOnly {
|
||||||
|
return reject
|
||||||
|
}
|
||||||
|
if !only(s.CapDrop, "ALL") || !only(s.SecurityOpt, "no-new-privileges:true") || !only(s.Networks, "backend") {
|
||||||
|
return reject
|
||||||
|
}
|
||||||
|
if s.Restart != "unless-stopped" && s.Restart != "no" {
|
||||||
|
return reject
|
||||||
|
}
|
||||||
|
if len(s.Volumes) > 128 {
|
||||||
|
return reject
|
||||||
|
}
|
||||||
|
targets := make([]string, 0, len(s.Volumes))
|
||||||
|
for _, v := range s.Volumes {
|
||||||
|
if _, exists := d.Volumes[v.Source]; !exists || used[v.Source] || v.Type != "volume" {
|
||||||
|
return reject
|
||||||
|
}
|
||||||
|
if len(v.Target) > 240 || !targetPath.MatchString(v.Target) || path.Clean(v.Target) != v.Target || v.Target == "/" {
|
||||||
|
return reject
|
||||||
|
}
|
||||||
|
// Deny runtime/system trees as well as overlapping mounts. Only
|
||||||
|
// application-data destinations belong in this initial profile.
|
||||||
|
for _, protected := range []string{"/proc", "/sys", "/dev", "/etc", "/run", "/var/run", "/bin", "/sbin", "/usr", "/lib", "/lib64"} {
|
||||||
|
if overlaps(v.Target, protected) {
|
||||||
|
return reject
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, previous := range targets {
|
||||||
|
if overlaps(v.Target, previous) {
|
||||||
|
return reject
|
||||||
|
}
|
||||||
|
}
|
||||||
|
targets = append(targets, v.Target)
|
||||||
|
used[v.Source] = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(used) != len(d.Volumes) {
|
||||||
|
return reject
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func only(values []string, expected string) bool { return len(values) == 1 && values[0] == expected }
|
||||||
|
func overlaps(a, b string) bool {
|
||||||
|
return a == b || strings.HasPrefix(a, b+"/") || strings.HasPrefix(b, a+"/")
|
||||||
|
}
|
||||||
|
func nonRootUser(value string) bool {
|
||||||
|
parts := strings.Split(value, ":")
|
||||||
|
if len(parts) != 2 {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
for _, part := range parts {
|
||||||
|
n, err := strconv.ParseUint(part, 10, 32)
|
||||||
|
if err != nil || n == 0 || n == 4294967295 || strconv.FormatUint(n, 10) != part {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
@@ -0,0 +1,161 @@
|
|||||||
|
package composepolicy
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"server-deploy/internal/appbundle"
|
||||||
|
)
|
||||||
|
|
||||||
|
func fixture() (appbundle.Manifest, map[string]any) {
|
||||||
|
image := "example/api@sha256:" + strings.Repeat("a", 64)
|
||||||
|
m := appbundle.Manifest{Components: []appbundle.Component{{Name: "api", Image: image}}}
|
||||||
|
d := map[string]any{
|
||||||
|
"services": map[string]any{"api": map[string]any{
|
||||||
|
"image": image, "user": "1000:1000", "read_only": true,
|
||||||
|
"cap_drop": []string{"ALL"}, "security_opt": []string{"no-new-privileges:true"},
|
||||||
|
"restart": "unless-stopped", "networks": []string{"backend"},
|
||||||
|
"volumes": []any{map[string]any{"type": "volume", "source": "data", "target": "/data", "read_only": false}},
|
||||||
|
}},
|
||||||
|
"networks": map[string]any{"backend": map[string]any{"internal": true}},
|
||||||
|
"volumes": map[string]any{"data": map[string]any{}},
|
||||||
|
}
|
||||||
|
return m, d
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAcceptRestrictedService(t *testing.T) {
|
||||||
|
m, d := fixture()
|
||||||
|
raw, _ := json.Marshal(d)
|
||||||
|
if err := Check(m, raw); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRejectPrivilegeAndExternalInputs(t *testing.T) {
|
||||||
|
for _, field := range []string{"privileged", "build", "container_name", "network_mode", "pid", "ipc", "userns_mode", "devices", "cap_add", "env_file", "environment", "extends", "ports", "labels", "use_api_socket", "post_start", "pre_stop", "command", "entrypoint", "volumes_from", "develop", "provider"} {
|
||||||
|
t.Run(field, func(t *testing.T) {
|
||||||
|
m, d := fixture()
|
||||||
|
d["services"].(map[string]any)["api"].(map[string]any)[field] = "secret-sentinel"
|
||||||
|
raw, _ := json.Marshal(d)
|
||||||
|
err := Check(m, raw)
|
||||||
|
if err == nil || strings.Contains(err.Error(), "secret-sentinel") {
|
||||||
|
t.Fatal("forbidden field accepted or echoed")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRejectUnsafeValuesAndReferences(t *testing.T) {
|
||||||
|
mutations := map[string]func(map[string]any, map[string]any){
|
||||||
|
"image drift": func(d, s map[string]any) { s["image"] = "example/api:latest" },
|
||||||
|
"root": func(d, s map[string]any) { s["user"] = "0:0" },
|
||||||
|
"named user": func(d, s map[string]any) { s["user"] = "root" },
|
||||||
|
"writable root": func(d, s map[string]any) { s["read_only"] = false },
|
||||||
|
"caps": func(d, s map[string]any) { s["cap_drop"] = []string{} },
|
||||||
|
"escalation": func(d, s map[string]any) { s["security_opt"] = []string{"seccomp:unconfined"} },
|
||||||
|
"implicit network": func(d, s map[string]any) { s["networks"] = []string{} },
|
||||||
|
"other network": func(d, s map[string]any) { s["networks"] = []string{"default"} },
|
||||||
|
"external network": func(d, s map[string]any) {
|
||||||
|
d["networks"] = map[string]any{"backend": map[string]any{"internal": true, "external": true}}
|
||||||
|
},
|
||||||
|
"outbound network": func(d, s map[string]any) {
|
||||||
|
d["networks"] = map[string]any{"backend": map[string]any{"internal": false}}
|
||||||
|
},
|
||||||
|
"bind": func(d, s map[string]any) { s["volumes"].([]any)[0].(map[string]any)["type"] = "bind" },
|
||||||
|
"host source": func(d, s map[string]any) { s["volumes"].([]any)[0].(map[string]any)["source"] = "/var/run/docker.sock" },
|
||||||
|
"missing volume": func(d, s map[string]any) { d["volumes"] = map[string]any{} },
|
||||||
|
"volume driver": func(d, s map[string]any) {
|
||||||
|
d["volumes"] = map[string]any{"data": map[string]any{"driver_opts": map[string]string{"device": "/"}}}
|
||||||
|
},
|
||||||
|
"root mount": func(d, s map[string]any) { s["volumes"].([]any)[0].(map[string]any)["target"] = "/" },
|
||||||
|
"traversal": func(d, s map[string]any) { s["volumes"].([]any)[0].(map[string]any)["target"] = "/data/../etc" },
|
||||||
|
"interpolation": func(d, s map[string]any) { s["user"] = "${UID}:1000" },
|
||||||
|
"include": func(d, s map[string]any) { d["include"] = []string{"/secret"} },
|
||||||
|
"null service": func(d, s map[string]any) { d["services"].(map[string]any)["api"] = nil },
|
||||||
|
"missing security": func(d, s map[string]any) { delete(s, "security_opt") },
|
||||||
|
"null volume": func(d, s map[string]any) { d["volumes"] = map[string]any{"data": nil} },
|
||||||
|
"extra service": func(d, s map[string]any) { d["services"].(map[string]any)["rogue"] = s },
|
||||||
|
"no services": func(d, s map[string]any) { d["services"] = map[string]any{} },
|
||||||
|
}
|
||||||
|
for name, mutate := range mutations {
|
||||||
|
t.Run(name, func(t *testing.T) {
|
||||||
|
m, d := fixture()
|
||||||
|
s := d["services"].(map[string]any)["api"].(map[string]any)
|
||||||
|
mutate(d, s)
|
||||||
|
raw, _ := json.Marshal(d)
|
||||||
|
if Check(m, raw) == nil {
|
||||||
|
t.Fatal("unsafe document accepted")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRejectMalformedDocument(t *testing.T) {
|
||||||
|
m, d := fixture()
|
||||||
|
raw, _ := json.Marshal(d)
|
||||||
|
for _, input := range []string{"services: {}", string(raw) + "{}", strings.Replace(string(raw), `"read_only":true`, `"read_only":true,"read_only":false`, 1), strings.Repeat(" ", 4<<20) + string(raw)} {
|
||||||
|
if Check(m, []byte(input)) == nil {
|
||||||
|
t.Fatal("malformed input accepted")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestMountAndIdentityBoundaries(t *testing.T) {
|
||||||
|
for _, target := range []string{"/proc/x", "/var", "/var/run/docker.sock", "/sys", "/dev", "/etc", "/run", "/usr/local", "/lib", "/lib64", "/bin", "/sbin", "/data/", "/data//x", "/data/$HOME", `C:\data`, "/" + strings.Repeat("a", 241)} {
|
||||||
|
m, d := fixture()
|
||||||
|
s := d["services"].(map[string]any)["api"].(map[string]any)
|
||||||
|
s["volumes"].([]any)[0].(map[string]any)["target"] = target
|
||||||
|
raw, _ := json.Marshal(d)
|
||||||
|
if Check(m, raw) == nil {
|
||||||
|
t.Errorf("accepted protected/noncanonical target %q", target)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, user := range []string{"1000:0", "01:1000", "+1:1000", "-1:1000", "4294967295:1000", "4294967296:1000", "1", "1:2:3"} {
|
||||||
|
m, d := fixture()
|
||||||
|
d["services"].(map[string]any)["api"].(map[string]any)["user"] = user
|
||||||
|
raw, _ := json.Marshal(d)
|
||||||
|
if Check(m, raw) == nil {
|
||||||
|
t.Errorf("accepted ambiguous/root identity %q", user)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestTwoServicesRequireExactManifestAndSeparateVolumes(t *testing.T) {
|
||||||
|
m, d := fixture()
|
||||||
|
raw, _ := json.Marshal(d)
|
||||||
|
var copyDoc map[string]any
|
||||||
|
if err := json.Unmarshal(raw, ©Doc); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
second := copyDoc["services"].(map[string]any)["api"].(map[string]any)
|
||||||
|
second["volumes"].([]any)[0].(map[string]any)["source"] = "db-data"
|
||||||
|
d["services"].(map[string]any)["db"] = second
|
||||||
|
d["volumes"].(map[string]any)["db-data"] = map[string]any{}
|
||||||
|
m.Components = append(m.Components, appbundle.Component{Name: "db", Image: m.Components[0].Image})
|
||||||
|
raw, _ = json.Marshal(d)
|
||||||
|
if err := Check(m, raw); err != nil {
|
||||||
|
t.Fatal("valid separate-service volumes rejected", err)
|
||||||
|
}
|
||||||
|
second["volumes"].([]any)[0].(map[string]any)["source"] = "data"
|
||||||
|
delete(d["volumes"].(map[string]any), "db-data")
|
||||||
|
raw, _ = json.Marshal(d)
|
||||||
|
if Check(m, raw) == nil {
|
||||||
|
t.Fatal("shared service data accepted")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestOverlappingAndUnusedVolumesRejected(t *testing.T) {
|
||||||
|
for _, target := range []string{"/data", "/data/nested", "/other"} {
|
||||||
|
m, d := fixture()
|
||||||
|
d["volumes"].(map[string]any)["second"] = map[string]any{}
|
||||||
|
if target != "/other" {
|
||||||
|
s := d["services"].(map[string]any)["api"].(map[string]any)
|
||||||
|
s["volumes"] = append(s["volumes"].([]any), map[string]any{"type": "volume", "source": "second", "target": target, "read_only": false})
|
||||||
|
}
|
||||||
|
raw, _ := json.Marshal(d)
|
||||||
|
if Check(m, raw) == nil {
|
||||||
|
t.Fatal("overlapping or unused volume accepted")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,33 @@
|
|||||||
|
# Read-only relevant dpkg inventory
|
||||||
|
|
||||||
|
`Inventory(fs.FS) Snapshot` reads fixed `var/lib/dpkg/status` (regular file,
|
||||||
|
nonempty, at most 16 MiB) and checks the fixed `var/lib/dpkg/updates` directory
|
||||||
|
is empty before and after reading. It invokes no package tools and writes nothing.
|
||||||
|
The filesystem must provide metadata via `fs.StatFS`; unsupported, inaccessible,
|
||||||
|
malformed, oversized, changing or journal-busy input returns `state=unknown`, an
|
||||||
|
empty digest and empty packages array. Missing status is not a clean machine.
|
||||||
|
|
||||||
|
It validates stanza structure/identity/status before filtering. Field names are
|
||||||
|
case-insensitive; duplicate keys, malformed scalar continuations and duplicate
|
||||||
|
package/architecture records are rejected. Descriptions/other values are never
|
||||||
|
returned. Distinct multiarch records are retained; ambiguous all/unspecified
|
||||||
|
architecture duplicates fail closed. Bare not-installed selections are permitted
|
||||||
|
and still returned when relevant, so a caller cannot mistake them for no record.
|
||||||
|
|
||||||
|
An observed snapshot contains the SHA-256 of the complete status file bytes and
|
||||||
|
deterministically ordered name/version/architecture/status records for:
|
||||||
|
docker-ce, docker-ce-cli, containerd.io, docker-buildx-plugin,
|
||||||
|
docker-compose-plugin, docker.io, docker-compose, docker-compose-v2, docker-doc,
|
||||||
|
docker-buildx, podman-docker, containerd and runc.
|
||||||
|
|
||||||
|
The `Installed` type means a database record exists, not that it is fully
|
||||||
|
installed. Callers must conservatively review **every** returned record, including
|
||||||
|
hold, partial installation, residual config and not-installed selections.
|
||||||
|
|
||||||
|
OS paths/ancestors and filesystem implementation are trusted. Metadata/journal
|
||||||
|
rechecks detect ordinary changes but do not lock dpkg or produce an atomic
|
||||||
|
transaction against concurrent writes. The digest is not a host identity or
|
||||||
|
approval. This does not scan custom package databases, rootless/manual runtimes,
|
||||||
|
APT sources, package dependencies, or unrelated packages' operational health.
|
||||||
|
|
||||||
|
Format reference: [Debian control files](https://www.debian.org/doc/debian-policy/ch-controlfields.html).
|
||||||
@@ -0,0 +1,267 @@
|
|||||||
|
// Package debian observes the dpkg database without executing package tools.
|
||||||
|
package debian
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bufio"
|
||||||
|
"bytes"
|
||||||
|
"crypto/sha256"
|
||||||
|
"encoding/hex"
|
||||||
|
"io"
|
||||||
|
"io/fs"
|
||||||
|
"regexp"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
"unicode/utf8"
|
||||||
|
)
|
||||||
|
|
||||||
|
type Snapshot struct {
|
||||||
|
State string `json:"state"`
|
||||||
|
Digest string `json:"digest"`
|
||||||
|
Packages []Installed `json:"packages"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Installed is a present database record, including residual or uninstalled selections.
|
||||||
|
type Installed struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
Version string `json:"version"`
|
||||||
|
Architecture string `json:"architecture"`
|
||||||
|
Status string `json:"status"`
|
||||||
|
}
|
||||||
|
|
||||||
|
const (
|
||||||
|
statusPath = "var/lib/dpkg/status"
|
||||||
|
updatesPath = "var/lib/dpkg/updates"
|
||||||
|
maxStatusBytes = 16 << 20
|
||||||
|
)
|
||||||
|
|
||||||
|
// Inventory reads only the fixed status and updates paths. An observed snapshot
|
||||||
|
// includes every relevant record, regardless of installation state; it is not
|
||||||
|
// an installation permission. Unknown never exposes a partial result or error.
|
||||||
|
// Paths and the FS implementation are trusted. Metadata and journal rechecks
|
||||||
|
// detect ordinary changes, but are not a lock or protection against a hostile
|
||||||
|
// administrator replacing paths between checks.
|
||||||
|
func Inventory(files fs.FS) Snapshot {
|
||||||
|
unknown := Snapshot{State: "unknown", Packages: []Installed{}}
|
||||||
|
// fs.Stat's fallback opens the path. Require a metadata operation so that
|
||||||
|
// checking an already-present FIFO cannot block before we reject its type.
|
||||||
|
metadata, ok := files.(fs.StatFS)
|
||||||
|
if !ok || !emptyJournal(files, metadata) {
|
||||||
|
return unknown
|
||||||
|
}
|
||||||
|
initial, err := metadata.Stat(statusPath)
|
||||||
|
if err != nil || !validStatusFile(initial) {
|
||||||
|
return unknown
|
||||||
|
}
|
||||||
|
f, err := files.Open(statusPath)
|
||||||
|
if err != nil {
|
||||||
|
return unknown
|
||||||
|
}
|
||||||
|
opened, err := f.Stat()
|
||||||
|
if err != nil || !sameMetadata(initial, opened) {
|
||||||
|
f.Close()
|
||||||
|
return unknown
|
||||||
|
}
|
||||||
|
raw, readErr := io.ReadAll(io.LimitReader(f, maxStatusBytes+1))
|
||||||
|
after, statErr := f.Stat()
|
||||||
|
closeErr := f.Close()
|
||||||
|
if readErr != nil || statErr != nil || closeErr != nil || len(raw) == 0 || len(raw) > maxStatusBytes || int64(len(raw)) != initial.Size() || !sameMetadata(initial, after) {
|
||||||
|
return unknown
|
||||||
|
}
|
||||||
|
current, err := metadata.Stat(statusPath)
|
||||||
|
if err != nil || !sameMetadata(initial, current) {
|
||||||
|
return unknown
|
||||||
|
}
|
||||||
|
packages, ok := parseStatus(raw)
|
||||||
|
if !ok || !emptyJournal(files, metadata) {
|
||||||
|
return unknown
|
||||||
|
}
|
||||||
|
sum := sha256.Sum256(raw)
|
||||||
|
return Snapshot{State: "observed", Digest: "sha256:" + hex.EncodeToString(sum[:]), Packages: packages}
|
||||||
|
}
|
||||||
|
|
||||||
|
func validStatusFile(info fs.FileInfo) bool {
|
||||||
|
return info != nil && info.Mode().IsRegular() && info.Size() > 0 && info.Size() <= maxStatusBytes
|
||||||
|
}
|
||||||
|
|
||||||
|
func sameMetadata(a, b fs.FileInfo) bool {
|
||||||
|
return a != nil && b != nil && a.Mode() == b.Mode() && a.Size() == b.Size() && a.ModTime().Equal(b.ModTime())
|
||||||
|
}
|
||||||
|
|
||||||
|
func emptyJournal(files fs.FS, metadata fs.StatFS) bool {
|
||||||
|
initial, err := metadata.Stat(updatesPath)
|
||||||
|
if err != nil || initial == nil || !initial.IsDir() {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
f, err := files.Open(updatesPath)
|
||||||
|
if err != nil {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
opened, statErr := f.Stat()
|
||||||
|
dir, ok := f.(fs.ReadDirFile)
|
||||||
|
if statErr != nil || !sameMetadata(initial, opened) || !ok {
|
||||||
|
f.Close()
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
// Read at most one entry: even a hidden file or a directory is pending work.
|
||||||
|
entries, readErr := dir.ReadDir(1)
|
||||||
|
closeErr := f.Close()
|
||||||
|
return len(entries) == 0 && readErr == io.EOF && closeErr == nil
|
||||||
|
}
|
||||||
|
|
||||||
|
var packageName = regexp.MustCompile(`^[a-z0-9][a-z0-9+.-]+$`)
|
||||||
|
var architectureName = regexp.MustCompile(`^[a-z0-9][a-z0-9-]*$`)
|
||||||
|
var packageVersion = regexp.MustCompile(`^(?:[0-9]+:)?[0-9][A-Za-z0-9.+:~\-]*$`)
|
||||||
|
|
||||||
|
func relevant(name string) bool {
|
||||||
|
switch name {
|
||||||
|
case "docker-ce", "docker-ce-cli", "containerd.io", "docker-buildx-plugin", "docker-compose-plugin", "docker.io", "docker-compose", "docker-compose-v2", "docker-doc", "docker-buildx", "podman-docker", "containerd", "runc":
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
func scalarField(key string) bool {
|
||||||
|
return key == "package" || key == "status" || key == "architecture" || key == "version"
|
||||||
|
}
|
||||||
|
|
||||||
|
// Validate structure and identifying fields in ALL stanzas, before filtering.
|
||||||
|
// Other field values (including descriptions) are never part of the snapshot.
|
||||||
|
func parseStatus(raw []byte) ([]Installed, bool) {
|
||||||
|
if !utf8.Valid(raw) {
|
||||||
|
return nil, false
|
||||||
|
}
|
||||||
|
packages := []Installed{}
|
||||||
|
seen := make(map[string]map[string]bool)
|
||||||
|
fields := make(map[string]string)
|
||||||
|
last := ""
|
||||||
|
count := 0
|
||||||
|
finish := func() bool {
|
||||||
|
if len(fields) == 0 {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
p := Installed{Name: fields["package"], Version: fields["version"], Architecture: fields["architecture"]}
|
||||||
|
status, ok := normalizedStatus(fields["status"])
|
||||||
|
if !ok || !packageName.MatchString(p.Name) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
p.Status = status
|
||||||
|
// dpkg may retain a bare selection for a never-installed package.
|
||||||
|
notInstalled := strings.HasSuffix(status, " not-installed")
|
||||||
|
if p.Architecture == "" {
|
||||||
|
if !notInstalled || hasField(fields, "architecture") {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
} else if !architectureName.MatchString(p.Architecture) || p.Architecture == "any" || p.Architecture == "source" || strings.HasPrefix(p.Architecture, "any-") || strings.HasSuffix(p.Architecture, "-any") {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
if p.Version == "" {
|
||||||
|
if !notInstalled || hasField(fields, "version") {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
} else if !packageVersion.MatchString(p.Version) || strings.HasSuffix(p.Version, "-") || strings.HasSuffix(p.Version, ":") {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
arches := seen[p.Name]
|
||||||
|
if len(arches) != 0 && (arches[p.Architecture] || arches[""] || arches["all"] || p.Architecture == "" || p.Architecture == "all") {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
if arches == nil {
|
||||||
|
arches = make(map[string]bool)
|
||||||
|
seen[p.Name] = arches
|
||||||
|
}
|
||||||
|
arches[p.Architecture] = true
|
||||||
|
if relevant(p.Name) {
|
||||||
|
packages = append(packages, p)
|
||||||
|
}
|
||||||
|
count++
|
||||||
|
fields = make(map[string]string)
|
||||||
|
last = ""
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
scanner := bufio.NewScanner(bytes.NewReader(raw))
|
||||||
|
// The file cap is also the token cap; long legitimate description lines
|
||||||
|
// must not be silently lost to Scanner's default 64 KiB limit.
|
||||||
|
scanner.Buffer(make([]byte, 4096), maxStatusBytes+1)
|
||||||
|
for scanner.Scan() {
|
||||||
|
line := scanner.Text()
|
||||||
|
for _, c := range line {
|
||||||
|
if (c < 32 && c != '\t') || c == 127 {
|
||||||
|
return nil, false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if strings.Trim(line, " \t") == "" {
|
||||||
|
if !finish() {
|
||||||
|
return nil, false
|
||||||
|
}
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if line[0] == ' ' || line[0] == '\t' {
|
||||||
|
if last == "" || scalarField(last) {
|
||||||
|
return nil, false
|
||||||
|
}
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
key, value, ok := strings.Cut(line, ":")
|
||||||
|
if !ok || !validFieldName(key) {
|
||||||
|
return nil, false
|
||||||
|
}
|
||||||
|
key = strings.ToLower(key)
|
||||||
|
if hasField(fields, key) {
|
||||||
|
return nil, false
|
||||||
|
}
|
||||||
|
// Retain only the values we project, but track every key for duplicates.
|
||||||
|
fields[key] = ""
|
||||||
|
if scalarField(key) {
|
||||||
|
fields[key] = strings.Trim(value, " \t")
|
||||||
|
}
|
||||||
|
last = key
|
||||||
|
}
|
||||||
|
if scanner.Err() != nil || !finish() || count == 0 {
|
||||||
|
return nil, false
|
||||||
|
}
|
||||||
|
sort.Slice(packages, func(i, j int) bool {
|
||||||
|
if packages[i].Name != packages[j].Name {
|
||||||
|
return packages[i].Name < packages[j].Name
|
||||||
|
}
|
||||||
|
return packages[i].Architecture < packages[j].Architecture
|
||||||
|
})
|
||||||
|
return packages, true
|
||||||
|
}
|
||||||
|
|
||||||
|
func hasField(fields map[string]string, key string) bool {
|
||||||
|
_, ok := fields[key]
|
||||||
|
return ok
|
||||||
|
}
|
||||||
|
|
||||||
|
func validFieldName(key string) bool {
|
||||||
|
if key == "" || key[0] == '#' || key[0] == '-' {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
for i := range len(key) {
|
||||||
|
if key[i] < 33 || key[i] > 126 || key[i] == ':' {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
func normalizedStatus(value string) (string, bool) {
|
||||||
|
parts := strings.FieldsFunc(value, func(r rune) bool { return r == ' ' || r == '\t' })
|
||||||
|
if len(parts) != 3 {
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
switch parts[0] {
|
||||||
|
case "unknown", "install", "hold", "deinstall", "purge":
|
||||||
|
default:
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
if parts[1] != "ok" && parts[1] != "reinstreq" {
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
switch parts[2] {
|
||||||
|
case "not-installed", "config-files", "half-installed", "unpacked", "half-configured", "triggers-awaited", "triggers-pending", "installed":
|
||||||
|
default:
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
return strings.Join(parts, " "), true
|
||||||
|
}
|
||||||
@@ -0,0 +1,386 @@
|
|||||||
|
package debian
|
||||||
|
|
||||||
|
import (
|
||||||
|
"crypto/sha256"
|
||||||
|
"encoding/hex"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"io"
|
||||||
|
"io/fs"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"reflect"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"testing/fstest"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
const fixtureStatus = "var/lib/dpkg/status"
|
||||||
|
const fixtureUpdates = "var/lib/dpkg/updates"
|
||||||
|
|
||||||
|
func stanza(name, arch, status string) string {
|
||||||
|
return "Package: " + name + "\nStatus: " + status + "\nArchitecture: " + arch + "\nVersion: 5:28.0.1-1~ubuntu.24.04~noble\nDescription: container runtime\n continuation with Package: ignored\n .\n\tUTF-8 description: 容器\n"
|
||||||
|
}
|
||||||
|
|
||||||
|
func statusFS(raw string) fstest.MapFS {
|
||||||
|
return fstest.MapFS{
|
||||||
|
fixtureStatus: &fstest.MapFile{Data: []byte(raw), Mode: 0644},
|
||||||
|
fixtureUpdates: &fstest.MapFile{Mode: fs.ModeDir | 0755},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func requireUnknown(t *testing.T, files fs.FS) {
|
||||||
|
t.Helper()
|
||||||
|
got := Inventory(files)
|
||||||
|
if got.State != "unknown" || got.Digest != "" || got.Packages == nil || len(got.Packages) != 0 {
|
||||||
|
t.Fatal("invalid input must produce unknown, empty digest, and non-nil empty packages")
|
||||||
|
}
|
||||||
|
raw, err := json.Marshal(got)
|
||||||
|
if err != nil || string(raw) != `{"state":"unknown","digest":"","packages":[]}` {
|
||||||
|
t.Fatal("unknown result must expose only the empty public JSON shape")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInventoryObservedAndExactDigest(t *testing.T) {
|
||||||
|
base := stanza("base-files", "amd64", "install ok installed")
|
||||||
|
for _, raw := range []string{base, "\n" + base + "\n\n", strings.ReplaceAll(base, "\n", "\r\n"), strings.TrimSuffix(base, "\n")} {
|
||||||
|
got := Inventory(statusFS(raw))
|
||||||
|
sum := sha256.Sum256([]byte(raw))
|
||||||
|
if got.State != "observed" || got.Packages == nil || len(got.Packages) != 0 || got.Digest != "sha256:"+hex.EncodeToString(sum[:]) {
|
||||||
|
t.Fatal("valid unrelated package must yield observed empty inventory and exact-byte digest")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInventoryAllRelevantNames(t *testing.T) {
|
||||||
|
names := []string{"containerd", "containerd.io", "docker-buildx", "docker-buildx-plugin", "docker-ce", "docker-ce-cli", "docker-compose", "docker-compose-plugin", "docker-compose-v2", "docker-doc", "docker.io", "podman-docker", "runc"}
|
||||||
|
var raw strings.Builder
|
||||||
|
for i := len(names) - 1; i >= 0; i-- {
|
||||||
|
raw.WriteString(stanza(names[i], "amd64", "install ok installed") + "\n")
|
||||||
|
}
|
||||||
|
raw.WriteString(stanza("docker-ce-extra", "amd64", "install ok installed"))
|
||||||
|
got := Inventory(statusFS(raw.String()))
|
||||||
|
if got.State != "observed" || len(got.Packages) != len(names) {
|
||||||
|
t.Fatal("inventory must include exactly the fixed relevant package set")
|
||||||
|
}
|
||||||
|
for i, name := range names {
|
||||||
|
want := Installed{Name: name, Version: "5:28.0.1-1~ubuntu.24.04~noble", Architecture: "amd64", Status: "install ok installed"}
|
||||||
|
if got.Packages[i] != want {
|
||||||
|
t.Fatal("relevant packages must preserve projected fields and sort by name")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
encoded, err := json.Marshal(got.Packages[0])
|
||||||
|
if err != nil || string(encoded) != `{"name":"containerd","version":"5:28.0.1-1~ubuntu.24.04~noble","architecture":"amd64","status":"install ok installed"}` {
|
||||||
|
t.Fatal("installed JSON must contain only the four specified fields")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInventoryKeepsEveryStatusAndMultiarch(t *testing.T) {
|
||||||
|
statuses := []string{"install ok installed", "hold ok installed", "deinstall ok config-files", "install reinstreq half-installed", "install ok unpacked", "install ok half-configured", "install ok triggers-awaited", "install ok triggers-pending", "purge ok not-installed", "unknown ok not-installed"}
|
||||||
|
for _, status := range statuses {
|
||||||
|
t.Run(status, func(t *testing.T) {
|
||||||
|
raw := stanza("runc", "arm64", status) + "\n" + stanza("runc", "amd64", status)
|
||||||
|
got := Inventory(statusFS(raw))
|
||||||
|
if got.State != "observed" || len(got.Packages) != 2 || got.Packages[0].Architecture != "amd64" || got.Packages[1].Architecture != "arm64" || got.Packages[0].Status != status || got.Packages[1].Status != status {
|
||||||
|
t.Fatal("every present record must survive regardless of installation state; sort multiarch by architecture")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
got := Inventory(statusFS("Package: docker-ce\nStatus: purge ok not-installed\n"))
|
||||||
|
if got.State != "observed" || !reflect.DeepEqual(got.Packages, []Installed{{Name: "docker-ce", Status: "purge ok not-installed"}}) {
|
||||||
|
t.Fatal("not-installed selection records may lack architecture and version but must still block")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInventoryCaseInsensitiveFieldsAndWhitespace(t *testing.T) {
|
||||||
|
raw := "pAcKaGe:\tdocker.io \nSTATUS: hold\t ok installed\narchitecture: all\nversion: 1.2+dfsg-3\nDESCRIPTION: first\n second\n .\n third\n\t \n" + stanza("base-files", "amd64", "install ok installed")
|
||||||
|
got := Inventory(statusFS(raw))
|
||||||
|
if got.State != "observed" || !reflect.DeepEqual(got.Packages, []Installed{{Name: "docker.io", Version: "1.2+dfsg-3", Architecture: "all", Status: "hold ok installed"}}) {
|
||||||
|
t.Fatal("field aliases must normalize safely, including status whitespace and whitespace-only stanza separators")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInventoryRejectsMalformedGlobally(t *testing.T) {
|
||||||
|
valid := stanza("base-files", "amd64", "install ok installed")
|
||||||
|
cases := map[string]string{
|
||||||
|
"empty": "", "blank": "\n \t\n", "orphan continuation": " unexpected\n" + valid,
|
||||||
|
"missing colon": valid + "broken line\n", "empty field name": valid + ": value\n",
|
||||||
|
"field whitespace": valid + "Bad Field: x\n", "field leading hyphen": valid + "-Bad: x\n",
|
||||||
|
"comment": valid + "# comment\n", "duplicate unrelated key": valid + "description: duplicate\n",
|
||||||
|
"duplicate package alias": valid + "PACKAGE: docker-ce\n", "duplicate status alias": valid + "status: purge ok not-installed\n",
|
||||||
|
"duplicate version alias": valid + "VERSION: 1\n", "duplicate architecture alias": valid + "ARCHITECTURE: arm64\n",
|
||||||
|
"duplicate irrelevant stanza": valid + "\n" + valid,
|
||||||
|
"duplicate relevant stanza": stanza("runc", "amd64", "install ok installed") + "\n" + stanza("runc", "amd64", "hold ok installed"),
|
||||||
|
"missing package": "Status: install ok installed\nArchitecture: amd64\nVersion: 1\n",
|
||||||
|
"missing status": "Package: base-files\nArchitecture: amd64\nVersion: 1\n",
|
||||||
|
"missing installed version": "Package: base-files\nStatus: install ok installed\nArchitecture: amd64\n",
|
||||||
|
"missing installed architecture": "Package: base-files\nStatus: install ok installed\nVersion: 1\n",
|
||||||
|
"unknown selection": stanza("base-files", "amd64", "selected ok installed"),
|
||||||
|
"unknown flag": stanza("base-files", "amd64", "install bad installed"),
|
||||||
|
"unknown state": stanza("base-files", "amd64", "install ok ready"),
|
||||||
|
"status suffix": stanza("base-files", "amd64", "install ok installed extra"),
|
||||||
|
"status short": stanza("base-files", "amd64", "ok installed"),
|
||||||
|
"status case": stanza("base-files", "amd64", "Install ok installed"),
|
||||||
|
"unicode status space": stanza("base-files", "amd64", "install\u00a0ok installed"),
|
||||||
|
"architecture list": stanza("base-files", "amd64 arm64", "install ok installed"),
|
||||||
|
"architecture wildcard": stanza("base-files", "any", "install ok installed"),
|
||||||
|
"architecture source": stanza("base-files", "source", "install ok installed"),
|
||||||
|
"architecture punctuation": stanza("base-files", "amd64!", "install ok installed"),
|
||||||
|
"architecture ambiguous": "Package: runc\nStatus: purge ok not-installed\n\n" + stanza("runc", "amd64", "install ok installed"),
|
||||||
|
"architecture all mixed": stanza("runc", "all", "install ok installed") + "\n" + stanza("runc", "amd64", "install ok installed"),
|
||||||
|
"scalar continuation": "Package: base-files\n unexpected\nStatus: install ok installed\nArchitecture: amd64\nVersion: 1\n",
|
||||||
|
"version continuation": "Package: base-files\nStatus: install ok installed\nArchitecture: amd64\nVersion: 1\n 2\n",
|
||||||
|
"nul": valid + "X-Note: hidden\x00value\n", "bare CR": valid + "X-Note: hidden\rvalue\n",
|
||||||
|
"invalid utf8": valid + "X-Note: \xff\n",
|
||||||
|
"unsafe version": strings.Replace(valid, "5:28.0.1-1~ubuntu.24.04~noble", "1;secret", 1),
|
||||||
|
"empty version": strings.Replace(valid, "5:28.0.1-1~ubuntu.24.04~noble", "", 1),
|
||||||
|
}
|
||||||
|
for _, name := range []string{"a", "Docker-ce", "docker_ce", "docker-ce:amd64", "-docker", "docker ce", "dockér"} {
|
||||||
|
cases["invalid package "+name] = stanza(name, "amd64", "install ok installed")
|
||||||
|
}
|
||||||
|
for name, raw := range cases {
|
||||||
|
t.Run(name, func(t *testing.T) {
|
||||||
|
requireUnknown(t, statusFS(raw))
|
||||||
|
// A valid relevant record before corrupt data must never leak a partial result.
|
||||||
|
if strings.TrimSpace(raw) != "" {
|
||||||
|
requireUnknown(t, statusFS(stanza("docker-ce", "amd64", "install ok installed")+"\n"+raw))
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInventoryFileAndJournalFailures(t *testing.T) {
|
||||||
|
for _, name := range []string{"missing status", "missing updates", "updates regular", "updates entry", "updates subdir", "updates hidden entry", "status directory", "status fifo", "status device", "status oversized"} {
|
||||||
|
t.Run(name, func(t *testing.T) {
|
||||||
|
files := statusFS(stanza("base-files", "amd64", "install ok installed"))
|
||||||
|
switch name {
|
||||||
|
case "missing status":
|
||||||
|
delete(files, fixtureStatus)
|
||||||
|
case "missing updates":
|
||||||
|
delete(files, fixtureUpdates)
|
||||||
|
case "updates regular":
|
||||||
|
files[fixtureUpdates].Mode = 0644
|
||||||
|
case "updates entry":
|
||||||
|
files[fixtureUpdates+"/0000"] = &fstest.MapFile{}
|
||||||
|
case "updates subdir":
|
||||||
|
files[fixtureUpdates+"/pending"] = &fstest.MapFile{Mode: fs.ModeDir | 0700}
|
||||||
|
case "updates hidden entry":
|
||||||
|
files[fixtureUpdates+"/.pending"] = &fstest.MapFile{}
|
||||||
|
case "status directory":
|
||||||
|
files[fixtureStatus].Mode = fs.ModeDir | 0755
|
||||||
|
case "status fifo":
|
||||||
|
files[fixtureStatus].Mode = fs.ModeNamedPipe | 0600
|
||||||
|
case "status device":
|
||||||
|
files[fixtureStatus].Mode = fs.ModeDevice | 0600
|
||||||
|
case "status oversized":
|
||||||
|
files[fixtureStatus].Data = []byte(strings.Repeat("x", (16<<20)+1))
|
||||||
|
}
|
||||||
|
requireUnknown(t, files)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
requireUnknown(t, nil)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInventoryBoundedLargeDescription(t *testing.T) {
|
||||||
|
// Exceed Scanner's default 64 KiB token size without exceeding the file cap.
|
||||||
|
raw := stanza("docker.io", "amd64", "install ok installed") + " " + strings.Repeat("x", 128<<10) + "\n"
|
||||||
|
if got := Inventory(statusFS(raw)); got.State != "observed" || len(got.Packages) != 1 {
|
||||||
|
t.Fatal("bounded long description continuation must not hide a relevant record")
|
||||||
|
}
|
||||||
|
// A valid file exactly at the cap is accepted; the next byte is rejected.
|
||||||
|
base := stanza("base-files", "amd64", "install ok installed")
|
||||||
|
raw = base + " " + strings.Repeat("x", (16<<20)-len(base)-2) + "\n"
|
||||||
|
if got := Inventory(statusFS(raw)); got.State != "observed" {
|
||||||
|
t.Fatal("exactly 16 MiB must be accepted")
|
||||||
|
}
|
||||||
|
requireUnknown(t, statusFS(raw+"\n"))
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInventoryRealDirectory(t *testing.T) {
|
||||||
|
root := t.TempDir()
|
||||||
|
if err := os.MkdirAll(filepath.Join(root, filepath.FromSlash(fixtureUpdates)), 0755); err != nil {
|
||||||
|
t.Fatal("create fixture directory")
|
||||||
|
}
|
||||||
|
raw := stanza("docker-compose-v2", "arm64", "deinstall ok config-files")
|
||||||
|
path := filepath.Join(root, filepath.FromSlash(fixtureStatus))
|
||||||
|
if err := os.WriteFile(path, []byte(raw), 0644); err != nil {
|
||||||
|
t.Fatal("write fixture status")
|
||||||
|
}
|
||||||
|
got := Inventory(os.DirFS(root))
|
||||||
|
if got.State != "observed" || len(got.Packages) != 1 || got.Packages[0].Status != "deinstall ok config-files" {
|
||||||
|
t.Fatal("real filesystem residual record missing")
|
||||||
|
}
|
||||||
|
after, err := os.ReadFile(path)
|
||||||
|
if err != nil || string(after) != raw {
|
||||||
|
t.Fatal("inventory must not modify the database")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Faults are confined to the FS boundary: permissions and read-time changes
|
||||||
|
// cannot be exercised portably with chmod (notably on Windows or as root).
|
||||||
|
type faultFS struct {
|
||||||
|
fstest.MapFS
|
||||||
|
fault string
|
||||||
|
journalOpens int
|
||||||
|
statusOpens int
|
||||||
|
statusStats int
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *faultFS) Stat(path string) (fs.FileInfo, error) {
|
||||||
|
if (f.fault == "status stat denied" && path == fixtureStatus) || (f.fault == "journal stat denied" && path == fixtureUpdates) {
|
||||||
|
return nil, errors.New("private stat diagnostic")
|
||||||
|
}
|
||||||
|
info, err := f.MapFS.Stat(path)
|
||||||
|
if path == fixtureStatus {
|
||||||
|
f.statusStats++
|
||||||
|
if f.fault == "path changed" && f.statusStats > 1 && err == nil {
|
||||||
|
return changedInfo{FileInfo: info, change: "time"}, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return info, err
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *faultFS) Open(path string) (fs.File, error) {
|
||||||
|
if path != fixtureStatus && path != fixtureUpdates {
|
||||||
|
return nil, errors.New("unexpected path")
|
||||||
|
}
|
||||||
|
if path == fixtureStatus {
|
||||||
|
f.statusOpens++
|
||||||
|
if f.fault == "status open denied" {
|
||||||
|
return nil, errors.New("private open diagnostic")
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
f.journalOpens++
|
||||||
|
if f.fault == "journal open denied" {
|
||||||
|
return nil, errors.New("private open diagnostic")
|
||||||
|
}
|
||||||
|
if f.fault == "journal becomes pending" && f.journalOpens == 2 {
|
||||||
|
f.MapFS[fixtureUpdates+"/0001"] = &fstest.MapFile{}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
file, err := f.MapFS.Open(path)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return &faultFile{File: file, owner: f, path: path}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
type faultFile struct {
|
||||||
|
fs.File
|
||||||
|
owner *faultFS
|
||||||
|
path string
|
||||||
|
stats int
|
||||||
|
bytesRead int
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *faultFile) Stat() (fs.FileInfo, error) {
|
||||||
|
f.stats++
|
||||||
|
info, err := f.File.Stat()
|
||||||
|
if f.path == fixtureStatus && err == nil {
|
||||||
|
switch f.owner.fault {
|
||||||
|
case "opened stat denied":
|
||||||
|
return nil, fs.ErrPermission
|
||||||
|
case "opened fifo":
|
||||||
|
return changedInfo{FileInfo: info, change: "fifo"}, nil
|
||||||
|
case "opened size changed":
|
||||||
|
return changedInfo{FileInfo: info, change: "size"}, nil
|
||||||
|
case "changed while reading":
|
||||||
|
if f.stats > 1 {
|
||||||
|
return changedInfo{FileInfo: info, change: "time"}, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return info, err
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *faultFile) Read(p []byte) (int, error) {
|
||||||
|
if f.path == fixtureStatus {
|
||||||
|
switch f.owner.fault {
|
||||||
|
case "read denied":
|
||||||
|
return 0, errors.New("private read diagnostic")
|
||||||
|
case "truncated":
|
||||||
|
return 0, io.EOF
|
||||||
|
case "growing":
|
||||||
|
// Ignore the advertised size, like a file growing after stat.
|
||||||
|
if f.bytesRead+len(p) > (16<<20)+1 {
|
||||||
|
return 0, errors.New("read exceeded bound")
|
||||||
|
}
|
||||||
|
for i := range p {
|
||||||
|
p[i] = 'x'
|
||||||
|
}
|
||||||
|
f.bytesRead += len(p)
|
||||||
|
return len(p), nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return f.File.Read(p)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *faultFile) ReadDir(n int) ([]fs.DirEntry, error) {
|
||||||
|
if f.owner.fault == "journal read denied" {
|
||||||
|
return nil, errors.New("private journal diagnostic")
|
||||||
|
}
|
||||||
|
return f.File.(fs.ReadDirFile).ReadDir(n)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *faultFile) Close() error {
|
||||||
|
err := f.File.Close()
|
||||||
|
if f.owner.fault == "close failure" {
|
||||||
|
return errors.New("private close diagnostic")
|
||||||
|
}
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
type changedInfo struct {
|
||||||
|
fs.FileInfo
|
||||||
|
change string
|
||||||
|
}
|
||||||
|
|
||||||
|
func (i changedInfo) Mode() fs.FileMode {
|
||||||
|
if i.change == "fifo" {
|
||||||
|
return fs.ModeNamedPipe | 0600
|
||||||
|
}
|
||||||
|
return i.FileInfo.Mode()
|
||||||
|
}
|
||||||
|
|
||||||
|
func (i changedInfo) Size() int64 {
|
||||||
|
if i.change == "size" {
|
||||||
|
return i.FileInfo.Size() + 1
|
||||||
|
}
|
||||||
|
return i.FileInfo.Size()
|
||||||
|
}
|
||||||
|
|
||||||
|
func (i changedInfo) ModTime() time.Time {
|
||||||
|
if i.change == "time" {
|
||||||
|
return i.FileInfo.ModTime().Add(time.Second)
|
||||||
|
}
|
||||||
|
return i.FileInfo.ModTime()
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInventoryFailsClosedOnReadFaultsAndChanges(t *testing.T) {
|
||||||
|
for _, fault := range []string{"status stat denied", "journal stat denied", "status open denied", "journal open denied", "opened stat denied", "opened fifo", "opened size changed", "changed while reading", "path changed", "read denied", "truncated", "growing", "journal read denied", "journal becomes pending", "close failure"} {
|
||||||
|
t.Run(fault, func(t *testing.T) {
|
||||||
|
files := &faultFS{MapFS: statusFS(stanza("runc", "amd64", "install ok installed")), fault: fault}
|
||||||
|
requireUnknown(t, files)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInventoryStatsBeforeOpeningNonregularFiles(t *testing.T) {
|
||||||
|
for _, path := range []string{fixtureStatus, fixtureUpdates} {
|
||||||
|
files := &faultFS{MapFS: statusFS(stanza("runc", "amd64", "install ok installed"))}
|
||||||
|
files.MapFS[path].Mode = fs.ModeNamedPipe | 0600
|
||||||
|
requireUnknown(t, files)
|
||||||
|
if files.statusOpens != 0 || (path == fixtureUpdates && files.journalOpens != 0) {
|
||||||
|
t.Fatal("must reject a FIFO using metadata before opening it")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// fs.Stat would use Open on this implementation, so fail closed instead.
|
||||||
|
requireUnknown(t, openOnlyFS{})
|
||||||
|
}
|
||||||
|
|
||||||
|
type openOnlyFS struct{}
|
||||||
|
|
||||||
|
func (openOnlyFS) Open(string) (fs.File, error) {
|
||||||
|
panic("must not open without safe metadata support")
|
||||||
|
}
|
||||||
@@ -0,0 +1,70 @@
|
|||||||
|
// Package inspect reports local prerequisite observations without executing
|
||||||
|
// commands, connecting to Docker or altering configuration.
|
||||||
|
package inspect
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"io/fs"
|
||||||
|
"os"
|
||||||
|
"runtime"
|
||||||
|
)
|
||||||
|
|
||||||
|
type Report struct {
|
||||||
|
ProtocolVersion int `json:"protocolVersion"`
|
||||||
|
OS string `json:"os"`
|
||||||
|
Architecture string `json:"architecture"`
|
||||||
|
SupportedPlatform bool `json:"supportedPlatform"`
|
||||||
|
SystemdRuntime string `json:"systemdRuntime"`
|
||||||
|
DockerClient string `json:"dockerClient"`
|
||||||
|
DockerDaemon string `json:"dockerDaemon"`
|
||||||
|
Compose string `json:"compose"`
|
||||||
|
Unchecked []string `json:"unchecked"`
|
||||||
|
DeploymentReady bool `json:"deploymentReady"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func Collect() Report { return Probe(runtime.GOOS, runtime.GOARCH, os.DirFS("/")) }
|
||||||
|
|
||||||
|
// Probe observes filesystem metadata only. "present" is not an assertion that
|
||||||
|
// systemd is responsive or the Docker executable is authentic or operational.
|
||||||
|
func Probe(goos, arch string, files fs.FS) Report {
|
||||||
|
r := Report{ProtocolVersion: 1, OS: goos, Architecture: arch, SupportedPlatform: goos == "linux" && (arch == "amd64" || arch == "arm64"), SystemdRuntime: "not_checked", DockerClient: "not_checked", DockerDaemon: "not_checked", Compose: "not_checked", Unchecked: []string{"docker_daemon", "compose_version", "host_identity", "distribution_support", "permissions", "disk_space", "ports", "dns", "firewall", "gateway"}}
|
||||||
|
if goos != "linux" {
|
||||||
|
return r
|
||||||
|
}
|
||||||
|
r.SystemdRuntime = observe(files, "run/systemd/system", true)
|
||||||
|
r.DockerClient = observe(files, "usr/bin/docker", false)
|
||||||
|
if r.DockerClient != "present" {
|
||||||
|
alternate := observe(files, "usr/local/bin/docker", false)
|
||||||
|
// Either known executable establishes presence. Otherwise retain any
|
||||||
|
// uncertainty instead of reporting absence from an incomplete check.
|
||||||
|
switch {
|
||||||
|
case alternate == "present":
|
||||||
|
r.DockerClient = "present"
|
||||||
|
case r.DockerClient == "unknown" || alternate == "unknown":
|
||||||
|
r.DockerClient = "unknown"
|
||||||
|
case r.DockerClient == "invalid" || alternate == "invalid":
|
||||||
|
r.DockerClient = "invalid"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return r
|
||||||
|
}
|
||||||
|
|
||||||
|
func observe(files fs.FS, path string, directory bool) string {
|
||||||
|
info, err := fs.Stat(files, path)
|
||||||
|
if errors.Is(err, fs.ErrNotExist) {
|
||||||
|
return "missing"
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return "unknown"
|
||||||
|
}
|
||||||
|
if directory {
|
||||||
|
if info.IsDir() {
|
||||||
|
return "present"
|
||||||
|
}
|
||||||
|
return "invalid"
|
||||||
|
}
|
||||||
|
if !info.Mode().IsRegular() || info.Mode().Perm()&0111 == 0 {
|
||||||
|
return "invalid"
|
||||||
|
}
|
||||||
|
return "present"
|
||||||
|
}
|
||||||
@@ -0,0 +1,110 @@
|
|||||||
|
package inspect
|
||||||
|
|
||||||
|
import (
|
||||||
|
"io/fs"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"testing/fstest"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestReportsFactsWithoutClaimingDeploymentReadiness(t *testing.T) {
|
||||||
|
files := fstest.MapFS{
|
||||||
|
"run/systemd/system": &fstest.MapFile{Mode: os.ModeDir | 0755},
|
||||||
|
"usr/bin/docker": &fstest.MapFile{Mode: 0755, Data: []byte("not executed")},
|
||||||
|
}
|
||||||
|
r := Probe("linux", "amd64", files)
|
||||||
|
if !r.SupportedPlatform || r.SystemdRuntime != "present" || r.DockerClient != "present" || r.DeploymentReady {
|
||||||
|
t.Fatalf("incorrect report: %+v", r)
|
||||||
|
}
|
||||||
|
if r.DockerDaemon != "not_checked" || r.Compose != "not_checked" {
|
||||||
|
t.Fatal("claimed unperformed checks")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestMissingAndUnsupportedEnvironment(t *testing.T) {
|
||||||
|
r := Probe("linux", "amd64", fstest.MapFS{})
|
||||||
|
if r.DockerClient != "missing" || r.SystemdRuntime != "missing" {
|
||||||
|
t.Fatalf("missing prerequisites masked: %+v", r)
|
||||||
|
}
|
||||||
|
r = Probe("windows", "amd64", fstest.MapFS{})
|
||||||
|
if r.SupportedPlatform || r.DeploymentReady || r.DockerClient != "not_checked" {
|
||||||
|
t.Fatal("Windows accepted as deployment target")
|
||||||
|
}
|
||||||
|
r = Probe("linux", "386", fstest.MapFS{})
|
||||||
|
if r.SupportedPlatform {
|
||||||
|
t.Fatal("unsupported architecture accepted")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestWrongFileKindsAndPermissions(t *testing.T) {
|
||||||
|
r := Probe("linux", "amd64", fstest.MapFS{
|
||||||
|
"run/systemd/system": &fstest.MapFile{Mode: 0644},
|
||||||
|
"usr/bin/docker": &fstest.MapFile{Mode: 0644},
|
||||||
|
})
|
||||||
|
if r.SystemdRuntime != "invalid" || r.DockerClient != "invalid" {
|
||||||
|
t.Fatalf("wrong file kinds accepted: %+v", r)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRealDirectoryProbeDoesNotWrite(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
if err := os.MkdirAll(filepath.Join(dir, "run/systemd/system"), 0700); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
r := Probe("linux", "arm64", os.DirFS(dir))
|
||||||
|
if r.SystemdRuntime != "present" || r.DockerClient != "missing" {
|
||||||
|
t.Fatalf("bad real probe: %+v", r)
|
||||||
|
}
|
||||||
|
items, err := os.ReadDir(dir)
|
||||||
|
if err != nil || len(items) != 1 || items[0].Name() != "run" {
|
||||||
|
t.Fatal("probe changed filesystem")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
type deniedFS struct{}
|
||||||
|
|
||||||
|
func (deniedFS) Open(string) (fs.File, error) { return nil, fs.ErrPermission }
|
||||||
|
|
||||||
|
func TestPermissionErrorsAreUnknownNotMissing(t *testing.T) {
|
||||||
|
r := Probe("linux", "amd64", deniedFS{})
|
||||||
|
if r.DockerClient != "unknown" || r.SystemdRuntime != "unknown" {
|
||||||
|
t.Fatalf("hid inspection failure: %+v", r)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDockerAlternatePath(t *testing.T) {
|
||||||
|
r := Probe("linux", "amd64", fstest.MapFS{"usr/local/bin/docker": &fstest.MapFile{Mode: 0755}})
|
||||||
|
if r.DockerClient != "present" {
|
||||||
|
t.Fatal("alternate installation not found")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
type pathDeniedFS struct{ fs.FS }
|
||||||
|
|
||||||
|
func (f pathDeniedFS) Open(name string) (fs.File, error) {
|
||||||
|
if name == "usr/bin/docker" {
|
||||||
|
return nil, fs.ErrPermission
|
||||||
|
}
|
||||||
|
return f.FS.Open(name)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAlternateDockerAfterInvalidOrUnknownPrimary(t *testing.T) {
|
||||||
|
for _, mode := range []fs.FileMode{0644, fs.ModeDir | 0755} {
|
||||||
|
files := fstest.MapFS{
|
||||||
|
"usr/bin/docker": &fstest.MapFile{Mode: mode},
|
||||||
|
"usr/local/bin/docker": &fstest.MapFile{Mode: 0755},
|
||||||
|
}
|
||||||
|
if r := Probe("linux", "amd64", files); r.DockerClient != "present" {
|
||||||
|
t.Errorf("valid alternate overlooked after invalid primary: %+v", r)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
files := pathDeniedFS{fstest.MapFS{"usr/local/bin/docker": &fstest.MapFile{Mode: 0755}}}
|
||||||
|
if r := Probe("linux", "amd64", files); r.DockerClient != "present" {
|
||||||
|
t.Errorf("valid alternate overlooked after inaccessible primary: %+v", r)
|
||||||
|
}
|
||||||
|
if r := Probe("linux", "amd64", pathDeniedFS{fstest.MapFS{}}); r.DockerClient != "unknown" {
|
||||||
|
t.Errorf("missing alternate masked inaccessible primary: %+v", r)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,50 @@
|
|||||||
|
# Environment lock and draft
|
||||||
|
|
||||||
|
`plan-environment` accepts strict JSON `{ "lock": <Lock> }`, collects local
|
||||||
|
preflight observations itself, and emits a non-executable draft. It does not
|
||||||
|
accept caller-supplied host observations, download anything, run apt, configure
|
||||||
|
sources or start services.
|
||||||
|
|
||||||
|
Lock fields, all required:
|
||||||
|
|
||||||
|
- protocolVersion: 1.
|
||||||
|
- repository: exactly `https://download.docker.com/linux/ubuntu`.
|
||||||
|
- suite: jammy, noble or resolute; architecture: amd64 or arm64.
|
||||||
|
- releaseDigest: `sha256:` plus 64 lowercase hex digits, supplied by the caller.
|
||||||
|
- packages: exactly docker-ce, docker-ce-cli, containerd.io,
|
||||||
|
docker-buildx-plugin and docker-compose-plugin, once each.
|
||||||
|
- Each package has name, version, filename, digest and size. Version is explicit
|
||||||
|
digit-leading Debian-style syntax (optional numeric epoch), at most 128 bytes;
|
||||||
|
digest uses the above SHA-256 format; size is 1 through 512 MiB.
|
||||||
|
- Filename must equal
|
||||||
|
`dists/<suite>/pool/stable/<architecture>/<name>_<version-without-epoch>_<architecture>.deb`.
|
||||||
|
Encoded paths, absolute paths, alternate domains, query strings and traversal
|
||||||
|
are not accepted. docker-ce and docker-ce-cli must use the same version.
|
||||||
|
|
||||||
|
This is a deliberately limited Docker lock format, not a complete Debian version
|
||||||
|
parser. A Linux host's observed distribution/suite and architecture must match;
|
||||||
|
unknown observations remain blockers. On a non-Linux machine a syntactically
|
||||||
|
valid lock can be reviewed, but unsupported-platform blockers remain.
|
||||||
|
|
||||||
|
## Digests and remaining trust boundary
|
||||||
|
|
||||||
|
lockDigest binds Go JSON encoding of the validated Lock in struct/array order.
|
||||||
|
observationDigest binds Go JSON encoding of the collected Report. These are
|
||||||
|
content hashes, not signatures, stable host IDs, freshness tokens or authorization.
|
||||||
|
The local observation is not atomic and changes (including free disk space) can
|
||||||
|
change its hash. No writing consumer may treat it as an approved plan.
|
||||||
|
|
||||||
|
The draft returns requestedPackages, not a complete APT dependency transaction.
|
||||||
|
It never proposes automatic removal or upgrade of existing installations.
|
||||||
|
RepositoryAuthenticated and executable are always false. There is no signature
|
||||||
|
verification, Release-to-Packages-to-deb digest chain verification, metadata
|
||||||
|
freshness policy, artifact download, package dependency resolution, or installation
|
||||||
|
executor yet. Matching a URL allowlist and a caller-provided hash proves none of
|
||||||
|
those. No actual versions are recommended or locked from live metadata in this batch.
|
||||||
|
|
||||||
|
Blockers explicitly retain these gaps along with host/network/runtime checks.
|
||||||
|
Potential APT database changes, dependency changes, service starts and network
|
||||||
|
rule effects are reported. Exit 0 only means a draft was produced.
|
||||||
|
|
||||||
|
Reference for package names and repository layout:
|
||||||
|
[Docker Ubuntu installation](https://docs.docker.com/engine/install/ubuntu/).
|
||||||
@@ -0,0 +1,68 @@
|
|||||||
|
package installplan
|
||||||
|
|
||||||
|
import (
|
||||||
|
"crypto/sha256"
|
||||||
|
"encoding/hex"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"regexp"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
type Lock struct {
|
||||||
|
ProtocolVersion int `json:"protocolVersion"`
|
||||||
|
Repository string `json:"repository"`
|
||||||
|
Suite string `json:"suite"`
|
||||||
|
Architecture string `json:"architecture"`
|
||||||
|
ReleaseDigest string `json:"releaseDigest"`
|
||||||
|
Packages []Package `json:"packages"`
|
||||||
|
}
|
||||||
|
type Package struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
Version string `json:"version"`
|
||||||
|
Filename string `json:"filename"`
|
||||||
|
Digest string `json:"digest"`
|
||||||
|
Size uint64 `json:"size"`
|
||||||
|
}
|
||||||
|
|
||||||
|
var versionPattern = regexp.MustCompile(`^(?:[0-9]+:)?[0-9][0-9A-Za-z.+~-]*$`)
|
||||||
|
var digestPattern = regexp.MustCompile(`^sha256:[0-9a-f]{64}$`)
|
||||||
|
var required = []string{"docker-ce", "docker-ce-cli", "containerd.io", "docker-buildx-plugin", "docker-compose-plugin"}
|
||||||
|
|
||||||
|
// Validate checks a caller-supplied lock against a fixed source policy. This is
|
||||||
|
// NOT repository signature validation or evidence these artifacts exist.
|
||||||
|
func Validate(lock Lock, suite, architecture string) (string, error) {
|
||||||
|
reject := errors.New("invalid Docker package lock")
|
||||||
|
if lock.ProtocolVersion != 1 || lock.Repository != "https://download.docker.com/linux/ubuntu" || lock.Suite != suite || lock.Architecture != architecture {
|
||||||
|
return "", reject
|
||||||
|
}
|
||||||
|
if (suite != "jammy" && suite != "noble" && suite != "resolute") || (architecture != "amd64" && architecture != "arm64") || !digestPattern.MatchString(lock.ReleaseDigest) || len(lock.Packages) != len(required) {
|
||||||
|
return "", reject
|
||||||
|
}
|
||||||
|
versions := map[string]string{}
|
||||||
|
for _, p := range lock.Packages {
|
||||||
|
allowed := false
|
||||||
|
for _, name := range required {
|
||||||
|
if p.Name == name {
|
||||||
|
allowed = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !allowed || versions[p.Name] != "" || len(p.Version) > 128 || !versionPattern.MatchString(p.Version) || !digestPattern.MatchString(p.Digest) || p.Size == 0 || p.Size > 512<<20 {
|
||||||
|
return "", reject
|
||||||
|
}
|
||||||
|
fileVersion := p.Version
|
||||||
|
if _, after, ok := strings.Cut(fileVersion, ":"); ok {
|
||||||
|
fileVersion = after
|
||||||
|
}
|
||||||
|
if p.Filename != "dists/"+suite+"/pool/stable/"+architecture+"/"+p.Name+"_"+fileVersion+"_"+architecture+".deb" {
|
||||||
|
return "", reject
|
||||||
|
}
|
||||||
|
versions[p.Name] = p.Version
|
||||||
|
}
|
||||||
|
if versions["docker-ce"] != versions["docker-ce-cli"] {
|
||||||
|
return "", reject
|
||||||
|
}
|
||||||
|
raw, _ := json.Marshal(lock)
|
||||||
|
sum := sha256.Sum256(raw)
|
||||||
|
return "sha256:" + hex.EncodeToString(sum[:]), nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,79 @@
|
|||||||
|
package installplan
|
||||||
|
|
||||||
|
import (
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func lockFixture() Lock {
|
||||||
|
l := Lock{ProtocolVersion: 1, Repository: "https://download.docker.com/linux/ubuntu", Suite: "resolute", Architecture: "amd64", ReleaseDigest: "sha256:" + strings.Repeat("b", 64), Packages: []Package{}}
|
||||||
|
for _, name := range []string{"docker-ce", "docker-ce-cli", "containerd.io", "docker-buildx-plugin", "docker-compose-plugin"} {
|
||||||
|
l.Packages = append(l.Packages, Package{Name: name, Version: "1.2.3-1", Filename: "dists/resolute/pool/stable/amd64/" + name + "_1.2.3-1_amd64.deb", Digest: "sha256:" + strings.Repeat("a", 64), Size: 123})
|
||||||
|
}
|
||||||
|
return l
|
||||||
|
}
|
||||||
|
func TestValidLockAndStableDigest(t *testing.T) {
|
||||||
|
l := lockFixture()
|
||||||
|
digest, err := Validate(l, "resolute", "amd64")
|
||||||
|
if err != nil || !strings.HasPrefix(digest, "sha256:") {
|
||||||
|
t.Fatal("valid lock rejected", err)
|
||||||
|
}
|
||||||
|
l.Packages[0].Size++
|
||||||
|
changed, err := Validate(l, "resolute", "amd64")
|
||||||
|
if err != nil || changed == digest {
|
||||||
|
t.Fatal("lock digest does not bind size")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestRejectUnsafeLock(t *testing.T) {
|
||||||
|
for name, mutate := range map[string]func(*Lock){
|
||||||
|
"protocol": func(l *Lock) { l.ProtocolVersion = 2 },
|
||||||
|
"repository": func(l *Lock) { l.Repository = "https://attacker.example/linux/ubuntu" },
|
||||||
|
"credentials": func(l *Lock) { l.Repository = "https://user:secret@download.docker.com/linux/ubuntu" },
|
||||||
|
"suite": func(l *Lock) { l.Suite = "noble" },
|
||||||
|
"architecture": func(l *Lock) { l.Architecture = "arm64" },
|
||||||
|
"release": func(l *Lock) { l.ReleaseDigest = "" },
|
||||||
|
"missing": func(l *Lock) { l.Packages = l.Packages[:4] },
|
||||||
|
"extra": func(l *Lock) { l.Packages = append(l.Packages, l.Packages[0]) },
|
||||||
|
"duplicate": func(l *Lock) { l.Packages[1] = l.Packages[0] },
|
||||||
|
"latest": func(l *Lock) { l.Packages[0].Version = "latest" },
|
||||||
|
"shell": func(l *Lock) { l.Packages[0].Version = "1;reboot" },
|
||||||
|
"wrong file": func(l *Lock) { l.Packages[0].Filename = "dists/resolute/pool/stable/amd64/other.deb" },
|
||||||
|
"traversal": func(l *Lock) { l.Packages[0].Filename = "../docker.deb" },
|
||||||
|
"encoded path": func(l *Lock) { l.Packages[0].Filename = "dists/resolute/pool/stable/amd64/%2e%2e.deb" },
|
||||||
|
"digest": func(l *Lock) { l.Packages[0].Digest = "bad" },
|
||||||
|
"size": func(l *Lock) { l.Packages[0].Size = 0 },
|
||||||
|
"engine mismatch": func(l *Lock) {
|
||||||
|
l.Packages[1].Version = "2.3.4-1"
|
||||||
|
l.Packages[1].Filename = "dists/resolute/pool/stable/amd64/docker-ce-cli_2.3.4-1_amd64.deb"
|
||||||
|
},
|
||||||
|
} {
|
||||||
|
t.Run(name, func(t *testing.T) {
|
||||||
|
l := lockFixture()
|
||||||
|
mutate(&l)
|
||||||
|
if _, err := Validate(l, "resolute", "amd64"); err == nil {
|
||||||
|
t.Fatal("unsafe lock accepted")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDockerEpochAndTargetArchitecture(t *testing.T) {
|
||||||
|
l := lockFixture()
|
||||||
|
for i := range l.Packages {
|
||||||
|
p := &l.Packages[i]
|
||||||
|
if p.Name == "docker-ce" || p.Name == "docker-ce-cli" {
|
||||||
|
p.Version = "5:29.1.0-1~ubuntu.26.04~resolute"
|
||||||
|
p.Filename = "dists/resolute/pool/stable/amd64/" + p.Name + "_29.1.0-1~ubuntu.26.04~resolute_amd64.deb"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if _, err := Validate(l, "resolute", "amd64"); err != nil {
|
||||||
|
t.Fatal("epoch version rejected", err)
|
||||||
|
}
|
||||||
|
if _, err := Validate(l, "resolute", "arm64"); err == nil {
|
||||||
|
t.Fatal("architecture mismatch accepted")
|
||||||
|
}
|
||||||
|
l.Packages[0].Filename = strings.Replace(l.Packages[0].Filename, "_29.", "_5:29.", 1)
|
||||||
|
if _, err := Validate(l, "resolute", "amd64"); err == nil {
|
||||||
|
t.Fatal("epoch in repository filename accepted")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,42 @@
|
|||||||
|
package installplan
|
||||||
|
|
||||||
|
import (
|
||||||
|
"crypto/sha256"
|
||||||
|
"encoding/hex"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"server-deploy/internal/preflight"
|
||||||
|
)
|
||||||
|
|
||||||
|
type Draft struct {
|
||||||
|
Executable bool `json:"executable"`
|
||||||
|
RepositoryAuthenticated bool `json:"repositoryAuthenticated"`
|
||||||
|
LockDigest string `json:"lockDigest"`
|
||||||
|
ObservationDigest string `json:"observationDigest"`
|
||||||
|
Blockers []string `json:"blockers"`
|
||||||
|
RequestedPackages []Package `json:"requestedPackages"`
|
||||||
|
Impacts []string `json:"impacts"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Build binds requested package pins to a local observation for review only.
|
||||||
|
// Neither digest is a signature, host identity, freshness token or approval.
|
||||||
|
func Build(report preflight.Report, lock Lock) (Draft, error) {
|
||||||
|
digest, err := Validate(lock, lock.Suite, lock.Architecture)
|
||||||
|
if err != nil {
|
||||||
|
return Draft{}, err
|
||||||
|
}
|
||||||
|
if report.Runtime.OS == "linux" && (report.Runtime.Architecture != lock.Architecture || (report.Distribution.State == "observed" && (report.Distribution.ID != "ubuntu" || report.Distribution.Codename != lock.Suite))) {
|
||||||
|
return Draft{}, errors.New("lock does not match local platform")
|
||||||
|
}
|
||||||
|
proposal := preflight.Plan(report)
|
||||||
|
blockers := []string{}
|
||||||
|
for _, b := range proposal.Blockers {
|
||||||
|
if b != "package_versions_unresolved" {
|
||||||
|
blockers = append(blockers, b)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
blockers = append(blockers, "dependency_transaction_unresolved", "artifact_bytes_unverified", "repository_metadata_freshness_unverified")
|
||||||
|
raw, _ := json.Marshal(report)
|
||||||
|
sum := sha256.Sum256(raw)
|
||||||
|
return Draft{LockDigest: digest, ObservationDigest: "sha256:" + hex.EncodeToString(sum[:]), Blockers: blockers, RequestedPackages: append([]Package{}, lock.Packages...), Impacts: []string{"package_database_and_repository_changes", "services_may_start_during_package_install", "host_network_rules_may_change", "additional_dependencies_not_yet_resolved"}}, nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,39 @@
|
|||||||
|
package installplan
|
||||||
|
|
||||||
|
import (
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"server-deploy/internal/inspect"
|
||||||
|
"server-deploy/internal/preflight"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestDraftPreservesSafetyBlockersAndBindsInputs(t *testing.T) {
|
||||||
|
r := preflight.Report{Runtime: inspect.Report{OS: "linux", Architecture: "amd64"}, Distribution: preflight.Distribution{State: "observed", ID: "ubuntu", Version: "26.04", Codename: "resolute"}}
|
||||||
|
d, err := Build(r, lockFixture())
|
||||||
|
if err != nil || d.Executable || d.RepositoryAuthenticated || len(d.Blockers) == 0 || !strings.HasPrefix(d.LockDigest, "sha256:") || !strings.HasPrefix(d.ObservationDigest, "sha256:") {
|
||||||
|
t.Fatalf("unsafe draft %+v %v", d, err)
|
||||||
|
}
|
||||||
|
for _, want := range []string{"repository_trust_unverified", "dependency_transaction_unresolved", "artifact_bytes_unverified"} {
|
||||||
|
found := false
|
||||||
|
for _, b := range d.Blockers {
|
||||||
|
if b == want {
|
||||||
|
found = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !found {
|
||||||
|
t.Fatalf("missing blocker %s", want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
r.Privilege = "non_root"
|
||||||
|
d2, _ := Build(r, lockFixture())
|
||||||
|
if d2.ObservationDigest == d.ObservationDigest {
|
||||||
|
t.Fatal("report change not bound")
|
||||||
|
}
|
||||||
|
l := lockFixture()
|
||||||
|
r.Distribution.Version = "24.04"
|
||||||
|
r.Distribution.Codename = "noble"
|
||||||
|
if _, err := Build(r, l); err == nil {
|
||||||
|
t.Fatal("host and lock mismatch accepted")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,56 @@
|
|||||||
|
// Package planner builds offline previews. It does not inspect or change a host.
|
||||||
|
package planner
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"net"
|
||||||
|
"regexp"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
const ProtocolVersion = 1
|
||||||
|
|
||||||
|
var (
|
||||||
|
idPattern = regexp.MustCompile(`^[a-z][a-z0-9-]{0,47}$`)
|
||||||
|
digestPattern = regexp.MustCompile(`^sha256:[a-f0-9]{64}$`)
|
||||||
|
labelPattern = regexp.MustCompile(`^[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?$`)
|
||||||
|
)
|
||||||
|
|
||||||
|
// Intent contains no secrets. ObservedStateDigest is caller-supplied in offline
|
||||||
|
// mode; a future executor must obtain it independently from the target host.
|
||||||
|
type Intent struct {
|
||||||
|
ProtocolVersion int `json:"protocolVersion"`
|
||||||
|
HostID string `json:"hostId"`
|
||||||
|
InstanceID string `json:"instanceId"`
|
||||||
|
AppID string `json:"appId"`
|
||||||
|
PackageDigest string `json:"packageDigest"`
|
||||||
|
ImageDigest string `json:"imageDigest"`
|
||||||
|
Domain string `json:"domain"`
|
||||||
|
ObservedStateDigest string `json:"observedStateDigest"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func (i Intent) Validate() error {
|
||||||
|
if i.ProtocolVersion != ProtocolVersion {
|
||||||
|
return errors.New("unsupported protocol version")
|
||||||
|
}
|
||||||
|
for _, id := range []string{i.HostID, i.InstanceID, i.AppID} {
|
||||||
|
if !idPattern.MatchString(id) {
|
||||||
|
return errors.New("invalid resource identifier")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, digest := range []string{i.PackageDigest, i.ImageDigest, i.ObservedStateDigest} {
|
||||||
|
if !digestPattern.MatchString(digest) {
|
||||||
|
return errors.New("expected a SHA-256 digest")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
labels := strings.Split(i.Domain, ".")
|
||||||
|
if len(i.Domain) > 253 || len(labels) < 2 || net.ParseIP(i.Domain) != nil {
|
||||||
|
return errors.New("expected an ASCII DNS hostname")
|
||||||
|
}
|
||||||
|
for _, label := range labels {
|
||||||
|
if !labelPattern.MatchString(label) {
|
||||||
|
return errors.New("invalid DNS hostname label")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,44 @@
|
|||||||
|
package planner
|
||||||
|
|
||||||
|
import (
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func validIntent() Intent {
|
||||||
|
return Intent{ProtocolVersion: 1, HostID: "host-one", InstanceID: "git-one", AppID: "gitea", PackageDigest: "sha256:" + strings.Repeat("a", 64), ImageDigest: "sha256:" + strings.Repeat("b", 64), Domain: "git.example.com", ObservedStateDigest: "sha256:" + strings.Repeat("c", 64)}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestIntentValidation(t *testing.T) {
|
||||||
|
if err := validIntent().Validate(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
cases := []struct {
|
||||||
|
name string
|
||||||
|
change func(*Intent)
|
||||||
|
}{
|
||||||
|
{"protocol", func(i *Intent) { i.ProtocolVersion = 2 }},
|
||||||
|
{"host empty", func(i *Intent) { i.HostID = "" }},
|
||||||
|
{"path escape", func(i *Intent) { i.InstanceID = "../git" }},
|
||||||
|
{"shell", func(i *Intent) { i.AppID = "git;id" }},
|
||||||
|
{"long id", func(i *Intent) { i.InstanceID = strings.Repeat("a", 49) }},
|
||||||
|
{"floating tag", func(i *Intent) { i.ImageDigest = "gitea:latest" }},
|
||||||
|
{"package hash", func(i *Intent) { i.PackageDigest = "sha256:xyz" }},
|
||||||
|
{"state missing", func(i *Intent) { i.ObservedStateDigest = "" }},
|
||||||
|
{"url", func(i *Intent) { i.Domain = "https://git.example.com" }},
|
||||||
|
{"wildcard", func(i *Intent) { i.Domain = "*.example.com" }},
|
||||||
|
{"label", func(i *Intent) { i.Domain = "-git.example.com" }},
|
||||||
|
{"empty label", func(i *Intent) { i.Domain = "git..com" }},
|
||||||
|
{"uppercase", func(i *Intent) { i.Domain = "Git.example.com" }},
|
||||||
|
{"ip", func(i *Intent) { i.Domain = "127.0.0.1" }},
|
||||||
|
}
|
||||||
|
for _, tc := range cases {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
i := validIntent()
|
||||||
|
tc.change(&i)
|
||||||
|
if i.Validate() == nil {
|
||||||
|
t.Fatal("accepted invalid intent")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,56 @@
|
|||||||
|
package planner
|
||||||
|
|
||||||
|
import (
|
||||||
|
"crypto/sha256"
|
||||||
|
"encoding/hex"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
const planLifetime = 15 * time.Minute
|
||||||
|
|
||||||
|
// Plan is an offline intent preview, not an executable authorization. Hash binds
|
||||||
|
// its contents but is not a signature. No secret may be added to this structure.
|
||||||
|
type Plan struct {
|
||||||
|
Intent Intent `json:"intent"`
|
||||||
|
ProjectName string `json:"projectName"`
|
||||||
|
DataPath string `json:"dataPath"`
|
||||||
|
CreatedAt time.Time `json:"createdAt"`
|
||||||
|
ExpiresAt time.Time `json:"expiresAt"`
|
||||||
|
Hash string `json:"hash"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func Build(i Intent, now time.Time) (Plan, error) {
|
||||||
|
if err := i.Validate(); err != nil {
|
||||||
|
return Plan{}, err
|
||||||
|
}
|
||||||
|
now = now.UTC().Truncate(time.Second)
|
||||||
|
p := Plan{Intent: i, ProjectName: "sd-" + i.InstanceID, DataPath: "/var/lib/server-deploy/instances/" + i.InstanceID, CreatedAt: now, ExpiresAt: now.Add(planLifetime)}
|
||||||
|
encoded, err := json.Marshal(p)
|
||||||
|
if err != nil {
|
||||||
|
return Plan{}, errors.New("cannot encode plan")
|
||||||
|
}
|
||||||
|
sum := sha256.Sum256(encoded)
|
||||||
|
p.Hash = "sha256:" + hex.EncodeToString(sum[:])
|
||||||
|
return p, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Verify checks offline consistency only. Current must be independently inspected
|
||||||
|
// under a host lock before any future write operation uses this comparison.
|
||||||
|
func (p Plan) Verify(current Intent, now time.Time) error {
|
||||||
|
if now.Before(p.CreatedAt) || !now.Before(p.ExpiresAt) {
|
||||||
|
return errors.New("plan is not within its validity window")
|
||||||
|
}
|
||||||
|
if p.Intent != current {
|
||||||
|
return errors.New("plan does not match current intent or observed state")
|
||||||
|
}
|
||||||
|
expected, err := Build(current, p.CreatedAt)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if p.ProjectName != expected.ProjectName || p.DataPath != expected.DataPath || !p.CreatedAt.Equal(expected.CreatedAt) || !p.ExpiresAt.Equal(expected.ExpiresAt) || p.Hash != expected.Hash {
|
||||||
|
return errors.New("plan integrity check failed")
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,71 @@
|
|||||||
|
package planner
|
||||||
|
|
||||||
|
import (
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestBuildPlan(t *testing.T) {
|
||||||
|
now := time.Date(2026, 9, 25, 12, 0, 0, 0, time.UTC)
|
||||||
|
i := validIntent()
|
||||||
|
p, err := Build(i, now)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if p.ProjectName != "sd-git-one" || p.DataPath != "/var/lib/server-deploy/instances/git-one" {
|
||||||
|
t.Fatalf("wrong instance resources: %+v", p)
|
||||||
|
}
|
||||||
|
if p.ExpiresAt.Sub(p.CreatedAt) != 15*time.Minute {
|
||||||
|
t.Fatal("wrong expiration")
|
||||||
|
}
|
||||||
|
if err := p.Verify(i, now); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
again, _ := Build(i, now)
|
||||||
|
if again.Hash != p.Hash {
|
||||||
|
t.Fatal("unstable plan hash")
|
||||||
|
}
|
||||||
|
i.InstanceID = "git-two"
|
||||||
|
other, _ := Build(i, now)
|
||||||
|
if other.Hash == p.Hash || other.ProjectName == p.ProjectName || other.DataPath == p.DataPath {
|
||||||
|
t.Fatal("instances share identity")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestVerifyRejectsChangedOrExpiredPlan(t *testing.T) {
|
||||||
|
now := time.Date(2026, 9, 25, 12, 0, 0, 0, time.UTC)
|
||||||
|
i := validIntent()
|
||||||
|
original, _ := Build(i, now)
|
||||||
|
cases := []struct {
|
||||||
|
name string
|
||||||
|
change func(*Plan, *Intent, *time.Time)
|
||||||
|
}{
|
||||||
|
{"expired", func(p *Plan, i *Intent, n *time.Time) { *n = p.ExpiresAt }},
|
||||||
|
{"future", func(p *Plan, i *Intent, n *time.Time) { *n = p.CreatedAt.Add(-time.Second) }},
|
||||||
|
{"tampered hash", func(p *Plan, i *Intent, n *time.Time) { p.Hash = "bad" }},
|
||||||
|
{"tampered path", func(p *Plan, i *Intent, n *time.Time) { p.DataPath = "/" }},
|
||||||
|
{"tampered project", func(p *Plan, i *Intent, n *time.Time) { p.ProjectName = "other" }},
|
||||||
|
{"tampered expiry", func(p *Plan, i *Intent, n *time.Time) { p.ExpiresAt = p.ExpiresAt.Add(time.Hour) }},
|
||||||
|
{"state drift", func(p *Plan, i *Intent, n *time.Time) { i.ObservedStateDigest = i.PackageDigest }},
|
||||||
|
{"host drift", func(p *Plan, i *Intent, n *time.Time) { i.HostID = "another-host" }},
|
||||||
|
{"domain drift", func(p *Plan, i *Intent, n *time.Time) { i.Domain = "other.example.com" }},
|
||||||
|
{"intent tamper", func(p *Plan, i *Intent, n *time.Time) { p.Intent.InstanceID = "other" }},
|
||||||
|
}
|
||||||
|
for _, tc := range cases {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
p, current, at := original, i, now
|
||||||
|
tc.change(&p, ¤t, &at)
|
||||||
|
if p.Verify(current, at) == nil {
|
||||||
|
t.Fatal("accepted invalid plan")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestBuildRejectsInvalidIntent(t *testing.T) {
|
||||||
|
i := validIntent()
|
||||||
|
i.InstanceID = "../bad"
|
||||||
|
if _, err := Build(i, time.Now()); err == nil {
|
||||||
|
t.Fatal("built invalid plan")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,57 @@
|
|||||||
|
# Read-only local environment proposal
|
||||||
|
|
||||||
|
`deployctl preflight` takes no stdin request. It collects facts from the local
|
||||||
|
machine and emits protocolVersion, mode=`local-environment-proposal`, observedAt,
|
||||||
|
report and proposal. It never connects to SSH/Docker, runs package managers,
|
||||||
|
sources shell files, writes configuration or starts/restarts services.
|
||||||
|
|
||||||
|
## Observations
|
||||||
|
|
||||||
|
- Reuses `inspect` for OS, architecture and systemd/Docker-client file presence.
|
||||||
|
- Reads only ID, VERSION_ID and VERSION_CODENAME from `/etc/os-release`, at most
|
||||||
|
64 KiB. Handles plain and simply quoted values, not a shell grammar. Ambiguous,
|
||||||
|
duplicated, missing or unsupported value syntax fails closed. Other keys are
|
||||||
|
ignored, not evaluated or returned. Trusted host files/ancestors are assumed.
|
||||||
|
- Linux effective UID is classified root/non_root/unknown.
|
||||||
|
- Linux statfs reports available bytes on the filesystem containing `/var/lib`.
|
||||||
|
This does not measure another configured data root, quotas, or inode capacity.
|
||||||
|
Unknown disk observations cannot authorize a fresh-install candidate.
|
||||||
|
- Resource checks inspect directory entries for `/var/lib/docker`,
|
||||||
|
`/var/lib/containerd`, `/etc/docker`, `/var/lib/server-deploy`, and the Docker
|
||||||
|
`.sources`/`.list` paths under `/etc/apt/sources.list.d`. Contents are not read.
|
||||||
|
Any link or non-directory intermediate component is treated as existing;
|
||||||
|
access failures become unknown, not absent. Checks are conservative hints,
|
||||||
|
not a complete scan of runtime installations or APT sources.
|
||||||
|
- Non-Linux hosts do not read Linux paths or report Linux disk availability.
|
||||||
|
- Reads a bounded local dpkg status snapshot and requires an empty update journal.
|
||||||
|
Reports only the Docker/runtime-related package records and the complete status
|
||||||
|
file digest. Missing/malformed/journal-busy input is unknown, not an empty host.
|
||||||
|
Installed, held, partial and residual relevant records all require manual review.
|
||||||
|
It does not audit unrelated dependency health or non-dpkg installations.
|
||||||
|
|
||||||
|
## Candidate policy
|
||||||
|
|
||||||
|
Linux amd64/arm64; Ubuntu version/codename pairs 22.04/jammy, 24.04/noble,
|
||||||
|
26.04/resolute; root; systemd path present; at least 5 GiB available on /var/lib;
|
||||||
|
Docker client absent; all listed resource paths observed absent. The 5 GiB floor
|
||||||
|
is only a bootstrap screening threshold, not a calculated application/image/backup
|
||||||
|
capacity requirement. Docker official Ubuntu support was checked at implementation:
|
||||||
|
[Docker installation requirements](https://docs.docker.com/engine/install/ubuntu/).
|
||||||
|
This project has not certified these distributions with actual installation tests.
|
||||||
|
|
||||||
|
If all these observations pass, proposal lists candidate source configuration,
|
||||||
|
version-locked package installation and engine/Compose verification steps, plus
|
||||||
|
APT/disk/service-start/firewall impacts. It does not produce shell commands or
|
||||||
|
claim those steps can yet execute. Existing resources cause manual-review blockers;
|
||||||
|
there is no automatic removal, adoption, migration or package conflict cleanup.
|
||||||
|
|
||||||
|
Every proposal remains `executable=false`, with blockers for unverified host
|
||||||
|
identity, unknown package inventory, unresolved versions, repository trust, network/firewall
|
||||||
|
and the unimplemented installer. Empty candidate steps mean preliminary host
|
||||||
|
observations also failed. Nonempty steps are NOT an approved install transaction.
|
||||||
|
|
||||||
|
Reports describe this process's environment (possibly a container/WSL instance),
|
||||||
|
not necessarily the intended cloud server. No snapshot hash, SSH identity binding,
|
||||||
|
freshness token or lock-based revalidation exists yet. These must be implemented
|
||||||
|
before any future write path consumes observations. Exit 0 means a report was
|
||||||
|
produced; inspect `proposal.blockers`, never exit status alone, for readiness.
|
||||||
@@ -0,0 +1,22 @@
|
|||||||
|
//go:build linux
|
||||||
|
|
||||||
|
package preflight
|
||||||
|
|
||||||
|
import (
|
||||||
|
"os"
|
||||||
|
"syscall"
|
||||||
|
|
||||||
|
"server-deploy/internal/inspect"
|
||||||
|
)
|
||||||
|
|
||||||
|
func Collect() Report {
|
||||||
|
return Probe(inspect.Collect(), os.DirFS("/"), os.Geteuid(), diskAvailable("/var/lib"))
|
||||||
|
}
|
||||||
|
|
||||||
|
func diskAvailable(path string) Disk {
|
||||||
|
var stat syscall.Statfs_t
|
||||||
|
if syscall.Statfs(path, &stat) != nil || stat.Bsize <= 0 || stat.Bavail > ^uint64(0)/uint64(stat.Bsize) {
|
||||||
|
return Disk{State: "unknown"}
|
||||||
|
}
|
||||||
|
return Disk{State: "observed", AvailableBytes: stat.Bavail * uint64(stat.Bsize)}
|
||||||
|
}
|
||||||
@@ -0,0 +1,31 @@
|
|||||||
|
//go:build linux
|
||||||
|
|
||||||
|
package preflight
|
||||||
|
|
||||||
|
import (
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestRealDiskObservation(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
if diskAvailable(dir).State != "observed" {
|
||||||
|
t.Fatal("existing filesystem not observed")
|
||||||
|
}
|
||||||
|
if diskAvailable(filepath.Join(dir, "missing")).State != "unknown" {
|
||||||
|
t.Fatal("missing target reported capacity")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestDanglingResourceLinkRequiresReview(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
if err := os.MkdirAll(filepath.Join(dir, "var/lib"), 0700); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.Symlink("missing", filepath.Join(dir, "var/lib/docker")); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if resourceState(os.DirFS(dir), "var/lib/docker") != "present" {
|
||||||
|
t.Fatal("dangling link treated as absent")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,7 @@
|
|||||||
|
//go:build !linux
|
||||||
|
|
||||||
|
package preflight
|
||||||
|
|
||||||
|
import "server-deploy/internal/inspect"
|
||||||
|
|
||||||
|
func Collect() Report { return Probe(inspect.Collect(), nil, -1, Disk{State: "not_checked"}) }
|
||||||
@@ -0,0 +1,208 @@
|
|||||||
|
package preflight
|
||||||
|
|
||||||
|
import (
|
||||||
|
"io"
|
||||||
|
"io/fs"
|
||||||
|
"regexp"
|
||||||
|
"server-deploy/internal/debian"
|
||||||
|
"server-deploy/internal/inspect"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
type Disk struct {
|
||||||
|
State string `json:"state"`
|
||||||
|
AvailableBytes uint64 `json:"availableBytes"`
|
||||||
|
}
|
||||||
|
type Distribution struct {
|
||||||
|
State string `json:"state"`
|
||||||
|
ID string `json:"id"`
|
||||||
|
Version string `json:"version"`
|
||||||
|
Codename string `json:"codename"`
|
||||||
|
}
|
||||||
|
type Resource struct {
|
||||||
|
Path string `json:"path"`
|
||||||
|
State string `json:"state"`
|
||||||
|
}
|
||||||
|
type Report struct {
|
||||||
|
Runtime inspect.Report `json:"runtime"`
|
||||||
|
Distribution Distribution `json:"distribution"`
|
||||||
|
Privilege string `json:"privilege"`
|
||||||
|
Disk Disk `json:"disk"`
|
||||||
|
Resources []Resource `json:"resources"`
|
||||||
|
Inventory debian.Snapshot `json:"inventory"`
|
||||||
|
}
|
||||||
|
type Proposal struct {
|
||||||
|
Executable bool `json:"executable"`
|
||||||
|
Blockers []string `json:"blockers"`
|
||||||
|
ProposedChanges []string `json:"proposedChanges"`
|
||||||
|
Impacts []string `json:"impacts"`
|
||||||
|
}
|
||||||
|
|
||||||
|
var resourcePaths = []string{"var/lib/docker", "var/lib/containerd", "etc/docker", "var/lib/server-deploy", "etc/apt/sources.list.d/docker.sources", "etc/apt/sources.list.d/docker.list"}
|
||||||
|
|
||||||
|
// Probe reads metadata and a bounded os-release file. It never sources shell
|
||||||
|
// files or invokes executables. Disk describes /var/lib, not an arbitrary target.
|
||||||
|
func Probe(runtime inspect.Report, files fs.FS, uid int, disk Disk) Report {
|
||||||
|
r := Report{Runtime: runtime, Distribution: Distribution{State: "not_checked"}, Privilege: "not_checked", Disk: Disk{State: "not_checked"}, Resources: []Resource{}, Inventory: debian.Snapshot{State: "not_checked", Packages: []debian.Installed{}}}
|
||||||
|
if runtime.OS != "linux" {
|
||||||
|
return r
|
||||||
|
}
|
||||||
|
r.Disk = disk
|
||||||
|
r.Privilege = "non_root"
|
||||||
|
if uid == 0 {
|
||||||
|
r.Privilege = "root"
|
||||||
|
} else if uid < 0 {
|
||||||
|
r.Privilege = "unknown"
|
||||||
|
}
|
||||||
|
r.Distribution = readDistribution(files)
|
||||||
|
r.Inventory = debian.Inventory(files)
|
||||||
|
for _, p := range resourcePaths {
|
||||||
|
r.Resources = append(r.Resources, Resource{Path: "/" + p, State: resourceState(files, p)})
|
||||||
|
}
|
||||||
|
return r
|
||||||
|
}
|
||||||
|
|
||||||
|
// Plan is a proposal, not an executable or approved installation plan. Missing
|
||||||
|
// package inventory/version locks and host identity always block execution.
|
||||||
|
func Plan(r Report) Proposal {
|
||||||
|
p := Proposal{Blockers: []string{}, ProposedChanges: []string{}, Impacts: []string{}}
|
||||||
|
if r.Runtime.OS != "linux" || (r.Runtime.Architecture != "amd64" && r.Runtime.Architecture != "arm64") {
|
||||||
|
p.Blockers = append(p.Blockers, "unsupported_platform")
|
||||||
|
}
|
||||||
|
if r.Distribution.State != "observed" {
|
||||||
|
p.Blockers = append(p.Blockers, "distribution_unverified")
|
||||||
|
} else if r.Distribution.ID != "ubuntu" || !supportedSuite(r.Distribution) {
|
||||||
|
p.Blockers = append(p.Blockers, "unsupported_distribution")
|
||||||
|
}
|
||||||
|
if r.Privilege != "root" {
|
||||||
|
p.Blockers = append(p.Blockers, "root_required")
|
||||||
|
}
|
||||||
|
if r.Runtime.SystemdRuntime != "present" {
|
||||||
|
p.Blockers = append(p.Blockers, "systemd_unverified")
|
||||||
|
}
|
||||||
|
if r.Disk.State != "observed" {
|
||||||
|
p.Blockers = append(p.Blockers, "disk_unverified")
|
||||||
|
} else if r.Disk.AvailableBytes < 5<<30 {
|
||||||
|
p.Blockers = append(p.Blockers, "disk_below_bootstrap_floor")
|
||||||
|
}
|
||||||
|
if r.Runtime.DockerClient != "missing" {
|
||||||
|
p.Blockers = append(p.Blockers, "existing_or_unknown_runtime_requires_review")
|
||||||
|
}
|
||||||
|
// Validate the complete path set as well: an incomplete report is not clean.
|
||||||
|
seen := make(map[string]bool)
|
||||||
|
resourcesClean := len(r.Resources) == len(resourcePaths)
|
||||||
|
for _, v := range r.Resources {
|
||||||
|
if v.State != "missing" || seen[v.Path] {
|
||||||
|
resourcesClean = false
|
||||||
|
}
|
||||||
|
seen[v.Path] = true
|
||||||
|
}
|
||||||
|
for _, path := range resourcePaths {
|
||||||
|
if !seen["/"+path] {
|
||||||
|
resourcesClean = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !resourcesClean {
|
||||||
|
p.Blockers = append(p.Blockers, "existing_resources_require_review")
|
||||||
|
}
|
||||||
|
if r.Inventory.State != "observed" {
|
||||||
|
p.Blockers = append(p.Blockers, "package_inventory_unverified")
|
||||||
|
} else if len(r.Inventory.Packages) > 0 {
|
||||||
|
p.Blockers = append(p.Blockers, "existing_packages_require_review")
|
||||||
|
}
|
||||||
|
if len(p.Blockers) == 0 {
|
||||||
|
p.ProposedChanges = []string{"configure_verified_docker_apt_source", "install_version_locked_docker_packages", "verify_local_engine_and_compose"}
|
||||||
|
p.Impacts = []string{"apt_configuration_and_package_database_changes", "docker_service_may_start_during_install", "docker_may_change_host_network_firewall_rules", "system_disk_usage_increases"}
|
||||||
|
}
|
||||||
|
p.Blockers = append(p.Blockers, "host_identity_unverified", "package_versions_unresolved", "repository_trust_unverified", "network_and_firewall_unverified", "installation_executor_unimplemented")
|
||||||
|
return p
|
||||||
|
}
|
||||||
|
|
||||||
|
func supportedSuite(d Distribution) bool {
|
||||||
|
return map[string]string{"22.04": "jammy", "24.04": "noble", "26.04": "resolute"}[d.Version] == d.Codename && d.Codename != ""
|
||||||
|
}
|
||||||
|
|
||||||
|
var releaseValue = regexp.MustCompile(`^[a-z0-9][a-z0-9._-]{0,63}$`)
|
||||||
|
|
||||||
|
func readDistribution(files fs.FS) Distribution {
|
||||||
|
initial, err := fs.Stat(files, "etc/os-release")
|
||||||
|
if err != nil {
|
||||||
|
return Distribution{State: "unknown"}
|
||||||
|
}
|
||||||
|
if !initial.Mode().IsRegular() || initial.Size() > 65536 {
|
||||||
|
return Distribution{State: "invalid"}
|
||||||
|
}
|
||||||
|
f, err := files.Open("etc/os-release")
|
||||||
|
if err != nil {
|
||||||
|
return Distribution{State: "unknown"}
|
||||||
|
}
|
||||||
|
defer f.Close()
|
||||||
|
info, err := f.Stat()
|
||||||
|
if err != nil {
|
||||||
|
return Distribution{State: "unknown"}
|
||||||
|
}
|
||||||
|
if !info.Mode().IsRegular() || info.Size() > 65536 {
|
||||||
|
return Distribution{State: "invalid"}
|
||||||
|
}
|
||||||
|
raw, err := io.ReadAll(io.LimitReader(f, 65537))
|
||||||
|
if err != nil {
|
||||||
|
return Distribution{State: "unknown"}
|
||||||
|
}
|
||||||
|
if len(raw) > 65536 {
|
||||||
|
return Distribution{State: "invalid"}
|
||||||
|
}
|
||||||
|
values := map[string]string{}
|
||||||
|
for _, line := range strings.Split(string(raw), "\n") {
|
||||||
|
key, value, ok := strings.Cut(strings.TrimSpace(line), "=")
|
||||||
|
if key != "ID" && key != "VERSION_ID" && key != "VERSION_CODENAME" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if !ok || values[key] != "" {
|
||||||
|
return Distribution{State: "invalid"}
|
||||||
|
}
|
||||||
|
if len(value) >= 2 && ((value[0] == '"' && value[len(value)-1] == '"') || (value[0] == '\'' && value[len(value)-1] == '\'')) {
|
||||||
|
value = value[1 : len(value)-1]
|
||||||
|
}
|
||||||
|
if !releaseValue.MatchString(value) {
|
||||||
|
return Distribution{State: "invalid"}
|
||||||
|
}
|
||||||
|
values[key] = value
|
||||||
|
}
|
||||||
|
if len(values) != 3 {
|
||||||
|
return Distribution{State: "invalid"}
|
||||||
|
}
|
||||||
|
return Distribution{State: "observed", ID: values["ID"], Version: values["VERSION_ID"], Codename: values["VERSION_CODENAME"]}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Walk directory entries to observe dangling/intermediate links as existing
|
||||||
|
// resources instead of following them and misreporting a clean install target.
|
||||||
|
func resourceState(files fs.FS, path string) string {
|
||||||
|
parent := "."
|
||||||
|
parts := strings.Split(path, "/")
|
||||||
|
for i, part := range parts {
|
||||||
|
entries, err := fs.ReadDir(files, parent)
|
||||||
|
if err != nil {
|
||||||
|
return "unknown"
|
||||||
|
}
|
||||||
|
found := false
|
||||||
|
for _, entry := range entries {
|
||||||
|
if entry.Name() != part {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
found = true
|
||||||
|
if i == len(parts)-1 || entry.Type()&fs.ModeSymlink != 0 || !entry.IsDir() {
|
||||||
|
return "present"
|
||||||
|
}
|
||||||
|
if parent == "." {
|
||||||
|
parent = part
|
||||||
|
} else {
|
||||||
|
parent += "/" + part
|
||||||
|
}
|
||||||
|
break
|
||||||
|
}
|
||||||
|
if !found {
|
||||||
|
return "missing"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return "unknown"
|
||||||
|
}
|
||||||
@@ -0,0 +1,116 @@
|
|||||||
|
package preflight
|
||||||
|
|
||||||
|
import (
|
||||||
|
"io/fs"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"testing/fstest"
|
||||||
|
|
||||||
|
"server-deploy/internal/inspect"
|
||||||
|
)
|
||||||
|
|
||||||
|
func hostFiles() fstest.MapFS {
|
||||||
|
return fstest.MapFS{
|
||||||
|
"etc/os-release": {Data: []byte("ID=ubuntu\nVERSION_ID=\"26.04\"\nVERSION_CODENAME=resolute\n")},
|
||||||
|
"var/lib/dpkg/status": {Data: []byte("Package: base-files\nStatus: install ok installed\nArchitecture: amd64\nVersion: 1.0\n")},
|
||||||
|
"var/lib/dpkg/updates": {Mode: fs.ModeDir | 0700},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPackageStateBlocksFreshInstallCandidates(t *testing.T) {
|
||||||
|
for _, state := range []string{"install ok installed", "deinstall ok config-files", "install reinstreq half-installed"} {
|
||||||
|
files := hostFiles()
|
||||||
|
files["var/lib/dpkg/status"].Data = []byte("Package: containerd\nStatus: " + state + "\nArchitecture: amd64\nVersion: 1.2.3\n")
|
||||||
|
p := Plan(Probe(baseline(), files, 0, Disk{State: "observed", AvailableBytes: 20 << 30}))
|
||||||
|
if len(p.ProposedChanges) != 0 || !contains(p.Blockers, "existing_packages_require_review") {
|
||||||
|
t.Errorf("existing package state %s overlooked", state)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
files := hostFiles()
|
||||||
|
delete(files, "var/lib/dpkg/status")
|
||||||
|
if p := Plan(Probe(baseline(), files, 0, Disk{State: "observed", AvailableBytes: 20 << 30})); len(p.ProposedChanges) != 0 {
|
||||||
|
t.Fatal("unknown inventory treated as empty")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func baseline() inspect.Report {
|
||||||
|
return inspect.Report{ProtocolVersion: 1, OS: "linux", Architecture: "amd64", SystemdRuntime: "present", DockerClient: "missing"}
|
||||||
|
}
|
||||||
|
func TestHostFactsAndNonExecutableProposal(t *testing.T) {
|
||||||
|
r := Probe(baseline(), hostFiles(), 0, Disk{State: "observed", AvailableBytes: 20 << 30})
|
||||||
|
if r.Distribution.ID != "ubuntu" || r.Distribution.Version != "26.04" || r.Distribution.State != "observed" || r.Privilege != "root" {
|
||||||
|
t.Fatalf("incorrect host facts: %+v", r)
|
||||||
|
}
|
||||||
|
p := Plan(r)
|
||||||
|
if p.Executable || len(p.ProposedChanges) == 0 || !contains(p.Blockers, "package_versions_unresolved") || !contains(p.Blockers, "host_identity_unverified") {
|
||||||
|
t.Fatalf("unsafe proposal: %+v", p)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestExistingResourcesNeverProposeFreshInstall(t *testing.T) {
|
||||||
|
for _, path := range []string{"var/lib/docker", "var/lib/containerd", "etc/docker", "var/lib/server-deploy", "etc/apt/sources.list.d/docker.sources", "etc/apt/sources.list.d/docker.list"} {
|
||||||
|
files := hostFiles()
|
||||||
|
files[path] = &fstest.MapFile{Mode: fs.ModeDir | 0700}
|
||||||
|
p := Plan(Probe(baseline(), files, 0, Disk{State: "observed", AvailableBytes: 20 << 30}))
|
||||||
|
if len(p.ProposedChanges) != 0 || !contains(p.Blockers, "existing_resources_require_review") {
|
||||||
|
t.Errorf("fresh install proposed over %s", path)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
runtime := baseline()
|
||||||
|
runtime.DockerClient = "present"
|
||||||
|
if p := Plan(Probe(runtime, hostFiles(), 0, Disk{State: "observed", AvailableBytes: 20 << 30})); len(p.ProposedChanges) != 0 {
|
||||||
|
t.Fatal("existing Docker overlooked")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestUnknownAndInsufficientHostFailsClosed(t *testing.T) {
|
||||||
|
for _, tc := range []struct {
|
||||||
|
name string
|
||||||
|
runtime inspect.Report
|
||||||
|
uid int
|
||||||
|
disk Disk
|
||||||
|
blocker string
|
||||||
|
}{
|
||||||
|
{"nonroot", baseline(), 1000, Disk{State: "observed", AvailableBytes: 20 << 30}, "root_required"},
|
||||||
|
{"disk unknown", baseline(), 0, Disk{State: "unknown"}, "disk_unverified"},
|
||||||
|
{"disk low", baseline(), 0, Disk{State: "observed", AvailableBytes: 1}, "disk_below_bootstrap_floor"},
|
||||||
|
{"unsupported", inspect.Report{OS: "windows", Architecture: "amd64"}, 0, Disk{}, "unsupported_platform"},
|
||||||
|
} {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
p := Plan(Probe(tc.runtime, hostFiles(), tc.uid, tc.disk))
|
||||||
|
if !contains(p.Blockers, tc.blocker) || p.Executable || len(p.ProposedChanges) != 0 {
|
||||||
|
t.Fatalf("unsafe proposal: %+v", p)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestOSReleaseRejectsAmbiguityAndNeverEvaluatesShell(t *testing.T) {
|
||||||
|
for _, content := range []string{"ID=ubuntu\nID=debian\nVERSION_ID=26.04\nVERSION_CODENAME=resolute", "ID=$(touch secret)\nVERSION_ID=26.04\nVERSION_CODENAME=resolute", "ID=ubuntu\nVERSION_ID=\"26.04\nVERSION_CODENAME=resolute", strings.Repeat("#", 65537)} {
|
||||||
|
files := hostFiles()
|
||||||
|
files["etc/os-release"].Data = []byte(content)
|
||||||
|
p := Plan(Probe(baseline(), files, 0, Disk{State: "observed", AvailableBytes: 20 << 30}))
|
||||||
|
if len(p.ProposedChanges) != 0 || !contains(p.Blockers, "distribution_unverified") {
|
||||||
|
t.Fatal("ambiguous distribution accepted")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
files := hostFiles()
|
||||||
|
files["etc/os-release"].Data = []byte("ID=ubuntu\nVERSION_ID=26.04\nVERSION_CODENAME=noble\n")
|
||||||
|
if p := Plan(Probe(baseline(), files, 0, Disk{State: "observed", AvailableBytes: 20 << 30})); !contains(p.Blockers, "unsupported_distribution") {
|
||||||
|
t.Fatal("mismatched suite accepted")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
type deniedFS struct{}
|
||||||
|
|
||||||
|
func (deniedFS) Open(string) (fs.File, error) { return nil, fs.ErrPermission }
|
||||||
|
func TestAccessFailuresAreNotAbsence(t *testing.T) {
|
||||||
|
r := Probe(baseline(), deniedFS{}, 0, Disk{State: "observed", AvailableBytes: 20 << 30})
|
||||||
|
if r.Distribution.State != "unknown" || r.Resources[0].State != "unknown" || len(Plan(r).ProposedChanges) != 0 {
|
||||||
|
t.Fatal("permission failure treated as clean host")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func contains(values []string, want string) bool {
|
||||||
|
for _, v := range values {
|
||||||
|
if v == want {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
@@ -0,0 +1,79 @@
|
|||||||
|
# Operation metadata store
|
||||||
|
|
||||||
|
Internal execution foundation; not a CLI write endpoint, application executor,
|
||||||
|
backup engine or systemd supervisor. The existing CLI remains read-only.
|
||||||
|
|
||||||
|
## Contract
|
||||||
|
|
||||||
|
Bootstrap must supply one fixed, existing, administrator-owned local directory
|
||||||
|
per host. Use the same canonical directory for every runner on that host.
|
||||||
|
Untrusted users/applications must not be able to replace the directory, its
|
||||||
|
ancestors or files. Linux deployment permissions should be 0700 for the directory
|
||||||
|
and 0600 for metadata. Windows ACL configuration belongs to the installer.
|
||||||
|
NFS/SMB/distributed locking is not supported.
|
||||||
|
|
||||||
|
`Acquire(directory, hostID)` opens an os.Root and takes a nonblocking exclusive
|
||||||
|
OS lock on `host.lock`. Different paths/roots are not a distributed host registry.
|
||||||
|
Hold the returned Session throughout any future application write operation.
|
||||||
|
Never remove/replace `host.lock`, including during cleanup: its inode is the lock
|
||||||
|
identity. Close releases it; abrupt process termination releases it at OS level.
|
||||||
|
|
||||||
|
- `Begin(id, planHash)` registers queued work and returns `(operation, created)`.
|
||||||
|
Reusing ID/hash returns the original record with created=false. Another hash
|
||||||
|
conflicts. Different IDs are blocked while an unresolved record exists.
|
||||||
|
- `Advance(id, revision, next)` compares revision and validates the transition.
|
||||||
|
A queued-to-running transition is the claim; a second claim fails.
|
||||||
|
- `Get(id)` returns a value copy. Closing or poisoning a session forbids its use.
|
||||||
|
|
||||||
|
Allowed transitions:
|
||||||
|
|
||||||
|
```text
|
||||||
|
queued → running | cancelled
|
||||||
|
running → succeeded | failed_recovered | needs_attention | unknown
|
||||||
|
needs_attention / unknown → succeeded | failed_recovered
|
||||||
|
terminal states → no transitions
|
||||||
|
```
|
||||||
|
|
||||||
|
Success/recovery labels are assertions by the caller, not proof. The future
|
||||||
|
executor must verify actual effects before recording them. Unknown/running work
|
||||||
|
survives restart without replay. Reconciliation requires inspecting actual state;
|
||||||
|
no automatic retry, forced reset, lease expiry or stale-lock deletion is provided.
|
||||||
|
|
||||||
|
## Persistence
|
||||||
|
|
||||||
|
state.json is a versioned, canonical JSON snapshot, maximum 16 MiB, with host
|
||||||
|
identity and SHA-256 integrity checksum. Unknown fields, duplicates, truncation,
|
||||||
|
changed data and host mismatch are rejected. The checksum is not authentication.
|
||||||
|
It is not an append-only audit log; event logging is a separate future component.
|
||||||
|
|
||||||
|
Mutation writes a unique private temporary file, syncs it, closes it, renames over
|
||||||
|
the snapshot and (Linux) syncs the containing directory. A save error poisons the
|
||||||
|
session because the rename may already have happened; close, reacquire and query
|
||||||
|
before deciding anything. Temporary remnants from a killed process are ignored,
|
||||||
|
never interpreted as successful work; automatic cleanup is not implemented.
|
||||||
|
|
||||||
|
Current snapshots retain all operation IDs. No pruning is provided, because
|
||||||
|
forgetting completed IDs can re-enable an old request. At the size limit, writes
|
||||||
|
fail closed. Admission reserves 64 bytes for the active record's later status
|
||||||
|
and revision growth; capacity rejection does not poison read access. A retention/tombstone design is required before bounded production
|
||||||
|
history cleanup is introduced.
|
||||||
|
|
||||||
|
host.lock also contains a synced initialization marker. Once work is persisted,
|
||||||
|
a missing snapshot is rejected rather than treated as a fresh store. A crash
|
||||||
|
between the first snapshot and marker can be repaired only from a valid snapshot.
|
||||||
|
Protect both files. Restoring an older valid snapshot still rolls back idempotency
|
||||||
|
history; do not restart execution without independent reconciliation. This package
|
||||||
|
cannot detect malicious administrator edits or rollback/deletion of the entire
|
||||||
|
state directory.
|
||||||
|
|
||||||
|
## Platform boundary
|
||||||
|
|
||||||
|
Linux uses flock and file/directory fsync. Windows uses LockFileEx for development
|
||||||
|
tests, file sync and rename; Windows power-loss durability is not promised.
|
||||||
|
Other OSes refuse acquisition. The local macOS panel will communicate with the
|
||||||
|
Linux runner, not use this package as its local SQLite replacement.
|
||||||
|
|
||||||
|
Kernel locks are advisory on Linux; all participating writers must obey them.
|
||||||
|
External Docker/Portainer operations are outside this lock and require drift
|
||||||
|
checks. Multi-process tests prove process-crash behavior, not sudden power failure
|
||||||
|
or storage-hardware reliability.
|
||||||
@@ -0,0 +1,26 @@
|
|||||||
|
//go:build linux
|
||||||
|
|
||||||
|
package state
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"os"
|
||||||
|
"syscall"
|
||||||
|
)
|
||||||
|
|
||||||
|
func lockExclusive(f *os.File) error {
|
||||||
|
err := syscall.Flock(int(f.Fd()), syscall.LOCK_EX|syscall.LOCK_NB)
|
||||||
|
if errors.Is(err, syscall.EWOULDBLOCK) || errors.Is(err, syscall.EAGAIN) {
|
||||||
|
return ErrBusy
|
||||||
|
}
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
func syncDirectory(r *os.Root) error {
|
||||||
|
f, err := r.Open(".")
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
defer f.Close()
|
||||||
|
return f.Sync()
|
||||||
|
}
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
//go:build !linux && !windows
|
||||||
|
|
||||||
|
package state
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"os"
|
||||||
|
)
|
||||||
|
|
||||||
|
func lockExclusive(*os.File) error {
|
||||||
|
return errors.New("operation store supports Linux and Windows only")
|
||||||
|
}
|
||||||
|
func syncDirectory(*os.Root) error { return errors.New("unsupported state durability platform") }
|
||||||
@@ -0,0 +1,30 @@
|
|||||||
|
//go:build windows
|
||||||
|
|
||||||
|
package state
|
||||||
|
|
||||||
|
import (
|
||||||
|
"os"
|
||||||
|
"runtime"
|
||||||
|
"syscall"
|
||||||
|
"unsafe"
|
||||||
|
)
|
||||||
|
|
||||||
|
var lockFileEx = syscall.NewLazyDLL("kernel32.dll").NewProc("LockFileEx")
|
||||||
|
|
||||||
|
func lockExclusive(f *os.File) error {
|
||||||
|
var overlapped syscall.Overlapped
|
||||||
|
// LOCKFILE_FAIL_IMMEDIATELY | LOCKFILE_EXCLUSIVE_LOCK, first byte only.
|
||||||
|
ok, _, err := lockFileEx.Call(f.Fd(), 3, 0, 1, 0, uintptr(unsafe.Pointer(&overlapped)))
|
||||||
|
runtime.KeepAlive(f)
|
||||||
|
if ok != 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if err == syscall.Errno(33) {
|
||||||
|
return ErrBusy
|
||||||
|
}
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
// Windows is a development platform. File.Sync is used, but Go does not provide
|
||||||
|
// a portable directory fsync here. Do not claim Windows power-loss durability.
|
||||||
|
func syncDirectory(*os.Root) error { return nil }
|
||||||
@@ -0,0 +1,56 @@
|
|||||||
|
// Package state persists task metadata. It never executes application actions.
|
||||||
|
package state
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"regexp"
|
||||||
|
)
|
||||||
|
|
||||||
|
var (
|
||||||
|
ErrBusy = errors.New("host state is locked")
|
||||||
|
ErrClosed = errors.New("session is closed")
|
||||||
|
ErrPoisoned = errors.New("state write outcome uncertain; reopen and reconcile")
|
||||||
|
ErrConflict = errors.New("operation identity or revision conflict")
|
||||||
|
ErrUnresolved = errors.New("another operation requires completion or reconciliation")
|
||||||
|
ErrTransition = errors.New("invalid operation transition")
|
||||||
|
ErrNotFound = errors.New("operation not found")
|
||||||
|
ErrCapacity = errors.New("operation history capacity exhausted")
|
||||||
|
idPattern = regexp.MustCompile(`^[a-z][a-z0-9-]{0,47}$`)
|
||||||
|
hashPattern = regexp.MustCompile(`^sha256:[a-f0-9]{64}$`)
|
||||||
|
)
|
||||||
|
|
||||||
|
type Status string
|
||||||
|
|
||||||
|
const (
|
||||||
|
Queued Status = "queued"
|
||||||
|
Running Status = "running"
|
||||||
|
Succeeded Status = "succeeded"
|
||||||
|
FailedRecovered Status = "failed_recovered"
|
||||||
|
NeedsAttention Status = "needs_attention"
|
||||||
|
Unknown Status = "unknown"
|
||||||
|
Cancelled Status = "cancelled"
|
||||||
|
)
|
||||||
|
|
||||||
|
type Operation struct {
|
||||||
|
ID string `json:"id"`
|
||||||
|
PlanHash string `json:"planHash"`
|
||||||
|
Status Status `json:"status"`
|
||||||
|
Revision uint64 `json:"revision"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s Status) terminal() bool { return s == Succeeded || s == FailedRecovered || s == Cancelled }
|
||||||
|
func (s Status) valid() bool {
|
||||||
|
return s.terminal() || s == Queued || s == Running || s == NeedsAttention || s == Unknown
|
||||||
|
}
|
||||||
|
func allowed(from, to Status) bool {
|
||||||
|
switch from {
|
||||||
|
case Queued:
|
||||||
|
return to == Running || to == Cancelled
|
||||||
|
case Running:
|
||||||
|
return to == Succeeded || to == FailedRecovered || to == NeedsAttention || to == Unknown
|
||||||
|
case NeedsAttention, Unknown:
|
||||||
|
return to == Succeeded || to == FailedRecovered
|
||||||
|
default:
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,293 @@
|
|||||||
|
package state
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"sync/atomic"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestIdempotencyAndTransitions(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
s, err := Acquire(dir, "host-one")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer s.Close()
|
||||||
|
op, created, err := s.Begin("op-one", testHash)
|
||||||
|
if err != nil || !created || op.Status != Queued || op.Revision != 1 {
|
||||||
|
t.Fatalf("begin: %+v %v", op, err)
|
||||||
|
}
|
||||||
|
duplicate, created, err := s.Begin("op-one", testHash)
|
||||||
|
if err != nil || created || duplicate != op {
|
||||||
|
t.Fatal("duplicate mutated operation")
|
||||||
|
}
|
||||||
|
if _, _, err := s.Begin("op-one", "sha256:"+strings.Repeat("b", 64)); !errors.Is(err, ErrConflict) {
|
||||||
|
t.Fatal("id rebound to another plan")
|
||||||
|
}
|
||||||
|
if _, _, err := s.Begin("op-two", testHash); !errors.Is(err, ErrUnresolved) {
|
||||||
|
t.Fatal("unresolved operation bypass")
|
||||||
|
}
|
||||||
|
if _, err := s.Advance(op.ID, op.Revision, Succeeded); !errors.Is(err, ErrTransition) {
|
||||||
|
t.Fatal("queued operation succeeded without running")
|
||||||
|
}
|
||||||
|
op, err = s.Advance(op.ID, op.Revision, Running)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := s.Advance(op.ID, 1, Succeeded); !errors.Is(err, ErrConflict) {
|
||||||
|
t.Fatal("stale revision accepted")
|
||||||
|
}
|
||||||
|
if _, err := s.Advance(op.ID, op.Revision, Running); !errors.Is(err, ErrTransition) {
|
||||||
|
t.Fatal("operation claimed twice")
|
||||||
|
}
|
||||||
|
op, err = s.Advance(op.ID, op.Revision, NeedsAttention)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, _, err := s.Begin("op-two", testHash); !errors.Is(err, ErrUnresolved) {
|
||||||
|
t.Fatal("ignored manual recovery")
|
||||||
|
}
|
||||||
|
op, err = s.Advance(op.ID, op.Revision, FailedRecovered)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := s.Advance(op.ID, op.Revision, Running); !errors.Is(err, ErrTransition) {
|
||||||
|
t.Fatal("terminal operation restarted")
|
||||||
|
}
|
||||||
|
s.Close()
|
||||||
|
s, err = Acquire(dir, "host-one")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer s.Close()
|
||||||
|
got, err := s.Get("op-one")
|
||||||
|
if err != nil || got != op {
|
||||||
|
t.Fatalf("persistence mismatch: %+v %v", got, err)
|
||||||
|
}
|
||||||
|
if _, created, err := s.Begin("op-two", testHash); err != nil || !created {
|
||||||
|
t.Fatal("completed operation blocks new work")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCorruptionAndHostMismatchFailClosed(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
s, err := Acquire(dir, "host-one")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, _, err := s.Begin("op-one", testHash); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
s.Close()
|
||||||
|
if other, err := Acquire(dir, "host-two"); err == nil {
|
||||||
|
other.Close()
|
||||||
|
t.Fatal("wrong host accepted")
|
||||||
|
}
|
||||||
|
for _, data := range []string{`{`, `{}`, `null`, strings.Repeat("x", maxSnapshotBytes+1)} {
|
||||||
|
if err := os.WriteFile(filepath.Join(dir, "state.json"), []byte(data), 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if other, err := Acquire(dir, "host-one"); err == nil {
|
||||||
|
other.Close()
|
||||||
|
t.Fatal("corrupt state silently reset")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInvalidOperationsDoNotCreateSnapshot(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
s, err := Acquire(dir, "host-one")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer s.Close()
|
||||||
|
for _, id := range []string{"", "../outside", "bad/id"} {
|
||||||
|
if _, _, err := s.Begin(id, testHash); err == nil {
|
||||||
|
t.Fatal("invalid operation id accepted")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if _, _, err := s.Begin("op-one", "latest"); err == nil {
|
||||||
|
t.Fatal("mutable plan binding accepted")
|
||||||
|
}
|
||||||
|
if _, err := s.Get("missing"); !errors.Is(err, ErrNotFound) {
|
||||||
|
t.Fatal("missing operation not reported")
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(filepath.Join(dir, "state.json")); !os.IsNotExist(err) {
|
||||||
|
t.Fatal("invalid request persisted state")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPersistenceFailurePoisonsSession(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
s, err := Acquire(dir, "host-one")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer s.Close()
|
||||||
|
// A directory at the destination makes atomic replacement fail on both OSes.
|
||||||
|
if err := os.Mkdir(filepath.Join(dir, "state.json"), 0700); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, _, err := s.Begin("op-one", testHash); err == nil {
|
||||||
|
t.Fatal("reported failed persistence as success")
|
||||||
|
}
|
||||||
|
if _, _, err := s.Begin("op-two", testHash); !errors.Is(err, ErrPoisoned) {
|
||||||
|
t.Fatalf("continued after ambiguous write: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestConcurrentClaimsHaveOneWinner(t *testing.T) {
|
||||||
|
s, err := Acquire(t.TempDir(), "host-one")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer s.Close()
|
||||||
|
var createdCount, claimedCount atomic.Int32
|
||||||
|
var wg sync.WaitGroup
|
||||||
|
for n := 0; n < 16; n++ {
|
||||||
|
wg.Add(1)
|
||||||
|
go func() {
|
||||||
|
defer wg.Done()
|
||||||
|
_, created, err := s.Begin("op-one", testHash)
|
||||||
|
if err != nil {
|
||||||
|
t.Error(err)
|
||||||
|
}
|
||||||
|
if created {
|
||||||
|
createdCount.Add(1)
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
}
|
||||||
|
wg.Wait()
|
||||||
|
for n := 0; n < 16; n++ {
|
||||||
|
wg.Add(1)
|
||||||
|
go func() {
|
||||||
|
defer wg.Done()
|
||||||
|
_, err := s.Advance("op-one", 1, Running)
|
||||||
|
if err == nil {
|
||||||
|
claimedCount.Add(1)
|
||||||
|
} else if !errors.Is(err, ErrConflict) {
|
||||||
|
t.Error(err)
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
}
|
||||||
|
wg.Wait()
|
||||||
|
if createdCount.Load() != 1 || claimedCount.Load() != 1 {
|
||||||
|
t.Fatalf("duplicate winners: %d %d", createdCount.Load(), claimedCount.Load())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestTamperedSnapshotRejected(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
s, err := Acquire(dir, "host-one")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, _, err := s.Begin("op-one", testHash); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
s.Close()
|
||||||
|
path := filepath.Join(dir, "state.json")
|
||||||
|
raw, err := os.ReadFile(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
for _, changed := range []string{
|
||||||
|
strings.Replace(string(raw), `"queued"`, `"succeeded"`, 1),
|
||||||
|
strings.Replace(string(raw), `"version":1`, `"version":1,"version":1`, 1),
|
||||||
|
string(raw) + ` {}`,
|
||||||
|
} {
|
||||||
|
if err := os.WriteFile(path, []byte(changed), 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if s, err := Acquire(dir, "host-one"); err == nil {
|
||||||
|
s.Close()
|
||||||
|
t.Fatal("tampered snapshot accepted")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAllTerminalAndReconciliationPaths(t *testing.T) {
|
||||||
|
for _, path := range [][]Status{{Cancelled}, {Running, Succeeded}, {Running, FailedRecovered}, {Running, Unknown, FailedRecovered}, {Running, NeedsAttention, Succeeded}} {
|
||||||
|
s, err := Acquire(t.TempDir(), "host-one")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
op, _, err := s.Begin("op-one", testHash)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
for _, next := range path {
|
||||||
|
op, err = s.Advance(op.ID, op.Revision, next)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if _, _, err := s.Begin("op-two", testHash); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
s.Close()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestMissingSnapshotDoesNotResetInitializedStore(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
s, err := Acquire(dir, "host-one")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, _, err := s.Begin("op-one", testHash); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
s.Close()
|
||||||
|
if err := os.Remove(filepath.Join(dir, "state.json")); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if s, err := Acquire(dir, "host-one"); err == nil {
|
||||||
|
s.Close()
|
||||||
|
t.Fatal("silently reset initialized store")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAdmissionReservesSpaceForCompletion(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
data := snapshot{Version: 1, HostID: "host-one", Operations: make(map[string]Operation)}
|
||||||
|
for n := 0; n < 113357; n++ {
|
||||||
|
id := fmt.Sprintf("op%06d", n)
|
||||||
|
if n < 2 {
|
||||||
|
id += strings.Repeat("a", 26)
|
||||||
|
}
|
||||||
|
data.Operations[id] = Operation{ID: id, PlanHash: testHash, Status: Cancelled, Revision: 2}
|
||||||
|
}
|
||||||
|
raw, err := encodeSnapshot(data)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(filepath.Join(dir, "state.json"), raw, 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
// Confirm this fixture reaches the actual bug boundary, not an arbitrary limit.
|
||||||
|
data.Operations["op-new"] = Operation{ID: "op-new", PlanHash: testHash, Status: Queued, Revision: 1}
|
||||||
|
queued, err := encodeSnapshot(data)
|
||||||
|
if err != nil || len(queued) != maxSnapshotBytes-2 {
|
||||||
|
t.Fatalf("fixture outside boundary: %d %v", len(queued), err)
|
||||||
|
}
|
||||||
|
s, err := Acquire(dir, "host-one")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer s.Close()
|
||||||
|
if _, _, err := s.Begin("op-new", testHash); err == nil {
|
||||||
|
t.Fatal("admitted task without room for terminal state")
|
||||||
|
}
|
||||||
|
if _, err := s.Get("op000002"); err != nil {
|
||||||
|
t.Fatalf("capacity rejection poisoned read access: %v", err)
|
||||||
|
}
|
||||||
|
if _, err := s.Get("op-new"); !errors.Is(err, ErrNotFound) {
|
||||||
|
t.Fatal("rejected admission persisted")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,120 @@
|
|||||||
|
package state
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"crypto/rand"
|
||||||
|
"crypto/sha256"
|
||||||
|
"encoding/hex"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"io"
|
||||||
|
"os"
|
||||||
|
)
|
||||||
|
|
||||||
|
const maxSnapshotBytes = 16 << 20
|
||||||
|
|
||||||
|
type snapshot struct {
|
||||||
|
Version int `json:"version"`
|
||||||
|
HostID string `json:"hostId"`
|
||||||
|
Operations map[string]Operation `json:"operations"`
|
||||||
|
Checksum string `json:"checksum"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func encodeSnapshot(s snapshot) ([]byte, error) {
|
||||||
|
s.Checksum = ""
|
||||||
|
raw, err := json.Marshal(s)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
sum := sha256.Sum256(raw)
|
||||||
|
s.Checksum = "sha256:" + hex.EncodeToString(sum[:])
|
||||||
|
raw, err = json.Marshal(s)
|
||||||
|
if len(raw) > maxSnapshotBytes {
|
||||||
|
return nil, errors.New("state capacity exceeded")
|
||||||
|
}
|
||||||
|
return raw, err
|
||||||
|
}
|
||||||
|
|
||||||
|
func readSnapshot(r *os.Root, hostID string, initialized bool) (snapshot, error) {
|
||||||
|
empty := snapshot{Version: 1, HostID: hostID, Operations: make(map[string]Operation)}
|
||||||
|
if err := regularOrMissing(r, "state.json"); err != nil {
|
||||||
|
return snapshot{}, err
|
||||||
|
}
|
||||||
|
f, err := r.Open("state.json")
|
||||||
|
if os.IsNotExist(err) {
|
||||||
|
if initialized {
|
||||||
|
return snapshot{}, errors.New("initialized store has lost its snapshot")
|
||||||
|
}
|
||||||
|
return empty, nil
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return snapshot{}, err
|
||||||
|
}
|
||||||
|
defer f.Close()
|
||||||
|
raw, err := io.ReadAll(io.LimitReader(f, maxSnapshotBytes+1))
|
||||||
|
if err != nil {
|
||||||
|
return snapshot{}, err
|
||||||
|
}
|
||||||
|
if len(raw) > maxSnapshotBytes {
|
||||||
|
return snapshot{}, errors.New("state exceeds size limit")
|
||||||
|
}
|
||||||
|
var s snapshot
|
||||||
|
if err := json.Unmarshal(raw, &s); err != nil {
|
||||||
|
return snapshot{}, errors.New("corrupt state")
|
||||||
|
}
|
||||||
|
if s.Version != 1 || s.HostID != hostID || s.Operations == nil {
|
||||||
|
return snapshot{}, errors.New("incompatible state or host mismatch")
|
||||||
|
}
|
||||||
|
canonical, err := encodeSnapshot(s)
|
||||||
|
if err != nil || !bytes.Equal(raw, canonical) {
|
||||||
|
return snapshot{}, errors.New("state integrity check failed")
|
||||||
|
}
|
||||||
|
unresolved := 0
|
||||||
|
for id, op := range s.Operations {
|
||||||
|
if id != op.ID || !idPattern.MatchString(id) || !hashPattern.MatchString(op.PlanHash) || !op.Status.valid() || op.Revision == 0 {
|
||||||
|
return snapshot{}, errors.New("invalid operation record")
|
||||||
|
}
|
||||||
|
if !op.Status.terminal() {
|
||||||
|
unresolved++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if unresolved > 1 {
|
||||||
|
return snapshot{}, errors.New("multiple unresolved operations")
|
||||||
|
}
|
||||||
|
return s, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func writeSnapshot(r *os.Root, s snapshot) error {
|
||||||
|
raw, err := encodeSnapshot(s)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := regularOrMissing(r, "state.json"); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
var entropy [16]byte
|
||||||
|
if _, err := rand.Read(entropy[:]); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
name := ".state-" + hex.EncodeToString(entropy[:]) + ".tmp"
|
||||||
|
f, err := r.OpenFile(name, os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0600)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
defer r.Remove(name) // Only the unique file just created; never state.json or host.lock.
|
||||||
|
if _, err := f.Write(raw); err != nil {
|
||||||
|
f.Close()
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := f.Sync(); err != nil {
|
||||||
|
f.Close()
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := f.Close(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := r.Rename(name, "state.json"); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return syncDirectory(r)
|
||||||
|
}
|
||||||
@@ -0,0 +1,225 @@
|
|||||||
|
package state
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"sync"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Session holds an exclusive host lock for its entire lifetime. The caller must
|
||||||
|
// retain it throughout any future write operation, not only metadata updates.
|
||||||
|
// Never unlink host.lock: replacing its inode defeats OS locking.
|
||||||
|
type Session struct {
|
||||||
|
mu sync.Mutex
|
||||||
|
root *os.Root
|
||||||
|
lock *os.File
|
||||||
|
data snapshot
|
||||||
|
closed bool
|
||||||
|
poisoned bool
|
||||||
|
initialized bool
|
||||||
|
}
|
||||||
|
|
||||||
|
const initializedMarker = "server-deploy-state-v1\n"
|
||||||
|
|
||||||
|
// Acquire requires an existing administrator-owned LOCAL directory. Bootstrap
|
||||||
|
// owns directory creation/permissions; this function never creates a new root.
|
||||||
|
func Acquire(directory, hostID string) (*Session, error) {
|
||||||
|
if !idPattern.MatchString(hostID) || !filepath.IsAbs(directory) {
|
||||||
|
return nil, errors.New("invalid host or state directory")
|
||||||
|
}
|
||||||
|
r, err := os.OpenRoot(directory)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
fail := func(err error) (*Session, error) { r.Close(); return nil, err }
|
||||||
|
if err := regularOrMissing(r, "host.lock"); err != nil {
|
||||||
|
return fail(err)
|
||||||
|
}
|
||||||
|
lock, err := r.OpenFile("host.lock", os.O_CREATE|os.O_RDWR, 0600)
|
||||||
|
if err != nil {
|
||||||
|
return fail(err)
|
||||||
|
}
|
||||||
|
if err := lockExclusive(lock); err != nil {
|
||||||
|
lock.Close()
|
||||||
|
return fail(err)
|
||||||
|
}
|
||||||
|
s := &Session{root: r, lock: lock}
|
||||||
|
info, err := lock.Stat()
|
||||||
|
if err != nil {
|
||||||
|
s.Close()
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if info.Size() != 0 {
|
||||||
|
if info.Size() != int64(len(initializedMarker)) {
|
||||||
|
s.Close()
|
||||||
|
return nil, errors.New("invalid state initialization marker")
|
||||||
|
}
|
||||||
|
marker := make([]byte, len(initializedMarker))
|
||||||
|
if _, err := lock.ReadAt(marker, 0); err != nil || string(marker) != initializedMarker {
|
||||||
|
s.Close()
|
||||||
|
return nil, errors.New("corrupt state initialization marker")
|
||||||
|
}
|
||||||
|
s.initialized = true
|
||||||
|
}
|
||||||
|
s.data, err = readSnapshot(r, hostID, s.initialized)
|
||||||
|
if err != nil {
|
||||||
|
s.Close()
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
// A crash may happen after the first snapshot rename but before its marker.
|
||||||
|
// Repair only from a fully validated existing snapshot, never from absence.
|
||||||
|
if len(s.data.Operations) > 0 {
|
||||||
|
if err := s.markInitialized(); err != nil {
|
||||||
|
s.Close()
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return s, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Session) markInitialized() error {
|
||||||
|
if s.initialized {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if _, err := s.lock.WriteAt([]byte(initializedMarker), 0); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := s.lock.Sync(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
s.initialized = true
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func regularOrMissing(r *os.Root, name string) error {
|
||||||
|
info, err := r.Lstat(name)
|
||||||
|
if os.IsNotExist(err) {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if !info.Mode().IsRegular() {
|
||||||
|
return errors.New("state path must be a regular file, not a link")
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Session) ready() error {
|
||||||
|
if s.closed {
|
||||||
|
return ErrClosed
|
||||||
|
}
|
||||||
|
if s.poisoned {
|
||||||
|
return ErrPoisoned
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Session) Close() error {
|
||||||
|
s.mu.Lock()
|
||||||
|
defer s.mu.Unlock()
|
||||||
|
if s.closed {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
s.closed = true
|
||||||
|
return errors.Join(s.lock.Close(), s.root.Close())
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Session) Get(id string) (Operation, error) {
|
||||||
|
s.mu.Lock()
|
||||||
|
defer s.mu.Unlock()
|
||||||
|
if err := s.ready(); err != nil {
|
||||||
|
return Operation{}, err
|
||||||
|
}
|
||||||
|
op, ok := s.data.Operations[id]
|
||||||
|
if !ok {
|
||||||
|
return Operation{}, ErrNotFound
|
||||||
|
}
|
||||||
|
return op, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Begin is idempotent, not an execution claim. A repeated ID cannot be rebound
|
||||||
|
// to a different plan, and any unresolved operation blocks unrelated work.
|
||||||
|
func (s *Session) Begin(id, planHash string) (Operation, bool, error) {
|
||||||
|
s.mu.Lock()
|
||||||
|
defer s.mu.Unlock()
|
||||||
|
if err := s.ready(); err != nil {
|
||||||
|
return Operation{}, false, err
|
||||||
|
}
|
||||||
|
if !idPattern.MatchString(id) || !hashPattern.MatchString(planHash) {
|
||||||
|
return Operation{}, false, errors.New("invalid operation identity")
|
||||||
|
}
|
||||||
|
if op, ok := s.data.Operations[id]; ok {
|
||||||
|
if op.PlanHash != planHash {
|
||||||
|
return Operation{}, false, ErrConflict
|
||||||
|
}
|
||||||
|
return op, false, nil
|
||||||
|
}
|
||||||
|
for _, op := range s.data.Operations {
|
||||||
|
if !op.Status.terminal() {
|
||||||
|
return Operation{}, false, ErrUnresolved
|
||||||
|
}
|
||||||
|
}
|
||||||
|
op := Operation{ID: id, PlanHash: planHash, Status: Queued, Revision: 1}
|
||||||
|
if err := s.save(op); err != nil {
|
||||||
|
return Operation{}, false, err
|
||||||
|
}
|
||||||
|
return op, true, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Advance uses a revision check so only one caller can claim queued work.
|
||||||
|
// Reconciliation to a terminal status requires external evidence; this metadata
|
||||||
|
// layer cannot prove that a deployment or restore actually succeeded.
|
||||||
|
func (s *Session) Advance(id string, revision uint64, next Status) (Operation, error) {
|
||||||
|
s.mu.Lock()
|
||||||
|
defer s.mu.Unlock()
|
||||||
|
if err := s.ready(); err != nil {
|
||||||
|
return Operation{}, err
|
||||||
|
}
|
||||||
|
op, ok := s.data.Operations[id]
|
||||||
|
if !ok {
|
||||||
|
return Operation{}, ErrNotFound
|
||||||
|
}
|
||||||
|
if op.Revision != revision {
|
||||||
|
return Operation{}, ErrConflict
|
||||||
|
}
|
||||||
|
if !allowed(op.Status, next) {
|
||||||
|
return Operation{}, ErrTransition
|
||||||
|
}
|
||||||
|
if op.Revision == ^uint64(0) {
|
||||||
|
return Operation{}, ErrConflict
|
||||||
|
}
|
||||||
|
op.Status = next
|
||||||
|
op.Revision++
|
||||||
|
if err := s.save(op); err != nil {
|
||||||
|
return Operation{}, err
|
||||||
|
}
|
||||||
|
return op, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Session) save(op Operation) error {
|
||||||
|
updated := snapshot{Version: 1, HostID: s.data.HostID, Operations: make(map[string]Operation, len(s.data.Operations)+1)}
|
||||||
|
for id, old := range s.data.Operations {
|
||||||
|
updated.Operations[id] = old
|
||||||
|
}
|
||||||
|
updated.Operations[op.ID] = op
|
||||||
|
if op.Revision == 1 {
|
||||||
|
// Only one unresolved operation is admitted. Reserve more than the
|
||||||
|
// longest status growth plus uint64 revision growth (at most 29 bytes).
|
||||||
|
raw, err := encodeSnapshot(updated)
|
||||||
|
if err != nil || len(raw) > maxSnapshotBytes-64 {
|
||||||
|
return ErrCapacity
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := writeSnapshot(s.root, updated); err != nil {
|
||||||
|
s.poisoned = true
|
||||||
|
return errors.Join(ErrPoisoned, err)
|
||||||
|
}
|
||||||
|
if err := s.markInitialized(); err != nil {
|
||||||
|
s.poisoned = true
|
||||||
|
return errors.Join(ErrPoisoned, err)
|
||||||
|
}
|
||||||
|
s.data = updated
|
||||||
|
return nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,134 @@
|
|||||||
|
package state
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bufio"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"os/exec"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
const testHash = "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
|
||||||
|
|
||||||
|
func TestExclusiveSession(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
s, err := Acquire(dir, "host-one")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
t.Cleanup(func() { s.Close() })
|
||||||
|
if second, err := Acquire(dir, "host-one"); second != nil || !errors.Is(err, ErrBusy) {
|
||||||
|
t.Fatalf("lock bypass: %v", err)
|
||||||
|
}
|
||||||
|
other, err := Acquire(t.TempDir(), "host-two")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
other.Close()
|
||||||
|
if err := s.Close(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, _, err := s.Begin("op-one", testHash); !errors.Is(err, ErrClosed) {
|
||||||
|
t.Fatal("used closed session")
|
||||||
|
}
|
||||||
|
next, err := Acquire(dir, "host-one")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
next.Close()
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInvalidHost(t *testing.T) {
|
||||||
|
for _, host := range []string{"", "../host", "Host", strings.Repeat("a", 49)} {
|
||||||
|
if s, err := Acquire(t.TempDir(), host); err == nil {
|
||||||
|
s.Close()
|
||||||
|
t.Fatal("accepted invalid host")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestStatePathsRejectSymlinks(t *testing.T) {
|
||||||
|
for _, name := range []string{"host.lock", "state.json"} {
|
||||||
|
t.Run(name, func(t *testing.T) {
|
||||||
|
dir, outside := t.TempDir(), filepath.Join(t.TempDir(), "outside")
|
||||||
|
if err := os.WriteFile(outside, []byte("protected"), 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.Symlink(outside, filepath.Join(dir, name)); err != nil {
|
||||||
|
t.Skipf("symlink privilege unavailable: %v", err)
|
||||||
|
}
|
||||||
|
if s, err := Acquire(dir, "host-one"); err == nil {
|
||||||
|
s.Close()
|
||||||
|
t.Fatal("accepted symlink state")
|
||||||
|
}
|
||||||
|
content, err := os.ReadFile(outside)
|
||||||
|
if err != nil || string(content) != "protected" {
|
||||||
|
t.Fatal("modified outside file")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The subprocess exits without Close: the OS must release its lock, but its
|
||||||
|
// running operation must remain recorded. No mock can test this boundary.
|
||||||
|
func TestProcessDeathPreservesRunningOperation(t *testing.T) {
|
||||||
|
if os.Getenv("DEPLOYCTL_STATE_CHILD") == "1" {
|
||||||
|
s, err := Acquire(os.Getenv("DEPLOYCTL_STATE_DIR"), "host-one")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
op, _, err := s.Begin("op-one", testHash)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err = s.Advance(op.ID, op.Revision, Running); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
fmt.Println("READY")
|
||||||
|
bufio.NewReader(os.Stdin).ReadByte()
|
||||||
|
os.Exit(3)
|
||||||
|
}
|
||||||
|
dir := t.TempDir()
|
||||||
|
cmd := exec.Command(os.Args[0], "-test.run=^TestProcessDeathPreservesRunningOperation$")
|
||||||
|
cmd.Env = append(os.Environ(), "DEPLOYCTL_STATE_CHILD=1", "DEPLOYCTL_STATE_DIR="+dir)
|
||||||
|
stdin, err := cmd.StdinPipe()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer stdin.Close()
|
||||||
|
stdout, err := cmd.StdoutPipe()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
cmd.Stderr = os.Stderr
|
||||||
|
if err := cmd.Start(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
t.Cleanup(func() { cmd.Process.Kill(); cmd.Wait() })
|
||||||
|
line, err := bufio.NewReader(stdout).ReadString('\n')
|
||||||
|
if err != nil || strings.TrimSpace(line) != "READY" {
|
||||||
|
t.Fatalf("child failed: %q %v", line, err)
|
||||||
|
}
|
||||||
|
if _, err := Acquire(dir, "host-one"); !errors.Is(err, ErrBusy) {
|
||||||
|
t.Fatalf("cross-process lock bypass: %v", err)
|
||||||
|
}
|
||||||
|
if err := cmd.Process.Kill(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
cmd.Wait()
|
||||||
|
s, err := Acquire(dir, "host-one")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer s.Close()
|
||||||
|
op, created, err := s.Begin("op-one", testHash)
|
||||||
|
if err != nil || created || op.Status != Running || op.Revision != 2 {
|
||||||
|
t.Fatalf("lost interrupted task: %+v %v", op, err)
|
||||||
|
}
|
||||||
|
if _, _, err := s.Begin("op-two", testHash); !errors.Is(err, ErrUnresolved) {
|
||||||
|
t.Fatal("allowed writes before reconciliation")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,143 @@
|
|||||||
|
// Package wire enforces the bounded, exact JSON protocol shared by CLI and packages.
|
||||||
|
package wire
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"io"
|
||||||
|
"reflect"
|
||||||
|
"time"
|
||||||
|
"unicode/utf8"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Decode refuses duplicate keys, case aliases, unknown/missing fields,
|
||||||
|
// nulls and trailing values. encoding/json alone accepts several of these.
|
||||||
|
func Decode(in io.Reader, target any, limit int64) error {
|
||||||
|
if limit <= 0 || limit > 16<<20 {
|
||||||
|
return errors.New("invalid input limit")
|
||||||
|
}
|
||||||
|
typ := reflect.TypeOf(target)
|
||||||
|
if typ == nil || typ.Kind() != reflect.Pointer || reflect.ValueOf(target).IsNil() {
|
||||||
|
return errors.New("expected nonnil decode target")
|
||||||
|
}
|
||||||
|
data, err := io.ReadAll(io.LimitReader(in, limit+1))
|
||||||
|
if err != nil || int64(len(data)) > limit || !utf8.Valid(data) {
|
||||||
|
return errors.New("invalid input")
|
||||||
|
}
|
||||||
|
decoder := json.NewDecoder(bytes.NewReader(data))
|
||||||
|
if err := uniqueValue(decoder, 0); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if _, err := decoder.Token(); err != io.EOF {
|
||||||
|
return errors.New("trailing input")
|
||||||
|
}
|
||||||
|
if err := exactFields(data, typ.Elem()); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return json.Unmarshal(data, target)
|
||||||
|
}
|
||||||
|
|
||||||
|
func uniqueValue(d *json.Decoder, depth int) error {
|
||||||
|
if depth > 32 {
|
||||||
|
return errors.New("input nesting exceeds limit")
|
||||||
|
}
|
||||||
|
token, err := d.Token()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
delim, composite := token.(json.Delim)
|
||||||
|
if !composite {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
switch delim {
|
||||||
|
case '{':
|
||||||
|
seen := make(map[string]bool)
|
||||||
|
for d.More() {
|
||||||
|
token, err := d.Token()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
key, ok := token.(string)
|
||||||
|
if !ok || seen[key] {
|
||||||
|
return errors.New("duplicate or invalid field")
|
||||||
|
}
|
||||||
|
seen[key] = true
|
||||||
|
if err := uniqueValue(d, depth+1); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
case '[':
|
||||||
|
for d.More() {
|
||||||
|
if err := uniqueValue(d, depth+1); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
return errors.New("unexpected delimiter")
|
||||||
|
}
|
||||||
|
_, err = d.Token()
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
func exactFields(data []byte, typ reflect.Type) error {
|
||||||
|
if bytes.Equal(bytes.TrimSpace(data), []byte("null")) {
|
||||||
|
return errors.New("null is not permitted")
|
||||||
|
}
|
||||||
|
if typ == reflect.TypeOf(time.Time{}) {
|
||||||
|
var text string
|
||||||
|
if err := json.Unmarshal(data, &text); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
parsed, err := time.Parse(time.RFC3339, text)
|
||||||
|
if err != nil || text != parsed.UTC().Format("2006-01-02T15:04:05Z") {
|
||||||
|
return errors.New("expected canonical UTC timestamp")
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if typ.Kind() == reflect.Slice {
|
||||||
|
var items []json.RawMessage
|
||||||
|
if err := json.Unmarshal(data, &items); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
for _, item := range items {
|
||||||
|
if err := exactFields(item, typ.Elem()); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if typ.Kind() == reflect.Map {
|
||||||
|
var entries map[string]json.RawMessage
|
||||||
|
if err := json.Unmarshal(data, &entries); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
for _, value := range entries {
|
||||||
|
if err := exactFields(value, typ.Elem()); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if typ.Kind() != reflect.Struct {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
var fields map[string]json.RawMessage
|
||||||
|
if err := json.Unmarshal(data, &fields); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if len(fields) != typ.NumField() {
|
||||||
|
return errors.New("unexpected field set")
|
||||||
|
}
|
||||||
|
for n := 0; n < typ.NumField(); n++ {
|
||||||
|
field := typ.Field(n)
|
||||||
|
value, exists := fields[field.Tag.Get("json")]
|
||||||
|
if !exists {
|
||||||
|
return errors.New("missing field")
|
||||||
|
}
|
||||||
|
if err := exactFields(value, field.Type); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,70 @@
|
|||||||
|
package wire
|
||||||
|
|
||||||
|
import (
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
type component struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
}
|
||||||
|
type document struct {
|
||||||
|
Components []component `json:"components"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestStrictNestedArrays(t *testing.T) {
|
||||||
|
var doc document
|
||||||
|
if err := Decode(strings.NewReader(`{"components":[{"name":"server"}]}`), &doc, 1024); err != nil || doc.Components[0].Name != "server" {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
for _, input := range []string{
|
||||||
|
`{"components":[{"name":"server","command":"secret"}]}`,
|
||||||
|
`{"components":[{}]}`,
|
||||||
|
`{"components":[{"Name":"server"}]}`,
|
||||||
|
`{"components":[null]}`,
|
||||||
|
`{"components":null}`,
|
||||||
|
`{"components":[{"name":null}]}`,
|
||||||
|
`{"components":[{"name":"one","name":"two"}]}`,
|
||||||
|
"{\"components\":[{\"name\":\"\xff\"}]}",
|
||||||
|
} {
|
||||||
|
if Decode(strings.NewReader(input), &doc, 1024) == nil {
|
||||||
|
t.Errorf("accepted malformed nested JSON: %q", input)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDecodeBounds(t *testing.T) {
|
||||||
|
var doc document
|
||||||
|
for _, input := range []string{`{"components":[]} {}`, strings.Repeat(" ", 1025) + `{"components":[]}`} {
|
||||||
|
if Decode(strings.NewReader(input), &doc, 1024) == nil {
|
||||||
|
t.Fatal("accepted trailing or oversized input")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInvalidDecoderArguments(t *testing.T) {
|
||||||
|
var doc document
|
||||||
|
for _, limit := range []int64{0, -1, 1<<63 - 1} {
|
||||||
|
if Decode(strings.NewReader(`{}`), &doc, limit) == nil {
|
||||||
|
t.Fatal("accepted invalid limit")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
var nilDoc *document
|
||||||
|
for _, target := range []any{nil, doc, nilDoc} {
|
||||||
|
if Decode(strings.NewReader(`{}`), target, 1024) == nil {
|
||||||
|
t.Fatal("accepted invalid target")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestStrictTypedMaps(t *testing.T) {
|
||||||
|
var target map[string]component
|
||||||
|
for _, input := range []string{`{"api":{"name":"ok","extra":true}}`, `{"api":{"Name":"ok"}}`, `{"api":{}}`, `{"api":null}`} {
|
||||||
|
if Decode(strings.NewReader(input), &target, 1024) == nil {
|
||||||
|
t.Errorf("accepted malformed map entry: %s", input)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := Decode(strings.NewReader(`{"api":{"name":"ok"}}`), &target, 1024); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,191 @@
|
|||||||
|
# deployctl protocol v1 — read-only foundation
|
||||||
|
|
||||||
|
# Supported commands
|
||||||
|
|
||||||
|
The runner supports `version`, `plan`, `verify-plan`, `inspect`, `preflight`, `plan-environment`, `verify-repository`, `verify-artifacts`, `verify-package`
|
||||||
|
and `check-package`.
|
||||||
|
It does not connect to a remote server, invoke Docker, install software, change system
|
||||||
|
configuration, deploy applications, or restore data. `apply`, `upgrade` and `restore`
|
||||||
|
fail closed. `verify-repository` and `verify-artifacts` create private temporary snapshots and invoke
|
||||||
|
local Linux GnuPG; it does not modify APT or persistent keyrings.
|
||||||
|
|
||||||
|
## Build and try
|
||||||
|
|
||||||
|
For staged official Docker metadata authentication, see
|
||||||
|
[verify-repository contract](../internal/aptrepo/README.md). Its output is not
|
||||||
|
accepted as execution authorization by `plan-environment`.
|
||||||
|
|
||||||
|
Requires Go 1.26 or newer. No third-party Go dependencies.
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
go test ./... -count=1
|
||||||
|
go vet ./...
|
||||||
|
go build -o dist/deployctl.exe ./cmd/deployctl
|
||||||
|
Get-Content -Raw protocol/examples/plan-request.json | ./dist/deployctl.exe plan
|
||||||
|
node --test tests/cli-smoke.test.mjs
|
||||||
|
```
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go build -o dist/deployctl ./cmd/deployctl
|
||||||
|
./dist/deployctl plan < protocol/examples/plan-request.json
|
||||||
|
```
|
||||||
|
|
||||||
|
The example hashes are synthetic, not deployable package/image references.
|
||||||
|
|
||||||
|
## Input
|
||||||
|
|
||||||
|
`plan`, `verify-plan`, `verify-package`, `check-package`, `plan-environment`, `verify-repository` and `verify-artifacts` receive one UTF-8 JSON value through
|
||||||
|
stdin, maximum 1 MiB. `version`, `inspect` and `preflight` require no stdin request.
|
||||||
|
Unknown fields, case aliases, duplicate keys, missing fields, nulls, additional
|
||||||
|
JSON values, excessive nesting and malformed types are rejected.
|
||||||
|
Errors go to stderr without echoing input; exit status is nonzero. stdout only
|
||||||
|
contains a JSON response on success (an output transport failure can truncate it).
|
||||||
|
|
||||||
|
`plan` requires exactly the fields in `examples/plan-request.json`:
|
||||||
|
|
||||||
|
- protocolVersion: integer 1.
|
||||||
|
- hostId, instanceId, appId: lower-case ASCII identifiers, first character a-z,
|
||||||
|
remaining characters a-z, 0-9 or hyphen; 1–48 characters.
|
||||||
|
- packageDigest, imageDigest, observedStateDigest: `sha256:` and 64 lowercase hex characters.
|
||||||
|
- domain: explicit lowercase ASCII DNS name, at least two labels, not an IP,
|
||||||
|
wildcard, URL, port or trailing-dot name. IDNs must already be in ASCII form.
|
||||||
|
|
||||||
|
`imageDigest` is a preliminary single-image binding. The deployable package
|
||||||
|
contract will bind all component images (including private databases); this
|
||||||
|
foundation does not claim to represent a complete multi-container release.
|
||||||
|
|
||||||
|
## Plan and verification
|
||||||
|
|
||||||
|
`plan` returns protocolVersion, mode=`offline-preview`, executable=false and plan.
|
||||||
|
The plan contains intent, projectName, dataPath, createdAt, expiresAt and hash.
|
||||||
|
Times are UTC RFC3339 with second precision; the validity window is 15 minutes.
|
||||||
|
Project name is `sd-<instanceId>`, data path is
|
||||||
|
`/var/lib/server-deploy/instances/<instanceId>`. The path is only a preview;
|
||||||
|
symlink-safe filesystem access is a separate executor responsibility.
|
||||||
|
|
||||||
|
Hash is SHA-256 over Go JSON encoding of Plan with an empty hash field, in its
|
||||||
|
declared field order. The CLI is the canonical plan producer; clients transport
|
||||||
|
the returned plan without rebuilding its hash. This is content binding, NOT a
|
||||||
|
signature, authorization, approval token or package trust check.
|
||||||
|
|
||||||
|
`verify-plan` receives exactly `{ "plan": <returned plan>, "current": <intent> }`.
|
||||||
|
It rejects changed identities, state digests, derived resources, hashes, future
|
||||||
|
creation times and expiry (including the exact expiration timestamp).
|
||||||
|
Success returns protocolVersion, mode, valid=true and executable=false.
|
||||||
|
|
||||||
|
Host identity and state digest are caller-supplied offline inputs. Verification
|
||||||
|
does NOT prove anything about a real server. Future apply must independently
|
||||||
|
inspect the host under the host lock, resolve the entire trusted package and
|
||||||
|
revalidate it; offline preview is never permission to deploy.
|
||||||
|
|
||||||
|
## Local prerequisite inspection
|
||||||
|
|
||||||
|
`inspect` takes no stdin request and examines the machine where deployctl runs.
|
||||||
|
It reports OS/architecture, the presence of `/run/systemd/system`, and an executable
|
||||||
|
regular Docker client at `/usr/bin/docker` or `/usr/local/bin/docker`. It does not
|
||||||
|
run that executable or read Docker configuration/credentials. Access failures
|
||||||
|
appear as `unknown`, not `missing`. Non-Linux targets do not probe Linux paths.
|
||||||
|
|
||||||
|
`supportedPlatform` only describes the initial Linux amd64/arm64 runtime family,
|
||||||
|
not distro certification. `deploymentReady` is always false in this foundation.
|
||||||
|
Daemon connectivity, Compose version, host identity, distro acceptance, permissions,
|
||||||
|
disk space, ports, DNS, firewall and gateway validation are listed as unchecked.
|
||||||
|
This report is not the authoritative observed state required by a future apply.
|
||||||
|
|
||||||
|
### Environment proposal
|
||||||
|
|
||||||
|
`preflight` adds local Ubuntu release, effective privilege, Linux `/var/lib`
|
||||||
|
filesystem available bytes and conservative existing-resource observations.
|
||||||
|
Output includes protocolVersion=1, mode=`local-environment-proposal`, observedAt,
|
||||||
|
report and proposal. It does not accept caller-supplied observations over stdin.
|
||||||
|
|
||||||
|
The proposal reports blockers, candidate changes and impacts; executable is always
|
||||||
|
false. Existing/unknown resources are never silently adopted or removed. Candidate
|
||||||
|
steps, when present, are not an executable installation plan: package inventory,
|
||||||
|
version locks, repository trust, host identity and network checks remain unresolved.
|
||||||
|
Exit 0 means the report was produced, not that the host is ready. See the
|
||||||
|
[preflight contract](../internal/preflight/README.md) for exact boundaries.
|
||||||
|
|
||||||
|
### Version-locked environment draft
|
||||||
|
|
||||||
|
`plan-environment` receives `{ "lock": <Docker package lock> }`. See the
|
||||||
|
[lock contract](../internal/installplan/README.md) for all required fields.
|
||||||
|
Local observations are collected by the CLI, not supplied by the caller.
|
||||||
|
Output includes protocolVersion=1, mode=`local-environment-draft`, observedAt,
|
||||||
|
report and draft. The draft includes requestedPackages, lockDigest,
|
||||||
|
observationDigest, impacts, blockers, executable=false and
|
||||||
|
repositoryAuthenticated=false. No shell command, download or installation runs.
|
||||||
|
|
||||||
|
Preflight now includes a local dpkg inventory observation. Relevant existing or
|
||||||
|
residual packages prevent fresh-install candidates; unknown status/journal cannot
|
||||||
|
be treated as absence. The lock is only syntactically pinned to an allowed source:
|
||||||
|
repository signature, signed metadata chain, package bytes, freshness and dependency
|
||||||
|
resolution remain blockers. This is not a complete or authorized APT transaction.
|
||||||
|
|
||||||
|
## Local application package verification
|
||||||
|
|
||||||
|
`verify-package` reads `{ "directory": "<absolute package directory>",
|
||||||
|
"expectedDigest": "sha256:<64 lowercase hex>" }` from stdin.
|
||||||
|
The expected digest pins the exact manifest.json bytes; the manifest pins every
|
||||||
|
payload file and every component's image reference. Obtain the expected digest
|
||||||
|
through a trusted channel, not from the untrusted package being checked.
|
||||||
|
|
||||||
|
The output includes the verified manifest, manifest digest, payload file count,
|
||||||
|
verified=true, executable=false and publisherAuthenticated=false. File contents
|
||||||
|
are never echoed. Integrity is not publisher authentication, Compose-policy
|
||||||
|
validation, container-image availability, backup compatibility or deployment safety.
|
||||||
|
|
||||||
|
Packages must be immutable, trusted staging directories during verification.
|
||||||
|
The internal verifier returns the verified payload snapshot for future consumers;
|
||||||
|
execution must not reopen mutable source files after a check. No package extraction,
|
||||||
|
download, signature trust-store management or image pull is implemented yet.
|
||||||
|
|
||||||
|
### Manifest contract
|
||||||
|
|
||||||
|
Every field is required; unknown fields are rejected:
|
||||||
|
|
||||||
|
- `protocolVersion`: 1; `runtime`: `compose`.
|
||||||
|
- `appId`: the same lowercase identifier rules as the offline intent.
|
||||||
|
- `version`: three numeric components separated by dots (no prerelease suffix).
|
||||||
|
- `entrypoint`: the path of one declared payload file.
|
||||||
|
- `platforms`: a nonempty unique subset of `linux/amd64`, `linux/arm64`.
|
||||||
|
- `components`: 1–32 unique `{name, image}` entries. Names use identifier rules;
|
||||||
|
images use lowercase repository paths followed by `@sha256:<64 lowercase hex>`.
|
||||||
|
This initial lexical subset excludes tags and registry ports; it is not a
|
||||||
|
complete Docker reference parser or a check against the Compose services.
|
||||||
|
- `files`: 1–128 unique `{path, digest}` entries. Digests pin payload bytes;
|
||||||
|
`manifest.json` itself must not appear in this list.
|
||||||
|
|
||||||
|
Paths are lowercase portable relative slash paths, at most 240 bytes overall and
|
||||||
|
100 bytes per segment. Segments begin with ASCII alphanumeric characters and use
|
||||||
|
only ASCII alphanumeric, dot, underscore or hyphen. Dotfiles, traversal, backslashes,
|
||||||
|
colons, Windows device names and trailing dots are rejected. The directory must
|
||||||
|
contain only `manifest.json`, listed payload files and their required parents.
|
||||||
|
|
||||||
|
Limits are 1 MiB for the manifest, 4 MiB per payload and 16 MiB for total payloads.
|
||||||
|
Symlinks and special files are rejected. See `internal/appbundle/README.md` for
|
||||||
|
filesystem trust boundaries and Linux-specific rejection tests.
|
||||||
|
|
||||||
|
## Restricted Compose policy check
|
||||||
|
|
||||||
|
`check-package` accepts the same request as `verify-package`. It performs the
|
||||||
|
complete integrity check first, then validates the returned entrypoint bytes
|
||||||
|
against `isolated-compose-v1`. No files are reopened and no Docker command runs.
|
||||||
|
|
||||||
|
Success returns protocolVersion=1, policyPassed=true, profile, the manifest digest,
|
||||||
|
executable=false and publisherAuthenticated=false. Failure returns no JSON result
|
||||||
|
and a fixed redacted diagnostic. `verify-package` remains integrity-only; a package
|
||||||
|
may pass that command while failing `check-package`.
|
||||||
|
|
||||||
|
The initial profile accepts only strict JSON, pinned manifest-matching services,
|
||||||
|
explicit non-root identities, read-only roots, dropped capabilities, no new
|
||||||
|
privileges, a private internal backend network and plain project-local named
|
||||||
|
volumes. Every field in the profile is required; other fields are rejected.
|
||||||
|
See [exact shape and limitations](../internal/composepolicy/README.md).
|
||||||
|
|
||||||
|
This is not a general YAML validator or completed application adapter. Traefik
|
||||||
|
routes, environment/secrets, application-specific permissions, Docker version
|
||||||
|
compatibility, volume ownership and resource limits are not implemented here.
|
||||||
|
Passing never establishes that an application can start or its data can be safely
|
||||||
|
upgraded/restored. Future execution still requires locked host/resource checks,
|
||||||
|
trusted images, a fixed Compose invocation and exact snapshot binding.
|
||||||
@@ -0,0 +1,10 @@
|
|||||||
|
{
|
||||||
|
"protocolVersion": 1,
|
||||||
|
"hostId": "example-host",
|
||||||
|
"instanceId": "git-one",
|
||||||
|
"appId": "gitea",
|
||||||
|
"packageDigest": "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
||||||
|
"imageDigest": "sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
|
||||||
|
"domain": "git.example.com",
|
||||||
|
"observedStateDigest": "sha256:cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc"
|
||||||
|
}
|
||||||
@@ -0,0 +1,32 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Local opt-in public deb download/hash test. No extraction, APT or installation.
|
||||||
|
# Called by probe-docker-repository.sh after authenticated metadata verification.
|
||||||
|
set -euo pipefail
|
||||||
|
[[ $# == 3 && -f "$1" && -d "$2" ]]
|
||||||
|
binary=$1
|
||||||
|
metadata_dir=$2
|
||||||
|
metadata_request=$3
|
||||||
|
probe_dir=$(mktemp -d /tmp/server-deploy-artifacts.XXXXXX)
|
||||||
|
chmod 700 "$probe_dir"
|
||||||
|
artifact_dir="$probe_dir/debs"
|
||||||
|
mkdir -m 700 "$artifact_dir"
|
||||||
|
# Only authenticated lock paths, fixed origin, no caller-provided download URLs.
|
||||||
|
python3 -I -c 'import json,re,sys; r=json.load(sys.stdin); assert r["repositoryAuthenticated"] is True; p=r["lock"]["packages"]; assert len(p)==5; [None if re.fullmatch(r"dists/resolute/pool/stable/amd64/[a-z0-9.+~_-]+\.deb",x["filename"]) and 0<x["size"]<=536870912 else sys.exit(1) for x in p]; [print(x["filename"],x["size"]) for x in p]' < "$metadata_dir/result.json" > "$probe_dir/downloads"
|
||||||
|
while read -r filename size; do
|
||||||
|
curl --fail --silent --show-error --proto '=https' --max-time 90 --max-filesize "$size" \
|
||||||
|
"https://download.docker.com/linux/ubuntu/$filename" -o "$artifact_dir/${filename##*/}"
|
||||||
|
done < "$probe_dir/downloads"
|
||||||
|
request=$(printf '%s\n' "$metadata_request" | python3 -I -c 'import json,sys; r=json.load(sys.stdin); r["artifactDirectory"]=sys.argv[1]; print(json.dumps(r))' "$artifact_dir")
|
||||||
|
printf '%s\n' "$request" | "$binary" verify-artifacts > "$probe_dir/result.json"
|
||||||
|
python3 -I -c 'import json,sys; r=json.load(sys.stdin); assert r["repositoryAuthenticated"] is True; assert r["packageBytesVerified"] is True; assert r["executable"] is False; assert len(r["lock"]["packages"])==5' < "$probe_dir/result.json"
|
||||||
|
cat "$probe_dir/result.json"
|
||||||
|
# Same-size mutation: digest comparison, not an earlier size check, must reject.
|
||||||
|
first=$(head -n 1 "$probe_dir/downloads")
|
||||||
|
filename=${first% *}
|
||||||
|
printf 'X' | dd of="$artifact_dir/${filename##*/}" bs=1 count=1 conv=notrunc status=none
|
||||||
|
if printf '%s\n' "$request" | "$binary" verify-artifacts > "$probe_dir/rejected.json"; then
|
||||||
|
printf 'ERROR: same-size deb tamper accepted\n' >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
[[ ! -s "$probe_dir/rejected.json" ]]
|
||||||
|
printf 'Five real deb files verified; same-size tamper rejected. Local artifacts: %s\n' "$artifact_dir"
|
||||||
@@ -0,0 +1,55 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Local metadata-only probe. No APT changes or software installation.
|
||||||
|
set -euo pipefail
|
||||||
|
task_dir=$(mktemp -d /tmp/server-deploy-repository.XXXXXX)
|
||||||
|
chmod 700 "$task_dir"
|
||||||
|
base=https://download.docker.com/linux/ubuntu
|
||||||
|
curl --fail --silent --show-error --proto '=https' --max-time 45 --max-filesize 1048576 "$base/gpg" -o "$task_dir/docker.asc"
|
||||||
|
curl --fail --silent --show-error --proto '=https' --max-time 45 --max-filesize 1048576 "$base/dists/resolute/Release" -o "$task_dir/Release"
|
||||||
|
curl --fail --silent --show-error --proto '=https' --max-time 45 --max-filesize 65536 "$base/dists/resolute/Release.gpg" -o "$task_dir/Release.gpg"
|
||||||
|
curl --fail --silent --show-error --proto '=https' --max-time 45 --max-filesize 16777216 "$base/dists/resolute/stable/binary-amd64/Packages" -o "$task_dir/Packages"
|
||||||
|
gpg --batch --no-options --homedir "$task_dir" --with-colons --show-keys "$task_dir/docker.asc"
|
||||||
|
sha256sum "$task_dir/docker.asc"
|
||||||
|
gpg --batch --no-options --homedir "$task_dir" --dearmor --output "$task_dir/docker.gpg" "$task_dir/docker.asc"
|
||||||
|
gpgv --homedir "$task_dir" --keyring "$task_dir/docker.gpg" --status-fd 1 "$task_dir/Release.gpg" "$task_dir/Release"
|
||||||
|
head -n 12 "$task_dir/Release"
|
||||||
|
awk '/^Package: (docker-ce|docker-ce-cli|containerd.io|docker-buildx-plugin|docker-compose-plugin)$/ {name=$2} /^Version:/ && name!="" {print name, $2; name=""}' "$task_dir/Packages"
|
||||||
|
# Optional prebuilt local binary: verifies the same downloaded snapshots. This
|
||||||
|
# selects explicit fixture versions, not an installation or latest-version policy.
|
||||||
|
if [[ $# == 1 ]]; then
|
||||||
|
engine=$(awk '/^Package: docker-ce$/ {p=1} p && /^Version:/ {print $2; exit}' "$task_dir/Packages")
|
||||||
|
containerd=$(awk '/^Package: containerd.io$/ {p=1} p && /^Version:/ {print $2; exit}' "$task_dir/Packages")
|
||||||
|
buildx=$(awk '/^Package: docker-buildx-plugin$/ {p=1} p && /^Version:/ {print $2; exit}' "$task_dir/Packages")
|
||||||
|
compose=$(awk '/^Package: docker-compose-plugin$/ {p=1} p && /^Version:/ {print $2; exit}' "$task_dir/Packages")
|
||||||
|
for version in "$engine" "$containerd" "$buildx" "$compose"; do
|
||||||
|
[[ "$version" =~ ^[0-9A-Za-z.+:~_-]+$ ]] || exit 1
|
||||||
|
done
|
||||||
|
request=$(printf '{"directory":"%s","suite":"resolute","architecture":"amd64","versions":{"docker-ce":"%s","docker-ce-cli":"%s","containerd.io":"%s","docker-buildx-plugin":"%s","docker-compose-plugin":"%s"}}' "$task_dir" "$engine" "$engine" "$containerd" "$buildx" "$compose")
|
||||||
|
printf '%s\n' "$request" | "$1" verify-repository > "$task_dir/result.json"
|
||||||
|
python3 -I -c 'import json,sys; r=json.load(sys.stdin); p=sys.argv[1:]; assert r["repositoryAuthenticated"] is True; assert r["executable"] is False; assert r["packageBytesVerified"] is False; assert {x["name"]:x["version"] for x in r["lock"]["packages"]} == dict(zip(["docker-ce","docker-ce-cli","containerd.io","docker-buildx-plugin","docker-compose-plugin"],p)); assert len(r["lock"]["packages"])==5' "$engine" "$engine" "$containerd" "$buildx" "$compose" < "$task_dir/result.json"
|
||||||
|
cat "$task_dir/result.json"
|
||||||
|
if [[ -n "${DEPLOYCTL_TEST_GO:-}" ]]; then
|
||||||
|
DEPLOYCTL_REPOSITORY_FIXTURE="$task_dir" "$DEPLOYCTL_TEST_GO" test ./internal/aptrepo -run TestStagedSignatureIntegration -count=1 -v
|
||||||
|
fi
|
||||||
|
if [[ "${DEPLOYCTL_ONLINE_ARTIFACT_PROBE:-0}" == 1 ]]; then
|
||||||
|
bash scripts/probe-docker-artifacts.sh "$1" "$task_dir" "$request"
|
||||||
|
fi
|
||||||
|
cp -- "$task_dir/Packages" "$task_dir/Packages.original"
|
||||||
|
awk '/^SHA256:/ && !changed {c=substr($2,1,1); $2=(c=="0" ? "1" : "0") substr($2,2); changed=1} {print}' "$task_dir/Packages.original" > "$task_dir/Packages"
|
||||||
|
if printf '%s\n' "$request" | "$1" verify-repository > "$task_dir/rejected.json"; then
|
||||||
|
printf 'ERROR: tampered Packages accepted\n' >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
[[ ! -s "$task_dir/rejected.json" ]]
|
||||||
|
mv -- "$task_dir/Packages.original" "$task_dir/Packages"
|
||||||
|
printf 'Tampered Packages rejected as expected.\n'
|
||||||
|
cp -- "$task_dir/Release" "$task_dir/Release.original"
|
||||||
|
awk '/^Date:/ {n=split($0,parts,":"); c=substr(parts[n],1,1); parts[n]=(c=="0" ? "1" : "0") substr(parts[n],2); line=parts[1]; for(i=2;i<=n;i++) line=line ":" parts[i]; print line; next} {print}' "$task_dir/Release.original" > "$task_dir/Release"
|
||||||
|
if printf '%s\n' "$request" | "$1" verify-repository > "$task_dir/rejected.json"; then
|
||||||
|
printf 'ERROR: tampered Release accepted\n' >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
[[ ! -s "$task_dir/rejected.json" ]]
|
||||||
|
printf 'Tampered signed Release rejected as expected.\n'
|
||||||
|
fi
|
||||||
|
printf 'Local temporary probe directory: %s\n' "$task_dir"
|
||||||
@@ -0,0 +1,23 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Run on Linux/WSL with a previously SHA-256-verified official Go tarball.
|
||||||
|
# The caller owns download provenance. No package installation or production SSH.
|
||||||
|
set -euo pipefail
|
||||||
|
if [[ $# != 1 || ! -f "$1" ]]; then
|
||||||
|
printf 'usage: bash scripts/verify-linux.sh /path/to/verified-go.linux-amd64.tar.gz\n' >&2
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
task_dir=$(mktemp -d /tmp/server-deploy-test.XXXXXX)
|
||||||
|
tar -xzf "$1" -C "$task_dir"
|
||||||
|
export GOCACHE="$task_dir/cache"
|
||||||
|
cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.."
|
||||||
|
"$task_dir/go/bin/go" test ./... -count=1 -v -timeout=60s
|
||||||
|
"$task_dir/go/bin/go" vet ./...
|
||||||
|
"$task_dir/go/bin/go" test -race ./... -count=1 -timeout=60s
|
||||||
|
"$task_dir/go/bin/go" build -o "$task_dir/deployctl" ./cmd/deployctl
|
||||||
|
"$task_dir/deployctl" plan < protocol/examples/plan-request.json
|
||||||
|
"$task_dir/deployctl" inspect
|
||||||
|
"$task_dir/deployctl" preflight
|
||||||
|
if [[ "${DEPLOYCTL_ONLINE_REPOSITORY_PROBE:-0}" == 1 ]]; then
|
||||||
|
DEPLOYCTL_TEST_GO="$task_dir/go/bin/go" bash scripts/probe-docker-repository.sh "$task_dir/deployctl"
|
||||||
|
fi
|
||||||
|
printf 'Verification artifacts retained at: %s\n' "$task_dir"
|
||||||
@@ -0,0 +1,195 @@
|
|||||||
|
// Run after building the native executable. No npm packages are required.
|
||||||
|
import { test } from 'node:test';
|
||||||
|
import assert from 'node:assert/strict';
|
||||||
|
import { spawnSync } from 'node:child_process';
|
||||||
|
import { readFileSync, writeFileSync, mkdtempSync, readdirSync, rmSync } from 'node:fs';
|
||||||
|
import { createHash } from 'node:crypto';
|
||||||
|
import { tmpdir } from 'node:os';
|
||||||
|
import { resolve, join } from 'node:path';
|
||||||
|
|
||||||
|
const binary = resolve(process.env.DEPLOYCTL_BIN ?? (process.platform === 'win32' ? 'dist/deployctl.exe' : 'dist/deployctl'));
|
||||||
|
const intent = JSON.parse(readFileSync(new URL('../protocol/examples/plan-request.json', import.meta.url), 'utf8'));
|
||||||
|
|
||||||
|
test('real CLI artifact verification rejects missing metadata without writes', () => {
|
||||||
|
const directory = mkdtempSync(join(tmpdir(), 'deployctl-artifact-smoke-'));
|
||||||
|
try {
|
||||||
|
const result = spawnSync(binary, ['verify-artifacts'], {
|
||||||
|
encoding:'utf8', timeout:10000,
|
||||||
|
input:JSON.stringify({directory, artifactDirectory:directory, suite:'resolute', architecture:'amd64', versions:{}}),
|
||||||
|
});
|
||||||
|
assert.notEqual(result.status, 0);
|
||||||
|
assert.equal(result.stdout, '');
|
||||||
|
assert.equal(result.stderr.includes('unsupported command'), false);
|
||||||
|
assert.equal(result.stderr.includes(directory), false);
|
||||||
|
assert.deepEqual(readdirSync(directory), []);
|
||||||
|
} finally {
|
||||||
|
rmSync(directory, {recursive:true});
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test('real CLI rejects caller-asserted repository authentication', () => {
|
||||||
|
const result = spawnSync(binary, ['verify-repository'], {
|
||||||
|
encoding: 'utf8', timeout: 10000,
|
||||||
|
input: JSON.stringify({directory:'secret-relative', suite:'resolute', architecture:'amd64',
|
||||||
|
versions:{}, repositoryAuthenticated:true}),
|
||||||
|
});
|
||||||
|
assert.notEqual(result.status, 0);
|
||||||
|
assert.equal(result.stdout, '');
|
||||||
|
assert.equal(result.stderr.includes('secret-relative'), false);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('real CLI environment draft rejects malformed locks without writing', () => {
|
||||||
|
const directory = mkdtempSync(join(tmpdir(), 'deployctl-environment-smoke-'));
|
||||||
|
try {
|
||||||
|
const probe = spawnSync(binary, ['preflight'], { encoding: 'utf8', timeout: 10000 });
|
||||||
|
assert.equal(probe.status, 0, probe.stderr);
|
||||||
|
const report = JSON.parse(probe.stdout).report;
|
||||||
|
const suite = report.runtime.os === 'linux' ? report.distribution.codename : 'resolute';
|
||||||
|
const arch = report.runtime.os === 'linux' ? report.runtime.architecture : 'amd64';
|
||||||
|
const lock = { protocolVersion: 1, repository: 'https://download.docker.com/linux/ubuntu',
|
||||||
|
suite, architecture: arch, releaseDigest: `sha256:${'a'.repeat(64)}`,
|
||||||
|
packages: ['docker-ce', 'docker-ce-cli', 'containerd.io', 'docker-buildx-plugin', 'docker-compose-plugin'].map(name => ({
|
||||||
|
name, version: '1.2.3-1', filename: `dists/${suite}/pool/stable/${arch}/${name}_1.2.3-1_${arch}.deb`, digest: `sha256:${'b'.repeat(64)}`, size: 123,
|
||||||
|
})),
|
||||||
|
};
|
||||||
|
const run = () => spawnSync(binary, ['plan-environment'], { cwd: directory, encoding: 'utf8', timeout: 10000, input: JSON.stringify({lock}) });
|
||||||
|
const result = run();
|
||||||
|
assert.equal(result.status, 0, result.stderr);
|
||||||
|
const draft = JSON.parse(result.stdout).draft;
|
||||||
|
assert.equal(draft.executable, false);
|
||||||
|
assert.equal(draft.repositoryAuthenticated, false);
|
||||||
|
assert.ok(draft.blockers.includes('dependency_transaction_unresolved'));
|
||||||
|
assert.equal(draft.requestedPackages.length, 5);
|
||||||
|
lock.packages[0].version = 'secret;reboot';
|
||||||
|
const invalid = run();
|
||||||
|
assert.notEqual(invalid.status, 0);
|
||||||
|
assert.equal(invalid.stdout, '');
|
||||||
|
assert.equal(invalid.stderr.includes('secret'), false);
|
||||||
|
assert.deepEqual(readdirSync(directory), []);
|
||||||
|
} finally {
|
||||||
|
rmSync(directory, { recursive: true });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test('real CLI preflight reports blockers without creating files', () => {
|
||||||
|
const directory = mkdtempSync(join(tmpdir(), 'deployctl-preflight-smoke-'));
|
||||||
|
try {
|
||||||
|
const result = spawnSync(binary, ['preflight'], { cwd: directory, encoding: 'utf8', timeout: 10000 });
|
||||||
|
assert.equal(result.status, 0, result.stderr);
|
||||||
|
const response = JSON.parse(result.stdout);
|
||||||
|
assert.equal(response.mode, 'local-environment-proposal');
|
||||||
|
assert.equal(response.proposal.executable, false);
|
||||||
|
assert.ok(response.proposal.blockers.includes('package_versions_unresolved'));
|
||||||
|
assert.ok(response.proposal.blockers.includes('host_identity_unverified'));
|
||||||
|
assert.deepEqual(readdirSync(directory), []);
|
||||||
|
} finally {
|
||||||
|
rmSync(directory, { recursive: true });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test('real CLI previews and verifies without writing into its working directory', () => {
|
||||||
|
const directory = mkdtempSync(join(tmpdir(), 'deployctl-smoke-'));
|
||||||
|
const run = (command, input) => spawnSync(binary, [command], {
|
||||||
|
input: JSON.stringify(input), encoding: 'utf8', cwd: directory, timeout: 10000,
|
||||||
|
});
|
||||||
|
try {
|
||||||
|
const preview = run('plan', intent);
|
||||||
|
assert.equal(preview.error, undefined);
|
||||||
|
assert.equal(preview.status, 0, preview.stderr);
|
||||||
|
const response = JSON.parse(preview.stdout);
|
||||||
|
assert.equal(response.executable, false);
|
||||||
|
assert.equal(response.mode, 'offline-preview');
|
||||||
|
assert.equal(response.plan.projectName, 'sd-git-one');
|
||||||
|
const check = run('verify-plan', { plan: response.plan, current: intent });
|
||||||
|
assert.equal(check.status, 0, check.stderr);
|
||||||
|
assert.equal(JSON.parse(check.stdout).valid, true);
|
||||||
|
const changed = run('verify-plan', { plan: response.plan, current: { ...intent, hostId: 'different-host' } });
|
||||||
|
assert.notEqual(changed.status, 0);
|
||||||
|
assert.equal(changed.stdout, '');
|
||||||
|
const apply = run('apply', response.plan);
|
||||||
|
assert.notEqual(apply.status, 0);
|
||||||
|
assert.equal(apply.stdout, '');
|
||||||
|
assert.deepEqual(readdirSync(directory), []);
|
||||||
|
} finally {
|
||||||
|
// Only remove the unique test directory this test just created.
|
||||||
|
rmSync(directory, { recursive: true });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test('real CLI inspects locally and verifies packages without claiming execution safety', () => {
|
||||||
|
const inspection = spawnSync(binary, ['inspect'], { encoding: 'utf8', timeout: 10000 });
|
||||||
|
assert.equal(inspection.status, 0, inspection.stderr);
|
||||||
|
assert.equal(JSON.parse(inspection.stdout).deploymentReady, false);
|
||||||
|
const directory = mkdtempSync(join(tmpdir(), 'deployctl-package-smoke-'));
|
||||||
|
const digest = bytes => `sha256:${createHash('sha256').update(bytes).digest('hex')}`;
|
||||||
|
try {
|
||||||
|
const payload = 'services: {}\n';
|
||||||
|
const manifest = JSON.stringify({
|
||||||
|
protocolVersion: 1, appId: 'example', version: '1.0.0', runtime: 'compose',
|
||||||
|
entrypoint: 'compose.yaml', platforms: ['linux/amd64'],
|
||||||
|
components: [{ name: 'server', image: `example/server@sha256:${'a'.repeat(64)}` }],
|
||||||
|
files: [{ path: 'compose.yaml', digest: digest(payload) }],
|
||||||
|
});
|
||||||
|
writeFileSync(join(directory, 'manifest.json'), manifest);
|
||||||
|
writeFileSync(join(directory, 'compose.yaml'), payload);
|
||||||
|
const result = spawnSync(binary, ['verify-package'], {
|
||||||
|
encoding: 'utf8', timeout: 10000,
|
||||||
|
input: JSON.stringify({ directory, expectedDigest: digest(manifest) }),
|
||||||
|
});
|
||||||
|
assert.equal(result.status, 0, result.stderr);
|
||||||
|
const verified = JSON.parse(result.stdout);
|
||||||
|
assert.equal(verified.verified, true);
|
||||||
|
assert.equal(verified.executable, false);
|
||||||
|
assert.equal(verified.publisherAuthenticated, false);
|
||||||
|
assert.deepEqual(readdirSync(directory).sort(), ['compose.yaml', 'manifest.json']);
|
||||||
|
assert.equal(readFileSync(join(directory, 'compose.yaml'), 'utf8'), payload);
|
||||||
|
} finally {
|
||||||
|
rmSync(directory, { recursive: true });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test('real CLI separates package integrity from restricted policy and rejects tampering', () => {
|
||||||
|
const directory = mkdtempSync(join(tmpdir(), 'deployctl-policy-smoke-'));
|
||||||
|
const digest = bytes => `sha256:${createHash('sha256').update(bytes).digest('hex')}`;
|
||||||
|
const image = `example/server@sha256:${'a'.repeat(64)}`;
|
||||||
|
const document = {
|
||||||
|
services: { server: { image, user: '1000:1000', read_only: true, cap_drop: ['ALL'],
|
||||||
|
security_opt: ['no-new-privileges:true'], restart: 'no', networks: ['backend'], volumes: [] } },
|
||||||
|
networks: { backend: { internal: true } }, volumes: {},
|
||||||
|
};
|
||||||
|
const run = (command, expectedDigest) => spawnSync(binary, [command], {
|
||||||
|
encoding: 'utf8', timeout: 10000, cwd: directory,
|
||||||
|
input: JSON.stringify({ directory, expectedDigest }),
|
||||||
|
});
|
||||||
|
const writePackage = () => {
|
||||||
|
const payload = JSON.stringify(document);
|
||||||
|
const manifest = JSON.stringify({ protocolVersion: 1, appId: 'example', version: '1.0.0', runtime: 'compose',
|
||||||
|
entrypoint: 'compose.json', platforms: ['linux/amd64'], components: [{ name: 'server', image }],
|
||||||
|
files: [{ path: 'compose.json', digest: digest(payload) }],
|
||||||
|
});
|
||||||
|
writeFileSync(join(directory, 'manifest.json'), manifest);
|
||||||
|
writeFileSync(join(directory, 'compose.json'), payload);
|
||||||
|
return digest(manifest);
|
||||||
|
};
|
||||||
|
try {
|
||||||
|
const pin = writePackage();
|
||||||
|
const check = run('check-package', pin);
|
||||||
|
assert.equal(check.status, 0, check.stderr);
|
||||||
|
const result = JSON.parse(check.stdout);
|
||||||
|
assert.equal(result.policyPassed, true);
|
||||||
|
assert.equal(result.executable, false);
|
||||||
|
assert.equal(result.publisherAuthenticated, false);
|
||||||
|
assert.equal(result.digest, pin);
|
||||||
|
writeFileSync(join(directory, 'compose.json'), '{}');
|
||||||
|
assert.notEqual(run('check-package', pin).status, 0);
|
||||||
|
document.services.server.privileged = true;
|
||||||
|
const unsafePin = writePackage();
|
||||||
|
assert.equal(run('verify-package', unsafePin).status, 0);
|
||||||
|
const rejected = run('check-package', unsafePin);
|
||||||
|
assert.notEqual(rejected.status, 0);
|
||||||
|
assert.equal(rejected.stdout, '');
|
||||||
|
assert.deepEqual(readdirSync(directory).sort(), ['compose.json', 'manifest.json']);
|
||||||
|
} finally {
|
||||||
|
rmSync(directory, { recursive: true });
|
||||||
|
}
|
||||||
|
});
|
||||||
Reference in New Issue
Block a user