Compare commits
45
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
f25f5df62d | ||
|
|
e965b0943d | ||
|
|
8ccb8b7c15 | ||
|
|
889d40299e | ||
|
|
95a7df0e29 | ||
|
|
b1473ca956 | ||
|
|
efb793c2f5 | ||
|
|
7504af2644 | ||
|
|
76a2332c9a | ||
|
|
a7154212e4 | ||
|
|
4264075d96 | ||
|
|
d9b19b404a | ||
|
|
3fd2c89020 | ||
|
|
0dc80ff144 | ||
|
|
34f698e5fb | ||
|
|
9f1e293d4c | ||
|
|
682b79852d | ||
|
|
22d1aa6a93 | ||
|
|
354ac8a58c | ||
|
|
d19d337bf2 | ||
|
|
dfcaf7e4d0 | ||
|
|
39b0aee2c9 | ||
|
|
920774d7d7 | ||
|
|
29236e8e1c | ||
|
|
938df26cbb | ||
|
|
9be05edc9a | ||
|
|
e8cc0162af | ||
|
|
5195986fc4 | ||
|
|
2afc2cade6 | ||
|
|
81928c6d33 | ||
|
|
56343db806 | ||
|
|
1bfd657f99 | ||
|
|
99e9ec1434 | ||
|
|
1b19054da1 | ||
|
|
353d3940a9 | ||
|
|
bffa517ed4 | ||
|
|
80ba4b046c | ||
|
|
be82a4eb70 | ||
|
|
b094cb85cd | ||
|
|
dd3eb24d0f | ||
|
|
e8693dad2a | ||
|
|
8ceccd75f1 | ||
|
|
18e24d40f0 | ||
|
|
a550a2675e | ||
|
|
afbbb031d3 |
+25
@@ -0,0 +1,25 @@
|
||||
# 环境变量 / 凭据(.env.example 模板除外)
|
||||
.env
|
||||
*.env
|
||||
!.env.example
|
||||
!*.env.example
|
||||
|
||||
# vps-xray:部署脚本生成的真实节点配置,含 UUID / 私钥 / 服务器 IP
|
||||
# vps-xray/client-config/*-info*.md
|
||||
# vps-xray/client-config/*.yaml
|
||||
# vps-xray/client-config/*.yml
|
||||
|
||||
# 备份产物
|
||||
*.bak
|
||||
*.bak.*
|
||||
backups/
|
||||
|
||||
# 系统 / 编辑器
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
*.swp
|
||||
|
||||
# New architecture build outputs
|
||||
/dist/
|
||||
coverage.out
|
||||
node_modules/
|
||||
+95
@@ -0,0 +1,95 @@
|
||||
# 跨设备接续入口
|
||||
|
||||
更新时间:2026-09-25。此文件用于在没有原聊天记录、没有原设备缓存的情况下接续开发。
|
||||
这是开发交接,不是生产安装说明。新设备先阅读此文件,不要执行根 README 下方的旧部署脚本。
|
||||
|
||||
## 1. 获取正确分支
|
||||
|
||||
新架构工作分支是 `codex/new-deployment-architecture`,不是 `master`。
|
||||
先为新设备配置仓库 SSH 权限,并从可信渠道核对 Git 服务器主机指纹;不要禁用主机校验。
|
||||
|
||||
```sh
|
||||
git clone --branch codex/new-deployment-architecture --single-branch ssh://git@git.joywaygames.cn:2222/joywaygamess/server-deploy.git
|
||||
cd server-deploy
|
||||
git branch --show-current
|
||||
git log -1 --oneline
|
||||
git status --short
|
||||
```
|
||||
|
||||
已有本地仓库:先检查未提交更改,不要 reset、clean 或自动 stash;获取远端分支后再切换。
|
||||
|
||||
```sh
|
||||
git status --short
|
||||
git fetch origin
|
||||
git switch codex/new-deployment-architecture
|
||||
git pull --ff-only
|
||||
```
|
||||
|
||||
若本地尚无该分支,用 `git switch --track origin/codex/new-deployment-architecture` 创建跟踪分支。
|
||||
有冲突或远端分叉时停止核对,不强制覆盖。跨设备轮换前,显式提交并推送本轮需要带走的文件。
|
||||
聊天记录、本机临时文件和未提交修改不会随 Git 同步。
|
||||
|
||||
## 2. 阅读顺序与权威性
|
||||
|
||||
1. 本文件:当前停止点、待确认项与安全边界。
|
||||
2. [实施状态](docs/implementation-status.md):已完成工作包和实际验证证据。
|
||||
3. [完整方案](docs/2026-09-25-complete-optimization-proposal.md):已批准的总体架构。
|
||||
4. [开发环境](docs/development.md):新设备安装前提、构建与测试。
|
||||
5. [CLI 协议](protocol/README.md)及所改模块的 `internal/<模块>/README.md`:接口与限制。
|
||||
6. 需要远程工作时阅读[服务器初检](docs/2026-09-25-target-server-preflight.md),并重新核对现场。
|
||||
|
||||
2026-09-24 的 Caddy/结构审查文档是历史讨论,不能覆盖已选定的 Traefik 方案。
|
||||
`docs/superpowers/plans/` 是各阶段计划,旧复选框不代表当前完成度;以实施状态和实际代码/测试为准。
|
||||
仓库不依赖某一 AI 工具的会话、记忆或个人技能目录才能开发。
|
||||
|
||||
## 3. 已确认架构与交付形式
|
||||
|
||||
- 为新服务器建设,不做旧服务器的数据迁移或旧脚本兼容;保留原有数据是最高约束。
|
||||
- 应用可独立部署,数据库随应用隔离;公共环境为 Docker Compose、Traefik 等。
|
||||
- 本地管理产品:React/TypeScript 页面 + Node 本地服务 + SQLite,经 SSH 调用远端 Go `deployctl`。
|
||||
- 用户界面由浏览器打开;本地 API 只监听回环并需会话、Host/Origin、CSRF 防护。
|
||||
- 不是已选定的 Electron/Tauri 桌面安装 App,也不是公网管理网站。
|
||||
- 便携目标:Windows/macOS 解压启动,运行时随包交付,不要求最终用户安装开发工具。
|
||||
启动器、端口分配、数据目录、退出行为、签名和更新机制尚未实现/验收。
|
||||
- Apple 风格 UI;面板关闭不应影响已部署应用,已提交远端任务最终由 systemd 独立监督。
|
||||
后一项仍是未完成的设计目标。
|
||||
- `claude-dev-stack` 不纳入新体系;旧脚本暂留对照,不得作为新执行器的后端。
|
||||
|
||||
## 4. 当前完成与停止点
|
||||
|
||||
已完成八个基础工作包:离线计划、状态/锁、应用包完整性、受限 Compose 策略、本机预检、
|
||||
dpkg 清单/版本锁、Docker 仓库签名认证、实际五个 Docker deb 的字节校验。
|
||||
|
||||
`cmd/deployctl` 和 `internal/` 为当前实现。CLI 没有可用的安装、部署、升级、恢复写入口;
|
||||
`executable` 始终为 false。状态库尚未接入执行器。
|
||||
`apps/ops-panel/` 尚不存在;本地服务、UI、SSH 管理层、便携打包尚未交付。
|
||||
真实 deb 测试通过不等于完整依赖解析或安装验收。
|
||||
|
||||
下一工作包:Ubuntu 依赖源认证与隔离的完整 APT 依赖事务预演。
|
||||
最近提出的细化方案是:原生 APT 求解器、可信元数据/包状态快照、独立配置/缓存、
|
||||
不继承宿主钩子、只模拟;删除/降级/修改保留包或现场漂移时拒绝,输出完整变更清单。
|
||||
**该 APT 适配器细化方案尚待用户确认,尚未实现;本次跨设备交接不是对它的批准。**
|
||||
接续时先复跑基础测试,再请用户确认此边界,随后编写实施计划和失败测试。
|
||||
|
||||
## 5. 服务器与凭据边界
|
||||
|
||||
- 指定新服务器及已核对指纹见服务器初检文档;该文档是历史观察,不是当前现场保证。
|
||||
- 截至本次交接,只完成该新服务器 SSH 只读初检;新执行器没有在那里安装/部署应用。
|
||||
- 新架构阶段没有操作旧服务器;不要复用旧升级任务的授权去迁移或恢复数据。
|
||||
- 新设备需要自行配置私钥或 SSH Agent、仓库访问权限和可信 known_hosts;均不通过 Git 交付。
|
||||
- 私钥、有效密码、token、真实 `.env`、个人 SSH 配置、备份和数据库不能新增到提交中。
|
||||
`.gitignore` 不会自动移除历史已跟踪的敏感文件;旧配置不得作为新部署输入。
|
||||
历史凭据轮换和 Git 历史清理需另行授权,不在本次交接范围。
|
||||
- 原设备已有的 `claude-dev-stack/.env` 本地删除不属于新代码交付清单,不恢复它,
|
||||
也不以这次交接擅自清理旧目录。新设备若见到历史文件,不要使用或复制凭据。
|
||||
- 远端写入前重新核对主机身份、数据/服务、容量和影响,并取得具体操作授权。
|
||||
不做整机升级、不重置防火墙、不删除卷/数据库、不把备份恢复覆盖到原数据。
|
||||
|
||||
## 6. 可直接交给接续助手的说明
|
||||
|
||||
> 请先阅读 HANDOFF.md、docs/development.md、docs/implementation-status.md 和已批准的完整方案。
|
||||
> 检查实际分支和 git status,保留用户修改,在当前目录接续,不依赖旧聊天或其他设备的绝对路径。
|
||||
> 先构建并运行本机测试,区分历史证据与本次结果。当前停在第八工作包之后,APT 隔离细化方案待确认。
|
||||
> 不执行旧部署脚本,不改服务器数据,不把校验结果当成安装授权;提交/推送按本轮用户要求处理。
|
||||
|
||||
后续每个工作包完成时更新本文件的停止点及实施状态;涉及协议、工具链或交付方式变化时同步开发指南。
|
||||
@@ -1,5 +1,14 @@
|
||||
# Docker 服务部署集合
|
||||
|
||||
> **换设备继续开发:先读 [HANDOFF.md](HANDOFF.md),按 [开发指南](docs/development.md) 准备环境。**
|
||||
> 新架构工作分支:`codex/new-deployment-architecture`;拉取 `master` 不包含该阶段成果。
|
||||
|
||||
> 新服务器架构正在重构:选定 **Traefik + 本地管理面板 + Go deployctl**。
|
||||
> 当前新执行器支持离线计划、应用包校验、受限 Compose 策略、本机预检、环境安装提案及 Docker 仓库签名/deb 文件校验,尚不能部署、升级或恢复应用。
|
||||
> 参见 [完整方案](docs/2026-09-25-complete-optimization-proposal.md)、
|
||||
> [实施状态](docs/implementation-status.md) 和 [CLI 使用说明](protocol/README.md)。
|
||||
> 下方是旧部署实现的历史使用说明,不是新架构的安装指南;旧脚本不会成为新执行后端。
|
||||
|
||||
一套完整的自托管服务部署方案,适用于阿里云等国内服务器环境。
|
||||
|
||||
## 服务列表
|
||||
|
||||
@@ -1,43 +0,0 @@
|
||||
# =============================================================
|
||||
# Claude Dev Stack 配置文件
|
||||
# 复制为 .env 并按需填写
|
||||
# =============================================================
|
||||
|
||||
# ── Claude / API 配置 ────────────────────────────────────────
|
||||
# 推荐:灵眸 AI(国内直连,无需代理)https://docs.lmuai.com/docs/tools/claude-code
|
||||
ANTHROPIC_AUTH_TOKEN=sk-你的灵眸API密钥
|
||||
ANTHROPIC_BASE_URL=https://api.lmuai.com
|
||||
|
||||
# 备选:Anthropic 官方 API Key(海外直连)
|
||||
# ANTHROPIC_API_KEY=
|
||||
# ANTHROPIC_BASE_URL=https://api.anthropic.com
|
||||
|
||||
# 备选:DeepSeek 兼容接口
|
||||
# ANTHROPIC_API_KEY=
|
||||
# ANTHROPIC_BASE_URL=https://api.deepseek.com/anthropic
|
||||
|
||||
# 默认使用的模型
|
||||
# 灵眸可选:claude-opus-4-7 | claude-sonnet-4-6 | claude-sonnet-4-5 | claude-haiku-4-5
|
||||
# DeepSeek可选:deepseek-v4-pro
|
||||
CLAUDE_MODEL=claude-sonnet-4-6
|
||||
|
||||
# ── WSL2 ────────────────────────────────────────────────────
|
||||
# WSL2 发行版名称(wsl --list 查看已安装发行版)
|
||||
WSL_DISTRO=Ubuntu
|
||||
|
||||
# 设为 true 可跳过 WSL2 安装步骤(已安装时使用)
|
||||
SKIP_WSL_INSTALL=false
|
||||
|
||||
# ── Unity MCP ───────────────────────────────────────────────
|
||||
# unity-mcp-server 自动克隆至 WSL2 ~/.mcp-servers/unity-mcp-server/
|
||||
# Unity Plugin 需手动通过 Package Manager 安装:
|
||||
# https://github.com/AnkleBreaker-Studio/unity-mcp-plugin.git
|
||||
# (无需额外配置)
|
||||
|
||||
# ── Rust / Token Killer ──────────────────────────────────────
|
||||
# (暂无需配置,预留扩展用)
|
||||
# CARGO_REGISTRY_MIRROR=https://rsproxy.cn/
|
||||
|
||||
# ── Docker 镜像加速(可选)──────────────────────────────────
|
||||
# 若 WSL2 内需要 Docker,可配置国内加速镜像(逗号分隔)
|
||||
# DOCKER_REGISTRY_MIRRORS=https://docker.m.daocloud.io,https://hub-mirror.c.163.com
|
||||
@@ -1,2 +0,0 @@
|
||||
.env
|
||||
node_modules/
|
||||
@@ -1,446 +0,0 @@
|
||||
# Claude Dev Stack
|
||||
|
||||
WSL2 + Claude Code CLI + Unity MCP + Rust Token Killer **全栈一键部署方案**,适用于 Windows 11 开发环境。
|
||||
|
||||
支持 **WSL2 镜像网络模式**(`autoProxy=true`),WSL2 自动继承 Windows 代理状态,无需脚本干预。
|
||||
|
||||
## 组件清单
|
||||
|
||||
| 组件 | 说明 | 安装位置 |
|
||||
|------|------|---------|
|
||||
| **WSL2** | Windows Subsystem for Linux 2 | Windows 功能 |
|
||||
| **Claude Code CLI** | `@anthropic-ai/claude-code` | WSL2 npm global |
|
||||
| **Unity MCP Server** | `AnkleBreaker-Studio/unity-mcp-server` — Claude ↔ Unity Editor 双向集成 | WSL2 独立 Node.js 服务 + Unity Plugin |
|
||||
| **Rust** | rustup stable 工具链 | WSL2 `~/.cargo` |
|
||||
| **RTK** | Rust Token Killer (`rtk`) — LLM token 统计与上下文优化 | WSL2 cargo bin |
|
||||
|
||||
## 目录结构
|
||||
|
||||
```
|
||||
claude-dev-stack/
|
||||
├── deploy.ps1 # Windows 一键部署(PowerShell 5.1+)
|
||||
├── wsl-setup.sh # WSL2 内部安装脚本(可单独运行)
|
||||
├── .env.example # 配置模板
|
||||
└── README.md
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 快速开始
|
||||
|
||||
### ℹ️ WSL2 默认用户
|
||||
|
||||
本方案使用 **root** 作为 WSL2 默认用户(通过 `/etc/wsl.conf` 配置)。
|
||||
WSL2 Ubuntu 首次启动若弹出用户创建提示,可直接跳过或按提示操作,deploy.ps1 会自动将默认用户切换为 root。
|
||||
|
||||
---
|
||||
|
||||
### 场景一:全新 Windows 系统(首次安装 WSL2)
|
||||
|
||||
> 需要**管理员权限**启用 WSL2 Windows 功能
|
||||
|
||||
```powershell
|
||||
# 1. 以管理员身份运行 PowerShell
|
||||
cd path\to\claude-dev-stack
|
||||
|
||||
# 2. 复制配置文件,填写灵眸 API Key(推荐)或 Anthropic API Key
|
||||
cp .env.example .env
|
||||
notepad .env
|
||||
|
||||
# 3. 运行部署脚本
|
||||
pwsh .\deploy.ps1
|
||||
```
|
||||
|
||||
首次安装 WSL2 特性后可能需要**重启系统**,重启后重新执行脚本。
|
||||
|
||||
### 场景二:已有 WSL2,跳过 WSL2 安装
|
||||
|
||||
```powershell
|
||||
pwsh .\deploy.ps1 -SkipWSL
|
||||
```
|
||||
|
||||
### 场景三:仅在 WSL2 内安装(无 Windows 脚本)
|
||||
|
||||
```bash
|
||||
# 在 WSL2 Ubuntu 终端中执行
|
||||
cp .env.example .env
|
||||
nano .env
|
||||
bash wsl-setup.sh
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 代理说明
|
||||
|
||||
### WSL2 镜像模式自动继承 Windows 代理
|
||||
|
||||
脚本自动配置 `~/.wslconfig` 启用 WSL2 镜像网络模式:
|
||||
|
||||
```ini
|
||||
[wsl2]
|
||||
networkingMode=mirrored
|
||||
autoProxy=true
|
||||
```
|
||||
|
||||
`autoProxy=true` 下,WSL2 自动继承 Windows 系统代理状态。
|
||||
**是否使用代理由 Windows 侧决定**(例如通过 v2rayN「设为系统代理」),无需脚本干预。
|
||||
|
||||
---
|
||||
|
||||
## 配置说明(`.env`)
|
||||
|
||||
### 推荐:灵眸 AI(国内直连,无需代理)
|
||||
|
||||
| 变量 | 默认值 | 说明 |
|
||||
|------|--------|------|
|
||||
| `ANTHROPIC_AUTH_TOKEN` | _(必填)_ | 灵眸 API Key,从 [lmuai.com](https://lmuai.com) 获取 |
|
||||
| `ANTHROPIC_BASE_URL` | `https://api.lmuai.com` | 灵眸 API 基础 URL |
|
||||
| `CLAUDE_MODEL` | `claude-sonnet-4-6` | 可选:`claude-sonnet-4-6` / `claude-opus-4-7` / `claude-sonnet-4-5` / `claude-haiku-4-5` |
|
||||
|
||||
> **灵眸模式自动写入 `settings.json` 的 env 参数**:
|
||||
> | 参数 | 值 | 说明 |
|
||||
> |------|----|------|
|
||||
> | `ANTHROPIC_BASE_URL` | `https://api.lmuai.com` | API 端点 |
|
||||
> | `ANTHROPIC_AUTH_TOKEN` | _(你的 Key)_ | 认证凭据 |
|
||||
> | `API_TIMEOUT_MS` | `3000000` | 超时 50 分钟,防止长任务中断 |
|
||||
> | `CLAUDE_CODE_ATTRIBUTION_HEADER` | `0` | 关闭来源标识,提高缓存命中率 |
|
||||
> | `CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC` | `1` | 关闭遥测/自动更新检查 |
|
||||
>
|
||||
> - Token 写入 `settings.json` 的 `env` 块(不写入 `~/.bashrc`,避免与 `ANTHROPIC_API_KEY` 冲突)
|
||||
> - 旧 `ANTHROPIC_API_KEY` / `ANTHROPIC_BASE_URL` 变量从 `~/.bashrc` 自动清除
|
||||
|
||||
### 备选:Anthropic 官方 API
|
||||
|
||||
| 变量 | 默认值 | 说明 |
|
||||
|------|--------|------|
|
||||
| `ANTHROPIC_API_KEY` | _(空)_ | Anthropic API Key,从 [console.anthropic.com](https://console.anthropic.com) 获取 |
|
||||
| `ANTHROPIC_BASE_URL` | `https://api.anthropic.com` | API 基础 URL;DeepSeek 等中转可改为对应 URL |
|
||||
| `CLAUDE_MODEL` | `claude-sonnet-4-6` | 默认使用的模型 |
|
||||
|
||||
| 变量 | 默认值 | 说明 |
|
||||
|------|--------|------|
|
||||
| `WSL_DISTRO` | `Ubuntu` | WSL2 发行版名称 |
|
||||
| `SKIP_WSL_INSTALL` | `false` | `true` 跳过 WSL2 安装步骤 |
|
||||
|
||||
---
|
||||
|
||||
## Unity MCP 完整安装教程
|
||||
|
||||
> **MCP Server 仓库**:[AnkleBreaker-Studio/unity-mcp-server](https://github.com/AnkleBreaker-Studio/unity-mcp-server)
|
||||
> **Unity Plugin 仓库**:[AnkleBreaker-Studio/unity-mcp-plugin](https://github.com/AnkleBreaker-Studio/unity-mcp-plugin)
|
||||
https://github.com/AnkleBreaker-Studio/unity-mcp-plugin.git
|
||||
> **要求**:Unity 2021.3+ + Node.js 18+
|
||||
|
||||
### 架构原理
|
||||
|
||||
Unity MCP 由**两个独立仓库**组成,各自独立安装:
|
||||
|
||||
```
|
||||
Claude Code CLI
|
||||
│ MCP 协议 (stdio)
|
||||
▼
|
||||
Node.js MCP Server ← 克隆自 unity-mcp-server,WSL2 中运行
|
||||
(~/.mcp-servers/unity-mcp-server/) ← deploy.ps1 自动完成克隆 + npm install
|
||||
│ WebSocket / HTTP
|
||||
▼
|
||||
Unity Editor Plugin (C#) ← Package Manager 安装 unity-mcp-plugin
|
||||
│
|
||||
▼
|
||||
Unity 场景 / 对象 / 脚本 / 测试...
|
||||
```
|
||||
|
||||
- **unity-mcp-server**:独立 Node.js MCP 服务,部署脚本自动克隆并安装依赖
|
||||
- **unity-mcp-plugin**:Unity C# 插件,需要手动通过 Package Manager 安装
|
||||
|
||||
---
|
||||
|
||||
### Step 1:运行部署脚本(自动完成服务器安装)
|
||||
|
||||
```powershell
|
||||
pwsh .\deploy.ps1
|
||||
```
|
||||
|
||||
脚本会自动:
|
||||
1. 克隆 `unity-mcp-server` 到 WSL2 `~/.mcp-servers/unity-mcp-server/`
|
||||
2. 执行 `npm install` 安装依赖
|
||||
3. 将 MCP Server 注册到 `%USERPROFILE%\.claude\claude_desktop_config.json`
|
||||
|
||||
---
|
||||
|
||||
### Step 2:在 Unity Editor 安装 unity-mcp-plugin
|
||||
|
||||
1. 打开 Unity Editor
|
||||
2. 菜单:**Window → Package Manager**
|
||||
3. 点击左上角 **"+"** 按钮 → **"Add package from git URL..."**
|
||||
4. 输入以下 URL,点击 **Add**:
|
||||
```
|
||||
https://github.com/AnkleBreaker-Studio/unity-mcp-plugin.git
|
||||
```
|
||||
5. 等待包下载安装完成
|
||||
|
||||
安装完成后,菜单栏会出现 **Tools → Unity MCP** 选项。
|
||||
|
||||
---
|
||||
|
||||
### Step 3:启动 Unity Editor 端 MCP 服务
|
||||
|
||||
每次使用 Claude Code 操作 Unity 之前,需要先在 Unity 中启动服务:
|
||||
|
||||
1. 打开 Unity Editor,确保项目已加载
|
||||
2. 菜单:**Tools → Unity MCP → Start Server**(具体菜单项名称以插件实际为准)
|
||||
3. 状态变为绿色 / Connected 表示服务就绪
|
||||
|
||||
---
|
||||
|
||||
### Step 4:确认 MCP 配置
|
||||
|
||||
部署脚本已自动写入 Windows Claude Code 配置。确认内容:
|
||||
|
||||
```powershell
|
||||
cat "$env:USERPROFILE\.claude\claude_desktop_config.json"
|
||||
```
|
||||
|
||||
应包含类似如下内容:
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"unity-mcp": {
|
||||
"command": "wsl",
|
||||
"args": ["-d", "Ubuntu", "--", "node", "/home/用户名/.mcp-servers/unity-mcp-server/src/index.js"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
WSL2 内的配置(`~/.config/Claude/claude_desktop_config.json`)直接使用 `node`:
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"unity-mcp": {
|
||||
"command": "node",
|
||||
"args": ["/home/用户名/.mcp-servers/unity-mcp-server/src/index.js"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Step 5:在 Claude Code 中使用 Unity MCP
|
||||
|
||||
```bash
|
||||
# 启动 Claude Code(WSL2 内)
|
||||
claude
|
||||
|
||||
# 验证 Unity MCP 连接
|
||||
> /mcp
|
||||
|
||||
# 示例:操作 Unity 场景
|
||||
> 在场景中创建一个名为 Player 的空 GameObject
|
||||
> 给 Player 添加 Rigidbody 组件,质量设为 5
|
||||
> 创建一个新场景 Level1,保存到 Assets/Scenes/
|
||||
> 运行所有 EditMode 测试
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 可用 MCP 工具列表
|
||||
|
||||
AnkleBreaker Unity MCP 采用**两级工具架构**,共 **288 个工具**:
|
||||
|
||||
- **核心工具(~70个)**:直接暴露,无需中转
|
||||
- **高级工具(200+)**:通过 `unity_advanced_tool` 代理访问(避免 MCP 客户端工具过多失效)
|
||||
|
||||
#### 核心工具(直接可用)
|
||||
|
||||
| 分类 | 工具名 | 功能 |
|
||||
|------|--------|------|
|
||||
| **编辑器状态** | `unity_editor_ping` / `unity_editor_state` / `unity_project_info` | 检测连接、获取编辑器/项目状态 |
|
||||
| **场景** | `unity_scene_info/open/save/new/hierarchy/stats` | 场景完整生命周期管理 |
|
||||
| **GameObject** | `unity_gameobject_create/delete/info/set_transform/duplicate/set_active/reparent` | 对象增删改查 |
|
||||
| **组件** | `unity_component_add/remove/get_properties/set_property/set_reference/batch_wire` | 组件与属性操作 |
|
||||
| **资产** | `unity_asset_list/import/delete/create_prefab/instantiate_prefab` | 资产管理 |
|
||||
| **脚本** | `unity_script_create/read/update` + `unity_execute_code` | C# 脚本读写与运行时执行 |
|
||||
| **材质** | `unity_material_create` / `unity_renderer_set_material` | 材质创建与赋值 |
|
||||
| **构建/运行** | `unity_build` / `unity_play_mode` | 多平台构建、Play Mode 控制 |
|
||||
| **控制台** | `unity_console_log/clear` + `unity_get_compilation_errors` | 日志读取与编译错误 |
|
||||
| **编辑器操作** | `unity_execute_menu_item` / `unity_undo/redo/undo_history` | 菜单执行、撤销重做 |
|
||||
| **选择/搜索** | `unity_selection_*` / `unity_search_by_*` / `unity_search_assets` | 对象查找与选中 |
|
||||
| **截图** | `unity_screenshot_game/scene` + `unity_graphics_*_capture` | 场景/游戏视图截图 |
|
||||
| **Prefab** | `unity_prefab_info` / `unity_set_object_reference` | Prefab 信息与引用 |
|
||||
| **包管理** | `unity_packages_list/add/remove/search/info` | Package Manager 操作 |
|
||||
| **Multi-Agent** | `unity_queue_info` / `unity_agents_list` / `unity_agent_log` | 多代理会话管理 |
|
||||
| **高级工具代理** | `unity_list_advanced_tools` / `unity_advanced_tool` | 访问 200+ 高级工具 |
|
||||
| **Unity Hub** | `unity_hub_list_editors/available_releases/install_editor/install_modules` | Hub 版本管理 |
|
||||
| **多实例** | `unity_list_instances` / `unity_select_instance` | 多 Unity 实例切换 |
|
||||
| **项目上下文** | `unity_get_project_context` | AI 代理项目文档注入 |
|
||||
|
||||
#### 高级工具分类(通过 `unity_advanced_tool` 访问)
|
||||
|
||||
动画、Prefab 模式、物理、光照、音频、地形、导航网格、粒子、UI、标签与层、输入系统、Shader Graph、VFX Graph、Amplify Shader Editor、性能分析器、帧调试器、内存分析器、ScriptableObject、约束、LOD、MPPM 多人 PlayMode、UMA Avatar 等 30+ 分类。
|
||||
|
||||
```bash
|
||||
# 列出所有高级工具
|
||||
> 使用 unity_list_advanced_tools 查看所有高级工具
|
||||
|
||||
# 按分类过滤
|
||||
> unity_list_advanced_tools { "category": "animation" }
|
||||
> unity_list_advanced_tools { "category": "terrain" }
|
||||
|
||||
# 执行高级工具
|
||||
> unity_advanced_tool { "tool": "unity_animation_create_controller", "params": {...} }
|
||||
```
|
||||
|
||||
`deploy.ps1` 会自动将所有工具写入 `~/.claude/settings.json` 的 `allowedTools` 白名单,**无需每次手动确认**。
|
||||
|
||||
---
|
||||
|
||||
### 防火墙放行(Bridge 端口)
|
||||
|
||||
`deploy.ps1` 会自动创建防火墙规则,放行 TCP **7890–7899**(Bridge Port 范围)。
|
||||
|
||||
手动放行:
|
||||
|
||||
```powershell
|
||||
New-NetFirewallRule -DisplayName "Unity MCP Bridge Inbound" `
|
||||
-Direction Inbound -Protocol TCP -LocalPort 7890-7899 -Action Allow -Profile Any
|
||||
New-NetFirewallRule -DisplayName "Unity MCP Bridge Outbound" `
|
||||
-Direction Outbound -Protocol TCP -LocalPort 7890-7899 -Action Allow -Profile Any
|
||||
```
|
||||
|
||||
验证 Bridge 是否可用:
|
||||
|
||||
```powershell
|
||||
# Unity Editor 打开后
|
||||
Invoke-WebRequest http://127.0.0.1:7890/api/ping
|
||||
# 返回 200 OK 表示 Bridge 正常
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 故障排查
|
||||
|
||||
**问题:Unity MCP 连接失败**
|
||||
```
|
||||
解决:
|
||||
1. 确认 Unity Editor 已打开,且 unity-mcp-plugin 已启动服务
|
||||
2. 测试 Bridge: Invoke-WebRequest http://127.0.0.1:7890/api/ping
|
||||
3. 确认防火墙 TCP 7890 已放行
|
||||
4. 确认 MCP Server 进程正常:wsl -d Ubuntu -- node ~/.mcp-servers/unity-mcp-server/src/index.js
|
||||
5. 重新执行 deploy.ps1 重新克隆并注册 MCP Server
|
||||
```
|
||||
|
||||
**问题:更新 unity-mcp-server 后连接失败**
|
||||
```
|
||||
解决:
|
||||
wsl -d Ubuntu -- bash -c "cd ~/.mcp-servers/unity-mcp-server && git pull && npm install"
|
||||
```
|
||||
|
||||
**问题:npm install 安装超时(国内网络)**
|
||||
```bash
|
||||
# 方案一:在 Windows 侧开启系统代理(v2rayN「设为系统代理」),WSL2 自动继承
|
||||
# 方案二:WSL2 内配置镜像
|
||||
npm config set registry https://registry.npmmirror.com
|
||||
```
|
||||
|
||||
**问题:WSL2 无法访问外网**
|
||||
```
|
||||
解决:
|
||||
1. 确认 Windows 侧代理已开启系统代理(v2rayN → 参数设置 → 勾选「允许来自局域网的连接」)
|
||||
2. 确认 ~/.wslconfig 包含 networkingMode=mirrored 和 autoProxy=true
|
||||
3. 重启 WSL2:wsl --shutdown,再重新启动
|
||||
```
|
||||
|
||||
**问题:Unity 版本兼容性**
|
||||
```
|
||||
请参考 unity-mcp-plugin 仓库的 README 了解所需 Unity 最低版本要求
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 使用方式
|
||||
|
||||
### Claude Code CLI
|
||||
|
||||
```bash
|
||||
# 进入 WSL2
|
||||
wsl -d Ubuntu
|
||||
|
||||
# 启动 Claude Code
|
||||
claude
|
||||
|
||||
# 或从 PowerShell 快捷命令(由 deploy.ps1 写入 profile)
|
||||
claude-wsl
|
||||
```
|
||||
|
||||
### RTK (Rust Token Killer)
|
||||
|
||||
RTK 作为 Claude Code 的 **PreToolUse hook** 运行,自动将 bash 命令重写为 `rtk` 等效命令,过滤噪音、压缩输出,减少 60–90% token 消耗。
|
||||
|
||||
`deploy.ps1` 会自动执行 `rtk init -g` 注册 hook,**重启 Claude Code 后即生效**。
|
||||
|
||||
```bash
|
||||
# 验证 hook 安装状态
|
||||
rtk init --show
|
||||
|
||||
# 查看 token 节省统计(需先在 Claude Code 中运行一些命令)
|
||||
rtk gain
|
||||
rtk gain --graph # ASCII 图表
|
||||
rtk gain --history # 最近命令历史
|
||||
|
||||
# 手动使用(无需 hook)
|
||||
rtk git status # 压缩 git status 输出
|
||||
rtk cargo test # 只显示失败的测试
|
||||
rtk grep "pattern" . # 分组搜索结果
|
||||
```
|
||||
|
||||
### 故障排查 — rtk 安装失败
|
||||
|
||||
```bash
|
||||
# 手动安装
|
||||
cargo install --git https://github.com/rtk-ai/rtk
|
||||
|
||||
# 手动初始化 hook
|
||||
rtk init -g
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 环境检测逻辑
|
||||
|
||||
脚本对每个组件均先检测是否已安装,**已安装则跳过**,实现幂等执行:
|
||||
|
||||
```
|
||||
WSL2 → wsl --list 检测发行版
|
||||
Node.js → node --version
|
||||
Claude Code → claude --version
|
||||
Rust → rustc --version
|
||||
rtk → rtk --version
|
||||
Unity MCP → 检测 ~/.mcp-servers/unity-mcp-server/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 故障排查
|
||||
|
||||
### WSL2 安装失败
|
||||
|
||||
```powershell
|
||||
Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux -NoRestart
|
||||
Enable-WindowsOptionalFeature -Online -FeatureName VirtualMachinePlatform -NoRestart
|
||||
# 重启后
|
||||
wsl --update
|
||||
wsl --install -d Ubuntu
|
||||
```
|
||||
|
||||
### Rust/cargo 安装慢
|
||||
|
||||
在 `.env` 或 `~/.bashrc` 中添加 RsProxy 镜像:
|
||||
|
||||
```bash
|
||||
export RUSTUP_DIST_SERVER=https://rsproxy.cn
|
||||
export RUSTUP_UPDATE_ROOT=https://rsproxy.cn/rustup
|
||||
```
|
||||
|
||||
@@ -1,757 +0,0 @@
|
||||
#Requires -Version 5.1
|
||||
<#
|
||||
.SYNOPSIS
|
||||
WSL2 + Claude Code CLI + Unity MCP + Rust Token Killer 全栈一键部署脚本
|
||||
.DESCRIPTION
|
||||
自动完成以下步骤:
|
||||
1. 启用 WSL2 功能 & 安装 Ubuntu 发行版
|
||||
2. 检测/安装 Windows 本机 Node.js(AI 客户端 MCP 需要)
|
||||
3. WSL2 内安装 Node.js LTS(Claude Code CLI 需要)
|
||||
4. 安装 Claude Code CLI (@anthropic-ai/claude-code)
|
||||
5. 安装 Unity MCP Server(Windows + WSL2 双侧)& 写入各 AI 客户端配置
|
||||
6. 配置 Windows 防火墙放行 MCP Bridge 端口
|
||||
7. 安装 Rust 工具链 & Token Killer (rtk)
|
||||
8. 写入 PowerShell Profile 快捷命令
|
||||
.PARAMETER BridgePort
|
||||
Unity MCP Bridge 端口,默认 7890
|
||||
.PARAMETER InstallDir
|
||||
Windows 侧 unity-mcp-server 安装目录
|
||||
.PARAMETER UnityHubPath
|
||||
Unity Hub 路径(写入 MCP 配置 env)
|
||||
.PARAMETER SkipFirewall
|
||||
跳过防火墙规则配置
|
||||
.PARAMETER SkipWSL
|
||||
跳过 WSL2 安装步骤
|
||||
.NOTES
|
||||
首次安装 WSL2 需以管理员身份运行。
|
||||
已有 WSL2 可普通终端运行,加 -SkipWSL 跳过 WSL2 安装检查。
|
||||
#>
|
||||
param(
|
||||
[int] $BridgePort = 7890,
|
||||
[string]$InstallDir = "$env:USERPROFILE\unity-mcp-server",
|
||||
[string]$UnityHubPath = "C:\Program Files\Unity Hub\Unity Hub.exe",
|
||||
[switch]$SkipFirewall,
|
||||
[switch]$SkipWSL
|
||||
)
|
||||
|
||||
Set-StrictMode -Version Latest
|
||||
$ErrorActionPreference = "Stop"
|
||||
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
# 颜色日志辅助
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
function Write-Step { param($msg) Write-Host "`n==== $msg ====" -ForegroundColor Cyan }
|
||||
function Write-OK { param($msg) Write-Host " [OK] $msg" -ForegroundColor Green }
|
||||
function Write-Info { param($msg) Write-Host " [..] $msg" -ForegroundColor DarkGray }
|
||||
function Write-Warn { param($msg) Write-Host " [!!] $msg" -ForegroundColor Yellow }
|
||||
function Write-Fail { param($msg) Write-Host " [ERR] $msg" -ForegroundColor Red }
|
||||
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
# 加载 .env 配置
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
$ScriptDir = $PSScriptRoot
|
||||
$EnvFile = Join-Path $ScriptDir ".env"
|
||||
$EnvExample = Join-Path $ScriptDir ".env.example"
|
||||
|
||||
if (-not (Test-Path $EnvFile)) {
|
||||
if (Test-Path $EnvExample) {
|
||||
Copy-Item $EnvExample $EnvFile
|
||||
Write-Warn "已从 .env.example 创建 .env,请按需编辑后重新运行"
|
||||
Write-Warn " notepad $EnvFile"
|
||||
exit 0
|
||||
}
|
||||
}
|
||||
|
||||
$Config = @{}
|
||||
if (Test-Path $EnvFile) {
|
||||
Get-Content $EnvFile | Where-Object { $_ -match "^\s*[^#].*=" } | ForEach-Object {
|
||||
$parts = $_ -split "=", 2
|
||||
$key = $parts[0].Trim()
|
||||
$val = $parts[1].Trim().Trim('"').Trim("'")
|
||||
$Config[$key] = $val
|
||||
}
|
||||
Write-Info "已加载配置:$EnvFile"
|
||||
}
|
||||
|
||||
$ANTHROPIC_API_KEY = if ($Config["ANTHROPIC_API_KEY"]) { $Config["ANTHROPIC_API_KEY"] } else { "" }
|
||||
$ANTHROPIC_AUTH_TOKEN = if ($Config["ANTHROPIC_AUTH_TOKEN"]) { $Config["ANTHROPIC_AUTH_TOKEN"] } else { "" }
|
||||
$ANTHROPIC_BASE_URL = if ($Config["ANTHROPIC_BASE_URL"]) { $Config["ANTHROPIC_BASE_URL"] } else { "https://api.lmuai.com" }
|
||||
$CLAUDE_MODEL = if ($Config["CLAUDE_MODEL"]) { $Config["CLAUDE_MODEL"] } else { "claude-sonnet-4-6" }
|
||||
$WSL_DISTRO = if ($Config["WSL_DISTRO"]) { $Config["WSL_DISTRO"] } else { "Ubuntu" }
|
||||
$SKIP_WSL_INSTALL = if ($Config["SKIP_WSL_INSTALL"]) { $Config["SKIP_WSL_INSTALL"] } else { "false" }
|
||||
|
||||
# 判断认证方式:灵眸/中转用 AUTH_TOKEN,官方用 API_KEY
|
||||
$UseLmuAuth = ($ANTHROPIC_AUTH_TOKEN -ne "" -and $ANTHROPIC_BASE_URL -ne "https://api.anthropic.com")
|
||||
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
# Banner
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
Write-Host ""
|
||||
Write-Host "╔══════════════════════════════════════════════════════════╗" -ForegroundColor Cyan
|
||||
Write-Host "║ WSL2 + Claude Code CLI + Unity MCP + RTK 全栈部署 ║" -ForegroundColor Cyan
|
||||
Write-Host "║ AnkleBreaker Unity MCP · WSL2 Mirror Mode ║" -ForegroundColor Cyan
|
||||
Write-Host "╚══════════════════════════════════════════════════════════╝" -ForegroundColor Cyan
|
||||
Write-Host ""
|
||||
|
||||
# ══════════════════════════════════════════════════════════════
|
||||
# Step 1: WSL2 安装
|
||||
# ══════════════════════════════════════════════════════════════
|
||||
Write-Step "1/8 WSL2 环境检测 & 安装"
|
||||
|
||||
$distroInstalled = $false
|
||||
try {
|
||||
$dl = wsl --list --quiet 2>&1 | ForEach-Object { $_ -replace "`0","" }
|
||||
if ($dl -match [regex]::Escape($WSL_DISTRO)) { $distroInstalled = $true }
|
||||
} catch {}
|
||||
|
||||
if ($distroInstalled) {
|
||||
Write-OK "WSL2 + $WSL_DISTRO 已安装,跳过"
|
||||
} elseif ($SkipWSL -or $SKIP_WSL_INSTALL -eq "true") {
|
||||
Write-Warn "跳过 WSL2 安装"
|
||||
} else {
|
||||
$isAdmin = ([Security.Principal.WindowsPrincipal][Security.Principal.WindowsIdentity]::GetCurrent()
|
||||
).IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator)
|
||||
if (-not $isAdmin) {
|
||||
Write-Fail "安装 WSL2 需要管理员权限,请以管理员身份运行 PowerShell"
|
||||
Write-Info "如已安装 WSL2,请加 -SkipWSL 参数或在 .env 设置 SKIP_WSL_INSTALL=true"
|
||||
exit 1
|
||||
}
|
||||
$f1 = Get-WindowsOptionalFeature -Online -FeatureName "Microsoft-Windows-Subsystem-Linux" -ErrorAction SilentlyContinue
|
||||
if ($f1.State -ne "Enabled") {
|
||||
Enable-WindowsOptionalFeature -Online -FeatureName "Microsoft-Windows-Subsystem-Linux" -NoRestart | Out-Null
|
||||
}
|
||||
$f2 = Get-WindowsOptionalFeature -Online -FeatureName "VirtualMachinePlatform" -ErrorAction SilentlyContinue
|
||||
if ($f2.State -ne "Enabled") {
|
||||
Enable-WindowsOptionalFeature -Online -FeatureName "VirtualMachinePlatform" -NoRestart | Out-Null
|
||||
}
|
||||
wsl --update 2>&1 | Out-Null
|
||||
wsl --set-default-version 2 2>&1 | Out-Null
|
||||
wsl --install -d $WSL_DISTRO --no-launch 2>&1
|
||||
if ($LASTEXITCODE -ne 0) {
|
||||
Write-Fail "WSL2 安装失败,系统可能需要重启后重新运行"
|
||||
exit 1
|
||||
}
|
||||
Write-OK "WSL2 $WSL_DISTRO 安装完成"
|
||||
Write-Warn "首次安装可能需要重启,重启后重新运行脚本"
|
||||
}
|
||||
|
||||
# ── Step 1b: 首次 OOBE 处理 & 设置 root 为默认用户 ──────────
|
||||
# Ubuntu 首次安装会停在 OOBE 要求创建用户;用 --root 启动可跳过
|
||||
Write-Info "初始化 WSL2 发行版(确保 root 可用)..."
|
||||
$oobeDone = $false
|
||||
for ($i = 0; $i -lt 3; $i++) {
|
||||
wsl -d $WSL_DISTRO --user root -- bash -c "exit 0" 2>$null
|
||||
if ($LASTEXITCODE -eq 0) { $oobeDone = $true; break }
|
||||
Start-Sleep 3
|
||||
}
|
||||
if (-not $oobeDone) {
|
||||
# 触发 OOBE 完成(无交互,接受默认)
|
||||
wsl -d $WSL_DISTRO -- bash -c "exit 0" 2>$null
|
||||
Start-Sleep 5
|
||||
}
|
||||
|
||||
# 写入 /etc/wsl.conf(完整覆盖,避免重复块)
|
||||
$wslConfContent = "[boot]`nsystemd=false`n`n[user]`ndefault=root`n"
|
||||
wsl -d $WSL_DISTRO --user root -- bash -c "printf '[boot]\nsystemd=false\n\n[user]\ndefault=root\n' > /etc/wsl.conf" 2>$null
|
||||
Write-OK "/etc/wsl.conf 已写入 (default=root, systemd=false)"
|
||||
|
||||
# 重启使 wsl.conf 生效
|
||||
wsl --terminate $WSL_DISTRO 2>$null
|
||||
Start-Sleep 2
|
||||
Write-OK "WSL2 已重启,后续命令将以 root 运行"
|
||||
|
||||
# ── WSL2 执行辅助函数 ─────────────────────────────────────────
|
||||
function Invoke-WSL {
|
||||
param([string]$Command, [switch]$IgnoreError)
|
||||
|
||||
if ($Command -match "`n") {
|
||||
# 多行命令:base64 编码后在 bash 内解码执行,完全规避 PowerShell 管道 CRLF 问题
|
||||
$CleanCmd = $Command -replace "`r`n","`n" -replace "`r","`n"
|
||||
$b64 = [Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($CleanCmd))
|
||||
$result = wsl -d $WSL_DISTRO --user root -- bash -c "echo '$b64' | base64 -d | bash" 2>&1 |
|
||||
ForEach-Object { ($_ -replace "`0","").ToString() }
|
||||
} else {
|
||||
$result = wsl -d $WSL_DISTRO --user root -- bash -c $Command 2>&1 |
|
||||
ForEach-Object { ($_ -replace "`0","").ToString() }
|
||||
}
|
||||
|
||||
if ($LASTEXITCODE -ne 0 -and -not $IgnoreError) {
|
||||
Write-Fail "WSL 命令失败 (exit $LASTEXITCODE)"
|
||||
Write-Fail ($result -join "`n")
|
||||
exit 1
|
||||
}
|
||||
return ($result -join "`n")
|
||||
}
|
||||
|
||||
# 用 exit code 方式测试连通性,避免输出编码干扰
|
||||
wsl -d $WSL_DISTRO --user root -- bash -c "exit 0" 2>$null
|
||||
if ($LASTEXITCODE -ne 0) {
|
||||
Write-Fail "无法访问 WSL2 发行版 '$WSL_DISTRO'"
|
||||
exit 1
|
||||
}
|
||||
Write-OK "WSL2 ($WSL_DISTRO) 连接正常 (root)"
|
||||
|
||||
# ── Step 1a: 配置 .wslconfig(mirrored 网络模式) ────────────
|
||||
$wslCfgPath = "$env:USERPROFILE\.wslconfig"
|
||||
$wslCfgContent = @"
|
||||
[wsl2]
|
||||
networkingMode=mirrored
|
||||
dnsTunneling=true
|
||||
firewall=true
|
||||
autoProxy=true
|
||||
"@
|
||||
$needRestart = $false
|
||||
if (Test-Path $wslCfgPath) {
|
||||
$existing = Get-Content $wslCfgPath -Raw
|
||||
if ($existing -notmatch "networkingMode=mirrored") {
|
||||
Set-Content $wslCfgPath $wslCfgContent -Encoding UTF8
|
||||
$needRestart = $true
|
||||
Write-OK ".wslconfig 已更新 -> networkingMode=mirrored"
|
||||
}
|
||||
} else {
|
||||
Set-Content $wslCfgPath $wslCfgContent -Encoding UTF8
|
||||
$needRestart = $true
|
||||
Write-OK ".wslconfig 已创建 -> networkingMode=mirrored"
|
||||
}
|
||||
if ($needRestart) {
|
||||
Write-Info "重启 WSL2 以应用镜像网络模式..."
|
||||
wsl --shutdown 2>$null
|
||||
Start-Sleep 3
|
||||
}
|
||||
|
||||
# ══════════════════════════════════════════════════════════════
|
||||
# Step 2: Windows 本机 Node.js 检查(AI 客户端 MCP 需要)
|
||||
# ══════════════════════════════════════════════════════════════
|
||||
Write-Step "2/8 Windows Node.js 检查"
|
||||
|
||||
$winNode = Get-Command node -ErrorAction SilentlyContinue
|
||||
if ($winNode) {
|
||||
$winNodeVer = & node --version
|
||||
Write-OK "Windows Node.js 已安装: $winNodeVer"
|
||||
} else {
|
||||
Write-Warn "Windows 本机未检测到 Node.js"
|
||||
Write-Info "尝试通过 winget 安装 Node.js LTS..."
|
||||
$wingetCmd = Get-Command winget -ErrorAction SilentlyContinue
|
||||
if ($wingetCmd) {
|
||||
winget install --id OpenJS.NodeJS.LTS --silent --accept-source-agreements --accept-package-agreements 2>&1 | Out-Null
|
||||
# 刷新 PATH
|
||||
$env:Path = [System.Environment]::GetEnvironmentVariable("Path","Machine") + ";" +
|
||||
[System.Environment]::GetEnvironmentVariable("Path","User")
|
||||
$winNode = Get-Command node -ErrorAction SilentlyContinue
|
||||
if ($winNode) {
|
||||
Write-OK "Node.js 安装成功: $(& node --version)"
|
||||
} else {
|
||||
Write-Warn "winget 安装后未找到 node,请手动安装 Node.js 18+:"
|
||||
Write-Warn " https://nodejs.org/zh-cn/download"
|
||||
Write-Warn " 或: winget install OpenJS.NodeJS.LTS"
|
||||
}
|
||||
} else {
|
||||
Write-Warn "未找到 winget,请手动安装 Node.js 18+:"
|
||||
Write-Warn " https://nodejs.org/zh-cn/download"
|
||||
}
|
||||
}
|
||||
|
||||
# ══════════════════════════════════════════════════════════════
|
||||
# Step 3: WSL2 系统依赖 & Node.js
|
||||
# ══════════════════════════════════════════════════════════════
|
||||
Write-Step "3/8 WSL2 Node.js LTS"
|
||||
|
||||
# 检测 WSL2 原生 node(排除通过 WSL interop 调用的 Windows node,路径含 /mnt/)
|
||||
$nodeVer = Invoke-WSL @'
|
||||
export NVM_DIR="$HOME/.nvm"
|
||||
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"
|
||||
_n=$(which node 2>/dev/null)
|
||||
if [ -n "$_n" ] && ! echo "$_n" | grep -q '/mnt/'; then
|
||||
node --version
|
||||
else
|
||||
echo MISSING
|
||||
fi
|
||||
true
|
||||
'@ -IgnoreError
|
||||
if ($nodeVer -match "v\d+") {
|
||||
Write-OK "WSL2 Node.js 已安装 (原生): $($nodeVer.Trim())"
|
||||
} else {
|
||||
Write-Info "安装 WSL2 Node.js LTS (via nvm)..."
|
||||
$installNodeCmd = @'
|
||||
export DEBIAN_FRONTEND=noninteractive
|
||||
sudo apt-get install -y -qq curl ca-certificates 2>/dev/null
|
||||
# 安装 nvm
|
||||
export NVM_DIR="$HOME/.nvm"
|
||||
if [ ! -d "$NVM_DIR" ]; then
|
||||
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.3/install.sh | bash
|
||||
fi
|
||||
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"
|
||||
# 安装 Node.js LTS
|
||||
nvm install --lts
|
||||
nvm use --lts
|
||||
# 写入 .bashrc(幂等)
|
||||
grep -q 'NVM_DIR' ~/.bashrc || cat >> ~/.bashrc << 'NVMEOF'
|
||||
|
||||
# nvm (Node Version Manager)
|
||||
export NVM_DIR="$HOME/.nvm"
|
||||
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"
|
||||
[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion"
|
||||
NVMEOF
|
||||
node --version
|
||||
npm --version
|
||||
true
|
||||
'@
|
||||
Invoke-WSL $installNodeCmd
|
||||
Write-OK "WSL2 Node.js 安装完成"
|
||||
}
|
||||
|
||||
# ══════════════════════════════════════════════════════════════
|
||||
# Step 4: Claude Code CLI (WSL2)
|
||||
# ══════════════════════════════════════════════════════════════
|
||||
Write-Step "4/8 Claude Code CLI"
|
||||
|
||||
# 检测 WSL2 原生 claude(排除通过 WSL interop 调用的 Windows claude.exe,路径含 /mnt/)
|
||||
$claudeVer = Invoke-WSL @'
|
||||
export NVM_DIR="$HOME/.nvm"
|
||||
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"
|
||||
_c=$(which claude 2>/dev/null)
|
||||
if [ -n "$_c" ] && ! echo "$_c" | grep -q '/mnt/'; then
|
||||
claude --version 2>/dev/null
|
||||
else
|
||||
echo MISSING
|
||||
fi
|
||||
true
|
||||
'@ -IgnoreError
|
||||
if ($claudeVer -notmatch "MISSING") {
|
||||
Write-OK "Claude Code 已安装 (WSL2 原生): $($claudeVer.Trim())"
|
||||
} else {
|
||||
Write-Info "安装 @anthropic-ai/claude-code (WSL2 原生)..."
|
||||
$installClaudeCmd = @'
|
||||
export NVM_DIR="$HOME/.nvm"
|
||||
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"
|
||||
npm install -g @anthropic-ai/claude-code --quiet
|
||||
which claude
|
||||
claude --version
|
||||
true
|
||||
'@
|
||||
Invoke-WSL $installClaudeCmd
|
||||
Write-OK "Claude Code 安装完成"
|
||||
}
|
||||
|
||||
# 写入 Claude 配置(WSL2 侧)— 含 allowedTools 完整白名单(自动信任,无需每次确认)
|
||||
$mcpAllowedTools = @(
|
||||
# Claude 内置工具
|
||||
"Bash","Read","Write","Edit","MultiEdit","Glob","Grep","LS","WebFetch","TodoRead","TodoWrite",
|
||||
# Editor State
|
||||
"mcp__unity-mcp__unity_editor_ping","mcp__unity-mcp__unity_editor_state","mcp__unity-mcp__unity_project_info",
|
||||
# Scene
|
||||
"mcp__unity-mcp__unity_scene_info","mcp__unity-mcp__unity_scene_open","mcp__unity-mcp__unity_scene_save",
|
||||
"mcp__unity-mcp__unity_scene_new","mcp__unity-mcp__unity_scene_hierarchy","mcp__unity-mcp__unity_scene_stats",
|
||||
# GameObject
|
||||
"mcp__unity-mcp__unity_gameobject_create","mcp__unity-mcp__unity_gameobject_delete",
|
||||
"mcp__unity-mcp__unity_gameobject_info","mcp__unity-mcp__unity_gameobject_set_transform",
|
||||
"mcp__unity-mcp__unity_gameobject_duplicate","mcp__unity-mcp__unity_gameobject_set_active",
|
||||
"mcp__unity-mcp__unity_gameobject_reparent",
|
||||
# Component
|
||||
"mcp__unity-mcp__unity_component_add","mcp__unity-mcp__unity_component_remove",
|
||||
"mcp__unity-mcp__unity_component_get_properties","mcp__unity-mcp__unity_component_set_property",
|
||||
"mcp__unity-mcp__unity_component_set_reference","mcp__unity-mcp__unity_component_batch_wire",
|
||||
"mcp__unity-mcp__unity_component_get_referenceable",
|
||||
# Asset
|
||||
"mcp__unity-mcp__unity_asset_list","mcp__unity-mcp__unity_asset_import",
|
||||
"mcp__unity-mcp__unity_asset_delete","mcp__unity-mcp__unity_asset_create_prefab",
|
||||
"mcp__unity-mcp__unity_asset_instantiate_prefab",
|
||||
# Script & Code
|
||||
"mcp__unity-mcp__unity_script_create","mcp__unity-mcp__unity_script_read",
|
||||
"mcp__unity-mcp__unity_script_update","mcp__unity-mcp__unity_execute_code",
|
||||
# Material
|
||||
"mcp__unity-mcp__unity_material_create","mcp__unity-mcp__unity_renderer_set_material",
|
||||
# Build & Play Mode
|
||||
"mcp__unity-mcp__unity_build","mcp__unity-mcp__unity_play_mode",
|
||||
# Console & Compilation
|
||||
"mcp__unity-mcp__unity_console_log","mcp__unity-mcp__unity_console_clear",
|
||||
"mcp__unity-mcp__unity_get_compilation_errors",
|
||||
# Editor Actions
|
||||
"mcp__unity-mcp__unity_execute_menu_item","mcp__unity-mcp__unity_undo",
|
||||
"mcp__unity-mcp__unity_redo","mcp__unity-mcp__unity_undo_history",
|
||||
# Selection & Search
|
||||
"mcp__unity-mcp__unity_selection_get","mcp__unity-mcp__unity_selection_set",
|
||||
"mcp__unity-mcp__unity_selection_focus_scene_view","mcp__unity-mcp__unity_selection_find_by_type",
|
||||
"mcp__unity-mcp__unity_search_by_component","mcp__unity-mcp__unity_search_by_tag",
|
||||
"mcp__unity-mcp__unity_search_by_layer","mcp__unity-mcp__unity_search_by_name",
|
||||
"mcp__unity-mcp__unity_search_assets","mcp__unity-mcp__unity_search_missing_references",
|
||||
# Screenshots & Graphics
|
||||
"mcp__unity-mcp__unity_screenshot_game","mcp__unity-mcp__unity_screenshot_scene",
|
||||
"mcp__unity-mcp__unity_graphics_scene_capture","mcp__unity-mcp__unity_graphics_game_capture",
|
||||
# Prefab
|
||||
"mcp__unity-mcp__unity_prefab_info","mcp__unity-mcp__unity_set_object_reference",
|
||||
# Packages
|
||||
"mcp__unity-mcp__unity_packages_list","mcp__unity-mcp__unity_packages_add",
|
||||
"mcp__unity-mcp__unity_packages_remove","mcp__unity-mcp__unity_packages_search",
|
||||
"mcp__unity-mcp__unity_packages_info",
|
||||
# Queue & Multi-Agent
|
||||
"mcp__unity-mcp__unity_queue_info","mcp__unity-mcp__unity_agents_list","mcp__unity-mcp__unity_agent_log",
|
||||
# Advanced Tools proxy(200+ 工具通过此代理访问)
|
||||
"mcp__unity-mcp__unity_list_advanced_tools","mcp__unity-mcp__unity_advanced_tool",
|
||||
# Unity Hub
|
||||
"mcp__unity-mcp__unity_hub_list_editors","mcp__unity-mcp__unity_hub_available_releases",
|
||||
"mcp__unity-mcp__unity_hub_install_editor","mcp__unity-mcp__unity_hub_install_modules",
|
||||
"mcp__unity-mcp__unity_hub_get_install_path","mcp__unity-mcp__unity_hub_set_install_path",
|
||||
# Multi-Instance & Project Context
|
||||
"mcp__unity-mcp__unity_list_instances","mcp__unity-mcp__unity_select_instance",
|
||||
"mcp__unity-mcp__unity_get_project_context"
|
||||
)
|
||||
$claudeSettingsJson = if ($UseLmuAuth) {
|
||||
# 灵眸 / 中转 API:使用 env.ANTHROPIC_AUTH_TOKEN(避免与 ANTHROPIC_API_KEY 冲突)
|
||||
[ordered]@{
|
||||
env = [ordered]@{
|
||||
ANTHROPIC_BASE_URL = $ANTHROPIC_BASE_URL
|
||||
ANTHROPIC_AUTH_TOKEN = $ANTHROPIC_AUTH_TOKEN
|
||||
API_TIMEOUT_MS = "3000000"
|
||||
CLAUDE_CODE_ATTRIBUTION_HEADER = "0"
|
||||
CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC = "1"
|
||||
}
|
||||
model = $CLAUDE_MODEL
|
||||
allowedTools = $mcpAllowedTools
|
||||
} | ConvertTo-Json -Depth 4
|
||||
} else {
|
||||
# Anthropic 官方:使用 model + allowedTools,API Key 由环境变量注入
|
||||
[ordered]@{ model = $CLAUDE_MODEL; allowedTools = $mcpAllowedTools } | ConvertTo-Json -Depth 3
|
||||
}
|
||||
$writeClaudeSettingsCmd = @"
|
||||
mkdir -p ~/.claude
|
||||
cat > ~/.claude/settings.json << 'SETTINGS'
|
||||
$claudeSettingsJson
|
||||
SETTINGS
|
||||
true
|
||||
"@
|
||||
Invoke-WSL $writeClaudeSettingsCmd -IgnoreError | Out-Null
|
||||
|
||||
# 写入 bash 环境变量(WSL2 侧)
|
||||
# 灵眸模式:auth token 已在 settings.json env 块中,清理旧 bashrc 变量避免冲突
|
||||
# 官方模式:写入 ANTHROPIC_API_KEY 到 bashrc
|
||||
$cleanOldVarsCmd = @'
|
||||
sed -i '/ANTHROPIC_API_KEY/d' ~/.bashrc ~/.profile 2>/dev/null || true
|
||||
sed -i '/ANTHROPIC_BASE_URL/d' ~/.bashrc ~/.profile 2>/dev/null || true
|
||||
sed -i '/CLAUDE_MODEL/d' ~/.bashrc ~/.profile 2>/dev/null || true
|
||||
sed -i '/# Claude Code CLI/d' ~/.bashrc ~/.profile 2>/dev/null || true
|
||||
true
|
||||
'@
|
||||
Invoke-WSL $cleanOldVarsCmd -IgnoreError | Out-Null
|
||||
|
||||
if (-not $UseLmuAuth -and $ANTHROPIC_API_KEY) {
|
||||
$profBlock = "export ANTHROPIC_API_KEY='$ANTHROPIC_API_KEY'\n"
|
||||
$profBlock += "export ANTHROPIC_BASE_URL='$ANTHROPIC_BASE_URL'\n"
|
||||
$profBlock += "export CLAUDE_MODEL='$CLAUDE_MODEL'\n"
|
||||
$addEnvCmd = @"
|
||||
printf '\n# Claude Code CLI\n$profBlock' >> ~/.bashrc
|
||||
true
|
||||
"@
|
||||
Invoke-WSL $addEnvCmd -IgnoreError | Out-Null
|
||||
}
|
||||
|
||||
# Windows 侧 Claude 配置
|
||||
$claudeDir = "$env:USERPROFILE\.claude"
|
||||
if (-not (Test-Path $claudeDir)) { New-Item -ItemType Directory -Path $claudeDir | Out-Null }
|
||||
$claudeSettingsJson | Set-Content "$claudeDir\settings.json" -Encoding UTF8
|
||||
if ($UseLmuAuth) {
|
||||
# 灵眸:token 已在 settings.json 中,清除可能冲突的 Windows 用户级变量
|
||||
[System.Environment]::SetEnvironmentVariable("ANTHROPIC_API_KEY", $null, "User")
|
||||
[System.Environment]::SetEnvironmentVariable("ANTHROPIC_BASE_URL", $null, "User")
|
||||
[System.Environment]::SetEnvironmentVariable("CLAUDE_MODEL", $null, "User")
|
||||
} elseif ($ANTHROPIC_API_KEY) {
|
||||
[System.Environment]::SetEnvironmentVariable("ANTHROPIC_API_KEY", $ANTHROPIC_API_KEY, "User")
|
||||
[System.Environment]::SetEnvironmentVariable("ANTHROPIC_BASE_URL", $ANTHROPIC_BASE_URL, "User")
|
||||
[System.Environment]::SetEnvironmentVariable("CLAUDE_MODEL", $CLAUDE_MODEL, "User")
|
||||
}
|
||||
Write-OK "Claude Code 配置已写入(Windows + WSL2)"
|
||||
|
||||
# ══════════════════════════════════════════════════════════════
|
||||
# Step 5: Unity MCP Server 安装
|
||||
# ══════════════════════════════════════════════════════════════
|
||||
Write-Step "5/8 Unity MCP Server"
|
||||
Write-Info "MCP Server: https://github.com/AnkleBreaker-Studio/unity-mcp-server"
|
||||
Write-Info "Unity Plugin: https://github.com/AnkleBreaker-Studio/unity-mcp-plugin"
|
||||
|
||||
# ── 5a. Windows 侧安装(供 Claude Desktop / Cursor / Windsurf / VS Code)────
|
||||
Write-Info "安装 Windows 侧 MCP Server -> $InstallDir"
|
||||
if (Test-Path "$InstallDir\.git") {
|
||||
Write-Info "已存在,执行 git pull..."
|
||||
Push-Location $InstallDir
|
||||
try { git pull --quiet 2>&1 | Out-Null } finally { Pop-Location }
|
||||
} else {
|
||||
if (Test-Path $InstallDir) { Remove-Item $InstallDir -Recurse -Force }
|
||||
git clone --quiet --depth 1 https://github.com/AnkleBreaker-Studio/unity-mcp-server.git $InstallDir
|
||||
}
|
||||
|
||||
$winNodeExists = (Get-Command node -ErrorAction SilentlyContinue) -ne $null
|
||||
if ($winNodeExists) {
|
||||
Push-Location $InstallDir
|
||||
try { & npm install --prefer-offline --quiet 2>&1 | Out-Null } finally { Pop-Location }
|
||||
Write-OK "Windows MCP Server 就绪: $InstallDir"
|
||||
} else {
|
||||
Write-Warn "Windows Node.js 未就绪,跳过 Windows 侧 npm install(请安装 Node.js 后重新运行)"
|
||||
}
|
||||
|
||||
# ── 5b. WSL2 侧安装(供 Claude Code CLI)─────────────────────
|
||||
Write-Info "安装 WSL2 侧 MCP Server..."
|
||||
$wslMcpDir = "~/.mcp-servers/unity-mcp-server"
|
||||
$wslMcpCmd = @'
|
||||
export NVM_DIR="$HOME/.nvm"
|
||||
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"
|
||||
mkdir -p $HOME/.mcp-servers
|
||||
if [ -d $HOME/.mcp-servers/unity-mcp-server/.git ]; then
|
||||
git -C $HOME/.mcp-servers/unity-mcp-server pull --quiet 2>/dev/null || true
|
||||
else
|
||||
git clone --quiet --depth 1 https://github.com/AnkleBreaker-Studio/unity-mcp-server.git \
|
||||
$HOME/.mcp-servers/unity-mcp-server 2>/dev/null \
|
||||
|| git -c http.proxy="" -c https.proxy="" clone --quiet --depth 1 \
|
||||
https://github.com/AnkleBreaker-Studio/unity-mcp-server.git \
|
||||
$HOME/.mcp-servers/unity-mcp-server
|
||||
fi
|
||||
if [ -d $HOME/.mcp-servers/unity-mcp-server ]; then
|
||||
cd $HOME/.mcp-servers/unity-mcp-server
|
||||
# 修复可能由 root 遗留的权限问题
|
||||
sudo chown -R $(whoami):$(whoami) . 2>/dev/null || true
|
||||
npm install --prefer-offline --quiet 2>/dev/null || npm install --quiet
|
||||
fi
|
||||
echo MCP_WSL_OK
|
||||
true
|
||||
'@
|
||||
Invoke-WSL $wslMcpCmd
|
||||
|
||||
$wslHome = (Invoke-WSL "echo `$HOME" -IgnoreError).Trim()
|
||||
$wslScript = "$wslHome/.mcp-servers/unity-mcp-server/src/index.js"
|
||||
Write-OK "WSL2 MCP Server 就绪: $wslScript"
|
||||
|
||||
# ── 5c. 写入各 AI 客户端 MCP 配置 ───────────────────────────
|
||||
$winScript = ($InstallDir -replace "\\","/") + "/src/index.js"
|
||||
|
||||
$mcpEntry = @{
|
||||
command = "node"
|
||||
args = @($winScript)
|
||||
env = @{
|
||||
UNITY_HUB_PATH = $UnityHubPath
|
||||
UNITY_BRIDGE_PORT = "$BridgePort"
|
||||
}
|
||||
}
|
||||
|
||||
$wslMcpEntry = @{
|
||||
command = "wsl"
|
||||
args = @("-d", $WSL_DISTRO, "--", "node", $wslScript)
|
||||
env = @{
|
||||
UNITY_BRIDGE_PORT = "$BridgePort"
|
||||
}
|
||||
}
|
||||
|
||||
function Merge-McpConfig {
|
||||
param([string]$ConfigFile, [hashtable]$Entry, [string]$Label)
|
||||
$dir = Split-Path $ConfigFile
|
||||
if (-not (Test-Path $dir)) { New-Item -ItemType Directory -Path $dir -Force | Out-Null }
|
||||
# PS5.1 兼容:手动将 PSCustomObject 转为普通 Hashtable
|
||||
function ConvertTo-Hashtable($obj) {
|
||||
if ($obj -is [System.Management.Automation.PSCustomObject]) {
|
||||
$h = @{}
|
||||
foreach ($p in $obj.PSObject.Properties) { $h[$p.Name] = ConvertTo-Hashtable $p.Value }
|
||||
return $h
|
||||
} elseif ($obj -is [System.Collections.IEnumerable] -and $obj -isnot [string]) {
|
||||
return @($obj | ForEach-Object { ConvertTo-Hashtable $_ })
|
||||
}
|
||||
return $obj
|
||||
}
|
||||
$cfg = @{ mcpServers = @{} }
|
||||
if (Test-Path $ConfigFile) {
|
||||
try {
|
||||
$raw = Get-Content $ConfigFile -Raw | ConvertFrom-Json
|
||||
$converted = ConvertTo-Hashtable $raw
|
||||
if ($converted -is [hashtable]) { $cfg = $converted }
|
||||
} catch {}
|
||||
}
|
||||
if (-not $cfg.ContainsKey("mcpServers")) { $cfg["mcpServers"] = @{} }
|
||||
$cfg["mcpServers"]["unity-mcp"] = $Entry
|
||||
$cfg | ConvertTo-Json -Depth 10 | Set-Content $ConfigFile -Encoding UTF8
|
||||
Write-OK "$Label -> $ConfigFile"
|
||||
}
|
||||
|
||||
# Claude Desktop(Windows)
|
||||
Merge-McpConfig "$env:APPDATA\Claude\claude_desktop_config.json" $mcpEntry "Claude Desktop"
|
||||
# Cursor
|
||||
Merge-McpConfig "$env:USERPROFILE\.cursor\mcp.json" $mcpEntry "Cursor"
|
||||
# Windsurf(两种路径)
|
||||
$windsurfDir = if (Test-Path "$env:APPDATA\Windsurf") { "$env:APPDATA\Windsurf" } else { "$env:USERPROFILE\.codeium\windsurf" }
|
||||
Merge-McpConfig "$windsurfDir\mcp_config.json" $mcpEntry "Windsurf"
|
||||
# VS Code
|
||||
Merge-McpConfig "$env:APPDATA\Code\User\mcp.json" $mcpEntry "VS Code"
|
||||
# Claude Code CLI(WSL2)——用 `claude mcp add --scope user` 写入全局用户级 MCP 配置
|
||||
$wslMcpCfgCmd = @"
|
||||
export NVM_DIR="`$HOME/.nvm"
|
||||
[ -s "`$NVM_DIR/nvm.sh" ] && . "`$NVM_DIR/nvm.sh"
|
||||
# 先移除旧条目(幂等),再添加
|
||||
claude mcp remove unity-mcp --scope user 2>/dev/null || true
|
||||
claude mcp add --scope user unity-mcp node $wslScript -e UNITY_BRIDGE_PORT=$BridgePort
|
||||
echo "Claude Code MCP configured"
|
||||
true
|
||||
"@
|
||||
Invoke-WSL $wslMcpCfgCmd -IgnoreError | Out-Null
|
||||
Write-OK "Claude Code CLI (WSL2) MCP -> unity-mcp (node $wslScript)"
|
||||
|
||||
# ══════════════════════════════════════════════════════════════
|
||||
# Step 6: Windows 防火墙放行 MCP Bridge 端口
|
||||
# ══════════════════════════════════════════════════════════════
|
||||
Write-Step "6/8 Windows 防火墙规则"
|
||||
|
||||
if ($SkipFirewall) {
|
||||
Write-Warn "已跳过防火墙配置(-SkipFirewall)"
|
||||
} else {
|
||||
$portRange = "$BridgePort-$($BridgePort + 9)"
|
||||
$isAdmin = ([Security.Principal.WindowsPrincipal][Security.Principal.WindowsIdentity]::GetCurrent()
|
||||
).IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator)
|
||||
if ($isAdmin) {
|
||||
foreach ($dir in @("Inbound","Outbound")) {
|
||||
$name = "Unity MCP Bridge $dir"
|
||||
Get-NetFirewallRule -DisplayName $name -ErrorAction SilentlyContinue |
|
||||
Remove-NetFirewallRule -ErrorAction SilentlyContinue
|
||||
New-NetFirewallRule -DisplayName $name -Direction $dir `
|
||||
-LocalPort $portRange -Protocol TCP -Action Allow -Profile Any | Out-Null
|
||||
Write-OK "防火墙规则: $name (TCP $portRange)"
|
||||
}
|
||||
} else {
|
||||
Write-Warn "非管理员权限,使用提升权限添加防火墙规则..."
|
||||
$cmd = @"
|
||||
`$p = '$portRange'
|
||||
foreach (`$d in @('Inbound','Outbound')) {
|
||||
`$n = "Unity MCP Bridge `$d"
|
||||
Get-NetFirewallRule -DisplayName `$n -ErrorAction SilentlyContinue | Remove-NetFirewallRule -ErrorAction SilentlyContinue
|
||||
New-NetFirewallRule -DisplayName `$n -Direction `$d -LocalPort `$p -Protocol TCP -Action Allow -Profile Any | Out-Null
|
||||
Write-Host "OK: `$n"
|
||||
}
|
||||
Start-Sleep 1
|
||||
"@
|
||||
$enc = [Convert]::ToBase64String([Text.Encoding]::Unicode.GetBytes($cmd))
|
||||
$proc = Start-Process powershell.exe -ArgumentList "-NoProfile -EncodedCommand $enc" `
|
||||
-Verb RunAs -Wait -PassThru
|
||||
if ($proc.ExitCode -eq 0) {
|
||||
Write-OK "防火墙规则已添加 (TCP $portRange)"
|
||||
} else {
|
||||
Write-Warn "防火墙配置可能未成功,请手动放行 TCP $portRange"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
# ══════════════════════════════════════════════════════════════
|
||||
# Step 7: Rust 工具链 & Token Killer (WSL2)
|
||||
# ══════════════════════════════════════════════════════════════
|
||||
Write-Step "7/8 Rust & Token Killer"
|
||||
|
||||
$rustVer = Invoke-WSL ". ~/.cargo/env 2>/dev/null; rustc --version 2>/dev/null || echo MISSING" -IgnoreError
|
||||
if ($rustVer -notmatch "MISSING") {
|
||||
Write-OK "Rust 已安装: $($rustVer.Trim())"
|
||||
} else {
|
||||
Write-Info "安装 Rust 工具链..."
|
||||
$installRustCmd = @"
|
||||
sudo apt-get install -y -qq build-essential pkg-config libssl-dev 2>/dev/null || true
|
||||
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs -o /tmp/rustup-init.sh
|
||||
sh /tmp/rustup-init.sh -y --default-toolchain stable --no-modify-path
|
||||
source ~/.cargo/env
|
||||
rustc --version
|
||||
"@
|
||||
Invoke-WSL $installRustCmd
|
||||
Invoke-WSL "grep -q 'cargo/env' ~/.bashrc || echo 'source ~/.cargo/env 2>/dev/null || true' >> ~/.bashrc" -IgnoreError | Out-Null
|
||||
Invoke-WSL "grep -q 'cargo/env' ~/.profile || echo 'source ~/.cargo/env 2>/dev/null || true' >> ~/.profile" -IgnoreError | Out-Null
|
||||
Write-OK "Rust 安装完成"
|
||||
}
|
||||
|
||||
$rtkVer = Invoke-WSL ". ~/.cargo/env 2>/dev/null; rtk --version 2>/dev/null || echo MISSING" -IgnoreError
|
||||
if ($rtkVer -notmatch "MISSING") {
|
||||
Write-OK "rtk 已安装: $($rtkVer.Trim())"
|
||||
} else {
|
||||
Write-Info "安装 rtk (Rust Token Killer)..."
|
||||
$installRtkCmd = @"
|
||||
. ~/.cargo/env 2>/dev/null || true
|
||||
CARGO_NET_GIT_FETCH_WITH_CLI=true cargo install --git https://github.com/rtk-ai/rtk 2>&1 | tail -5
|
||||
"@
|
||||
Invoke-WSL $installRtkCmd -IgnoreError
|
||||
$rtkCheck = Invoke-WSL ". ~/.cargo/env 2>/dev/null; rtk --version 2>/dev/null || echo FAILED" -IgnoreError
|
||||
if ($rtkCheck -match "FAILED") {
|
||||
Write-Warn "rtk 安装失败,可手动运行: cargo install --git https://github.com/rtk-ai/rtk"
|
||||
} else {
|
||||
Write-OK "rtk 安装成功: $($rtkCheck.Trim())"
|
||||
}
|
||||
}
|
||||
|
||||
# rtk init -g:安装 Claude Code PreToolUse hook(幂等,yes 管道自动确认所有提示)
|
||||
Invoke-WSL ". ~/.cargo/env 2>/dev/null; yes | rtk init -g --auto-patch 2>/dev/null || true" -IgnoreError | Out-Null
|
||||
Write-OK "rtk hook 已注册 (rtk init -g),重启 Claude Code 后生效"
|
||||
|
||||
# ══════════════════════════════════════════════════════════════
|
||||
# Step 8: PowerShell Profile 配置
|
||||
# ══════════════════════════════════════════════════════════════
|
||||
Write-Step "8/8 PowerShell Profile"
|
||||
|
||||
$pwshProfile = "$env:USERPROFILE\Documents\PowerShell\Microsoft.PowerShell_profile.ps1"
|
||||
$profileDir = Split-Path $pwshProfile
|
||||
if (-not (Test-Path $profileDir)) { New-Item -ItemType Directory -Path $profileDir | Out-Null }
|
||||
|
||||
$profileBlock = @"
|
||||
|
||||
# ── Claude Dev Stack (generated by deploy.ps1) ────────────────
|
||||
`$env:ANTHROPIC_BASE_URL = "$ANTHROPIC_BASE_URL"
|
||||
$(if ($ANTHROPIC_API_KEY) { "`$env:ANTHROPIC_API_KEY = `"$ANTHROPIC_API_KEY`"" } else { "# ANTHROPIC_API_KEY= (配置 .env 后重新运行)" })
|
||||
`$env:CLAUDE_MODEL = "$CLAUDE_MODEL"
|
||||
function claude-wsl { wsl -d $WSL_DISTRO -- bash -ic 'claude' }
|
||||
function unity-mcp-status { Invoke-RestMethod http://127.0.0.1:$BridgePort/api/ping -ErrorAction SilentlyContinue }
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
"@
|
||||
|
||||
$existing = if (Test-Path $pwshProfile) { Get-Content $pwshProfile -Raw } else { "" }
|
||||
if ($existing -match "Claude Dev Stack") {
|
||||
$existing = $existing -replace "(?ms)# ── Claude Dev Stack.*?# ─{60}", $profileBlock
|
||||
Set-Content $pwshProfile $existing -Encoding UTF8
|
||||
} else {
|
||||
Add-Content $pwshProfile $profileBlock -Encoding UTF8
|
||||
}
|
||||
|
||||
$policy = Get-ExecutionPolicy -Scope CurrentUser
|
||||
if ($policy -in @("Restricted","Undefined")) {
|
||||
Set-ExecutionPolicy -Scope CurrentUser -ExecutionPolicy RemoteSigned -Force
|
||||
}
|
||||
Write-OK "PowerShell Profile 已配置: $pwshProfile"
|
||||
|
||||
# ══════════════════════════════════════════════════════════════
|
||||
# 安装总结
|
||||
# ══════════════════════════════════════════════════════════════
|
||||
Write-Host ""
|
||||
Write-Host "╔══════════════════════════════════════════════════════════════╗" -ForegroundColor Green
|
||||
Write-Host "║ ✅ 全栈部署完成 ║" -ForegroundColor Green
|
||||
Write-Host "╚══════════════════════════════════════════════════════════════╝" -ForegroundColor Green
|
||||
|
||||
$items = @(
|
||||
@("WSL2 内核", (Invoke-WSL "uname -r 2>/dev/null" -IgnoreError).Trim()),
|
||||
@("Node.js (Win)", (& node --version 2>$null)),
|
||||
@("Node.js (WSL)", $(if ($nodeVer -match "v\d+") { $nodeVer.Trim() } else { "" })),
|
||||
@("Claude Code", $(if ($claudeVer -notmatch "MISSING") { $claudeVer.Trim() } else { "" })),
|
||||
@("Rust", (Invoke-WSL ". ~/.cargo/env && rustc --version 2>/dev/null" -IgnoreError).Trim()),
|
||||
@("rtk", (Invoke-WSL ". ~/.cargo/env && rtk --version 2>/dev/null" -IgnoreError).Trim()),
|
||||
@("MCP Server", $InstallDir),
|
||||
@("Bridge Port", "TCP $BridgePort (防火墙已放行 $BridgePort-$($BridgePort+9))")
|
||||
)
|
||||
foreach ($it in $items) {
|
||||
Write-Host (" {0,-16} {1}" -f $it[0], $(if ($it[1]) { $it[1] } else { "(未安装)" })) -ForegroundColor White
|
||||
}
|
||||
|
||||
Write-Host ""
|
||||
Write-Host " ━━━━ 后续步骤 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" -ForegroundColor Cyan
|
||||
Write-Host " 1) 在 Unity 项目中安装 MCP Plugin(每个项目一次):" -ForegroundColor White
|
||||
Write-Host " Window > Package Manager > + > Add from git URL:" -ForegroundColor DarkGray
|
||||
Write-Host " https://github.com/AnkleBreaker-Studio/unity-mcp-plugin.git" -ForegroundColor Yellow
|
||||
Write-Host ""
|
||||
Write-Host " 2) 打开 Unity 后确认 Bridge 在线(浏览器验证):" -ForegroundColor White
|
||||
Write-Host " http://127.0.0.1:$BridgePort/api/ping" -ForegroundColor Yellow
|
||||
Write-Host ""
|
||||
Write-Host " 3) 重启 AI 客户端(Claude Desktop / Cursor / Windsurf)" -ForegroundColor White
|
||||
Write-Host " MCP 配置已自动写入各客户端配置文件" -ForegroundColor DarkGray
|
||||
Write-Host ""
|
||||
Write-Host " 4) 在 Claude Code (WSL2) 中使用:" -ForegroundColor White
|
||||
Write-Host " claude-wsl → claude → /mcp" -ForegroundColor DarkGray
|
||||
Write-Host ""
|
||||
if (-not $ANTHROPIC_API_KEY -and -not $ANTHROPIC_AUTH_TOKEN) {
|
||||
Write-Warn " ⚠ 未设置 API Key,请编辑 .env 后重新运行"
|
||||
Write-Warn " 灵眸用户:填写 ANTHROPIC_AUTH_TOKEN=sk-xxx"
|
||||
Write-Warn " 官方用户:填写 ANTHROPIC_API_KEY=sk-xxx"
|
||||
}
|
||||
Write-Host "╚══════════════════════════════════════════════════════════════╝" -ForegroundColor Green
|
||||
Generated
-6
@@ -1,6 +0,0 @@
|
||||
{
|
||||
"name": "claude-dev-stack",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {}
|
||||
}
|
||||
@@ -1,272 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# =============================================================
|
||||
# WSL2 内部环境检测 & 安装脚本
|
||||
# 可单独在 WSL2 Ubuntu 中运行:bash wsl-setup.sh
|
||||
# 也会被 deploy.ps1 自动调用(通过 Invoke-WSL)
|
||||
# =============================================================
|
||||
set -euo pipefail
|
||||
|
||||
RED='\033[0;31m'; GREEN='\033[0;32m'; YELLOW='\033[1;33m'
|
||||
CYAN='\033[0;36m'; NC='\033[0m'
|
||||
|
||||
log() { echo -e "${GREEN}[INFO]${NC} $*"; }
|
||||
warn() { echo -e "${YELLOW}[WARN]${NC} $*"; }
|
||||
error() { echo -e "${RED}[ERROR]${NC} $*" >&2; }
|
||||
step() { echo -e "\n${CYAN}====== $* ======${NC}"; }
|
||||
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
# 加载 .env(与脚本同目录)
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
ENV_FILE="$SCRIPT_DIR/.env"
|
||||
ENV_EXAMPLE="$SCRIPT_DIR/.env.example"
|
||||
|
||||
if [ ! -f "$ENV_FILE" ]; then
|
||||
if [ -f "$ENV_EXAMPLE" ]; then
|
||||
cp "$ENV_EXAMPLE" "$ENV_FILE"
|
||||
warn "已从 .env.example 创建 .env,请按需编辑后重新运行"
|
||||
exit 0
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ -f "$ENV_FILE" ]; then
|
||||
sed -i 's/\r$//' "$ENV_FILE"
|
||||
set -a; source "$ENV_FILE"; set +a
|
||||
fi
|
||||
|
||||
ANTHROPIC_API_KEY="${ANTHROPIC_API_KEY:-}"
|
||||
ANTHROPIC_BASE_URL="${ANTHROPIC_BASE_URL:-https://api.anthropic.com}"
|
||||
CLAUDE_MODEL="${CLAUDE_MODEL:-claude-opus-4-5}"
|
||||
BRIDGE_PORT="${BRIDGE_PORT:-7890}"
|
||||
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
# Step 1: v2rayN 代理配置(WSL2 侧)
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
step "1/6 代理配置"
|
||||
|
||||
# 优先读取 deploy.ps1 写入的 ~/.mcp-proxy.env
|
||||
if [ -f ~/.mcp-proxy.env ]; then
|
||||
source ~/.mcp-proxy.env 2>/dev/null || true
|
||||
if [ -n "${http_proxy:-}" ]; then
|
||||
log "代理已从 ~/.mcp-proxy.env 加载: $http_proxy"
|
||||
fi
|
||||
fi
|
||||
|
||||
# 若代理未设置但提供了 PROXY_PORT,则动态检测 Windows 主机 IP
|
||||
PROXY_PORT="${PROXY_PORT:-0}"
|
||||
if [ -z "${http_proxy:-}" ] && [ "$PROXY_PORT" -gt 0 ] 2>/dev/null; then
|
||||
WIN_HOST=$(ip route 2>/dev/null | grep default | awk '{print $3}' | head -1)
|
||||
[ -z "$WIN_HOST" ] && WIN_HOST=$(grep nameserver /etc/resolv.conf 2>/dev/null | awk '{print $2}' | head -1)
|
||||
if [ -n "$WIN_HOST" ]; then
|
||||
export http_proxy="http://${WIN_HOST}:${PROXY_PORT}"
|
||||
export https_proxy="$http_proxy"
|
||||
export HTTP_PROXY="$http_proxy"
|
||||
export HTTPS_PROXY="$http_proxy"
|
||||
export no_proxy="localhost,127.0.0.1,::1"
|
||||
log "WSL2 代理已设置: $http_proxy (Windows 主机: $WIN_HOST)"
|
||||
else
|
||||
warn "无法获取 Windows 主机 IP,代理未设置"
|
||||
fi
|
||||
fi
|
||||
|
||||
# 写入 ~/.mcp-proxy.env 并持久化到 ~/.bashrc
|
||||
if [ -n "${http_proxy:-}" ] && [ -z "$(grep -s 'mcp-proxy.env' ~/.bashrc)" ]; then
|
||||
cat >> ~/.bashrc << 'BASHRCBLOCK'
|
||||
|
||||
# Unity MCP WSL2 proxy (generated by wsl-setup.sh)
|
||||
[ -f ~/.mcp-proxy.env ] && . ~/.mcp-proxy.env
|
||||
BASHRCBLOCK
|
||||
log "代理配置已写入 ~/.bashrc"
|
||||
fi
|
||||
|
||||
# 写入 git 代理
|
||||
if [ -n "${http_proxy:-}" ]; then
|
||||
git config --global http.proxy "$http_proxy" 2>/dev/null || true
|
||||
git config --global https.proxy "$https_proxy" 2>/dev/null || true
|
||||
log "git 代理已配置"
|
||||
fi
|
||||
|
||||
if [ -z "${http_proxy:-}" ]; then
|
||||
warn "未配置代理,将使用直连网络(如下载失败请通过 PROXY_PORT=10809 重新运行)"
|
||||
fi
|
||||
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
# Step 2: 系统依赖
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
step "2/6 系统依赖"
|
||||
|
||||
export DEBIAN_FRONTEND=noninteractive
|
||||
sudo apt-get update -qq
|
||||
sudo apt-get install -y -qq \
|
||||
curl wget git build-essential pkg-config libssl-dev \
|
||||
ca-certificates gnupg lsb-release unzip python3
|
||||
log "系统依赖安装完成"
|
||||
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
# Step 3: Node.js LTS
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
step "3/6 Node.js LTS"
|
||||
|
||||
if command -v node &>/dev/null; then
|
||||
log "Node.js 已安装: $(node --version)"
|
||||
else
|
||||
log "通过 NodeSource 安装 Node.js LTS..."
|
||||
curl -fsSL https://deb.nodesource.com/setup_lts.x | sudo -E bash - > /dev/null 2>&1
|
||||
sudo apt-get install -y -qq nodejs
|
||||
log "Node.js 安装完成: $(node --version)"
|
||||
fi
|
||||
|
||||
# 设置 npm 代理
|
||||
if [ -n "${http_proxy:-}" ] && command -v npm &>/dev/null; then
|
||||
npm config set proxy "$http_proxy" 2>/dev/null || true
|
||||
npm config set https-proxy "$https_proxy" 2>/dev/null || true
|
||||
log "npm 代理已配置"
|
||||
fi
|
||||
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
# Step 4: Claude Code CLI
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
step "4/6 Claude Code CLI"
|
||||
|
||||
if command -v claude &>/dev/null; then
|
||||
log "Claude Code 已安装: $(claude --version 2>&1 | head -1)"
|
||||
else
|
||||
log "安装 @anthropic-ai/claude-code..."
|
||||
sudo npm install -g @anthropic-ai/claude-code --quiet
|
||||
log "Claude Code 安装完成: $(claude --version 2>&1 | head -1)"
|
||||
fi
|
||||
|
||||
mkdir -p ~/.claude
|
||||
cat > ~/.claude/settings.json << SETTINGS
|
||||
{"model": "$CLAUDE_MODEL"}
|
||||
SETTINGS
|
||||
log "已写入 ~/.claude/settings.json"
|
||||
|
||||
PROFILE_BLOCK=""
|
||||
[ -n "$ANTHROPIC_API_KEY" ] && PROFILE_BLOCK+="export ANTHROPIC_API_KEY='$ANTHROPIC_API_KEY'\n"
|
||||
[ -n "$ANTHROPIC_BASE_URL" ] && PROFILE_BLOCK+="export ANTHROPIC_BASE_URL='$ANTHROPIC_BASE_URL'\n"
|
||||
PROFILE_BLOCK+="export CLAUDE_MODEL='$CLAUDE_MODEL'\n"
|
||||
|
||||
for rc in ~/.bashrc ~/.profile; do
|
||||
if ! grep -q "ANTHROPIC_API_KEY" "$rc" 2>/dev/null; then
|
||||
printf "\n# Claude Code CLI\n${PROFILE_BLOCK}" >> "$rc"
|
||||
log "已追加环境变量到 $rc"
|
||||
fi
|
||||
done
|
||||
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
# Step 5: Unity MCP Server
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
step "5/6 Unity MCP Server (AnkleBreaker-Studio)"
|
||||
|
||||
MCP_SERVER_DIR="$HOME/.mcp-servers/unity-mcp-server"
|
||||
mkdir -p "$HOME/.mcp-servers"
|
||||
|
||||
if [ -d "$MCP_SERVER_DIR/.git" ]; then
|
||||
log "unity-mcp-server 已存在,更新至最新..."
|
||||
git -C "$MCP_SERVER_DIR" pull --quiet
|
||||
else
|
||||
log "克隆 unity-mcp-server..."
|
||||
git clone --quiet https://github.com/AnkleBreaker-Studio/unity-mcp-server.git "$MCP_SERVER_DIR"
|
||||
fi
|
||||
|
||||
log "安装 npm 依赖..."
|
||||
(cd "$MCP_SERVER_DIR" && npm install --prefer-offline --quiet 2>/dev/null || npm install --quiet)
|
||||
|
||||
# 确定入口文件
|
||||
if [ -f "$MCP_SERVER_DIR/src/index.js" ]; then
|
||||
SERVER_ENTRY="$MCP_SERVER_DIR/src/index.js"
|
||||
elif [ -f "$MCP_SERVER_DIR/build/index.js" ]; then
|
||||
SERVER_ENTRY="$MCP_SERVER_DIR/build/index.js"
|
||||
else
|
||||
SERVER_ENTRY="$MCP_SERVER_DIR/index.js"
|
||||
fi
|
||||
log "MCP Server 入口:$SERVER_ENTRY"
|
||||
|
||||
# 写入 Claude Code (WSL2) MCP 配置
|
||||
MCP_CONFIG_DIR="$HOME/.config/Claude"
|
||||
MCP_CONFIG_FILE="$MCP_CONFIG_DIR/claude_desktop_config.json"
|
||||
mkdir -p "$MCP_CONFIG_DIR"
|
||||
|
||||
python3 - << PYEOF
|
||||
import json, os
|
||||
|
||||
cfg_file = "$MCP_CONFIG_FILE"
|
||||
try:
|
||||
with open(cfg_file) as f:
|
||||
cfg = json.load(f)
|
||||
except:
|
||||
cfg = {}
|
||||
|
||||
cfg.setdefault("mcpServers", {})
|
||||
cfg["mcpServers"]["unity-mcp"] = {
|
||||
"command": "node",
|
||||
"args": ["$SERVER_ENTRY"],
|
||||
"env": {"UNITY_BRIDGE_PORT": "$BRIDGE_PORT"}
|
||||
}
|
||||
|
||||
with open(cfg_file, "w") as f:
|
||||
json.dump(cfg, f, indent=2)
|
||||
print("Claude Code (WSL2) MCP config written")
|
||||
PYEOF
|
||||
|
||||
log "MCP 配置已写入 $MCP_CONFIG_FILE"
|
||||
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
# Step 6: Rust 工具链 & Token Killer
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
step "6/6 Rust 工具链 & Token Killer"
|
||||
|
||||
[ -f "$HOME/.cargo/env" ] && source "$HOME/.cargo/env"
|
||||
|
||||
if command -v rustc &>/dev/null; then
|
||||
log "Rust 已安装: $(rustc --version)"
|
||||
else
|
||||
log "通过 rustup 安装 Rust stable..."
|
||||
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs \
|
||||
| sh -s -- -y --default-toolchain stable --no-modify-path
|
||||
source "$HOME/.cargo/env"
|
||||
log "Rust 安装完成: $(rustc --version)"
|
||||
fi
|
||||
|
||||
for rc in ~/.bashrc ~/.profile; do
|
||||
grep -q 'cargo/env' "$rc" 2>/dev/null || echo 'source ~/.cargo/env 2>/dev/null || true' >> "$rc"
|
||||
done
|
||||
|
||||
if command -v rtk &>/dev/null; then
|
||||
log "rtk 已安装: $(rtk --version 2>/dev/null || echo ok)"
|
||||
else
|
||||
log "安装 rtk (Rust Token Killer)..."
|
||||
sudo apt-get install -y -qq pkg-config libssl-dev 2>/dev/null || true
|
||||
if cargo install --git https://github.com/rtk-ai/rtk 2>&1 | tail -3; then
|
||||
log "rtk 安装成功"
|
||||
else
|
||||
warn "rtk 安装失败,请手动运行: cargo install --git https://github.com/rtk-ai/rtk"
|
||||
fi
|
||||
fi
|
||||
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
# 安装摘要
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
echo ""
|
||||
echo -e "${CYAN}═══════════════════════════════════════════════════${NC}"
|
||||
echo -e "${GREEN} ✅ WSL2 环境部署完成${NC}"
|
||||
echo -e "${CYAN}═══════════════════════════════════════════════════${NC}"
|
||||
echo -e " Node.js $(node --version 2>/dev/null || echo 未安装)"
|
||||
echo -e " npm $(npm --version 2>/dev/null || echo 未安装)"
|
||||
echo -e " Claude Code $(claude --version 2>/dev/null | head -1 || echo 未安装)"
|
||||
echo -e " Rust $(rustc --version 2>/dev/null || echo 未安装)"
|
||||
echo -e " rtk $(rtk --version 2>/dev/null || echo 未安装)"
|
||||
echo -e " MCP Server $SERVER_ENTRY"
|
||||
if [ -n "${http_proxy:-}" ]; then
|
||||
echo -e " Proxy $http_proxy"
|
||||
fi
|
||||
echo ""
|
||||
echo -e " ${YELLOW}⚠ 后续步骤(手动):${NC}"
|
||||
echo -e " 1. Unity Package Manager 安装插件:"
|
||||
echo -e " ${CYAN}https://github.com/AnkleBreaker-Studio/unity-mcp-plugin.git${NC}"
|
||||
echo -e " 2. 确认 Unity Bridge 在线: http://127.0.0.1:${BRIDGE_PORT}/api/ping"
|
||||
echo -e " 3. 重启 AI 客户端(Claude Desktop / Cursor / Windsurf)"
|
||||
echo -e "${CYAN}═══════════════════════════════════════════════════${NC}"
|
||||
echo ""
|
||||
log "请重新加载 shell: source ~/.bashrc"
|
||||
@@ -0,0 +1,12 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"os"
|
||||
"time"
|
||||
|
||||
"server-deploy/internal/cli"
|
||||
)
|
||||
|
||||
func main() {
|
||||
os.Exit(cli.Run(os.Args[1:], os.Stdin, os.Stdout, os.Stderr, time.Now))
|
||||
}
|
||||
@@ -0,0 +1,294 @@
|
||||
# 部署架构审查与管理工具接入建议
|
||||
|
||||
日期:2026-09-24。状态:历史审查与讨论记录,尚未实施。用户已明确以全新服务器为目标,不需要渐进改造或存量兼容;本文的源码发现仍供参考,涉及保留旧结构、接管与过渡层的建议已撤回。当前候选架构以 [全新部署架构](2026-09-24-greenfield-architecture.md) 为准。
|
||||
|
||||
## 1. 范围与结论
|
||||
|
||||
本次检查本仓库全部顶层目录、7 份 Compose、共享基础脚本、各工具部署/备份/卸载入口、Gitea/Vaultwarden 升级逻辑、Nginx 模板与已有测试目录。参考项目为 `G:/Works/YouleGames/tools/docker-ops-panel`,检查其文档、连接与身份核验、任务执行、服务清单、会话校验和打包结构。
|
||||
|
||||
这是本地源码与架构审查,没有连接生产服务器,没有执行部署、恢复、清理、安装或迁移;不把历史会话的线上结果当作当前生产状态。没有读取实际 `.env` 内容;在普通源码中发现硬编码凭据,本报告不收录其值。
|
||||
|
||||
结论:目前应用运行单元已基本分离,主要问题是部署生命周期仍耦合主机环境,缺少稳定的实例身份、操作协议和恢复边界。推荐保留“一工具一独立部署单元”,先规范脚本,再建立可选的本地 SSH 管理面板。无需合并成一份总 Compose,也无需为此引入 Kubernetes。
|
||||
|
||||
## 2. 当前目录与运行单元
|
||||
|
||||
- `base/`:Docker、Nginx、Certbot、防火墙、系统初始化及共享 Shell 函数;同时承担安装程序和公共库两种职责。
|
||||
- `gitea/`:独立 Compose,应用 + 私有 MySQL 8.4;含部署、备份、升级、卸载及 Nginx 模板。
|
||||
- `joplin/`:独立 Compose,应用 + 私有 PostgreSQL;含部署、备份、卸载及 Nginx 模板。
|
||||
- `vaultwarden/`:独立 Compose,应用持久数据;含部署、备份、升级、卸载及 Nginx 模板。
|
||||
- `certd/`、`siyuan/`、`portainer/`:各自独立 Compose、数据目录和运维脚本。
|
||||
- `rustdesk/`:独立 Compose,hbbs + hbbr 组成一个工具,共享该工具的数据目录,包含公网 TCP/UDP 和回环 WebSocket 入口。
|
||||
- `vps-xray/`:独立 systemd 部署,不使用 Compose;另含内核网络参数、防火墙、定时重启及客户端配置生成。应支持不同主机部署。
|
||||
- `claude/`:开发机 CLI 配置脚本,不是 Linux 服务器应用栈。
|
||||
- `claude-dev-stack/`:用户已明确废弃,目录与其中唯一的 `.env` 已从工作区删除,不纳入工具目录、管理界面、适配器或部署计划;历史版本仍保留该文件。
|
||||
- `docs/`:已有 Xray 设计与测试计划。
|
||||
- `.claude/`:本地辅助目录,不作为发布单元。
|
||||
|
||||
当前 Web 路径为:宿主 Nginx → 各应用的回环映射端口。MySQL/PostgreSQL 没有发布宿主端口。应用之间未发现业务依赖。Gitea SSH、RustDesk TCP/UDP 为公网直达例外。Certd 是独立证书管理应用,当前脚本中的站点证书仍由 Certbot 处理,不能认为 Certd 已接管所有证书。
|
||||
|
||||
## 3. 主要发现及证据
|
||||
|
||||
### P0:发布内容中存在凭据边界问题
|
||||
|
||||
初次审查时 `git ls-files` 确认 `claude-dev-stack/.env` 已被跟踪,随后按用户要求从工作区删除并排除管理范围;`claude/setup-claude-deepseek.sh:14` 和 PowerShell 对应脚本存在硬编码 API 凭据。根 `.gitignore` 中 Xray 生成配置的忽略规则被注释,多个节点信息文件也被跟踪。
|
||||
|
||||
新增面板的上传/打包必须使用发布文件白名单,不能递归上传整个工作区。需要另行确认并轮换已进入版本历史的有效凭据,将实际环境文件移出跟踪;仅添加 ignore 不会移除已经跟踪的文件。当前仅按用户要求删除废弃的 claude-dev-stack,不重写历史或轮换密钥。
|
||||
|
||||
### P1:应用部署会影响整台主机
|
||||
|
||||
`certd/deploy.sh:145` 等应用入口会调用 `init_system`;`base/setup.sh:72` 执行整机 `apt-get upgrade`。应用配置校验在这些步骤之后。Gitea 在自己的脚本里重复了同一套初始化逻辑。
|
||||
|
||||
`base/setup.sh:157` 在已有 Docker 配置缺少 registry-mirrors 字段时覆盖整个 daemon.json,随后重启 Docker。应用部署不应隐式执行这些操作。
|
||||
|
||||
建议拆分主机初始化、主机维护和应用生命周期;应用只检查基础能力。公共库被 source 时不得隐式安装或修改系统。
|
||||
|
||||
### P1:脚本位置依赖阻碍单独分发
|
||||
|
||||
Certd、Joplin、Vaultwarden、SiYuan、Portainer、RustDesk 的 deploy.sh 均要求 `../base/setup.sh` 存在,即使服务器已安装所需环境也无法跳过。它们运行时独立,部署包却不独立。
|
||||
|
||||
共享源代码可以保留,但单工具发布包应带版本固定的最小运维库,或使用显式安装并验证版本的公共运行器;不能依赖相邻仓库目录。建议先采用发布时打包最小库,减少服务端额外依赖。
|
||||
|
||||
### P1:同主机多实例隔离不完整
|
||||
|
||||
7 份 Compose 都使用固定 container_name;大部分宿主端口和 Nginx 上游固定。Vaultwarden 虽支持 VAULTWARDEN_PORT,Nginx 模板仍固定 8080。Gitea 升级检测支持 GITEA_HTTP_PORT,但 Compose 和 Nginx 仍固定 3000。
|
||||
|
||||
仅设置 Compose 项目名还不够,容器名、端口、数据路径、Nginx 站点名称和备份范围必须同时隔离。项目身份不能只靠当前文件夹名推断。
|
||||
|
||||
现有实例应读取 Docker Compose 标签与挂载建立绑定,不应直接删除 container_name 或改项目名后重建;这可能创建第二套容器或误连数据。
|
||||
|
||||
### P1:共享入口和证书缺少事务与归属管理
|
||||
|
||||
`base/setup.sh:342` 起先把站点替换成临时 HTTP 配置,再申请证书;失败后没有恢复旧站点。`deploy_nginx_conf` 覆盖目标文件后才测试,失败时磁盘上可能留着无效配置。
|
||||
|
||||
各工具卸载入口还会删除证书并可能移除共享 Certbot cron。应由独立入口模块持有站点和证书引用,应用卸载只释放自己的绑定;续期调度由环境模块负责。
|
||||
|
||||
建议保持宿主 Nginx,使用主机级锁,备份旧配置、生成候选、验证、切换、reload、探测,失败时还原。每张证书只有一个明确的管理者,Certbot 与 Certd 不同时自动修改同一证书。
|
||||
|
||||
### P1:备份/恢复契约不足以支撑通用面板
|
||||
|
||||
`certd/backup.sh`、`portainer/backup.sh`、`joplin/backup.sh` 等把配置打包错误通过 `|| true` 忽略,随后仍输出成功。多个脚本仅按目录年龄清理备份,缺少成功清单、引用保护与明确的自有文件边界。运行中的数据库/文件目录归档也不能统一视为一致快照。
|
||||
|
||||
Gitea 的备份有单独 flock,但 upgrade.sh 没有使用同一把实例锁,不能阻止备份和升级交叠。统一面板锁也必须覆盖 CLI 入口。
|
||||
|
||||
恢复应声明类型:配置恢复、镜像回退、数据库恢复、完整快照恢复。数据库迁移不能套用参考面板的通用镜像回退。备份目录需要 manifest、完成标记、文件摘要、版本、数据位置及一致性模式;被回滚点引用的备份禁止自动清理。
|
||||
|
||||
### P1:现有升级脚本不应直接视为经过全面验证的执行后端
|
||||
|
||||
`gitea/upgrade.sh:617` 附近的 wait_healthy 仍接受任意非 000 HTTP 响应,包括 500。自动和手动回滚忽略 stop 失败后继续恢复数据库,存在写入方未停就恢复数据的风险。自动回滚还可能在服务未就绪后输出“已回滚”。
|
||||
|
||||
恢复文件为合并解压,数据库恢复保留新版本额外表;这不等于精确还原。全量备份回灌也会覆盖备份内的 Git 引用文件,不能声称升级后的新提交天然保留。
|
||||
|
||||
此前“跨小版本必有 schema 版本增长”“表行数不减少即数据完好”“遗留表普遍无害”都不宜作为所有版本的通用保证。应绑定目标容器、实际数据库、目标版本兼容要求和具体恢复演练;通过有限场景只能证明对应场景。
|
||||
|
||||
### P2:非 Docker 工具及端口资源需要单独建模
|
||||
|
||||
Xray 默认 XRAY_PORT=443(`vps-xray/deploy.sh:411`),与同一地址上的 Nginx TLS 入口存在绑定冲突的可能;它还执行 sysctl 和防火墙修改。不能把所有工具都抽象成 docker compose up。
|
||||
|
||||
面板应支持 Compose 与 systemd 两类执行器;Xray 先支持状态及明确的维护动作。主机网络调优必须作为单独环境计划展示,安装前检查真实监听地址、协议和端口。
|
||||
|
||||
### P2:版本、能力、文档存在漂移
|
||||
|
||||
多个镜像使用 latest/lts/小版本浮动标签;仓库 Gitea 默认仍为 1.25。根 README 未覆盖 Xray、CLI 工具和残留目录,列出不存在的 gitea/migrate.sh,并把所有端口描述为回环,遗漏实际公网例外。
|
||||
|
||||
只有 Gitea/Vaultwarden 有专门 upgrade.sh,其余工具不能仅因存在 deploy.sh 就在面板上显示“安全升级/回滚”。面板应按适配器实际能力显示操作,解析并记录具体镜像 digest。
|
||||
|
||||
## 4. 推荐目标架构
|
||||
|
||||
采用三层结构,管理界面是可选控制入口:
|
||||
|
||||
1. **主机环境**:Docker/Compose、宿主 Nginx、证书续期、防火墙。单独检查、安装、维护,主机级变更说明影响范围。
|
||||
2. **独立应用实例**:每实例独立 Compose 项目/或 systemd 单元、配置、数据、备份与任务记录。应用私有数据库留在实例内。Gitea 不依赖 Joplin、Certd 或 Portainer。
|
||||
3. **本地管理面板与 CLI**:共用操作协议和执行器。面板停止后,应用继续工作,已提交的远程任务可以继续并被重新核对。
|
||||
|
||||
应用实例目标由 hostId + appId + instanceId + 规范化部署路径 + 实际项目/单元身份组成。所有写计划绑定目标、配置/编排摘要、明确版本、影响资源和恢复能力;执行前在锁内复核。
|
||||
|
||||
独立部署应同时满足:只选择一个工具即可准备它自己的依赖;不要求安装其他应用或面板;部署/升级不修改其他应用;独立停启、备份、恢复、卸载;同一台主机可部署第二实例;环境能力缺失时给出独立环境准备计划。
|
||||
|
||||
数据库可以在高级配置中外置,但默认不合并共享 MySQL/PostgreSQL;避免将升级、故障和恢复范围扩大到多个工具。
|
||||
|
||||
## 5. 管理工具参考项目的取舍
|
||||
|
||||
可借鉴的机制:
|
||||
|
||||
- React/Vite + Node 本地服务、Windows/macOS 便携启动与打包。
|
||||
- SSH 私钥/Agent、SHA256 指纹确认,传输层明确区分断线和任务失败。
|
||||
- 本地 Host/Origin/会话/CSRF 校验。
|
||||
- 目标绑定与过期预览、配置漂移检查、受控参数、秘密遮蔽。
|
||||
- 远端任务目录、退出码、日志、进程状态、断线后的状态核对。
|
||||
- 主机环境操作与应用生命周期分离。
|
||||
|
||||
需要重建的业务层:
|
||||
|
||||
- `src/contracts.ts:1` 固定了 game-docker 的服务列表。
|
||||
- `src/connections.ts:388` 会拒绝不存在 game-docker 服务的目录,不能直接连接本仓库的 Gitea/Joplin 部署。
|
||||
- 部署目录、环境字段、微信/QQ、发布文件清单与业务站点路由均为专用逻辑。
|
||||
- 现有档案偏向“一个连接目标绑定一个项目”;这里需要“一台主机登记多个独立实例”。
|
||||
|
||||
因此建议在本仓库新增 `tools/ops-panel/`,复用经抽取和测试的机制,应用适配器独立定义。不要直接搬整个参考项目再堆条件判断。参考项目自己也将真实 Linux Docker 验收列为未完成,不能把其本地测试当作生产保证。
|
||||
|
||||
Portainer 保持可选:它提供容器层管理,本工具负责仓库应用生命周期、备份与恢复计划。直接在 Portainer 修改编排可能造成漂移,面板须检测并要求重新识别。docker.sock 的 `:ro` 挂载不能当作 Docker API 只读权限。
|
||||
|
||||
## 6. 候选目录与交付方式
|
||||
|
||||
第一阶段保留现有应用目录和线上路径,新增职责明确的目录:
|
||||
|
||||
```text
|
||||
server-deploy/
|
||||
base/ # 兼容旧入口;逐步成为显式主机环境命令
|
||||
lib/ # 可版本化、无隐式副作用的通用运维函数
|
||||
contracts/ # 应用清单、计划、实例、任务、备份 schema
|
||||
tools/ops-panel/ # 可独立启动的本地管理工具
|
||||
gitea/ # 原路径保留,逐个添加应用清单与适配入口
|
||||
joplin/
|
||||
vaultwarden/
|
||||
certd/ siyuan/ portainer/ rustdesk/
|
||||
vps-xray/ # systemd 适配器
|
||||
claude/ # 开发机配置工具,不默认纳入服务器部署
|
||||
tests/ # 协议、脚本故障、真实隔离验收
|
||||
docs/
|
||||
```
|
||||
|
||||
应用清单声明:标识/版本、运行器、Compose 服务角色、环境要求、端口及入口、秘密字段、数据挂载、健康探测、升级与恢复策略、支持的动作。清单不是任意命令执行接口,执行器仍使用受控命令和参数校验。
|
||||
|
||||
远端先保持 `/opt/gitea` 等既有目录;已有数据路径按真实挂载登记。新实例可采用 `/opt/selfhost/instances/<实例>/`、`/var/lib/selfhost/<实例>/` 和 `/var/backups/selfhost/<实例>/`,但不自动移动旧目录。单工具发布包只包含自身清单、Compose、脚本/模板和所需版本的通用库,避免依赖整仓库。
|
||||
|
||||
## 7. 方案比较与实施顺序
|
||||
|
||||
**推荐:独立应用 + 共享环境 + 本地 SSH 面板。** 最接近当前结构,改造可以逐工具进行,日常运维不依赖服务器额外常驻控制服务。
|
||||
|
||||
**备选:独立应用 + 服务器 Web 面板。** 适合多人协作,但增加集中凭据、用户权限、审计、HTTPS 和面板自身可用性的维护成本。
|
||||
|
||||
**不推荐:全服务合并 Compose,通过 profiles 选择。** 能集中启动,却容易重新引入共享环境、项目级操作和恢复范围耦合,不符合本次强调的独立部署目标。
|
||||
|
||||
建议实施顺序:
|
||||
|
||||
1. 凭据与发布边界清理;明确环境与应用责任;登记所有已有实例,不重建容器。
|
||||
2. 固化实例/任务/备份协议;统一实例锁及主机环境锁,应用配置校验先于所有修改。
|
||||
3. 选择无外部数据库的 SiYuan 做独立部署试点,验证端口参数、反代、数据保留、多实例及失败恢复。
|
||||
4. 建立面板只读版:多主机、多实例、状态、日志、配置脱敏、备份目录和任务核对。
|
||||
5. 接入经过验收的部署/配置更新/备份动作;Gitea、Vaultwarden、Joplin 各自定义迁移与恢复策略。
|
||||
6. 单独接入主机环境维护、Xray;按需扩展 Certd 与 Portainer。
|
||||
|
||||
升级应选择明确目标版本;预览可以查询新版本,但确认后的执行不能重新解析 latest。取消选择工具只表示本次不操作,不能转换为卸载。卸载默认保留数据,销毁数据是独立动作。
|
||||
|
||||
## 8. 验收边界
|
||||
|
||||
- 只拿单个工具发布包,在准备好的环境中完成部署;同机启动第二实例且端口、数据、路由不交叉。
|
||||
- 面板或 SSH 断开后,远程任务可以核对,不自动重发有副作用的步骤。
|
||||
- 部署单个工具时,其他工具容器 ID、挂载、配置与运行状态不发生非预期变化。
|
||||
- backup/upgrade/restore/uninstall 共用锁;公共环境维护与应用变更互斥。
|
||||
- 镜像拉取失败、磁盘不足、数据库未就绪、配置漂移、Nginx 校验失败,均有明确阶段和退出状态。
|
||||
- 停机失败时禁止恢复数据库;备份失败禁止继续迁移;恢复失败明确进入需人工检查,不输出成功。
|
||||
- 恢复既验证归档也实际恢复数据库/仓库,并核验应用功能;数据行数只是辅助证据。
|
||||
- 保留现有数据路径、镜像和回滚资料;接管只建立基线,执行迁移另有计划。
|
||||
|
||||
## 9. UI 方向(用户已确认)
|
||||
|
||||
管理工具采用 Apple 风格,参考 macOS 原生工具的信息组织与交互节奏。此项是视觉与交互要求,不意味着必须改用 Swift、Electron 或其他桌面框架,也不等同于已选择本地或服务端部署形态。
|
||||
|
||||
- **整体视觉**:中性背景、克制的蓝色强调、清晰字体层级、充足留白、适度圆角与轻分隔。避免大面积渐变、过度毛玻璃和满屏同质卡片。
|
||||
- **主框架**:侧边栏负责导航;顶部持续显示当前服务器、环境与应用实例;内容区按概览、应用、环境、任务和备份组织。切换目标后清除旧预览,避免操作到上一台服务器。
|
||||
- **应用列表**:优先用紧凑列表展示名称、版本、状态和入口;详情页提供配置、日志、备份与升级等能力,未支持的动作不伪装成可用功能。
|
||||
- **操作流程**:部署按“选择工具 → 配置 → 预检与变更预览 → 执行结果”推进。配置采用分组表单,长任务使用明确的阶段和可展开日志,保留任务恢复入口。
|
||||
- **生产操作**:预览明确展示目标、影响的实例、预计停机及数据恢复范围;危险动作使用独立且具体的确认。颜色辅助提示,不能代替文字说明。
|
||||
- **可访问性**:键盘焦点可见、文本对比度充足、状态不只依靠颜色;以系统字体栈适配 Windows/macOS,不分发 Apple 专有字体资源。
|
||||
|
||||
下一步设计应先明确页面结构与核心流程,再制作少量关键页面原型;当前尚未实现 UI。
|
||||
|
||||
## 10. 官方参考
|
||||
|
||||
- Compose 项目命名与隔离:https://docs.docker.com/compose/how-tos/project-name/
|
||||
- Compose 默认网络:https://docs.docker.com/compose/how-tos/networking/
|
||||
- 服务定义与 container_name:https://docs.docker.com/reference/compose-file/services/
|
||||
- Docker daemon 权限:https://docs.docker.com/engine/security/
|
||||
- SSH 保护 Docker 访问:https://docs.docker.com/engine/security/protect-access/
|
||||
|
||||
待确认的产品选择:管理工具是本地 SSH 面板还是服务器 Web 平台。本文优先推荐前者;选择后再形成实施规格与任务拆分。
|
||||
|
||||
## 11. 第二轮评估:收敛为可实施的结构
|
||||
|
||||
### 11.1 判断与纠偏
|
||||
|
||||
第一版对独立应用、共享环境、渐进接管的方向判断成立,但仍不足以作为直接开发的架构规格:执行责任不明确,容易让 Node、CLI、Shell 各自实现一遍操作;对多实例、外部数据库、双入口提供者等扩展的优先级偏高;UI 罗列功能而没有明确使用层级。
|
||||
|
||||
现代化的衡量标准是可复现、声明清楚、可观测、失败可判定、模块可独立测试,而不是增加框架、微服务或自动发现组件。本项目适合模块化单体管理工具 + 独立应用发布包 + 明确的环境模块。
|
||||
|
||||
### 11.2 独立部署的实际边界
|
||||
|
||||
首版必须保证每个工具能单独安装、停启、升级和备份,不依赖其他业务应用及面板在线。共享 Docker、入口和系统资源意味着仍有共同故障域,不能承诺同主机上的绝对故障隔离。
|
||||
|
||||
同主机多实例作为架构约束预留(独立身份、参数化资源),先在试点应用验证;不要求首版所有工具完成多实例 UI 和复杂迁移。外部数据库作为未来扩展,不在首版同时支持所有组合。
|
||||
|
||||
Gitea + 私有 MySQL、Joplin + 私有 PostgreSQL、RustDesk 两组件分别是一个部署单元。应用镜像更新与其数据库引擎升级是不同操作,不能因全栈 pull 顺带升级数据库。
|
||||
|
||||
应用包只声明所需环境与入口,不负责无条件安装 Nginx、签发证书或重启 Docker。入口模式至少在模型中区分“受管入口”“外部入口”“仅内网”,实际支持范围由适配器声明;应用自身要求公网 HTTPS 时仍需阻止不满足要求的配置。
|
||||
|
||||
### 11.3 入口方案与部署合理性
|
||||
|
||||
当前改造基线继续使用宿主 Nginx + Certbot:保留既有证书、站点和路径,集中到环境模块管理,避免与应用迁移同时更换网关。每台主机的监听地址和端口必须有明确归属,应用只申请自己的路由。
|
||||
|
||||
Caddy 的自动 HTTPS 可以减少新服务器的证书管理代码,是合理的后续选择;但它不自动解决既有 Nginx 路由、特殊长连接、证书接管和端口迁移。因此首版不同时实现两套网关,不强迫存量服务切换。入口操作接口应保持小而具体:检查、规划站点、应用站点、验证、移除本站点绑定。
|
||||
|
||||
容器化网关也可行,但不能把所有数据库加入公共代理网络。若未来改用容器网关,需单独设计前端网络与应用私有网络。当前回环端口 + 宿主代理在此规模下足够清楚。
|
||||
|
||||
### 11.4 管理工具采用模块化单体
|
||||
|
||||
候选仍为 React + TypeScript 前端及一个 Node 本地服务;技术版本在实施时锁定。首版不引入微服务、独立消息队列、常驻远端 Agent、插件市场或 Electron/Tauri 外壳。
|
||||
|
||||
面板内部按职责组织,而不是持续扩大一个 operations.ts:
|
||||
|
||||
```text
|
||||
tools/ops-panel/src/
|
||||
domain/ # Host、AppDefinition、Instance、Plan、Operation、Backup
|
||||
application/ # 发现、接管、预览、执行、核对等用例
|
||||
infrastructure/ # SSH/SFTP、存储、应用目录读取
|
||||
http/ # 本地会话、校验、API、任务事件
|
||||
web/
|
||||
app/ # 路由、当前目标、整体布局
|
||||
features/ # 服务器、应用实例、任务、设置
|
||||
ui/ # 基础组件与设计 token
|
||||
```
|
||||
|
||||
依赖方向为 UI/API → 用例 → 领域模型;用例通过小接口使用 SSH 和存储实现。领域模型不依赖 React、HTTP 或 SSH。不要为每个目录强建独立 npm 包;出现第二个真实消费者时再提取公共包。
|
||||
|
||||
主机档案与应用实例分开:一台主机有多个实例;应用定义描述模板和能力,实例记录具体路径、配置引用和现场身份。Plan 表示有时效的目标变更;Operation 表示一次执行;Backup 表示可验证的恢复资料。不要把这些全部塞进一个不断扩展的 ServerProfile。
|
||||
|
||||
### 11.5 单一执行入口,避免三套实现
|
||||
|
||||
前端负责输入、预览和展示;本地服务负责连接、调用用例与核对任务;远端版本化运行器负责现场检查、实例锁、执行步骤和结果记录。应用适配器提供真正有差异的备份、迁移、健康验证及恢复步骤。
|
||||
|
||||
GUI 与服务器上的 CLI 都调用同一个远端执行入口。旧 deploy.sh/backup.sh/upgrade.sh 在逐项验收后才改成兼容入口;不新增一套 TypeScript Shell 字符串流程与原脚本并行维护。
|
||||
|
||||
目标流程:发现/检查 → 生成计划 → 展示变更 → 确认 → 锁内重验 → 执行 → 验证 → 记录结果。远端运行器不成为常驻服务;每个任务保留固定版本代码、计划标识、实例身份、阶段日志和退出状态,SSH 断线后继续运行或进入明确的待核对状态。
|
||||
|
||||
任务结果必须区分:成功、失败且已恢复、失败需处理、结果待核对。不能将网络断开直接标记为失败或自动重试,也不能把“发出恢复命令”当作“已恢复”。中断恢复指核对现场和受控续做,不代表每个 Shell 步骤都能从中间续跑。
|
||||
|
||||
首版同主机写任务串行、跨主机可并行,读取不取写锁;CLI 使用相同锁约定。后续证明有必要时才细分实例并行锁与环境排他锁,避免过早设计复杂调度器。手工操作及 Portainer 不会自动遵守本工具的锁,因此执行前仍要核对配置与现场漂移。
|
||||
|
||||
### 11.6 单一事实来源与分发
|
||||
|
||||
应用目录内的清单是该工具字段、能力和运行入口的唯一声明来源;Compose 是容器编排来源。面板不再手写第二套服务列表或镜像默认值。旧实例以实际挂载与项目标签建立记录,不由新模板覆盖现场。
|
||||
|
||||
发布时产出独立、版本化的应用包,包含本应用定义、Compose/模板、适配步骤与固定版本的最小运行器。源代码共享,发布包自包含;更新管理面板不隐式更新服务器应用包。
|
||||
|
||||
版本需区分面板版本、应用包版本、运行器协议版本和上游镜像 digest。计划锁定具体包与镜像;包摘要只证明内容一致,若要防范供应链替换还需要可信发布来源或签名,不能将普通 checksum 当作来源认证。
|
||||
|
||||
本地保存主机档案和任务索引,远端任务结果是执行核对依据。首版可以用经串行化和原子写入的 JSON 文件存储小规模元数据;不为几台服务器强加本地数据库。通过 Store 接口隔离,确有查询、规模或事务需求时再换 SQLite。秘密不进入任务摘要、前端持久存储或普通日志。
|
||||
|
||||
### 11.7 Apple 风格落实到信息架构
|
||||
|
||||
全局导航保持简洁:服务器、任务、设置。进入服务器后显示其应用实例与独立的运行环境页;“添加应用”进入工具目录。实例详情再提供概览、配置、日志、备份与版本历史,避免把每个动作升级为全局导航项。
|
||||
|
||||
常用操作贴近当前对象:重启在实例工具栏,升级在版本信息附近,恢复从具体备份进入。预览使用稳定的页面或侧栏,只对真正需要确认的影响做确认,不层层叠弹窗。任务状态与阶段、日志分开,真实可计算时才展示百分比。
|
||||
|
||||
视觉采用中性色、统一字级和间距、小面积状态色、系统字体、清楚的选中与焦点状态;适量材质不妨碍日志阅读。Apple 风格是一致性、层级和反馈,不是简单地增加圆角或玻璃背景。离线显示最后更新时间与“状态未确认”,不把历史绿色状态当作当前健康。
|
||||
|
||||
### 11.8 缩小首版并调整交付方式
|
||||
|
||||
第一条闭环采用一个应用:应用包 → 独立 CLI → 面板识别 → 部署预览 → 执行 → 健康验证 → 备份/恢复演练。完成后接入第二个带数据库的应用,验证抽象是否成立,再批量扩展。
|
||||
|
||||
这比先重写全部脚本、再开发一个大型面板更容易发现协议设计问题。共享环境准备作为单独入口随试点一起提供,但 Docker 版本升级、证书提供者切换、所有工具多实例和外部数据库组合不同时纳入首版。
|
||||
|
||||
验收应覆盖:只分发单工具包、另一应用持续正常、断线后任务核对、停机失败拒绝恢复、备份失败阻止迁移、失败结果真实、已有实例原路径接管。UI 原型与领域接口可以先设计,但本节仍是评估后的建议,未开始应用代码实现。
|
||||
|
||||
补充官方参考:Caddy 自动 HTTPS https://caddyserver.com/docs/automatic-https 。其能力说明不构成对当前生产网关切换的建议。
|
||||
@@ -0,0 +1,155 @@
|
||||
# 全新服务器部署与管理工具架构
|
||||
|
||||
日期:2026-09-24。状态:候选设计,待评审,不代表实现完成。
|
||||
|
||||
## 1. 已确认的约束
|
||||
|
||||
- 为全新服务器设计;不保留旧部署目录、脚本接口或迁移兼容层。
|
||||
- 工具可以独立部署,公共环境可以共享;管理面板不是应用运行依赖。
|
||||
- 优先考虑职责清晰、设计一致、可复现及数据保护。
|
||||
- UI 采用 Apple 风格。
|
||||
- claude-dev-stack 已废弃,不进入新设计。
|
||||
|
||||
本文取代旧审查报告中以渐进兼容为前提的建议。完整架构统一设计;工程实施仍需分模块验证,不等于维持两代部署体系。
|
||||
|
||||
## 2. 总体选择
|
||||
|
||||
推荐本地 React/TypeScript 管理面板 + Node 本地 API,通过 SSH 调用 Linux 上的单一运维执行器。执行器建议用 Go 编译为独立 CLI 二进制,以避免远端 Node 依赖和大规模 Shell 字符串拼接;代价是仓库需维护 TypeScript 与 Go 两种语言及对应测试。
|
||||
|
||||
公共环境包括 Docker Engine/Compose、宿主 Caddy、systemd 与必要的主机准备。Caddy 负责反向代理和自动 HTTPS;默认不再安装 Nginx + Certbot,不再由应用写续期 cron。DNS、证书签发可达性与云安全组仍需实际校验。
|
||||
|
||||
应用采用各自独立的 Compose 项目。Xray 采用 systemd 适配器,保持适合其运行方式的部署。没有总 Compose,也没有跨业务应用共享数据库。
|
||||
|
||||
本地面板形态是推荐选择,用户尚未明确指定运行位置;若选服务器 Web 平台,认证、凭据存储与多用户边界需要另行设计,但应用包和执行器协议保持相同。
|
||||
|
||||
## 3. 服务器布局与职责
|
||||
|
||||
```text
|
||||
/usr/local/bin/deployctl 唯一 CLI 入口
|
||||
/opt/server-deploy/packages/<id>/<ver>/ 只读、版本化应用包与校验信息
|
||||
/etc/server-deploy/ 主机配置、实例声明、秘密引用
|
||||
/var/lib/server-deploy/ 实例登记、任务状态、应用持久数据
|
||||
/var/backups/server-deploy/<instance>/ 备份集与恢复清单
|
||||
/etc/caddy/ 受管网关配置
|
||||
```
|
||||
|
||||
具体子路径由实例 ID 生成并校验,不允许用户输入直接成为任意写入/删除目标。应用包与可变数据分离;更新包不覆盖数据目录。秘密文件使用严格文件权限并从备份与日志展示中脱敏。
|
||||
|
||||
Gitea + MySQL、Joplin + PostgreSQL、RustDesk hbbs + hbbr 分别作为一个应用单元。它们的数据库/配套组件有明确角色,应用镜像升级不默认更新数据库引擎。
|
||||
|
||||
Compose 项目名由实例 ID 固定生成,不使用固定 container_name。同一工具的不同实例拥有不同配置、挂载、端口、项目网络和备份目录;多实例是统一模型的一部分,不依靠复制脚本修改名称。
|
||||
|
||||
## 4. 入口与网络
|
||||
|
||||
选择宿主 systemd 管理的 Caddy,HTTP 应用只向宿主回环地址发布端口,Caddy 转发到对应端口。该方案使网关不依赖 Docker socket,也不需要将不同应用的数据库放进公共网络。
|
||||
|
||||
端口由执行器分配、登记并在执行前检查占用,不能每次启动随机变化。应用对外 URL、Compose 映射和网关上游都来自同一份实例配置。计划阶段的可用端口检查不消除抢占竞争,绑定失败仍必须作为可恢复错误处理。
|
||||
|
||||
每个应用的数据库只连接本项目网络,不发布宿主端口。Compose 项目隔离不是对同一 Docker daemon 的强安全隔离;有高权限 Docker 访问的主体仍可跨项目操作。
|
||||
|
||||
Gitea SSH、RustDesk TCP/UDP、Xray 是协议直达入口,单独声明监听地址、端口、协议及防火墙要求。Xray 与 HTTPS 网关不能同时绑定同一地址的 TCP 443;部署规划应拒绝冲突,或明确使用不同端口/IP/主机。
|
||||
|
||||
网关变更由唯一入口模块管理:收集已登记路由 → 生成候选配置 → 校验 → 切换并 reload → 访问验证。失败保留旧配置及诊断。证书状态和持久数据独立保存;删除一个应用不得删除其他应用的证书或共享网关。
|
||||
|
||||
Certd 保留为可选的独立工具,用于额外证书工作流;不自动管理 Caddy 已负责的证书。Portainer 同样可选,属于高权限容器管理入口,不是本工具的依赖。
|
||||
|
||||
## 5. 应用包与单一事实来源
|
||||
|
||||
每个应用包包含:
|
||||
|
||||
- 版本化清单:参数类型、秘密标记、组件角色、入口与数据位置、支持动作。
|
||||
- Compose 模板或 systemd 模板。
|
||||
- 版本锁:明确的上游版本和平台对应镜像 digest。
|
||||
- 应用特有的备份、迁移与验证策略声明。
|
||||
- 协议版本、文件清单和完整性摘要。
|
||||
|
||||
声明以可校验的数据表达,不能退化为任意远程命令字段。复杂恢复由内置且经过测试的应用适配器处理;不为简单需求创造通用工作流编程语言。
|
||||
|
||||
面板表单和能力显示读取应用清单,不手写第二份服务列表。应用包发布与面板发布解耦;执行器在预检中校验应用包协议兼容性。校验和验证内容一致性,可信分发还需可信来源/发布签名。
|
||||
|
||||
只分发一个应用包及兼容的 deployctl,就应能在准备好的主机上通过 CLI 部署该工具。公共环境初始化是独立命令,应用部署只检查环境或生成明确的环境准备计划,不隐式整机升级。
|
||||
|
||||
## 6. 唯一运维执行器
|
||||
|
||||
deployctl 同时服务本机 CLI 与 SSH 调用,承担现场检查、规划、互斥锁、执行、健康验证及结果记录。面板后端不再生成另一套安装/回滚脚本;GUI 与 CLI 的业务操作经过同一执行路径。
|
||||
|
||||
协议使用版本化 JSON 请求/响应及阶段事件,应用核心不依赖终端彩色输出。参数作为独立进程参数传递,避免经过 shell 二次解释;日志统一脱敏。高权限操作只在明确的主机维护或应用执行路径发生。
|
||||
|
||||
远程任务通过 systemd 管理的任务单元脱离 SSH 会话,执行固定版本程序。持久记录 operation ID、计划摘要、目标身份、运行阶段及退出状态。机器重启后先核对现场,不假设数据库迁移可以自动从任意位置续跑。
|
||||
|
||||
操作顺序为:inspect → plan → approve → apply → verify。apply 在锁内重验计划条件;过期、配置变化、版本变化时要求重新规划。同一主机写任务先统一串行,跨主机可并行,读取独立执行;后续若需要并发,可在保持接口不变的前提下细分锁范围。
|
||||
|
||||
结果明确区分成功、失败已恢复、失败需处理、状态待核对。恢复数据库前必须确认写入方已停止。镜像回退、配置恢复、数据恢复是不同动作,不承诺统一“回滚一切”。
|
||||
|
||||
备份包含完成标志、源实例、版本、内容摘要、一致性模式与恢复要求;归档通过校验不等于恢复演练通过。恢复覆盖升级后数据的风险必须出现在计划中。被恢复点引用的备份禁止自动清理。
|
||||
|
||||
## 7. 管理工具结构
|
||||
|
||||
管理工具为一个模块化单体,前后端随一个便携产品发布:
|
||||
|
||||
```text
|
||||
apps/ops-panel/
|
||||
src/domain/ 主机、定义、实例、计划、任务、备份
|
||||
src/application/ 主机连接、应用目录、计划确认、任务核对
|
||||
src/infrastructure/ SSH/SFTP、应用包仓库、存储
|
||||
src/http/ 本地会话、API、任务事件
|
||||
src/web/app/ 布局、路由与当前目标
|
||||
src/web/features/ 服务器、实例、任务、设置
|
||||
src/web/ui/ 组件和设计 token
|
||||
```
|
||||
|
||||
前端不直接执行命令;HTTP 层不实现部署逻辑;连接模块不识别具体 Gitea 字段;领域模型不依赖 React 或 SSH。远端执行器是操作结果的权威来源,本地保存连接档案、用户偏好和任务索引。
|
||||
|
||||
本地持久化采用 SQLite,适合关联主机/实例/计划/任务、执行记录查询及事务写入;它是嵌入式文件,不引入额外数据库服务。具体驱动需结合便携 Node 运行时验证。SSH 优先使用 Agent/本地私钥引用,数据库中不保存私钥内容。
|
||||
|
||||
本地服务只监听回环地址,保留 Host/Origin、会话及 CSRF 校验。面板关闭不影响应用和远程任务;重新打开后按任务 ID 核对结果。
|
||||
|
||||
## 8. 新仓库目录
|
||||
|
||||
```text
|
||||
server-deploy/
|
||||
apps/ops-panel/ 本地管理产品
|
||||
cmd/deployctl/ Go CLI 入口
|
||||
internal/ 执行器内部模块
|
||||
planner/
|
||||
executor/
|
||||
state/
|
||||
runtime/ compose、systemd、gateway 适配
|
||||
applications/ 必要的应用专有策略
|
||||
catalog/ gitea、joplin、vaultwarden 等版本化应用包源
|
||||
platform/ Docker、Caddy、主机初始化与维护定义
|
||||
protocol/ 请求、计划、事件及清单的 schema
|
||||
scripts/ 构建、发布、打包;不承载另一套运维业务
|
||||
tests/ 协议一致性、隔离端到端、故障与恢复验收
|
||||
docs/ 架构决策、操作说明和应用恢复边界
|
||||
```
|
||||
|
||||
catalog 放部署对象,apps 放管理产品,platform 放公共环境,执行逻辑集中在 deployctl。旧顶层工具目录不作为兼容结构保留;实际重组在设计批准后的实施中完成,不能把设计文档新增解释为已完成重构。
|
||||
|
||||
## 9. Apple 风格与信息结构
|
||||
|
||||
主导航:服务器、任务、设置。服务器详情下分应用实例与运行环境;添加应用打开应用目录。实例详情提供概览、配置、日志、备份和版本历史,操作位于具体对象附近。
|
||||
|
||||
采用清晰留白、统一字级/圆角/间距、克制的状态色、系统字体、键盘操作、深浅色主题和可访问焦点。日志区域强调密度与可读性,不套用大面积玻璃材质。任务阶段与日志分离;离线状态显示最后核对时间。
|
||||
|
||||
部署以明确目标和差异预览为中心;默认值来自应用包,高级项逐步展开。需要数据恢复或停机的操作显示具体范围,不用重复弹窗代替设计清晰的操作流程。
|
||||
|
||||
## 10. 设计质量的验收标准
|
||||
|
||||
- 新增常规 Compose 应用主要增加 catalog 定义,而非修改 UI/连接/任务核心。
|
||||
- GUI 与 CLI 对同一请求产生相同现场计划和执行结果。
|
||||
- 应用 A 升级不变更应用 B 的配置、数据库、数据或镜像。
|
||||
- 同工具多实例的项目名、端口、数据和入口从配置生成,无手工改源码。
|
||||
- 面板失联、SSH 中断、磁盘不足、镜像不可用、停机失败和恢复失败均有可核对结果。
|
||||
- Caddy、数据库及应用数据有明确所有者与备份边界。
|
||||
- 模板、协议、执行器和界面有自动测试;真实隔离服务器覆盖部署、升级、备份及恢复。
|
||||
|
||||
共享主机仍有共同故障域。本方案提供生命周期和资源归属隔离,不声称达到多租户强隔离或多节点高可用。
|
||||
|
||||
## 官方依据
|
||||
|
||||
- Caddy 自动 HTTPS:https://caddyserver.com/docs/automatic-https
|
||||
- Compose 项目身份:https://docs.docker.com/compose/how-tos/project-name/
|
||||
- systemd transient services:https://www.freedesktop.org/software/systemd/man/latest/systemd-run.html
|
||||
|
||||
待评审:本地面板运行形态、Go 单二进制执行器以及统一 Caddy 入口均为本方案的明确推荐,尚未视为用户逐项批准的实现选型。
|
||||
@@ -0,0 +1,299 @@
|
||||
# 新服务器部署与管理工具:完整优化方案
|
||||
|
||||
日期:2026-09-25。状态:用户已批准开始实施;选定 Traefik、本地面板、Go 执行器。不代表已实现或经过服务器验收。
|
||||
|
||||
## 1. 范围与决策状态
|
||||
|
||||
已确认:为新服务器设计;不做旧服务器迁移或旧脚本兼容;每个工具独立部署;公共环境可共享;Apple 风格界面;claude-dev-stack 不进入新体系。
|
||||
|
||||
实施默认仅在本地开发和隔离测试环境进行,不连接生产服务器,不自动提交代码。之后用户已指定新服务器并授权只读初检,记录见 target-server-preflight 文档;这不代表已授权任意写操作。2026-09-25 用户要求跨设备拉取接续,本次据此整理并提交/推送新架构交接成果。此前已删除的 claude-dev-stack/.env 保持现状。
|
||||
|
||||
用户在完整方案后明确要求“按照方案,开始”,因此以本方案的 Traefik + 本地面板 + Go 执行器作为实施基线。2026-09-24 的 Caddy 方案保留为历史讨论,不与本方案同时实施。
|
||||
|
||||
推荐组合:本地 React/TypeScript 面板 + Node API + SQLite;SSH 调用 Go deployctl;远端 Docker Compose + Traefik + systemd;应用包独立分发。首版面向单管理员、可信主机,不承诺多租户强隔离或集群高可用。
|
||||
|
||||
## 2. 现有项目的结构性问题
|
||||
|
||||
依据 2026-09-24 源码审查及本轮目录核对:
|
||||
|
||||
- base 混合函数库和主机安装程序;应用部署可触发整机升级、Docker 配置覆盖与重启。
|
||||
- 应用运行单元基本独立,但脚本依赖相邻 base 目录;固定容器名和端口妨碍多实例。
|
||||
- 网关、证书、定时任务由多个应用脚本各自维护,资源归属不清。
|
||||
- 部分备份失败被忽略,一致性、完成标记、恢复演练与保留策略不足。
|
||||
- Gitea 的健康判断、停止写入后恢复、回滚结果判断等问题,不能作为新执行后端继续继承。
|
||||
- 版本浮动、文档漂移、实际秘密进入源码或历史,影响可复现与发布安全。
|
||||
|
||||
旧脚本只作为功能和故障案例来源,不包装成新管理工具的执行后端。完整重构允许按模块开发验证,但不建立长期双轨兼容系统。
|
||||
|
||||
## 3. 四个明确的职责层
|
||||
|
||||
### 3.1 控制界面
|
||||
|
||||
本地面板负责主机连接、应用选择、参数输入、差异展示、任务观察。面板停止不影响应用,也不终止已经提交的远端任务。
|
||||
|
||||
### 3.2 运维执行器
|
||||
|
||||
deployctl 是唯一业务执行入口:检查、计划、应用、备份、恢复、升级、卸载、任务核对。CLI 与面板走相同路径,不各写一套脚本。
|
||||
|
||||
建议 Go:远端独立二进制,无需安装 Node;进程调用、文件操作和状态机更易测试。代价是维护 Go 与 TypeScript 两套构建链,不以减少语言数量牺牲远端可靠性。
|
||||
|
||||
### 3.3 公共环境
|
||||
|
||||
Docker Engine/Compose、Traefik、受限 Docker API 代理、systemd 任务与计时器、必要防火墙规则。环境初始化和维护是独立操作,必须展示整机影响。
|
||||
|
||||
### 3.4 应用实例
|
||||
|
||||
各实例独立编排、配置、数据、备份、版本记录。私有数据库属于应用,不提升为跨应用共享环境数据库。
|
||||
|
||||
## 4. 网关、网络与证书
|
||||
|
||||
### 4.1 Traefik 候选的取舍
|
||||
|
||||
应用路由从包内声明生成 Compose 标签;Traefik 使用 Docker provider 动态发现。部署工具校验域名归属和标签,但不再另写一份同域名的文件路由。普通容器 HTTP 服务不发布宿主端口。
|
||||
|
||||
采用此方案的理由是应用包可携带完整入口声明,而非 Traefik 比 Caddy 更现代。若希望网关不访问 Docker API,则采用此前宿主 Caddy + 回环上游方案更简单。首版只实现其中一种,不做多网关切换平台。
|
||||
|
||||
Traefik 是独立环境 Compose 项目,静态配置变化或自身升级作为主机级维护;动态应用路由不要求重启整个网关。主机和网关故障仍可能影响全部应用 HTTP 入口。
|
||||
|
||||
### 4.2 首版网络边界
|
||||
|
||||
- 一张受控共享入口网络,仅连接 Traefik 与需要公开 HTTP 的应用组件。
|
||||
- 每个应用另有私有后端网络,数据库只连接本应用后端网络、不发布宿主端口。
|
||||
- 一张专用 Docker API 管理网络,仅连接 Traefik 与 socket proxy,不发布宿主端口。
|
||||
- Traefik 设置 exposedByDefault=false,并过滤受管应用标签;显式指定上游端口和入口网络。
|
||||
- 路由、服务名称包含实例 ID,域名绑定有唯一性校验,避免跨应用覆盖。
|
||||
|
||||
共享入口网络上的应用可以相互连接:这是首版同一管理员、同一信任域的明确取舍,不是强隔离。若未来运行不可信应用,应使用独立主机或专门设计的隔离网络,不把 Compose 项目名当安全边界。
|
||||
|
||||
socket proxy 必须按实际需要限制 Docker API 路径和方法,验收确认拒绝容器创建、exec、删除等写操作。只读挂载 socket 不是 API 授权。允许读取的容器元数据仍可能泄露秘密,因此凭据尽量使用文件挂载而非标签或环境变量。
|
||||
|
||||
代理本身仍持有高权限 socket,必须固定版本、保护网络、审查供应链;不能描述成彻底消除了 Docker 权限风险。
|
||||
|
||||
### 4.3 证书所有权
|
||||
|
||||
Traefik 内置 ACME 管理入口证书:自动申请、续期和加载。默认普通子域名证书,用户预先配置 A/AAAA;部署前检查解析及验证端口,不默认索取 DNS API 密钥。
|
||||
|
||||
泛域名或不能使用公网端口验证时,单独启用 DNS challenge;凭据采用支持的最小权限策略。DNS 验证 TXT 与网站 A/AAAA 解析是两回事。
|
||||
|
||||
acme.json 放在独立持久目录、严格权限保护、加密备份;不进入 Git/日志/普通导出。单实例管理该存储,不能靠多个 Traefik 共享该文件实现高可用。
|
||||
|
||||
Certd 可单独部署处理额外证书用途,但不同时管理 Traefik 的证书。应用删除只移除其路由,不随意编辑共享 acme.json。
|
||||
|
||||
网关管理接口默认不对公网开放,通过 SSH 隧道访问。证书监控检查实际 TLS 端点而不仅是本地文件时间。
|
||||
|
||||
### 4.4 非 HTTP 入口
|
||||
|
||||
Gitea SSH、RustDesk TCP/UDP、Xray 独立声明端口和防火墙要求,不强制套进 HTTP 代理。Xray 保持 systemd,默认单独主机或非冲突端口;同一 IP 的 TCP 443 冲突必须拒绝,不默认设计复杂协议复用。
|
||||
|
||||
## 5. 应用包与实例模型
|
||||
|
||||
AppDefinition 表示应用种类,PackageRelease 表示部署包版本,Instance 表示某主机上的实际安装,不能混为一个“服务”。
|
||||
|
||||
应用包包含 manifest、参数 schema、Compose/systemd 模板、上游版本锁、健康规则、数据位置、一致性与升级策略声明、文件摘要及可信发布信息。
|
||||
|
||||
规则:
|
||||
|
||||
- 不使用固定 container_name;Compose 项目名、资源名和路径由稳定实例 ID 生成。
|
||||
- 包版本、程序版本、数据库版本、镜像 digest 分别记录;发布时确认目标 CPU 架构支持。
|
||||
- 禁止 latest 作为可复现部署依据;升级前解析并锁定具体目标。
|
||||
- 数据目录不在包目录内;应用程序更新不覆盖持久数据。
|
||||
- UI 字段和可用动作从 schema/能力声明生成,不维护第二套应用清单。
|
||||
- 常规应用新增包即可接入;复杂备份或迁移需要内置适配器及测试,不允许包声明任意 root Shell 工作流。
|
||||
- 包只从受信来源发布;摘要是完整性检查,不是身份认证。
|
||||
- 面板、执行器、包分别发布,显式校验协议兼容,不隐式互相升级。
|
||||
|
||||
准备好公共环境后,独立应用包 + deployctl 即可通过 CLI 部署,不依赖整个仓库或面板。
|
||||
|
||||
## 6. 现有工具的目标归属
|
||||
|
||||
- Gitea:应用 + 私有 MySQL 一个部署单元;数据库大版本升级单列维护计划。
|
||||
- Joplin:应用 + 私有 PostgreSQL 一个部署单元。
|
||||
- Vaultwarden:独立应用,数据库类型在包中明确;SQLite 不可直接在线复制当作可靠备份。
|
||||
- SiYuan:独立应用,数据目录与客户端写入的一致性需要专门验证。
|
||||
- RustDesk:hbbs/hbbr 同一应用单元,保存身份密钥,验证直连/中继链路。
|
||||
- Certd:可选证书工具,不是公共入口证书的第二管理者。
|
||||
- Portainer:可选高权限管理入口;其人工操作会造成漂移,不能与本工具并发修改同一实例。
|
||||
- Xray:systemd 应用适配器;内核调优、整机网络修改属于环境维护。
|
||||
- claude:开发机辅助脚本,归类到 extras/workstation,不混进服务器应用目录。
|
||||
- claude-dev-stack:排除,不恢复、不保留适配器。
|
||||
|
||||
支持部署不自动等于支持安全升级与恢复;每项能力完成验收后才在面板开放。
|
||||
|
||||
## 7. 操作协议与任务状态
|
||||
|
||||
核心流程:inspect → plan → 确认 → 锁内复核 → apply → verify。
|
||||
|
||||
计划绑定主机身份、实例 ID、当前状态摘要、目标包/digest、端口和域名、停机与数据影响、备份要求、失败处置、有效期和计划哈希。只读检查不修改配置或秘密。
|
||||
|
||||
apply 拒绝过期或发生漂移的计划。operation ID / 幂等键在远端去重,SSH 断开后先查询原任务,不自动重做升级或恢复。
|
||||
|
||||
任务由 systemd 服务运行,不依附 SSH 会话;记录固定执行器版本、事件序号、阶段检查点和最终结果。重启后先核对现场,禁止从任意数据库迁移步骤盲目续跑。
|
||||
|
||||
首版同一主机写操作串行,跨主机可并行。自动备份、CLI 和 UI 使用同一把锁。外部人工操作无法被锁约束,必须做现场复核和漂移提示。
|
||||
|
||||
状态至少包括排队、执行中、成功、失败已恢复、失败需处理、结果待核对、已取消。取消是受控请求,仅在安全检查点停止;不能在恢复数据库时强杀进程然后宣告“取消成功”。
|
||||
|
||||
SSH 启动命令使用固定入口,JSON 经 stdin/受保护文件传递,不把用户参数拼进远程 shell。子进程使用参数数组;目录校验包括归属、规范化路径和符号链接逃逸。
|
||||
|
||||
远端写状态是权威,本地 SQLite 是连接档案、任务索引与视图缓存。协议含版本号,日志统一脱敏,不记录秘密请求体。
|
||||
|
||||
## 8. 数据保护、备份和恢复
|
||||
|
||||
### 8.1 备份契约
|
||||
|
||||
先确认源实例和数据范围、可用空间与恢复所需版本;按适配器冻结全部写入方,或使用明确可证明的一致性机制。
|
||||
|
||||
- Gitea:统一考虑数据库、Git 仓库、LFS、附件、配置与秘密;Git SSH/后台任务同样是写入方。仅做数据库 dump 不算完整备份。
|
||||
- Joplin:数据库及配置;如支持额外外部存储,必须纳入范围。
|
||||
- Vaultwarden:SQLite 使用受支持备份方式或停机备份,同时覆盖附件等文件。
|
||||
- SiYuan、RustDesk、Certd、Portainer:按各自数据格式设计停写和导出策略,不能套一个在线 tar。
|
||||
|
||||
临时输出 → 检查导出退出码和内容 → 生成 manifest/摘要 → 原子标记完成。失败备份不可选为恢复点,不触发成功备份清理。
|
||||
|
||||
manifest 记录源身份、时间、包与应用/数据库版本、数据范围、一致性方式、工具版本和文件摘要。区分“生成成功”“完整性检查通过”“实际恢复验证通过”。
|
||||
|
||||
### 8.2 保留与异地
|
||||
|
||||
建议初始策略:每日一次、保留 7 个日备份和 4 个周备份;按数据规模、变更频率与停机窗口调整。每日一次意味着潜在丢失接近一天的数据,不等于零数据损失。
|
||||
|
||||
至少有一份加密异机/对象存储副本;本机备份不防磁盘损坏和主机丢失。备份目标和凭据由用户配置,禁止自动采购或上传到未指定外部服务。
|
||||
|
||||
在确认可用替代恢复点前不得清除最后可恢复备份;升级恢复点、正在恢复或被引用备份禁止自动清理。加密密钥应有独立安全副本,否则备份可能无法恢复。
|
||||
|
||||
定时任务运行在远端,不依赖面板常开。远端通知使用用户配置的通道;没有通道时不能声称面板关闭仍会主动提醒。
|
||||
|
||||
### 8.3 恢复契约
|
||||
|
||||
默认先恢复到隔离实例验证,避免直接覆盖当前数据。原地恢复必须明确展示会覆盖备份时间之后的新增数据,并生成当前状态救援备份。
|
||||
|
||||
恢复前确认全部写入方确已停止;停止失败立即中止。完整恢复进入干净目标,禁止用合并解压假装精确还原。校验权限、秘密、版本兼容后启动并执行功能验证。
|
||||
|
||||
恢复后的邮件、Webhook、定时任务和公网入口在演练环境默认禁用,防止恢复测试触发真实外部操作。
|
||||
|
||||
归档摘要通过只能证明文件未变化;数据库行数不能证明业务完整。RTO 通过实际恢复计时测量后给出,不提前承诺。
|
||||
|
||||
## 9. 升级、回退与卸载
|
||||
|
||||
升级前检查受支持升级路径和迁移要求,下载并验证目标包/镜像,预估空间与停机时间,再备份、迁移、启动、验证。不能先停业务才发现镜像不可下载。
|
||||
|
||||
健康判定包含进程/容器状态、应用就绪、预期 HTTP 状态与必要功能探测;500 不是健康,200 登录页面也不等于所有功能正常。
|
||||
|
||||
Gitea 验收需覆盖登录、读取仓库、clone、受控测试仓库 push、附件/LFS(启用时)及重启后数据。其他应用按自身业务定义探测,不复用单一 HTTP 判断。
|
||||
|
||||
镜像回退、配置还原、数据恢复是三个独立能力。发生不可逆数据库迁移后不能自动降级旧镜像;自动补偿仅适用于适配器明确支持的步骤。无法证明可恢复时保持诊断并请求人工处理,不输出“已回滚成功”。
|
||||
|
||||
默认卸载只停止并移除实例运行资源与自有路由,保留数据和备份。清除数据是独立高风险动作,要求目标实例确认、路径归属验证及备份检查;不删除共享网关、其他实例、宿主证书或全局卷。
|
||||
|
||||
## 10. 管理工具架构与 UI
|
||||
|
||||
模块化单体,不拆微服务。React/TypeScript + Vite 前端,Node 本地 API,SQLite 元数据;具体运行时/依赖版本在实施时选定并锁定,完成 Windows/macOS 便携包验证。
|
||||
|
||||
交付形式为本地服务配合浏览器页面,不是 Electron/Tauri 桌面安装 App,也不是公网管理网站。
|
||||
便携目标为解压后启动、随包携带运行时;启动器、端口/数据目录和退出行为尚待实现,不能当成现有能力。
|
||||
|
||||
模块边界:
|
||||
|
||||
- domain:Host、AppDefinition、PackageRelease、Instance、Plan、Operation、Backup、RouteBinding。
|
||||
- application:连接、计划确认、任务核对、备份查询等用例,不执行任意 shell。
|
||||
- infrastructure:SSH/SFTP、SQLite、包仓库、凭据引用。
|
||||
- http:会话、验证、API、事件传输,不包含部署策略。
|
||||
- web:页面、表单、任务状态和设计系统。
|
||||
|
||||
SSH 主机指纹首次信任需确认,变化阻止连接;优先 Agent 或密钥文件引用,不保存私钥内容。API 只监听回环,校验 Host/Origin、会话与 CSRF;本地服务也不是无认证裸 API。
|
||||
|
||||
参考 docker-ops-panel 的连接验证、会话保护、远程任务追踪和便携分发方式,不复制其业务固定服务列表、单项目连接模型或巨型操作模块。
|
||||
|
||||
Apple 风格信息架构:全局服务器/任务/设置;服务器内应用/环境/入口与证书;实例内概览/配置/日志/备份/版本。添加应用是目录与参数流程,而不是额外复杂全局导航。
|
||||
|
||||
视觉以系统字体、留白、克制色彩、统一间距和圆角、浅深色、键盘焦点为主;日志使用高可读密度,不做满屏玻璃效果。不可计算的任务不伪造百分比;离线显示最后核对时间。
|
||||
|
||||
部署和恢复展示真实目标、差异、停机与数据影响。危险确认只用于真正有风险的动作;不靠频繁弹窗代替清楚的权限和资源归属。
|
||||
|
||||
## 11. 源码与服务器目录
|
||||
|
||||
源码:
|
||||
|
||||
```text
|
||||
apps/ops-panel/ 本地管理产品
|
||||
cmd/deployctl/ Go CLI
|
||||
internal/
|
||||
planner/ 计划、前提与差异
|
||||
executor/ 状态机、锁、补偿
|
||||
state/ 远端登记、事件与恢复核对
|
||||
runtime/ Compose、systemd、网关适配
|
||||
applications/ 应用专用备份、升级和验证
|
||||
security/ 路径、秘密、包验证
|
||||
catalog/<app>/ 清单、模板、版本锁与应用测试资料
|
||||
platform/ 环境定义和模板
|
||||
protocol/ JSON schema 和协议兼容测试
|
||||
extras/workstation/ 非服务器辅助工具
|
||||
scripts/ 构建发布;不放第二套运维逻辑
|
||||
tests/ 集成、故障注入、恢复与端到端测试
|
||||
docs/ 架构决策、操作手册、恢复手册
|
||||
```
|
||||
|
||||
远端:
|
||||
|
||||
```text
|
||||
/usr/local/bin/deployctl
|
||||
/opt/server-deploy/packages/<app>/<version>/
|
||||
/etc/server-deploy/instances/<id>/
|
||||
/etc/server-deploy/secrets/<id>/
|
||||
/var/lib/server-deploy/instances/<id>/
|
||||
/var/lib/server-deploy/operations/<id>/
|
||||
/var/lib/server-deploy/platform/traefik/
|
||||
/var/backups/server-deploy/<instance>/<backup>/
|
||||
```
|
||||
|
||||
配置、包、秘密、数据、备份、日志分别管理权限和保留策略。不把整个仓库上传到服务器。应用进程不应可修改执行器、包信任配置或操作记录。
|
||||
|
||||
## 12. 主机安全与供应链
|
||||
|
||||
首版只承诺一套经过端到端验收的 Linux LTS 系统和 CPU 架构,其他组合通过兼容矩阵扩展,不声称支持所有发行版。
|
||||
|
||||
bootstrap 是经过审查的显式管理员操作,安装可信、固定版本的执行器和环境。不把任意用户可写路径的程序加入无限 sudo;若后续实现受限运维账户,需要验证包、路径和模板不能绕过权限边界。
|
||||
|
||||
整机升级、Docker 重启、sysctl 和 SSH/防火墙修改单独计划。防火墙操作保留现有 SSH 通路;验证云安全组与 Docker 实际端口暴露,不能只看主机防火墙配置。
|
||||
|
||||
秘密不进 Git/前端存储/标签/日志;历史已泄露的有效凭据需要轮换,删除文件不等于消除泄露。密钥轮换和历史清理另行授权执行。
|
||||
|
||||
发布采用白名单打包、依赖锁定、构建校验、秘密扫描、受信签名或分发通道。容器尽可能降权、限制资源并配置日志轮转;具体只读根目录/能力限制按应用实测,不盲目统一启用。
|
||||
|
||||
## 13. 实施工作包与交付顺序
|
||||
|
||||
这是完整新架构的工程顺序,不是存量渐进迁移:
|
||||
|
||||
1. 冻结三项关键选择,确定协议、清单、实例和备份契约;补全威胁模型及验收用例。
|
||||
2. 构建 deployctl、任务存储、锁、计划复核和可信分发;验证断线、重启和幂等。
|
||||
3. 实现公共环境及选定网关;验证网络权限、解析、证书续期与入口隔离。
|
||||
4. 用 Gitea 完成 CLI 部署—备份—升级—隔离恢复闭环,用 Joplin 验证第二种数据库适配。
|
||||
5. 建立本地面板与 Apple 风格设计系统,将已验证操作接入,不把 UI 作为执行逻辑试验场。
|
||||
6. 接入其余应用和 Xray,能力按实际验收逐项开放。
|
||||
7. 完成便携打包、灾难恢复、文档和全新服务器安装验收;删除旧部署实现并切换根 README,不保留旧兼容入口。
|
||||
|
||||
每个工作包可单独测试和审查,协议约束贯穿始终。不是先把所有旧脚本复制进新目录再统一包装。
|
||||
|
||||
## 14. 发布验收门槛
|
||||
|
||||
- 单个应用包在干净主机上可独立安装;不依赖面板或相邻仓库目录。
|
||||
- 同应用两个实例并存,名称、端口、数据、域名、备份不冲突。
|
||||
- 应用 A 升级不改应用 B 数据和编排;共同环境维护明确报告共享影响。
|
||||
- 无公网数据库端口、Docker API 或未保护的网关管理接口;socket proxy 写请求确实被拒绝。
|
||||
- ACME 测试环境验证申请/续期路径,实际 TLS 握手验证加载结果;不靠频繁生产签发做测试。
|
||||
- 注入磁盘不足、备份失败、停止失败、镜像拉取失败、数据库迁移失败、恢复失败,结果真实且可核对。
|
||||
- SSH 中断/面板关闭/主机重启后不会重复执行升级,任务能重新关联或明确标记待处理。
|
||||
- 每种声明支持恢复的应用都有真实恢复演练记录,检查业务数据和身份密钥,不只检查归档。
|
||||
- 默认卸载保留数据,清理只涉及明确自有目标;操作记录和安全测试覆盖路径逃逸。
|
||||
- 本地界面访问控制、SSH 指纹、日志脱敏、包完整性与来源验证有自动测试。
|
||||
- 前端单元/E2E、协议一致性、Go 单元/集成、隔离 Linux 主机验收全部通过,并区分未测项。
|
||||
|
||||
不能承诺“绝不丢数据”或“所有升级自动回滚”。可以承诺的工程目标是:危险动作有明确边界、恢复点真实可用、失败不伪装成功、每项能力有可复核验收证据。
|
||||
|
||||
## 15. 官方参考
|
||||
|
||||
- [Traefik Docker provider 与安全边界](https://doc.traefik.io/traefik/reference/install-configuration/providers/docker/)
|
||||
- [Traefik ACME 与自动续期](https://doc.traefik.io/traefik/reference/install-configuration/tls/certificate-resolvers/acme/)
|
||||
- [Compose 项目身份](https://docs.docker.com/compose/how-tos/project-name/)
|
||||
- [Caddy 候选的自动 HTTPS 能力](https://caddyserver.com/docs/automatic-https)
|
||||
|
||||
这些文档支持产品能力说明;本项目的目录、权限、流程和测试门槛是设计建议,不是已经完成的实现。
|
||||
@@ -0,0 +1,35 @@
|
||||
# 新服务器只读初检
|
||||
|
||||
目标:`root@121.199.168.54`。用户明确指定此服务器,并核对了控制台主机指纹。
|
||||
本记录是初步现场观察,不是部署验收,不保证主机上没有其他数据。
|
||||
|
||||
## 身份与范围
|
||||
|
||||
- ED25519:`SHA256:7sEaEMqm0ZqfIRkTIQ/7a3wgMfo9mS0JcDBWWmDvBf8`。
|
||||
- 已在本机 SSH known_hosts 记录信任;后续连接使用严格主机校验。
|
||||
- 使用已有密钥登录成功,未读取或输出私钥。
|
||||
- 远端只执行系统、服务、端口、软件包和目录元数据查询。
|
||||
- 未安装软件、上传执行器、修改防火墙、重启服务、迁移或删除数据。
|
||||
|
||||
## 实际观察
|
||||
|
||||
- Ubuntu 26.04.1 LTS,Linux 7.0.0-31-generic,x86_64。
|
||||
- 2 个逻辑 CPU,内存 3622 MiB,无 swap。
|
||||
- 根分区 ext4,约 79 GiB,总可用约 73 GiB,inode 使用约 3%。
|
||||
- systemd 状态 running,没有 failed 单元;NTP 已同步,Asia/Shanghai 时区。
|
||||
- 监听包含 SSH 22、本机 DNS、DHCP 与时间服务;80/443 未监听。
|
||||
- 没有找到 Docker 命令;dpkg 未发现 docker-ce、docker.io、containerd.io、
|
||||
containerd、docker-compose-plugin、nginx 或 certbot 软件包。
|
||||
- UFW inactive;nft 查询未输出规则。云安全组不属于这些主机查询的覆盖范围。
|
||||
- /opt、/srv 存在,未发现一级子目录;常见 Docker、containerd、server-deploy、
|
||||
Nginx、Let's Encrypt 数据/配置根目录未发现。未扫描全盘或读取应用文件。
|
||||
|
||||
## 部署前仍需完成
|
||||
|
||||
1. 校验 Ubuntu 26.04 的 Docker 官方支持及安装来源,固定并验证版本。
|
||||
2. 完成 Compose 安全策略、执行器运行前检查、环境安装和失败处理实现。
|
||||
3. 确定首批应用、域名、ACME 联系邮箱及备份目标。
|
||||
4. 核对域名解析和云安全组;主机未监听不等于公网可达。
|
||||
5. 正式写操作前重新检查现场,展示并确认环境安装影响。
|
||||
|
||||
本次没有对旧服务器 121.43.180.59 执行任何操作。
|
||||
@@ -0,0 +1,99 @@
|
||||
# 新设备开发与验证
|
||||
|
||||
先看根目录 [HANDOFF.md](../HANDOFF.md)。以下命令在仓库根目录执行,不依赖原开发者的磁盘路径。
|
||||
这里只构建/测试新执行器,不调用旧部署脚本,不连接服务器,不安装 Docker。
|
||||
|
||||
## 环境前提
|
||||
|
||||
- Git 和 SSH 客户端:克隆工作分支并访问仓库。服务器 SSH 权限与仓库权限是两件事。
|
||||
- Go:`go.mod` 要求 1.26.0 或更新版本;本次基线实际使用 Go 1.27.1。
|
||||
- Node.js:仅用于无第三方依赖的 CLI 冒烟测试;本次 Windows 基线为 v24.19.0。
|
||||
尚无面板 package.json,不需要 `npm install`。
|
||||
- Windows:PowerShell;Linux:Bash。Linux 专有行为使用 Ubuntu/WSL 验证。
|
||||
- Linux `go test -race` 需要可用的 C 编译器/CGO;检查 `go env CGO_ENABLED` 和 `cc --version`。
|
||||
- 在线仓库测试另外需要 curl、GnuPG 的 `/usr/bin/gpg` 与 `/usr/bin/gpgv`、Python 3、awk、tar 和常规核心工具。
|
||||
|
||||
从组织认可的软件渠道或 [Go 官方下载](https://go.dev/dl/) / [Node 官方下载](https://nodejs.org/en/download)
|
||||
获取工具链;核对下载摘要。不要复制旧机器临时目录里的未知二进制。
|
||||
上述版本是实测基线,不表示所有更新版本已验收。macOS 便携产品尚未验证;完整 Linux 专有测试需 Linux。
|
||||
|
||||
## Windows:基础离线检查
|
||||
|
||||
确认 `go` 和 `node` 已在 PATH。新设备无需沿用原设备的 GOCACHE、临时 Go 目录或磁盘盘符。
|
||||
|
||||
```powershell
|
||||
go version
|
||||
node --version
|
||||
go test ./... -count=1
|
||||
if ($LASTEXITCODE -ne 0) { throw 'Go tests failed' }
|
||||
go vet ./...
|
||||
if ($LASTEXITCODE -ne 0) { throw 'Go vet failed' }
|
||||
go build -o dist/deployctl.exe ./cmd/deployctl
|
||||
if ($LASTEXITCODE -ne 0) { throw 'Build failed' }
|
||||
node --test tests/cli-smoke.test.mjs
|
||||
if ($LASTEXITCODE -ne 0) { throw 'CLI smoke tests failed' }
|
||||
./dist/deployctl.exe version
|
||||
Get-Content -Raw protocol/examples/plan-request.json | ./dist/deployctl.exe plan
|
||||
```
|
||||
|
||||
Windows 可能因权限限制跳过符号链接测试;Linux 必须补验。仓库 `.gitattributes` 已将 Shell 脚本固定为 LF,
|
||||
不要把脚本转为 CRLF。Git 换行提示不等于测试失败;实际错误须单独排查。
|
||||
|
||||
## Linux/WSL:基础离线检查
|
||||
|
||||
```bash
|
||||
set -euo pipefail
|
||||
go version
|
||||
node --version
|
||||
go test ./... -count=1
|
||||
go vet ./...
|
||||
go test -race ./... -count=1
|
||||
go build -o dist/deployctl ./cmd/deployctl
|
||||
node --test tests/cli-smoke.test.mjs
|
||||
./dist/deployctl version
|
||||
./dist/deployctl plan < protocol/examples/plan-request.json
|
||||
./dist/deployctl preflight
|
||||
```
|
||||
|
||||
`preflight` 报告运行它的本机/WSL,不是云服务器。它可以报告“不支持/有阻塞项”,不代表测试失败,
|
||||
也不得为消除阻塞项擅自修改机器。WSL 临时目录可能随实例生命周期消失,不能作为跨会话依赖。
|
||||
|
||||
另有 `bash scripts/verify-linux.sh /绝对路径/已验证的Go.linux-amd64.tar.gz`:
|
||||
会解包到新建临时目录、使用独立缓存执行 Go 全量测试/vet/race/构建及 CLI 只读检查。
|
||||
它不下载 Go,不自动校验调用者提供的 Go 压缩包,不运行 Node 冒烟;调用前验证官方压缩包摘要,
|
||||
Node 冒烟按上面命令另跑。该脚本按 amd64 测试流程提供,其他架构没有同等实测证据。
|
||||
|
||||
## 可选在线测试:不要默认运行
|
||||
|
||||
先完成离线检查。以下测试只在本地下载公开测试资料,不使用 SSH、不运行 APT、不解包或安装 deb。
|
||||
需访问 Docker 官方仓库;网络、上游签名/密钥或元数据有效期变化都可能使测试失败,不能绕过校验。
|
||||
|
||||
```bash
|
||||
set -euo pipefail
|
||||
go build -o dist/deployctl ./cmd/deployctl
|
||||
# 只验证官方仓库签名、索引及篡改拒绝
|
||||
bash scripts/probe-docker-repository.sh "$PWD/dist/deployctl"
|
||||
# 另外下载约 90 MB(随测试版本变化)的五个 deb,验证并故意损坏其中一个
|
||||
DEPLOYCTL_ONLINE_ARTIFACT_PROBE=1 bash scripts/probe-docker-repository.sh "$PWD/dist/deployctl"
|
||||
```
|
||||
|
||||
需要连同 Go 全量 Linux 检查及真实签名单元测试一起运行时:
|
||||
|
||||
```bash
|
||||
DEPLOYCTL_ONLINE_REPOSITORY_PROBE=1 DEPLOYCTL_ONLINE_ARTIFACT_PROBE=1 \
|
||||
bash scripts/verify-linux.sh /绝对路径/已验证的Go.linux-amd64.tar.gz
|
||||
```
|
||||
|
||||
命令末尾的临时目录仅用于该次诊断。负向测试会故意篡改资料,不能当成安装输入或备份。
|
||||
日志中出现预期的 `artifact verification failed` / `repository verification failed` 后,
|
||||
应跟随篡改已拒绝的提示且整个脚本退出码为 0;不能只看一条成功输出。
|
||||
未提供真实元数据夹具时,`TestStagedSignatureIntegration` 会跳过,这是明确的在线覆盖缺口,
|
||||
不是已完成在线验收。
|
||||
|
||||
## 接续纪律
|
||||
|
||||
- 当前没有可运行的面板/便携包;开发依赖与未来用户免安装的产品依赖不要混淆。
|
||||
- 先失败测试、再实现;用真实 CLI 和隔离 Linux 环境覆盖涉及的边界。
|
||||
- 未通过验证、不支持或未检查的能力须保持不可执行;不要临时开放 `apply` 以便演示。
|
||||
- 每次交接更新 HANDOFF 和实施状态,记录运行环境、命令、结果、跳过项及待确认事项。
|
||||
- `dist/`、下载包、缓存、日志、SSH 密钥与真实配置不纳入源码提交。
|
||||
@@ -0,0 +1,210 @@
|
||||
# 新架构实施状态
|
||||
|
||||
更新:2026-09-25。这是工程状态,不是生产部署验收报告。
|
||||
|
||||
跨设备接续从 [HANDOFF.md](../HANDOFF.md) 开始;工具链准备及可复现命令见[开发指南](development.md)。
|
||||
本文件各工作包的测试结论属于相应阶段的历史证据,新设备仍需复跑。
|
||||
|
||||
## 已固定的选择
|
||||
|
||||
Traefik、本地 React/Node 管理面板、Go deployctl。当前目录开发,分支为
|
||||
`codex/new-deployment-architecture`,不自动提交或推送。
|
||||
|
||||
## 第一工作包:只读执行器基础
|
||||
|
||||
已实现:
|
||||
|
||||
- 独立 Go module,标准库实现,无第三方 Go 依赖。
|
||||
- `version`、`plan`、`verify-plan` CLI;JSON stdin/stdout 协议。
|
||||
- 严格输入校验:字段大小写、未知/重复/缺失字段、null、类型、体积限制。
|
||||
- 实例标识、DNS 名称和 SHA-256 摘要校验。
|
||||
- 实例项目名/数据路径推导、15 分钟有效期、内容绑定和状态漂移拒绝。
|
||||
- 所有输出标记为不可执行的离线预览;未提供任何部署写命令。
|
||||
- Go 单元测试以及真实二进制 Node 冒烟测试。
|
||||
|
||||
此阶段输入中的主机身份和状态摘要由调用方提供,不代表远程探测。
|
||||
单镜像摘要只是预览基础字段,正式应用包必须完整绑定所有组件镜像。
|
||||
摘要不是签名;只读校验通过不是写操作授权;路径拼接不是安全文件系统访问实现。
|
||||
|
||||
## 验证记录
|
||||
|
||||
- 在 Windows 使用临时 Go 1.27.1 官方工具链(下载 SHA-256 校验通过)。
|
||||
- `go test ./... -count=1` 通过。
|
||||
- `go vet ./...` 通过。
|
||||
- Windows amd64 二进制构建、运行预览、校验、拒绝 apply 通过。
|
||||
- `node --test tests/cli-smoke.test.mjs` 通过;确认测试工作目录无新增文件。
|
||||
- 独立审查未发现 Critical/Important 问题;发现时间格式接受范围过宽,已通过先失败后通过的回归测试收紧为 UTC 秒级格式。
|
||||
- Linux amd64 / CGO_ENABLED=0 交叉编译通过;未在 Linux 执行。
|
||||
- 构建产物位于忽略目录 dist,不纳入源码发布。
|
||||
|
||||
## 第二工作包:任务状态与互斥基础
|
||||
|
||||
已实现 internal/state 内部模块,尚未接入 CLI 写操作:
|
||||
|
||||
- Linux flock / Windows LockFileEx 主机目录互斥,支持真实进程终止后释放锁。
|
||||
- 操作 ID 与计划摘要绑定;重复请求返回原记录,不能改绑计划。
|
||||
- 修订号比较确保只有一个调用方能领取任务;未决任务阻止其他写任务。
|
||||
- 状态机区分成功、失败已恢复、需处理、未知及取消;终态不可重跑。
|
||||
- 有界、带完整性摘要的快照;私有临时文件、同步、替换及 Linux 目录同步。
|
||||
- 写入失败后冻结会话,必须重新打开并核对现场;不伪装保存成功。
|
||||
- 初始化标记避免已有快照丢失后静默重置;新任务入库预留后续状态空间。
|
||||
- 不自动清理幂等历史,不自动重跑中断任务;尚无事件审计日志和任务监督器。
|
||||
|
||||
验证使用临时目录,不涉及应用数据。Windows 测试通过;两项符号链接用例在
|
||||
Windows 因创建链接权限不足跳过,在 Ubuntu WSL 中实际通过。Linux 全套测试、
|
||||
状态模块竞态检测、go vet 和真实 CLI 运行已完成。进程终止测试不等于断电测试。
|
||||
独立审查指出接近容量上限时缺少终态空间,已用 113,357 条历史记录复现并修复。
|
||||
|
||||
## 第三工作包:应用包校验与本机只读检查
|
||||
|
||||
已实现:
|
||||
|
||||
- 共享严格 JSON 解码,递归校验组件/文件数组,拒绝未知字段、null 和非法 UTF-8。
|
||||
- 应用包清单绑定所有组件镜像摘要、平台、入口和文件内容摘要。
|
||||
- 校验调用方提供的清单摘要,精确检查文件清单;拒绝缺失、多余、越界路径、符号链接和特殊文件。
|
||||
- 清单、单文件、总内容和文件数量均设上限;返回已验证字节,供后续适配器使用。
|
||||
- `verify-package` 仅输出清单和结果,不输出文件正文,不授予执行权限。
|
||||
- `inspect` 只读取本机平台和 systemd/Docker 客户端文件状态,不启动进程或连接 Docker。
|
||||
- 未检查的条件明确列出,`deploymentReady` 始终为 false。
|
||||
|
||||
调用方必须通过可信渠道获取期望摘要,并保证暂存目录及其父目录可信、校验期间不被修改。
|
||||
内容摘要匹配不是发布者身份认证;本批不解析 Compose 语义,也不证明镜像能运行。
|
||||
本机文件存在不等于服务健康,尚未检测 Docker daemon、Compose 版本、网络或磁盘容量。
|
||||
|
||||
本批验证:Windows 全套测试、go vet、构建及两项真实 CLI 冒烟测试通过;Ubuntu WSL
|
||||
全套测试、go vet、全模块竞态检测、构建和真实 plan/inspect 运行通过。
|
||||
Linux 实际执行了符号链接和 FIFO 拒绝测试。第一次 Linux 竞态构建曾报标准库 fmt
|
||||
导入异常;待并行开发结束后以全新工具链目录和缓存重跑通过,异常未再复现,根因未确认。
|
||||
独立审查发现默认 Docker 路径无效或不可读时漏查备用路径,已先复现三个失败用例再修复;
|
||||
复审无遗留发现,修复后 Windows/Linux 全套检查再次通过。
|
||||
|
||||
## 第四工作包:受限 Compose 策略基础
|
||||
|
||||
- 新增 `check-package`:先校验完整应用包,再检查内存中的入口字节,不重新打开文件。
|
||||
- `isolated-compose-v1` 仅接受严格 JSON;字段白名单拒绝未覆盖功能。
|
||||
- 服务与清单组件/镜像精确匹配;要求显式非 root UID:GID、只读根文件系统、
|
||||
丢弃全部 capabilities 和 no-new-privileges。
|
||||
- 首版仅允许 backend 内部网络及普通项目命名卷;拒绝宿主挂载、外部资源、
|
||||
跨服务共享卷、重叠挂载、系统目录挂载、Docker socket、主机网络、额外服务和构建/钩子。
|
||||
- 扩展共享 JSON 解码器,递归检查动态服务字典的未知字段、缺失字段和 null。
|
||||
- 包摘要与策略名在结果中明确返回,`executable` 和发布者认证始终为 false。
|
||||
|
||||
这是策略基础,不是完整 Compose 适配器。环境/秘密注入、Traefik 路由、健康规则、
|
||||
应用权限例外、卷初始化权限和资源上限尚未实现;当前策略无法直接覆盖 Gitea 等完整应用。
|
||||
它不验证镜像内部路径/默认行为,不证明 Docker 资源归属,也不替代实际部署/恢复测试。
|
||||
精确契约见 [策略说明](../internal/composepolicy/README.md)。本批未连接或修改云服务器。
|
||||
|
||||
验证:先观察策略缺失、字典递归校验缺失和 CLI 命令缺失导致测试失败,再实现通过。
|
||||
Windows 全套 Go 测试、go vet、构建和 3 项真实 CLI 冒烟测试通过;Ubuntu WSL
|
||||
全套测试、go vet、全模块竞态检测、构建及现有只读 CLI 检查通过。
|
||||
独立审查未发现本批范围内的问题,并独立重跑 composepolicy/wire/cli 测试通过。
|
||||
尚未用真实 Docker/Compose 加载或运行策略示例,不能视为容器运行验收。
|
||||
|
||||
## 第五工作包:本机预检与环境安装提案
|
||||
|
||||
- 新增只读 `preflight`,输出实际观察、阻塞项、候选步骤和影响;不是可执行安装计划。
|
||||
- 在 Linux 读取有限大小的 os-release、有效 UID 和 /var/lib 所在文件系统可用空间。
|
||||
- 对常见运行时/数据/安装源路径作保守存在性检查,链接或不可读路径不视作干净环境。
|
||||
- 仅在支持的 Ubuntu 版本/代号组合、root、systemd、磁盘初筛及资源检查通过时显示候选步骤。
|
||||
- 明确指出软件包安装可能启动 Docker 并影响主机网络规则;不生成或执行 Shell 命令。
|
||||
- 版本锁、软件包清单、源信任、主机身份和网络检查未完成,任何提案始终不可执行。
|
||||
|
||||
目前只是现场观察和安装提案基础,尚未实现完整运行前检查、APT 事务计划或环境安装。
|
||||
5 GiB 是安装初筛阈值,不是应用/备份容量保证。没有扫描所有安装源或非标准数据路径。
|
||||
本批未上传或在云服务器执行新二进制;CLI 只报告它实际运行的本机环境。
|
||||
详细边界见 [预检说明](../internal/preflight/README.md)。
|
||||
|
||||
验证:Windows 全套测试、go vet、构建和 4 项真实 CLI 冒烟测试通过;Ubuntu WSL
|
||||
全套测试、go vet、竞态检测、构建和实际 preflight 输出通过。Linux 用临时目录验证了
|
||||
真实 statfs 查询和悬空符号链接拒绝作为“空环境”的行为,未涉及应用数据。
|
||||
独立静态审查未发现本批范围内的问题;未完成真实软件包安装验收。
|
||||
|
||||
## 第六工作包:dpkg 清单、版本锁与安装事务草案
|
||||
|
||||
- 只读检查 dpkg 状态及更新日志目录,报告 Docker/容器运行时相关包和状态文件摘要。
|
||||
- 相关包无论已安装、残留配置或部分安装均阻止全新安装候选;读取失败不视为未安装。
|
||||
- 新增 `plan-environment`,输入明确的 Docker 五包版本锁,现场信息由本机收集。
|
||||
- 限制官方源地址、发行版、架构、文件路径、版本、大小和 SHA-256;Engine/CLI 版本须一致。
|
||||
- 草案绑定锁文件与现场观察摘要,列明请求包、影响和阻塞项;不执行下载、apt 或配置写入。
|
||||
|
||||
本批实现的是源地址约束和调用方版本锁校验,不是仓库签名认证。仍未验证签名 Release、
|
||||
Packages 到 deb 的摘要链、元数据时效、实际文件或完整依赖事务;始终返回不可执行且源未认证。
|
||||
未从在线仓库选择真实安装版本;没有对云服务器执行安装、升级或数据改动。
|
||||
|
||||
验证:Windows 全套 Go 测试、go vet、构建和 5 项真实 CLI 冒烟测试通过;Ubuntu WSL
|
||||
全套测试、go vet、竞态检测、构建及真实 dpkg 清单读取通过。独立审查指出两项测试
|
||||
被提前校验掩盖,已修正输入并通过临时停用对应校验的反向测试证明能捕获错误;
|
||||
恢复生产校验后全套复验通过,复审无遗留发现。未执行真实 APT 安装事务或仓库签名验收。
|
||||
|
||||
## 第七工作包:官方仓库签名与元数据摘要链
|
||||
|
||||
- 新增 `verify-repository`:在 Linux 验证暂存的 Docker 公钥、Release 签名和 Packages 摘要。
|
||||
- 公钥字节摘要及主指纹固定在代码中;使用隔离临时目录调用系统 GnuPG,不修改 APT 或个人密钥环。
|
||||
- 校验发行版、架构、stable 组件、元数据日期与有效期,再按五个明确版本生成绑定 Release 的版本锁。
|
||||
- 文件大小、类型、重复字段/记录、无效路径及未知签名状态均受限制;错误不回显原始输入。
|
||||
- 认证输出不是执行授权;实际 deb 文件尚未下载验证,`packageBytesVerified`、`executable` 仍为 false。
|
||||
|
||||
真实 Docker resolute/amd64 元数据在本地 WSL 验证通过,篡改 Release 和 Packages 的端到端
|
||||
CLI 测试均被拒绝。验证所选版本仅作测试样本,不代表最终安装版本。没有访问或修改云服务器,
|
||||
没有安装软件、变更防火墙或接触应用数据。GnuPG 只写入本次创建的临时目录。
|
||||
该校验依赖可信系统时钟、GnuPG 和无并发写入的可信暂存目录;30 天时效窗口不是持久防回退账本。
|
||||
详细契约见 [仓库验证说明](../internal/aptrepo/README.md)。
|
||||
|
||||
验证:Windows 全套 Go 测试、go vet、构建及 6 项真实 CLI 冒烟测试通过;Linux 全套测试、
|
||||
go vet、竞态检测、构建与官方在线元数据验证通过。独立审查发现并修复了输出缓冲的
|
||||
io.Copy 大小限制绕过,失败回归用例已转绿;篡改测试改为合法语法,并增加交叉验证以防
|
||||
其他解析错误掩盖签名/摘要检查。测试用 Python 已隔离环境,实测 PYTHONOPTIMIZE=1
|
||||
不能禁用断言。没有完成实际 deb 文件验证或依赖事务预演。
|
||||
|
||||
## 第八工作包:实际 deb 文件完整性验证
|
||||
|
||||
- 新增 `verify-artifacts`:每次重新认证仓库元数据,再验证对应五个真实 deb 文件。
|
||||
- 暂存目录必须精确包含这五个文件;拒绝缺失、多余、符号链接、特殊文件及大小/摘要不符。
|
||||
- 流式计算 SHA-256,不将大型 deb 整体载入内存,不解包、不执行包内安装脚本。
|
||||
- 任一文件失败时整个请求失败;只有全部通过才返回 `packageBytesVerified: true`。
|
||||
- `executable` 保持 false;验证结果是当次观察,不是以后可复用的安装授权。
|
||||
|
||||
本地 Ubuntu WSL 下载并验证了五个官方 resolute/amd64 deb 测试样本(约 90 MB);
|
||||
同长度单字节篡改测试被拒绝。没有安装任何下载包,没有调用 APT,没有连接或修改云服务器。
|
||||
测试样本暂存于本地临时目录,其中一个被测试故意修改,不可拿去安装。
|
||||
|
||||
验证:Windows 全套测试、go vet、构建与 7 项 CLI 冒烟通过;Linux 全套测试、go vet、
|
||||
竞态检测、构建及真实 deb 下载/签名/摘要/篡改测试通过。独立审查指出的临时辅助文件权限
|
||||
边界和协议测试被其他错误掩盖的问题已修复,复审无遗留发现。协议测试通过临时换用宽松
|
||||
解码器确认能够失败,恢复严格解码后复验通过。以上是本批范围内的验证,不是安装验收。
|
||||
|
||||
本批完成的是 Docker 五包的字节校验,不是完整安装事务预演。Ubuntu 依赖源的认证、
|
||||
全部依赖的版本/摘要锁定,以及隔离 APT 配置/包状态的预演仍未交付。
|
||||
APT 会读取多层配置,后续适配器须隔离配置与状态,不直接复用宿主默认配置;
|
||||
参考 [APT 配置加载顺序](https://manpages.debian.org/trixie/apt/apt.conf.5.en.html)。
|
||||
|
||||
## 跨设备开发交接(2026-09-25)
|
||||
|
||||
用户要求补充记录并实现其他设备拉取接续。本次将此前八个工作包的源码、测试、协议及文档
|
||||
纳入 `codex/new-deployment-architecture` 分支交付,不合并到 master;交接入口为根目录 HANDOFF.md。
|
||||
开发指南记录工具链基线、Windows/Linux 命令、可选在线测试和凭据边界;补充了本地服务+浏览器
|
||||
的便携交付目标,并明确 APT 隔离细化方案仍待确认。原有旧 `.env` 本地删除不纳入本次提交。
|
||||
|
||||
对 Git 暂存快照导出的新架构干净源码副本复验:Windows 全量 Go 测试、vet、构建、7 项 CLI
|
||||
冒烟及示例运行通过;Linux 全量测试、vet、race、构建及 CLI 只读检查通过。Windows 最初在
|
||||
创建构建目录时遇到本地沙箱权限拒绝,授权重跑同一副本后通过,未修改代码或放宽目录权限。
|
||||
这是同一设备上的隔离源码副本/双环境验证,不是另一台物理设备或 macOS 的验收。
|
||||
文档相对链接已检查。本轮未重复在线下载;其通过证据保留在第七、八工作包,新设备可按指南重跑。
|
||||
|
||||
## 尚未交付
|
||||
|
||||
1. 远程现场 inspect、持久化状态与执行器集成、systemd 任务监督及重启现场核对。
|
||||
2. 应用包可信分发、发布者认证、完整 Compose 安全策略和运行适配器,以及应用包与计划的完整绑定。
|
||||
3. Traefik 环境安装、受限 Docker API 代理、网络与 ACME 实测。
|
||||
4. Gitea/Joplin 和其余工具的部署、升级、备份与隔离恢复。
|
||||
5. 本地面板、SSH、SQLite 与 Apple 风格交互界面。
|
||||
6. Windows/macOS 便携产品、Linux 故障注入与生产发布验收。
|
||||
|
||||
前三个工作包未连接服务器。用户随后指定新服务器 `121.199.168.54` 并确认主机指纹,
|
||||
现已完成 SSH 只读初检,详见 [现场记录](2026-09-25-target-server-preflight.md)。
|
||||
未安装环境、迁移或清除任何应用数据,未操作旧服务器,未删除旧部署实现。
|
||||
旧脚本暂留作开发对照,不作为新执行器后端;新架构完成验收后按总方案移除。
|
||||
|
||||
下一工作包是 Ubuntu 依赖源认证与隔离的完整依赖事务预演,之后才接入安装执行,
|
||||
再逐步接入受控配置/秘密、入口路由和任务执行器。
|
||||
完整产品尚未完成,不可将当前 CLI 用作部署工具。
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,62 @@
|
||||
# Operation State Implementation Plan
|
||||
|
||||
> **For agentic workers:** Use superpowers:executing-plans with test-driven-development. No production access or automatic commits.
|
||||
|
||||
**Goal:** Persist operation identity and status under an exclusive OS lock, refusing duplicate execution and ambiguous state.
|
||||
|
||||
**Architecture:** An internal state package acquires a nonblocking per-directory host lock and returns a session. Every mutation saves a complete bounded snapshot using write/sync/rename; the session becomes unusable on persistence errors. The directory must be a pre-existing trusted local directory, not a network share.
|
||||
|
||||
**Tech Stack:** Go standard library, Linux flock / Windows LockFileEx, os.Root.
|
||||
|
||||
**Spec:** docs/2026-09-25-complete-optimization-proposal.md sections 7, 8, 12, 14.
|
||||
|
||||
## Constraints
|
||||
|
||||
- Current directory and codex/new-deployment-architecture branch; preserve prior edits.
|
||||
- No CLI write endpoint or Docker/application operation is enabled by this package.
|
||||
- No automatic deletion of lock files, stale-task replay or corrupted-state reset.
|
||||
- Lock-file inode must remain stable; close releases the OS lock.
|
||||
- Metadata integrity is not authorization, and terminal success must be verified by the future executor.
|
||||
- Linux is the production target; Windows provides development tests. Power-loss durability on Windows is not claimed.
|
||||
|
||||
## Task 1: Exclusive host sessions
|
||||
|
||||
Files: internal/state/store.go, lock_linux.go, lock_windows.go, lock_unsupported.go, store_test.go.
|
||||
|
||||
Interface: `Acquire(directory, hostID string) (*Session, error)`; `Session.Close() error`.
|
||||
|
||||
- [x] Write tests: second session returns ErrBusy, closed session rejects use, independent directories do not block each other, invalid IDs fail, killed child releases OS lock.
|
||||
- [x] Run `go test ./internal/state`; observe missing implementation failure.
|
||||
- [x] Implement nonblocking OS lock with private regular file inside os.Root; refuse unsupported systems and symlink state files.
|
||||
- [x] Run tests using real temp directories and subprocesses, not mocked locks.
|
||||
|
||||
```go
|
||||
second, err := Acquire(dir, "host-one")
|
||||
if second != nil || !errors.Is(err, ErrBusy) { t.Fatal("host lock bypassed") }
|
||||
```
|
||||
|
||||
## Task 2: Persistent idempotency and state transitions
|
||||
|
||||
Files: internal/state/operation.go, snapshot.go, operation_test.go.
|
||||
|
||||
Interfaces: `Begin(id, planHash string) (Operation, bool, error)`; `Get(id string) (Operation, error)`; `Advance(id string, revision uint64, next Status) (Operation, error)`.
|
||||
|
||||
- [x] Tests: duplicate ID/hash returns original with created=false; same ID/different hash conflicts; reopened state persists; unrelated new operation blocked by unresolved prior operation; stale revision and running-to-running rejected; terminal results immutable.
|
||||
- [x] Tests: corrupt/truncated/oversized/host-mismatched snapshots rejected; write failure poisons session and never returns success; surviving running state is not silently reset.
|
||||
- [x] Implement canonical checksummed snapshot, bounded read, unique temporary file, file sync, atomic rename and Linux directory sync. Any persistence failure requires closing/reopening and checking the actual state.
|
||||
- [x] Run full tests and real killed-child recovery test. Abrupt process death is not a power-loss durability test.
|
||||
|
||||
```go
|
||||
again, created, err := session.Begin("op-one", hash)
|
||||
if err != nil || created || again.Revision != 1 { t.Fatal("duplicate submission changed state") }
|
||||
```
|
||||
|
||||
## Task 3: Verification and documentation
|
||||
|
||||
- [x] Run gofmt, `go test ./... -count=1`, `go vet ./...`, native CLI smoke tests and Linux cross-build including state tests.
|
||||
- [x] Independent read-only code review; repair actionable findings with regression tests.
|
||||
- [x] Update docs/implementation-status.md and internal/state/README.md with lifecycle, trusted-root requirements, residual risks, and remaining systemd/executor work.
|
||||
|
||||
Additional evidence: Ubuntu WSL full suite and `go test -race ./internal/state` passed.
|
||||
Windows symlink tests skipped for unavailable privilege; both passed on Linux.
|
||||
Regression tests cover missing initialized snapshots and 113,357-record capacity boundary.
|
||||
@@ -0,0 +1,56 @@
|
||||
# Package Verification and Inspection Implementation Plan
|
||||
|
||||
> **For agentic workers:** Execute with test-driven-development; package worker and local inspection work have disjoint write scopes. No commits, production connections, or deployment writes.
|
||||
|
||||
**Goal:** Add read-only package verification and a factual local prerequisite report to deployctl.
|
||||
|
||||
**Architecture:** A shared bounded strict JSON decoder supports nested typed arrays. An immutable package verifier returns verified manifest and file bytes, never executes templates. A local probe reports OS/architecture and tool/runtime presence, with unperformed checks explicitly marked unknown.
|
||||
|
||||
**Tech Stack:** Go standard library. Existing local Windows and Ubuntu WSL test runtimes.
|
||||
|
||||
**Spec:** docs/2026-09-25-complete-optimization-proposal.md sections 5, 7, 12, 14.
|
||||
|
||||
## Boundaries and integration
|
||||
|
||||
- Parent owns internal/wire, internal/inspect, internal/cli, docs and CLI tests.
|
||||
- Package worker owns only internal/appbundle; requirements are in package-verifier-brief.md beside this plan.
|
||||
- Package code consumes `wire.Decode(reader io.Reader, target any, limit int64) error`.
|
||||
- CLI consumes `appbundle.Verify(directory, expectedDigest string) (Verified, error)`; Verified has Manifest, Files map[string][]byte, Digest string. CLI never prints file bodies.
|
||||
- Local inspect is not full host acceptance; Docker daemon, Compose, firewall, DNS, permissions and port availability remain explicitly unverified.
|
||||
- Expected digest must come from a trusted channel. Matching it does not establish publisher identity on its own.
|
||||
|
||||
## Task 1: Shared strict wire decoder
|
||||
|
||||
- [x] Move strict JSON decoder from internal/cli to internal/wire, keeping all CLI tests passing.
|
||||
- [x] Add failing tests for nested array unknown/missing/case-alias fields and null elements, duplicate keys and input limit.
|
||||
- [x] Extend recursive validation through typed slices and reject invalid UTF-8. Preserve canonical UTC timestamp policy.
|
||||
- [x] Test the API `wire.Decode(strings.NewReader(input), &value, 1024)` using real JSON.
|
||||
|
||||
## Task 2: Application package verifier
|
||||
|
||||
- [x] Worker implements bounded manifest validation, pinned multi-component images, exact file inventory, SHA-256 content checks and path/symlink rejection using tests first.
|
||||
- [x] Parent reviews written code and runs all tests, including Linux symlink cases.
|
||||
- [x] Independent review checks all file access and integrity boundaries.
|
||||
|
||||
## Task 3: Read-only inspection and CLI
|
||||
|
||||
- [x] internal/inspect provides `Collect() Report` and a testable filesystem probe. Report includes OS, architecture, Linux support, systemd runtime presence and Docker client presence, with explicit unchecked prerequisites.
|
||||
- [x] Missing or inaccessible files produce missing/unknown observations, never false success. No shell commands, network, credentials or environment changes.
|
||||
- [x] CLI `inspect` emits observations with deploymentReady=false.
|
||||
- [x] CLI `verify-package` accepts `{directory, expectedDigest}`, emits manifest, digest, file count and executable=false; rejects invalid input with redacted diagnostics.
|
||||
- [x] Add actual CLI tests for new commands and unsupported writes.
|
||||
|
||||
## Verification
|
||||
|
||||
- [x] Windows go test/vet, CLI smoke; Linux full tests, race check and build.
|
||||
- [x] Record separate implemented, unverified and not-yet-supported capabilities.
|
||||
|
||||
## Execution notes
|
||||
|
||||
Task 1 + Task 2 share only the Decode interface; Task 2 must not modify it. Task 3 consumes their exports without exposing package bytes. Directory paths are accepted only for read-only local verification. Files remain subject to trusted staging ownership; future execution must use the verified snapshot rather than reopening mutable package files.
|
||||
|
||||
Independent review found an alternate Docker path omitted when the primary path
|
||||
was invalid or inaccessible. Three failing regression cases reproduced it before
|
||||
the fix. Fallback now prefers presence, then uncertainty, invalidity and absence.
|
||||
Focused re-review found no remaining issues. Final Windows and Linux checks passed;
|
||||
see implementation-status.md for the initial non-reproduced Linux compiler error.
|
||||
@@ -0,0 +1,77 @@
|
||||
# Runner Foundation Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox syntax for tracking.
|
||||
|
||||
**Goal:** Deliver a runnable, read-only deployctl foundation that strictly validates deployment requests and produces expiring, state-bound plans without executing legacy scripts.
|
||||
|
||||
**Architecture:** Go standard-library CLI consumes bounded JSON over stdin. Domain validation and planning are separate from CLI transport. No deployment writes are enabled before the executor and application recovery contracts pass their own acceptance tests.
|
||||
|
||||
**Tech Stack:** Go, standard testing package, JSON protocol v1.
|
||||
|
||||
**Spec:** docs/2026-09-25-complete-optimization-proposal.md
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- Traefik + local React/Node panel + Go runner are approved choices.
|
||||
- Do not connect to production or modify existing application data.
|
||||
- Preserve current uncommitted documents and claude-dev-stack deletion.
|
||||
- No implicit system upgrade, arbitrary shell field, mutable image tag or fake successful apply.
|
||||
- This plan is the first independently testable work package, not the full product.
|
||||
- No commit or push without user request.
|
||||
|
||||
## Delivery sequence after this package
|
||||
|
||||
Remote state/locking/idempotency → trusted packages and Compose runtime → Traefik environment → Gitea backup/upgrade/restore → Joplin verification → local panel → other applications → isolated Linux acceptance and replacement of legacy source. These require separate implementation plans; they are not represented as completed by this package.
|
||||
|
||||
## Task 1: Validated deployment intent
|
||||
|
||||
Files: go.mod; internal/planner/intent.go; internal/planner/intent_test.go.
|
||||
|
||||
Interface: Intent.Validate() error. Intent includes protocolVersion, hostId, instanceId, appId, packageDigest, imageDigest, domain, observedStateDigest.
|
||||
|
||||
- [x] Write table-driven tests: valid request succeeds; traversal IDs, missing fields, unsupported protocol, floating images, invalid domains and invalid hashes fail.
|
||||
- [x] Run `go test ./internal/planner` and observe the missing implementation failure.
|
||||
- [x] Implement strict lower-case IDs, sha256 digest validation and ASCII DNS names; do not normalize invalid input silently.
|
||||
- [x] Run tests and `go vet ./...`.
|
||||
|
||||
Example boundary assertion:
|
||||
|
||||
```go
|
||||
request.InstanceID = "../gitea"
|
||||
if request.Validate() == nil { t.Fatal("accepted path traversal") }
|
||||
```
|
||||
|
||||
## Task 2: Expiring, state-bound plan
|
||||
|
||||
Files: internal/planner/plan.go; internal/planner/plan_test.go.
|
||||
|
||||
Interfaces: Build(Intent, time.Time) (Plan, error); Plan.Verify(Intent, time.Time) error.
|
||||
|
||||
- [x] Test repeatable hashes at fixed time, separate instance identities, expiry boundary, altered intent/hash and observed-state drift rejection.
|
||||
- [x] Observe failing tests before implementation.
|
||||
- [x] Implement SHA-256 over typed JSON with an empty hash field, 15-minute TTL, deterministic project/data paths and verification of all derived fields.
|
||||
- [x] Run `go test ./internal/planner`.
|
||||
|
||||
```go
|
||||
if err := plan.Verify(intent, plan.ExpiresAt); err == nil { t.Fatal("accepted expired plan") }
|
||||
```
|
||||
|
||||
Plan hash is an integrity binding, NOT authentication or approval. Supplied host/state are unverified offline input; no write operation may trust them without future remote inspection.
|
||||
|
||||
## Task 3: Read-only CLI and contract
|
||||
|
||||
Files: cmd/deployctl/main.go; internal/cli/run.go; internal/cli/run_test.go; protocol/README.md; protocol/examples/plan-request.json.
|
||||
|
||||
Interface: cli.Run(args []string, in io.Reader, out, diagnostics io.Writer, now func() time.Time) int.
|
||||
|
||||
- [x] Tests exercise actual JSON input/output: plan and verify success, unknown fields, duplicate fields, trailing objects, oversized input, unsupported commands including apply, and no input echo on errors.
|
||||
- [x] Observe missing CLI behavior failure.
|
||||
- [x] Implement `version`, `plan`, `verify-plan` only; strict size-bounded decoder; generic diagnostics; nonzero exit on refusal. `apply` stays unavailable.
|
||||
- [x] Build and execute the real CLI with sample stdin; verify output parses and `apply` fails.
|
||||
- [x] Document exact wire format, offline limitation, commands and remaining stages.
|
||||
|
||||
## Verification and handoff
|
||||
|
||||
- [x] gofmt; go test ./... -count=1; go vet ./...; build Windows and Linux binaries.
|
||||
- [x] Confirm no external dependencies and no production connection/process execution in CLI.
|
||||
- [x] Record actual verification results and remaining work in docs/implementation-status.md.
|
||||
@@ -0,0 +1,19 @@
|
||||
# Package verifier worker requirements
|
||||
|
||||
Implement ONLY internal/appbundle/*.go and optional internal/appbundle/README.md in G:/Works/server-deploy. No commits, no other files, no subagents, no production access. Use apply_patch, tests first. Parent handles CLI, shared decoder, environment inspection and integration review.
|
||||
|
||||
API: `Verify(directory, expectedDigest string) (Verified, error)`.
|
||||
Verified: `Manifest Manifest`, `Files map[string][]byte`, `Digest string`.
|
||||
Manifest JSON fields (all required, no extras): protocolVersion (1), appId (lowercase ID), version (numeric x.y.z), runtime (compose), entrypoint (relative path to listed file), platforms ([]string of linux/amd64 or linux/arm64, unique, nonempty), components ([]Component), files ([]File).
|
||||
Component fields: name (lowercase ID), image (lowercase repository@sha256:64hex; no tag, port, whitespace or shell syntax in initial subset). Components 1..32, names unique; repository components use lowercase ASCII alnum with separators dot/underscore/hyphen, slash between components, no empty segments.
|
||||
File fields: path (portable relative slash path), digest (sha256:64 lowercase hex). Files 1..128, unique; manifest.json itself excluded from files. No slash root, drive, backslash, colon, dot/dotdot segments, dotfiles, empty segments, trailing spaces/dots, Windows device-name segments (including extensions), or case-colliding names. Allowed path segment alphabet lowercase ASCII a-z0-9 underscore hyphen dot, first character alphanumeric. Max path 240 bytes, segment 100. ID rules consistent with planner: first a-z, remaining a-z0-9-, max48.
|
||||
|
||||
Manifest file name manifest.json; maximum 1MiB. ExpectedDigest pins SHA-256 of the exact raw manifest bytes (manifest pins every file). Validate digest before parsing. Use shared `server-deploy/internal/wire.Decode(io.Reader, target, int64 limit)` being implemented by parent; it rejects duplicate/unknown/missing/case-alias fields, nulls including slice elements, trailing data and oversized input.
|
||||
|
||||
Read only regular files under os.Root, reject symlinks including intermediate directory links, file size max4MiB, total payload <=16MiB. Exact inventory: reject any unlisted files, secrets, symlinks, special files or unnecessary directories; allow only parent directories of declared files plus manifest.json. Bound enumeration to declared inventory rather than unbounded walk; reject extras immediately. No extraction, writes, processes, network or arbitrary command execution. Require absolute directory; staging root is caller-selected trusted directory (ancestors trusted), don't claim hostile concurrent mutation is fully prevented. Return verified bytes for later consumers; do not reopen files to execute anything.
|
||||
|
||||
Entrypoint must be listed. DO NOT claim Compose semantics safe/validated: contents are authenticated as bytes only. Signature trust store and executable policy come later; expected hash is a caller trust anchor, not publisher authentication. Do not hardcode live app image hashes or fetch versions.
|
||||
|
||||
Tests with real temp directories: valid two-component package, wrong manifest digest, changed/missing/extra file, nested valid file, symlink/intermediate symlink (skip only missing Windows privilege, Linux tested by parent), traversal, duplicate components/file paths, unpinned image, unsupported protocol/runtime/platform, missing/unknown/null component fields, oversized payload, entrypoint missing. Fixture hashes may be calculated from known test bytes, but test expected acceptance/rejection independently.
|
||||
|
||||
Toolchain: C:/Users/Joywayer/AppData/Local/Temp/server-deploy-go-d67bef26e1ee49718e2b62311429f729/go/bin/go.exe. Set GOCACHE to a dedicated temp path if needed. Parent creates wire shortly; don't change it to unblock. Report test results and changed paths, plus any blockers/concerns, without declaring full deployment readiness.
|
||||
@@ -0,0 +1,182 @@
|
||||
# Xray 定时重启:可配置的「每 N 天 + 固定时间点」
|
||||
|
||||
日期:2026-08-06
|
||||
涉及文件:`vps-xray/deploy.sh`、`vps-xray/.env.example`、`vps-xray/uninstall.sh`、`vps-xray/README.md`
|
||||
|
||||
## 背景
|
||||
|
||||
当前 `deploy.sh` 只有一个布尔开关 `XRAY_DAILY_RESTART`,为真时创建 `OnCalendar=*-*-* 04:00:00` 的每日重启 timer,为假时删除该 timer。默认关闭,理由是崩溃恢复已由 `Restart=always` + `RestartSec=3` 覆盖(秒级),而每日硬重启会切断全部活动连接。
|
||||
|
||||
这个判断本身没变,但「一天一次」粒度太粗:想要「每 7 天凌晨 4 点重启一次」这种低频维护性重启时,只能在「每天重启」和「完全不重启」之间二选一。
|
||||
|
||||
## 目标
|
||||
|
||||
1. 支持任意 N 天间隔 + 指定时间点,如「每 7 天 04:00」「每 3 天 05:30」
|
||||
2. 保持默认关闭,已部署机器重跑脚本时行为不变
|
||||
3. 提供一个轻量命令行入口,只改重启计划,不触碰 Xray 与现有 vless 链接
|
||||
|
||||
## 非目标
|
||||
|
||||
- 不做「每月/每季度」等日历语义(真有需要时用户可直接编辑 timer)
|
||||
- 不做多时间点(如一天重启两次)
|
||||
- 不改动 `Restart=always` 这一层崩溃恢复逻辑
|
||||
|
||||
## 关键约束:systemd 表达不了任意 N 天
|
||||
|
||||
`OnCalendar` 的 `*-*-1/7 04:00:00` 是「每月的 1、8、15、22、29 号」,**跨月会重置**——29 号到下月 1 号只隔 2~3 天。因此不能直接用 `OnCalendar` 表达「每 N 天」。
|
||||
|
||||
`OnUnitActiveSec=7d` 是真实间隔,但无法钉住具体时间点,且会随每次触发缓慢漂移。
|
||||
|
||||
本设计采用第三种:**timer 每天到点唤醒,由 service 里的守卫脚本决定这次要不要真的重启。**
|
||||
|
||||
## 配置模型
|
||||
|
||||
`.env` 三个字段:
|
||||
|
||||
```bash
|
||||
XRAY_RESTART_EVERY_DAYS=0 # 0 或缺失 = 关闭;N = 每 N 天重启一次
|
||||
XRAY_RESTART_TIME=04:00 # 24 小时制 HH:MM,默认 04:00
|
||||
XRAY_RESTART_TIMEZONE=Asia/Shanghai # 默认保持现状
|
||||
```
|
||||
|
||||
首次部署默认 `0`(关闭),与当前行为一致。
|
||||
|
||||
### 旧字段迁移
|
||||
|
||||
只在 `XRAY_RESTART_EVERY_DAYS` **未设置**时才回退去读 `XRAY_DAILY_RESTART`:
|
||||
|
||||
| 旧值 | 迁移结果 |
|
||||
|---|---|
|
||||
| `true` / `yes` / `on` / `1`(大小写不敏感) | `XRAY_RESTART_EVERY_DAYS=1` |
|
||||
| 其它或缺失 | `XRAY_RESTART_EVERY_DAYS=0` |
|
||||
|
||||
`save_env` 之后只写新字段,不再写 `XRAY_DAILY_RESTART`,`.env` 随重跑自动换代。两个字段同时存在时以新字段为准。
|
||||
|
||||
现存四台服务器的 `.env` 都是 `XRAY_DAILY_RESTART=false`,迁移后为 `0`,行为不变。
|
||||
|
||||
## 实现架构
|
||||
|
||||
```
|
||||
xray-restart.timer OnCalendar=*-*-* <HH:MM>:00 每天到点唤醒
|
||||
TimeZone=<tz>
|
||||
Persistent=true
|
||||
↓
|
||||
xray-restart.service Type=oneshot
|
||||
ExecStart=/usr/local/bin/xray-restart-guard
|
||||
↓
|
||||
xray-restart-guard 读 /var/lib/xray/last-restart
|
||||
距上次不足 N 天 → 打日志后 exit 0
|
||||
满 N 天 → 写入新时间戳 → systemctl restart xray
|
||||
```
|
||||
|
||||
「每 N 天」因此是**距上次实际重启**的真实间隔,不受跨月重置影响,N=3 与 N=7 同样准确。
|
||||
|
||||
### 守卫脚本的两个关键细节
|
||||
|
||||
**300 秒容差。** systemd timer 默认 `AccuracySec=1min`,第 N 次触发的实际间隔可能是 `N×86400 - 30s`。严格比较会判定「不满 N 天」从而顺延整整一天,且误差会持续累积。阈值取 `N×86400 - 300` 规避。容差远小于一天,不会造成重复触发。
|
||||
|
||||
**先写时间戳,再重启。** `systemctl restart xray` 会中断 guard 所在的 systemd 事务。顺序反了会丢失时间戳记录,导致每天都重启。代价是:重启失败时时间戳仍已更新,本轮被跳过——这比陷入每日重启循环可接受。
|
||||
|
||||
### 状态文件
|
||||
|
||||
`/var/lib/xray/last-restart`,内容为一个 Unix 时间戳。guard 以 root 运行,读写无权限问题。内容非法(非纯数字)时按 `0` 处理,即立即允许重启。
|
||||
|
||||
## 参数校验
|
||||
|
||||
在脚本早期(`.env` 载入后)完成,不合法直接报错退出,**不静默回退到默认值**——避免用户以为设了每 7 天、实际在每天重启。
|
||||
|
||||
| 字段 | 规则 |
|
||||
|---|---|
|
||||
| `XRAY_RESTART_EVERY_DAYS` | `^[0-9]+$` |
|
||||
| `XRAY_RESTART_TIME` | `^([01]?[0-9]\|2[0-3]):[0-5][0-9]$`,校验通过后**补零归一化**为 `HH:MM` |
|
||||
| `XRAY_RESTART_TIMEZONE` | 非空即可,交由 systemd 校验并在启动失败时报错 |
|
||||
|
||||
时间同时接受 `4:00` 与 `04:00`,但写入 `.env` 和 timer 前一律归一化成两位小时(`04:00`),保证 `OnCalendar` 字符串格式统一、`.env` 内容可预测。
|
||||
|
||||
## `--restart` 轻量入口
|
||||
|
||||
### 用法
|
||||
|
||||
```bash
|
||||
bash deploy.sh --restart 7 04:00 # 每 7 天 04:00
|
||||
bash deploy.sh --restart 3 # 每 3 天,时间沿用 .env
|
||||
bash deploy.sh --restart 0 # 关闭
|
||||
bash deploy.sh --restart # 按 .env 现有值重建 timer
|
||||
```
|
||||
|
||||
### 参数解析
|
||||
|
||||
`--restart` 后可跟 0~2 个位置参数,按形态判断,不会误吞后续选项:
|
||||
|
||||
- 下一个 token 匹配 `^[0-9]+$` → 取作 N
|
||||
- 再下一个匹配 `^([01]?[0-9]|2[0-3]):[0-5][0-9]$` → 取作 HH:MM
|
||||
- 不匹配则不消费,沿用 `.env` 现值
|
||||
|
||||
`--restart` 与 `--mode` / `--show` / `--redetect` 互斥,同时给出直接报错。
|
||||
|
||||
### 执行路径
|
||||
|
||||
在 `main()` 中早退,与 `--show` 同一位置:
|
||||
|
||||
1. 校验 N 与 HH:MM,时间归一化为 `HH:MM`
|
||||
2. 就地更新 `.env`:命令行给了 N 就写 `XRAY_RESTART_EVERY_DAYS`,给了时间才写 `XRAY_RESTART_TIME`;两者都没给则不改 `.env`,只按现值重建 timer。字段不存在时追加(`TIME` 追加时用默认值 `04:00`)
|
||||
3. 调用 `configure_restart_timer()`
|
||||
4. `systemctl daemon-reload`,按需 enable / disable timer
|
||||
5. 打印结果与 `systemctl list-timers xray-restart.timer`
|
||||
|
||||
### 明确不做的事
|
||||
|
||||
不装依赖、不升级 Xray、不生成密钥、不选伪装目标、不重写 `config.json`、不动防火墙与 sysctl、**不重启 xray**。因此 vless 链接与现有连接均不受影响。这些约束会以注释形式写在函数头部。
|
||||
|
||||
### 前置检查
|
||||
|
||||
- `.env` 不存在 → 报错退出(无法确认这是已部署的机器)
|
||||
- xray 主单元不存在 → 报错退出(避免在未部署的机器上留下孤儿 timer)
|
||||
|
||||
## 代码组织
|
||||
|
||||
**timer / service / guard 的创建逻辑抽成独立函数 `configure_restart_timer()`**,由 `harden_system()` 与 `--restart` 路径共用。两条路径各写一份实现迟早会漂移成不一致。
|
||||
|
||||
`.env` 更新走两条不同路径,这是有意的:
|
||||
|
||||
- 完整部署走既有的 `save_env()`(`cat >` 全量重写)
|
||||
- `--restart` 走 `sed` 就地替换 —— 轻量入口只应改自己那两行,不该抹掉用户在 `.env` 里加的注释和自定义字段
|
||||
|
||||
## 关闭与清理
|
||||
|
||||
`XRAY_RESTART_EVERY_DAYS=0` 时:
|
||||
|
||||
- `systemctl disable --now xray-restart.timer`
|
||||
- 删除 `xray-restart.timer`、`xray-restart.service`、`/usr/local/bin/xray-restart-guard`
|
||||
- **保留** `/var/lib/xray/last-restart`
|
||||
|
||||
保留状态文件的理由:以后重新启用时,若距上次已超过 N 天则首次触发即重启,符合直觉;若刚重启过则不会立刻再来一次。
|
||||
|
||||
`uninstall.sh` 需补上 guard 脚本与状态文件的清理(当前只删了 timer 和 service)。
|
||||
|
||||
## 受影响文件
|
||||
|
||||
| 文件 | 改动 |
|
||||
|---|---|
|
||||
| `vps-xray/deploy.sh` | 参数解析、配置校验与迁移、`configure_restart_timer()`、`--restart` 路径、`save_env`、`usage` |
|
||||
| `vps-xray/.env.example` | 三个新字段与说明 |
|
||||
| `vps-xray/uninstall.sh` | 清理 guard 脚本与状态文件 |
|
||||
| `vps-xray/README.md` | 定时重启一节 |
|
||||
|
||||
## 验证方案
|
||||
|
||||
在真实 VPS 上执行,不接受仅靠阅读代码判断:
|
||||
|
||||
1. **语法** —— `bash -n deploy.sh`
|
||||
2. **启用** —— `bash deploy.sh --restart 7 04:00`,确认:`.env` 两字段已更新;`systemctl list-timers xray-restart.timer` 有下次触发时间;xray 主服务的 `ActiveEnterTimestamp` **未变**(证明没重启)
|
||||
3. **守卫生效** —— 手动连跑 `xray-restart-guard` 两次,第一次应重启并写入时间戳,第二次应打印「跳过」且 xray 未重启
|
||||
4. **关闭** —— `bash deploy.sh --restart 0`,确认 timer / service / guard 三个文件均被删除,状态文件仍在
|
||||
5. **迁移** —— 构造一份只含 `XRAY_DAILY_RESTART=true` 的 `.env`,跑完整部署,确认得到 `XRAY_RESTART_EVERY_DAYS=1` 且 timer 已建立
|
||||
6. **校验** —— `bash deploy.sh --restart 7 25:00` 与 `--restart abc` 均应报错退出,且不留下任何单元文件
|
||||
7. **互斥** —— `bash deploy.sh --restart 7 --redetect` 应报错退出
|
||||
|
||||
## 已知取舍
|
||||
|
||||
- **间隔锚点是「上次实际重启时间」,不是固定日历日期。** 若某次因机器关机错过了触发点,下次开机后的第一个触发点会补上(`Persistent=true`),此后间隔重新从那一刻起算,会缓慢偏移触发日期。这是「真实间隔」语义的必然结果,也是用户选择的语义。
|
||||
- **重启失败时时间戳仍已更新**,本轮被跳过,下一轮照常。理由见上文「先写时间戳」。
|
||||
- **时区错误要到 timer 启动时才暴露**。脚本不预校验时区字符串,靠 `systemctl` 报错——自行维护时区白名单成本高且易过期。
|
||||
@@ -1,3 +1,5 @@
|
||||
data/
|
||||
backups/
|
||||
.env
|
||||
.env.bak-*
|
||||
.upgrade-state
|
||||
|
||||
+83
-30
@@ -326,42 +326,93 @@ docker compose up -d # 启动
|
||||
|
||||
#### 升级 Gitea
|
||||
|
||||
Gitea 以 Docker 容器运行,升级 = 拉取新镜像 + 重启容器。数据库结构变更会在启动时自动迁移。
|
||||
使用 `upgrade.sh` 升级,**不要**手动 `docker compose pull && up -d`(见下方「为什么不要手动升级」)。
|
||||
|
||||
```bash
|
||||
cd /opt/gitea
|
||||
|
||||
# 1. 备份(必须!)
|
||||
bash backup.sh
|
||||
|
||||
# 2. 查看当前版本
|
||||
curl -s http://127.0.0.1:3000/api/v1/version
|
||||
|
||||
# 3. 修改目标版本(编辑 .env 中的 GITEA_IMAGE)
|
||||
# ● 查看最新版本号: https://github.com/go-gitea/gitea/releases
|
||||
# ● 或访问: https://hub.docker.com/r/gitea/gitea/tags
|
||||
vi .env
|
||||
# 修改: GITEA_IMAGE=gitea/gitea:1.25.5 ← 替换为目标版本号
|
||||
|
||||
# 4. 拉取新镜像
|
||||
docker compose pull server
|
||||
|
||||
# 5. 重启(自动执行数据库迁移)
|
||||
docker compose up -d server
|
||||
|
||||
# 6. 检查日志确认启动成功
|
||||
docker compose logs -f server
|
||||
# 看到 "Starting new Web server: tcp:0.0.0.0:3000" 表示成功
|
||||
# Ctrl+C 退出日志
|
||||
|
||||
# 7. 验证新版本
|
||||
curl -s http://127.0.0.1:3000/api/v1/version
|
||||
bash upgrade.sh --check # 先看看会发生什么,不做任何改动
|
||||
bash upgrade.sh # 正式升级到 GitHub 最新 release
|
||||
```
|
||||
|
||||
**注意事项:**
|
||||
- 务必查阅 [Gitea 发版说明](https://github.com/go-gitea/gitea/releases) 了解 Breaking Changes
|
||||
- 不支持版本降级,升级前务必备份
|
||||
- 跨多个大版本建议逐版本升级(如 1.21 → 1.22 → 1.23)
|
||||
脚本做的事:
|
||||
|
||||
| 步骤 | 内容 | 失败时 |
|
||||
|------|------|--------|
|
||||
| 1 | 预检:root、依赖命令、MySQL 连通性、数据表数量、磁盘空间 | 直接退出,服务零影响 |
|
||||
| 2 | 拉取目标镜像 | 直接退出,服务零影响 |
|
||||
| 3 | 把当前镜像打上 `pre-upgrade-<时间戳>` 标签作为回滚锚点 | — |
|
||||
| 4 | 停止 Gitea 容器(**MySQL 保持运行**) | 自动拉起原版本 |
|
||||
| 5 | mysqldump + 打包数据目录 + 校验备份完整性 | 自动拉起原版本 |
|
||||
| 6 | 切换镜像启动,Gitea 自动执行数据库迁移 | 自动回滚 |
|
||||
| 7 | 校验镜像 ID / 运行版本 / 接口 / 表行数 / 迁移版本 | 自动回滚 |
|
||||
|
||||
常用参数:
|
||||
|
||||
```bash
|
||||
bash upgrade.sh --check # 只检查,零改动
|
||||
bash upgrade.sh --version 1.26.4 # 升级到指定版本
|
||||
bash upgrade.sh --full-backup # 连仓库和 LFS 一起备份(小实例推荐)
|
||||
bash upgrade.sh --timeout 600 # 大库迁移慢,放宽就绪超时(默认 300 秒)
|
||||
bash upgrade.sh --doctor # 升级后跑一次 gitea doctor 一致性检查
|
||||
bash upgrade.sh --yes # 跳过交互确认(自动化场景)
|
||||
bash upgrade.sh --rollback # 回滚到上次升级前的状态
|
||||
```
|
||||
|
||||
**备份范围说明**
|
||||
|
||||
默认只打包会被升级影响的部分:`gitea/conf`(app.ini)、`gitea/jwt`(OAuth2 签名密钥)、
|
||||
`ssh`(SSH host key)、`gitea/indexers`、`gitea/queues`、头像目录。
|
||||
|
||||
仓库(`git/repositories`)、LFS、软件包**默认不打包** —— 它们是内容寻址的追加式存储,
|
||||
版本迁移只改数据库结构,不会改写这些文件。上百 GB 的实例每次升级都全量打包并不现实。
|
||||
数据库则**每次都完整 mysqldump**,因为那才是升级真正会动的东西。
|
||||
|
||||
小实例想要更保险,加 `--full-backup` 连仓库一起打包。
|
||||
|
||||
**回滚**
|
||||
|
||||
```bash
|
||||
bash upgrade.sh --rollback
|
||||
```
|
||||
|
||||
读取 `.upgrade-state` 里记录的备份与镜像锚点,**回灌 SQL 备份** + 恢复配置目录 + 切回旧镜像。
|
||||
|
||||
> 回滚会把数据库整体退回到升级前的状态,升级之后新产生的 issue、PR、评论、仓库记录都会丢失。
|
||||
> 数据目录用的是合并解压,**不会删除任何仓库或 LFS 文件**。
|
||||
|
||||
#### 为什么不要手动升级
|
||||
|
||||
**一、跨小版本升级不可逆,只切镜像回滚是无效的**
|
||||
|
||||
按 [Gitea 官方文档](https://docs.gitea.com/installation/upgrade-from-gitea/),只有
|
||||
`a.b.x → a.b.y`(补丁级)才保证数据库结构不变、可以来回切。跨小版本(如 1.25 → 1.27)
|
||||
新版会在启动时改写数据库结构,之后旧版二进制看到新库会直接拒绝启动:
|
||||
|
||||
```
|
||||
Your database (migration version: 286) is for a newer Gitea,
|
||||
newer database for this old Gitea release (280).
|
||||
Gitea will exit to keep your database safe and unchanged.
|
||||
```
|
||||
|
||||
所以想退回旧版本,**唯一正确的办法是回灌升级前的 SQL 备份**。手动改 `version` 表是官方
|
||||
明确警告会丢数据的操作。`upgrade.sh` 的回滚就是按这个路径做的,并且会拒绝执行跨小版本降级。
|
||||
|
||||
**二、`docker compose up -d` 不会因为镜像变了就重建容器**
|
||||
|
||||
容器处于 stopped 状态时,`up -d` 只是把原容器重新 start,结果是「改了配置但还跑着旧镜像」。
|
||||
必须 `--force-recreate`。脚本在启动后强制校验「容器实际运行的镜像 ID == 目标镜像 ID」,
|
||||
不匹配直接回滚。
|
||||
|
||||
**三、`.env` 里的变量已经被 export 到当前 shell**
|
||||
|
||||
部署脚本用 `set -a; source .env` 导出过 `GITEA_IMAGE`,而 docker compose 对环境变量的
|
||||
优先级高于 `.env` 文件 —— 只改文件不改环境变量,compose 仍会读到旧镜像。
|
||||
|
||||
**其它注意事项:**
|
||||
- 升级前查阅 [Gitea 发版说明](https://github.com/go-gitea/gitea/releases) 了解 Breaking Changes
|
||||
- 自定义模板与新版不兼容会导致 5xx 或页面错乱,脚本的首页检查能发现这类问题
|
||||
- MySQL 容器不在 `upgrade.sh` 的升级范围内,需要单独处理(见下)
|
||||
|
||||
#### 升级 MySQL
|
||||
|
||||
@@ -749,6 +800,8 @@ docker compose exec server gitea admin user change-password -u 管理员用户
|
||||
├── .env # 运行时配置(自动生成)
|
||||
├── deploy.sh # 全新服务器一键部署脚本
|
||||
├── backup.sh # MySQL + 数据备份脚本
|
||||
├── upgrade.sh # 安全升级脚本(备份 → 校验 → 自动回滚)
|
||||
├── .upgrade-state # 升级回滚锚点(upgrade.sh 自动生成)
|
||||
├── uninstall.sh # 完全卸载脚本
|
||||
├── .gitignore
|
||||
├── README.md
|
||||
|
||||
+253
-35
@@ -1,68 +1,286 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
# -E 让 ERR trap 在函数内部同样生效(冷备份模式下必须靠它把服务拉回来)
|
||||
set -Eeuo pipefail
|
||||
|
||||
# ============================================
|
||||
# Gitea 备份脚本 (MySQL 版)
|
||||
# 备份内容:MySQL 数据库 + Gitea 数据(含 LFS) + 配置
|
||||
#
|
||||
# 备份内容:MySQL 数据库 + Gitea 数据(含仓库/LFS/密钥) + 部署配置
|
||||
#
|
||||
# 用法:
|
||||
# bash backup.sh 热备份(服务不停机,默认)
|
||||
# bash backup.sh --cold 冷备份(停 Gitea 取快照,最一致)
|
||||
# bash backup.sh --keep 60 保留天数(默认 30)
|
||||
# bash backup.sh --verify-last 只校验最近一次备份,不产生新备份
|
||||
#
|
||||
# 定时执行: crontab -e → 0 3 * * * /opt/gitea/backup.sh
|
||||
# ============================================
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
cd "$SCRIPT_DIR"
|
||||
|
||||
# 加载配置
|
||||
if [ ! -f .env ]; then
|
||||
echo "[ERROR] .env 文件不存在" >&2
|
||||
exit 1
|
||||
fi
|
||||
DATE=$(date +%Y%m%d_%H%M%S)
|
||||
KEEP_DAYS=30
|
||||
MIN_KEEP_SETS=3 # 无论多旧,至少保留这么多份完整备份
|
||||
MODE="hot" # hot | cold | verify
|
||||
SVC_APP="server"
|
||||
SVC_DB="db"
|
||||
DB_NAME="gitea"
|
||||
STOPPED=0
|
||||
|
||||
log() { echo "[$(date '+%Y-%m-%d %H:%M:%S')] $*"; }
|
||||
warn() { echo "[$(date '+%Y-%m-%d %H:%M:%S')] [WARN] $*" >&2; }
|
||||
die() { echo "[$(date '+%Y-%m-%d %H:%M:%S')] [ERROR] $*" >&2; exit 1; }
|
||||
|
||||
usage() {
|
||||
sed -n '5,18p' "$0" | sed 's/^# \{0,1\}//'
|
||||
exit 0
|
||||
}
|
||||
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--cold) MODE="cold" ;;
|
||||
--verify-last) MODE="verify" ;;
|
||||
--keep) shift; [ $# -gt 0 ] || die "--keep 需要一个参数"
|
||||
[[ "$1" =~ ^[0-9]+$ ]] || die "--keep 必须是整数天"
|
||||
KEEP_DAYS="$1" ;;
|
||||
-h|--help) usage ;;
|
||||
*) die "未知参数: $1(--help 查看用法)" ;;
|
||||
esac
|
||||
shift
|
||||
done
|
||||
|
||||
# ===== 加载配置 =====
|
||||
[ -f .env ] || die ".env 文件不存在"
|
||||
if grep -q $'\r' .env 2>/dev/null; then sed -i 's/\r$//' .env; fi
|
||||
set -a
|
||||
sed -i 's/\r$//' .env
|
||||
# shellcheck disable=SC1091
|
||||
source .env
|
||||
set +a
|
||||
|
||||
BACKUP_DIR="${BACKUP_DIR:-/var/backups/gitea}"
|
||||
GITEA_DATA_DIR="${GITEA_DATA_DIR:-/var/lib/gitea}"
|
||||
DATE=$(date +%Y%m%d_%H%M%S)
|
||||
KEEP_DAYS=30
|
||||
[ -n "${DB_ROOT_PASSWORD:-}" ] || die ".env 中缺少 DB_ROOT_PASSWORD"
|
||||
|
||||
log() { echo "[$(date '+%Y-%m-%d %H:%M:%S')] $*"; }
|
||||
# ===== 前置检查 =====
|
||||
[ "$(id -u)" -eq 0 ] || die "需要 root 权限运行"
|
||||
for c in docker tar gzip find awk; do
|
||||
command -v "$c" >/dev/null 2>&1 || die "缺少命令: $c"
|
||||
done
|
||||
docker compose version >/dev/null 2>&1 || die "docker compose 不可用"
|
||||
[ -d "$GITEA_DATA_DIR" ] || die "数据目录不存在: $GITEA_DATA_DIR"
|
||||
|
||||
mkdir -p "$BACKUP_DIR"
|
||||
chmod 700 "$BACKUP_DIR"
|
||||
|
||||
# 1. 备份 MySQL 数据库
|
||||
# 防止 cron 叠加执行:上一次还没跑完就不再起新的
|
||||
exec 9>"$BACKUP_DIR/.backup.lock"
|
||||
flock -n 9 || die "已有另一个备份进程在运行,本次退出"
|
||||
|
||||
# MySQL 容器用 MYSQL_PWD 传密码,避免密码出现在 ps 的命令行里
|
||||
db_exec() { docker compose exec -T -e MYSQL_PWD="$DB_ROOT_PASSWORD" "$SVC_DB" "$@"; }
|
||||
|
||||
db_running() {
|
||||
local cid
|
||||
cid=$(docker compose ps -a -q "$SVC_DB" 2>/dev/null | head -1)
|
||||
[ -n "$cid" ] || return 1
|
||||
[ "$(docker inspect "$cid" --format '{{.State.Running}}' 2>/dev/null)" = "true" ]
|
||||
}
|
||||
|
||||
# =============================================================
|
||||
# 校验:备份产出后必须自证可用,否则等于没备
|
||||
# =============================================================
|
||||
verify_db_dump() {
|
||||
local f="$1" summary tables completed
|
||||
[ -f "$f" ] || { warn "SQL 备份不存在: $f"; return 1; }
|
||||
|
||||
gzip -t "$f" 2>/dev/null || { warn "SQL 备份 gzip 校验失败: $f"; return 1; }
|
||||
|
||||
# 一次解压同时数建表语句和找结束标记。用 awk 读完整个流,不能写成
|
||||
# `gunzip -c | grep -q` —— grep 命中即退出会让 gunzip 收到 SIGPIPE,
|
||||
# 在 pipefail 下把好备份误判成坏的。
|
||||
summary="$(gunzip -c "$f" | awk '
|
||||
/^CREATE TABLE / { n++ }
|
||||
/-- Dump completed/ { d = 1 }
|
||||
END { print n+0, d+0 }
|
||||
')" || { warn "SQL 备份无法完整解压: $f"; return 1; }
|
||||
|
||||
tables="${summary%% *}"; completed="${summary##* }"
|
||||
# "Dump completed" 只有 mysqldump 正常收尾才会写;被 OOM / 断连截断的没有
|
||||
[ "$completed" = "1" ] || { warn "SQL 备份缺少结束标记,导出被截断: $f"; return 1; }
|
||||
[ "$tables" -ge 10 ] || { warn "SQL 备份只有 $tables 条 CREATE TABLE: $f"; return 1; }
|
||||
|
||||
log " SQL 备份校验通过(${tables} 张表,含结束标记)"
|
||||
return 0
|
||||
}
|
||||
|
||||
verify_data_tar() {
|
||||
local f="$1" listing repos
|
||||
[ -f "$f" ] || { warn "数据备份不存在: $f"; return 1; }
|
||||
|
||||
gzip -t "$f" 2>/dev/null || { warn "数据备份 gzip 校验失败: $f"; return 1; }
|
||||
listing="$(tar tzf "$f")" || { warn "数据备份无法列出内容: $f"; return 1; }
|
||||
|
||||
grep -q 'gitea/conf/app\.ini$' <<<"$listing" || { warn "数据备份中没有 app.ini"; return 1; }
|
||||
grep -q '/ssh/' <<<"$listing" || warn "数据备份中没有 SSH host key 目录"
|
||||
grep -q '/jwt/' <<<"$listing" || warn "数据备份中没有 JWT 密钥目录"
|
||||
|
||||
repos=$(grep -c 'repositories/[^/]*/[^/]*\.git/$' <<<"$listing" || true)
|
||||
log " 数据备份校验通过($(wc -l <<<"$listing") 个条目,${repos} 个仓库)"
|
||||
return 0
|
||||
}
|
||||
|
||||
verify_set() {
|
||||
local ts="$1" ok=0
|
||||
verify_db_dump "${BACKUP_DIR}/db_${ts}.sql.gz" || ok=1
|
||||
verify_data_tar "${BACKUP_DIR}/gitea_data_${ts}.tar.gz" || ok=1
|
||||
[ -f "${BACKUP_DIR}/config_${ts}.tar.gz" ] || { warn "配置备份缺失"; ok=1; }
|
||||
return $ok
|
||||
}
|
||||
|
||||
# ===== --verify-last:只校验,不产生新备份 =====
|
||||
if [ "$MODE" = "verify" ]; then
|
||||
last=$(find "$BACKUP_DIR" -maxdepth 1 -name 'db_*.sql.gz' -printf '%f\n' 2>/dev/null \
|
||||
| sed 's/^db_//; s/\.sql\.gz$//' | sort | tail -1)
|
||||
[ -n "$last" ] || die "$BACKUP_DIR 中没有找到任何备份"
|
||||
log "校验最近一次备份: $last"
|
||||
if verify_set "$last"; then log "===== 备份 $last 校验通过 ====="; exit 0
|
||||
else die "备份 $last 校验未通过"; fi
|
||||
fi
|
||||
|
||||
# =============================================================
|
||||
# 冷备份:停 Gitea 取一致快照,MySQL 全程保持运行
|
||||
# =============================================================
|
||||
restart_app_if_stopped() {
|
||||
[ "$STOPPED" -eq 1 ] || return 0
|
||||
STOPPED=0
|
||||
log "正在恢复 Gitea 服务..."
|
||||
docker compose up -d "$SVC_APP" >/dev/null 2>&1 || \
|
||||
warn "Gitea 恢复失败!请手动执行: cd $SCRIPT_DIR && docker compose up -d"
|
||||
}
|
||||
trap 'restart_app_if_stopped' EXIT
|
||||
|
||||
db_running || die "MySQL 容器未运行,无法备份数据库: docker compose up -d $SVC_DB"
|
||||
|
||||
# ===== 磁盘空间预检 =====
|
||||
data_kb=$(du -sk "$GITEA_DATA_DIR" 2>/dev/null | cut -f1 || echo 0)
|
||||
[[ "$data_kb" =~ ^[0-9]+$ ]] || data_kb=0
|
||||
avail_kb=$(df -Pk "$BACKUP_DIR" | awk 'NR==2{print $4}')
|
||||
need_kb=$(( data_kb * 6 / 5 + 512000 ))
|
||||
if [ -n "$avail_kb" ] && [ "$avail_kb" -lt "$need_kb" ]; then
|
||||
die "磁盘空间不足: 可用 $((avail_kb/1024))MB,预计需要 $((need_kb/1024))MB"
|
||||
fi
|
||||
|
||||
log "===== 开始备份(模式: $MODE)====="
|
||||
|
||||
if [ "$MODE" = "cold" ]; then
|
||||
log "停止 Gitea 以取得一致快照(MySQL 不停)..."
|
||||
docker compose stop "$SVC_APP" >/dev/null 2>&1
|
||||
STOPPED=1
|
||||
log "Gitea 已停止"
|
||||
fi
|
||||
|
||||
# =============================================================
|
||||
# 1. 数据库
|
||||
# =============================================================
|
||||
# 顺序很重要:先导数据库、后打包文件。
|
||||
# 热备份下两者必然是两个时刻的快照,若中间发生 push:
|
||||
# 先库后文件 → 文件里多出库中没有的对象(孤儿对象,无害)
|
||||
# 先文件后库 → 库里记录了文件中不存在的提交(引用悬空,有害)
|
||||
# 所以先导库是两害相权取其轻。要完全一致请用 --cold。
|
||||
log "正在备份 MySQL 数据库..."
|
||||
docker compose exec -T db mysqldump \
|
||||
-u root -p"${DB_ROOT_PASSWORD}" \
|
||||
--single-transaction \
|
||||
--routines \
|
||||
--triggers \
|
||||
--databases gitea \
|
||||
| gzip > "${BACKUP_DIR}/db_${DATE}.sql.gz"
|
||||
log "数据库备份完成: db_${DATE}.sql.gz ($(du -h "${BACKUP_DIR}/db_${DATE}.sql.gz" | cut -f1))"
|
||||
DB_FILE="${BACKUP_DIR}/db_${DATE}.sql.gz"
|
||||
|
||||
# 2. 备份 Gitea 数据(仓库、LFS、附件、头像等)
|
||||
log "正在备份 Gitea 数据目录(含 LFS)..."
|
||||
tar czf "${BACKUP_DIR}/gitea_data_${DATE}.tar.gz" \
|
||||
dump_opts=(-u root --single-transaction --routines --triggers
|
||||
--hex-blob --default-character-set=utf8mb4 --databases "$DB_NAME")
|
||||
# 无 GTID 的实例上 mysqldump 会写 SET @@GLOBAL.GTID_PURGED,回灌时报错。
|
||||
# 先收进变量再匹配,避免 grep -q 的 SIGPIPE 让探测结果反转。
|
||||
dump_help="$(db_exec mysqldump --help 2>/dev/null || true)"
|
||||
if grep -q 'set-gtid-purged' <<<"$dump_help"; then
|
||||
dump_opts+=(--set-gtid-purged=OFF)
|
||||
fi
|
||||
|
||||
db_exec mysqldump "${dump_opts[@]}" | gzip > "$DB_FILE"
|
||||
chmod 600 "$DB_FILE"
|
||||
verify_db_dump "$DB_FILE" || die "数据库备份未通过校验,已中止(不清理旧备份)"
|
||||
log "数据库备份完成: db_${DATE}.sql.gz ($(du -h "$DB_FILE" | cut -f1))"
|
||||
|
||||
# =============================================================
|
||||
# 2. Gitea 数据目录(仓库、LFS、附件、密钥)
|
||||
# =============================================================
|
||||
log "正在备份 Gitea 数据目录(含仓库与 LFS)..."
|
||||
DATA_FILE="${BACKUP_DIR}/gitea_data_${DATE}.tar.gz"
|
||||
tar czf "$DATA_FILE" \
|
||||
-C "$(dirname "$GITEA_DATA_DIR")" \
|
||||
"$(basename "$GITEA_DATA_DIR")"
|
||||
log "数据目录备份完成: gitea_data_${DATE}.tar.gz ($(du -h "${BACKUP_DIR}/gitea_data_${DATE}.tar.gz" | cut -f1))"
|
||||
# 包里有 SSH host key 和 OAuth2 JWT 签名密钥,权限必须收紧
|
||||
chmod 600 "$DATA_FILE"
|
||||
verify_data_tar "$DATA_FILE" || die "数据目录备份未通过校验,已中止(不清理旧备份)"
|
||||
log "数据目录备份完成: gitea_data_${DATE}.tar.gz ($(du -h "$DATA_FILE" | cut -f1))"
|
||||
|
||||
# 3. 备份配置文件
|
||||
# =============================================================
|
||||
# 3. 部署配置
|
||||
# =============================================================
|
||||
log "正在备份配置文件..."
|
||||
tar czf "${BACKUP_DIR}/config_${DATE}.tar.gz" \
|
||||
-C "$SCRIPT_DIR" \
|
||||
.env docker-compose.yml nginx/
|
||||
CONF_FILE="${BACKUP_DIR}/config_${DATE}.tar.gz"
|
||||
conf_items=(.env docker-compose.yml)
|
||||
[ -d nginx ] && conf_items+=(nginx)
|
||||
tar czf "$CONF_FILE" -C "$SCRIPT_DIR" "${conf_items[@]}"
|
||||
chmod 600 "$CONF_FILE" # 含 .env 里的数据库密码
|
||||
gzip -t "$CONF_FILE" 2>/dev/null || die "配置备份 gzip 校验失败"
|
||||
log "配置备份完成: config_${DATE}.tar.gz"
|
||||
|
||||
# 4. 清理过期备份
|
||||
log "清理 ${KEEP_DAYS} 天前的备份..."
|
||||
deleted=$(find "$BACKUP_DIR" -type f -mtime +${KEEP_DAYS} -print -delete | wc -l)
|
||||
log "已清理 ${deleted} 个过期文件"
|
||||
# 冷备份到此结束,尽早把服务放回去,后面的清理不该占用停机时间
|
||||
restart_app_if_stopped
|
||||
|
||||
# 5. 输出备份摘要
|
||||
# =============================================================
|
||||
# 4. 清理过期备份
|
||||
# =============================================================
|
||||
# 三条安全约束,缺一不可:
|
||||
# a) 只删本脚本自己产出的三种文件名,绝不碰 upgrade.sh 的 pre-upgrade-*
|
||||
# 回滚锚点,也不碰别人放进这个目录的东西
|
||||
# b) 按「备份集」整组判断,不按单个文件,避免删出半套残废备份
|
||||
# c) 无论多旧,至少保留 MIN_KEEP_SETS 份,防止长期备份失败后
|
||||
# 把最后一份能用的也清掉
|
||||
log "清理 ${KEEP_DAYS} 天前的备份(至少保留最近 ${MIN_KEEP_SETS} 份)..."
|
||||
|
||||
mapfile -t all_sets < <(
|
||||
find "$BACKUP_DIR" -maxdepth 1 -name 'db_*.sql.gz' -printf '%f\n' 2>/dev/null \
|
||||
| sed 's/^db_//; s/\.sql\.gz$//' | sort
|
||||
)
|
||||
total=${#all_sets[@]}
|
||||
deleted=0
|
||||
|
||||
if [ "$total" -gt "$MIN_KEEP_SETS" ]; then
|
||||
# 只考察最老的那些,末尾 MIN_KEEP_SETS 份无条件保留
|
||||
candidates=$(( total - MIN_KEEP_SETS ))
|
||||
for (( i = 0; i < candidates; i++ )); do
|
||||
ts="${all_sets[$i]}"
|
||||
f="${BACKUP_DIR}/db_${ts}.sql.gz"
|
||||
# 用 mtime 判断是否过期;-mmin 比 -mtime 精度更好
|
||||
if [ -n "$(find "$f" -maxdepth 0 -mmin +$(( KEEP_DAYS * 1440 )) 2>/dev/null)" ]; then
|
||||
rm -f "${BACKUP_DIR}/db_${ts}.sql.gz" \
|
||||
"${BACKUP_DIR}/gitea_data_${ts}.tar.gz" \
|
||||
"${BACKUP_DIR}/config_${ts}.tar.gz"
|
||||
log " 已删除过期备份集: $ts"
|
||||
deleted=$(( deleted + 1 ))
|
||||
fi
|
||||
done
|
||||
fi
|
||||
log "已清理 ${deleted} 份过期备份集,当前保留 $(( total - deleted )) 份"
|
||||
|
||||
# =============================================================
|
||||
# 5. 摘要
|
||||
# =============================================================
|
||||
echo ""
|
||||
log "===== 备份完成 ====="
|
||||
log "备份目录: ${BACKUP_DIR}/"
|
||||
log "备份集: ${DATE}(模式: ${MODE})"
|
||||
ls -lh "${BACKUP_DIR}/"*"${DATE}"* 2>/dev/null || true
|
||||
echo ""
|
||||
log "总备份空间占用: $(du -sh "${BACKUP_DIR}" | cut -f1)"
|
||||
log "备份目录: ${BACKUP_DIR}/"
|
||||
log "总占用: $(du -sh "$BACKUP_DIR" | cut -f1)"
|
||||
if [ "$MODE" = "hot" ]; then
|
||||
log "提示: 这是热备份,数据库与仓库文件取自不同时刻。"
|
||||
log " 需要完全一致的快照请用: bash backup.sh --cold"
|
||||
fi
|
||||
log "校验最近备份: bash backup.sh --verify-last"
|
||||
|
||||
+1143
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,36 @@
|
||||
# Local package verifier
|
||||
|
||||
`Verify(directory, expectedDigest)` accepts an absolute staging directory and a
|
||||
canonical `sha256:` digest of its exact `manifest.json` bytes. It returns the
|
||||
validated manifest, matching payload bytes keyed by relative path, and manifest
|
||||
digest. On failure it returns a zero `Verified` and fixed diagnostic text without
|
||||
embedding file contents, decoder errors, or filesystem paths.
|
||||
|
||||
The manifest is limited to 1 MiB, each payload to 4 MiB, and all payloads together
|
||||
to 16 MiB. Metadata checks precede opening regular files; actual reads also have
|
||||
a limit plus one overflow-detection byte and must match the observed size.
|
||||
The manifest digest is checked before shared `wire.Decode` parses it.
|
||||
|
||||
Inventory is derived from at most 128 declared files and their parent directories.
|
||||
Each directory is enumerated one entry at a time; an unexpected entry fails
|
||||
immediately. Symlinks, special files, empty/unnecessary directories, and unlisted
|
||||
files are rejected. Lowercase portable paths prevent case collisions. Files and
|
||||
subdirectories are opened relative to `os.Root`, with identity checks around open.
|
||||
|
||||
The caller must select a trusted staging directory with trusted ancestors and
|
||||
prevent concurrent mutation. These checks do not provide a transactional snapshot
|
||||
or complete protection against hostile concurrent filesystem changes. Consumers
|
||||
should use the returned bytes rather than reopen package files.
|
||||
|
||||
This authenticates bytes against a caller-provided trust anchor. It does not
|
||||
authenticate a publisher, validate Compose semantics, or authorize execution.
|
||||
Signature trust stores and executable policy are outside this package.
|
||||
|
||||
Tests use local temporary directories. Symlink cases skip only Windows error 1314
|
||||
(missing symlink privilege). Linux-specific FIFO cases verify rejection without
|
||||
opening the FIFO; run tests with a timeout, for example:
|
||||
|
||||
```text
|
||||
go test ./internal/appbundle -count=1 -timeout=30s
|
||||
go vet ./internal/appbundle
|
||||
```
|
||||
@@ -0,0 +1,48 @@
|
||||
package appbundle
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// Check lexical rejection independently of missing-file rejection. Invalid paths
|
||||
// cannot always be materialized on Windows, so a missing file is not sufficient
|
||||
// evidence that the manifest validator enforces portable paths.
|
||||
func TestPortablePathValidation(t *testing.T) {
|
||||
for _, path := range []string{
|
||||
"", "../escape", "a/../b", "a/./b", "/root", "c:/file", `a\b`,
|
||||
"a//b", "a/", ".env", "a/.secret", "a/secret ", "a/secret.",
|
||||
"con", "prn.txt", "aux.txt", "nul", "a/com1.log", "lpt9",
|
||||
"UPPER.txt", "a:stream", "café", strings.Repeat("a", 101),
|
||||
strings.Repeat("a/", 120) + "b",
|
||||
} {
|
||||
if validPath(path) {
|
||||
t.Errorf("accepted invalid path %q", path)
|
||||
}
|
||||
}
|
||||
for _, path := range []string{
|
||||
"compose.yaml", "dir-a/1_config.json", "com0.txt", "lpt10.txt",
|
||||
strings.Repeat("a", 100), strings.Repeat("a", 100) + "/" + strings.Repeat("b", 100) + "/" + strings.Repeat("c", 38),
|
||||
} {
|
||||
if !validPath(path) {
|
||||
t.Errorf("rejected valid path %q", path)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestImageDigestAndRepositoryValidation(t *testing.T) {
|
||||
for _, image := range []string{
|
||||
"api@sha256:" + strings.Repeat("a", 63), "api@sha256:" + strings.Repeat("A", 64),
|
||||
"api@sha256:" + strings.Repeat("a", 65), "api@sha512:" + strings.Repeat("a", 64),
|
||||
"api:tag@sha256:" + strings.Repeat("a", 64), "api@sha256:" + strings.Repeat("a", 64) + "@extra",
|
||||
} {
|
||||
if validImage(image) {
|
||||
t.Errorf("accepted invalid image %q", image)
|
||||
}
|
||||
}
|
||||
for _, repo := range []string{"a", "registry.example/team/api", "team_name/app-v2.0"} {
|
||||
if !validImage(repo + "@sha256:" + strings.Repeat("a", 64)) {
|
||||
t.Errorf("rejected valid repository %q", repo)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,312 @@
|
||||
// Package appbundle authenticates a bounded local bundle as bytes.
|
||||
package appbundle
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strings"
|
||||
|
||||
"server-deploy/internal/wire"
|
||||
)
|
||||
|
||||
const (
|
||||
manifestLimit int64 = 1 << 20
|
||||
fileLimit int64 = 4 << 20
|
||||
payloadLimit int64 = 16 << 20
|
||||
)
|
||||
|
||||
var (
|
||||
idPattern = regexp.MustCompile(`^[a-z][a-z0-9-]{0,47}$`)
|
||||
versionPattern = regexp.MustCompile(`^[0-9]+\.[0-9]+\.[0-9]+$`)
|
||||
digestPattern = regexp.MustCompile(`^sha256:[0-9a-f]{64}$`)
|
||||
repositoryPart = regexp.MustCompile(`^[a-z0-9]+(?:[._-]+[a-z0-9]+)*$`)
|
||||
pathPart = regexp.MustCompile(`^[a-z0-9][a-z0-9_.-]*$`)
|
||||
)
|
||||
|
||||
type Manifest struct {
|
||||
ProtocolVersion int `json:"protocolVersion"`
|
||||
AppID string `json:"appId"`
|
||||
Version string `json:"version"`
|
||||
Runtime string `json:"runtime"`
|
||||
Entrypoint string `json:"entrypoint"`
|
||||
Platforms []string `json:"platforms"`
|
||||
Components []Component `json:"components"`
|
||||
Files []File `json:"files"`
|
||||
}
|
||||
|
||||
type Component struct {
|
||||
Name string `json:"name"`
|
||||
Image string `json:"image"`
|
||||
}
|
||||
|
||||
type File struct {
|
||||
Path string `json:"path"`
|
||||
Digest string `json:"digest"`
|
||||
}
|
||||
|
||||
type Verified struct {
|
||||
Manifest Manifest
|
||||
Files map[string][]byte
|
||||
Digest string
|
||||
}
|
||||
|
||||
// Verify authenticates the exact manifest and listed payload bytes. The expected
|
||||
// digest is a caller trust anchor, not publisher authentication. Compose syntax
|
||||
// and execution safety are not evaluated. The caller must use a trusted staging
|
||||
// directory with trusted ancestors and prevent concurrent mutation; os.Root and
|
||||
// identity checks do not provide a transaction against hostile concurrent writers.
|
||||
func Verify(directory, expectedDigest string) (Verified, error) {
|
||||
if !filepath.IsAbs(directory) {
|
||||
return Verified{}, errors.New("bundle directory must be absolute")
|
||||
}
|
||||
// A trailing separator can make Lstat follow a directory symlink on Unix.
|
||||
// Normalize it before checking the caller-selected root itself.
|
||||
directory = filepath.Clean(directory)
|
||||
if !digestPattern.MatchString(expectedDigest) {
|
||||
return Verified{}, errors.New("invalid expected manifest digest")
|
||||
}
|
||||
info, err := os.Lstat(directory)
|
||||
if err != nil || !info.IsDir() || info.Mode()&os.ModeSymlink != 0 {
|
||||
return Verified{}, errors.New("invalid bundle directory")
|
||||
}
|
||||
root, err := os.OpenRoot(directory)
|
||||
if err != nil {
|
||||
return Verified{}, errors.New("cannot open bundle directory")
|
||||
}
|
||||
defer root.Close()
|
||||
openedInfo, err := root.Stat(".")
|
||||
if err != nil || !os.SameFile(info, openedInfo) {
|
||||
return Verified{}, errors.New("bundle directory changed")
|
||||
}
|
||||
raw, err := readRegular(root, "manifest.json", manifestLimit)
|
||||
if err != nil {
|
||||
return Verified{}, errors.New("cannot read bounded regular manifest")
|
||||
}
|
||||
if hash(raw) != expectedDigest {
|
||||
return Verified{}, errors.New("manifest digest mismatch")
|
||||
}
|
||||
var manifest Manifest
|
||||
if err := wire.Decode(bytes.NewReader(raw), &manifest, manifestLimit); err != nil {
|
||||
// Never propagate decoder errors: some include offending input values.
|
||||
return Verified{}, errors.New("invalid manifest JSON")
|
||||
}
|
||||
tree, err := validate(manifest)
|
||||
if err != nil {
|
||||
return Verified{}, err
|
||||
}
|
||||
files := make(map[string][]byte, len(manifest.Files))
|
||||
remaining := payloadLimit
|
||||
if err := readInventory(root, tree, "", files, &remaining); err != nil {
|
||||
return Verified{}, err
|
||||
}
|
||||
return Verified{Manifest: manifest, Files: files, Digest: expectedDigest}, nil
|
||||
}
|
||||
|
||||
func hash(data []byte) string {
|
||||
sum := sha256.Sum256(data)
|
||||
return "sha256:" + hex.EncodeToString(sum[:])
|
||||
}
|
||||
|
||||
// readRegular caps both the pre-open size and actual bytes read. The one extra
|
||||
// byte detects growth or oversize without an unbounded read, even after Stat.
|
||||
// Names are single validated path segments relative to an already-open Root.
|
||||
func readRegular(root *os.Root, name string, limit int64) ([]byte, error) {
|
||||
info, err := root.Lstat(name)
|
||||
if err != nil || !info.Mode().IsRegular() || info.Size() < 0 || info.Size() > limit {
|
||||
return nil, errors.New("invalid file type or size")
|
||||
}
|
||||
f, err := root.Open(name)
|
||||
if err != nil {
|
||||
return nil, errors.New("cannot open file")
|
||||
}
|
||||
defer f.Close()
|
||||
openedInfo, err := f.Stat()
|
||||
if err != nil || !openedInfo.Mode().IsRegular() || !os.SameFile(info, openedInfo) || openedInfo.Size() != info.Size() {
|
||||
return nil, errors.New("file changed before read")
|
||||
}
|
||||
data, err := io.ReadAll(io.LimitReader(f, limit+1))
|
||||
if err != nil || int64(len(data)) > limit || int64(len(data)) != openedInfo.Size() {
|
||||
return nil, errors.New("file read exceeds bounds or changed")
|
||||
}
|
||||
return data, nil
|
||||
}
|
||||
|
||||
type inventory struct {
|
||||
children map[string]*inventory
|
||||
digest string
|
||||
}
|
||||
|
||||
func validate(m Manifest) (*inventory, error) {
|
||||
if m.ProtocolVersion != 1 || m.Runtime != "compose" || !idPattern.MatchString(m.AppID) || !versionPattern.MatchString(m.Version) {
|
||||
return nil, errors.New("unsupported or invalid manifest identity")
|
||||
}
|
||||
if len(m.Platforms) < 1 || len(m.Platforms) > 2 {
|
||||
return nil, errors.New("invalid platforms")
|
||||
}
|
||||
platforms := make(map[string]bool)
|
||||
for _, platform := range m.Platforms {
|
||||
if (platform != "linux/amd64" && platform != "linux/arm64") || platforms[platform] {
|
||||
return nil, errors.New("invalid platforms")
|
||||
}
|
||||
platforms[platform] = true
|
||||
}
|
||||
if len(m.Components) < 1 || len(m.Components) > 32 {
|
||||
return nil, errors.New("invalid component count")
|
||||
}
|
||||
names := make(map[string]bool)
|
||||
for _, c := range m.Components {
|
||||
if !idPattern.MatchString(c.Name) || names[c.Name] || !validImage(c.Image) {
|
||||
return nil, errors.New("invalid component")
|
||||
}
|
||||
names[c.Name] = true
|
||||
}
|
||||
if len(m.Files) < 1 || len(m.Files) > 128 {
|
||||
return nil, errors.New("invalid file count")
|
||||
}
|
||||
tree := &inventory{children: map[string]*inventory{"manifest.json": {}}}
|
||||
paths := make(map[string]bool)
|
||||
for _, file := range m.Files {
|
||||
if !validPath(file.Path) || file.Path == "manifest.json" || !digestPattern.MatchString(file.Digest) || paths[file.Path] {
|
||||
return nil, errors.New("invalid file declaration")
|
||||
}
|
||||
paths[file.Path] = true
|
||||
node := tree
|
||||
parts := strings.Split(file.Path, "/")
|
||||
for i, part := range parts {
|
||||
next, exists := node.children[part]
|
||||
if i == len(parts)-1 {
|
||||
if exists {
|
||||
return nil, errors.New("conflicting file paths")
|
||||
}
|
||||
node.children[part] = &inventory{digest: file.Digest}
|
||||
} else {
|
||||
if exists && next.children == nil {
|
||||
return nil, errors.New("conflicting file paths")
|
||||
}
|
||||
if !exists {
|
||||
next = &inventory{children: make(map[string]*inventory)}
|
||||
node.children[part] = next
|
||||
}
|
||||
node = next
|
||||
}
|
||||
}
|
||||
}
|
||||
if !paths[m.Entrypoint] {
|
||||
return nil, errors.New("entrypoint must be a listed file")
|
||||
}
|
||||
return tree, nil
|
||||
}
|
||||
|
||||
func validImage(image string) bool {
|
||||
repo, pin, found := strings.Cut(image, "@")
|
||||
if !found || !digestPattern.MatchString(pin) {
|
||||
return false
|
||||
}
|
||||
for _, part := range strings.Split(repo, "/") {
|
||||
if !repositoryPart.MatchString(part) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func validPath(path string) bool {
|
||||
if len(path) == 0 || len(path) > 240 {
|
||||
return false
|
||||
}
|
||||
for _, part := range strings.Split(path, "/") {
|
||||
if len(part) > 100 || !pathPart.MatchString(part) || strings.HasSuffix(part, ".") {
|
||||
return false
|
||||
}
|
||||
base, _, _ := strings.Cut(part, ".")
|
||||
switch base {
|
||||
case "con", "prn", "aux", "nul", "conin$", "conout$":
|
||||
return false
|
||||
}
|
||||
if len(base) == 4 && (strings.HasPrefix(base, "com") || strings.HasPrefix(base, "lpt")) && base[3] >= '1' && base[3] <= '9' {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// readInventory enumerates only declared directories, one entry at a time. Each
|
||||
// directory consumes at most its declared child count plus one entry; extras
|
||||
// fail immediately, with no unbounded ReadDir or recursive filesystem walk.
|
||||
func readInventory(root *os.Root, node *inventory, prefix string, files map[string][]byte, remaining *int64) error {
|
||||
dir, err := root.Open(".")
|
||||
if err != nil {
|
||||
return errors.New("cannot enumerate bundle")
|
||||
}
|
||||
defer dir.Close()
|
||||
seen := make(map[string]bool, len(node.children))
|
||||
for {
|
||||
entries, err := dir.ReadDir(1)
|
||||
if err != nil && err != io.EOF {
|
||||
return errors.New("cannot enumerate bundle")
|
||||
}
|
||||
if len(entries) == 0 {
|
||||
if err == io.EOF {
|
||||
break
|
||||
}
|
||||
return errors.New("invalid directory enumeration")
|
||||
}
|
||||
entry := entries[0]
|
||||
name := entry.Name()
|
||||
child, exists := node.children[name]
|
||||
if !exists || seen[name] {
|
||||
return errors.New("unexpected bundle entry")
|
||||
}
|
||||
seen[name] = true
|
||||
info, statErr := root.Lstat(name)
|
||||
if statErr != nil || info.Mode()&os.ModeSymlink != 0 {
|
||||
return errors.New("invalid bundle entry")
|
||||
}
|
||||
if child.children != nil {
|
||||
if !info.IsDir() {
|
||||
return errors.New("expected bundle directory")
|
||||
}
|
||||
sub, openErr := root.OpenRoot(name)
|
||||
if openErr != nil {
|
||||
return errors.New("cannot open bundle subdirectory")
|
||||
}
|
||||
openedInfo, statErr := sub.Stat(".")
|
||||
if statErr != nil || !os.SameFile(info, openedInfo) {
|
||||
sub.Close()
|
||||
return errors.New("bundle subdirectory changed")
|
||||
}
|
||||
err := readInventory(sub, child, prefix+name+"/", files, remaining)
|
||||
sub.Close()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
continue
|
||||
}
|
||||
if !info.Mode().IsRegular() {
|
||||
return errors.New("expected regular bundle file")
|
||||
}
|
||||
if prefix == "" && name == "manifest.json" {
|
||||
continue
|
||||
}
|
||||
limit := min(fileLimit, *remaining)
|
||||
data, readErr := readRegular(root, name, limit)
|
||||
if readErr != nil {
|
||||
return errors.New("cannot read bounded regular payload")
|
||||
}
|
||||
if hash(data) != child.digest {
|
||||
return errors.New("payload digest mismatch")
|
||||
}
|
||||
*remaining -= int64(len(data))
|
||||
files[prefix+name] = data
|
||||
}
|
||||
if len(seen) != len(node.children) {
|
||||
return errors.New("missing bundle entry")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,29 @@
|
||||
package appbundle_test
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"syscall"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestRejectFIFOWithoutOpening(t *testing.T) {
|
||||
for _, name := range []string{"manifest.json", "config.txt", "extra"} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
files := basicFiles()
|
||||
dir, pin := packageDir(t, manifest(files), files)
|
||||
path := filepath.Join(dir, name)
|
||||
if name != "extra" {
|
||||
if err := os.Remove(path); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if err := syscall.Mkfifo(path, 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// Opening a FIFO for reading would block. Verification must reject its
|
||||
// metadata before opening it; the parent Linux run uses a test timeout.
|
||||
rejected(t, dir, pin)
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,408 @@
|
||||
package appbundle_test
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"runtime"
|
||||
"strings"
|
||||
"syscall"
|
||||
"testing"
|
||||
|
||||
"server-deploy/internal/appbundle"
|
||||
)
|
||||
|
||||
func digest(data []byte) string {
|
||||
sum := sha256.Sum256(data)
|
||||
return "sha256:" + hex.EncodeToString(sum[:])
|
||||
}
|
||||
|
||||
func manifest(files map[string][]byte) map[string]any {
|
||||
entries := []any{}
|
||||
for path, data := range files {
|
||||
entries = append(entries, map[string]any{"path": path, "digest": digest(data)})
|
||||
}
|
||||
return map[string]any{
|
||||
"protocolVersion": 1, "appId": "demo-app", "version": "1.2.3",
|
||||
"runtime": "compose", "entrypoint": "compose.yaml",
|
||||
"platforms": []any{"linux/amd64", "linux/arm64"},
|
||||
"components": []any{
|
||||
map[string]any{"name": "api", "image": "registry.example/team/api@sha256:" + strings.Repeat("a", 64)},
|
||||
map[string]any{"name": "db", "image": "db@sha256:" + strings.Repeat("b", 64)},
|
||||
},
|
||||
"files": entries,
|
||||
}
|
||||
}
|
||||
|
||||
func basicFiles() map[string][]byte {
|
||||
return map[string][]byte{"compose.yaml": []byte("services: {}\n"), "config.txt": []byte("known fixture\n")}
|
||||
}
|
||||
|
||||
func write(t *testing.T, dir, path string, data []byte) {
|
||||
t.Helper()
|
||||
full := filepath.Join(dir, filepath.FromSlash(path))
|
||||
if err := os.MkdirAll(filepath.Dir(full), 0700); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(full, data, 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func packageDir(t *testing.T, m map[string]any, files map[string][]byte) (string, string) {
|
||||
t.Helper()
|
||||
dir := t.TempDir()
|
||||
for path, data := range files {
|
||||
write(t, dir, path, data)
|
||||
}
|
||||
return dir, saveManifest(t, dir, m)
|
||||
}
|
||||
|
||||
func saveManifest(t *testing.T, dir string, m map[string]any) string {
|
||||
t.Helper()
|
||||
raw, err := json.Marshal(m)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
write(t, dir, "manifest.json", raw)
|
||||
return digest(raw)
|
||||
}
|
||||
|
||||
func rejected(t *testing.T, dir, pin string) error {
|
||||
t.Helper()
|
||||
v, err := appbundle.Verify(dir, pin)
|
||||
if err == nil {
|
||||
t.Fatal("accepted invalid package")
|
||||
}
|
||||
if v.Digest != "" || v.Files != nil || v.Manifest.AppID != "" {
|
||||
t.Fatal("returned partial verified data on error")
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func TestVerifyReturnsAuthenticatedBytes(t *testing.T) {
|
||||
files := basicFiles()
|
||||
files["nested/config/data.txt"] = []byte("nested bytes")
|
||||
dir, pin := packageDir(t, manifest(files), files)
|
||||
v, err := appbundle.Verify(dir, pin)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if v.Digest != pin || v.Manifest.AppID != "demo-app" || len(v.Manifest.Components) != 2 || len(v.Files) != 3 {
|
||||
t.Fatalf("incorrect verified result: digest=%q files=%d", v.Digest, len(v.Files))
|
||||
}
|
||||
for path, data := range files {
|
||||
if !bytes.Equal(v.Files[path], data) {
|
||||
t.Fatalf("incorrect bytes for %s", path)
|
||||
}
|
||||
}
|
||||
write(t, dir, "compose.yaml", []byte("later mutation"))
|
||||
if !bytes.Equal(v.Files["compose.yaml"], []byte("services: {}\n")) {
|
||||
t.Fatal("verified bytes changed after disk mutation")
|
||||
}
|
||||
}
|
||||
|
||||
func TestManifestPinAndParseBoundary(t *testing.T) {
|
||||
for _, tc := range []struct{ name, raw, pin string }{
|
||||
{"wrong pin", "not json", "sha256:" + strings.Repeat("0", 64)},
|
||||
{"empty pin", "{}", ""},
|
||||
{"bare pin", "{}", strings.Repeat("a", 64)},
|
||||
{"uppercase pin", "{}", "sha256:" + strings.Repeat("A", 64)},
|
||||
{"malformed authenticated manifest", "{SECRET_CONTENT", "auto"},
|
||||
{"oversized manifest", strings.Repeat(" ", (1<<20)+1), "auto"},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
write(t, dir, "manifest.json", []byte(tc.raw))
|
||||
pin := tc.pin
|
||||
if pin == "auto" {
|
||||
pin = digest([]byte(tc.raw))
|
||||
}
|
||||
err := rejected(t, dir, pin)
|
||||
if strings.Contains(err.Error(), "SECRET_CONTENT") {
|
||||
t.Fatal("manifest bytes leaked in error")
|
||||
}
|
||||
if tc.name == "wrong pin" && !strings.Contains(strings.ToLower(err.Error()), "digest") {
|
||||
t.Fatal("digest must be checked before JSON parsing")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestManifestValidation(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
change func(map[string]any)
|
||||
}{
|
||||
{"protocol", func(m map[string]any) { m["protocolVersion"] = 2 }},
|
||||
{"runtime", func(m map[string]any) { m["runtime"] = "shell" }},
|
||||
{"app id", func(m map[string]any) { m["appId"] = "Demo" }},
|
||||
{"long id", func(m map[string]any) { m["appId"] = strings.Repeat("a", 49) }},
|
||||
{"version", func(m map[string]any) { m["version"] = "1.2.3-beta" }},
|
||||
{"empty platforms", func(m map[string]any) { m["platforms"] = []any{} }},
|
||||
{"platform", func(m map[string]any) { m["platforms"] = []any{"windows/amd64"} }},
|
||||
{"duplicate platform", func(m map[string]any) { m["platforms"] = []any{"linux/amd64", "linux/amd64"} }},
|
||||
{"null platform", func(m map[string]any) { m["platforms"] = []any{nil} }},
|
||||
{"empty components", func(m map[string]any) { m["components"] = []any{} }},
|
||||
{"duplicate component", func(m map[string]any) { c := m["components"].([]any); c[1].(map[string]any)["name"] = "api" }},
|
||||
{"invalid component name", func(m map[string]any) { m["components"].([]any)[0].(map[string]any)["name"] = "1api" }},
|
||||
{"missing component field", func(m map[string]any) { delete(m["components"].([]any)[0].(map[string]any), "image") }},
|
||||
{"unknown component field", func(m map[string]any) { m["components"].([]any)[0].(map[string]any)["secret"] = "SECRET_CONTENT" }},
|
||||
{"null component field", func(m map[string]any) { m["components"].([]any)[0].(map[string]any)["image"] = nil }},
|
||||
{"null component", func(m map[string]any) { m["components"] = []any{nil} }},
|
||||
{"too many components", func(m map[string]any) {
|
||||
c := []any{}
|
||||
for i := 0; i < 33; i++ {
|
||||
c = append(c, map[string]any{"name": "c-" + strings.Repeat("a", i), "image": "api@sha256:" + strings.Repeat("a", 64)})
|
||||
}
|
||||
m["components"] = c
|
||||
}},
|
||||
{"empty files", func(m map[string]any) { m["files"] = []any{} }},
|
||||
{"duplicate file", func(m map[string]any) { f := m["files"].([]any); m["files"] = append(f, f[0]) }},
|
||||
{"null file", func(m map[string]any) { m["files"] = []any{nil} }},
|
||||
{"bad file digest", func(m map[string]any) {
|
||||
m["files"].([]any)[0].(map[string]any)["digest"] = "sha256:" + strings.Repeat("A", 64)
|
||||
}},
|
||||
{"entrypoint absent", func(m map[string]any) { m["entrypoint"] = "missing.yaml" }},
|
||||
{"null files", func(m map[string]any) { m["files"] = nil }},
|
||||
{"missing top field", func(m map[string]any) { delete(m, "appId") }},
|
||||
{"unknown top field", func(m map[string]any) { m["secret"] = "SECRET_CONTENT" }},
|
||||
{"case alias", func(m map[string]any) { m["AppId"] = m["appId"]; delete(m, "appId") }},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
files := basicFiles()
|
||||
m := manifest(files)
|
||||
tc.change(m)
|
||||
dir, pin := packageDir(t, m, files)
|
||||
err := rejected(t, dir, pin)
|
||||
if strings.Contains(err.Error(), "SECRET_CONTENT") {
|
||||
t.Fatal("manifest content leaked")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestRejectImageSubsetViolations(t *testing.T) {
|
||||
for _, repo := range []string{"api:latest", "api", "host:5000/api", "UPPER/api", "a//b", "/api", "api/", "a/$b", "a;b", "a b", ".api", "api."} {
|
||||
t.Run(repo, func(t *testing.T) {
|
||||
files := basicFiles()
|
||||
m := manifest(files)
|
||||
img := repo + "@sha256:" + strings.Repeat("a", 64)
|
||||
if repo == "api:latest" || repo == "api" {
|
||||
img = repo
|
||||
}
|
||||
m["components"].([]any)[0].(map[string]any)["image"] = img
|
||||
dir, pin := packageDir(t, m, files)
|
||||
rejected(t, dir, pin)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestRejectNonportablePaths(t *testing.T) {
|
||||
for _, path := range []string{"../escape", "a/../b", "a/./b", "/root", "c:/file", `a\b`, "a//b", "a/", ".env", "a/.secret", "a/secret ", "a/secret.", "con", "aux.txt", "a/com1.log", "lpt9", "CLOCK$", "UPPER.txt", "a:stream", "manifest.json", strings.Repeat("a", 101), strings.Repeat("a/", 120) + "b"} {
|
||||
t.Run(path, func(t *testing.T) {
|
||||
files := basicFiles()
|
||||
m := manifest(files)
|
||||
m["files"] = append(m["files"].([]any), map[string]any{"path": path, "digest": digest(nil)})
|
||||
dir, pin := packageDir(t, m, files)
|
||||
rejected(t, dir, pin)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestExactInventory(t *testing.T) {
|
||||
for _, mode := range []string{"changed", "missing", "extra", "secret", "directory", "nested extra", "file directory conflict", "case collision"} {
|
||||
t.Run(mode, func(t *testing.T) {
|
||||
files := basicFiles()
|
||||
files["nested/file.txt"] = []byte("nested")
|
||||
m := manifest(files)
|
||||
dir, pin := packageDir(t, m, files)
|
||||
switch mode {
|
||||
case "changed":
|
||||
write(t, dir, "config.txt", []byte("SECRET_CONTENT"))
|
||||
case "missing":
|
||||
if err := os.Remove(filepath.Join(dir, "config.txt")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
case "extra":
|
||||
write(t, dir, "extra.txt", []byte("SECRET_CONTENT"))
|
||||
case "secret":
|
||||
write(t, dir, ".env", []byte("SECRET_CONTENT"))
|
||||
case "directory":
|
||||
if err := os.Mkdir(filepath.Join(dir, "unused"), 0700); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
case "nested extra":
|
||||
write(t, dir, "nested/extra.txt", []byte("SECRET_CONTENT"))
|
||||
case "file directory conflict":
|
||||
m["files"] = append(m["files"].([]any), map[string]any{"path": "config.txt/child", "digest": digest(nil)})
|
||||
pin = saveManifest(t, dir, m)
|
||||
case "case collision":
|
||||
m["files"] = append(m["files"].([]any), map[string]any{"path": "CONFIG.txt", "digest": digest(nil)})
|
||||
pin = saveManifest(t, dir, m)
|
||||
}
|
||||
err := rejected(t, dir, pin)
|
||||
if strings.Contains(err.Error(), "SECRET_CONTENT") {
|
||||
t.Fatal("file contents leaked")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestPayloadBounds(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
sizes []int
|
||||
valid bool
|
||||
}{
|
||||
{"file at limit", []int{4 << 20}, true},
|
||||
{"file over limit", []int{(4 << 20) + 1}, false},
|
||||
{"total at limit", []int{4 << 20, 4 << 20, 4 << 20, 4 << 20}, true},
|
||||
{"total over limit", []int{4 << 20, 4 << 20, 4 << 20, 4 << 20, 1}, false},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
files := map[string][]byte{}
|
||||
for i, size := range tc.sizes {
|
||||
path := string(rune('a'+i)) + ".txt"
|
||||
if i == 0 {
|
||||
path = "compose.yaml"
|
||||
}
|
||||
files[path] = bytes.Repeat([]byte("x"), size)
|
||||
}
|
||||
dir, pin := packageDir(t, manifest(files), files)
|
||||
if tc.valid {
|
||||
if _, err := appbundle.Verify(dir, pin); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
} else {
|
||||
rejected(t, dir, pin)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func symlink(t *testing.T, target, link string) {
|
||||
t.Helper()
|
||||
if err := os.Symlink(target, link); err != nil {
|
||||
if runtime.GOOS == "windows" && errors.Is(err, syscall.Errno(1314)) {
|
||||
t.Skip("Windows symlink privilege unavailable")
|
||||
}
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRejectSymlinks(t *testing.T) {
|
||||
for _, mode := range []string{"file", "manifest", "intermediate", "extra", "root", "root trailing separator"} {
|
||||
t.Run(mode, func(t *testing.T) {
|
||||
files := basicFiles()
|
||||
dir, pin := packageDir(t, manifest(files), files)
|
||||
switch mode {
|
||||
case "file", "manifest":
|
||||
name := "config.txt"
|
||||
if mode == "manifest" {
|
||||
name = "manifest.json"
|
||||
}
|
||||
path := filepath.Join(dir, name)
|
||||
target := filepath.Join(t.TempDir(), "target")
|
||||
if err := os.Rename(path, target); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
symlink(t, target, path)
|
||||
case "intermediate":
|
||||
outside := t.TempDir()
|
||||
write(t, outside, "data.txt", []byte("outside"))
|
||||
symlink(t, outside, filepath.Join(dir, "nested"))
|
||||
m := manifest(files)
|
||||
m["files"] = append(m["files"].([]any), map[string]any{"path": "nested/data.txt", "digest": digest([]byte("outside"))})
|
||||
pin = saveManifest(t, dir, m)
|
||||
case "extra":
|
||||
symlink(t, t.TempDir(), filepath.Join(dir, "extra"))
|
||||
case "root", "root trailing separator":
|
||||
link := filepath.Join(t.TempDir(), "bundle")
|
||||
symlink(t, dir, link)
|
||||
dir = link
|
||||
if mode == "root trailing separator" {
|
||||
dir += string(os.PathSeparator)
|
||||
}
|
||||
}
|
||||
rejected(t, dir, pin)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestAbsoluteDirectoryRequired(t *testing.T) { rejected(t, ".", "sha256:"+strings.Repeat("a", 64)) }
|
||||
|
||||
func TestRejectTrailingAndDuplicateJSON(t *testing.T) {
|
||||
for _, suffix := range []string{" {}", ",\"appId\":\"other\"}"} {
|
||||
t.Run(suffix, func(t *testing.T) {
|
||||
files := basicFiles()
|
||||
dir, _ := packageDir(t, manifest(files), files)
|
||||
raw, err := os.ReadFile(filepath.Join(dir, "manifest.json"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if strings.HasPrefix(suffix, ",") {
|
||||
raw = raw[:len(raw)-1]
|
||||
}
|
||||
raw = append(raw, []byte(suffix)...)
|
||||
write(t, dir, "manifest.json", raw)
|
||||
rejected(t, dir, digest(raw))
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestManifestExactByteLimit(t *testing.T) {
|
||||
files := basicFiles()
|
||||
dir, _ := packageDir(t, manifest(files), files)
|
||||
raw, err := os.ReadFile(filepath.Join(dir, "manifest.json"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
originalPin := digest(raw)
|
||||
raw = append(raw, bytes.Repeat([]byte(" "), (1<<20)-len(raw))...)
|
||||
write(t, dir, "manifest.json", raw)
|
||||
rejected(t, dir, originalPin) // Whitespace must change the exact-byte pin.
|
||||
if _, err := appbundle.Verify(dir, digest(raw)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestInventoryCountBoundaries(t *testing.T) {
|
||||
for _, n := range []int{128, 129} {
|
||||
t.Run(fmt.Sprint(n), func(t *testing.T) {
|
||||
files := map[string][]byte{"compose.yaml": {}}
|
||||
for i := 1; i < n; i++ {
|
||||
files[fmt.Sprintf("file-%d.txt", i)] = []byte{}
|
||||
}
|
||||
dir, pin := packageDir(t, manifest(files), files)
|
||||
if n == 128 {
|
||||
if _, err := appbundle.Verify(dir, pin); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
} else {
|
||||
rejected(t, dir, pin)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestComponentCountBoundary(t *testing.T) {
|
||||
files := basicFiles()
|
||||
m := manifest(files)
|
||||
components := []any{}
|
||||
for i := 0; i < 32; i++ {
|
||||
components = append(components, map[string]any{"name": fmt.Sprintf("component-%d", i), "image": "api@sha256:" + strings.Repeat("a", 64)})
|
||||
}
|
||||
m["components"] = components
|
||||
m["appId"] = strings.Repeat("a", 48)
|
||||
dir, pin := packageDir(t, m, files)
|
||||
if _, err := appbundle.Verify(dir, pin); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,90 @@
|
||||
# Staged Docker repository authentication
|
||||
|
||||
`deployctl verify-repository` accepts strict JSON with `directory` (absolute trusted
|
||||
staging directory), `suite`, `architecture`, and `versions` (exact version strings
|
||||
for docker-ce, docker-ce-cli, containerd.io, docker-buildx-plugin and
|
||||
docker-compose-plugin). It does not select a latest version or resolve dependencies.
|
||||
|
||||
The directory must contain `docker.asc`, `Release`, `Release.gpg` and uncompressed
|
||||
`Packages`. The caller obtains these from the Docker Ubuntu repository. No runtime
|
||||
network request is performed by this command. Limits are 1 MiB, 1 MiB, 64 KiB and
|
||||
16 MiB respectively. Files must be nonempty regular files; symlinks are rejected.
|
||||
Trusted ancestors and absence of concurrent writers are required. This is not an
|
||||
atomic filesystem snapshot against hostile writers. Additional staging files are
|
||||
ignored, never executed.
|
||||
|
||||
## Trust and verification
|
||||
|
||||
1. Exact armored key SHA-256 is pinned in source. Initial trust was bootstrapped
|
||||
from `https://download.docker.com/linux/ubuntu/gpg`, not supplied by the request.
|
||||
Primary fingerprint: `9DC858229FC7DD38854AE2D88D81803C0EBFCD88`.
|
||||
Rotation or formatting changes require a reviewed source update; fail closed.
|
||||
2. Linux `/usr/bin/gpg` dearmors into a new private temporary directory; `/usr/bin/gpgv`
|
||||
verifies the detached signature against the copied Release bytes, with that
|
||||
keyring and isolated homedir. No shell, ambient GnuPG config or personal keyring.
|
||||
Each child has a 15-second timeout and bounded output. Errors do not expose raw
|
||||
GnuPG diagnostics or metadata. Non-Linux or missing tools fail closed.
|
||||
3. Successful process exit and a single accepted primary-fingerprint signature
|
||||
are required; weak digests, expired/revoked/bad signatures and unknown status
|
||||
types fail closed. SHA-256/384/512 are accepted.
|
||||
4. Authenticated Release must describe Docker CE and the requested supported Ubuntu
|
||||
suite, architecture and stable component. Date must be within 30 days and no
|
||||
more than 10 minutes in the future; Valid-Until is enforced when present.
|
||||
5. The exact uncompressed stable Packages entry's SHA-256 and byte size must match.
|
||||
Five explicitly requested versions are resolved to authenticated index records;
|
||||
the derived lock is checked by installplan's source/path/version constraints.
|
||||
|
||||
The machine clock, OS and installed GnuPG are trusted. This is not a persistent
|
||||
anti-rollback ledger: an older authentic Release inside the freshness window can
|
||||
pass, and a missing Valid-Until is governed by the local 30-day policy. No online
|
||||
key revocation lookup is performed. Key pin maintenance is an operator responsibility.
|
||||
|
||||
## Result boundaries
|
||||
|
||||
`repositoryAuthenticated: true` attests to these metadata bytes at `verifiedAt`.
|
||||
For `verify-repository`, `packageBytesVerified: false` and `executable: false` remain explicit: no deb bytes,
|
||||
Ubuntu dependency repository, dependency closure, installation scripts, system
|
||||
compatibility or service behavior have been verified. The result is not a signed
|
||||
capability. `plan-environment` still treats a supplied lock as untrusted and does
|
||||
not accept a caller's authentication claim to clear its trust blockers.
|
||||
|
||||
Signature scratch data is removed on normal return; a killed process can leave its own
|
||||
private temporary directory. Deployment writes remain disabled. `writesEnabled`
|
||||
in `version` refers to deployment writes, not verification scratch files.
|
||||
|
||||
`scripts/probe-docker-repository.sh /absolute/path/to/deployctl` is an optional
|
||||
local online integration check. It downloads public metadata over HTTPS into a
|
||||
new temporary directory, tests real authentication and rejects tampering. Versions
|
||||
selected by this test are fixtures, not recommended installation versions. It
|
||||
does not install packages, alter APT, use SSH or touch application data.
|
||||
|
||||
## Verify actual deb bytes
|
||||
|
||||
`verify-artifacts` requires the same request fields as `verify-repository`, plus
|
||||
`artifactDirectory`: an absolute trusted directory containing exactly the five deb
|
||||
files named by the basename of each authenticated `Filename`. No other entries,
|
||||
subdirectories, symlinks or special files are accepted. The two directories must
|
||||
not be concurrently modified. Repository signatures and metadata are verified
|
||||
anew in the same call; the command accepts neither a supplied lock nor trust flags.
|
||||
|
||||
Each file is streamed through SHA-256 with a 64 KiB copy buffer and its signed
|
||||
index size as the read bound (plus one overflow-detection byte). The existing
|
||||
lock policy caps each file at 512 MiB and requires exactly five files. A last-file
|
||||
failure rejects the entire result. No partial successful report is emitted.
|
||||
The verifier never unpacks a deb or executes its contents. It does not change
|
||||
the staged files. Metadata identity checks supplement, not replace, the trusted
|
||||
directory/no concurrent writer requirement.
|
||||
|
||||
Only `verify-artifacts` may set `packageBytesVerified: true`; `executable` stays
|
||||
false. This verifies the selected five Docker package bytes, not the complete
|
||||
Ubuntu dependency closure, package-internal safety or installation compatibility.
|
||||
It is a point-in-time observation: do not reuse this JSON to authorize later
|
||||
execution of paths that may have changed. A future installer must recheck or
|
||||
own immutable verified snapshots under its operation lock.
|
||||
|
||||
Set `DEPLOYCTL_ONLINE_ARTIFACT_PROBE=1` along with
|
||||
`DEPLOYCTL_ONLINE_REPOSITORY_PROBE=1` when running `scripts/verify-linux.sh` to
|
||||
also download the five real public deb fixtures into a new local temporary
|
||||
directory. The optional probe verifies all files, then changes one byte without
|
||||
changing length and asserts rejection. It retains test files for inspection;
|
||||
they are not installed and one is intentionally damaged by the negative test.
|
||||
@@ -0,0 +1,117 @@
|
||||
package aptrepo
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
"io"
|
||||
"os"
|
||||
"path"
|
||||
"path/filepath"
|
||||
"time"
|
||||
|
||||
"server-deploy/internal/installplan"
|
||||
)
|
||||
|
||||
// VerifyArtifacts authenticates repository metadata anew before checking the
|
||||
// selected five deb files. It never accepts a caller-asserted authenticated lock.
|
||||
// The staging directories and ancestors must be trusted and not concurrently
|
||||
// modified. This is observation evidence, not a capability to later execute paths.
|
||||
func VerifyArtifacts(metadataDirectory, artifactDirectory, suite, arch string, versions map[string]string, now time.Time) (Result, error) {
|
||||
result, err := Verify(metadataDirectory, suite, arch, versions, now)
|
||||
if err != nil {
|
||||
return Result{}, err
|
||||
}
|
||||
if err := verifyArtifacts(artifactDirectory, result.Lock); err != nil {
|
||||
return Result{}, err
|
||||
}
|
||||
result.PackageBytesVerified = true
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func verifyArtifacts(directory string, lock installplan.Lock) error {
|
||||
reject := errors.New("invalid staged Docker artifacts")
|
||||
if _, err := installplan.Validate(lock, lock.Suite, lock.Architecture); err != nil {
|
||||
return reject
|
||||
}
|
||||
if !filepath.IsAbs(directory) {
|
||||
return reject
|
||||
}
|
||||
directory = filepath.Clean(directory)
|
||||
before, err := os.Lstat(directory)
|
||||
if err != nil || !before.IsDir() || before.Mode()&os.ModeSymlink != 0 {
|
||||
return reject
|
||||
}
|
||||
root, err := os.OpenRoot(directory)
|
||||
if err != nil {
|
||||
return reject
|
||||
}
|
||||
defer root.Close()
|
||||
opened, err := root.Stat(".")
|
||||
if err != nil || !os.SameFile(before, opened) {
|
||||
return reject
|
||||
}
|
||||
expected := make(map[string]installplan.Package, len(lock.Packages))
|
||||
for _, p := range lock.Packages {
|
||||
expected[path.Base(p.Filename)] = p
|
||||
}
|
||||
dir, err := root.Open(".")
|
||||
if err != nil {
|
||||
return reject
|
||||
}
|
||||
defer dir.Close()
|
||||
// Enumerate at most the expected count plus one, never an unbounded directory.
|
||||
seen := make(map[string]bool)
|
||||
for {
|
||||
entries, err := dir.ReadDir(1)
|
||||
if err != nil && err != io.EOF {
|
||||
return reject
|
||||
}
|
||||
if len(entries) == 0 {
|
||||
if err == io.EOF {
|
||||
break
|
||||
}
|
||||
return reject
|
||||
}
|
||||
name := entries[0].Name()
|
||||
p, ok := expected[name]
|
||||
if !ok || seen[name] {
|
||||
return reject
|
||||
}
|
||||
seen[name] = true
|
||||
if err := verifyArtifact(root, name, p); err != nil {
|
||||
return reject
|
||||
}
|
||||
}
|
||||
if len(seen) != len(expected) {
|
||||
return reject
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func verifyArtifact(root *os.Root, name string, p installplan.Package) error {
|
||||
reject := errors.New("artifact size or digest mismatch")
|
||||
before, err := root.Lstat(name)
|
||||
if err != nil || !before.Mode().IsRegular() || before.Size() != int64(p.Size) {
|
||||
return reject
|
||||
}
|
||||
file, err := root.Open(name)
|
||||
if err != nil {
|
||||
return reject
|
||||
}
|
||||
defer file.Close()
|
||||
opened, err := file.Stat()
|
||||
if err != nil || !opened.Mode().IsRegular() || !os.SameFile(before, opened) || opened.Size() != before.Size() {
|
||||
return reject
|
||||
}
|
||||
digest := sha256.New()
|
||||
n, err := io.CopyBuffer(digest, io.LimitReader(file, int64(p.Size)+1), make([]byte, 64<<10))
|
||||
if err != nil || n != int64(p.Size) || "sha256:"+hex.EncodeToString(digest.Sum(nil)) != p.Digest {
|
||||
return reject
|
||||
}
|
||||
after, err := file.Stat()
|
||||
if err != nil || after.Size() != opened.Size() || !after.ModTime().Equal(opened.ModTime()) {
|
||||
return reject
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,44 @@
|
||||
package aptrepo
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path"
|
||||
"path/filepath"
|
||||
"syscall"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestArtifactRejectsSymlinksAndFIFO(t *testing.T) {
|
||||
for _, kind := range []string{"root-link", "file-link", "fifo"} {
|
||||
t.Run(kind, func(t *testing.T) {
|
||||
dir, lock := artifactFixture(t)
|
||||
filename := filepath.Join(dir, path.Base(lock.Packages[0].Filename))
|
||||
switch kind {
|
||||
case "root-link":
|
||||
link := filepath.Join(t.TempDir(), "link")
|
||||
if err := os.Symlink(dir, link); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
dir = link + "/"
|
||||
case "file-link":
|
||||
target := filepath.Join(t.TempDir(), "target")
|
||||
if err := os.Rename(filename, target); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.Symlink(target, filename); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
case "fifo":
|
||||
if err := os.Remove(filename); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := syscall.Mkfifo(filename, 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if err := verifyArtifacts(dir, lock); err == nil {
|
||||
t.Fatal("unsafe file accepted")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,99 @@
|
||||
package aptrepo
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"fmt"
|
||||
"os"
|
||||
"path"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"testing"
|
||||
|
||||
"server-deploy/internal/installplan"
|
||||
)
|
||||
|
||||
func artifactFixture(t *testing.T) (string, installplan.Lock) {
|
||||
t.Helper()
|
||||
dir := t.TempDir()
|
||||
index := fixtureIndex()
|
||||
lock, err := Resolve(fixtureRelease(index), index, "resolute", "amd64", fixturePins(), fixtureNow)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for i := range lock.Packages {
|
||||
p := &lock.Packages[i]
|
||||
data := []byte("artifact fixture " + p.Name)
|
||||
p.Size = uint64(len(data))
|
||||
p.Digest = fmt.Sprintf("sha256:%x", sha256.Sum256(data))
|
||||
if err := os.WriteFile(filepath.Join(dir, path.Base(p.Filename)), data, 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
return dir, lock
|
||||
}
|
||||
|
||||
func TestArtifactBytesAndIdentity(t *testing.T) {
|
||||
dir, lock := artifactFixture(t)
|
||||
before := lock
|
||||
before.Packages = append([]installplan.Package(nil), lock.Packages...)
|
||||
if err := verifyArtifacts(dir, lock); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !reflect.DeepEqual(before, lock) {
|
||||
t.Fatal("lock mutated")
|
||||
}
|
||||
for _, p := range lock.Packages {
|
||||
data, err := os.ReadFile(filepath.Join(dir, path.Base(p.Filename)))
|
||||
if err != nil || string(data) != "artifact fixture "+p.Name {
|
||||
t.Fatal("artifact modified")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestArtifactRejectsTamperWithoutPartialSuccess(t *testing.T) {
|
||||
for _, kind := range []string{"same-size", "truncated", "extra-byte", "missing", "directory", "bad-lock", "relative-root", "extra-file"} {
|
||||
t.Run(kind, func(t *testing.T) {
|
||||
dir, lock := artifactFixture(t)
|
||||
p := lock.Packages[4] // The last artifact must fail the whole set.
|
||||
filename := filepath.Join(dir, path.Base(p.Filename))
|
||||
switch kind {
|
||||
case "same-size":
|
||||
data := []byte("artifact fixture " + p.Name)
|
||||
data[0] = 'X'
|
||||
if err := os.WriteFile(filename, data, 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
case "truncated":
|
||||
if err := os.Truncate(filename, int64(p.Size)-1); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
case "extra-byte":
|
||||
if err := os.Truncate(filename, int64(p.Size)+1); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
case "missing":
|
||||
if err := os.Remove(filename); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
case "directory":
|
||||
if err := os.Remove(filename); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.Mkdir(filename, 0700); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
case "bad-lock":
|
||||
lock.Packages[0].Filename = "../../secret.deb"
|
||||
case "relative-root":
|
||||
dir = "relative"
|
||||
case "extra-file":
|
||||
if err := os.WriteFile(filepath.Join(dir, "unexpected.deb"), []byte("x"), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if err := verifyArtifacts(dir, lock); err == nil {
|
||||
t.Fatal("invalid artifacts accepted")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,232 @@
|
||||
package aptrepo
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
"unicode"
|
||||
"unicode/utf8"
|
||||
|
||||
"server-deploy/internal/installplan"
|
||||
)
|
||||
|
||||
var metadataError = errors.New("invalid Docker repository metadata")
|
||||
|
||||
var pinnedPackageNames = [...]string{"docker-ce", "docker-ce-cli", "containerd.io", "docker-buildx-plugin", "docker-compose-plugin"}
|
||||
|
||||
// Resolve parses Release bytes already authenticated by the caller and binds
|
||||
// explicitly pinned packages to that Release. It does not verify signatures.
|
||||
func Resolve(release, index []byte, suite, arch string, versions map[string]string, now time.Time) (installplan.Lock, error) {
|
||||
if len(release) > 1<<20 || len(index) > 16<<20 || len(versions) != len(pinnedPackageNames) {
|
||||
return installplan.Lock{}, metadataError
|
||||
}
|
||||
for _, name := range pinnedPackageNames {
|
||||
if versions[name] == "" {
|
||||
return installplan.Lock{}, metadataError
|
||||
}
|
||||
}
|
||||
|
||||
var fields map[string]string
|
||||
if err := parseControl(release, func(stanza map[string]string) error {
|
||||
if fields != nil {
|
||||
return metadataError
|
||||
}
|
||||
fields = stanza
|
||||
return nil
|
||||
}); err != nil {
|
||||
return installplan.Lock{}, metadataError
|
||||
}
|
||||
if fields["suite"] != suite || fields["origin"] != "Docker" || fields["label"] != "Docker CE" || !hasWord(fields["architectures"], arch) || !hasWord(fields["components"], "stable") {
|
||||
return installplan.Lock{}, metadataError
|
||||
}
|
||||
date, err := time.Parse(time.RFC1123Z, fields["date"])
|
||||
if err != nil || date.After(now.Add(10*time.Minute)) || date.Before(now.Add(-30*24*time.Hour)) {
|
||||
return installplan.Lock{}, metadataError
|
||||
}
|
||||
if value, ok := fields["valid-until"]; ok {
|
||||
expiry, err := time.Parse(time.RFC1123Z, value)
|
||||
if err != nil || !expiry.After(now) || expiry.Before(date) {
|
||||
return installplan.Lock{}, metadataError
|
||||
}
|
||||
}
|
||||
// Bind the exact uncompressed index bytes before interpreting any records.
|
||||
indexSum := sha256.Sum256(index)
|
||||
expectedPath := "stable/binary-" + arch + "/Packages"
|
||||
seenPaths := make(map[string]bool)
|
||||
matched := false
|
||||
for _, line := range strings.Split(fields["sha256"], "\n") {
|
||||
if strings.TrimSpace(line) == "" {
|
||||
continue
|
||||
}
|
||||
entry := strings.Fields(line)
|
||||
if len(entry) != 3 || !validSHA256(entry[0]) || seenPaths[entry[2]] {
|
||||
return installplan.Lock{}, metadataError
|
||||
}
|
||||
seenPaths[entry[2]] = true
|
||||
size, err := decimalSize(entry[1])
|
||||
if err != nil {
|
||||
return installplan.Lock{}, metadataError
|
||||
}
|
||||
if entry[2] == expectedPath {
|
||||
if size != uint64(len(index)) || entry[0] != hex.EncodeToString(indexSum[:]) {
|
||||
return installplan.Lock{}, metadataError
|
||||
}
|
||||
matched = true
|
||||
}
|
||||
}
|
||||
if !matched {
|
||||
return installplan.Lock{}, metadataError
|
||||
}
|
||||
|
||||
selected := make(map[string]installplan.Package)
|
||||
seenRecords := make(map[[3]string]bool)
|
||||
if err := parseControl(index, func(record map[string]string) error {
|
||||
identity := [3]string{record["package"], record["version"], record["architecture"]}
|
||||
if identity[0] != "" && identity[1] != "" && identity[2] != "" {
|
||||
if seenRecords[identity] {
|
||||
return metadataError
|
||||
}
|
||||
seenRecords[identity] = true
|
||||
}
|
||||
version, target := versions[identity[0]]
|
||||
if !target {
|
||||
return nil
|
||||
}
|
||||
for _, field := range []string{"package", "version", "architecture", "filename", "size", "sha256"} {
|
||||
if record[field] == "" || strings.Contains(record[field], "\n") {
|
||||
return metadataError
|
||||
}
|
||||
}
|
||||
if identity[1] != version || identity[2] != arch {
|
||||
return nil
|
||||
}
|
||||
size, err := decimalSize(record["size"])
|
||||
if err != nil {
|
||||
return metadataError
|
||||
}
|
||||
selected[identity[0]] = installplan.Package{
|
||||
Name: identity[0], Version: version, Filename: record["filename"],
|
||||
Size: size, Digest: "sha256:" + record["sha256"],
|
||||
}
|
||||
return nil
|
||||
}); err != nil {
|
||||
return installplan.Lock{}, metadataError
|
||||
}
|
||||
|
||||
releaseSum := sha256.Sum256(release)
|
||||
lock := installplan.Lock{
|
||||
ProtocolVersion: 1, Repository: "https://download.docker.com/linux/ubuntu",
|
||||
Suite: suite, Architecture: arch, ReleaseDigest: "sha256:" + hex.EncodeToString(releaseSum[:]),
|
||||
}
|
||||
for _, name := range pinnedPackageNames {
|
||||
p, ok := selected[name]
|
||||
if !ok {
|
||||
return installplan.Lock{}, metadataError
|
||||
}
|
||||
lock.Packages = append(lock.Packages, p)
|
||||
}
|
||||
if _, err := installplan.Validate(lock, suite, arch); err != nil {
|
||||
return installplan.Lock{}, metadataError
|
||||
}
|
||||
return lock, nil
|
||||
}
|
||||
|
||||
// parseControl preserves continuation boundaries and rejects duplicate fields
|
||||
// before invoking visit. Processing one stanza at a time bounds retained values.
|
||||
func parseControl(raw []byte, visit func(map[string]string) error) error {
|
||||
if !utf8.Valid(raw) {
|
||||
return metadataError
|
||||
}
|
||||
text := strings.ReplaceAll(string(raw), "\r\n", "\n")
|
||||
for _, r := range text {
|
||||
if unicode.IsControl(r) && r != '\n' && r != '\t' {
|
||||
return metadataError
|
||||
}
|
||||
}
|
||||
fields := make(map[string]*strings.Builder)
|
||||
current := ""
|
||||
flush := func() error {
|
||||
if len(fields) == 0 {
|
||||
return nil
|
||||
}
|
||||
stanza := make(map[string]string, len(fields))
|
||||
for name, value := range fields {
|
||||
stanza[name] = value.String()
|
||||
}
|
||||
if err := visit(stanza); err != nil {
|
||||
return err
|
||||
}
|
||||
fields = make(map[string]*strings.Builder)
|
||||
current = ""
|
||||
return nil
|
||||
}
|
||||
for line := range strings.SplitSeq(text, "\n") {
|
||||
if line == "" {
|
||||
if err := flush(); err != nil {
|
||||
return err
|
||||
}
|
||||
continue
|
||||
}
|
||||
if line[0] == ' ' || line[0] == '\t' {
|
||||
if current == "" {
|
||||
return metadataError
|
||||
}
|
||||
fields[current].WriteByte('\n')
|
||||
fields[current].WriteString(strings.Trim(line, " \t"))
|
||||
continue
|
||||
}
|
||||
name, value, ok := strings.Cut(line, ":")
|
||||
if !ok || name == "" {
|
||||
return metadataError
|
||||
}
|
||||
for _, r := range name {
|
||||
if !(r >= 'a' && r <= 'z' || r >= 'A' && r <= 'Z' || r >= '0' && r <= '9' || r == '-') {
|
||||
return metadataError
|
||||
}
|
||||
}
|
||||
current = strings.ToLower(name)
|
||||
if _, exists := fields[current]; exists {
|
||||
return metadataError
|
||||
}
|
||||
fields[current] = &strings.Builder{}
|
||||
fields[current].WriteString(strings.Trim(value, " \t"))
|
||||
}
|
||||
return flush()
|
||||
}
|
||||
|
||||
func hasWord(value, word string) bool {
|
||||
for _, item := range strings.Fields(value) {
|
||||
if item == word {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func decimalSize(value string) (uint64, error) {
|
||||
for _, r := range value {
|
||||
if r < '0' || r > '9' {
|
||||
return 0, metadataError
|
||||
}
|
||||
}
|
||||
size, err := strconv.ParseUint(value, 10, 64)
|
||||
if err != nil {
|
||||
return 0, metadataError
|
||||
}
|
||||
return size, nil
|
||||
}
|
||||
|
||||
func validSHA256(value string) bool {
|
||||
if len(value) != 64 {
|
||||
return false
|
||||
}
|
||||
for _, r := range value {
|
||||
if !(r >= '0' && r <= '9' || r >= 'a' && r <= 'f') {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
@@ -0,0 +1,327 @@
|
||||
package aptrepo
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"fmt"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"server-deploy/internal/installplan"
|
||||
)
|
||||
|
||||
var fixtureNow = time.Date(2026, 9, 25, 6, 38, 50, 0, time.UTC)
|
||||
|
||||
func fixturePins() map[string]string {
|
||||
return map[string]string{
|
||||
"docker-ce": "5:29.1.0-1~ubuntu.26.04~resolute",
|
||||
"docker-ce-cli": "5:29.1.0-1~ubuntu.26.04~resolute",
|
||||
"containerd.io": "2.1.5-1~ubuntu.26.04~resolute",
|
||||
"docker-buildx-plugin": "0.30.1-1~ubuntu.26.04~resolute",
|
||||
"docker-compose-plugin": "2.40.3-1~ubuntu.26.04~resolute",
|
||||
}
|
||||
}
|
||||
|
||||
// Inline Debian control fixtures retain Docker's Release field layout (notably
|
||||
// no Codename) and epoch-free pool filenames. Artifact hashes are test data;
|
||||
// authentication of Release is outside Resolve's contract.
|
||||
func fixtureRecord(name, version, arch string) string {
|
||||
fileVersion := version
|
||||
if _, after, ok := strings.Cut(version, ":"); ok {
|
||||
fileVersion = after
|
||||
}
|
||||
return fmt.Sprintf("Package: %s\nVersion: %s\nArchitecture: %s\nMaintainer: Docker <support@docker.com>\nFilename: dists/resolute/pool/stable/%s/%s_%s_%s.deb\nSize: 12345\nSHA256: %s\nDescription: Docker package\n continuation with a colon: allowed\n .\n another paragraph\n\n", name, version, arch, arch, name, fileVersion, arch, strings.Repeat("a", 64))
|
||||
}
|
||||
|
||||
func fixtureIndex() []byte {
|
||||
pins := fixturePins()
|
||||
var index strings.Builder
|
||||
for _, name := range []string{"docker-ce", "docker-ce-cli", "containerd.io", "docker-buildx-plugin", "docker-compose-plugin"} {
|
||||
index.WriteString(fixtureRecord(name, pins[name], "amd64"))
|
||||
}
|
||||
return []byte(index.String())
|
||||
}
|
||||
|
||||
func fixtureRelease(index []byte) []byte {
|
||||
return []byte(fmt.Sprintf("Architectures: amd64 arm64 armhf s390x ppc64el\nComponents: stable edge test nightly\nDate: Thu, 24 Sep 2026 06:38:50 +0000\nLabel: Docker CE\nOrigin: Docker\nSuite: resolute\nSHA256:\n %x %d stable/binary-amd64/Packages\n", sha256.Sum256(index), len(index)))
|
||||
}
|
||||
|
||||
func replace(raw []byte, old, new string) []byte {
|
||||
return []byte(strings.Replace(string(raw), old, new, 1))
|
||||
}
|
||||
|
||||
func assertRejected(t *testing.T, release, index []byte, suite, arch string, pins map[string]string, now time.Time) {
|
||||
t.Helper()
|
||||
lock, err := Resolve(release, index, suite, arch, pins, now)
|
||||
if err == nil {
|
||||
t.Fatal("invalid metadata accepted")
|
||||
}
|
||||
if !reflect.DeepEqual(lock, installplan.Lock{}) {
|
||||
t.Fatal("failure returned a partial lock")
|
||||
}
|
||||
// Rejection messages must not disclose any untrusted metadata or pins.
|
||||
if strings.Contains(err.Error(), "PRIVATE-MARKER") {
|
||||
t.Fatal("error echoed metadata")
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveDockerMetadataChain(t *testing.T) {
|
||||
index := fixtureIndex()
|
||||
release := fixtureRelease(index)
|
||||
lock, err := Resolve(release, index, "resolute", "amd64", fixturePins(), fixtureNow)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if lock.ProtocolVersion != 1 || lock.Repository != "https://download.docker.com/linux/ubuntu" || lock.Suite != "resolute" || lock.Architecture != "amd64" || lock.ReleaseDigest != fmt.Sprintf("sha256:%x", sha256.Sum256(release)) {
|
||||
t.Fatalf("incorrect release binding: %+v", lock)
|
||||
}
|
||||
if len(lock.Packages) != 5 {
|
||||
t.Fatal("incorrect package count")
|
||||
}
|
||||
want := installplan.Package{Name: "docker-ce", Version: "5:29.1.0-1~ubuntu.26.04~resolute", Filename: "dists/resolute/pool/stable/amd64/docker-ce_29.1.0-1~ubuntu.26.04~resolute_amd64.deb", Digest: "sha256:" + strings.Repeat("a", 64), Size: 12345}
|
||||
if lock.Packages[0] != want {
|
||||
t.Fatalf("wrong selected package: %+v", lock.Packages[0])
|
||||
}
|
||||
for _, p := range lock.Packages {
|
||||
if p.Version != fixturePins()[p.Name] {
|
||||
t.Fatal("pin was not preserved")
|
||||
}
|
||||
}
|
||||
if _, err := installplan.Validate(lock, "resolute", "amd64"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveCompatibleControlFormatting(t *testing.T) {
|
||||
for _, mode := range []string{"mixed case", "CRLF", "no final newline", "other versions and architectures", "arm64", "valid expiry", "future boundary", "age boundary"} {
|
||||
t.Run(mode, func(t *testing.T) {
|
||||
index, arch, now := fixtureIndex(), "amd64", fixtureNow
|
||||
switch mode {
|
||||
case "mixed case":
|
||||
index = []byte(strings.ReplaceAll(string(index), "Package:", "pAcKaGe:"))
|
||||
case "CRLF":
|
||||
index = []byte(strings.ReplaceAll(string(index), "\n", "\r\n"))
|
||||
case "no final newline":
|
||||
index = []byte(strings.TrimRight(string(index), "\n"))
|
||||
case "other versions and architectures":
|
||||
index = append(index, fixtureRecord("docker-ce", "5:99.0-1", "amd64")...)
|
||||
index = append(index, fixtureRecord("docker-ce", fixturePins()["docker-ce"], "arm64")...)
|
||||
case "arm64":
|
||||
arch = "arm64"
|
||||
index = []byte(strings.ReplaceAll(string(index), "amd64", arch))
|
||||
case "future boundary":
|
||||
now = fixtureNow.Add(-24*time.Hour - 10*time.Minute)
|
||||
case "age boundary":
|
||||
now = fixtureNow.Add(29 * 24 * time.Hour)
|
||||
}
|
||||
release := fixtureRelease(index)
|
||||
switch mode {
|
||||
case "mixed case":
|
||||
release = replace(release, "SHA256:", "sHa256:")
|
||||
release = replace(release, "Suite:", "sUiTe:")
|
||||
case "CRLF":
|
||||
release = []byte(strings.ReplaceAll(string(release), "\n", "\r\n"))
|
||||
case "no final newline":
|
||||
release = []byte(strings.TrimRight(string(release), "\n"))
|
||||
case "arm64":
|
||||
release = replace(release, "binary-amd64/Packages", "binary-arm64/Packages")
|
||||
case "valid expiry":
|
||||
release = append(release, "Valid-Until: Sat, 26 Sep 2026 06:38:50 +0000\n"...)
|
||||
}
|
||||
lock, err := Resolve(release, index, "resolute", arch, fixturePins(), now)
|
||||
if err != nil || len(lock.Packages) != 5 || lock.Architecture != arch {
|
||||
t.Fatalf("compatible metadata rejected: %v", err)
|
||||
}
|
||||
if lock.ReleaseDigest != fmt.Sprintf("sha256:%x", sha256.Sum256(release)) {
|
||||
t.Fatal("digest did not bind original bytes")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveRejectsReleaseMetadata(t *testing.T) {
|
||||
index := fixtureIndex()
|
||||
for name, mutate := range map[string]func([]byte) []byte{
|
||||
"suite": func(r []byte) []byte { return replace(r, "Suite: resolute", "Suite: noble") },
|
||||
"codename cannot replace suite": func(r []byte) []byte { return replace(r, "Suite:", "Codename:") },
|
||||
"origin": func(r []byte) []byte { return replace(r, "Origin: Docker", "Origin: PRIVATE-MARKER") },
|
||||
"label": func(r []byte) []byte { return replace(r, "Label: Docker CE", "Label: Other") },
|
||||
"arch token": func(r []byte) []byte { return replace(r, "amd64 arm64", "xamd64 arm64") },
|
||||
"component token": func(r []byte) []byte { return replace(r, "stable edge", "unstable edge") },
|
||||
"invalid date": func(r []byte) []byte { return replace(r, "Thu, 24 Sep 2026 06:38:50 +0000", "PRIVATE-MARKER") },
|
||||
"future date": func(r []byte) []byte {
|
||||
return replace(r, "Thu, 24 Sep 2026 06:38:50 +0000", "Fri, 25 Sep 2026 06:48:51 +0000")
|
||||
},
|
||||
"stale date": func(r []byte) []byte {
|
||||
return replace(r, "Thu, 24 Sep 2026 06:38:50 +0000", "Wed, 26 Aug 2026 06:38:49 +0000")
|
||||
},
|
||||
"expired": func(r []byte) []byte { return append(r, "Valid-Until: Fri, 25 Sep 2026 06:38:50 +0000\n"...) },
|
||||
"invalid expiry": func(r []byte) []byte { return append(r, "Valid-Until: PRIVATE-MARKER\n"...) },
|
||||
"duplicate case insensitive field": func(r []byte) []byte { return append(r, "oRiGiN: Docker\n"...) },
|
||||
"duplicate unrelated field": func(r []byte) []byte { return append(r, "X-Info: one\nx-info: two\n"...) },
|
||||
"second stanza": func(r []byte) []byte { return append(r, "\nSuite: resolute\n"...) },
|
||||
"MD5 only": func(r []byte) []byte { return replace(r, "SHA256:", "MD5Sum:") },
|
||||
"SHA1 only": func(r []byte) []byte { return replace(r, "SHA256:", "SHA1:") },
|
||||
"compressed only": func(r []byte) []byte { return replace(r, "/Packages", "/Packages.gz") },
|
||||
"path prefix": func(r []byte) []byte { return replace(r, "stable/binary", "./stable/binary") },
|
||||
"checksum arch": func(r []byte) []byte { return replace(r, "binary-amd64", "binary-arm64") },
|
||||
"checksum size": func(r []byte) []byte {
|
||||
return replace(r, fmt.Sprintf(" %d ", len(index)), fmt.Sprintf(" %d ", len(index)+1))
|
||||
},
|
||||
"checksum negative size": func(r []byte) []byte { return replace(r, fmt.Sprintf(" %d ", len(index)), " -1 ") },
|
||||
"checksum extra column": func(r []byte) []byte { return replace(r, "/Packages\n", "/Packages extra\n") },
|
||||
"checksum invalid digest": func(r []byte) []byte {
|
||||
return replace(r, fmt.Sprintf("%x", sha256.Sum256(index)), strings.Repeat("g", 64))
|
||||
},
|
||||
"duplicate checksum entry": func(r []byte) []byte {
|
||||
return append(r, fmt.Sprintf(" %x %d stable/binary-amd64/Packages\n", sha256.Sum256(index), len(index))...)
|
||||
},
|
||||
"conflicting checksum entry": func(r []byte) []byte {
|
||||
return append(r, fmt.Sprintf(" %s %d stable/binary-amd64/Packages\n", strings.Repeat("b", 64), len(index))...)
|
||||
},
|
||||
"duplicate other checksum entry": func(r []byte) []byte {
|
||||
return append(r, strings.Repeat(" "+strings.Repeat("a", 64)+" 1 other/Packages\n", 2)...)
|
||||
},
|
||||
"orphan continuation": func(r []byte) []byte { return append([]byte(" orphan\n"), r...) },
|
||||
"invalid field name": func(r []byte) []byte { return append(r, "Bad Field: value\n"...) },
|
||||
} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
assertRejected(t, mutate(fixtureRelease(index)), index, "resolute", "amd64", fixturePins(), fixtureNow)
|
||||
})
|
||||
}
|
||||
for _, field := range []string{"Architectures", "Components", "Date", "Label", "Origin", "Suite"} {
|
||||
t.Run("missing "+field, func(t *testing.T) {
|
||||
r := fixtureRelease(index)
|
||||
lines := strings.Split(string(r), "\n")
|
||||
for i, line := range lines {
|
||||
if strings.HasPrefix(line, field+":") {
|
||||
lines = append(lines[:i], lines[i+1:]...)
|
||||
break
|
||||
}
|
||||
}
|
||||
assertRejected(t, []byte(strings.Join(lines, "\n")), index, "resolute", "amd64", fixturePins(), fixtureNow)
|
||||
})
|
||||
}
|
||||
// Both times are in the future relative to now, but expiry precedes Date.
|
||||
r := append(fixtureRelease(index), "Valid-Until: Thu, 24 Sep 2026 06:37:50 +0000\n"...)
|
||||
assertRejected(t, r, index, "resolute", "amd64", fixturePins(), fixtureNow.Add(-24*time.Hour-2*time.Minute))
|
||||
}
|
||||
|
||||
func TestResolveRejectsTamperedIndex(t *testing.T) {
|
||||
index := fixtureIndex()
|
||||
r := fixtureRelease(index)
|
||||
index = replace(index, "Size: 12345", "Size: 12346") // Same byte size, different digest.
|
||||
assertRejected(t, r, index, "resolute", "amd64", fixturePins(), fixtureNow)
|
||||
}
|
||||
|
||||
func TestResolveRejectsPackageAmbiguityAndInvalidLock(t *testing.T) {
|
||||
for name, mutate := range map[string]func([]byte) []byte{
|
||||
"duplicate field": func(p []byte) []byte {
|
||||
return replace(p, "Package: docker-ce\n", "Package: docker-ce\npAcKaGe: docker-ce\n")
|
||||
},
|
||||
"duplicate unrelated field": func(p []byte) []byte { return replace(p, "Description:", "X-Info: one\nx-info: two\nDescription:") },
|
||||
"duplicate record": func(p []byte) []byte {
|
||||
return append(p, fixtureRecord("docker-ce", fixturePins()["docker-ce"], "amd64")...)
|
||||
},
|
||||
"conflicting record": func(p []byte) []byte {
|
||||
return append(p, strings.Replace(fixtureRecord("docker-ce", fixturePins()["docker-ce"], "amd64"), "Size: 12345", "Size: 999", 1)...)
|
||||
},
|
||||
"missing record": func(p []byte) []byte { return []byte(strings.SplitN(string(p), "\n\n", 2)[1]) },
|
||||
"wrong architecture": func(p []byte) []byte { return replace(p, "Architecture: amd64", "Architecture: all") },
|
||||
"wrong version": func(p []byte) []byte { return replace(p, "Version: 5:29.1.0", "Version: 5:29.2.0") },
|
||||
"unsafe filename": func(p []byte) []byte { return replace(p, "Filename: dists/resolute", "Filename: ../PRIVATE-MARKER") },
|
||||
"wrong filename version": func(p []byte) []byte { return replace(p, "docker-ce_29.1.0", "docker-ce_29.2.0") },
|
||||
"epoch filename": func(p []byte) []byte { return replace(p, "docker-ce_29.1.0", "docker-ce_5:29.1.0") },
|
||||
"wrong filename suite": func(p []byte) []byte { return replace(p, "Filename: dists/resolute", "Filename: dists/noble") },
|
||||
"zero size": func(p []byte) []byte { return replace(p, "Size: 12345", "Size: 0") },
|
||||
"negative size": func(p []byte) []byte { return replace(p, "Size: 12345", "Size: -1") },
|
||||
"overflow size": func(p []byte) []byte { return replace(p, "Size: 12345", "Size: 18446744073709551616") },
|
||||
"excess package size": func(p []byte) []byte { return replace(p, "Size: 12345", "Size: 536870913") },
|
||||
"bad digest": func(p []byte) []byte { return replace(p, "SHA256: "+strings.Repeat("a", 64), "SHA256: PRIVATE-MARKER") },
|
||||
"folded required field": func(p []byte) []byte { return replace(p, "Size: 12345", "Size: 12345\n 6") },
|
||||
"invalid syntax": func(p []byte) []byte { return append(p, "PRIVATE-MARKER\n"...) },
|
||||
} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
index := mutate(fixtureIndex())
|
||||
assertRejected(t, fixtureRelease(index), index, "resolute", "amd64", fixturePins(), fixtureNow)
|
||||
})
|
||||
}
|
||||
for _, field := range []string{"Package", "Version", "Architecture", "Filename", "Size", "SHA256"} {
|
||||
t.Run("missing "+field, func(t *testing.T) {
|
||||
index := replace(fixtureIndex(), field+":", "X-Removed:")
|
||||
assertRejected(t, fixtureRelease(index), index, "resolute", "amd64", fixturePins(), fixtureNow)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveRequiresExactExplicitPins(t *testing.T) {
|
||||
for _, mode := range []string{"nil", "missing", "extra", "empty", "latest", "engine mismatch", "invalid version"} {
|
||||
t.Run(mode, func(t *testing.T) {
|
||||
pins := fixturePins()
|
||||
switch mode {
|
||||
case "nil":
|
||||
pins = nil
|
||||
case "missing":
|
||||
delete(pins, "containerd.io")
|
||||
case "extra":
|
||||
pins["unexpected"] = "1.0"
|
||||
case "empty":
|
||||
pins["containerd.io"] = ""
|
||||
case "latest":
|
||||
pins["containerd.io"] = "latest"
|
||||
case "engine mismatch":
|
||||
pins["docker-ce-cli"] = "5:29.2.0-1~ubuntu.26.04~resolute"
|
||||
case "invalid version":
|
||||
pins["containerd.io"] = "1;PRIVATE-MARKER"
|
||||
}
|
||||
// Make invalid versions available too: Validate, rather than a missing
|
||||
// match, must enforce version syntax and engine/CLI equality.
|
||||
index := fixtureIndex()
|
||||
for _, name := range []string{"containerd.io", "docker-ce-cli"} {
|
||||
if v := pins[name]; v != "" && v != fixturePins()[name] {
|
||||
index = replace(index, fixtureRecord(name, fixturePins()[name], "amd64"), fixtureRecord(name, v, "amd64"))
|
||||
}
|
||||
}
|
||||
assertRejected(t, fixtureRelease(index), index, "resolute", "amd64", pins, fixtureNow)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveRejectsInvalidText(t *testing.T) {
|
||||
for _, invalid := range []string{"\x00", "\x01", "\x1b", "\x7f", "\u0085", "\r", "\xff"} {
|
||||
t.Run(fmt.Sprintf("%x", invalid), func(t *testing.T) {
|
||||
index := fixtureIndex()
|
||||
r := append(fixtureRelease(index), "X-Info: PRIVATE-MARKER"+invalid+"suffix\n"...)
|
||||
assertRejected(t, r, index, "resolute", "amd64", fixturePins(), fixtureNow)
|
||||
index = append(index, "Package: unrelated\nDescription: PRIVATE-MARKER"+invalid+"suffix\n"...)
|
||||
assertRejected(t, fixtureRelease(index), index, "resolute", "amd64", fixturePins(), fixtureNow)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveExactByteLimits(t *testing.T) {
|
||||
for _, target := range []string{"release", "index"} {
|
||||
for _, excess := range []int{0, 1} {
|
||||
t.Run(fmt.Sprintf("%s+%d", target, excess), func(t *testing.T) {
|
||||
index := fixtureIndex()
|
||||
if target == "index" {
|
||||
index = append(index, "Package: unrelated\nDescription: "...)
|
||||
index = append(index, strings.Repeat("x", (16<<20)+excess-len(index)-1)...)
|
||||
index = append(index, '\n')
|
||||
}
|
||||
release := fixtureRelease(index)
|
||||
if target == "release" {
|
||||
release = append(release, "X-Padding: "...)
|
||||
release = append(release, strings.Repeat("x", (1<<20)+excess-len(release)-1)...)
|
||||
release = append(release, '\n')
|
||||
}
|
||||
if excess != 0 {
|
||||
assertRejected(t, release, index, "resolute", "amd64", fixturePins(), fixtureNow)
|
||||
} else if _, err := Resolve(release, index, "resolute", "amd64", fixturePins(), fixtureNow); err != nil {
|
||||
t.Fatalf("exact limit rejected: %v", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,169 @@
|
||||
package aptrepo
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
"io"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"runtime"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Bootstrapped from Docker's official HTTPS key endpoint. Rotation requires a
|
||||
// reviewed code update, never a caller-supplied key or fingerprint override.
|
||||
const dockerFingerprint = "9DC858229FC7DD38854AE2D88D81803C0EBFCD88"
|
||||
const dockerKeySHA256 = "1500c1f56fa9e26b9b8f42452a553675796ade0807cdce11975eb98170b3a570"
|
||||
|
||||
func readStaging(directory string) (map[string][]byte, error) {
|
||||
fail := errors.New("invalid repository staging files")
|
||||
if !filepath.IsAbs(directory) {
|
||||
return nil, fail
|
||||
}
|
||||
directory = filepath.Clean(directory)
|
||||
info, err := os.Lstat(directory)
|
||||
if err != nil || !info.IsDir() || info.Mode()&os.ModeSymlink != 0 {
|
||||
return nil, fail
|
||||
}
|
||||
root, err := os.OpenRoot(directory)
|
||||
if err != nil {
|
||||
return nil, fail
|
||||
}
|
||||
defer root.Close()
|
||||
opened, err := root.Stat(".")
|
||||
if err != nil || !os.SameFile(info, opened) {
|
||||
return nil, fail
|
||||
}
|
||||
result := make(map[string][]byte)
|
||||
for name, limit := range map[string]int64{"docker.asc": 1 << 20, "Release": 1 << 20, "Release.gpg": 65536, "Packages": 16 << 20} {
|
||||
data, err := readFile(root, name, limit)
|
||||
if err != nil {
|
||||
return nil, fail
|
||||
}
|
||||
result[name] = data
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func readFile(root *os.Root, name string, limit int64) ([]byte, error) {
|
||||
fail := errors.New("invalid metadata file")
|
||||
before, err := root.Lstat(name)
|
||||
if err != nil || !before.Mode().IsRegular() || before.Size() <= 0 || before.Size() > limit {
|
||||
return nil, fail
|
||||
}
|
||||
f, err := root.Open(name)
|
||||
if err != nil {
|
||||
return nil, fail
|
||||
}
|
||||
defer f.Close()
|
||||
opened, err := f.Stat()
|
||||
if err != nil || !opened.Mode().IsRegular() || !os.SameFile(before, opened) || before.Size() != opened.Size() {
|
||||
return nil, fail
|
||||
}
|
||||
data, err := io.ReadAll(io.LimitReader(f, limit+1))
|
||||
if err != nil || int64(len(data)) != opened.Size() || int64(len(data)) > limit {
|
||||
return nil, fail
|
||||
}
|
||||
return data, nil
|
||||
}
|
||||
|
||||
func authenticate(files map[string][]byte, now time.Time) error {
|
||||
sum := sha256.Sum256(files["docker.asc"])
|
||||
if hex.EncodeToString(sum[:]) != dockerKeySHA256 {
|
||||
return errors.New("repository trust anchor mismatch")
|
||||
}
|
||||
if runtime.GOOS != "linux" {
|
||||
return errors.New("repository authentication requires Linux GnuPG")
|
||||
}
|
||||
// All untrusted bytes are copied to a private snapshot. No caller path reaches
|
||||
// a subprocess, and neither system nor personal keyrings are consulted.
|
||||
dir, err := os.MkdirTemp("", "deployctl-signature-")
|
||||
if err != nil {
|
||||
return errors.New("cannot create signature workspace")
|
||||
}
|
||||
defer os.RemoveAll(dir) // Only our own freshly allocated directory.
|
||||
for _, name := range []string{"docker.asc", "Release", "Release.gpg"} {
|
||||
if err := os.WriteFile(filepath.Join(dir, name), files[name], 0600); err != nil {
|
||||
return errors.New("cannot snapshot repository metadata")
|
||||
}
|
||||
}
|
||||
if _, err := runGPG(dir, "/usr/bin/gpg", "--batch", "--no-options", "--homedir", dir, "--dearmor", "--output", filepath.Join(dir, "docker.gpg"), filepath.Join(dir, "docker.asc")); err != nil {
|
||||
return err
|
||||
}
|
||||
status, err := runGPG(dir, "/usr/bin/gpgv", "--homedir", dir, "--keyring", filepath.Join(dir, "docker.gpg"), "--status-fd", "1", filepath.Join(dir, "Release.gpg"), filepath.Join(dir, "Release"))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return checkStatus(status, now)
|
||||
}
|
||||
|
||||
type cappedOutput struct{ buffer bytes.Buffer }
|
||||
|
||||
func (b *cappedOutput) Len() int { return b.buffer.Len() }
|
||||
func (b *cappedOutput) Bytes() []byte { return b.buffer.Bytes() }
|
||||
|
||||
func (b *cappedOutput) Write(p []byte) (int, error) {
|
||||
if len(p) > 65536-b.Len() {
|
||||
return 0, errors.New("signature output exceeds limit")
|
||||
}
|
||||
return b.buffer.Write(p)
|
||||
}
|
||||
|
||||
func runGPG(dir, program string, args ...string) ([]byte, error) {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
|
||||
defer cancel()
|
||||
cmd := exec.CommandContext(ctx, program, args...)
|
||||
cmd.Dir = dir
|
||||
cmd.Env = []string{"LC_ALL=C", "LANG=C", "HOME=" + dir, "GNUPGHOME=" + dir, "PATH=/usr/bin:/bin"}
|
||||
var output, diagnostics cappedOutput
|
||||
cmd.Stdout = &output
|
||||
cmd.Stderr = &diagnostics
|
||||
cmd.WaitDelay = time.Second
|
||||
if err := cmd.Run(); err != nil {
|
||||
return nil, errors.New("repository signature tool failed")
|
||||
}
|
||||
return output.Bytes(), nil
|
||||
}
|
||||
|
||||
func checkStatus(status []byte, now time.Time) error {
|
||||
fail := errors.New("repository signature rejected")
|
||||
valid, good := 0, 0
|
||||
for _, line := range strings.Split(string(status), "\n") {
|
||||
if line == "" {
|
||||
continue
|
||||
}
|
||||
f := strings.Fields(line)
|
||||
if len(f) < 2 || f[0] != "[GNUPG:]" {
|
||||
return fail
|
||||
}
|
||||
switch f[1] {
|
||||
case "NEWSIG", "KEY_CONSIDERED", "SIG_ID":
|
||||
case "GOODSIG":
|
||||
good++
|
||||
case "VALIDSIG":
|
||||
// fingerprint, date, timestamp, expiry, version, reserved, public-key
|
||||
// algorithm, digest algorithm, signature class, primary fingerprint.
|
||||
if len(f) != 12 || f[11] != dockerFingerprint || (f[9] != "8" && f[9] != "9" && f[9] != "10") || f[10] != "00" {
|
||||
return fail
|
||||
}
|
||||
issued, e1 := strconv.ParseInt(f[4], 10, 64)
|
||||
expiry, e2 := strconv.ParseInt(f[5], 10, 64)
|
||||
if e1 != nil || e2 != nil || issued <= 0 || expiry < 0 || issued > now.Add(10*time.Minute).Unix() || (expiry != 0 && expiry <= now.Unix()) {
|
||||
return fail
|
||||
}
|
||||
valid++
|
||||
default:
|
||||
return fail // Includes expired/revoked/bad/unknown signatures.
|
||||
}
|
||||
}
|
||||
if valid != 1 || good != 1 {
|
||||
return fail
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,105 @@
|
||||
package aptrepo
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"server-deploy/internal/installplan"
|
||||
"syscall"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestStagingRejectsLinksAndFIFO(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
for _, name := range []string{"docker.asc", "Release", "Release.gpg", "Packages"} {
|
||||
if err := os.WriteFile(filepath.Join(dir, name), []byte("metadata"), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
link := filepath.Join(t.TempDir(), "link")
|
||||
if err := os.Symlink(dir, link); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := readStaging(link + "/"); err == nil {
|
||||
t.Fatal("root symlink accepted")
|
||||
}
|
||||
if err := os.Remove(filepath.Join(dir, "Release")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.Symlink(filepath.Join(dir, "Packages"), filepath.Join(dir, "Release")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := readStaging(dir); err == nil {
|
||||
t.Fatal("file symlink accepted")
|
||||
}
|
||||
if err := os.Remove(filepath.Join(dir, "Release")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := syscall.Mkfifo(filepath.Join(dir, "Release"), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := readStaging(dir); err == nil {
|
||||
t.Fatal("FIFO accepted")
|
||||
}
|
||||
}
|
||||
|
||||
func TestGPGFailureRedactsDiagnostics(t *testing.T) {
|
||||
if _, err := runGPG(t.TempDir(), "/nonexistent/signature-tool-secret"); err == nil || err.Error() != "repository signature tool failed" {
|
||||
t.Fatal("unredacted or absent error", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestOutputBound(t *testing.T) {
|
||||
var b cappedOutput
|
||||
if _, err := b.Write(make([]byte, 65536)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := b.Write([]byte{1}); err == nil {
|
||||
t.Fatal("output limit missing")
|
||||
}
|
||||
if b.Len() != 65536 {
|
||||
t.Fatal("buffer grew beyond limit")
|
||||
}
|
||||
}
|
||||
|
||||
func TestStagedSignatureIntegration(t *testing.T) {
|
||||
// Explicit opt-in public fixture, downloaded by the metadata-only probe.
|
||||
dir := os.Getenv("DEPLOYCTL_REPOSITORY_FIXTURE")
|
||||
if dir == "" {
|
||||
t.Skip("public signed fixture not supplied; run repository probe for real signature evidence")
|
||||
}
|
||||
files, err := readStaging(dir)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := authenticate(files, time.Now()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// Result comes from the successful real CLI invocation. Recheck the exact
|
||||
// fixture versions before making a semantically valid signature mutation.
|
||||
var response struct {
|
||||
Lock installplan.Lock `json:"lock"`
|
||||
}
|
||||
raw, err := os.ReadFile(filepath.Join(dir, "result.json"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err = json.Unmarshal(raw, &response); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
versions := make(map[string]string)
|
||||
for _, p := range response.Lock.Packages {
|
||||
versions[p.Name] = p.Version
|
||||
}
|
||||
// Unknown Release extension remains valid metadata; only its signature
|
||||
// should reject it. This catches a bypass hidden by syntax failures.
|
||||
files["Release"] = append(bytes.TrimRight(files["Release"], "\n"), []byte("\nX-Verification-Probe: changed\n")...)
|
||||
if _, err := Resolve(files["Release"], files["Packages"], response.Lock.Suite, response.Lock.Architecture, versions, time.Now()); err != nil {
|
||||
t.Fatal("mutation masked by metadata rejection", err)
|
||||
}
|
||||
if err := authenticate(files, time.Now()); err == nil {
|
||||
t.Fatal("tampered signature accepted")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,77 @@
|
||||
package aptrepo
|
||||
|
||||
import (
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestOutputCopyBound(t *testing.T) {
|
||||
var b cappedOutput
|
||||
_, err := io.Copy(&b, io.LimitReader(strings.NewReader(strings.Repeat("x", 65537)), 65537))
|
||||
if err == nil || b.Len() > 65536 {
|
||||
t.Fatal("io.Copy bypassed output limit", b.Len(), err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestWellFormedPackageTamperNeedsRebinding(t *testing.T) {
|
||||
index := fixtureIndex()
|
||||
release := fixtureRelease(index)
|
||||
changed := []byte(strings.Replace(string(index), "SHA256: a", "SHA256: b", 1))
|
||||
if _, err := Resolve(release, changed, "resolute", "amd64", fixturePins(), fixtureNow); err == nil {
|
||||
t.Fatal("unbound index accepted")
|
||||
}
|
||||
if _, err := Resolve(fixtureRelease(changed), changed, "resolute", "amd64", fixturePins(), fixtureNow); err != nil {
|
||||
t.Fatal("tamper test masked by parser rejection", err)
|
||||
}
|
||||
}
|
||||
|
||||
const goodStatus = "[GNUPG:] NEWSIG\n[GNUPG:] GOODSIG 7EA0A9C3F273FCD8 Docker\n[GNUPG:] VALIDSIG D3306A018370199E527AE7997EA0A9C3F273FCD8 2026-09-24 1790231932 0 4 0 1 10 00 9DC858229FC7DD38854AE2D88D81803C0EBFCD88\n"
|
||||
|
||||
func TestSignatureStatus(t *testing.T) {
|
||||
now := time.Date(2026, 9, 25, 0, 0, 0, 0, time.UTC)
|
||||
if err := checkStatus([]byte(goodStatus), now); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, s := range []string{"", "[GNUPG:] GOODSIG key name\n", goodStatus + goodStatus,
|
||||
strings.ReplaceAll(goodStatus, dockerFingerprint, strings.Repeat("A", 40)),
|
||||
strings.Replace(goodStatus, " 10 00 ", " 2 00 ", 1),
|
||||
strings.Replace(goodStatus, "1790231932 0", "1990231932 0", 1),
|
||||
strings.Replace(goodStatus, "1790231932 0", "1790231932 1790231933", 1),
|
||||
goodStatus + "[GNUPG:] EXPKEYSIG bad\n", goodStatus + "[GNUPG:] BADSIG bad\n",
|
||||
} {
|
||||
if checkStatus([]byte(s), now) == nil {
|
||||
t.Errorf("accepted invalid signature status: %q", s)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestReadStaging(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
for _, name := range []string{"docker.asc", "Release", "Release.gpg", "Packages"} {
|
||||
if err := os.WriteFile(filepath.Join(dir, name), []byte("bytes"), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if _, err := readStaging(dir); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := readStaging("relative"); err == nil {
|
||||
t.Fatal("relative directory accepted")
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(dir, "Release.gpg"), make([]byte, 65537), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := readStaging(dir); err == nil {
|
||||
t.Fatal("oversize signature accepted")
|
||||
}
|
||||
}
|
||||
|
||||
func TestWrongKeyFailsBeforeExternalProcess(t *testing.T) {
|
||||
if err := authenticate(map[string][]byte{"docker.asc": []byte("untrusted")}, time.Now()); err == nil {
|
||||
t.Fatal("untrusted key accepted")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,34 @@
|
||||
// Package aptrepo authenticates staged Docker APT metadata, never installs it.
|
||||
package aptrepo
|
||||
|
||||
import (
|
||||
"server-deploy/internal/installplan"
|
||||
"time"
|
||||
)
|
||||
|
||||
type Result struct {
|
||||
ProtocolVersion int `json:"protocolVersion"`
|
||||
RepositoryAuthenticated bool `json:"repositoryAuthenticated"`
|
||||
PackageBytesVerified bool `json:"packageBytesVerified"`
|
||||
Executable bool `json:"executable"`
|
||||
VerifiedAt time.Time `json:"verifiedAt"`
|
||||
PrimaryFingerprint string `json:"primaryFingerprint"`
|
||||
Lock installplan.Lock `json:"lock"`
|
||||
}
|
||||
|
||||
// Verify requires a trusted staging directory and trusted ancestors, with no
|
||||
// concurrent writers. Returned JSON is evidence, not an execution capability.
|
||||
func Verify(directory, suite, arch string, versions map[string]string, now time.Time) (Result, error) {
|
||||
files, err := readStaging(directory)
|
||||
if err != nil {
|
||||
return Result{}, err
|
||||
}
|
||||
if err := authenticate(files, now); err != nil {
|
||||
return Result{}, err
|
||||
}
|
||||
lock, err := Resolve(files["Release"], files["Packages"], suite, arch, versions, now)
|
||||
if err != nil {
|
||||
return Result{}, err
|
||||
}
|
||||
return Result{ProtocolVersion: 1, RepositoryAuthenticated: true, VerifiedAt: now.UTC().Truncate(time.Second), PrimaryFingerprint: dockerFingerprint, Lock: lock}, nil
|
||||
}
|
||||
@@ -0,0 +1,43 @@
|
||||
package cli
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"server-deploy/internal/installplan"
|
||||
"server-deploy/internal/preflight"
|
||||
)
|
||||
|
||||
func TestEnvironmentPlanCLI(t *testing.T) {
|
||||
r := preflight.Collect()
|
||||
suite, arch := "resolute", "amd64"
|
||||
if r.Runtime.OS == "linux" {
|
||||
suite = r.Distribution.Codename
|
||||
arch = r.Runtime.Architecture
|
||||
}
|
||||
if suite != "resolute" && suite != "noble" && suite != "jammy" {
|
||||
t.Skip("local Linux distribution outside initial lock policy")
|
||||
}
|
||||
l := installplan.Lock{ProtocolVersion: 1, Repository: "https://download.docker.com/linux/ubuntu", Suite: suite, Architecture: arch, ReleaseDigest: "sha256:" + strings.Repeat("a", 64), Packages: []installplan.Package{}}
|
||||
for _, name := range []string{"docker-ce", "docker-ce-cli", "containerd.io", "docker-buildx-plugin", "docker-compose-plugin"} {
|
||||
l.Packages = append(l.Packages, installplan.Package{Name: name, Version: "1.2.3-1", Filename: "dists/" + suite + "/pool/stable/" + arch + "/" + name + "_1.2.3-1_" + arch + ".deb", Digest: "sha256:" + strings.Repeat("b", 64), Size: 123})
|
||||
}
|
||||
raw, _ := json.Marshal(struct {
|
||||
Lock installplan.Lock `json:"lock"`
|
||||
}{l})
|
||||
code, out, diagnostics := run([]string{"plan-environment"}, string(raw))
|
||||
var result struct {
|
||||
Mode string `json:"mode"`
|
||||
Draft installplan.Draft `json:"draft"`
|
||||
}
|
||||
if code != 0 || json.Unmarshal([]byte(out), &result) != nil || result.Mode != "local-environment-draft" || result.Draft.Executable || result.Draft.RepositoryAuthenticated || len(result.Draft.RequestedPackages) != 5 || len(result.Draft.Blockers) == 0 {
|
||||
t.Fatalf("bad draft %s %s", out, diagnostics)
|
||||
}
|
||||
for _, bad := range []string{`{}`, strings.Replace(string(raw), `"1.2.3-1"`, `"secret;reboot"`, 1)} {
|
||||
code, out, diagnostics := run([]string{"plan-environment"}, bad)
|
||||
if code == 0 || out != "" || strings.Contains(diagnostics, "secret") {
|
||||
t.Fatal("invalid request accepted or leaked")
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
package cli
|
||||
|
||||
import (
|
||||
"io"
|
||||
"server-deploy/internal/wire"
|
||||
)
|
||||
|
||||
func decodeStrict(in io.Reader, target any) error { return wire.Decode(in, target, 1<<20) }
|
||||
@@ -0,0 +1,117 @@
|
||||
package cli
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func packageFixture(t *testing.T) (string, string) {
|
||||
return packageFixturePayload(t, []byte("services: {}\n"))
|
||||
}
|
||||
|
||||
func packageFixturePayload(t *testing.T, payload []byte) (string, string) {
|
||||
t.Helper()
|
||||
dir := t.TempDir()
|
||||
digest := func(data []byte) string { sum := sha256.Sum256(data); return "sha256:" + hex.EncodeToString(sum[:]) }
|
||||
manifest := map[string]any{
|
||||
"protocolVersion": 1, "appId": "example", "version": "1.0.0", "runtime": "compose", "entrypoint": "compose.yaml",
|
||||
"platforms": []string{"linux/amd64"},
|
||||
"components": []map[string]string{{"name": "server", "image": "example/server@sha256:" + strings.Repeat("a", 64)}},
|
||||
"files": []map[string]string{{"path": "compose.yaml", "digest": digest(payload)}},
|
||||
}
|
||||
raw, err := json.Marshal(manifest)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(dir, "manifest.json"), raw, 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(dir, "compose.yaml"), payload, 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return dir, digest(raw)
|
||||
}
|
||||
|
||||
func TestCheckPackagePolicyCLI(t *testing.T) {
|
||||
payload := `{"services":{"server":{"image":"example/server@sha256:` + strings.Repeat("a", 64) + `","user":"1000:1000","read_only":true,"cap_drop":["ALL"],"security_opt":["no-new-privileges:true"],"restart":"no","networks":["backend"],"volumes":[]}},"networks":{"backend":{"internal":true}},"volumes":{}}`
|
||||
for _, tc := range []struct {
|
||||
payload string
|
||||
accepted bool
|
||||
}{
|
||||
{payload, true},
|
||||
{strings.Replace(payload, `"read_only":true`, `"read_only":false`, 1), false},
|
||||
{`services: {}`, false},
|
||||
} {
|
||||
dir, digest := packageFixturePayload(t, []byte(tc.payload))
|
||||
input, _ := json.Marshal(map[string]string{"directory": dir, "expectedDigest": digest})
|
||||
code, out, diagnostics := run([]string{"check-package"}, string(input))
|
||||
if tc.accepted {
|
||||
var result struct {
|
||||
PolicyPassed bool `json:"policyPassed"`
|
||||
Executable bool `json:"executable"`
|
||||
Profile string `json:"profile"`
|
||||
Digest string `json:"digest"`
|
||||
}
|
||||
if code != 0 || json.Unmarshal([]byte(out), &result) != nil || !result.PolicyPassed || result.Executable || result.Profile == "" || result.Digest != digest {
|
||||
t.Fatalf("bad check result: %s %s", out, diagnostics)
|
||||
}
|
||||
if strings.Contains(out, "1000:1000") {
|
||||
t.Fatal("payload leaked")
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(dir, "compose.yaml"), []byte(tc.payload+" "), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if code, _, _ := run([]string{"check-package"}, string(input)); code == 0 {
|
||||
t.Fatal("policy bypassed integrity check")
|
||||
}
|
||||
} else if code == 0 || out != "" {
|
||||
t.Fatal("unsafe package accepted")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestVerifyPackageCLI(t *testing.T) {
|
||||
dir, digest := packageFixture(t)
|
||||
input, _ := json.Marshal(map[string]string{"directory": dir, "expectedDigest": digest})
|
||||
code, out, diagnostics := run([]string{"verify-package"}, string(input))
|
||||
if code != 0 || diagnostics != "" {
|
||||
t.Fatalf("package verification failed: %s", diagnostics)
|
||||
}
|
||||
var response struct {
|
||||
Verified bool `json:"verified"`
|
||||
Executable bool `json:"executable"`
|
||||
PublisherAuthenticated bool `json:"publisherAuthenticated"`
|
||||
FileCount int `json:"fileCount"`
|
||||
Digest string `json:"digest"`
|
||||
}
|
||||
if err := json.Unmarshal([]byte(out), &response); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !response.Verified || response.Executable || response.PublisherAuthenticated || response.FileCount != 1 || response.Digest != digest {
|
||||
t.Fatalf("misleading verification: %s", out)
|
||||
}
|
||||
if strings.Contains(out, "services: {}") {
|
||||
t.Fatal("file contents leaked")
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(dir, "compose.yaml"), []byte("secret tampering"), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
code, out, diagnostics = run([]string{"verify-package"}, string(input))
|
||||
if code == 0 || out != "" || strings.Contains(diagnostics, "secret tampering") {
|
||||
t.Fatal("tampering accepted or leaked")
|
||||
}
|
||||
}
|
||||
|
||||
func TestVerifyPackageInvalidRequests(t *testing.T) {
|
||||
for _, input := range []string{`{}`, `{"directory":".","expectedDigest":"latest"}`, `{"directory":"secret","expectedDigest":"bad","password":"do-not-echo"}`} {
|
||||
code, out, diagnostics := run([]string{"verify-package"}, input)
|
||||
if code == 0 || out != "" || strings.Contains(diagnostics, "do-not-echo") {
|
||||
t.Fatal("invalid package request accepted or leaked")
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,21 @@
|
||||
package cli
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestPreflightDoesNotAuthorizeInstallation(t *testing.T) {
|
||||
code, out, diagnostics := run([]string{"preflight"}, "")
|
||||
var response struct {
|
||||
ProtocolVersion int `json:"protocolVersion"`
|
||||
Mode string `json:"mode"`
|
||||
Proposal struct {
|
||||
Executable bool `json:"executable"`
|
||||
Blockers []string `json:"blockers"`
|
||||
} `json:"proposal"`
|
||||
}
|
||||
if code != 0 || json.Unmarshal([]byte(out), &response) != nil || response.ProtocolVersion != 1 || response.Mode != "local-environment-proposal" || response.Proposal.Executable || len(response.Proposal.Blockers) == 0 {
|
||||
t.Fatalf("unsafe report: %s %s", out, diagnostics)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,37 @@
|
||||
package cli
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestRepositoryRejectsUntrustedRequests(t *testing.T) {
|
||||
for _, input := range []string{`{}`, `{"directory":"secret-relative","suite":"resolute","architecture":"amd64","versions":{}}`,
|
||||
`{"directory":"secret-relative","suite":"resolute","architecture":"amd64","versions":{},"repositoryAuthenticated":true}`} {
|
||||
var out, diagnostics bytes.Buffer
|
||||
if Run([]string{"verify-repository"}, strings.NewReader(input), &out, &diagnostics, time.Now) == 0 {
|
||||
t.Fatal("accepted invalid request")
|
||||
}
|
||||
if out.Len() != 0 || strings.Contains(diagnostics.String(), "secret-relative") {
|
||||
t.Fatal("invalid response leaked input")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestArtifactCommandRejectsCallerTrust(t *testing.T) {
|
||||
for _, field := range []string{`"packageBytesVerified":true`, `"repositoryAuthenticated":true`, `"lock":{}`} {
|
||||
var out, diagnostics bytes.Buffer
|
||||
input := `{"directory":"secret-relative","artifactDirectory":"secret-artifacts","suite":"resolute","architecture":"amd64","versions":{},` + field + `}`
|
||||
if Run([]string{"verify-artifacts"}, strings.NewReader(input), &out, &diagnostics, time.Now) == 0 {
|
||||
t.Fatal("caller trust accepted")
|
||||
}
|
||||
if out.Len() != 0 || strings.Contains(diagnostics.String(), "secret") {
|
||||
t.Fatal("input leaked")
|
||||
}
|
||||
if diagnostics.String() != "invalid artifact verification request\n" {
|
||||
t.Fatal("request was not rejected at the protocol boundary", diagnostics.String())
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,167 @@
|
||||
// Package cli exposes read-only protocol endpoints, not a shell wrapper.
|
||||
package cli
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"time"
|
||||
|
||||
"server-deploy/internal/appbundle"
|
||||
"server-deploy/internal/aptrepo"
|
||||
"server-deploy/internal/composepolicy"
|
||||
"server-deploy/internal/inspect"
|
||||
"server-deploy/internal/installplan"
|
||||
"server-deploy/internal/planner"
|
||||
"server-deploy/internal/preflight"
|
||||
)
|
||||
|
||||
func Run(args []string, in io.Reader, out, diagnostics io.Writer, now func() time.Time) int {
|
||||
fail := func(message string) int { fmt.Fprintln(diagnostics, message); return 1 }
|
||||
if len(args) != 1 {
|
||||
return fail("usage: deployctl version | inspect | preflight | plan-environment | verify-repository | verify-artifacts | plan | verify-plan | verify-package | check-package")
|
||||
}
|
||||
var response any
|
||||
switch args[0] {
|
||||
case "verify-artifacts":
|
||||
var request struct {
|
||||
Directory string `json:"directory"`
|
||||
ArtifactDirectory string `json:"artifactDirectory"`
|
||||
Suite string `json:"suite"`
|
||||
Architecture string `json:"architecture"`
|
||||
Versions map[string]string `json:"versions"`
|
||||
}
|
||||
if decodeStrict(in, &request) != nil {
|
||||
return fail("invalid artifact verification request")
|
||||
}
|
||||
verified, err := aptrepo.VerifyArtifacts(request.Directory, request.ArtifactDirectory, request.Suite, request.Architecture, request.Versions, now())
|
||||
if err != nil {
|
||||
return fail("artifact verification failed")
|
||||
}
|
||||
response = verified
|
||||
case "verify-repository":
|
||||
var request struct {
|
||||
Directory string `json:"directory"`
|
||||
Suite string `json:"suite"`
|
||||
Architecture string `json:"architecture"`
|
||||
Versions map[string]string `json:"versions"`
|
||||
}
|
||||
if decodeStrict(in, &request) != nil {
|
||||
return fail("invalid repository verification request")
|
||||
}
|
||||
verified, err := aptrepo.Verify(request.Directory, request.Suite, request.Architecture, request.Versions, now())
|
||||
if err != nil {
|
||||
return fail("repository verification failed")
|
||||
}
|
||||
response = verified
|
||||
case "plan-environment":
|
||||
var request struct {
|
||||
Lock installplan.Lock `json:"lock"`
|
||||
}
|
||||
if decodeStrict(in, &request) != nil {
|
||||
return fail("invalid environment lock request")
|
||||
}
|
||||
report := preflight.Collect()
|
||||
draft, err := installplan.Build(report, request.Lock)
|
||||
if err != nil {
|
||||
return fail("environment lock rejected")
|
||||
}
|
||||
response = struct {
|
||||
ProtocolVersion int `json:"protocolVersion"`
|
||||
Mode string `json:"mode"`
|
||||
ObservedAt time.Time `json:"observedAt"`
|
||||
Report preflight.Report `json:"report"`
|
||||
Draft installplan.Draft `json:"draft"`
|
||||
}{1, "local-environment-draft", now().UTC().Truncate(time.Second), report, draft}
|
||||
case "preflight":
|
||||
report := preflight.Collect()
|
||||
response = struct {
|
||||
ProtocolVersion int `json:"protocolVersion"`
|
||||
Mode string `json:"mode"`
|
||||
ObservedAt time.Time `json:"observedAt"`
|
||||
Report preflight.Report `json:"report"`
|
||||
Proposal preflight.Proposal `json:"proposal"`
|
||||
}{1, "local-environment-proposal", now().UTC().Truncate(time.Second), report, preflight.Plan(report)}
|
||||
case "verify-package", "check-package":
|
||||
var request struct {
|
||||
Directory string `json:"directory"`
|
||||
ExpectedDigest string `json:"expectedDigest"`
|
||||
}
|
||||
if err := decodeStrict(in, &request); err != nil {
|
||||
return fail("invalid package verification request")
|
||||
}
|
||||
verified, err := appbundle.Verify(request.Directory, request.ExpectedDigest)
|
||||
if err != nil {
|
||||
return fail("package verification failed: invalid manifest, inventory or digest")
|
||||
}
|
||||
if args[0] == "check-package" {
|
||||
if composepolicy.Check(verified.Manifest, verified.Files[verified.Manifest.Entrypoint]) != nil {
|
||||
return fail("package rejected by restricted Compose policy")
|
||||
}
|
||||
response = struct {
|
||||
ProtocolVersion int `json:"protocolVersion"`
|
||||
PolicyPassed bool `json:"policyPassed"`
|
||||
Profile string `json:"profile"`
|
||||
Digest string `json:"digest"`
|
||||
Executable bool `json:"executable"`
|
||||
PublisherAuthenticated bool `json:"publisherAuthenticated"`
|
||||
}{1, true, composepolicy.Profile, verified.Digest, false, false}
|
||||
break
|
||||
}
|
||||
response = struct {
|
||||
ProtocolVersion int `json:"protocolVersion"`
|
||||
Verified bool `json:"verified"`
|
||||
Executable bool `json:"executable"`
|
||||
PublisherAuthenticated bool `json:"publisherAuthenticated"`
|
||||
Digest string `json:"digest"`
|
||||
FileCount int `json:"fileCount"`
|
||||
Manifest appbundle.Manifest `json:"manifest"`
|
||||
}{1, true, false, false, verified.Digest, len(verified.Files), verified.Manifest}
|
||||
case "inspect":
|
||||
response = inspect.Collect()
|
||||
case "version":
|
||||
response = struct {
|
||||
Version string `json:"version"`
|
||||
ProtocolVersion int `json:"protocolVersion"`
|
||||
WritesEnabled bool `json:"writesEnabled"`
|
||||
}{"0.1.0-dev", planner.ProtocolVersion, false}
|
||||
case "plan":
|
||||
var intent planner.Intent
|
||||
if err := decodeStrict(in, &intent); err != nil {
|
||||
return fail("invalid request: expected strict protocol JSON (maximum 1 MiB)")
|
||||
}
|
||||
plan, err := planner.Build(intent, now())
|
||||
if err != nil {
|
||||
return fail("invalid deployment intent")
|
||||
}
|
||||
response = struct {
|
||||
ProtocolVersion int `json:"protocolVersion"`
|
||||
Mode string `json:"mode"`
|
||||
Executable bool `json:"executable"`
|
||||
Plan planner.Plan `json:"plan"`
|
||||
}{planner.ProtocolVersion, "offline-preview", false, plan}
|
||||
case "verify-plan":
|
||||
var request struct {
|
||||
Plan planner.Plan `json:"plan"`
|
||||
Current planner.Intent `json:"current"`
|
||||
}
|
||||
if err := decodeStrict(in, &request); err != nil {
|
||||
return fail("invalid verification request")
|
||||
}
|
||||
if err := request.Plan.Verify(request.Current, now()); err != nil {
|
||||
return fail("plan rejected: expired, changed or invalid")
|
||||
}
|
||||
response = struct {
|
||||
ProtocolVersion int `json:"protocolVersion"`
|
||||
Mode string `json:"mode"`
|
||||
Valid bool `json:"valid"`
|
||||
Executable bool `json:"executable"`
|
||||
}{planner.ProtocolVersion, "offline-preview", true, false}
|
||||
default:
|
||||
return fail("unsupported command; deployment writes are not enabled")
|
||||
}
|
||||
if err := json.NewEncoder(out).Encode(response); err != nil {
|
||||
return fail("cannot write response")
|
||||
}
|
||||
return 0
|
||||
}
|
||||
@@ -0,0 +1,187 @@
|
||||
package cli
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"server-deploy/internal/planner"
|
||||
)
|
||||
|
||||
func fixture() string {
|
||||
return `{"protocolVersion":1,"hostId":"host-one","instanceId":"git-one","appId":"gitea","packageDigest":"sha256:` + strings.Repeat("a", 64) + `","imageDigest":"sha256:` + strings.Repeat("b", 64) + `","domain":"git.example.com","observedStateDigest":"sha256:` + strings.Repeat("c", 64) + `"}`
|
||||
}
|
||||
|
||||
func run(args []string, input string) (int, string, string) {
|
||||
var out, diagnostic bytes.Buffer
|
||||
code := Run(args, strings.NewReader(input), &out, &diagnostic, func() time.Time { return time.Date(2026, 9, 25, 12, 0, 0, 0, time.UTC) })
|
||||
return code, out.String(), diagnostic.String()
|
||||
}
|
||||
|
||||
func TestPlanAndVerify(t *testing.T) {
|
||||
code, output, diagnostic := run([]string{"plan"}, fixture())
|
||||
if code != 0 || diagnostic != "" {
|
||||
t.Fatalf("plan failed: %d %s", code, diagnostic)
|
||||
}
|
||||
var response struct {
|
||||
ProtocolVersion int `json:"protocolVersion"`
|
||||
Mode string `json:"mode"`
|
||||
Executable bool `json:"executable"`
|
||||
Plan planner.Plan `json:"plan"`
|
||||
}
|
||||
if err := json.Unmarshal([]byte(output), &response); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if response.Mode != "offline-preview" || response.Executable || response.ProtocolVersion != 1 || response.Plan.ProjectName != "sd-git-one" {
|
||||
t.Fatalf("misleading plan: %s", output)
|
||||
}
|
||||
request, _ := json.Marshal(struct {
|
||||
Plan planner.Plan `json:"plan"`
|
||||
Current planner.Intent `json:"current"`
|
||||
}{response.Plan, response.Plan.Intent})
|
||||
code, output, diagnostic = run([]string{"verify-plan"}, string(request))
|
||||
if code != 0 || diagnostic != "" || !strings.Contains(output, `"valid":true`) {
|
||||
t.Fatalf("verify failed: %d %s %s", code, output, diagnostic)
|
||||
}
|
||||
response.Plan.Hash = "tampered"
|
||||
request, _ = json.Marshal(struct {
|
||||
Plan planner.Plan `json:"plan"`
|
||||
Current planner.Intent `json:"current"`
|
||||
}{response.Plan, response.Plan.Intent})
|
||||
code, _, _ = run([]string{"verify-plan"}, string(request))
|
||||
if code == 0 {
|
||||
t.Fatal("accepted tampered plan")
|
||||
}
|
||||
}
|
||||
|
||||
func TestRejectsAmbiguousAndOversizedInput(t *testing.T) {
|
||||
cases := []string{
|
||||
``, `{}`, `null`, `[]`,
|
||||
strings.TrimSuffix(fixture(), "}") + `,"password":"do-not-echo"}`,
|
||||
strings.TrimSuffix(fixture(), "}") + `,"hostId":"other"}`,
|
||||
strings.TrimSuffix(fixture(), "}") + `,"HostId":"other"}`,
|
||||
strings.Replace(fixture(), `"hostId"`, `"HostId"`, 1),
|
||||
strings.Replace(fixture(), `"protocolVersion":1`, `"protocolVersion":null`, 1),
|
||||
fixture() + ` {}`, fixture() + ` trailing`,
|
||||
strings.Repeat(" ", 1024*1024) + fixture(),
|
||||
}
|
||||
for _, input := range cases {
|
||||
code, out, diagnostic := run([]string{"plan"}, input)
|
||||
if code == 0 || out != "" || diagnostic == "" {
|
||||
t.Fatalf("invalid input accepted: code=%d", code)
|
||||
}
|
||||
if strings.Contains(diagnostic, "do-not-echo") {
|
||||
t.Fatal("secret leaked")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRejectsWriteCommandsAndUnexpectedArguments(t *testing.T) {
|
||||
for _, args := range [][]string{nil, {"apply"}, {"upgrade"}, {"restore"}, {"plan", "extra"}, {"version", "extra"}} {
|
||||
code, out, _ := run(args, fixture())
|
||||
if code == 0 || out != "" {
|
||||
t.Fatalf("accepted command %v", args)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestVersion(t *testing.T) {
|
||||
code, out, diagnostic := run([]string{"version"}, "")
|
||||
if code != 0 || diagnostic != "" || !json.Valid([]byte(out)) {
|
||||
t.Fatal("version failed")
|
||||
}
|
||||
}
|
||||
|
||||
func TestInspectIsReadOnlyAndExplicitlyIncomplete(t *testing.T) {
|
||||
code, out, diagnostics := run([]string{"inspect"}, "")
|
||||
if code != 0 || diagnostics != "" {
|
||||
t.Fatalf("inspect failed: %s", diagnostics)
|
||||
}
|
||||
var report struct {
|
||||
ProtocolVersion int `json:"protocolVersion"`
|
||||
OS string `json:"os"`
|
||||
DeploymentReady bool `json:"deploymentReady"`
|
||||
DockerDaemon string `json:"dockerDaemon"`
|
||||
}
|
||||
if err := json.Unmarshal([]byte(out), &report); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if report.ProtocolVersion != 1 || report.OS == "" || report.DeploymentReady || report.DockerDaemon != "not_checked" {
|
||||
t.Fatalf("misleading report: %s", out)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNestedVerificationRejectsDuplicateAndNullFields(t *testing.T) {
|
||||
_, output, _ := run([]string{"plan"}, fixture())
|
||||
var response struct {
|
||||
Plan json.RawMessage `json:"plan"`
|
||||
}
|
||||
if err := json.Unmarshal([]byte(output), &response); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
valid := `{"plan":` + string(response.Plan) + `,"current":` + fixture() + `}`
|
||||
for _, input := range []string{
|
||||
strings.Replace(valid, `"projectName":"sd-git-one"`, `"projectName":"sd-git-one","projectName":"sd-git-one"`, 1),
|
||||
strings.Replace(valid, `"createdAt":"2026-09-25T12:00:00Z"`, `"createdAt":null`, 1),
|
||||
strings.Replace(valid, `"intent":`, `"Intent":`, 1),
|
||||
} {
|
||||
code, out, _ := run([]string{"verify-plan"}, input)
|
||||
if code == 0 || out != "" {
|
||||
t.Fatal("accepted ambiguous nested request")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestVerificationRequiresCanonicalUTCTimestamps(t *testing.T) {
|
||||
_, output, _ := run([]string{"plan"}, fixture())
|
||||
var response struct {
|
||||
Plan json.RawMessage `json:"plan"`
|
||||
}
|
||||
if err := json.Unmarshal([]byte(output), &response); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
valid := `{"plan":` + string(response.Plan) + `,"current":` + fixture() + `}`
|
||||
for _, timestamp := range []string{
|
||||
"2026-09-25T12:00:00,000Z",
|
||||
"2026-09-25T12:00:00.000Z",
|
||||
"2026-09-25T12:00:00+00:00",
|
||||
"2026-09-26T12:00:00+24:00",
|
||||
"2026-09-25T13:00:00+00:60",
|
||||
} {
|
||||
t.Run(timestamp, func(t *testing.T) {
|
||||
input := strings.Replace(valid, "2026-09-25T12:00:00Z", timestamp, 1)
|
||||
code, out, _ := run([]string{"verify-plan"}, input)
|
||||
if code == 0 || out != "" {
|
||||
t.Fatal("accepted noncanonical timestamp")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
type brokenReader struct{}
|
||||
|
||||
func (brokenReader) Read([]byte) (int, error) { return 0, errors.New("secret reader error") }
|
||||
|
||||
type brokenWriter struct{}
|
||||
|
||||
func (brokenWriter) Write([]byte) (int, error) { return 0, errors.New("secret writer error") }
|
||||
|
||||
func TestIOErrorsFailWithoutLeakingDetails(t *testing.T) {
|
||||
var out, diagnostics bytes.Buffer
|
||||
if Run([]string{"plan"}, brokenReader{}, &out, &diagnostics, time.Now) == 0 {
|
||||
t.Fatal("ignored read error")
|
||||
}
|
||||
if out.Len() != 0 || strings.Contains(diagnostics.String(), "secret") {
|
||||
t.Fatal("leaked input error")
|
||||
}
|
||||
diagnostics.Reset()
|
||||
if Run([]string{"version"}, strings.NewReader(""), brokenWriter{}, &diagnostics, time.Now) == 0 {
|
||||
t.Fatal("ignored write error")
|
||||
}
|
||||
if strings.Contains(diagnostics.String(), "secret") {
|
||||
t.Fatal("leaked output error")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,78 @@
|
||||
# Restricted Compose policy
|
||||
|
||||
`Check(manifest, entrypointBytes)` is a pure, offline policy check. The CLI
|
||||
`check-package` first calls `appbundle.Verify`, then passes the authenticated
|
||||
entrypoint snapshot here. It never reopens the entrypoint, runs Compose, reads
|
||||
`.env`, resolves templates, or contacts a daemon. Direct internal callers must
|
||||
perform the same integrity/trust check first.
|
||||
|
||||
Profile: `isolated-compose-v1`. This is an initial restricted backend profile,
|
||||
not the finished application's Compose contract. It deliberately rejects public
|
||||
routing, secrets/config injection, environment settings, healthchecks, dependency
|
||||
ordering and application-specific privilege exceptions until adapters exist.
|
||||
Existing YAML packages can still pass `verify-package`; that does not mean they
|
||||
pass `check-package`. Only JSON entrypoint content is accepted by this policy.
|
||||
|
||||
## Exact shape
|
||||
|
||||
All fields below are mandatory, all unlisted fields are rejected:
|
||||
|
||||
```json
|
||||
{
|
||||
"services": {
|
||||
"api": {
|
||||
"image": "example/api@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
||||
"user": "1000:1000",
|
||||
"read_only": true,
|
||||
"cap_drop": ["ALL"],
|
||||
"security_opt": ["no-new-privileges:true"],
|
||||
"restart": "no",
|
||||
"networks": ["backend"],
|
||||
"volumes": [
|
||||
{"type": "volume", "source": "data", "target": "/data", "read_only": false}
|
||||
]
|
||||
}
|
||||
},
|
||||
"networks": {"backend": {"internal": true}},
|
||||
"volumes": {"data": {}}
|
||||
}
|
||||
```
|
||||
|
||||
The example digest is synthetic, not an installable release.
|
||||
|
||||
- 4 MiB input limit. Shared strict decoder rejects duplicate, case-alias,
|
||||
unknown, missing and null fields, including typed map values.
|
||||
- 1–32 services, exactly matching manifest component names and pinned images.
|
||||
- UID and GID must be canonical positive uint32 decimals, excluding 4294967295.
|
||||
No root, account-name lookup, interpolation or inherited user defaults.
|
||||
- Restart must be `no` or `unless-stopped`; root filesystem must be read-only,
|
||||
all capabilities dropped and privilege escalation disabled.
|
||||
- Exactly one network: `backend`, with `internal: true`. No default network,
|
||||
external network, host networking, published port or arbitrary router label.
|
||||
- At most 128 plain named volumes; every declaration must be mounted exactly
|
||||
once. Empty volume maps/lists are permitted for stateless services. No external
|
||||
names, drivers, driver options, bind mounts or cross-service sharing.
|
||||
- Mount paths must be absolute canonical ASCII paths, at most 240 bytes, with
|
||||
no root, overlapping mount or system-tree mount. Denied trees: /proc, /sys,
|
||||
/dev, /etc, /run, /var/run, /bin, /sbin, /usr, /lib, /lib64 (including ancestors).
|
||||
- No command overrides, hooks, build, include, extends, profile, socket access,
|
||||
devices or arbitrary privilege additions. Unknown future keys also fail closed.
|
||||
|
||||
## What passing does not prove
|
||||
|
||||
This check does not authenticate a publisher, validate image contents or mount
|
||||
destinations inside an image, provision usable volume ownership, reserve resource
|
||||
names, verify engine/Compose compatibility, limit resource consumption, prove
|
||||
application readiness, or provide backup/restore. Image defaults and existing
|
||||
Docker resources must still be validated by future adapters and preflight.
|
||||
|
||||
Future execution must bind an explicit instance project name, verify resource
|
||||
ownership under the host lock, and use the exact checked snapshot without extra
|
||||
Compose files, ambient overrides or subsequent interpolation. An integrity hash
|
||||
and this restricted policy are not authorization to run a deployment.
|
||||
|
||||
References checked during implementation:
|
||||
[Compose services](https://docs.docker.com/reference/compose-file/services/),
|
||||
[Compose networks](https://docs.docker.com/reference/compose-file/networks/),
|
||||
[Compose config](https://docs.docker.com/reference/cli/docker/compose/config/).
|
||||
No real Docker/Compose execution has been validated in this batch.
|
||||
@@ -0,0 +1,135 @@
|
||||
// Package composepolicy validates a deliberately restricted, offline Compose
|
||||
// profile. Passing this policy never authorizes execution or proves image safety.
|
||||
package composepolicy
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"errors"
|
||||
"path"
|
||||
"regexp"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"server-deploy/internal/appbundle"
|
||||
"server-deploy/internal/wire"
|
||||
)
|
||||
|
||||
const Profile = "isolated-compose-v1"
|
||||
|
||||
type document struct {
|
||||
Services map[string]service `json:"services"`
|
||||
Networks map[string]network `json:"networks"`
|
||||
Volumes map[string]struct{} `json:"volumes"`
|
||||
}
|
||||
type network struct {
|
||||
Internal bool `json:"internal"`
|
||||
}
|
||||
type service struct {
|
||||
Image string `json:"image"`
|
||||
User string `json:"user"`
|
||||
ReadOnly bool `json:"read_only"`
|
||||
CapDrop []string `json:"cap_drop"`
|
||||
SecurityOpt []string `json:"security_opt"`
|
||||
Restart string `json:"restart"`
|
||||
Networks []string `json:"networks"`
|
||||
Volumes []mount `json:"volumes"`
|
||||
}
|
||||
type mount struct {
|
||||
Type string `json:"type"`
|
||||
Source string `json:"source"`
|
||||
Target string `json:"target"`
|
||||
ReadOnly bool `json:"read_only"`
|
||||
}
|
||||
|
||||
var identifier = regexp.MustCompile(`^[a-z][a-z0-9-]{0,47}$`)
|
||||
var pinnedImage = regexp.MustCompile(`^[a-z0-9][a-z0-9._/-]*@sha256:[0-9a-f]{64}$`)
|
||||
var targetPath = regexp.MustCompile(`^/[a-zA-Z0-9_./-]+$`)
|
||||
|
||||
// Check consumes only the authenticated entrypoint bytes and manifest supplied
|
||||
// by appbundle.Verify. It neither reads files nor renders/interpolates templates.
|
||||
// All fields in the profile are mandatory; unknown Compose features fail closed.
|
||||
func Check(manifest appbundle.Manifest, data []byte) error {
|
||||
reject := errors.New("Compose document rejected by restricted policy")
|
||||
var d document
|
||||
if wire.Decode(bytes.NewReader(data), &d, 4<<20) != nil {
|
||||
return reject
|
||||
}
|
||||
if len(manifest.Components) < 1 || len(manifest.Components) > 32 || len(d.Services) != len(manifest.Components) {
|
||||
return reject
|
||||
}
|
||||
if len(d.Networks) != 1 || !d.Networks["backend"].Internal || len(d.Volumes) > 128 {
|
||||
return reject
|
||||
}
|
||||
images := make(map[string]string, len(manifest.Components))
|
||||
for _, c := range manifest.Components {
|
||||
if !identifier.MatchString(c.Name) || !pinnedImage.MatchString(c.Image) || images[c.Name] != "" {
|
||||
return reject
|
||||
}
|
||||
images[c.Name] = c.Image
|
||||
}
|
||||
for name := range d.Volumes {
|
||||
if !identifier.MatchString(name) {
|
||||
return reject
|
||||
}
|
||||
}
|
||||
used := make(map[string]bool)
|
||||
for name, s := range d.Services {
|
||||
if images[name] == "" || s.Image != images[name] || !nonRootUser(s.User) || !s.ReadOnly {
|
||||
return reject
|
||||
}
|
||||
if !only(s.CapDrop, "ALL") || !only(s.SecurityOpt, "no-new-privileges:true") || !only(s.Networks, "backend") {
|
||||
return reject
|
||||
}
|
||||
if s.Restart != "unless-stopped" && s.Restart != "no" {
|
||||
return reject
|
||||
}
|
||||
if len(s.Volumes) > 128 {
|
||||
return reject
|
||||
}
|
||||
targets := make([]string, 0, len(s.Volumes))
|
||||
for _, v := range s.Volumes {
|
||||
if _, exists := d.Volumes[v.Source]; !exists || used[v.Source] || v.Type != "volume" {
|
||||
return reject
|
||||
}
|
||||
if len(v.Target) > 240 || !targetPath.MatchString(v.Target) || path.Clean(v.Target) != v.Target || v.Target == "/" {
|
||||
return reject
|
||||
}
|
||||
// Deny runtime/system trees as well as overlapping mounts. Only
|
||||
// application-data destinations belong in this initial profile.
|
||||
for _, protected := range []string{"/proc", "/sys", "/dev", "/etc", "/run", "/var/run", "/bin", "/sbin", "/usr", "/lib", "/lib64"} {
|
||||
if overlaps(v.Target, protected) {
|
||||
return reject
|
||||
}
|
||||
}
|
||||
for _, previous := range targets {
|
||||
if overlaps(v.Target, previous) {
|
||||
return reject
|
||||
}
|
||||
}
|
||||
targets = append(targets, v.Target)
|
||||
used[v.Source] = true
|
||||
}
|
||||
}
|
||||
if len(used) != len(d.Volumes) {
|
||||
return reject
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func only(values []string, expected string) bool { return len(values) == 1 && values[0] == expected }
|
||||
func overlaps(a, b string) bool {
|
||||
return a == b || strings.HasPrefix(a, b+"/") || strings.HasPrefix(b, a+"/")
|
||||
}
|
||||
func nonRootUser(value string) bool {
|
||||
parts := strings.Split(value, ":")
|
||||
if len(parts) != 2 {
|
||||
return false
|
||||
}
|
||||
for _, part := range parts {
|
||||
n, err := strconv.ParseUint(part, 10, 32)
|
||||
if err != nil || n == 0 || n == 4294967295 || strconv.FormatUint(n, 10) != part {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
@@ -0,0 +1,161 @@
|
||||
package composepolicy
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"server-deploy/internal/appbundle"
|
||||
)
|
||||
|
||||
func fixture() (appbundle.Manifest, map[string]any) {
|
||||
image := "example/api@sha256:" + strings.Repeat("a", 64)
|
||||
m := appbundle.Manifest{Components: []appbundle.Component{{Name: "api", Image: image}}}
|
||||
d := map[string]any{
|
||||
"services": map[string]any{"api": map[string]any{
|
||||
"image": image, "user": "1000:1000", "read_only": true,
|
||||
"cap_drop": []string{"ALL"}, "security_opt": []string{"no-new-privileges:true"},
|
||||
"restart": "unless-stopped", "networks": []string{"backend"},
|
||||
"volumes": []any{map[string]any{"type": "volume", "source": "data", "target": "/data", "read_only": false}},
|
||||
}},
|
||||
"networks": map[string]any{"backend": map[string]any{"internal": true}},
|
||||
"volumes": map[string]any{"data": map[string]any{}},
|
||||
}
|
||||
return m, d
|
||||
}
|
||||
|
||||
func TestAcceptRestrictedService(t *testing.T) {
|
||||
m, d := fixture()
|
||||
raw, _ := json.Marshal(d)
|
||||
if err := Check(m, raw); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRejectPrivilegeAndExternalInputs(t *testing.T) {
|
||||
for _, field := range []string{"privileged", "build", "container_name", "network_mode", "pid", "ipc", "userns_mode", "devices", "cap_add", "env_file", "environment", "extends", "ports", "labels", "use_api_socket", "post_start", "pre_stop", "command", "entrypoint", "volumes_from", "develop", "provider"} {
|
||||
t.Run(field, func(t *testing.T) {
|
||||
m, d := fixture()
|
||||
d["services"].(map[string]any)["api"].(map[string]any)[field] = "secret-sentinel"
|
||||
raw, _ := json.Marshal(d)
|
||||
err := Check(m, raw)
|
||||
if err == nil || strings.Contains(err.Error(), "secret-sentinel") {
|
||||
t.Fatal("forbidden field accepted or echoed")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestRejectUnsafeValuesAndReferences(t *testing.T) {
|
||||
mutations := map[string]func(map[string]any, map[string]any){
|
||||
"image drift": func(d, s map[string]any) { s["image"] = "example/api:latest" },
|
||||
"root": func(d, s map[string]any) { s["user"] = "0:0" },
|
||||
"named user": func(d, s map[string]any) { s["user"] = "root" },
|
||||
"writable root": func(d, s map[string]any) { s["read_only"] = false },
|
||||
"caps": func(d, s map[string]any) { s["cap_drop"] = []string{} },
|
||||
"escalation": func(d, s map[string]any) { s["security_opt"] = []string{"seccomp:unconfined"} },
|
||||
"implicit network": func(d, s map[string]any) { s["networks"] = []string{} },
|
||||
"other network": func(d, s map[string]any) { s["networks"] = []string{"default"} },
|
||||
"external network": func(d, s map[string]any) {
|
||||
d["networks"] = map[string]any{"backend": map[string]any{"internal": true, "external": true}}
|
||||
},
|
||||
"outbound network": func(d, s map[string]any) {
|
||||
d["networks"] = map[string]any{"backend": map[string]any{"internal": false}}
|
||||
},
|
||||
"bind": func(d, s map[string]any) { s["volumes"].([]any)[0].(map[string]any)["type"] = "bind" },
|
||||
"host source": func(d, s map[string]any) { s["volumes"].([]any)[0].(map[string]any)["source"] = "/var/run/docker.sock" },
|
||||
"missing volume": func(d, s map[string]any) { d["volumes"] = map[string]any{} },
|
||||
"volume driver": func(d, s map[string]any) {
|
||||
d["volumes"] = map[string]any{"data": map[string]any{"driver_opts": map[string]string{"device": "/"}}}
|
||||
},
|
||||
"root mount": func(d, s map[string]any) { s["volumes"].([]any)[0].(map[string]any)["target"] = "/" },
|
||||
"traversal": func(d, s map[string]any) { s["volumes"].([]any)[0].(map[string]any)["target"] = "/data/../etc" },
|
||||
"interpolation": func(d, s map[string]any) { s["user"] = "${UID}:1000" },
|
||||
"include": func(d, s map[string]any) { d["include"] = []string{"/secret"} },
|
||||
"null service": func(d, s map[string]any) { d["services"].(map[string]any)["api"] = nil },
|
||||
"missing security": func(d, s map[string]any) { delete(s, "security_opt") },
|
||||
"null volume": func(d, s map[string]any) { d["volumes"] = map[string]any{"data": nil} },
|
||||
"extra service": func(d, s map[string]any) { d["services"].(map[string]any)["rogue"] = s },
|
||||
"no services": func(d, s map[string]any) { d["services"] = map[string]any{} },
|
||||
}
|
||||
for name, mutate := range mutations {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
m, d := fixture()
|
||||
s := d["services"].(map[string]any)["api"].(map[string]any)
|
||||
mutate(d, s)
|
||||
raw, _ := json.Marshal(d)
|
||||
if Check(m, raw) == nil {
|
||||
t.Fatal("unsafe document accepted")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestRejectMalformedDocument(t *testing.T) {
|
||||
m, d := fixture()
|
||||
raw, _ := json.Marshal(d)
|
||||
for _, input := range []string{"services: {}", string(raw) + "{}", strings.Replace(string(raw), `"read_only":true`, `"read_only":true,"read_only":false`, 1), strings.Repeat(" ", 4<<20) + string(raw)} {
|
||||
if Check(m, []byte(input)) == nil {
|
||||
t.Fatal("malformed input accepted")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestMountAndIdentityBoundaries(t *testing.T) {
|
||||
for _, target := range []string{"/proc/x", "/var", "/var/run/docker.sock", "/sys", "/dev", "/etc", "/run", "/usr/local", "/lib", "/lib64", "/bin", "/sbin", "/data/", "/data//x", "/data/$HOME", `C:\data`, "/" + strings.Repeat("a", 241)} {
|
||||
m, d := fixture()
|
||||
s := d["services"].(map[string]any)["api"].(map[string]any)
|
||||
s["volumes"].([]any)[0].(map[string]any)["target"] = target
|
||||
raw, _ := json.Marshal(d)
|
||||
if Check(m, raw) == nil {
|
||||
t.Errorf("accepted protected/noncanonical target %q", target)
|
||||
}
|
||||
}
|
||||
for _, user := range []string{"1000:0", "01:1000", "+1:1000", "-1:1000", "4294967295:1000", "4294967296:1000", "1", "1:2:3"} {
|
||||
m, d := fixture()
|
||||
d["services"].(map[string]any)["api"].(map[string]any)["user"] = user
|
||||
raw, _ := json.Marshal(d)
|
||||
if Check(m, raw) == nil {
|
||||
t.Errorf("accepted ambiguous/root identity %q", user)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestTwoServicesRequireExactManifestAndSeparateVolumes(t *testing.T) {
|
||||
m, d := fixture()
|
||||
raw, _ := json.Marshal(d)
|
||||
var copyDoc map[string]any
|
||||
if err := json.Unmarshal(raw, ©Doc); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
second := copyDoc["services"].(map[string]any)["api"].(map[string]any)
|
||||
second["volumes"].([]any)[0].(map[string]any)["source"] = "db-data"
|
||||
d["services"].(map[string]any)["db"] = second
|
||||
d["volumes"].(map[string]any)["db-data"] = map[string]any{}
|
||||
m.Components = append(m.Components, appbundle.Component{Name: "db", Image: m.Components[0].Image})
|
||||
raw, _ = json.Marshal(d)
|
||||
if err := Check(m, raw); err != nil {
|
||||
t.Fatal("valid separate-service volumes rejected", err)
|
||||
}
|
||||
second["volumes"].([]any)[0].(map[string]any)["source"] = "data"
|
||||
delete(d["volumes"].(map[string]any), "db-data")
|
||||
raw, _ = json.Marshal(d)
|
||||
if Check(m, raw) == nil {
|
||||
t.Fatal("shared service data accepted")
|
||||
}
|
||||
}
|
||||
|
||||
func TestOverlappingAndUnusedVolumesRejected(t *testing.T) {
|
||||
for _, target := range []string{"/data", "/data/nested", "/other"} {
|
||||
m, d := fixture()
|
||||
d["volumes"].(map[string]any)["second"] = map[string]any{}
|
||||
if target != "/other" {
|
||||
s := d["services"].(map[string]any)["api"].(map[string]any)
|
||||
s["volumes"] = append(s["volumes"].([]any), map[string]any{"type": "volume", "source": "second", "target": target, "read_only": false})
|
||||
}
|
||||
raw, _ := json.Marshal(d)
|
||||
if Check(m, raw) == nil {
|
||||
t.Fatal("overlapping or unused volume accepted")
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,33 @@
|
||||
# Read-only relevant dpkg inventory
|
||||
|
||||
`Inventory(fs.FS) Snapshot` reads fixed `var/lib/dpkg/status` (regular file,
|
||||
nonempty, at most 16 MiB) and checks the fixed `var/lib/dpkg/updates` directory
|
||||
is empty before and after reading. It invokes no package tools and writes nothing.
|
||||
The filesystem must provide metadata via `fs.StatFS`; unsupported, inaccessible,
|
||||
malformed, oversized, changing or journal-busy input returns `state=unknown`, an
|
||||
empty digest and empty packages array. Missing status is not a clean machine.
|
||||
|
||||
It validates stanza structure/identity/status before filtering. Field names are
|
||||
case-insensitive; duplicate keys, malformed scalar continuations and duplicate
|
||||
package/architecture records are rejected. Descriptions/other values are never
|
||||
returned. Distinct multiarch records are retained; ambiguous all/unspecified
|
||||
architecture duplicates fail closed. Bare not-installed selections are permitted
|
||||
and still returned when relevant, so a caller cannot mistake them for no record.
|
||||
|
||||
An observed snapshot contains the SHA-256 of the complete status file bytes and
|
||||
deterministically ordered name/version/architecture/status records for:
|
||||
docker-ce, docker-ce-cli, containerd.io, docker-buildx-plugin,
|
||||
docker-compose-plugin, docker.io, docker-compose, docker-compose-v2, docker-doc,
|
||||
docker-buildx, podman-docker, containerd and runc.
|
||||
|
||||
The `Installed` type means a database record exists, not that it is fully
|
||||
installed. Callers must conservatively review **every** returned record, including
|
||||
hold, partial installation, residual config and not-installed selections.
|
||||
|
||||
OS paths/ancestors and filesystem implementation are trusted. Metadata/journal
|
||||
rechecks detect ordinary changes but do not lock dpkg or produce an atomic
|
||||
transaction against concurrent writes. The digest is not a host identity or
|
||||
approval. This does not scan custom package databases, rootless/manual runtimes,
|
||||
APT sources, package dependencies, or unrelated packages' operational health.
|
||||
|
||||
Format reference: [Debian control files](https://www.debian.org/doc/debian-policy/ch-controlfields.html).
|
||||
@@ -0,0 +1,267 @@
|
||||
// Package debian observes the dpkg database without executing package tools.
|
||||
package debian
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"bytes"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"io"
|
||||
"io/fs"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strings"
|
||||
"unicode/utf8"
|
||||
)
|
||||
|
||||
type Snapshot struct {
|
||||
State string `json:"state"`
|
||||
Digest string `json:"digest"`
|
||||
Packages []Installed `json:"packages"`
|
||||
}
|
||||
|
||||
// Installed is a present database record, including residual or uninstalled selections.
|
||||
type Installed struct {
|
||||
Name string `json:"name"`
|
||||
Version string `json:"version"`
|
||||
Architecture string `json:"architecture"`
|
||||
Status string `json:"status"`
|
||||
}
|
||||
|
||||
const (
|
||||
statusPath = "var/lib/dpkg/status"
|
||||
updatesPath = "var/lib/dpkg/updates"
|
||||
maxStatusBytes = 16 << 20
|
||||
)
|
||||
|
||||
// Inventory reads only the fixed status and updates paths. An observed snapshot
|
||||
// includes every relevant record, regardless of installation state; it is not
|
||||
// an installation permission. Unknown never exposes a partial result or error.
|
||||
// Paths and the FS implementation are trusted. Metadata and journal rechecks
|
||||
// detect ordinary changes, but are not a lock or protection against a hostile
|
||||
// administrator replacing paths between checks.
|
||||
func Inventory(files fs.FS) Snapshot {
|
||||
unknown := Snapshot{State: "unknown", Packages: []Installed{}}
|
||||
// fs.Stat's fallback opens the path. Require a metadata operation so that
|
||||
// checking an already-present FIFO cannot block before we reject its type.
|
||||
metadata, ok := files.(fs.StatFS)
|
||||
if !ok || !emptyJournal(files, metadata) {
|
||||
return unknown
|
||||
}
|
||||
initial, err := metadata.Stat(statusPath)
|
||||
if err != nil || !validStatusFile(initial) {
|
||||
return unknown
|
||||
}
|
||||
f, err := files.Open(statusPath)
|
||||
if err != nil {
|
||||
return unknown
|
||||
}
|
||||
opened, err := f.Stat()
|
||||
if err != nil || !sameMetadata(initial, opened) {
|
||||
f.Close()
|
||||
return unknown
|
||||
}
|
||||
raw, readErr := io.ReadAll(io.LimitReader(f, maxStatusBytes+1))
|
||||
after, statErr := f.Stat()
|
||||
closeErr := f.Close()
|
||||
if readErr != nil || statErr != nil || closeErr != nil || len(raw) == 0 || len(raw) > maxStatusBytes || int64(len(raw)) != initial.Size() || !sameMetadata(initial, after) {
|
||||
return unknown
|
||||
}
|
||||
current, err := metadata.Stat(statusPath)
|
||||
if err != nil || !sameMetadata(initial, current) {
|
||||
return unknown
|
||||
}
|
||||
packages, ok := parseStatus(raw)
|
||||
if !ok || !emptyJournal(files, metadata) {
|
||||
return unknown
|
||||
}
|
||||
sum := sha256.Sum256(raw)
|
||||
return Snapshot{State: "observed", Digest: "sha256:" + hex.EncodeToString(sum[:]), Packages: packages}
|
||||
}
|
||||
|
||||
func validStatusFile(info fs.FileInfo) bool {
|
||||
return info != nil && info.Mode().IsRegular() && info.Size() > 0 && info.Size() <= maxStatusBytes
|
||||
}
|
||||
|
||||
func sameMetadata(a, b fs.FileInfo) bool {
|
||||
return a != nil && b != nil && a.Mode() == b.Mode() && a.Size() == b.Size() && a.ModTime().Equal(b.ModTime())
|
||||
}
|
||||
|
||||
func emptyJournal(files fs.FS, metadata fs.StatFS) bool {
|
||||
initial, err := metadata.Stat(updatesPath)
|
||||
if err != nil || initial == nil || !initial.IsDir() {
|
||||
return false
|
||||
}
|
||||
f, err := files.Open(updatesPath)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
opened, statErr := f.Stat()
|
||||
dir, ok := f.(fs.ReadDirFile)
|
||||
if statErr != nil || !sameMetadata(initial, opened) || !ok {
|
||||
f.Close()
|
||||
return false
|
||||
}
|
||||
// Read at most one entry: even a hidden file or a directory is pending work.
|
||||
entries, readErr := dir.ReadDir(1)
|
||||
closeErr := f.Close()
|
||||
return len(entries) == 0 && readErr == io.EOF && closeErr == nil
|
||||
}
|
||||
|
||||
var packageName = regexp.MustCompile(`^[a-z0-9][a-z0-9+.-]+$`)
|
||||
var architectureName = regexp.MustCompile(`^[a-z0-9][a-z0-9-]*$`)
|
||||
var packageVersion = regexp.MustCompile(`^(?:[0-9]+:)?[0-9][A-Za-z0-9.+:~\-]*$`)
|
||||
|
||||
func relevant(name string) bool {
|
||||
switch name {
|
||||
case "docker-ce", "docker-ce-cli", "containerd.io", "docker-buildx-plugin", "docker-compose-plugin", "docker.io", "docker-compose", "docker-compose-v2", "docker-doc", "docker-buildx", "podman-docker", "containerd", "runc":
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func scalarField(key string) bool {
|
||||
return key == "package" || key == "status" || key == "architecture" || key == "version"
|
||||
}
|
||||
|
||||
// Validate structure and identifying fields in ALL stanzas, before filtering.
|
||||
// Other field values (including descriptions) are never part of the snapshot.
|
||||
func parseStatus(raw []byte) ([]Installed, bool) {
|
||||
if !utf8.Valid(raw) {
|
||||
return nil, false
|
||||
}
|
||||
packages := []Installed{}
|
||||
seen := make(map[string]map[string]bool)
|
||||
fields := make(map[string]string)
|
||||
last := ""
|
||||
count := 0
|
||||
finish := func() bool {
|
||||
if len(fields) == 0 {
|
||||
return true
|
||||
}
|
||||
p := Installed{Name: fields["package"], Version: fields["version"], Architecture: fields["architecture"]}
|
||||
status, ok := normalizedStatus(fields["status"])
|
||||
if !ok || !packageName.MatchString(p.Name) {
|
||||
return false
|
||||
}
|
||||
p.Status = status
|
||||
// dpkg may retain a bare selection for a never-installed package.
|
||||
notInstalled := strings.HasSuffix(status, " not-installed")
|
||||
if p.Architecture == "" {
|
||||
if !notInstalled || hasField(fields, "architecture") {
|
||||
return false
|
||||
}
|
||||
} else if !architectureName.MatchString(p.Architecture) || p.Architecture == "any" || p.Architecture == "source" || strings.HasPrefix(p.Architecture, "any-") || strings.HasSuffix(p.Architecture, "-any") {
|
||||
return false
|
||||
}
|
||||
if p.Version == "" {
|
||||
if !notInstalled || hasField(fields, "version") {
|
||||
return false
|
||||
}
|
||||
} else if !packageVersion.MatchString(p.Version) || strings.HasSuffix(p.Version, "-") || strings.HasSuffix(p.Version, ":") {
|
||||
return false
|
||||
}
|
||||
arches := seen[p.Name]
|
||||
if len(arches) != 0 && (arches[p.Architecture] || arches[""] || arches["all"] || p.Architecture == "" || p.Architecture == "all") {
|
||||
return false
|
||||
}
|
||||
if arches == nil {
|
||||
arches = make(map[string]bool)
|
||||
seen[p.Name] = arches
|
||||
}
|
||||
arches[p.Architecture] = true
|
||||
if relevant(p.Name) {
|
||||
packages = append(packages, p)
|
||||
}
|
||||
count++
|
||||
fields = make(map[string]string)
|
||||
last = ""
|
||||
return true
|
||||
}
|
||||
scanner := bufio.NewScanner(bytes.NewReader(raw))
|
||||
// The file cap is also the token cap; long legitimate description lines
|
||||
// must not be silently lost to Scanner's default 64 KiB limit.
|
||||
scanner.Buffer(make([]byte, 4096), maxStatusBytes+1)
|
||||
for scanner.Scan() {
|
||||
line := scanner.Text()
|
||||
for _, c := range line {
|
||||
if (c < 32 && c != '\t') || c == 127 {
|
||||
return nil, false
|
||||
}
|
||||
}
|
||||
if strings.Trim(line, " \t") == "" {
|
||||
if !finish() {
|
||||
return nil, false
|
||||
}
|
||||
continue
|
||||
}
|
||||
if line[0] == ' ' || line[0] == '\t' {
|
||||
if last == "" || scalarField(last) {
|
||||
return nil, false
|
||||
}
|
||||
continue
|
||||
}
|
||||
key, value, ok := strings.Cut(line, ":")
|
||||
if !ok || !validFieldName(key) {
|
||||
return nil, false
|
||||
}
|
||||
key = strings.ToLower(key)
|
||||
if hasField(fields, key) {
|
||||
return nil, false
|
||||
}
|
||||
// Retain only the values we project, but track every key for duplicates.
|
||||
fields[key] = ""
|
||||
if scalarField(key) {
|
||||
fields[key] = strings.Trim(value, " \t")
|
||||
}
|
||||
last = key
|
||||
}
|
||||
if scanner.Err() != nil || !finish() || count == 0 {
|
||||
return nil, false
|
||||
}
|
||||
sort.Slice(packages, func(i, j int) bool {
|
||||
if packages[i].Name != packages[j].Name {
|
||||
return packages[i].Name < packages[j].Name
|
||||
}
|
||||
return packages[i].Architecture < packages[j].Architecture
|
||||
})
|
||||
return packages, true
|
||||
}
|
||||
|
||||
func hasField(fields map[string]string, key string) bool {
|
||||
_, ok := fields[key]
|
||||
return ok
|
||||
}
|
||||
|
||||
func validFieldName(key string) bool {
|
||||
if key == "" || key[0] == '#' || key[0] == '-' {
|
||||
return false
|
||||
}
|
||||
for i := range len(key) {
|
||||
if key[i] < 33 || key[i] > 126 || key[i] == ':' {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func normalizedStatus(value string) (string, bool) {
|
||||
parts := strings.FieldsFunc(value, func(r rune) bool { return r == ' ' || r == '\t' })
|
||||
if len(parts) != 3 {
|
||||
return "", false
|
||||
}
|
||||
switch parts[0] {
|
||||
case "unknown", "install", "hold", "deinstall", "purge":
|
||||
default:
|
||||
return "", false
|
||||
}
|
||||
if parts[1] != "ok" && parts[1] != "reinstreq" {
|
||||
return "", false
|
||||
}
|
||||
switch parts[2] {
|
||||
case "not-installed", "config-files", "half-installed", "unpacked", "half-configured", "triggers-awaited", "triggers-pending", "installed":
|
||||
default:
|
||||
return "", false
|
||||
}
|
||||
return strings.Join(parts, " "), true
|
||||
}
|
||||
@@ -0,0 +1,386 @@
|
||||
package debian
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"io"
|
||||
"io/fs"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
"testing/fstest"
|
||||
"time"
|
||||
)
|
||||
|
||||
const fixtureStatus = "var/lib/dpkg/status"
|
||||
const fixtureUpdates = "var/lib/dpkg/updates"
|
||||
|
||||
func stanza(name, arch, status string) string {
|
||||
return "Package: " + name + "\nStatus: " + status + "\nArchitecture: " + arch + "\nVersion: 5:28.0.1-1~ubuntu.24.04~noble\nDescription: container runtime\n continuation with Package: ignored\n .\n\tUTF-8 description: 容器\n"
|
||||
}
|
||||
|
||||
func statusFS(raw string) fstest.MapFS {
|
||||
return fstest.MapFS{
|
||||
fixtureStatus: &fstest.MapFile{Data: []byte(raw), Mode: 0644},
|
||||
fixtureUpdates: &fstest.MapFile{Mode: fs.ModeDir | 0755},
|
||||
}
|
||||
}
|
||||
|
||||
func requireUnknown(t *testing.T, files fs.FS) {
|
||||
t.Helper()
|
||||
got := Inventory(files)
|
||||
if got.State != "unknown" || got.Digest != "" || got.Packages == nil || len(got.Packages) != 0 {
|
||||
t.Fatal("invalid input must produce unknown, empty digest, and non-nil empty packages")
|
||||
}
|
||||
raw, err := json.Marshal(got)
|
||||
if err != nil || string(raw) != `{"state":"unknown","digest":"","packages":[]}` {
|
||||
t.Fatal("unknown result must expose only the empty public JSON shape")
|
||||
}
|
||||
}
|
||||
|
||||
func TestInventoryObservedAndExactDigest(t *testing.T) {
|
||||
base := stanza("base-files", "amd64", "install ok installed")
|
||||
for _, raw := range []string{base, "\n" + base + "\n\n", strings.ReplaceAll(base, "\n", "\r\n"), strings.TrimSuffix(base, "\n")} {
|
||||
got := Inventory(statusFS(raw))
|
||||
sum := sha256.Sum256([]byte(raw))
|
||||
if got.State != "observed" || got.Packages == nil || len(got.Packages) != 0 || got.Digest != "sha256:"+hex.EncodeToString(sum[:]) {
|
||||
t.Fatal("valid unrelated package must yield observed empty inventory and exact-byte digest")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestInventoryAllRelevantNames(t *testing.T) {
|
||||
names := []string{"containerd", "containerd.io", "docker-buildx", "docker-buildx-plugin", "docker-ce", "docker-ce-cli", "docker-compose", "docker-compose-plugin", "docker-compose-v2", "docker-doc", "docker.io", "podman-docker", "runc"}
|
||||
var raw strings.Builder
|
||||
for i := len(names) - 1; i >= 0; i-- {
|
||||
raw.WriteString(stanza(names[i], "amd64", "install ok installed") + "\n")
|
||||
}
|
||||
raw.WriteString(stanza("docker-ce-extra", "amd64", "install ok installed"))
|
||||
got := Inventory(statusFS(raw.String()))
|
||||
if got.State != "observed" || len(got.Packages) != len(names) {
|
||||
t.Fatal("inventory must include exactly the fixed relevant package set")
|
||||
}
|
||||
for i, name := range names {
|
||||
want := Installed{Name: name, Version: "5:28.0.1-1~ubuntu.24.04~noble", Architecture: "amd64", Status: "install ok installed"}
|
||||
if got.Packages[i] != want {
|
||||
t.Fatal("relevant packages must preserve projected fields and sort by name")
|
||||
}
|
||||
}
|
||||
encoded, err := json.Marshal(got.Packages[0])
|
||||
if err != nil || string(encoded) != `{"name":"containerd","version":"5:28.0.1-1~ubuntu.24.04~noble","architecture":"amd64","status":"install ok installed"}` {
|
||||
t.Fatal("installed JSON must contain only the four specified fields")
|
||||
}
|
||||
}
|
||||
|
||||
func TestInventoryKeepsEveryStatusAndMultiarch(t *testing.T) {
|
||||
statuses := []string{"install ok installed", "hold ok installed", "deinstall ok config-files", "install reinstreq half-installed", "install ok unpacked", "install ok half-configured", "install ok triggers-awaited", "install ok triggers-pending", "purge ok not-installed", "unknown ok not-installed"}
|
||||
for _, status := range statuses {
|
||||
t.Run(status, func(t *testing.T) {
|
||||
raw := stanza("runc", "arm64", status) + "\n" + stanza("runc", "amd64", status)
|
||||
got := Inventory(statusFS(raw))
|
||||
if got.State != "observed" || len(got.Packages) != 2 || got.Packages[0].Architecture != "amd64" || got.Packages[1].Architecture != "arm64" || got.Packages[0].Status != status || got.Packages[1].Status != status {
|
||||
t.Fatal("every present record must survive regardless of installation state; sort multiarch by architecture")
|
||||
}
|
||||
})
|
||||
}
|
||||
got := Inventory(statusFS("Package: docker-ce\nStatus: purge ok not-installed\n"))
|
||||
if got.State != "observed" || !reflect.DeepEqual(got.Packages, []Installed{{Name: "docker-ce", Status: "purge ok not-installed"}}) {
|
||||
t.Fatal("not-installed selection records may lack architecture and version but must still block")
|
||||
}
|
||||
}
|
||||
|
||||
func TestInventoryCaseInsensitiveFieldsAndWhitespace(t *testing.T) {
|
||||
raw := "pAcKaGe:\tdocker.io \nSTATUS: hold\t ok installed\narchitecture: all\nversion: 1.2+dfsg-3\nDESCRIPTION: first\n second\n .\n third\n\t \n" + stanza("base-files", "amd64", "install ok installed")
|
||||
got := Inventory(statusFS(raw))
|
||||
if got.State != "observed" || !reflect.DeepEqual(got.Packages, []Installed{{Name: "docker.io", Version: "1.2+dfsg-3", Architecture: "all", Status: "hold ok installed"}}) {
|
||||
t.Fatal("field aliases must normalize safely, including status whitespace and whitespace-only stanza separators")
|
||||
}
|
||||
}
|
||||
|
||||
func TestInventoryRejectsMalformedGlobally(t *testing.T) {
|
||||
valid := stanza("base-files", "amd64", "install ok installed")
|
||||
cases := map[string]string{
|
||||
"empty": "", "blank": "\n \t\n", "orphan continuation": " unexpected\n" + valid,
|
||||
"missing colon": valid + "broken line\n", "empty field name": valid + ": value\n",
|
||||
"field whitespace": valid + "Bad Field: x\n", "field leading hyphen": valid + "-Bad: x\n",
|
||||
"comment": valid + "# comment\n", "duplicate unrelated key": valid + "description: duplicate\n",
|
||||
"duplicate package alias": valid + "PACKAGE: docker-ce\n", "duplicate status alias": valid + "status: purge ok not-installed\n",
|
||||
"duplicate version alias": valid + "VERSION: 1\n", "duplicate architecture alias": valid + "ARCHITECTURE: arm64\n",
|
||||
"duplicate irrelevant stanza": valid + "\n" + valid,
|
||||
"duplicate relevant stanza": stanza("runc", "amd64", "install ok installed") + "\n" + stanza("runc", "amd64", "hold ok installed"),
|
||||
"missing package": "Status: install ok installed\nArchitecture: amd64\nVersion: 1\n",
|
||||
"missing status": "Package: base-files\nArchitecture: amd64\nVersion: 1\n",
|
||||
"missing installed version": "Package: base-files\nStatus: install ok installed\nArchitecture: amd64\n",
|
||||
"missing installed architecture": "Package: base-files\nStatus: install ok installed\nVersion: 1\n",
|
||||
"unknown selection": stanza("base-files", "amd64", "selected ok installed"),
|
||||
"unknown flag": stanza("base-files", "amd64", "install bad installed"),
|
||||
"unknown state": stanza("base-files", "amd64", "install ok ready"),
|
||||
"status suffix": stanza("base-files", "amd64", "install ok installed extra"),
|
||||
"status short": stanza("base-files", "amd64", "ok installed"),
|
||||
"status case": stanza("base-files", "amd64", "Install ok installed"),
|
||||
"unicode status space": stanza("base-files", "amd64", "install\u00a0ok installed"),
|
||||
"architecture list": stanza("base-files", "amd64 arm64", "install ok installed"),
|
||||
"architecture wildcard": stanza("base-files", "any", "install ok installed"),
|
||||
"architecture source": stanza("base-files", "source", "install ok installed"),
|
||||
"architecture punctuation": stanza("base-files", "amd64!", "install ok installed"),
|
||||
"architecture ambiguous": "Package: runc\nStatus: purge ok not-installed\n\n" + stanza("runc", "amd64", "install ok installed"),
|
||||
"architecture all mixed": stanza("runc", "all", "install ok installed") + "\n" + stanza("runc", "amd64", "install ok installed"),
|
||||
"scalar continuation": "Package: base-files\n unexpected\nStatus: install ok installed\nArchitecture: amd64\nVersion: 1\n",
|
||||
"version continuation": "Package: base-files\nStatus: install ok installed\nArchitecture: amd64\nVersion: 1\n 2\n",
|
||||
"nul": valid + "X-Note: hidden\x00value\n", "bare CR": valid + "X-Note: hidden\rvalue\n",
|
||||
"invalid utf8": valid + "X-Note: \xff\n",
|
||||
"unsafe version": strings.Replace(valid, "5:28.0.1-1~ubuntu.24.04~noble", "1;secret", 1),
|
||||
"empty version": strings.Replace(valid, "5:28.0.1-1~ubuntu.24.04~noble", "", 1),
|
||||
}
|
||||
for _, name := range []string{"a", "Docker-ce", "docker_ce", "docker-ce:amd64", "-docker", "docker ce", "dockér"} {
|
||||
cases["invalid package "+name] = stanza(name, "amd64", "install ok installed")
|
||||
}
|
||||
for name, raw := range cases {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
requireUnknown(t, statusFS(raw))
|
||||
// A valid relevant record before corrupt data must never leak a partial result.
|
||||
if strings.TrimSpace(raw) != "" {
|
||||
requireUnknown(t, statusFS(stanza("docker-ce", "amd64", "install ok installed")+"\n"+raw))
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestInventoryFileAndJournalFailures(t *testing.T) {
|
||||
for _, name := range []string{"missing status", "missing updates", "updates regular", "updates entry", "updates subdir", "updates hidden entry", "status directory", "status fifo", "status device", "status oversized"} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
files := statusFS(stanza("base-files", "amd64", "install ok installed"))
|
||||
switch name {
|
||||
case "missing status":
|
||||
delete(files, fixtureStatus)
|
||||
case "missing updates":
|
||||
delete(files, fixtureUpdates)
|
||||
case "updates regular":
|
||||
files[fixtureUpdates].Mode = 0644
|
||||
case "updates entry":
|
||||
files[fixtureUpdates+"/0000"] = &fstest.MapFile{}
|
||||
case "updates subdir":
|
||||
files[fixtureUpdates+"/pending"] = &fstest.MapFile{Mode: fs.ModeDir | 0700}
|
||||
case "updates hidden entry":
|
||||
files[fixtureUpdates+"/.pending"] = &fstest.MapFile{}
|
||||
case "status directory":
|
||||
files[fixtureStatus].Mode = fs.ModeDir | 0755
|
||||
case "status fifo":
|
||||
files[fixtureStatus].Mode = fs.ModeNamedPipe | 0600
|
||||
case "status device":
|
||||
files[fixtureStatus].Mode = fs.ModeDevice | 0600
|
||||
case "status oversized":
|
||||
files[fixtureStatus].Data = []byte(strings.Repeat("x", (16<<20)+1))
|
||||
}
|
||||
requireUnknown(t, files)
|
||||
})
|
||||
}
|
||||
requireUnknown(t, nil)
|
||||
}
|
||||
|
||||
func TestInventoryBoundedLargeDescription(t *testing.T) {
|
||||
// Exceed Scanner's default 64 KiB token size without exceeding the file cap.
|
||||
raw := stanza("docker.io", "amd64", "install ok installed") + " " + strings.Repeat("x", 128<<10) + "\n"
|
||||
if got := Inventory(statusFS(raw)); got.State != "observed" || len(got.Packages) != 1 {
|
||||
t.Fatal("bounded long description continuation must not hide a relevant record")
|
||||
}
|
||||
// A valid file exactly at the cap is accepted; the next byte is rejected.
|
||||
base := stanza("base-files", "amd64", "install ok installed")
|
||||
raw = base + " " + strings.Repeat("x", (16<<20)-len(base)-2) + "\n"
|
||||
if got := Inventory(statusFS(raw)); got.State != "observed" {
|
||||
t.Fatal("exactly 16 MiB must be accepted")
|
||||
}
|
||||
requireUnknown(t, statusFS(raw+"\n"))
|
||||
}
|
||||
|
||||
func TestInventoryRealDirectory(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
if err := os.MkdirAll(filepath.Join(root, filepath.FromSlash(fixtureUpdates)), 0755); err != nil {
|
||||
t.Fatal("create fixture directory")
|
||||
}
|
||||
raw := stanza("docker-compose-v2", "arm64", "deinstall ok config-files")
|
||||
path := filepath.Join(root, filepath.FromSlash(fixtureStatus))
|
||||
if err := os.WriteFile(path, []byte(raw), 0644); err != nil {
|
||||
t.Fatal("write fixture status")
|
||||
}
|
||||
got := Inventory(os.DirFS(root))
|
||||
if got.State != "observed" || len(got.Packages) != 1 || got.Packages[0].Status != "deinstall ok config-files" {
|
||||
t.Fatal("real filesystem residual record missing")
|
||||
}
|
||||
after, err := os.ReadFile(path)
|
||||
if err != nil || string(after) != raw {
|
||||
t.Fatal("inventory must not modify the database")
|
||||
}
|
||||
}
|
||||
|
||||
// Faults are confined to the FS boundary: permissions and read-time changes
|
||||
// cannot be exercised portably with chmod (notably on Windows or as root).
|
||||
type faultFS struct {
|
||||
fstest.MapFS
|
||||
fault string
|
||||
journalOpens int
|
||||
statusOpens int
|
||||
statusStats int
|
||||
}
|
||||
|
||||
func (f *faultFS) Stat(path string) (fs.FileInfo, error) {
|
||||
if (f.fault == "status stat denied" && path == fixtureStatus) || (f.fault == "journal stat denied" && path == fixtureUpdates) {
|
||||
return nil, errors.New("private stat diagnostic")
|
||||
}
|
||||
info, err := f.MapFS.Stat(path)
|
||||
if path == fixtureStatus {
|
||||
f.statusStats++
|
||||
if f.fault == "path changed" && f.statusStats > 1 && err == nil {
|
||||
return changedInfo{FileInfo: info, change: "time"}, nil
|
||||
}
|
||||
}
|
||||
return info, err
|
||||
}
|
||||
|
||||
func (f *faultFS) Open(path string) (fs.File, error) {
|
||||
if path != fixtureStatus && path != fixtureUpdates {
|
||||
return nil, errors.New("unexpected path")
|
||||
}
|
||||
if path == fixtureStatus {
|
||||
f.statusOpens++
|
||||
if f.fault == "status open denied" {
|
||||
return nil, errors.New("private open diagnostic")
|
||||
}
|
||||
} else {
|
||||
f.journalOpens++
|
||||
if f.fault == "journal open denied" {
|
||||
return nil, errors.New("private open diagnostic")
|
||||
}
|
||||
if f.fault == "journal becomes pending" && f.journalOpens == 2 {
|
||||
f.MapFS[fixtureUpdates+"/0001"] = &fstest.MapFile{}
|
||||
}
|
||||
}
|
||||
file, err := f.MapFS.Open(path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &faultFile{File: file, owner: f, path: path}, nil
|
||||
}
|
||||
|
||||
type faultFile struct {
|
||||
fs.File
|
||||
owner *faultFS
|
||||
path string
|
||||
stats int
|
||||
bytesRead int
|
||||
}
|
||||
|
||||
func (f *faultFile) Stat() (fs.FileInfo, error) {
|
||||
f.stats++
|
||||
info, err := f.File.Stat()
|
||||
if f.path == fixtureStatus && err == nil {
|
||||
switch f.owner.fault {
|
||||
case "opened stat denied":
|
||||
return nil, fs.ErrPermission
|
||||
case "opened fifo":
|
||||
return changedInfo{FileInfo: info, change: "fifo"}, nil
|
||||
case "opened size changed":
|
||||
return changedInfo{FileInfo: info, change: "size"}, nil
|
||||
case "changed while reading":
|
||||
if f.stats > 1 {
|
||||
return changedInfo{FileInfo: info, change: "time"}, nil
|
||||
}
|
||||
}
|
||||
}
|
||||
return info, err
|
||||
}
|
||||
|
||||
func (f *faultFile) Read(p []byte) (int, error) {
|
||||
if f.path == fixtureStatus {
|
||||
switch f.owner.fault {
|
||||
case "read denied":
|
||||
return 0, errors.New("private read diagnostic")
|
||||
case "truncated":
|
||||
return 0, io.EOF
|
||||
case "growing":
|
||||
// Ignore the advertised size, like a file growing after stat.
|
||||
if f.bytesRead+len(p) > (16<<20)+1 {
|
||||
return 0, errors.New("read exceeded bound")
|
||||
}
|
||||
for i := range p {
|
||||
p[i] = 'x'
|
||||
}
|
||||
f.bytesRead += len(p)
|
||||
return len(p), nil
|
||||
}
|
||||
}
|
||||
return f.File.Read(p)
|
||||
}
|
||||
|
||||
func (f *faultFile) ReadDir(n int) ([]fs.DirEntry, error) {
|
||||
if f.owner.fault == "journal read denied" {
|
||||
return nil, errors.New("private journal diagnostic")
|
||||
}
|
||||
return f.File.(fs.ReadDirFile).ReadDir(n)
|
||||
}
|
||||
|
||||
func (f *faultFile) Close() error {
|
||||
err := f.File.Close()
|
||||
if f.owner.fault == "close failure" {
|
||||
return errors.New("private close diagnostic")
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
type changedInfo struct {
|
||||
fs.FileInfo
|
||||
change string
|
||||
}
|
||||
|
||||
func (i changedInfo) Mode() fs.FileMode {
|
||||
if i.change == "fifo" {
|
||||
return fs.ModeNamedPipe | 0600
|
||||
}
|
||||
return i.FileInfo.Mode()
|
||||
}
|
||||
|
||||
func (i changedInfo) Size() int64 {
|
||||
if i.change == "size" {
|
||||
return i.FileInfo.Size() + 1
|
||||
}
|
||||
return i.FileInfo.Size()
|
||||
}
|
||||
|
||||
func (i changedInfo) ModTime() time.Time {
|
||||
if i.change == "time" {
|
||||
return i.FileInfo.ModTime().Add(time.Second)
|
||||
}
|
||||
return i.FileInfo.ModTime()
|
||||
}
|
||||
|
||||
func TestInventoryFailsClosedOnReadFaultsAndChanges(t *testing.T) {
|
||||
for _, fault := range []string{"status stat denied", "journal stat denied", "status open denied", "journal open denied", "opened stat denied", "opened fifo", "opened size changed", "changed while reading", "path changed", "read denied", "truncated", "growing", "journal read denied", "journal becomes pending", "close failure"} {
|
||||
t.Run(fault, func(t *testing.T) {
|
||||
files := &faultFS{MapFS: statusFS(stanza("runc", "amd64", "install ok installed")), fault: fault}
|
||||
requireUnknown(t, files)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestInventoryStatsBeforeOpeningNonregularFiles(t *testing.T) {
|
||||
for _, path := range []string{fixtureStatus, fixtureUpdates} {
|
||||
files := &faultFS{MapFS: statusFS(stanza("runc", "amd64", "install ok installed"))}
|
||||
files.MapFS[path].Mode = fs.ModeNamedPipe | 0600
|
||||
requireUnknown(t, files)
|
||||
if files.statusOpens != 0 || (path == fixtureUpdates && files.journalOpens != 0) {
|
||||
t.Fatal("must reject a FIFO using metadata before opening it")
|
||||
}
|
||||
}
|
||||
// fs.Stat would use Open on this implementation, so fail closed instead.
|
||||
requireUnknown(t, openOnlyFS{})
|
||||
}
|
||||
|
||||
type openOnlyFS struct{}
|
||||
|
||||
func (openOnlyFS) Open(string) (fs.File, error) {
|
||||
panic("must not open without safe metadata support")
|
||||
}
|
||||
@@ -0,0 +1,70 @@
|
||||
// Package inspect reports local prerequisite observations without executing
|
||||
// commands, connecting to Docker or altering configuration.
|
||||
package inspect
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"io/fs"
|
||||
"os"
|
||||
"runtime"
|
||||
)
|
||||
|
||||
type Report struct {
|
||||
ProtocolVersion int `json:"protocolVersion"`
|
||||
OS string `json:"os"`
|
||||
Architecture string `json:"architecture"`
|
||||
SupportedPlatform bool `json:"supportedPlatform"`
|
||||
SystemdRuntime string `json:"systemdRuntime"`
|
||||
DockerClient string `json:"dockerClient"`
|
||||
DockerDaemon string `json:"dockerDaemon"`
|
||||
Compose string `json:"compose"`
|
||||
Unchecked []string `json:"unchecked"`
|
||||
DeploymentReady bool `json:"deploymentReady"`
|
||||
}
|
||||
|
||||
func Collect() Report { return Probe(runtime.GOOS, runtime.GOARCH, os.DirFS("/")) }
|
||||
|
||||
// Probe observes filesystem metadata only. "present" is not an assertion that
|
||||
// systemd is responsive or the Docker executable is authentic or operational.
|
||||
func Probe(goos, arch string, files fs.FS) Report {
|
||||
r := Report{ProtocolVersion: 1, OS: goos, Architecture: arch, SupportedPlatform: goos == "linux" && (arch == "amd64" || arch == "arm64"), SystemdRuntime: "not_checked", DockerClient: "not_checked", DockerDaemon: "not_checked", Compose: "not_checked", Unchecked: []string{"docker_daemon", "compose_version", "host_identity", "distribution_support", "permissions", "disk_space", "ports", "dns", "firewall", "gateway"}}
|
||||
if goos != "linux" {
|
||||
return r
|
||||
}
|
||||
r.SystemdRuntime = observe(files, "run/systemd/system", true)
|
||||
r.DockerClient = observe(files, "usr/bin/docker", false)
|
||||
if r.DockerClient != "present" {
|
||||
alternate := observe(files, "usr/local/bin/docker", false)
|
||||
// Either known executable establishes presence. Otherwise retain any
|
||||
// uncertainty instead of reporting absence from an incomplete check.
|
||||
switch {
|
||||
case alternate == "present":
|
||||
r.DockerClient = "present"
|
||||
case r.DockerClient == "unknown" || alternate == "unknown":
|
||||
r.DockerClient = "unknown"
|
||||
case r.DockerClient == "invalid" || alternate == "invalid":
|
||||
r.DockerClient = "invalid"
|
||||
}
|
||||
}
|
||||
return r
|
||||
}
|
||||
|
||||
func observe(files fs.FS, path string, directory bool) string {
|
||||
info, err := fs.Stat(files, path)
|
||||
if errors.Is(err, fs.ErrNotExist) {
|
||||
return "missing"
|
||||
}
|
||||
if err != nil {
|
||||
return "unknown"
|
||||
}
|
||||
if directory {
|
||||
if info.IsDir() {
|
||||
return "present"
|
||||
}
|
||||
return "invalid"
|
||||
}
|
||||
if !info.Mode().IsRegular() || info.Mode().Perm()&0111 == 0 {
|
||||
return "invalid"
|
||||
}
|
||||
return "present"
|
||||
}
|
||||
@@ -0,0 +1,110 @@
|
||||
package inspect
|
||||
|
||||
import (
|
||||
"io/fs"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"testing/fstest"
|
||||
)
|
||||
|
||||
func TestReportsFactsWithoutClaimingDeploymentReadiness(t *testing.T) {
|
||||
files := fstest.MapFS{
|
||||
"run/systemd/system": &fstest.MapFile{Mode: os.ModeDir | 0755},
|
||||
"usr/bin/docker": &fstest.MapFile{Mode: 0755, Data: []byte("not executed")},
|
||||
}
|
||||
r := Probe("linux", "amd64", files)
|
||||
if !r.SupportedPlatform || r.SystemdRuntime != "present" || r.DockerClient != "present" || r.DeploymentReady {
|
||||
t.Fatalf("incorrect report: %+v", r)
|
||||
}
|
||||
if r.DockerDaemon != "not_checked" || r.Compose != "not_checked" {
|
||||
t.Fatal("claimed unperformed checks")
|
||||
}
|
||||
}
|
||||
|
||||
func TestMissingAndUnsupportedEnvironment(t *testing.T) {
|
||||
r := Probe("linux", "amd64", fstest.MapFS{})
|
||||
if r.DockerClient != "missing" || r.SystemdRuntime != "missing" {
|
||||
t.Fatalf("missing prerequisites masked: %+v", r)
|
||||
}
|
||||
r = Probe("windows", "amd64", fstest.MapFS{})
|
||||
if r.SupportedPlatform || r.DeploymentReady || r.DockerClient != "not_checked" {
|
||||
t.Fatal("Windows accepted as deployment target")
|
||||
}
|
||||
r = Probe("linux", "386", fstest.MapFS{})
|
||||
if r.SupportedPlatform {
|
||||
t.Fatal("unsupported architecture accepted")
|
||||
}
|
||||
}
|
||||
|
||||
func TestWrongFileKindsAndPermissions(t *testing.T) {
|
||||
r := Probe("linux", "amd64", fstest.MapFS{
|
||||
"run/systemd/system": &fstest.MapFile{Mode: 0644},
|
||||
"usr/bin/docker": &fstest.MapFile{Mode: 0644},
|
||||
})
|
||||
if r.SystemdRuntime != "invalid" || r.DockerClient != "invalid" {
|
||||
t.Fatalf("wrong file kinds accepted: %+v", r)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRealDirectoryProbeDoesNotWrite(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
if err := os.MkdirAll(filepath.Join(dir, "run/systemd/system"), 0700); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
r := Probe("linux", "arm64", os.DirFS(dir))
|
||||
if r.SystemdRuntime != "present" || r.DockerClient != "missing" {
|
||||
t.Fatalf("bad real probe: %+v", r)
|
||||
}
|
||||
items, err := os.ReadDir(dir)
|
||||
if err != nil || len(items) != 1 || items[0].Name() != "run" {
|
||||
t.Fatal("probe changed filesystem")
|
||||
}
|
||||
}
|
||||
|
||||
type deniedFS struct{}
|
||||
|
||||
func (deniedFS) Open(string) (fs.File, error) { return nil, fs.ErrPermission }
|
||||
|
||||
func TestPermissionErrorsAreUnknownNotMissing(t *testing.T) {
|
||||
r := Probe("linux", "amd64", deniedFS{})
|
||||
if r.DockerClient != "unknown" || r.SystemdRuntime != "unknown" {
|
||||
t.Fatalf("hid inspection failure: %+v", r)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDockerAlternatePath(t *testing.T) {
|
||||
r := Probe("linux", "amd64", fstest.MapFS{"usr/local/bin/docker": &fstest.MapFile{Mode: 0755}})
|
||||
if r.DockerClient != "present" {
|
||||
t.Fatal("alternate installation not found")
|
||||
}
|
||||
}
|
||||
|
||||
type pathDeniedFS struct{ fs.FS }
|
||||
|
||||
func (f pathDeniedFS) Open(name string) (fs.File, error) {
|
||||
if name == "usr/bin/docker" {
|
||||
return nil, fs.ErrPermission
|
||||
}
|
||||
return f.FS.Open(name)
|
||||
}
|
||||
|
||||
func TestAlternateDockerAfterInvalidOrUnknownPrimary(t *testing.T) {
|
||||
for _, mode := range []fs.FileMode{0644, fs.ModeDir | 0755} {
|
||||
files := fstest.MapFS{
|
||||
"usr/bin/docker": &fstest.MapFile{Mode: mode},
|
||||
"usr/local/bin/docker": &fstest.MapFile{Mode: 0755},
|
||||
}
|
||||
if r := Probe("linux", "amd64", files); r.DockerClient != "present" {
|
||||
t.Errorf("valid alternate overlooked after invalid primary: %+v", r)
|
||||
}
|
||||
}
|
||||
files := pathDeniedFS{fstest.MapFS{"usr/local/bin/docker": &fstest.MapFile{Mode: 0755}}}
|
||||
if r := Probe("linux", "amd64", files); r.DockerClient != "present" {
|
||||
t.Errorf("valid alternate overlooked after inaccessible primary: %+v", r)
|
||||
}
|
||||
if r := Probe("linux", "amd64", pathDeniedFS{fstest.MapFS{}}); r.DockerClient != "unknown" {
|
||||
t.Errorf("missing alternate masked inaccessible primary: %+v", r)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,50 @@
|
||||
# Environment lock and draft
|
||||
|
||||
`plan-environment` accepts strict JSON `{ "lock": <Lock> }`, collects local
|
||||
preflight observations itself, and emits a non-executable draft. It does not
|
||||
accept caller-supplied host observations, download anything, run apt, configure
|
||||
sources or start services.
|
||||
|
||||
Lock fields, all required:
|
||||
|
||||
- protocolVersion: 1.
|
||||
- repository: exactly `https://download.docker.com/linux/ubuntu`.
|
||||
- suite: jammy, noble or resolute; architecture: amd64 or arm64.
|
||||
- releaseDigest: `sha256:` plus 64 lowercase hex digits, supplied by the caller.
|
||||
- packages: exactly docker-ce, docker-ce-cli, containerd.io,
|
||||
docker-buildx-plugin and docker-compose-plugin, once each.
|
||||
- Each package has name, version, filename, digest and size. Version is explicit
|
||||
digit-leading Debian-style syntax (optional numeric epoch), at most 128 bytes;
|
||||
digest uses the above SHA-256 format; size is 1 through 512 MiB.
|
||||
- Filename must equal
|
||||
`dists/<suite>/pool/stable/<architecture>/<name>_<version-without-epoch>_<architecture>.deb`.
|
||||
Encoded paths, absolute paths, alternate domains, query strings and traversal
|
||||
are not accepted. docker-ce and docker-ce-cli must use the same version.
|
||||
|
||||
This is a deliberately limited Docker lock format, not a complete Debian version
|
||||
parser. A Linux host's observed distribution/suite and architecture must match;
|
||||
unknown observations remain blockers. On a non-Linux machine a syntactically
|
||||
valid lock can be reviewed, but unsupported-platform blockers remain.
|
||||
|
||||
## Digests and remaining trust boundary
|
||||
|
||||
lockDigest binds Go JSON encoding of the validated Lock in struct/array order.
|
||||
observationDigest binds Go JSON encoding of the collected Report. These are
|
||||
content hashes, not signatures, stable host IDs, freshness tokens or authorization.
|
||||
The local observation is not atomic and changes (including free disk space) can
|
||||
change its hash. No writing consumer may treat it as an approved plan.
|
||||
|
||||
The draft returns requestedPackages, not a complete APT dependency transaction.
|
||||
It never proposes automatic removal or upgrade of existing installations.
|
||||
RepositoryAuthenticated and executable are always false. There is no signature
|
||||
verification, Release-to-Packages-to-deb digest chain verification, metadata
|
||||
freshness policy, artifact download, package dependency resolution, or installation
|
||||
executor yet. Matching a URL allowlist and a caller-provided hash proves none of
|
||||
those. No actual versions are recommended or locked from live metadata in this batch.
|
||||
|
||||
Blockers explicitly retain these gaps along with host/network/runtime checks.
|
||||
Potential APT database changes, dependency changes, service starts and network
|
||||
rule effects are reported. Exit 0 only means a draft was produced.
|
||||
|
||||
Reference for package names and repository layout:
|
||||
[Docker Ubuntu installation](https://docs.docker.com/engine/install/ubuntu/).
|
||||
@@ -0,0 +1,68 @@
|
||||
package installplan
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"regexp"
|
||||
"strings"
|
||||
)
|
||||
|
||||
type Lock struct {
|
||||
ProtocolVersion int `json:"protocolVersion"`
|
||||
Repository string `json:"repository"`
|
||||
Suite string `json:"suite"`
|
||||
Architecture string `json:"architecture"`
|
||||
ReleaseDigest string `json:"releaseDigest"`
|
||||
Packages []Package `json:"packages"`
|
||||
}
|
||||
type Package struct {
|
||||
Name string `json:"name"`
|
||||
Version string `json:"version"`
|
||||
Filename string `json:"filename"`
|
||||
Digest string `json:"digest"`
|
||||
Size uint64 `json:"size"`
|
||||
}
|
||||
|
||||
var versionPattern = regexp.MustCompile(`^(?:[0-9]+:)?[0-9][0-9A-Za-z.+~-]*$`)
|
||||
var digestPattern = regexp.MustCompile(`^sha256:[0-9a-f]{64}$`)
|
||||
var required = []string{"docker-ce", "docker-ce-cli", "containerd.io", "docker-buildx-plugin", "docker-compose-plugin"}
|
||||
|
||||
// Validate checks a caller-supplied lock against a fixed source policy. This is
|
||||
// NOT repository signature validation or evidence these artifacts exist.
|
||||
func Validate(lock Lock, suite, architecture string) (string, error) {
|
||||
reject := errors.New("invalid Docker package lock")
|
||||
if lock.ProtocolVersion != 1 || lock.Repository != "https://download.docker.com/linux/ubuntu" || lock.Suite != suite || lock.Architecture != architecture {
|
||||
return "", reject
|
||||
}
|
||||
if (suite != "jammy" && suite != "noble" && suite != "resolute") || (architecture != "amd64" && architecture != "arm64") || !digestPattern.MatchString(lock.ReleaseDigest) || len(lock.Packages) != len(required) {
|
||||
return "", reject
|
||||
}
|
||||
versions := map[string]string{}
|
||||
for _, p := range lock.Packages {
|
||||
allowed := false
|
||||
for _, name := range required {
|
||||
if p.Name == name {
|
||||
allowed = true
|
||||
}
|
||||
}
|
||||
if !allowed || versions[p.Name] != "" || len(p.Version) > 128 || !versionPattern.MatchString(p.Version) || !digestPattern.MatchString(p.Digest) || p.Size == 0 || p.Size > 512<<20 {
|
||||
return "", reject
|
||||
}
|
||||
fileVersion := p.Version
|
||||
if _, after, ok := strings.Cut(fileVersion, ":"); ok {
|
||||
fileVersion = after
|
||||
}
|
||||
if p.Filename != "dists/"+suite+"/pool/stable/"+architecture+"/"+p.Name+"_"+fileVersion+"_"+architecture+".deb" {
|
||||
return "", reject
|
||||
}
|
||||
versions[p.Name] = p.Version
|
||||
}
|
||||
if versions["docker-ce"] != versions["docker-ce-cli"] {
|
||||
return "", reject
|
||||
}
|
||||
raw, _ := json.Marshal(lock)
|
||||
sum := sha256.Sum256(raw)
|
||||
return "sha256:" + hex.EncodeToString(sum[:]), nil
|
||||
}
|
||||
@@ -0,0 +1,79 @@
|
||||
package installplan
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func lockFixture() Lock {
|
||||
l := Lock{ProtocolVersion: 1, Repository: "https://download.docker.com/linux/ubuntu", Suite: "resolute", Architecture: "amd64", ReleaseDigest: "sha256:" + strings.Repeat("b", 64), Packages: []Package{}}
|
||||
for _, name := range []string{"docker-ce", "docker-ce-cli", "containerd.io", "docker-buildx-plugin", "docker-compose-plugin"} {
|
||||
l.Packages = append(l.Packages, Package{Name: name, Version: "1.2.3-1", Filename: "dists/resolute/pool/stable/amd64/" + name + "_1.2.3-1_amd64.deb", Digest: "sha256:" + strings.Repeat("a", 64), Size: 123})
|
||||
}
|
||||
return l
|
||||
}
|
||||
func TestValidLockAndStableDigest(t *testing.T) {
|
||||
l := lockFixture()
|
||||
digest, err := Validate(l, "resolute", "amd64")
|
||||
if err != nil || !strings.HasPrefix(digest, "sha256:") {
|
||||
t.Fatal("valid lock rejected", err)
|
||||
}
|
||||
l.Packages[0].Size++
|
||||
changed, err := Validate(l, "resolute", "amd64")
|
||||
if err != nil || changed == digest {
|
||||
t.Fatal("lock digest does not bind size")
|
||||
}
|
||||
}
|
||||
func TestRejectUnsafeLock(t *testing.T) {
|
||||
for name, mutate := range map[string]func(*Lock){
|
||||
"protocol": func(l *Lock) { l.ProtocolVersion = 2 },
|
||||
"repository": func(l *Lock) { l.Repository = "https://attacker.example/linux/ubuntu" },
|
||||
"credentials": func(l *Lock) { l.Repository = "https://user:secret@download.docker.com/linux/ubuntu" },
|
||||
"suite": func(l *Lock) { l.Suite = "noble" },
|
||||
"architecture": func(l *Lock) { l.Architecture = "arm64" },
|
||||
"release": func(l *Lock) { l.ReleaseDigest = "" },
|
||||
"missing": func(l *Lock) { l.Packages = l.Packages[:4] },
|
||||
"extra": func(l *Lock) { l.Packages = append(l.Packages, l.Packages[0]) },
|
||||
"duplicate": func(l *Lock) { l.Packages[1] = l.Packages[0] },
|
||||
"latest": func(l *Lock) { l.Packages[0].Version = "latest" },
|
||||
"shell": func(l *Lock) { l.Packages[0].Version = "1;reboot" },
|
||||
"wrong file": func(l *Lock) { l.Packages[0].Filename = "dists/resolute/pool/stable/amd64/other.deb" },
|
||||
"traversal": func(l *Lock) { l.Packages[0].Filename = "../docker.deb" },
|
||||
"encoded path": func(l *Lock) { l.Packages[0].Filename = "dists/resolute/pool/stable/amd64/%2e%2e.deb" },
|
||||
"digest": func(l *Lock) { l.Packages[0].Digest = "bad" },
|
||||
"size": func(l *Lock) { l.Packages[0].Size = 0 },
|
||||
"engine mismatch": func(l *Lock) {
|
||||
l.Packages[1].Version = "2.3.4-1"
|
||||
l.Packages[1].Filename = "dists/resolute/pool/stable/amd64/docker-ce-cli_2.3.4-1_amd64.deb"
|
||||
},
|
||||
} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
l := lockFixture()
|
||||
mutate(&l)
|
||||
if _, err := Validate(l, "resolute", "amd64"); err == nil {
|
||||
t.Fatal("unsafe lock accepted")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestDockerEpochAndTargetArchitecture(t *testing.T) {
|
||||
l := lockFixture()
|
||||
for i := range l.Packages {
|
||||
p := &l.Packages[i]
|
||||
if p.Name == "docker-ce" || p.Name == "docker-ce-cli" {
|
||||
p.Version = "5:29.1.0-1~ubuntu.26.04~resolute"
|
||||
p.Filename = "dists/resolute/pool/stable/amd64/" + p.Name + "_29.1.0-1~ubuntu.26.04~resolute_amd64.deb"
|
||||
}
|
||||
}
|
||||
if _, err := Validate(l, "resolute", "amd64"); err != nil {
|
||||
t.Fatal("epoch version rejected", err)
|
||||
}
|
||||
if _, err := Validate(l, "resolute", "arm64"); err == nil {
|
||||
t.Fatal("architecture mismatch accepted")
|
||||
}
|
||||
l.Packages[0].Filename = strings.Replace(l.Packages[0].Filename, "_29.", "_5:29.", 1)
|
||||
if _, err := Validate(l, "resolute", "amd64"); err == nil {
|
||||
t.Fatal("epoch in repository filename accepted")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,42 @@
|
||||
package installplan
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"server-deploy/internal/preflight"
|
||||
)
|
||||
|
||||
type Draft struct {
|
||||
Executable bool `json:"executable"`
|
||||
RepositoryAuthenticated bool `json:"repositoryAuthenticated"`
|
||||
LockDigest string `json:"lockDigest"`
|
||||
ObservationDigest string `json:"observationDigest"`
|
||||
Blockers []string `json:"blockers"`
|
||||
RequestedPackages []Package `json:"requestedPackages"`
|
||||
Impacts []string `json:"impacts"`
|
||||
}
|
||||
|
||||
// Build binds requested package pins to a local observation for review only.
|
||||
// Neither digest is a signature, host identity, freshness token or approval.
|
||||
func Build(report preflight.Report, lock Lock) (Draft, error) {
|
||||
digest, err := Validate(lock, lock.Suite, lock.Architecture)
|
||||
if err != nil {
|
||||
return Draft{}, err
|
||||
}
|
||||
if report.Runtime.OS == "linux" && (report.Runtime.Architecture != lock.Architecture || (report.Distribution.State == "observed" && (report.Distribution.ID != "ubuntu" || report.Distribution.Codename != lock.Suite))) {
|
||||
return Draft{}, errors.New("lock does not match local platform")
|
||||
}
|
||||
proposal := preflight.Plan(report)
|
||||
blockers := []string{}
|
||||
for _, b := range proposal.Blockers {
|
||||
if b != "package_versions_unresolved" {
|
||||
blockers = append(blockers, b)
|
||||
}
|
||||
}
|
||||
blockers = append(blockers, "dependency_transaction_unresolved", "artifact_bytes_unverified", "repository_metadata_freshness_unverified")
|
||||
raw, _ := json.Marshal(report)
|
||||
sum := sha256.Sum256(raw)
|
||||
return Draft{LockDigest: digest, ObservationDigest: "sha256:" + hex.EncodeToString(sum[:]), Blockers: blockers, RequestedPackages: append([]Package{}, lock.Packages...), Impacts: []string{"package_database_and_repository_changes", "services_may_start_during_package_install", "host_network_rules_may_change", "additional_dependencies_not_yet_resolved"}}, nil
|
||||
}
|
||||
@@ -0,0 +1,39 @@
|
||||
package installplan
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"server-deploy/internal/inspect"
|
||||
"server-deploy/internal/preflight"
|
||||
)
|
||||
|
||||
func TestDraftPreservesSafetyBlockersAndBindsInputs(t *testing.T) {
|
||||
r := preflight.Report{Runtime: inspect.Report{OS: "linux", Architecture: "amd64"}, Distribution: preflight.Distribution{State: "observed", ID: "ubuntu", Version: "26.04", Codename: "resolute"}}
|
||||
d, err := Build(r, lockFixture())
|
||||
if err != nil || d.Executable || d.RepositoryAuthenticated || len(d.Blockers) == 0 || !strings.HasPrefix(d.LockDigest, "sha256:") || !strings.HasPrefix(d.ObservationDigest, "sha256:") {
|
||||
t.Fatalf("unsafe draft %+v %v", d, err)
|
||||
}
|
||||
for _, want := range []string{"repository_trust_unverified", "dependency_transaction_unresolved", "artifact_bytes_unverified"} {
|
||||
found := false
|
||||
for _, b := range d.Blockers {
|
||||
if b == want {
|
||||
found = true
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
t.Fatalf("missing blocker %s", want)
|
||||
}
|
||||
}
|
||||
r.Privilege = "non_root"
|
||||
d2, _ := Build(r, lockFixture())
|
||||
if d2.ObservationDigest == d.ObservationDigest {
|
||||
t.Fatal("report change not bound")
|
||||
}
|
||||
l := lockFixture()
|
||||
r.Distribution.Version = "24.04"
|
||||
r.Distribution.Codename = "noble"
|
||||
if _, err := Build(r, l); err == nil {
|
||||
t.Fatal("host and lock mismatch accepted")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,56 @@
|
||||
// Package planner builds offline previews. It does not inspect or change a host.
|
||||
package planner
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"net"
|
||||
"regexp"
|
||||
"strings"
|
||||
)
|
||||
|
||||
const ProtocolVersion = 1
|
||||
|
||||
var (
|
||||
idPattern = regexp.MustCompile(`^[a-z][a-z0-9-]{0,47}$`)
|
||||
digestPattern = regexp.MustCompile(`^sha256:[a-f0-9]{64}$`)
|
||||
labelPattern = regexp.MustCompile(`^[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?$`)
|
||||
)
|
||||
|
||||
// Intent contains no secrets. ObservedStateDigest is caller-supplied in offline
|
||||
// mode; a future executor must obtain it independently from the target host.
|
||||
type Intent struct {
|
||||
ProtocolVersion int `json:"protocolVersion"`
|
||||
HostID string `json:"hostId"`
|
||||
InstanceID string `json:"instanceId"`
|
||||
AppID string `json:"appId"`
|
||||
PackageDigest string `json:"packageDigest"`
|
||||
ImageDigest string `json:"imageDigest"`
|
||||
Domain string `json:"domain"`
|
||||
ObservedStateDigest string `json:"observedStateDigest"`
|
||||
}
|
||||
|
||||
func (i Intent) Validate() error {
|
||||
if i.ProtocolVersion != ProtocolVersion {
|
||||
return errors.New("unsupported protocol version")
|
||||
}
|
||||
for _, id := range []string{i.HostID, i.InstanceID, i.AppID} {
|
||||
if !idPattern.MatchString(id) {
|
||||
return errors.New("invalid resource identifier")
|
||||
}
|
||||
}
|
||||
for _, digest := range []string{i.PackageDigest, i.ImageDigest, i.ObservedStateDigest} {
|
||||
if !digestPattern.MatchString(digest) {
|
||||
return errors.New("expected a SHA-256 digest")
|
||||
}
|
||||
}
|
||||
labels := strings.Split(i.Domain, ".")
|
||||
if len(i.Domain) > 253 || len(labels) < 2 || net.ParseIP(i.Domain) != nil {
|
||||
return errors.New("expected an ASCII DNS hostname")
|
||||
}
|
||||
for _, label := range labels {
|
||||
if !labelPattern.MatchString(label) {
|
||||
return errors.New("invalid DNS hostname label")
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,44 @@
|
||||
package planner
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func validIntent() Intent {
|
||||
return Intent{ProtocolVersion: 1, HostID: "host-one", InstanceID: "git-one", AppID: "gitea", PackageDigest: "sha256:" + strings.Repeat("a", 64), ImageDigest: "sha256:" + strings.Repeat("b", 64), Domain: "git.example.com", ObservedStateDigest: "sha256:" + strings.Repeat("c", 64)}
|
||||
}
|
||||
|
||||
func TestIntentValidation(t *testing.T) {
|
||||
if err := validIntent().Validate(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cases := []struct {
|
||||
name string
|
||||
change func(*Intent)
|
||||
}{
|
||||
{"protocol", func(i *Intent) { i.ProtocolVersion = 2 }},
|
||||
{"host empty", func(i *Intent) { i.HostID = "" }},
|
||||
{"path escape", func(i *Intent) { i.InstanceID = "../git" }},
|
||||
{"shell", func(i *Intent) { i.AppID = "git;id" }},
|
||||
{"long id", func(i *Intent) { i.InstanceID = strings.Repeat("a", 49) }},
|
||||
{"floating tag", func(i *Intent) { i.ImageDigest = "gitea:latest" }},
|
||||
{"package hash", func(i *Intent) { i.PackageDigest = "sha256:xyz" }},
|
||||
{"state missing", func(i *Intent) { i.ObservedStateDigest = "" }},
|
||||
{"url", func(i *Intent) { i.Domain = "https://git.example.com" }},
|
||||
{"wildcard", func(i *Intent) { i.Domain = "*.example.com" }},
|
||||
{"label", func(i *Intent) { i.Domain = "-git.example.com" }},
|
||||
{"empty label", func(i *Intent) { i.Domain = "git..com" }},
|
||||
{"uppercase", func(i *Intent) { i.Domain = "Git.example.com" }},
|
||||
{"ip", func(i *Intent) { i.Domain = "127.0.0.1" }},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
i := validIntent()
|
||||
tc.change(&i)
|
||||
if i.Validate() == nil {
|
||||
t.Fatal("accepted invalid intent")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,56 @@
|
||||
package planner
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"time"
|
||||
)
|
||||
|
||||
const planLifetime = 15 * time.Minute
|
||||
|
||||
// Plan is an offline intent preview, not an executable authorization. Hash binds
|
||||
// its contents but is not a signature. No secret may be added to this structure.
|
||||
type Plan struct {
|
||||
Intent Intent `json:"intent"`
|
||||
ProjectName string `json:"projectName"`
|
||||
DataPath string `json:"dataPath"`
|
||||
CreatedAt time.Time `json:"createdAt"`
|
||||
ExpiresAt time.Time `json:"expiresAt"`
|
||||
Hash string `json:"hash"`
|
||||
}
|
||||
|
||||
func Build(i Intent, now time.Time) (Plan, error) {
|
||||
if err := i.Validate(); err != nil {
|
||||
return Plan{}, err
|
||||
}
|
||||
now = now.UTC().Truncate(time.Second)
|
||||
p := Plan{Intent: i, ProjectName: "sd-" + i.InstanceID, DataPath: "/var/lib/server-deploy/instances/" + i.InstanceID, CreatedAt: now, ExpiresAt: now.Add(planLifetime)}
|
||||
encoded, err := json.Marshal(p)
|
||||
if err != nil {
|
||||
return Plan{}, errors.New("cannot encode plan")
|
||||
}
|
||||
sum := sha256.Sum256(encoded)
|
||||
p.Hash = "sha256:" + hex.EncodeToString(sum[:])
|
||||
return p, nil
|
||||
}
|
||||
|
||||
// Verify checks offline consistency only. Current must be independently inspected
|
||||
// under a host lock before any future write operation uses this comparison.
|
||||
func (p Plan) Verify(current Intent, now time.Time) error {
|
||||
if now.Before(p.CreatedAt) || !now.Before(p.ExpiresAt) {
|
||||
return errors.New("plan is not within its validity window")
|
||||
}
|
||||
if p.Intent != current {
|
||||
return errors.New("plan does not match current intent or observed state")
|
||||
}
|
||||
expected, err := Build(current, p.CreatedAt)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if p.ProjectName != expected.ProjectName || p.DataPath != expected.DataPath || !p.CreatedAt.Equal(expected.CreatedAt) || !p.ExpiresAt.Equal(expected.ExpiresAt) || p.Hash != expected.Hash {
|
||||
return errors.New("plan integrity check failed")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,71 @@
|
||||
package planner
|
||||
|
||||
import (
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestBuildPlan(t *testing.T) {
|
||||
now := time.Date(2026, 9, 25, 12, 0, 0, 0, time.UTC)
|
||||
i := validIntent()
|
||||
p, err := Build(i, now)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if p.ProjectName != "sd-git-one" || p.DataPath != "/var/lib/server-deploy/instances/git-one" {
|
||||
t.Fatalf("wrong instance resources: %+v", p)
|
||||
}
|
||||
if p.ExpiresAt.Sub(p.CreatedAt) != 15*time.Minute {
|
||||
t.Fatal("wrong expiration")
|
||||
}
|
||||
if err := p.Verify(i, now); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
again, _ := Build(i, now)
|
||||
if again.Hash != p.Hash {
|
||||
t.Fatal("unstable plan hash")
|
||||
}
|
||||
i.InstanceID = "git-two"
|
||||
other, _ := Build(i, now)
|
||||
if other.Hash == p.Hash || other.ProjectName == p.ProjectName || other.DataPath == p.DataPath {
|
||||
t.Fatal("instances share identity")
|
||||
}
|
||||
}
|
||||
|
||||
func TestVerifyRejectsChangedOrExpiredPlan(t *testing.T) {
|
||||
now := time.Date(2026, 9, 25, 12, 0, 0, 0, time.UTC)
|
||||
i := validIntent()
|
||||
original, _ := Build(i, now)
|
||||
cases := []struct {
|
||||
name string
|
||||
change func(*Plan, *Intent, *time.Time)
|
||||
}{
|
||||
{"expired", func(p *Plan, i *Intent, n *time.Time) { *n = p.ExpiresAt }},
|
||||
{"future", func(p *Plan, i *Intent, n *time.Time) { *n = p.CreatedAt.Add(-time.Second) }},
|
||||
{"tampered hash", func(p *Plan, i *Intent, n *time.Time) { p.Hash = "bad" }},
|
||||
{"tampered path", func(p *Plan, i *Intent, n *time.Time) { p.DataPath = "/" }},
|
||||
{"tampered project", func(p *Plan, i *Intent, n *time.Time) { p.ProjectName = "other" }},
|
||||
{"tampered expiry", func(p *Plan, i *Intent, n *time.Time) { p.ExpiresAt = p.ExpiresAt.Add(time.Hour) }},
|
||||
{"state drift", func(p *Plan, i *Intent, n *time.Time) { i.ObservedStateDigest = i.PackageDigest }},
|
||||
{"host drift", func(p *Plan, i *Intent, n *time.Time) { i.HostID = "another-host" }},
|
||||
{"domain drift", func(p *Plan, i *Intent, n *time.Time) { i.Domain = "other.example.com" }},
|
||||
{"intent tamper", func(p *Plan, i *Intent, n *time.Time) { p.Intent.InstanceID = "other" }},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
p, current, at := original, i, now
|
||||
tc.change(&p, ¤t, &at)
|
||||
if p.Verify(current, at) == nil {
|
||||
t.Fatal("accepted invalid plan")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildRejectsInvalidIntent(t *testing.T) {
|
||||
i := validIntent()
|
||||
i.InstanceID = "../bad"
|
||||
if _, err := Build(i, time.Now()); err == nil {
|
||||
t.Fatal("built invalid plan")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,57 @@
|
||||
# Read-only local environment proposal
|
||||
|
||||
`deployctl preflight` takes no stdin request. It collects facts from the local
|
||||
machine and emits protocolVersion, mode=`local-environment-proposal`, observedAt,
|
||||
report and proposal. It never connects to SSH/Docker, runs package managers,
|
||||
sources shell files, writes configuration or starts/restarts services.
|
||||
|
||||
## Observations
|
||||
|
||||
- Reuses `inspect` for OS, architecture and systemd/Docker-client file presence.
|
||||
- Reads only ID, VERSION_ID and VERSION_CODENAME from `/etc/os-release`, at most
|
||||
64 KiB. Handles plain and simply quoted values, not a shell grammar. Ambiguous,
|
||||
duplicated, missing or unsupported value syntax fails closed. Other keys are
|
||||
ignored, not evaluated or returned. Trusted host files/ancestors are assumed.
|
||||
- Linux effective UID is classified root/non_root/unknown.
|
||||
- Linux statfs reports available bytes on the filesystem containing `/var/lib`.
|
||||
This does not measure another configured data root, quotas, or inode capacity.
|
||||
Unknown disk observations cannot authorize a fresh-install candidate.
|
||||
- Resource checks inspect directory entries for `/var/lib/docker`,
|
||||
`/var/lib/containerd`, `/etc/docker`, `/var/lib/server-deploy`, and the Docker
|
||||
`.sources`/`.list` paths under `/etc/apt/sources.list.d`. Contents are not read.
|
||||
Any link or non-directory intermediate component is treated as existing;
|
||||
access failures become unknown, not absent. Checks are conservative hints,
|
||||
not a complete scan of runtime installations or APT sources.
|
||||
- Non-Linux hosts do not read Linux paths or report Linux disk availability.
|
||||
- Reads a bounded local dpkg status snapshot and requires an empty update journal.
|
||||
Reports only the Docker/runtime-related package records and the complete status
|
||||
file digest. Missing/malformed/journal-busy input is unknown, not an empty host.
|
||||
Installed, held, partial and residual relevant records all require manual review.
|
||||
It does not audit unrelated dependency health or non-dpkg installations.
|
||||
|
||||
## Candidate policy
|
||||
|
||||
Linux amd64/arm64; Ubuntu version/codename pairs 22.04/jammy, 24.04/noble,
|
||||
26.04/resolute; root; systemd path present; at least 5 GiB available on /var/lib;
|
||||
Docker client absent; all listed resource paths observed absent. The 5 GiB floor
|
||||
is only a bootstrap screening threshold, not a calculated application/image/backup
|
||||
capacity requirement. Docker official Ubuntu support was checked at implementation:
|
||||
[Docker installation requirements](https://docs.docker.com/engine/install/ubuntu/).
|
||||
This project has not certified these distributions with actual installation tests.
|
||||
|
||||
If all these observations pass, proposal lists candidate source configuration,
|
||||
version-locked package installation and engine/Compose verification steps, plus
|
||||
APT/disk/service-start/firewall impacts. It does not produce shell commands or
|
||||
claim those steps can yet execute. Existing resources cause manual-review blockers;
|
||||
there is no automatic removal, adoption, migration or package conflict cleanup.
|
||||
|
||||
Every proposal remains `executable=false`, with blockers for unverified host
|
||||
identity, unknown package inventory, unresolved versions, repository trust, network/firewall
|
||||
and the unimplemented installer. Empty candidate steps mean preliminary host
|
||||
observations also failed. Nonempty steps are NOT an approved install transaction.
|
||||
|
||||
Reports describe this process's environment (possibly a container/WSL instance),
|
||||
not necessarily the intended cloud server. No snapshot hash, SSH identity binding,
|
||||
freshness token or lock-based revalidation exists yet. These must be implemented
|
||||
before any future write path consumes observations. Exit 0 means a report was
|
||||
produced; inspect `proposal.blockers`, never exit status alone, for readiness.
|
||||
@@ -0,0 +1,22 @@
|
||||
//go:build linux
|
||||
|
||||
package preflight
|
||||
|
||||
import (
|
||||
"os"
|
||||
"syscall"
|
||||
|
||||
"server-deploy/internal/inspect"
|
||||
)
|
||||
|
||||
func Collect() Report {
|
||||
return Probe(inspect.Collect(), os.DirFS("/"), os.Geteuid(), diskAvailable("/var/lib"))
|
||||
}
|
||||
|
||||
func diskAvailable(path string) Disk {
|
||||
var stat syscall.Statfs_t
|
||||
if syscall.Statfs(path, &stat) != nil || stat.Bsize <= 0 || stat.Bavail > ^uint64(0)/uint64(stat.Bsize) {
|
||||
return Disk{State: "unknown"}
|
||||
}
|
||||
return Disk{State: "observed", AvailableBytes: stat.Bavail * uint64(stat.Bsize)}
|
||||
}
|
||||
@@ -0,0 +1,31 @@
|
||||
//go:build linux
|
||||
|
||||
package preflight
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestRealDiskObservation(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
if diskAvailable(dir).State != "observed" {
|
||||
t.Fatal("existing filesystem not observed")
|
||||
}
|
||||
if diskAvailable(filepath.Join(dir, "missing")).State != "unknown" {
|
||||
t.Fatal("missing target reported capacity")
|
||||
}
|
||||
}
|
||||
func TestDanglingResourceLinkRequiresReview(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
if err := os.MkdirAll(filepath.Join(dir, "var/lib"), 0700); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.Symlink("missing", filepath.Join(dir, "var/lib/docker")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if resourceState(os.DirFS(dir), "var/lib/docker") != "present" {
|
||||
t.Fatal("dangling link treated as absent")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
//go:build !linux
|
||||
|
||||
package preflight
|
||||
|
||||
import "server-deploy/internal/inspect"
|
||||
|
||||
func Collect() Report { return Probe(inspect.Collect(), nil, -1, Disk{State: "not_checked"}) }
|
||||
@@ -0,0 +1,208 @@
|
||||
package preflight
|
||||
|
||||
import (
|
||||
"io"
|
||||
"io/fs"
|
||||
"regexp"
|
||||
"server-deploy/internal/debian"
|
||||
"server-deploy/internal/inspect"
|
||||
"strings"
|
||||
)
|
||||
|
||||
type Disk struct {
|
||||
State string `json:"state"`
|
||||
AvailableBytes uint64 `json:"availableBytes"`
|
||||
}
|
||||
type Distribution struct {
|
||||
State string `json:"state"`
|
||||
ID string `json:"id"`
|
||||
Version string `json:"version"`
|
||||
Codename string `json:"codename"`
|
||||
}
|
||||
type Resource struct {
|
||||
Path string `json:"path"`
|
||||
State string `json:"state"`
|
||||
}
|
||||
type Report struct {
|
||||
Runtime inspect.Report `json:"runtime"`
|
||||
Distribution Distribution `json:"distribution"`
|
||||
Privilege string `json:"privilege"`
|
||||
Disk Disk `json:"disk"`
|
||||
Resources []Resource `json:"resources"`
|
||||
Inventory debian.Snapshot `json:"inventory"`
|
||||
}
|
||||
type Proposal struct {
|
||||
Executable bool `json:"executable"`
|
||||
Blockers []string `json:"blockers"`
|
||||
ProposedChanges []string `json:"proposedChanges"`
|
||||
Impacts []string `json:"impacts"`
|
||||
}
|
||||
|
||||
var resourcePaths = []string{"var/lib/docker", "var/lib/containerd", "etc/docker", "var/lib/server-deploy", "etc/apt/sources.list.d/docker.sources", "etc/apt/sources.list.d/docker.list"}
|
||||
|
||||
// Probe reads metadata and a bounded os-release file. It never sources shell
|
||||
// files or invokes executables. Disk describes /var/lib, not an arbitrary target.
|
||||
func Probe(runtime inspect.Report, files fs.FS, uid int, disk Disk) Report {
|
||||
r := Report{Runtime: runtime, Distribution: Distribution{State: "not_checked"}, Privilege: "not_checked", Disk: Disk{State: "not_checked"}, Resources: []Resource{}, Inventory: debian.Snapshot{State: "not_checked", Packages: []debian.Installed{}}}
|
||||
if runtime.OS != "linux" {
|
||||
return r
|
||||
}
|
||||
r.Disk = disk
|
||||
r.Privilege = "non_root"
|
||||
if uid == 0 {
|
||||
r.Privilege = "root"
|
||||
} else if uid < 0 {
|
||||
r.Privilege = "unknown"
|
||||
}
|
||||
r.Distribution = readDistribution(files)
|
||||
r.Inventory = debian.Inventory(files)
|
||||
for _, p := range resourcePaths {
|
||||
r.Resources = append(r.Resources, Resource{Path: "/" + p, State: resourceState(files, p)})
|
||||
}
|
||||
return r
|
||||
}
|
||||
|
||||
// Plan is a proposal, not an executable or approved installation plan. Missing
|
||||
// package inventory/version locks and host identity always block execution.
|
||||
func Plan(r Report) Proposal {
|
||||
p := Proposal{Blockers: []string{}, ProposedChanges: []string{}, Impacts: []string{}}
|
||||
if r.Runtime.OS != "linux" || (r.Runtime.Architecture != "amd64" && r.Runtime.Architecture != "arm64") {
|
||||
p.Blockers = append(p.Blockers, "unsupported_platform")
|
||||
}
|
||||
if r.Distribution.State != "observed" {
|
||||
p.Blockers = append(p.Blockers, "distribution_unverified")
|
||||
} else if r.Distribution.ID != "ubuntu" || !supportedSuite(r.Distribution) {
|
||||
p.Blockers = append(p.Blockers, "unsupported_distribution")
|
||||
}
|
||||
if r.Privilege != "root" {
|
||||
p.Blockers = append(p.Blockers, "root_required")
|
||||
}
|
||||
if r.Runtime.SystemdRuntime != "present" {
|
||||
p.Blockers = append(p.Blockers, "systemd_unverified")
|
||||
}
|
||||
if r.Disk.State != "observed" {
|
||||
p.Blockers = append(p.Blockers, "disk_unverified")
|
||||
} else if r.Disk.AvailableBytes < 5<<30 {
|
||||
p.Blockers = append(p.Blockers, "disk_below_bootstrap_floor")
|
||||
}
|
||||
if r.Runtime.DockerClient != "missing" {
|
||||
p.Blockers = append(p.Blockers, "existing_or_unknown_runtime_requires_review")
|
||||
}
|
||||
// Validate the complete path set as well: an incomplete report is not clean.
|
||||
seen := make(map[string]bool)
|
||||
resourcesClean := len(r.Resources) == len(resourcePaths)
|
||||
for _, v := range r.Resources {
|
||||
if v.State != "missing" || seen[v.Path] {
|
||||
resourcesClean = false
|
||||
}
|
||||
seen[v.Path] = true
|
||||
}
|
||||
for _, path := range resourcePaths {
|
||||
if !seen["/"+path] {
|
||||
resourcesClean = false
|
||||
}
|
||||
}
|
||||
if !resourcesClean {
|
||||
p.Blockers = append(p.Blockers, "existing_resources_require_review")
|
||||
}
|
||||
if r.Inventory.State != "observed" {
|
||||
p.Blockers = append(p.Blockers, "package_inventory_unverified")
|
||||
} else if len(r.Inventory.Packages) > 0 {
|
||||
p.Blockers = append(p.Blockers, "existing_packages_require_review")
|
||||
}
|
||||
if len(p.Blockers) == 0 {
|
||||
p.ProposedChanges = []string{"configure_verified_docker_apt_source", "install_version_locked_docker_packages", "verify_local_engine_and_compose"}
|
||||
p.Impacts = []string{"apt_configuration_and_package_database_changes", "docker_service_may_start_during_install", "docker_may_change_host_network_firewall_rules", "system_disk_usage_increases"}
|
||||
}
|
||||
p.Blockers = append(p.Blockers, "host_identity_unverified", "package_versions_unresolved", "repository_trust_unverified", "network_and_firewall_unverified", "installation_executor_unimplemented")
|
||||
return p
|
||||
}
|
||||
|
||||
func supportedSuite(d Distribution) bool {
|
||||
return map[string]string{"22.04": "jammy", "24.04": "noble", "26.04": "resolute"}[d.Version] == d.Codename && d.Codename != ""
|
||||
}
|
||||
|
||||
var releaseValue = regexp.MustCompile(`^[a-z0-9][a-z0-9._-]{0,63}$`)
|
||||
|
||||
func readDistribution(files fs.FS) Distribution {
|
||||
initial, err := fs.Stat(files, "etc/os-release")
|
||||
if err != nil {
|
||||
return Distribution{State: "unknown"}
|
||||
}
|
||||
if !initial.Mode().IsRegular() || initial.Size() > 65536 {
|
||||
return Distribution{State: "invalid"}
|
||||
}
|
||||
f, err := files.Open("etc/os-release")
|
||||
if err != nil {
|
||||
return Distribution{State: "unknown"}
|
||||
}
|
||||
defer f.Close()
|
||||
info, err := f.Stat()
|
||||
if err != nil {
|
||||
return Distribution{State: "unknown"}
|
||||
}
|
||||
if !info.Mode().IsRegular() || info.Size() > 65536 {
|
||||
return Distribution{State: "invalid"}
|
||||
}
|
||||
raw, err := io.ReadAll(io.LimitReader(f, 65537))
|
||||
if err != nil {
|
||||
return Distribution{State: "unknown"}
|
||||
}
|
||||
if len(raw) > 65536 {
|
||||
return Distribution{State: "invalid"}
|
||||
}
|
||||
values := map[string]string{}
|
||||
for _, line := range strings.Split(string(raw), "\n") {
|
||||
key, value, ok := strings.Cut(strings.TrimSpace(line), "=")
|
||||
if key != "ID" && key != "VERSION_ID" && key != "VERSION_CODENAME" {
|
||||
continue
|
||||
}
|
||||
if !ok || values[key] != "" {
|
||||
return Distribution{State: "invalid"}
|
||||
}
|
||||
if len(value) >= 2 && ((value[0] == '"' && value[len(value)-1] == '"') || (value[0] == '\'' && value[len(value)-1] == '\'')) {
|
||||
value = value[1 : len(value)-1]
|
||||
}
|
||||
if !releaseValue.MatchString(value) {
|
||||
return Distribution{State: "invalid"}
|
||||
}
|
||||
values[key] = value
|
||||
}
|
||||
if len(values) != 3 {
|
||||
return Distribution{State: "invalid"}
|
||||
}
|
||||
return Distribution{State: "observed", ID: values["ID"], Version: values["VERSION_ID"], Codename: values["VERSION_CODENAME"]}
|
||||
}
|
||||
|
||||
// Walk directory entries to observe dangling/intermediate links as existing
|
||||
// resources instead of following them and misreporting a clean install target.
|
||||
func resourceState(files fs.FS, path string) string {
|
||||
parent := "."
|
||||
parts := strings.Split(path, "/")
|
||||
for i, part := range parts {
|
||||
entries, err := fs.ReadDir(files, parent)
|
||||
if err != nil {
|
||||
return "unknown"
|
||||
}
|
||||
found := false
|
||||
for _, entry := range entries {
|
||||
if entry.Name() != part {
|
||||
continue
|
||||
}
|
||||
found = true
|
||||
if i == len(parts)-1 || entry.Type()&fs.ModeSymlink != 0 || !entry.IsDir() {
|
||||
return "present"
|
||||
}
|
||||
if parent == "." {
|
||||
parent = part
|
||||
} else {
|
||||
parent += "/" + part
|
||||
}
|
||||
break
|
||||
}
|
||||
if !found {
|
||||
return "missing"
|
||||
}
|
||||
}
|
||||
return "unknown"
|
||||
}
|
||||
@@ -0,0 +1,116 @@
|
||||
package preflight
|
||||
|
||||
import (
|
||||
"io/fs"
|
||||
"strings"
|
||||
"testing"
|
||||
"testing/fstest"
|
||||
|
||||
"server-deploy/internal/inspect"
|
||||
)
|
||||
|
||||
func hostFiles() fstest.MapFS {
|
||||
return fstest.MapFS{
|
||||
"etc/os-release": {Data: []byte("ID=ubuntu\nVERSION_ID=\"26.04\"\nVERSION_CODENAME=resolute\n")},
|
||||
"var/lib/dpkg/status": {Data: []byte("Package: base-files\nStatus: install ok installed\nArchitecture: amd64\nVersion: 1.0\n")},
|
||||
"var/lib/dpkg/updates": {Mode: fs.ModeDir | 0700},
|
||||
}
|
||||
}
|
||||
|
||||
func TestPackageStateBlocksFreshInstallCandidates(t *testing.T) {
|
||||
for _, state := range []string{"install ok installed", "deinstall ok config-files", "install reinstreq half-installed"} {
|
||||
files := hostFiles()
|
||||
files["var/lib/dpkg/status"].Data = []byte("Package: containerd\nStatus: " + state + "\nArchitecture: amd64\nVersion: 1.2.3\n")
|
||||
p := Plan(Probe(baseline(), files, 0, Disk{State: "observed", AvailableBytes: 20 << 30}))
|
||||
if len(p.ProposedChanges) != 0 || !contains(p.Blockers, "existing_packages_require_review") {
|
||||
t.Errorf("existing package state %s overlooked", state)
|
||||
}
|
||||
}
|
||||
files := hostFiles()
|
||||
delete(files, "var/lib/dpkg/status")
|
||||
if p := Plan(Probe(baseline(), files, 0, Disk{State: "observed", AvailableBytes: 20 << 30})); len(p.ProposedChanges) != 0 {
|
||||
t.Fatal("unknown inventory treated as empty")
|
||||
}
|
||||
}
|
||||
func baseline() inspect.Report {
|
||||
return inspect.Report{ProtocolVersion: 1, OS: "linux", Architecture: "amd64", SystemdRuntime: "present", DockerClient: "missing"}
|
||||
}
|
||||
func TestHostFactsAndNonExecutableProposal(t *testing.T) {
|
||||
r := Probe(baseline(), hostFiles(), 0, Disk{State: "observed", AvailableBytes: 20 << 30})
|
||||
if r.Distribution.ID != "ubuntu" || r.Distribution.Version != "26.04" || r.Distribution.State != "observed" || r.Privilege != "root" {
|
||||
t.Fatalf("incorrect host facts: %+v", r)
|
||||
}
|
||||
p := Plan(r)
|
||||
if p.Executable || len(p.ProposedChanges) == 0 || !contains(p.Blockers, "package_versions_unresolved") || !contains(p.Blockers, "host_identity_unverified") {
|
||||
t.Fatalf("unsafe proposal: %+v", p)
|
||||
}
|
||||
}
|
||||
func TestExistingResourcesNeverProposeFreshInstall(t *testing.T) {
|
||||
for _, path := range []string{"var/lib/docker", "var/lib/containerd", "etc/docker", "var/lib/server-deploy", "etc/apt/sources.list.d/docker.sources", "etc/apt/sources.list.d/docker.list"} {
|
||||
files := hostFiles()
|
||||
files[path] = &fstest.MapFile{Mode: fs.ModeDir | 0700}
|
||||
p := Plan(Probe(baseline(), files, 0, Disk{State: "observed", AvailableBytes: 20 << 30}))
|
||||
if len(p.ProposedChanges) != 0 || !contains(p.Blockers, "existing_resources_require_review") {
|
||||
t.Errorf("fresh install proposed over %s", path)
|
||||
}
|
||||
}
|
||||
runtime := baseline()
|
||||
runtime.DockerClient = "present"
|
||||
if p := Plan(Probe(runtime, hostFiles(), 0, Disk{State: "observed", AvailableBytes: 20 << 30})); len(p.ProposedChanges) != 0 {
|
||||
t.Fatal("existing Docker overlooked")
|
||||
}
|
||||
}
|
||||
func TestUnknownAndInsufficientHostFailsClosed(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
runtime inspect.Report
|
||||
uid int
|
||||
disk Disk
|
||||
blocker string
|
||||
}{
|
||||
{"nonroot", baseline(), 1000, Disk{State: "observed", AvailableBytes: 20 << 30}, "root_required"},
|
||||
{"disk unknown", baseline(), 0, Disk{State: "unknown"}, "disk_unverified"},
|
||||
{"disk low", baseline(), 0, Disk{State: "observed", AvailableBytes: 1}, "disk_below_bootstrap_floor"},
|
||||
{"unsupported", inspect.Report{OS: "windows", Architecture: "amd64"}, 0, Disk{}, "unsupported_platform"},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
p := Plan(Probe(tc.runtime, hostFiles(), tc.uid, tc.disk))
|
||||
if !contains(p.Blockers, tc.blocker) || p.Executable || len(p.ProposedChanges) != 0 {
|
||||
t.Fatalf("unsafe proposal: %+v", p)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
func TestOSReleaseRejectsAmbiguityAndNeverEvaluatesShell(t *testing.T) {
|
||||
for _, content := range []string{"ID=ubuntu\nID=debian\nVERSION_ID=26.04\nVERSION_CODENAME=resolute", "ID=$(touch secret)\nVERSION_ID=26.04\nVERSION_CODENAME=resolute", "ID=ubuntu\nVERSION_ID=\"26.04\nVERSION_CODENAME=resolute", strings.Repeat("#", 65537)} {
|
||||
files := hostFiles()
|
||||
files["etc/os-release"].Data = []byte(content)
|
||||
p := Plan(Probe(baseline(), files, 0, Disk{State: "observed", AvailableBytes: 20 << 30}))
|
||||
if len(p.ProposedChanges) != 0 || !contains(p.Blockers, "distribution_unverified") {
|
||||
t.Fatal("ambiguous distribution accepted")
|
||||
}
|
||||
}
|
||||
files := hostFiles()
|
||||
files["etc/os-release"].Data = []byte("ID=ubuntu\nVERSION_ID=26.04\nVERSION_CODENAME=noble\n")
|
||||
if p := Plan(Probe(baseline(), files, 0, Disk{State: "observed", AvailableBytes: 20 << 30})); !contains(p.Blockers, "unsupported_distribution") {
|
||||
t.Fatal("mismatched suite accepted")
|
||||
}
|
||||
}
|
||||
|
||||
type deniedFS struct{}
|
||||
|
||||
func (deniedFS) Open(string) (fs.File, error) { return nil, fs.ErrPermission }
|
||||
func TestAccessFailuresAreNotAbsence(t *testing.T) {
|
||||
r := Probe(baseline(), deniedFS{}, 0, Disk{State: "observed", AvailableBytes: 20 << 30})
|
||||
if r.Distribution.State != "unknown" || r.Resources[0].State != "unknown" || len(Plan(r).ProposedChanges) != 0 {
|
||||
t.Fatal("permission failure treated as clean host")
|
||||
}
|
||||
}
|
||||
func contains(values []string, want string) bool {
|
||||
for _, v := range values {
|
||||
if v == want {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
@@ -0,0 +1,79 @@
|
||||
# Operation metadata store
|
||||
|
||||
Internal execution foundation; not a CLI write endpoint, application executor,
|
||||
backup engine or systemd supervisor. The existing CLI remains read-only.
|
||||
|
||||
## Contract
|
||||
|
||||
Bootstrap must supply one fixed, existing, administrator-owned local directory
|
||||
per host. Use the same canonical directory for every runner on that host.
|
||||
Untrusted users/applications must not be able to replace the directory, its
|
||||
ancestors or files. Linux deployment permissions should be 0700 for the directory
|
||||
and 0600 for metadata. Windows ACL configuration belongs to the installer.
|
||||
NFS/SMB/distributed locking is not supported.
|
||||
|
||||
`Acquire(directory, hostID)` opens an os.Root and takes a nonblocking exclusive
|
||||
OS lock on `host.lock`. Different paths/roots are not a distributed host registry.
|
||||
Hold the returned Session throughout any future application write operation.
|
||||
Never remove/replace `host.lock`, including during cleanup: its inode is the lock
|
||||
identity. Close releases it; abrupt process termination releases it at OS level.
|
||||
|
||||
- `Begin(id, planHash)` registers queued work and returns `(operation, created)`.
|
||||
Reusing ID/hash returns the original record with created=false. Another hash
|
||||
conflicts. Different IDs are blocked while an unresolved record exists.
|
||||
- `Advance(id, revision, next)` compares revision and validates the transition.
|
||||
A queued-to-running transition is the claim; a second claim fails.
|
||||
- `Get(id)` returns a value copy. Closing or poisoning a session forbids its use.
|
||||
|
||||
Allowed transitions:
|
||||
|
||||
```text
|
||||
queued → running | cancelled
|
||||
running → succeeded | failed_recovered | needs_attention | unknown
|
||||
needs_attention / unknown → succeeded | failed_recovered
|
||||
terminal states → no transitions
|
||||
```
|
||||
|
||||
Success/recovery labels are assertions by the caller, not proof. The future
|
||||
executor must verify actual effects before recording them. Unknown/running work
|
||||
survives restart without replay. Reconciliation requires inspecting actual state;
|
||||
no automatic retry, forced reset, lease expiry or stale-lock deletion is provided.
|
||||
|
||||
## Persistence
|
||||
|
||||
state.json is a versioned, canonical JSON snapshot, maximum 16 MiB, with host
|
||||
identity and SHA-256 integrity checksum. Unknown fields, duplicates, truncation,
|
||||
changed data and host mismatch are rejected. The checksum is not authentication.
|
||||
It is not an append-only audit log; event logging is a separate future component.
|
||||
|
||||
Mutation writes a unique private temporary file, syncs it, closes it, renames over
|
||||
the snapshot and (Linux) syncs the containing directory. A save error poisons the
|
||||
session because the rename may already have happened; close, reacquire and query
|
||||
before deciding anything. Temporary remnants from a killed process are ignored,
|
||||
never interpreted as successful work; automatic cleanup is not implemented.
|
||||
|
||||
Current snapshots retain all operation IDs. No pruning is provided, because
|
||||
forgetting completed IDs can re-enable an old request. At the size limit, writes
|
||||
fail closed. Admission reserves 64 bytes for the active record's later status
|
||||
and revision growth; capacity rejection does not poison read access. A retention/tombstone design is required before bounded production
|
||||
history cleanup is introduced.
|
||||
|
||||
host.lock also contains a synced initialization marker. Once work is persisted,
|
||||
a missing snapshot is rejected rather than treated as a fresh store. A crash
|
||||
between the first snapshot and marker can be repaired only from a valid snapshot.
|
||||
Protect both files. Restoring an older valid snapshot still rolls back idempotency
|
||||
history; do not restart execution without independent reconciliation. This package
|
||||
cannot detect malicious administrator edits or rollback/deletion of the entire
|
||||
state directory.
|
||||
|
||||
## Platform boundary
|
||||
|
||||
Linux uses flock and file/directory fsync. Windows uses LockFileEx for development
|
||||
tests, file sync and rename; Windows power-loss durability is not promised.
|
||||
Other OSes refuse acquisition. The local macOS panel will communicate with the
|
||||
Linux runner, not use this package as its local SQLite replacement.
|
||||
|
||||
Kernel locks are advisory on Linux; all participating writers must obey them.
|
||||
External Docker/Portainer operations are outside this lock and require drift
|
||||
checks. Multi-process tests prove process-crash behavior, not sudden power failure
|
||||
or storage-hardware reliability.
|
||||
@@ -0,0 +1,26 @@
|
||||
//go:build linux
|
||||
|
||||
package state
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"os"
|
||||
"syscall"
|
||||
)
|
||||
|
||||
func lockExclusive(f *os.File) error {
|
||||
err := syscall.Flock(int(f.Fd()), syscall.LOCK_EX|syscall.LOCK_NB)
|
||||
if errors.Is(err, syscall.EWOULDBLOCK) || errors.Is(err, syscall.EAGAIN) {
|
||||
return ErrBusy
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func syncDirectory(r *os.Root) error {
|
||||
f, err := r.Open(".")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer f.Close()
|
||||
return f.Sync()
|
||||
}
|
||||
@@ -0,0 +1,13 @@
|
||||
//go:build !linux && !windows
|
||||
|
||||
package state
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"os"
|
||||
)
|
||||
|
||||
func lockExclusive(*os.File) error {
|
||||
return errors.New("operation store supports Linux and Windows only")
|
||||
}
|
||||
func syncDirectory(*os.Root) error { return errors.New("unsupported state durability platform") }
|
||||
@@ -0,0 +1,30 @@
|
||||
//go:build windows
|
||||
|
||||
package state
|
||||
|
||||
import (
|
||||
"os"
|
||||
"runtime"
|
||||
"syscall"
|
||||
"unsafe"
|
||||
)
|
||||
|
||||
var lockFileEx = syscall.NewLazyDLL("kernel32.dll").NewProc("LockFileEx")
|
||||
|
||||
func lockExclusive(f *os.File) error {
|
||||
var overlapped syscall.Overlapped
|
||||
// LOCKFILE_FAIL_IMMEDIATELY | LOCKFILE_EXCLUSIVE_LOCK, first byte only.
|
||||
ok, _, err := lockFileEx.Call(f.Fd(), 3, 0, 1, 0, uintptr(unsafe.Pointer(&overlapped)))
|
||||
runtime.KeepAlive(f)
|
||||
if ok != 0 {
|
||||
return nil
|
||||
}
|
||||
if err == syscall.Errno(33) {
|
||||
return ErrBusy
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
// Windows is a development platform. File.Sync is used, but Go does not provide
|
||||
// a portable directory fsync here. Do not claim Windows power-loss durability.
|
||||
func syncDirectory(*os.Root) error { return nil }
|
||||
@@ -0,0 +1,56 @@
|
||||
// Package state persists task metadata. It never executes application actions.
|
||||
package state
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"regexp"
|
||||
)
|
||||
|
||||
var (
|
||||
ErrBusy = errors.New("host state is locked")
|
||||
ErrClosed = errors.New("session is closed")
|
||||
ErrPoisoned = errors.New("state write outcome uncertain; reopen and reconcile")
|
||||
ErrConflict = errors.New("operation identity or revision conflict")
|
||||
ErrUnresolved = errors.New("another operation requires completion or reconciliation")
|
||||
ErrTransition = errors.New("invalid operation transition")
|
||||
ErrNotFound = errors.New("operation not found")
|
||||
ErrCapacity = errors.New("operation history capacity exhausted")
|
||||
idPattern = regexp.MustCompile(`^[a-z][a-z0-9-]{0,47}$`)
|
||||
hashPattern = regexp.MustCompile(`^sha256:[a-f0-9]{64}$`)
|
||||
)
|
||||
|
||||
type Status string
|
||||
|
||||
const (
|
||||
Queued Status = "queued"
|
||||
Running Status = "running"
|
||||
Succeeded Status = "succeeded"
|
||||
FailedRecovered Status = "failed_recovered"
|
||||
NeedsAttention Status = "needs_attention"
|
||||
Unknown Status = "unknown"
|
||||
Cancelled Status = "cancelled"
|
||||
)
|
||||
|
||||
type Operation struct {
|
||||
ID string `json:"id"`
|
||||
PlanHash string `json:"planHash"`
|
||||
Status Status `json:"status"`
|
||||
Revision uint64 `json:"revision"`
|
||||
}
|
||||
|
||||
func (s Status) terminal() bool { return s == Succeeded || s == FailedRecovered || s == Cancelled }
|
||||
func (s Status) valid() bool {
|
||||
return s.terminal() || s == Queued || s == Running || s == NeedsAttention || s == Unknown
|
||||
}
|
||||
func allowed(from, to Status) bool {
|
||||
switch from {
|
||||
case Queued:
|
||||
return to == Running || to == Cancelled
|
||||
case Running:
|
||||
return to == Succeeded || to == FailedRecovered || to == NeedsAttention || to == Unknown
|
||||
case NeedsAttention, Unknown:
|
||||
return to == Succeeded || to == FailedRecovered
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,293 @@
|
||||
package state
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestIdempotencyAndTransitions(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
s, err := Acquire(dir, "host-one")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer s.Close()
|
||||
op, created, err := s.Begin("op-one", testHash)
|
||||
if err != nil || !created || op.Status != Queued || op.Revision != 1 {
|
||||
t.Fatalf("begin: %+v %v", op, err)
|
||||
}
|
||||
duplicate, created, err := s.Begin("op-one", testHash)
|
||||
if err != nil || created || duplicate != op {
|
||||
t.Fatal("duplicate mutated operation")
|
||||
}
|
||||
if _, _, err := s.Begin("op-one", "sha256:"+strings.Repeat("b", 64)); !errors.Is(err, ErrConflict) {
|
||||
t.Fatal("id rebound to another plan")
|
||||
}
|
||||
if _, _, err := s.Begin("op-two", testHash); !errors.Is(err, ErrUnresolved) {
|
||||
t.Fatal("unresolved operation bypass")
|
||||
}
|
||||
if _, err := s.Advance(op.ID, op.Revision, Succeeded); !errors.Is(err, ErrTransition) {
|
||||
t.Fatal("queued operation succeeded without running")
|
||||
}
|
||||
op, err = s.Advance(op.ID, op.Revision, Running)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := s.Advance(op.ID, 1, Succeeded); !errors.Is(err, ErrConflict) {
|
||||
t.Fatal("stale revision accepted")
|
||||
}
|
||||
if _, err := s.Advance(op.ID, op.Revision, Running); !errors.Is(err, ErrTransition) {
|
||||
t.Fatal("operation claimed twice")
|
||||
}
|
||||
op, err = s.Advance(op.ID, op.Revision, NeedsAttention)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, _, err := s.Begin("op-two", testHash); !errors.Is(err, ErrUnresolved) {
|
||||
t.Fatal("ignored manual recovery")
|
||||
}
|
||||
op, err = s.Advance(op.ID, op.Revision, FailedRecovered)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := s.Advance(op.ID, op.Revision, Running); !errors.Is(err, ErrTransition) {
|
||||
t.Fatal("terminal operation restarted")
|
||||
}
|
||||
s.Close()
|
||||
s, err = Acquire(dir, "host-one")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer s.Close()
|
||||
got, err := s.Get("op-one")
|
||||
if err != nil || got != op {
|
||||
t.Fatalf("persistence mismatch: %+v %v", got, err)
|
||||
}
|
||||
if _, created, err := s.Begin("op-two", testHash); err != nil || !created {
|
||||
t.Fatal("completed operation blocks new work")
|
||||
}
|
||||
}
|
||||
|
||||
func TestCorruptionAndHostMismatchFailClosed(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
s, err := Acquire(dir, "host-one")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, _, err := s.Begin("op-one", testHash); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
s.Close()
|
||||
if other, err := Acquire(dir, "host-two"); err == nil {
|
||||
other.Close()
|
||||
t.Fatal("wrong host accepted")
|
||||
}
|
||||
for _, data := range []string{`{`, `{}`, `null`, strings.Repeat("x", maxSnapshotBytes+1)} {
|
||||
if err := os.WriteFile(filepath.Join(dir, "state.json"), []byte(data), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if other, err := Acquire(dir, "host-one"); err == nil {
|
||||
other.Close()
|
||||
t.Fatal("corrupt state silently reset")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestInvalidOperationsDoNotCreateSnapshot(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
s, err := Acquire(dir, "host-one")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer s.Close()
|
||||
for _, id := range []string{"", "../outside", "bad/id"} {
|
||||
if _, _, err := s.Begin(id, testHash); err == nil {
|
||||
t.Fatal("invalid operation id accepted")
|
||||
}
|
||||
}
|
||||
if _, _, err := s.Begin("op-one", "latest"); err == nil {
|
||||
t.Fatal("mutable plan binding accepted")
|
||||
}
|
||||
if _, err := s.Get("missing"); !errors.Is(err, ErrNotFound) {
|
||||
t.Fatal("missing operation not reported")
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(dir, "state.json")); !os.IsNotExist(err) {
|
||||
t.Fatal("invalid request persisted state")
|
||||
}
|
||||
}
|
||||
|
||||
func TestPersistenceFailurePoisonsSession(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
s, err := Acquire(dir, "host-one")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer s.Close()
|
||||
// A directory at the destination makes atomic replacement fail on both OSes.
|
||||
if err := os.Mkdir(filepath.Join(dir, "state.json"), 0700); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, _, err := s.Begin("op-one", testHash); err == nil {
|
||||
t.Fatal("reported failed persistence as success")
|
||||
}
|
||||
if _, _, err := s.Begin("op-two", testHash); !errors.Is(err, ErrPoisoned) {
|
||||
t.Fatalf("continued after ambiguous write: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestConcurrentClaimsHaveOneWinner(t *testing.T) {
|
||||
s, err := Acquire(t.TempDir(), "host-one")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer s.Close()
|
||||
var createdCount, claimedCount atomic.Int32
|
||||
var wg sync.WaitGroup
|
||||
for n := 0; n < 16; n++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
_, created, err := s.Begin("op-one", testHash)
|
||||
if err != nil {
|
||||
t.Error(err)
|
||||
}
|
||||
if created {
|
||||
createdCount.Add(1)
|
||||
}
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
for n := 0; n < 16; n++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
_, err := s.Advance("op-one", 1, Running)
|
||||
if err == nil {
|
||||
claimedCount.Add(1)
|
||||
} else if !errors.Is(err, ErrConflict) {
|
||||
t.Error(err)
|
||||
}
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
if createdCount.Load() != 1 || claimedCount.Load() != 1 {
|
||||
t.Fatalf("duplicate winners: %d %d", createdCount.Load(), claimedCount.Load())
|
||||
}
|
||||
}
|
||||
|
||||
func TestTamperedSnapshotRejected(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
s, err := Acquire(dir, "host-one")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, _, err := s.Begin("op-one", testHash); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
s.Close()
|
||||
path := filepath.Join(dir, "state.json")
|
||||
raw, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, changed := range []string{
|
||||
strings.Replace(string(raw), `"queued"`, `"succeeded"`, 1),
|
||||
strings.Replace(string(raw), `"version":1`, `"version":1,"version":1`, 1),
|
||||
string(raw) + ` {}`,
|
||||
} {
|
||||
if err := os.WriteFile(path, []byte(changed), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if s, err := Acquire(dir, "host-one"); err == nil {
|
||||
s.Close()
|
||||
t.Fatal("tampered snapshot accepted")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAllTerminalAndReconciliationPaths(t *testing.T) {
|
||||
for _, path := range [][]Status{{Cancelled}, {Running, Succeeded}, {Running, FailedRecovered}, {Running, Unknown, FailedRecovered}, {Running, NeedsAttention, Succeeded}} {
|
||||
s, err := Acquire(t.TempDir(), "host-one")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
op, _, err := s.Begin("op-one", testHash)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, next := range path {
|
||||
op, err = s.Advance(op.ID, op.Revision, next)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if _, _, err := s.Begin("op-two", testHash); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
s.Close()
|
||||
}
|
||||
}
|
||||
|
||||
func TestMissingSnapshotDoesNotResetInitializedStore(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
s, err := Acquire(dir, "host-one")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, _, err := s.Begin("op-one", testHash); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
s.Close()
|
||||
if err := os.Remove(filepath.Join(dir, "state.json")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if s, err := Acquire(dir, "host-one"); err == nil {
|
||||
s.Close()
|
||||
t.Fatal("silently reset initialized store")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAdmissionReservesSpaceForCompletion(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
data := snapshot{Version: 1, HostID: "host-one", Operations: make(map[string]Operation)}
|
||||
for n := 0; n < 113357; n++ {
|
||||
id := fmt.Sprintf("op%06d", n)
|
||||
if n < 2 {
|
||||
id += strings.Repeat("a", 26)
|
||||
}
|
||||
data.Operations[id] = Operation{ID: id, PlanHash: testHash, Status: Cancelled, Revision: 2}
|
||||
}
|
||||
raw, err := encodeSnapshot(data)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(dir, "state.json"), raw, 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// Confirm this fixture reaches the actual bug boundary, not an arbitrary limit.
|
||||
data.Operations["op-new"] = Operation{ID: "op-new", PlanHash: testHash, Status: Queued, Revision: 1}
|
||||
queued, err := encodeSnapshot(data)
|
||||
if err != nil || len(queued) != maxSnapshotBytes-2 {
|
||||
t.Fatalf("fixture outside boundary: %d %v", len(queued), err)
|
||||
}
|
||||
s, err := Acquire(dir, "host-one")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer s.Close()
|
||||
if _, _, err := s.Begin("op-new", testHash); err == nil {
|
||||
t.Fatal("admitted task without room for terminal state")
|
||||
}
|
||||
if _, err := s.Get("op000002"); err != nil {
|
||||
t.Fatalf("capacity rejection poisoned read access: %v", err)
|
||||
}
|
||||
if _, err := s.Get("op-new"); !errors.Is(err, ErrNotFound) {
|
||||
t.Fatal("rejected admission persisted")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,120 @@
|
||||
package state
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"crypto/rand"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"io"
|
||||
"os"
|
||||
)
|
||||
|
||||
const maxSnapshotBytes = 16 << 20
|
||||
|
||||
type snapshot struct {
|
||||
Version int `json:"version"`
|
||||
HostID string `json:"hostId"`
|
||||
Operations map[string]Operation `json:"operations"`
|
||||
Checksum string `json:"checksum"`
|
||||
}
|
||||
|
||||
func encodeSnapshot(s snapshot) ([]byte, error) {
|
||||
s.Checksum = ""
|
||||
raw, err := json.Marshal(s)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
sum := sha256.Sum256(raw)
|
||||
s.Checksum = "sha256:" + hex.EncodeToString(sum[:])
|
||||
raw, err = json.Marshal(s)
|
||||
if len(raw) > maxSnapshotBytes {
|
||||
return nil, errors.New("state capacity exceeded")
|
||||
}
|
||||
return raw, err
|
||||
}
|
||||
|
||||
func readSnapshot(r *os.Root, hostID string, initialized bool) (snapshot, error) {
|
||||
empty := snapshot{Version: 1, HostID: hostID, Operations: make(map[string]Operation)}
|
||||
if err := regularOrMissing(r, "state.json"); err != nil {
|
||||
return snapshot{}, err
|
||||
}
|
||||
f, err := r.Open("state.json")
|
||||
if os.IsNotExist(err) {
|
||||
if initialized {
|
||||
return snapshot{}, errors.New("initialized store has lost its snapshot")
|
||||
}
|
||||
return empty, nil
|
||||
}
|
||||
if err != nil {
|
||||
return snapshot{}, err
|
||||
}
|
||||
defer f.Close()
|
||||
raw, err := io.ReadAll(io.LimitReader(f, maxSnapshotBytes+1))
|
||||
if err != nil {
|
||||
return snapshot{}, err
|
||||
}
|
||||
if len(raw) > maxSnapshotBytes {
|
||||
return snapshot{}, errors.New("state exceeds size limit")
|
||||
}
|
||||
var s snapshot
|
||||
if err := json.Unmarshal(raw, &s); err != nil {
|
||||
return snapshot{}, errors.New("corrupt state")
|
||||
}
|
||||
if s.Version != 1 || s.HostID != hostID || s.Operations == nil {
|
||||
return snapshot{}, errors.New("incompatible state or host mismatch")
|
||||
}
|
||||
canonical, err := encodeSnapshot(s)
|
||||
if err != nil || !bytes.Equal(raw, canonical) {
|
||||
return snapshot{}, errors.New("state integrity check failed")
|
||||
}
|
||||
unresolved := 0
|
||||
for id, op := range s.Operations {
|
||||
if id != op.ID || !idPattern.MatchString(id) || !hashPattern.MatchString(op.PlanHash) || !op.Status.valid() || op.Revision == 0 {
|
||||
return snapshot{}, errors.New("invalid operation record")
|
||||
}
|
||||
if !op.Status.terminal() {
|
||||
unresolved++
|
||||
}
|
||||
}
|
||||
if unresolved > 1 {
|
||||
return snapshot{}, errors.New("multiple unresolved operations")
|
||||
}
|
||||
return s, nil
|
||||
}
|
||||
|
||||
func writeSnapshot(r *os.Root, s snapshot) error {
|
||||
raw, err := encodeSnapshot(s)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := regularOrMissing(r, "state.json"); err != nil {
|
||||
return err
|
||||
}
|
||||
var entropy [16]byte
|
||||
if _, err := rand.Read(entropy[:]); err != nil {
|
||||
return err
|
||||
}
|
||||
name := ".state-" + hex.EncodeToString(entropy[:]) + ".tmp"
|
||||
f, err := r.OpenFile(name, os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0600)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer r.Remove(name) // Only the unique file just created; never state.json or host.lock.
|
||||
if _, err := f.Write(raw); err != nil {
|
||||
f.Close()
|
||||
return err
|
||||
}
|
||||
if err := f.Sync(); err != nil {
|
||||
f.Close()
|
||||
return err
|
||||
}
|
||||
if err := f.Close(); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := r.Rename(name, "state.json"); err != nil {
|
||||
return err
|
||||
}
|
||||
return syncDirectory(r)
|
||||
}
|
||||
@@ -0,0 +1,225 @@
|
||||
package state
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sync"
|
||||
)
|
||||
|
||||
// Session holds an exclusive host lock for its entire lifetime. The caller must
|
||||
// retain it throughout any future write operation, not only metadata updates.
|
||||
// Never unlink host.lock: replacing its inode defeats OS locking.
|
||||
type Session struct {
|
||||
mu sync.Mutex
|
||||
root *os.Root
|
||||
lock *os.File
|
||||
data snapshot
|
||||
closed bool
|
||||
poisoned bool
|
||||
initialized bool
|
||||
}
|
||||
|
||||
const initializedMarker = "server-deploy-state-v1\n"
|
||||
|
||||
// Acquire requires an existing administrator-owned LOCAL directory. Bootstrap
|
||||
// owns directory creation/permissions; this function never creates a new root.
|
||||
func Acquire(directory, hostID string) (*Session, error) {
|
||||
if !idPattern.MatchString(hostID) || !filepath.IsAbs(directory) {
|
||||
return nil, errors.New("invalid host or state directory")
|
||||
}
|
||||
r, err := os.OpenRoot(directory)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
fail := func(err error) (*Session, error) { r.Close(); return nil, err }
|
||||
if err := regularOrMissing(r, "host.lock"); err != nil {
|
||||
return fail(err)
|
||||
}
|
||||
lock, err := r.OpenFile("host.lock", os.O_CREATE|os.O_RDWR, 0600)
|
||||
if err != nil {
|
||||
return fail(err)
|
||||
}
|
||||
if err := lockExclusive(lock); err != nil {
|
||||
lock.Close()
|
||||
return fail(err)
|
||||
}
|
||||
s := &Session{root: r, lock: lock}
|
||||
info, err := lock.Stat()
|
||||
if err != nil {
|
||||
s.Close()
|
||||
return nil, err
|
||||
}
|
||||
if info.Size() != 0 {
|
||||
if info.Size() != int64(len(initializedMarker)) {
|
||||
s.Close()
|
||||
return nil, errors.New("invalid state initialization marker")
|
||||
}
|
||||
marker := make([]byte, len(initializedMarker))
|
||||
if _, err := lock.ReadAt(marker, 0); err != nil || string(marker) != initializedMarker {
|
||||
s.Close()
|
||||
return nil, errors.New("corrupt state initialization marker")
|
||||
}
|
||||
s.initialized = true
|
||||
}
|
||||
s.data, err = readSnapshot(r, hostID, s.initialized)
|
||||
if err != nil {
|
||||
s.Close()
|
||||
return nil, err
|
||||
}
|
||||
// A crash may happen after the first snapshot rename but before its marker.
|
||||
// Repair only from a fully validated existing snapshot, never from absence.
|
||||
if len(s.data.Operations) > 0 {
|
||||
if err := s.markInitialized(); err != nil {
|
||||
s.Close()
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
return s, nil
|
||||
}
|
||||
|
||||
func (s *Session) markInitialized() error {
|
||||
if s.initialized {
|
||||
return nil
|
||||
}
|
||||
if _, err := s.lock.WriteAt([]byte(initializedMarker), 0); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := s.lock.Sync(); err != nil {
|
||||
return err
|
||||
}
|
||||
s.initialized = true
|
||||
return nil
|
||||
}
|
||||
|
||||
func regularOrMissing(r *os.Root, name string) error {
|
||||
info, err := r.Lstat(name)
|
||||
if os.IsNotExist(err) {
|
||||
return nil
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !info.Mode().IsRegular() {
|
||||
return errors.New("state path must be a regular file, not a link")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Session) ready() error {
|
||||
if s.closed {
|
||||
return ErrClosed
|
||||
}
|
||||
if s.poisoned {
|
||||
return ErrPoisoned
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Session) Close() error {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
if s.closed {
|
||||
return nil
|
||||
}
|
||||
s.closed = true
|
||||
return errors.Join(s.lock.Close(), s.root.Close())
|
||||
}
|
||||
|
||||
func (s *Session) Get(id string) (Operation, error) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
if err := s.ready(); err != nil {
|
||||
return Operation{}, err
|
||||
}
|
||||
op, ok := s.data.Operations[id]
|
||||
if !ok {
|
||||
return Operation{}, ErrNotFound
|
||||
}
|
||||
return op, nil
|
||||
}
|
||||
|
||||
// Begin is idempotent, not an execution claim. A repeated ID cannot be rebound
|
||||
// to a different plan, and any unresolved operation blocks unrelated work.
|
||||
func (s *Session) Begin(id, planHash string) (Operation, bool, error) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
if err := s.ready(); err != nil {
|
||||
return Operation{}, false, err
|
||||
}
|
||||
if !idPattern.MatchString(id) || !hashPattern.MatchString(planHash) {
|
||||
return Operation{}, false, errors.New("invalid operation identity")
|
||||
}
|
||||
if op, ok := s.data.Operations[id]; ok {
|
||||
if op.PlanHash != planHash {
|
||||
return Operation{}, false, ErrConflict
|
||||
}
|
||||
return op, false, nil
|
||||
}
|
||||
for _, op := range s.data.Operations {
|
||||
if !op.Status.terminal() {
|
||||
return Operation{}, false, ErrUnresolved
|
||||
}
|
||||
}
|
||||
op := Operation{ID: id, PlanHash: planHash, Status: Queued, Revision: 1}
|
||||
if err := s.save(op); err != nil {
|
||||
return Operation{}, false, err
|
||||
}
|
||||
return op, true, nil
|
||||
}
|
||||
|
||||
// Advance uses a revision check so only one caller can claim queued work.
|
||||
// Reconciliation to a terminal status requires external evidence; this metadata
|
||||
// layer cannot prove that a deployment or restore actually succeeded.
|
||||
func (s *Session) Advance(id string, revision uint64, next Status) (Operation, error) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
if err := s.ready(); err != nil {
|
||||
return Operation{}, err
|
||||
}
|
||||
op, ok := s.data.Operations[id]
|
||||
if !ok {
|
||||
return Operation{}, ErrNotFound
|
||||
}
|
||||
if op.Revision != revision {
|
||||
return Operation{}, ErrConflict
|
||||
}
|
||||
if !allowed(op.Status, next) {
|
||||
return Operation{}, ErrTransition
|
||||
}
|
||||
if op.Revision == ^uint64(0) {
|
||||
return Operation{}, ErrConflict
|
||||
}
|
||||
op.Status = next
|
||||
op.Revision++
|
||||
if err := s.save(op); err != nil {
|
||||
return Operation{}, err
|
||||
}
|
||||
return op, nil
|
||||
}
|
||||
|
||||
func (s *Session) save(op Operation) error {
|
||||
updated := snapshot{Version: 1, HostID: s.data.HostID, Operations: make(map[string]Operation, len(s.data.Operations)+1)}
|
||||
for id, old := range s.data.Operations {
|
||||
updated.Operations[id] = old
|
||||
}
|
||||
updated.Operations[op.ID] = op
|
||||
if op.Revision == 1 {
|
||||
// Only one unresolved operation is admitted. Reserve more than the
|
||||
// longest status growth plus uint64 revision growth (at most 29 bytes).
|
||||
raw, err := encodeSnapshot(updated)
|
||||
if err != nil || len(raw) > maxSnapshotBytes-64 {
|
||||
return ErrCapacity
|
||||
}
|
||||
}
|
||||
if err := writeSnapshot(s.root, updated); err != nil {
|
||||
s.poisoned = true
|
||||
return errors.Join(ErrPoisoned, err)
|
||||
}
|
||||
if err := s.markInitialized(); err != nil {
|
||||
s.poisoned = true
|
||||
return errors.Join(ErrPoisoned, err)
|
||||
}
|
||||
s.data = updated
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,134 @@
|
||||
package state
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
const testHash = "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
|
||||
|
||||
func TestExclusiveSession(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
s, err := Acquire(dir, "host-one")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(func() { s.Close() })
|
||||
if second, err := Acquire(dir, "host-one"); second != nil || !errors.Is(err, ErrBusy) {
|
||||
t.Fatalf("lock bypass: %v", err)
|
||||
}
|
||||
other, err := Acquire(t.TempDir(), "host-two")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
other.Close()
|
||||
if err := s.Close(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, _, err := s.Begin("op-one", testHash); !errors.Is(err, ErrClosed) {
|
||||
t.Fatal("used closed session")
|
||||
}
|
||||
next, err := Acquire(dir, "host-one")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
next.Close()
|
||||
}
|
||||
|
||||
func TestInvalidHost(t *testing.T) {
|
||||
for _, host := range []string{"", "../host", "Host", strings.Repeat("a", 49)} {
|
||||
if s, err := Acquire(t.TempDir(), host); err == nil {
|
||||
s.Close()
|
||||
t.Fatal("accepted invalid host")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestStatePathsRejectSymlinks(t *testing.T) {
|
||||
for _, name := range []string{"host.lock", "state.json"} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
dir, outside := t.TempDir(), filepath.Join(t.TempDir(), "outside")
|
||||
if err := os.WriteFile(outside, []byte("protected"), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.Symlink(outside, filepath.Join(dir, name)); err != nil {
|
||||
t.Skipf("symlink privilege unavailable: %v", err)
|
||||
}
|
||||
if s, err := Acquire(dir, "host-one"); err == nil {
|
||||
s.Close()
|
||||
t.Fatal("accepted symlink state")
|
||||
}
|
||||
content, err := os.ReadFile(outside)
|
||||
if err != nil || string(content) != "protected" {
|
||||
t.Fatal("modified outside file")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// The subprocess exits without Close: the OS must release its lock, but its
|
||||
// running operation must remain recorded. No mock can test this boundary.
|
||||
func TestProcessDeathPreservesRunningOperation(t *testing.T) {
|
||||
if os.Getenv("DEPLOYCTL_STATE_CHILD") == "1" {
|
||||
s, err := Acquire(os.Getenv("DEPLOYCTL_STATE_DIR"), "host-one")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
op, _, err := s.Begin("op-one", testHash)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = s.Advance(op.ID, op.Revision, Running); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
fmt.Println("READY")
|
||||
bufio.NewReader(os.Stdin).ReadByte()
|
||||
os.Exit(3)
|
||||
}
|
||||
dir := t.TempDir()
|
||||
cmd := exec.Command(os.Args[0], "-test.run=^TestProcessDeathPreservesRunningOperation$")
|
||||
cmd.Env = append(os.Environ(), "DEPLOYCTL_STATE_CHILD=1", "DEPLOYCTL_STATE_DIR="+dir)
|
||||
stdin, err := cmd.StdinPipe()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer stdin.Close()
|
||||
stdout, err := cmd.StdoutPipe()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cmd.Stderr = os.Stderr
|
||||
if err := cmd.Start(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(func() { cmd.Process.Kill(); cmd.Wait() })
|
||||
line, err := bufio.NewReader(stdout).ReadString('\n')
|
||||
if err != nil || strings.TrimSpace(line) != "READY" {
|
||||
t.Fatalf("child failed: %q %v", line, err)
|
||||
}
|
||||
if _, err := Acquire(dir, "host-one"); !errors.Is(err, ErrBusy) {
|
||||
t.Fatalf("cross-process lock bypass: %v", err)
|
||||
}
|
||||
if err := cmd.Process.Kill(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cmd.Wait()
|
||||
s, err := Acquire(dir, "host-one")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer s.Close()
|
||||
op, created, err := s.Begin("op-one", testHash)
|
||||
if err != nil || created || op.Status != Running || op.Revision != 2 {
|
||||
t.Fatalf("lost interrupted task: %+v %v", op, err)
|
||||
}
|
||||
if _, _, err := s.Begin("op-two", testHash); !errors.Is(err, ErrUnresolved) {
|
||||
t.Fatal("allowed writes before reconciliation")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,143 @@
|
||||
// Package wire enforces the bounded, exact JSON protocol shared by CLI and packages.
|
||||
package wire
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"io"
|
||||
"reflect"
|
||||
"time"
|
||||
"unicode/utf8"
|
||||
)
|
||||
|
||||
// Decode refuses duplicate keys, case aliases, unknown/missing fields,
|
||||
// nulls and trailing values. encoding/json alone accepts several of these.
|
||||
func Decode(in io.Reader, target any, limit int64) error {
|
||||
if limit <= 0 || limit > 16<<20 {
|
||||
return errors.New("invalid input limit")
|
||||
}
|
||||
typ := reflect.TypeOf(target)
|
||||
if typ == nil || typ.Kind() != reflect.Pointer || reflect.ValueOf(target).IsNil() {
|
||||
return errors.New("expected nonnil decode target")
|
||||
}
|
||||
data, err := io.ReadAll(io.LimitReader(in, limit+1))
|
||||
if err != nil || int64(len(data)) > limit || !utf8.Valid(data) {
|
||||
return errors.New("invalid input")
|
||||
}
|
||||
decoder := json.NewDecoder(bytes.NewReader(data))
|
||||
if err := uniqueValue(decoder, 0); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := decoder.Token(); err != io.EOF {
|
||||
return errors.New("trailing input")
|
||||
}
|
||||
if err := exactFields(data, typ.Elem()); err != nil {
|
||||
return err
|
||||
}
|
||||
return json.Unmarshal(data, target)
|
||||
}
|
||||
|
||||
func uniqueValue(d *json.Decoder, depth int) error {
|
||||
if depth > 32 {
|
||||
return errors.New("input nesting exceeds limit")
|
||||
}
|
||||
token, err := d.Token()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
delim, composite := token.(json.Delim)
|
||||
if !composite {
|
||||
return nil
|
||||
}
|
||||
switch delim {
|
||||
case '{':
|
||||
seen := make(map[string]bool)
|
||||
for d.More() {
|
||||
token, err := d.Token()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
key, ok := token.(string)
|
||||
if !ok || seen[key] {
|
||||
return errors.New("duplicate or invalid field")
|
||||
}
|
||||
seen[key] = true
|
||||
if err := uniqueValue(d, depth+1); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
case '[':
|
||||
for d.More() {
|
||||
if err := uniqueValue(d, depth+1); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
default:
|
||||
return errors.New("unexpected delimiter")
|
||||
}
|
||||
_, err = d.Token()
|
||||
return err
|
||||
}
|
||||
|
||||
func exactFields(data []byte, typ reflect.Type) error {
|
||||
if bytes.Equal(bytes.TrimSpace(data), []byte("null")) {
|
||||
return errors.New("null is not permitted")
|
||||
}
|
||||
if typ == reflect.TypeOf(time.Time{}) {
|
||||
var text string
|
||||
if err := json.Unmarshal(data, &text); err != nil {
|
||||
return err
|
||||
}
|
||||
parsed, err := time.Parse(time.RFC3339, text)
|
||||
if err != nil || text != parsed.UTC().Format("2006-01-02T15:04:05Z") {
|
||||
return errors.New("expected canonical UTC timestamp")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
if typ.Kind() == reflect.Slice {
|
||||
var items []json.RawMessage
|
||||
if err := json.Unmarshal(data, &items); err != nil {
|
||||
return err
|
||||
}
|
||||
for _, item := range items {
|
||||
if err := exactFields(item, typ.Elem()); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
if typ.Kind() == reflect.Map {
|
||||
var entries map[string]json.RawMessage
|
||||
if err := json.Unmarshal(data, &entries); err != nil {
|
||||
return err
|
||||
}
|
||||
for _, value := range entries {
|
||||
if err := exactFields(value, typ.Elem()); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
if typ.Kind() != reflect.Struct {
|
||||
return nil
|
||||
}
|
||||
var fields map[string]json.RawMessage
|
||||
if err := json.Unmarshal(data, &fields); err != nil {
|
||||
return err
|
||||
}
|
||||
if len(fields) != typ.NumField() {
|
||||
return errors.New("unexpected field set")
|
||||
}
|
||||
for n := 0; n < typ.NumField(); n++ {
|
||||
field := typ.Field(n)
|
||||
value, exists := fields[field.Tag.Get("json")]
|
||||
if !exists {
|
||||
return errors.New("missing field")
|
||||
}
|
||||
if err := exactFields(value, field.Type); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,70 @@
|
||||
package wire
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
type component struct {
|
||||
Name string `json:"name"`
|
||||
}
|
||||
type document struct {
|
||||
Components []component `json:"components"`
|
||||
}
|
||||
|
||||
func TestStrictNestedArrays(t *testing.T) {
|
||||
var doc document
|
||||
if err := Decode(strings.NewReader(`{"components":[{"name":"server"}]}`), &doc, 1024); err != nil || doc.Components[0].Name != "server" {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, input := range []string{
|
||||
`{"components":[{"name":"server","command":"secret"}]}`,
|
||||
`{"components":[{}]}`,
|
||||
`{"components":[{"Name":"server"}]}`,
|
||||
`{"components":[null]}`,
|
||||
`{"components":null}`,
|
||||
`{"components":[{"name":null}]}`,
|
||||
`{"components":[{"name":"one","name":"two"}]}`,
|
||||
"{\"components\":[{\"name\":\"\xff\"}]}",
|
||||
} {
|
||||
if Decode(strings.NewReader(input), &doc, 1024) == nil {
|
||||
t.Errorf("accepted malformed nested JSON: %q", input)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestDecodeBounds(t *testing.T) {
|
||||
var doc document
|
||||
for _, input := range []string{`{"components":[]} {}`, strings.Repeat(" ", 1025) + `{"components":[]}`} {
|
||||
if Decode(strings.NewReader(input), &doc, 1024) == nil {
|
||||
t.Fatal("accepted trailing or oversized input")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestInvalidDecoderArguments(t *testing.T) {
|
||||
var doc document
|
||||
for _, limit := range []int64{0, -1, 1<<63 - 1} {
|
||||
if Decode(strings.NewReader(`{}`), &doc, limit) == nil {
|
||||
t.Fatal("accepted invalid limit")
|
||||
}
|
||||
}
|
||||
var nilDoc *document
|
||||
for _, target := range []any{nil, doc, nilDoc} {
|
||||
if Decode(strings.NewReader(`{}`), target, 1024) == nil {
|
||||
t.Fatal("accepted invalid target")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestStrictTypedMaps(t *testing.T) {
|
||||
var target map[string]component
|
||||
for _, input := range []string{`{"api":{"name":"ok","extra":true}}`, `{"api":{"Name":"ok"}}`, `{"api":{}}`, `{"api":null}`} {
|
||||
if Decode(strings.NewReader(input), &target, 1024) == nil {
|
||||
t.Errorf("accepted malformed map entry: %s", input)
|
||||
}
|
||||
}
|
||||
if err := Decode(strings.NewReader(`{"api":{"name":"ok"}}`), &target, 1024); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,191 @@
|
||||
# deployctl protocol v1 — read-only foundation
|
||||
|
||||
# Supported commands
|
||||
|
||||
The runner supports `version`, `plan`, `verify-plan`, `inspect`, `preflight`, `plan-environment`, `verify-repository`, `verify-artifacts`, `verify-package`
|
||||
and `check-package`.
|
||||
It does not connect to a remote server, invoke Docker, install software, change system
|
||||
configuration, deploy applications, or restore data. `apply`, `upgrade` and `restore`
|
||||
fail closed. `verify-repository` and `verify-artifacts` create private temporary snapshots and invoke
|
||||
local Linux GnuPG; it does not modify APT or persistent keyrings.
|
||||
|
||||
## Build and try
|
||||
|
||||
For staged official Docker metadata authentication, see
|
||||
[verify-repository contract](../internal/aptrepo/README.md). Its output is not
|
||||
accepted as execution authorization by `plan-environment`.
|
||||
|
||||
Requires Go 1.26 or newer. No third-party Go dependencies.
|
||||
|
||||
```powershell
|
||||
go test ./... -count=1
|
||||
go vet ./...
|
||||
go build -o dist/deployctl.exe ./cmd/deployctl
|
||||
Get-Content -Raw protocol/examples/plan-request.json | ./dist/deployctl.exe plan
|
||||
node --test tests/cli-smoke.test.mjs
|
||||
```
|
||||
|
||||
```sh
|
||||
go build -o dist/deployctl ./cmd/deployctl
|
||||
./dist/deployctl plan < protocol/examples/plan-request.json
|
||||
```
|
||||
|
||||
The example hashes are synthetic, not deployable package/image references.
|
||||
|
||||
## Input
|
||||
|
||||
`plan`, `verify-plan`, `verify-package`, `check-package`, `plan-environment`, `verify-repository` and `verify-artifacts` receive one UTF-8 JSON value through
|
||||
stdin, maximum 1 MiB. `version`, `inspect` and `preflight` require no stdin request.
|
||||
Unknown fields, case aliases, duplicate keys, missing fields, nulls, additional
|
||||
JSON values, excessive nesting and malformed types are rejected.
|
||||
Errors go to stderr without echoing input; exit status is nonzero. stdout only
|
||||
contains a JSON response on success (an output transport failure can truncate it).
|
||||
|
||||
`plan` requires exactly the fields in `examples/plan-request.json`:
|
||||
|
||||
- protocolVersion: integer 1.
|
||||
- hostId, instanceId, appId: lower-case ASCII identifiers, first character a-z,
|
||||
remaining characters a-z, 0-9 or hyphen; 1–48 characters.
|
||||
- packageDigest, imageDigest, observedStateDigest: `sha256:` and 64 lowercase hex characters.
|
||||
- domain: explicit lowercase ASCII DNS name, at least two labels, not an IP,
|
||||
wildcard, URL, port or trailing-dot name. IDNs must already be in ASCII form.
|
||||
|
||||
`imageDigest` is a preliminary single-image binding. The deployable package
|
||||
contract will bind all component images (including private databases); this
|
||||
foundation does not claim to represent a complete multi-container release.
|
||||
|
||||
## Plan and verification
|
||||
|
||||
`plan` returns protocolVersion, mode=`offline-preview`, executable=false and plan.
|
||||
The plan contains intent, projectName, dataPath, createdAt, expiresAt and hash.
|
||||
Times are UTC RFC3339 with second precision; the validity window is 15 minutes.
|
||||
Project name is `sd-<instanceId>`, data path is
|
||||
`/var/lib/server-deploy/instances/<instanceId>`. The path is only a preview;
|
||||
symlink-safe filesystem access is a separate executor responsibility.
|
||||
|
||||
Hash is SHA-256 over Go JSON encoding of Plan with an empty hash field, in its
|
||||
declared field order. The CLI is the canonical plan producer; clients transport
|
||||
the returned plan without rebuilding its hash. This is content binding, NOT a
|
||||
signature, authorization, approval token or package trust check.
|
||||
|
||||
`verify-plan` receives exactly `{ "plan": <returned plan>, "current": <intent> }`.
|
||||
It rejects changed identities, state digests, derived resources, hashes, future
|
||||
creation times and expiry (including the exact expiration timestamp).
|
||||
Success returns protocolVersion, mode, valid=true and executable=false.
|
||||
|
||||
Host identity and state digest are caller-supplied offline inputs. Verification
|
||||
does NOT prove anything about a real server. Future apply must independently
|
||||
inspect the host under the host lock, resolve the entire trusted package and
|
||||
revalidate it; offline preview is never permission to deploy.
|
||||
|
||||
## Local prerequisite inspection
|
||||
|
||||
`inspect` takes no stdin request and examines the machine where deployctl runs.
|
||||
It reports OS/architecture, the presence of `/run/systemd/system`, and an executable
|
||||
regular Docker client at `/usr/bin/docker` or `/usr/local/bin/docker`. It does not
|
||||
run that executable or read Docker configuration/credentials. Access failures
|
||||
appear as `unknown`, not `missing`. Non-Linux targets do not probe Linux paths.
|
||||
|
||||
`supportedPlatform` only describes the initial Linux amd64/arm64 runtime family,
|
||||
not distro certification. `deploymentReady` is always false in this foundation.
|
||||
Daemon connectivity, Compose version, host identity, distro acceptance, permissions,
|
||||
disk space, ports, DNS, firewall and gateway validation are listed as unchecked.
|
||||
This report is not the authoritative observed state required by a future apply.
|
||||
|
||||
### Environment proposal
|
||||
|
||||
`preflight` adds local Ubuntu release, effective privilege, Linux `/var/lib`
|
||||
filesystem available bytes and conservative existing-resource observations.
|
||||
Output includes protocolVersion=1, mode=`local-environment-proposal`, observedAt,
|
||||
report and proposal. It does not accept caller-supplied observations over stdin.
|
||||
|
||||
The proposal reports blockers, candidate changes and impacts; executable is always
|
||||
false. Existing/unknown resources are never silently adopted or removed. Candidate
|
||||
steps, when present, are not an executable installation plan: package inventory,
|
||||
version locks, repository trust, host identity and network checks remain unresolved.
|
||||
Exit 0 means the report was produced, not that the host is ready. See the
|
||||
[preflight contract](../internal/preflight/README.md) for exact boundaries.
|
||||
|
||||
### Version-locked environment draft
|
||||
|
||||
`plan-environment` receives `{ "lock": <Docker package lock> }`. See the
|
||||
[lock contract](../internal/installplan/README.md) for all required fields.
|
||||
Local observations are collected by the CLI, not supplied by the caller.
|
||||
Output includes protocolVersion=1, mode=`local-environment-draft`, observedAt,
|
||||
report and draft. The draft includes requestedPackages, lockDigest,
|
||||
observationDigest, impacts, blockers, executable=false and
|
||||
repositoryAuthenticated=false. No shell command, download or installation runs.
|
||||
|
||||
Preflight now includes a local dpkg inventory observation. Relevant existing or
|
||||
residual packages prevent fresh-install candidates; unknown status/journal cannot
|
||||
be treated as absence. The lock is only syntactically pinned to an allowed source:
|
||||
repository signature, signed metadata chain, package bytes, freshness and dependency
|
||||
resolution remain blockers. This is not a complete or authorized APT transaction.
|
||||
|
||||
## Local application package verification
|
||||
|
||||
`verify-package` reads `{ "directory": "<absolute package directory>",
|
||||
"expectedDigest": "sha256:<64 lowercase hex>" }` from stdin.
|
||||
The expected digest pins the exact manifest.json bytes; the manifest pins every
|
||||
payload file and every component's image reference. Obtain the expected digest
|
||||
through a trusted channel, not from the untrusted package being checked.
|
||||
|
||||
The output includes the verified manifest, manifest digest, payload file count,
|
||||
verified=true, executable=false and publisherAuthenticated=false. File contents
|
||||
are never echoed. Integrity is not publisher authentication, Compose-policy
|
||||
validation, container-image availability, backup compatibility or deployment safety.
|
||||
|
||||
Packages must be immutable, trusted staging directories during verification.
|
||||
The internal verifier returns the verified payload snapshot for future consumers;
|
||||
execution must not reopen mutable source files after a check. No package extraction,
|
||||
download, signature trust-store management or image pull is implemented yet.
|
||||
|
||||
### Manifest contract
|
||||
|
||||
Every field is required; unknown fields are rejected:
|
||||
|
||||
- `protocolVersion`: 1; `runtime`: `compose`.
|
||||
- `appId`: the same lowercase identifier rules as the offline intent.
|
||||
- `version`: three numeric components separated by dots (no prerelease suffix).
|
||||
- `entrypoint`: the path of one declared payload file.
|
||||
- `platforms`: a nonempty unique subset of `linux/amd64`, `linux/arm64`.
|
||||
- `components`: 1–32 unique `{name, image}` entries. Names use identifier rules;
|
||||
images use lowercase repository paths followed by `@sha256:<64 lowercase hex>`.
|
||||
This initial lexical subset excludes tags and registry ports; it is not a
|
||||
complete Docker reference parser or a check against the Compose services.
|
||||
- `files`: 1–128 unique `{path, digest}` entries. Digests pin payload bytes;
|
||||
`manifest.json` itself must not appear in this list.
|
||||
|
||||
Paths are lowercase portable relative slash paths, at most 240 bytes overall and
|
||||
100 bytes per segment. Segments begin with ASCII alphanumeric characters and use
|
||||
only ASCII alphanumeric, dot, underscore or hyphen. Dotfiles, traversal, backslashes,
|
||||
colons, Windows device names and trailing dots are rejected. The directory must
|
||||
contain only `manifest.json`, listed payload files and their required parents.
|
||||
|
||||
Limits are 1 MiB for the manifest, 4 MiB per payload and 16 MiB for total payloads.
|
||||
Symlinks and special files are rejected. See `internal/appbundle/README.md` for
|
||||
filesystem trust boundaries and Linux-specific rejection tests.
|
||||
|
||||
## Restricted Compose policy check
|
||||
|
||||
`check-package` accepts the same request as `verify-package`. It performs the
|
||||
complete integrity check first, then validates the returned entrypoint bytes
|
||||
against `isolated-compose-v1`. No files are reopened and no Docker command runs.
|
||||
|
||||
Success returns protocolVersion=1, policyPassed=true, profile, the manifest digest,
|
||||
executable=false and publisherAuthenticated=false. Failure returns no JSON result
|
||||
and a fixed redacted diagnostic. `verify-package` remains integrity-only; a package
|
||||
may pass that command while failing `check-package`.
|
||||
|
||||
The initial profile accepts only strict JSON, pinned manifest-matching services,
|
||||
explicit non-root identities, read-only roots, dropped capabilities, no new
|
||||
privileges, a private internal backend network and plain project-local named
|
||||
volumes. Every field in the profile is required; other fields are rejected.
|
||||
See [exact shape and limitations](../internal/composepolicy/README.md).
|
||||
|
||||
This is not a general YAML validator or completed application adapter. Traefik
|
||||
routes, environment/secrets, application-specific permissions, Docker version
|
||||
compatibility, volume ownership and resource limits are not implemented here.
|
||||
Passing never establishes that an application can start or its data can be safely
|
||||
upgraded/restored. Future execution still requires locked host/resource checks,
|
||||
trusted images, a fixed Compose invocation and exact snapshot binding.
|
||||
@@ -0,0 +1,10 @@
|
||||
{
|
||||
"protocolVersion": 1,
|
||||
"hostId": "example-host",
|
||||
"instanceId": "git-one",
|
||||
"appId": "gitea",
|
||||
"packageDigest": "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
||||
"imageDigest": "sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
|
||||
"domain": "git.example.com",
|
||||
"observedStateDigest": "sha256:cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc"
|
||||
}
|
||||
@@ -0,0 +1,32 @@
|
||||
#!/usr/bin/env bash
|
||||
# Local opt-in public deb download/hash test. No extraction, APT or installation.
|
||||
# Called by probe-docker-repository.sh after authenticated metadata verification.
|
||||
set -euo pipefail
|
||||
[[ $# == 3 && -f "$1" && -d "$2" ]]
|
||||
binary=$1
|
||||
metadata_dir=$2
|
||||
metadata_request=$3
|
||||
probe_dir=$(mktemp -d /tmp/server-deploy-artifacts.XXXXXX)
|
||||
chmod 700 "$probe_dir"
|
||||
artifact_dir="$probe_dir/debs"
|
||||
mkdir -m 700 "$artifact_dir"
|
||||
# Only authenticated lock paths, fixed origin, no caller-provided download URLs.
|
||||
python3 -I -c 'import json,re,sys; r=json.load(sys.stdin); assert r["repositoryAuthenticated"] is True; p=r["lock"]["packages"]; assert len(p)==5; [None if re.fullmatch(r"dists/resolute/pool/stable/amd64/[a-z0-9.+~_-]+\.deb",x["filename"]) and 0<x["size"]<=536870912 else sys.exit(1) for x in p]; [print(x["filename"],x["size"]) for x in p]' < "$metadata_dir/result.json" > "$probe_dir/downloads"
|
||||
while read -r filename size; do
|
||||
curl --fail --silent --show-error --proto '=https' --max-time 90 --max-filesize "$size" \
|
||||
"https://download.docker.com/linux/ubuntu/$filename" -o "$artifact_dir/${filename##*/}"
|
||||
done < "$probe_dir/downloads"
|
||||
request=$(printf '%s\n' "$metadata_request" | python3 -I -c 'import json,sys; r=json.load(sys.stdin); r["artifactDirectory"]=sys.argv[1]; print(json.dumps(r))' "$artifact_dir")
|
||||
printf '%s\n' "$request" | "$binary" verify-artifacts > "$probe_dir/result.json"
|
||||
python3 -I -c 'import json,sys; r=json.load(sys.stdin); assert r["repositoryAuthenticated"] is True; assert r["packageBytesVerified"] is True; assert r["executable"] is False; assert len(r["lock"]["packages"])==5' < "$probe_dir/result.json"
|
||||
cat "$probe_dir/result.json"
|
||||
# Same-size mutation: digest comparison, not an earlier size check, must reject.
|
||||
first=$(head -n 1 "$probe_dir/downloads")
|
||||
filename=${first% *}
|
||||
printf 'X' | dd of="$artifact_dir/${filename##*/}" bs=1 count=1 conv=notrunc status=none
|
||||
if printf '%s\n' "$request" | "$binary" verify-artifacts > "$probe_dir/rejected.json"; then
|
||||
printf 'ERROR: same-size deb tamper accepted\n' >&2
|
||||
exit 1
|
||||
fi
|
||||
[[ ! -s "$probe_dir/rejected.json" ]]
|
||||
printf 'Five real deb files verified; same-size tamper rejected. Local artifacts: %s\n' "$artifact_dir"
|
||||
@@ -0,0 +1,55 @@
|
||||
#!/usr/bin/env bash
|
||||
# Local metadata-only probe. No APT changes or software installation.
|
||||
set -euo pipefail
|
||||
task_dir=$(mktemp -d /tmp/server-deploy-repository.XXXXXX)
|
||||
chmod 700 "$task_dir"
|
||||
base=https://download.docker.com/linux/ubuntu
|
||||
curl --fail --silent --show-error --proto '=https' --max-time 45 --max-filesize 1048576 "$base/gpg" -o "$task_dir/docker.asc"
|
||||
curl --fail --silent --show-error --proto '=https' --max-time 45 --max-filesize 1048576 "$base/dists/resolute/Release" -o "$task_dir/Release"
|
||||
curl --fail --silent --show-error --proto '=https' --max-time 45 --max-filesize 65536 "$base/dists/resolute/Release.gpg" -o "$task_dir/Release.gpg"
|
||||
curl --fail --silent --show-error --proto '=https' --max-time 45 --max-filesize 16777216 "$base/dists/resolute/stable/binary-amd64/Packages" -o "$task_dir/Packages"
|
||||
gpg --batch --no-options --homedir "$task_dir" --with-colons --show-keys "$task_dir/docker.asc"
|
||||
sha256sum "$task_dir/docker.asc"
|
||||
gpg --batch --no-options --homedir "$task_dir" --dearmor --output "$task_dir/docker.gpg" "$task_dir/docker.asc"
|
||||
gpgv --homedir "$task_dir" --keyring "$task_dir/docker.gpg" --status-fd 1 "$task_dir/Release.gpg" "$task_dir/Release"
|
||||
head -n 12 "$task_dir/Release"
|
||||
awk '/^Package: (docker-ce|docker-ce-cli|containerd.io|docker-buildx-plugin|docker-compose-plugin)$/ {name=$2} /^Version:/ && name!="" {print name, $2; name=""}' "$task_dir/Packages"
|
||||
# Optional prebuilt local binary: verifies the same downloaded snapshots. This
|
||||
# selects explicit fixture versions, not an installation or latest-version policy.
|
||||
if [[ $# == 1 ]]; then
|
||||
engine=$(awk '/^Package: docker-ce$/ {p=1} p && /^Version:/ {print $2; exit}' "$task_dir/Packages")
|
||||
containerd=$(awk '/^Package: containerd.io$/ {p=1} p && /^Version:/ {print $2; exit}' "$task_dir/Packages")
|
||||
buildx=$(awk '/^Package: docker-buildx-plugin$/ {p=1} p && /^Version:/ {print $2; exit}' "$task_dir/Packages")
|
||||
compose=$(awk '/^Package: docker-compose-plugin$/ {p=1} p && /^Version:/ {print $2; exit}' "$task_dir/Packages")
|
||||
for version in "$engine" "$containerd" "$buildx" "$compose"; do
|
||||
[[ "$version" =~ ^[0-9A-Za-z.+:~_-]+$ ]] || exit 1
|
||||
done
|
||||
request=$(printf '{"directory":"%s","suite":"resolute","architecture":"amd64","versions":{"docker-ce":"%s","docker-ce-cli":"%s","containerd.io":"%s","docker-buildx-plugin":"%s","docker-compose-plugin":"%s"}}' "$task_dir" "$engine" "$engine" "$containerd" "$buildx" "$compose")
|
||||
printf '%s\n' "$request" | "$1" verify-repository > "$task_dir/result.json"
|
||||
python3 -I -c 'import json,sys; r=json.load(sys.stdin); p=sys.argv[1:]; assert r["repositoryAuthenticated"] is True; assert r["executable"] is False; assert r["packageBytesVerified"] is False; assert {x["name"]:x["version"] for x in r["lock"]["packages"]} == dict(zip(["docker-ce","docker-ce-cli","containerd.io","docker-buildx-plugin","docker-compose-plugin"],p)); assert len(r["lock"]["packages"])==5' "$engine" "$engine" "$containerd" "$buildx" "$compose" < "$task_dir/result.json"
|
||||
cat "$task_dir/result.json"
|
||||
if [[ -n "${DEPLOYCTL_TEST_GO:-}" ]]; then
|
||||
DEPLOYCTL_REPOSITORY_FIXTURE="$task_dir" "$DEPLOYCTL_TEST_GO" test ./internal/aptrepo -run TestStagedSignatureIntegration -count=1 -v
|
||||
fi
|
||||
if [[ "${DEPLOYCTL_ONLINE_ARTIFACT_PROBE:-0}" == 1 ]]; then
|
||||
bash scripts/probe-docker-artifacts.sh "$1" "$task_dir" "$request"
|
||||
fi
|
||||
cp -- "$task_dir/Packages" "$task_dir/Packages.original"
|
||||
awk '/^SHA256:/ && !changed {c=substr($2,1,1); $2=(c=="0" ? "1" : "0") substr($2,2); changed=1} {print}' "$task_dir/Packages.original" > "$task_dir/Packages"
|
||||
if printf '%s\n' "$request" | "$1" verify-repository > "$task_dir/rejected.json"; then
|
||||
printf 'ERROR: tampered Packages accepted\n' >&2
|
||||
exit 1
|
||||
fi
|
||||
[[ ! -s "$task_dir/rejected.json" ]]
|
||||
mv -- "$task_dir/Packages.original" "$task_dir/Packages"
|
||||
printf 'Tampered Packages rejected as expected.\n'
|
||||
cp -- "$task_dir/Release" "$task_dir/Release.original"
|
||||
awk '/^Date:/ {n=split($0,parts,":"); c=substr(parts[n],1,1); parts[n]=(c=="0" ? "1" : "0") substr(parts[n],2); line=parts[1]; for(i=2;i<=n;i++) line=line ":" parts[i]; print line; next} {print}' "$task_dir/Release.original" > "$task_dir/Release"
|
||||
if printf '%s\n' "$request" | "$1" verify-repository > "$task_dir/rejected.json"; then
|
||||
printf 'ERROR: tampered Release accepted\n' >&2
|
||||
exit 1
|
||||
fi
|
||||
[[ ! -s "$task_dir/rejected.json" ]]
|
||||
printf 'Tampered signed Release rejected as expected.\n'
|
||||
fi
|
||||
printf 'Local temporary probe directory: %s\n' "$task_dir"
|
||||
@@ -0,0 +1,23 @@
|
||||
#!/usr/bin/env bash
|
||||
# Run on Linux/WSL with a previously SHA-256-verified official Go tarball.
|
||||
# The caller owns download provenance. No package installation or production SSH.
|
||||
set -euo pipefail
|
||||
if [[ $# != 1 || ! -f "$1" ]]; then
|
||||
printf 'usage: bash scripts/verify-linux.sh /path/to/verified-go.linux-amd64.tar.gz\n' >&2
|
||||
exit 2
|
||||
fi
|
||||
task_dir=$(mktemp -d /tmp/server-deploy-test.XXXXXX)
|
||||
tar -xzf "$1" -C "$task_dir"
|
||||
export GOCACHE="$task_dir/cache"
|
||||
cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.."
|
||||
"$task_dir/go/bin/go" test ./... -count=1 -v -timeout=60s
|
||||
"$task_dir/go/bin/go" vet ./...
|
||||
"$task_dir/go/bin/go" test -race ./... -count=1 -timeout=60s
|
||||
"$task_dir/go/bin/go" build -o "$task_dir/deployctl" ./cmd/deployctl
|
||||
"$task_dir/deployctl" plan < protocol/examples/plan-request.json
|
||||
"$task_dir/deployctl" inspect
|
||||
"$task_dir/deployctl" preflight
|
||||
if [[ "${DEPLOYCTL_ONLINE_REPOSITORY_PROBE:-0}" == 1 ]]; then
|
||||
DEPLOYCTL_TEST_GO="$task_dir/go/bin/go" bash scripts/probe-docker-repository.sh "$task_dir/deployctl"
|
||||
fi
|
||||
printf 'Verification artifacts retained at: %s\n' "$task_dir"
|
||||
@@ -0,0 +1,195 @@
|
||||
// Run after building the native executable. No npm packages are required.
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { spawnSync } from 'node:child_process';
|
||||
import { readFileSync, writeFileSync, mkdtempSync, readdirSync, rmSync } from 'node:fs';
|
||||
import { createHash } from 'node:crypto';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { resolve, join } from 'node:path';
|
||||
|
||||
const binary = resolve(process.env.DEPLOYCTL_BIN ?? (process.platform === 'win32' ? 'dist/deployctl.exe' : 'dist/deployctl'));
|
||||
const intent = JSON.parse(readFileSync(new URL('../protocol/examples/plan-request.json', import.meta.url), 'utf8'));
|
||||
|
||||
test('real CLI artifact verification rejects missing metadata without writes', () => {
|
||||
const directory = mkdtempSync(join(tmpdir(), 'deployctl-artifact-smoke-'));
|
||||
try {
|
||||
const result = spawnSync(binary, ['verify-artifacts'], {
|
||||
encoding:'utf8', timeout:10000,
|
||||
input:JSON.stringify({directory, artifactDirectory:directory, suite:'resolute', architecture:'amd64', versions:{}}),
|
||||
});
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.equal(result.stdout, '');
|
||||
assert.equal(result.stderr.includes('unsupported command'), false);
|
||||
assert.equal(result.stderr.includes(directory), false);
|
||||
assert.deepEqual(readdirSync(directory), []);
|
||||
} finally {
|
||||
rmSync(directory, {recursive:true});
|
||||
}
|
||||
});
|
||||
|
||||
test('real CLI rejects caller-asserted repository authentication', () => {
|
||||
const result = spawnSync(binary, ['verify-repository'], {
|
||||
encoding: 'utf8', timeout: 10000,
|
||||
input: JSON.stringify({directory:'secret-relative', suite:'resolute', architecture:'amd64',
|
||||
versions:{}, repositoryAuthenticated:true}),
|
||||
});
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.equal(result.stdout, '');
|
||||
assert.equal(result.stderr.includes('secret-relative'), false);
|
||||
});
|
||||
|
||||
test('real CLI environment draft rejects malformed locks without writing', () => {
|
||||
const directory = mkdtempSync(join(tmpdir(), 'deployctl-environment-smoke-'));
|
||||
try {
|
||||
const probe = spawnSync(binary, ['preflight'], { encoding: 'utf8', timeout: 10000 });
|
||||
assert.equal(probe.status, 0, probe.stderr);
|
||||
const report = JSON.parse(probe.stdout).report;
|
||||
const suite = report.runtime.os === 'linux' ? report.distribution.codename : 'resolute';
|
||||
const arch = report.runtime.os === 'linux' ? report.runtime.architecture : 'amd64';
|
||||
const lock = { protocolVersion: 1, repository: 'https://download.docker.com/linux/ubuntu',
|
||||
suite, architecture: arch, releaseDigest: `sha256:${'a'.repeat(64)}`,
|
||||
packages: ['docker-ce', 'docker-ce-cli', 'containerd.io', 'docker-buildx-plugin', 'docker-compose-plugin'].map(name => ({
|
||||
name, version: '1.2.3-1', filename: `dists/${suite}/pool/stable/${arch}/${name}_1.2.3-1_${arch}.deb`, digest: `sha256:${'b'.repeat(64)}`, size: 123,
|
||||
})),
|
||||
};
|
||||
const run = () => spawnSync(binary, ['plan-environment'], { cwd: directory, encoding: 'utf8', timeout: 10000, input: JSON.stringify({lock}) });
|
||||
const result = run();
|
||||
assert.equal(result.status, 0, result.stderr);
|
||||
const draft = JSON.parse(result.stdout).draft;
|
||||
assert.equal(draft.executable, false);
|
||||
assert.equal(draft.repositoryAuthenticated, false);
|
||||
assert.ok(draft.blockers.includes('dependency_transaction_unresolved'));
|
||||
assert.equal(draft.requestedPackages.length, 5);
|
||||
lock.packages[0].version = 'secret;reboot';
|
||||
const invalid = run();
|
||||
assert.notEqual(invalid.status, 0);
|
||||
assert.equal(invalid.stdout, '');
|
||||
assert.equal(invalid.stderr.includes('secret'), false);
|
||||
assert.deepEqual(readdirSync(directory), []);
|
||||
} finally {
|
||||
rmSync(directory, { recursive: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('real CLI preflight reports blockers without creating files', () => {
|
||||
const directory = mkdtempSync(join(tmpdir(), 'deployctl-preflight-smoke-'));
|
||||
try {
|
||||
const result = spawnSync(binary, ['preflight'], { cwd: directory, encoding: 'utf8', timeout: 10000 });
|
||||
assert.equal(result.status, 0, result.stderr);
|
||||
const response = JSON.parse(result.stdout);
|
||||
assert.equal(response.mode, 'local-environment-proposal');
|
||||
assert.equal(response.proposal.executable, false);
|
||||
assert.ok(response.proposal.blockers.includes('package_versions_unresolved'));
|
||||
assert.ok(response.proposal.blockers.includes('host_identity_unverified'));
|
||||
assert.deepEqual(readdirSync(directory), []);
|
||||
} finally {
|
||||
rmSync(directory, { recursive: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('real CLI previews and verifies without writing into its working directory', () => {
|
||||
const directory = mkdtempSync(join(tmpdir(), 'deployctl-smoke-'));
|
||||
const run = (command, input) => spawnSync(binary, [command], {
|
||||
input: JSON.stringify(input), encoding: 'utf8', cwd: directory, timeout: 10000,
|
||||
});
|
||||
try {
|
||||
const preview = run('plan', intent);
|
||||
assert.equal(preview.error, undefined);
|
||||
assert.equal(preview.status, 0, preview.stderr);
|
||||
const response = JSON.parse(preview.stdout);
|
||||
assert.equal(response.executable, false);
|
||||
assert.equal(response.mode, 'offline-preview');
|
||||
assert.equal(response.plan.projectName, 'sd-git-one');
|
||||
const check = run('verify-plan', { plan: response.plan, current: intent });
|
||||
assert.equal(check.status, 0, check.stderr);
|
||||
assert.equal(JSON.parse(check.stdout).valid, true);
|
||||
const changed = run('verify-plan', { plan: response.plan, current: { ...intent, hostId: 'different-host' } });
|
||||
assert.notEqual(changed.status, 0);
|
||||
assert.equal(changed.stdout, '');
|
||||
const apply = run('apply', response.plan);
|
||||
assert.notEqual(apply.status, 0);
|
||||
assert.equal(apply.stdout, '');
|
||||
assert.deepEqual(readdirSync(directory), []);
|
||||
} finally {
|
||||
// Only remove the unique test directory this test just created.
|
||||
rmSync(directory, { recursive: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('real CLI inspects locally and verifies packages without claiming execution safety', () => {
|
||||
const inspection = spawnSync(binary, ['inspect'], { encoding: 'utf8', timeout: 10000 });
|
||||
assert.equal(inspection.status, 0, inspection.stderr);
|
||||
assert.equal(JSON.parse(inspection.stdout).deploymentReady, false);
|
||||
const directory = mkdtempSync(join(tmpdir(), 'deployctl-package-smoke-'));
|
||||
const digest = bytes => `sha256:${createHash('sha256').update(bytes).digest('hex')}`;
|
||||
try {
|
||||
const payload = 'services: {}\n';
|
||||
const manifest = JSON.stringify({
|
||||
protocolVersion: 1, appId: 'example', version: '1.0.0', runtime: 'compose',
|
||||
entrypoint: 'compose.yaml', platforms: ['linux/amd64'],
|
||||
components: [{ name: 'server', image: `example/server@sha256:${'a'.repeat(64)}` }],
|
||||
files: [{ path: 'compose.yaml', digest: digest(payload) }],
|
||||
});
|
||||
writeFileSync(join(directory, 'manifest.json'), manifest);
|
||||
writeFileSync(join(directory, 'compose.yaml'), payload);
|
||||
const result = spawnSync(binary, ['verify-package'], {
|
||||
encoding: 'utf8', timeout: 10000,
|
||||
input: JSON.stringify({ directory, expectedDigest: digest(manifest) }),
|
||||
});
|
||||
assert.equal(result.status, 0, result.stderr);
|
||||
const verified = JSON.parse(result.stdout);
|
||||
assert.equal(verified.verified, true);
|
||||
assert.equal(verified.executable, false);
|
||||
assert.equal(verified.publisherAuthenticated, false);
|
||||
assert.deepEqual(readdirSync(directory).sort(), ['compose.yaml', 'manifest.json']);
|
||||
assert.equal(readFileSync(join(directory, 'compose.yaml'), 'utf8'), payload);
|
||||
} finally {
|
||||
rmSync(directory, { recursive: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('real CLI separates package integrity from restricted policy and rejects tampering', () => {
|
||||
const directory = mkdtempSync(join(tmpdir(), 'deployctl-policy-smoke-'));
|
||||
const digest = bytes => `sha256:${createHash('sha256').update(bytes).digest('hex')}`;
|
||||
const image = `example/server@sha256:${'a'.repeat(64)}`;
|
||||
const document = {
|
||||
services: { server: { image, user: '1000:1000', read_only: true, cap_drop: ['ALL'],
|
||||
security_opt: ['no-new-privileges:true'], restart: 'no', networks: ['backend'], volumes: [] } },
|
||||
networks: { backend: { internal: true } }, volumes: {},
|
||||
};
|
||||
const run = (command, expectedDigest) => spawnSync(binary, [command], {
|
||||
encoding: 'utf8', timeout: 10000, cwd: directory,
|
||||
input: JSON.stringify({ directory, expectedDigest }),
|
||||
});
|
||||
const writePackage = () => {
|
||||
const payload = JSON.stringify(document);
|
||||
const manifest = JSON.stringify({ protocolVersion: 1, appId: 'example', version: '1.0.0', runtime: 'compose',
|
||||
entrypoint: 'compose.json', platforms: ['linux/amd64'], components: [{ name: 'server', image }],
|
||||
files: [{ path: 'compose.json', digest: digest(payload) }],
|
||||
});
|
||||
writeFileSync(join(directory, 'manifest.json'), manifest);
|
||||
writeFileSync(join(directory, 'compose.json'), payload);
|
||||
return digest(manifest);
|
||||
};
|
||||
try {
|
||||
const pin = writePackage();
|
||||
const check = run('check-package', pin);
|
||||
assert.equal(check.status, 0, check.stderr);
|
||||
const result = JSON.parse(check.stdout);
|
||||
assert.equal(result.policyPassed, true);
|
||||
assert.equal(result.executable, false);
|
||||
assert.equal(result.publisherAuthenticated, false);
|
||||
assert.equal(result.digest, pin);
|
||||
writeFileSync(join(directory, 'compose.json'), '{}');
|
||||
assert.notEqual(run('check-package', pin).status, 0);
|
||||
document.services.server.privileged = true;
|
||||
const unsafePin = writePackage();
|
||||
assert.equal(run('verify-package', unsafePin).status, 0);
|
||||
const rejected = run('check-package', unsafePin);
|
||||
assert.notEqual(rejected.status, 0);
|
||||
assert.equal(rejected.stdout, '');
|
||||
assert.deepEqual(readdirSync(directory).sort(), ['compose.json', 'manifest.json']);
|
||||
} finally {
|
||||
rmSync(directory, { recursive: true });
|
||||
}
|
||||
});
|
||||
@@ -25,7 +25,11 @@ SENDS_ALLOWED=true
|
||||
# ===== 镜像配置 =====
|
||||
|
||||
# Vaultwarden 镜像
|
||||
VAULTWARDEN_IMAGE=vaultwarden/server:latest
|
||||
# 留空或填 latest 时,deploy.sh 会自动查询 GitHub 最新 release 并把具体版本号写回这里。
|
||||
# 不要长期使用 latest:该标签只在 docker compose pull 时才重新解析,容器不重建就会
|
||||
# 一直跑旧版本,而且从配置上看不出实际运行的是哪一版。
|
||||
# 后续升级请用 upgrade.sh,它会自动改写此处的版本号。
|
||||
VAULTWARDEN_IMAGE=
|
||||
|
||||
# ===== 目录与端口 =====
|
||||
|
||||
|
||||
+112
-16
@@ -16,7 +16,7 @@ Bitwarden 兼容的自托管密码管理器,轻量、安全、功能完整。
|
||||
|
||||
| 组件 | 版本 | 说明 |
|
||||
|------|------|------|
|
||||
| Vaultwarden | latest | Bitwarden 兼容服务端(Rust 实现) |
|
||||
| Vaultwarden | 固定版本号 | Bitwarden 兼容服务端(Rust 实现),由 `upgrade.sh` 升级 |
|
||||
| SQLite | 内置 | 轻量数据库,无需额外部署 |
|
||||
| Nginx | 系统包 | 反向代理 + HTTPS(Bitwarden 客户端必须 HTTPS) |
|
||||
| Docker | 最新版 | 容器运行环境 |
|
||||
@@ -35,6 +35,7 @@ vaultwarden/
|
||||
├── docker-compose.yml # 容器编排
|
||||
├── .env.example # 配置模板
|
||||
├── deploy.sh # 一键部署脚本
|
||||
├── upgrade.sh # 安全升级脚本(带备份、校验、自动回滚)
|
||||
├── backup.sh # 备份脚本
|
||||
├── uninstall.sh # 完全卸载脚本
|
||||
├── nginx/
|
||||
@@ -97,9 +98,12 @@ bash deploy.sh
|
||||
cd /opt/vaultwarden
|
||||
# 编辑 .env,将 SIGNUPS_ALLOWED 改为 false
|
||||
vi .env
|
||||
docker compose restart
|
||||
docker compose up -d # 注意:不是 restart
|
||||
```
|
||||
|
||||
> **不要用 `docker compose restart`**。环境变量是在容器**创建**时注入的,`restart` 只是重启
|
||||
> 原容器、不会重建,改了 `.env` 也不会生效。必须用 `up -d` 让 compose 检测到配置变更并重建容器。
|
||||
|
||||
### 第五步:安装客户端
|
||||
|
||||
1. 下载 Bitwarden 客户端:https://bitwarden.com/download/
|
||||
@@ -224,23 +228,65 @@ docker compose up -d
|
||||
|
||||
### 升级
|
||||
|
||||
使用 `upgrade.sh` 升级,**不要**手动 `docker compose pull && up -d`(见下方「为什么不要手动升级」)。
|
||||
|
||||
```bash
|
||||
cd /opt/vaultwarden
|
||||
|
||||
# 1. 备份数据
|
||||
bash backup.sh
|
||||
|
||||
# 2. 拉取新镜像
|
||||
docker compose pull
|
||||
|
||||
# 3. 重启
|
||||
docker compose up -d
|
||||
|
||||
# 4. 检查运行状态
|
||||
docker compose ps
|
||||
docker compose logs --tail 20
|
||||
bash upgrade.sh --check # 先看看会发生什么,不做任何改动
|
||||
bash upgrade.sh # 正式升级到 GitHub 最新 release
|
||||
```
|
||||
|
||||
脚本执行流程:
|
||||
|
||||
| 步骤 | 动作 | 失败时 |
|
||||
|------|------|--------|
|
||||
| 1 | 预检:root、依赖命令、数据目录、磁盘空间 | 直接退出,未动服务 |
|
||||
| 2 | 拉取新镜像 | 直接退出,**服务零影响**,可安全重试 |
|
||||
| 3 | 把当前镜像打上 `pre-upgrade-<时间戳>` 标签作为回滚锚点 | — |
|
||||
| 4 | 停止容器 | 自动拉起原服务 |
|
||||
| 5 | WAL checkpoint + `integrity_check` + 冷备份整个数据目录 | 自动拉起原服务 |
|
||||
| 6 | 切换镜像并 `--force-recreate` 重建容器 | 自动回滚 |
|
||||
| 7 | 校验容器实际镜像 == 目标镜像 | 自动回滚 |
|
||||
| 8 | 校验 `/alive`、新旧 prelogin 路由、Nginx 反代 | 自动回滚 |
|
||||
| 9 | 比对升级前后各表行数,任一表减少即判定数据丢失 | 自动回滚 |
|
||||
|
||||
常用选项:
|
||||
|
||||
```bash
|
||||
bash upgrade.sh --check # 只检查,零改动
|
||||
bash upgrade.sh --version 1.37.0 # 升级到指定版本
|
||||
bash upgrade.sh --yes # 跳过交互确认(自动化场景)
|
||||
bash upgrade.sh --rollback # 回滚到上次升级前的状态
|
||||
```
|
||||
|
||||
升级完成后请**登录 Web 端确认数据无误**,再清理回滚镜像:
|
||||
|
||||
```bash
|
||||
docker image rm vaultwarden/server:pre-upgrade-<时间戳>
|
||||
```
|
||||
|
||||
### 回滚
|
||||
|
||||
```bash
|
||||
cd /opt/vaultwarden
|
||||
bash upgrade.sh --rollback
|
||||
```
|
||||
|
||||
回滚会读取 `.upgrade-state` 里记录的备份和镜像锚点,恢复数据目录并切回旧镜像。
|
||||
|
||||
> **注意**:回滚会丢弃升级后新增/修改的密码条目。
|
||||
> 回滚**不会删除**升级后的数据,而是把它改名保留为 `<数据目录>.failed-<时间戳>`,确认无需后再自行删除。
|
||||
|
||||
### 为什么不要手动升级
|
||||
|
||||
手动 `docker compose pull && docker compose up -d` 有两个坑,都会导致「看起来升级了,其实没有」:
|
||||
|
||||
1. **停止状态的容器不会因镜像变更而重建**。如果先 `stop` 再 `up -d`,compose 只是把原容器重新 `start`,仍然跑旧镜像。必须 `--force-recreate`。
|
||||
2. **环境变量优先级高于 `.env` 文件**。如果当前 shell 里已经 `export` 过 `VAULTWARDEN_IMAGE`(比如脚本里 `source .env` 过),改 `.env` 文件无效,compose 依然用旧值。
|
||||
|
||||
`upgrade.sh` 处理了这两点,并在启动后强制校验「容器实际运行的镜像 ID == 目标镜像 ID」,不匹配直接回滚。
|
||||
|
||||
### 停止 / 启动
|
||||
|
||||
```bash
|
||||
@@ -316,10 +362,24 @@ rm -rf /opt/vaultwarden
|
||||
注册好所有需要的账号后:
|
||||
|
||||
```bash
|
||||
# .env 中设置
|
||||
SIGNUPS_ALLOWED=false
|
||||
cd /opt/vaultwarden
|
||||
sed -i 's/^SIGNUPS_ALLOWED=.*/SIGNUPS_ALLOWED=false/' .env
|
||||
docker compose up -d # 必须 up -d,restart 不生效
|
||||
```
|
||||
|
||||
验证是否真的关闭了(应返回 `Registration not allowed or user already exists`):
|
||||
|
||||
```bash
|
||||
curl -s -X POST http://127.0.0.1:8080/identity/accounts/register/send-verification-email \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"email":"probe@example.invalid","name":"probe"}'
|
||||
```
|
||||
|
||||
> **不要用 `/api/config` 里的 `disableUserRegistration` 判断**。该字段来自
|
||||
> `is_signup_disabled()`,含义是「是否隐藏 UI 上的注册链接」,而非「注册是否被拒绝」。
|
||||
> 当 `INVITATIONS_ALLOWED=true` 且未配置 SMTP 时,即使 `SIGNUPS_ALLOWED=false`
|
||||
> 它也会显示为 `false`(因为管理员邀请的用户仍需走注册流程)。真正的拦截在注册接口里。
|
||||
|
||||
### 2. 限制管理面板访问
|
||||
|
||||
在 `nginx/vaultwarden.conf` 中取消注释 `/admin` 的 IP 限制部分,仅允许你的 IP 访问。
|
||||
@@ -347,6 +407,42 @@ docker compose ps
|
||||
curl http://127.0.0.1:8080/alive
|
||||
```
|
||||
|
||||
### 客户端登录报错,控制台 404 /identity/accounts/prelogin/password
|
||||
|
||||
**症状**:已登录的会话正常,但重新登录失败,浏览器控制台显示:
|
||||
|
||||
```
|
||||
POST https://vault.example.com/identity/accounts/prelogin/password 404 (Not Found)
|
||||
```
|
||||
|
||||
**原因**:服务端版本过旧。Bitwarden 客户端从 v2026.4.0 起改用 `/identity/accounts/prelogin/password`
|
||||
这个新路由,而 Vaultwarden 在 **1.36.0** 才实现它。客户端会自动更新,服务端不会。
|
||||
|
||||
**确认**:
|
||||
|
||||
```bash
|
||||
# 看服务端实到底跑的是哪个版本
|
||||
curl -s https://vault.example.com/api/config | grep -o '"gitHash":"[^"]*"'
|
||||
|
||||
# 直接探测新旧两个路由
|
||||
curl -s -o /dev/null -w '%{http_code}\n' -X POST -H 'Content-Type: application/json' \
|
||||
-d '{"email":"x@example.com"}' https://vault.example.com/identity/accounts/prelogin/password
|
||||
```
|
||||
|
||||
旧路由返回 200、新路由返回 404,即可确诊。
|
||||
|
||||
**修复**:升级到 1.36.0 以上。
|
||||
|
||||
```bash
|
||||
cd /opt/vaultwarden && bash upgrade.sh
|
||||
```
|
||||
|
||||
如果暂时无法升级,可在 Nginx 的 443 server 块内加一条重写作为临时过渡:
|
||||
|
||||
```nginx
|
||||
rewrite ^/identity/accounts/prelogin/password/?$ /identity/accounts/prelogin last;
|
||||
```
|
||||
|
||||
### 502 Bad Gateway
|
||||
|
||||
```bash
|
||||
|
||||
+84
-3
@@ -24,11 +24,72 @@ source "$BASE_DIR/setup.sh"
|
||||
# Vaultwarden 专用函数
|
||||
# =============================================================
|
||||
|
||||
GITHUB_REPO="dani-garcia/vaultwarden"
|
||||
|
||||
# 生成随机密码
|
||||
generate_password() {
|
||||
openssl rand -base64 32 | tr -d '/+=' | head -c 32
|
||||
}
|
||||
|
||||
# 查询 GitHub 上的最新 release 版本号(不含前缀 v),失败返回非零
|
||||
resolve_latest_version() {
|
||||
local ver
|
||||
ver=$(curl -sf --max-time 20 \
|
||||
"https://api.github.com/repos/${GITHUB_REPO}/releases/latest" \
|
||||
| grep -o '"tag_name"[[:space:]]*:[[:space:]]*"[^"]*"' \
|
||||
| head -1 | sed 's/.*"\([^"]*\)"$/\1/' | sed 's/^v//') || true
|
||||
[ -n "$ver" ] || return 1
|
||||
echo "$ver"
|
||||
}
|
||||
|
||||
# 把镜像固定到具体版本号。
|
||||
# 必须同时更新 .env 和已导出的环境变量:init_env 里 `set -a; source .env` 已把旧值
|
||||
# 导出到本进程,而 docker compose 对环境变量的优先级高于 .env 文件,只改文件不生效。
|
||||
pin_image_version() {
|
||||
local ref="$1"
|
||||
if grep -q '^VAULTWARDEN_IMAGE=' .env; then
|
||||
sed -i "s|^VAULTWARDEN_IMAGE=.*|VAULTWARDEN_IMAGE=${ref}|" .env
|
||||
else
|
||||
printf '\nVAULTWARDEN_IMAGE=%s\n' "$ref" >> .env
|
||||
fi
|
||||
export VAULTWARDEN_IMAGE="$ref"
|
||||
}
|
||||
|
||||
# 确定本次部署使用的镜像版本。
|
||||
# 不使用 latest 标签的原因:它只在 docker compose pull 时才重新解析,容器不重建就会
|
||||
# 一直跑旧版本,且从配置上看不出实际运行的是哪一版。
|
||||
setup_image_version() {
|
||||
local current="${VAULTWARDEN_IMAGE:-}"
|
||||
|
||||
# 已固定到具体版本 → 尊重现有配置,升级交给 upgrade.sh
|
||||
if [ -n "$current" ] && [[ "$current" != *":latest" ]]; then
|
||||
log " 镜像: ${current}"
|
||||
return 0
|
||||
fi
|
||||
|
||||
# 已有实例却仍在用 latest:此时固定版本等同于一次升级,
|
||||
# 而升级必须走 upgrade.sh(带冷备份、完整性校验、自动回滚),这里不擅自处理。
|
||||
if docker compose ps -a -q vaultwarden 2>/dev/null | grep -q .; then
|
||||
warn "检测到已存在的实例,且 .env 仍在使用 latest 标签"
|
||||
warn "latest 不会自动更新,也看不出实际运行版本,建议固定到具体版本号:"
|
||||
warn " bash upgrade.sh # 安全升级到最新版,并自动写回版本号"
|
||||
warn "本次部署不改动镜像配置"
|
||||
return 0
|
||||
fi
|
||||
|
||||
log "正在查询 Vaultwarden 最新版本..."
|
||||
local ver
|
||||
if ! ver="$(resolve_latest_version)"; then
|
||||
warn "无法从 GitHub 获取最新版本号(网络问题?),本次回退使用 latest 标签"
|
||||
warn "部署完成后建议手动固定: VAULTWARDEN_IMAGE=vaultwarden/server:<版本号>"
|
||||
[ -n "$current" ] || pin_image_version "vaultwarden/server:latest"
|
||||
return 0
|
||||
fi
|
||||
|
||||
pin_image_version "vaultwarden/server:${ver}"
|
||||
log " 镜像: vaultwarden/server:${ver}(当前最新 release,已写入 .env)"
|
||||
}
|
||||
|
||||
init_env() {
|
||||
step "初始化 Vaultwarden 配置"
|
||||
|
||||
@@ -83,6 +144,8 @@ init_env() {
|
||||
log " 域名: ${VAULTWARDEN_DOMAIN}"
|
||||
log " 邮箱: ${CERTBOT_EMAIL}"
|
||||
log " 注册: ${SIGNUPS_ALLOWED:-true}"
|
||||
|
||||
setup_image_version
|
||||
}
|
||||
|
||||
create_dirs() {
|
||||
@@ -95,21 +158,39 @@ create_dirs() {
|
||||
log "备份目录: $backup_dir"
|
||||
}
|
||||
|
||||
# 确认容器真的跑在配置指定的镜像上,避免「配置改了但容器没重建」这类静默失败
|
||||
verify_running_version() {
|
||||
local cid actual expect="${VAULTWARDEN_IMAGE:-}"
|
||||
cid=$(docker compose ps -a -q vaultwarden 2>/dev/null | head -1)
|
||||
[ -n "$cid" ] || return 0
|
||||
|
||||
actual=$(docker inspect "$cid" --format '{{.Config.Image}}' 2>/dev/null || true)
|
||||
if [ -n "$expect" ] && [ "$actual" != "$expect" ]; then
|
||||
warn "容器实际镜像 (${actual}) 与配置 (${expect}) 不一致"
|
||||
warn "请检查: docker compose up -d --force-recreate"
|
||||
return 0
|
||||
fi
|
||||
log " 运行镜像: ${actual}"
|
||||
}
|
||||
|
||||
start_services() {
|
||||
step "启动 Vaultwarden 服务"
|
||||
|
||||
log "正在拉取镜像..."
|
||||
docker compose pull
|
||||
|
||||
# --force-recreate:容器处于 stopped 状态时,up -d 只会把原容器重新 start,
|
||||
# 不会因镜像或环境变量变更而重建,导致「配置改了但没生效」。
|
||||
log "正在启动容器..."
|
||||
docker compose up -d
|
||||
docker compose up -d --force-recreate
|
||||
|
||||
local port="${VAULTWARDEN_PORT:-8080}"
|
||||
log "等待 Vaultwarden 就绪..."
|
||||
local max_wait=30
|
||||
for i in $(seq 1 "$max_wait"); do
|
||||
for _ in $(seq 1 "$max_wait"); do
|
||||
if curl -sf "http://127.0.0.1:${port}/alive" &> /dev/null; then
|
||||
log "Vaultwarden 启动成功!"
|
||||
verify_running_version
|
||||
return
|
||||
fi
|
||||
sleep 2
|
||||
@@ -144,7 +225,7 @@ show_info() {
|
||||
if [[ "${SIGNUPS_ALLOWED:-true}" == "true" ]]; then
|
||||
echo -e "${GREEN}║${NC} ${YELLOW}⚠ 注册功能已开启,注册完账号后建议关闭:${NC}"
|
||||
echo -e "${GREEN}║${NC} ${YELLOW} 修改 .env 中 SIGNUPS_ALLOWED=false${NC}"
|
||||
echo -e "${GREEN}║${NC} ${YELLOW} 然后 docker compose restart${NC}"
|
||||
echo -e "${GREEN}║${NC} ${YELLOW} 然后 docker compose up -d(不是 restart)${NC}"
|
||||
fi
|
||||
echo -e "${GREEN}║${NC}"
|
||||
echo -e "${GREEN}╚══════════════════════════════════════════════════════════╝${NC}"
|
||||
|
||||
@@ -0,0 +1,700 @@
|
||||
#!/usr/bin/env bash
|
||||
# -E 让 ERR trap 能在函数内部生效,保证不可逆区间的异常一定触发回滚
|
||||
set -Eeuo pipefail
|
||||
|
||||
# ============================================
|
||||
# Vaultwarden 安全升级脚本
|
||||
#
|
||||
# 设计原则:
|
||||
# 1. 先拉镜像再动服务 —— 网络失败时服务零影响
|
||||
# 2. 冷备份(停容器 + WAL checkpoint)—— 保证快照一致
|
||||
# 3. 升级前后比对数据指纹 —— 数据异常自动回滚
|
||||
# 4. 回滚不删数据 —— 旧数据目录改名保留,不 rm
|
||||
#
|
||||
# 用法:
|
||||
# bash upgrade.sh 升级到 GitHub 最新 release
|
||||
# bash upgrade.sh --check 只检查,不做任何改动
|
||||
# bash upgrade.sh --version 1.37.0
|
||||
# bash upgrade.sh --rollback 回滚到上次升级前的状态
|
||||
# bash upgrade.sh --yes 跳过交互确认(用于自动化)
|
||||
# ============================================
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
cd "$SCRIPT_DIR"
|
||||
|
||||
# ===== 输出样式 =====
|
||||
if [ -t 1 ]; then
|
||||
RED=$'\033[0;31m'; GREEN=$'\033[0;32m'; YELLOW=$'\033[1;33m'
|
||||
CYAN=$'\033[0;36m'; BOLD=$'\033[1m'; NC=$'\033[0m'
|
||||
else
|
||||
RED=''; GREEN=''; YELLOW=''; CYAN=''; BOLD=''; NC=''
|
||||
fi
|
||||
|
||||
log() { echo -e "${GREEN}[✓]${NC} $*"; }
|
||||
info() { echo -e "${CYAN}[i]${NC} $*"; }
|
||||
warn() { echo -e "${YELLOW}[!]${NC} $*"; }
|
||||
error(){ echo -e "${RED}[✗]${NC} $*" >&2; }
|
||||
step() { echo ""; echo -e "${BOLD}${CYAN}━━━ $* ━━━${NC}"; }
|
||||
|
||||
# ===== 全局状态 =====
|
||||
GITHUB_REPO="dani-garcia/vaultwarden"
|
||||
TIMESTAMP="$(date +%Y%m%d_%H%M%S)"
|
||||
STATE_FILE="$SCRIPT_DIR/.upgrade-state"
|
||||
MODE="upgrade" # upgrade | check | rollback
|
||||
TARGET_VERSION=""
|
||||
ASSUME_YES=0
|
||||
SAFETY_STAGE="none" # none → stopped → armed → done,决定异常退出时如何兜底
|
||||
BACKUP_PATH=""
|
||||
ROLLBACK_TAG=""
|
||||
OLD_IMAGE_REF=""
|
||||
COUNTS_BEFORE=""
|
||||
|
||||
# =============================================================
|
||||
# 参数解析
|
||||
# =============================================================
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Vaultwarden 安全升级脚本
|
||||
|
||||
用法: bash upgrade.sh [选项]
|
||||
|
||||
选项:
|
||||
-c, --check 只做预检和版本对比,不做任何改动
|
||||
-v, --version <ver> 指定目标版本(如 1.37.0),默认取 GitHub 最新 release
|
||||
-r, --rollback 回滚到上次升级前的状态(读取 .upgrade-state)
|
||||
-y, --yes 跳过交互确认
|
||||
-h, --help 显示本帮助
|
||||
|
||||
示例:
|
||||
bash upgrade.sh --check # 先看看会发生什么
|
||||
bash upgrade.sh # 正式升级
|
||||
bash upgrade.sh --rollback # 升级后发现问题,回滚
|
||||
EOF
|
||||
}
|
||||
|
||||
parse_args() {
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
-c|--check) MODE="check" ;;
|
||||
-r|--rollback) MODE="rollback" ;;
|
||||
-y|--yes) ASSUME_YES=1 ;;
|
||||
-v|--version)
|
||||
shift
|
||||
[ $# -gt 0 ] || { error "--version 需要一个参数"; exit 1; }
|
||||
TARGET_VERSION="${1#v}"
|
||||
;;
|
||||
-h|--help) usage; exit 0 ;;
|
||||
*) error "未知参数: $1"; echo ""; usage; exit 1 ;;
|
||||
esac
|
||||
shift
|
||||
done
|
||||
}
|
||||
|
||||
confirm() {
|
||||
[ "$ASSUME_YES" -eq 1 ] && return 0
|
||||
local prompt="$1" answer
|
||||
echo ""
|
||||
read -r -p "${prompt} [y/N] " answer
|
||||
[[ "$answer" =~ ^[Yy]$ ]]
|
||||
}
|
||||
|
||||
# =============================================================
|
||||
# 环境加载与预检
|
||||
# =============================================================
|
||||
load_env() {
|
||||
[ -f .env ] || { error ".env 不存在,请先运行 deploy.sh"; exit 1; }
|
||||
sed -i 's/\r$//' .env
|
||||
set -a; source .env; set +a
|
||||
|
||||
DATA_DIR="${VAULTWARDEN_DATA_DIR:-/var/lib/vaultwarden}"
|
||||
BACKUP_BASE="${BACKUP_DIR:-/var/backups/vaultwarden}"
|
||||
PORT="${VAULTWARDEN_PORT:-8080}"
|
||||
DOMAIN="${VAULTWARDEN_DOMAIN:-}"
|
||||
CURRENT_IMAGE="${VAULTWARDEN_IMAGE:-vaultwarden/server:latest}"
|
||||
DB_FILE="$DATA_DIR/db.sqlite3"
|
||||
}
|
||||
|
||||
preflight() {
|
||||
step "环境预检"
|
||||
|
||||
[ "$(id -u)" -eq 0 ] || { error "需要 root 权限运行"; exit 1; }
|
||||
|
||||
local missing=0
|
||||
for cmd in docker curl tar; do
|
||||
command -v "$cmd" >/dev/null 2>&1 || { error "缺少命令: $cmd"; missing=1; }
|
||||
done
|
||||
docker compose version >/dev/null 2>&1 || { error "docker compose 不可用"; missing=1; }
|
||||
[ "$missing" -eq 0 ] || exit 1
|
||||
|
||||
[ -f docker-compose.yml ] || { error "docker-compose.yml 不存在"; exit 1; }
|
||||
[ -d "$DATA_DIR" ] || { error "数据目录不存在: $DATA_DIR"; exit 1; }
|
||||
[ -f "$DB_FILE" ] || { error "数据库不存在: $DB_FILE"; exit 1; }
|
||||
|
||||
# 磁盘空间:要求可用空间 ≥ 数据目录的 3 倍(备份 + 解压余量),至少 500MB
|
||||
mkdir -p "$BACKUP_BASE"
|
||||
local data_kb avail_kb need_kb
|
||||
data_kb=$(du -sk "$DATA_DIR" | cut -f1)
|
||||
avail_kb=$(df -Pk "$BACKUP_BASE" | awk 'NR==2{print $4}')
|
||||
[ -n "$avail_kb" ] || { error "无法检测 $BACKUP_BASE 的可用空间"; exit 1; }
|
||||
need_kb=$(( data_kb * 3 ))
|
||||
[ "$need_kb" -lt 512000 ] && need_kb=512000
|
||||
if [ "$avail_kb" -lt "$need_kb" ]; then
|
||||
error "磁盘空间不足: 可用 $((avail_kb/1024))MB,需要 $((need_kb/1024))MB"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
log "root 权限、依赖命令、数据目录、磁盘空间 均正常"
|
||||
info "数据目录 $DATA_DIR ($((data_kb/1024))MB),可用空间 $((avail_kb/1024))MB"
|
||||
}
|
||||
|
||||
ensure_sqlite3() {
|
||||
command -v sqlite3 >/dev/null 2>&1 && return 0
|
||||
warn "sqlite3 未安装,正在安装(用于 WAL checkpoint 和完整性校验)..."
|
||||
if command -v apt-get >/dev/null 2>&1; then
|
||||
DEBIAN_FRONTEND=noninteractive apt-get update -qq
|
||||
DEBIAN_FRONTEND=noninteractive apt-get install -y -qq sqlite3
|
||||
elif command -v yum >/dev/null 2>&1; then
|
||||
yum install -y -q sqlite
|
||||
else
|
||||
error "无法自动安装 sqlite3,请手动安装后重试"
|
||||
exit 1
|
||||
fi
|
||||
command -v sqlite3 >/dev/null 2>&1 || { error "sqlite3 安装失败"; exit 1; }
|
||||
log "sqlite3 已安装"
|
||||
}
|
||||
|
||||
# =============================================================
|
||||
# 版本解析
|
||||
# =============================================================
|
||||
resolve_target_version() {
|
||||
if [ -n "$TARGET_VERSION" ]; then
|
||||
info "使用指定版本: $TARGET_VERSION"
|
||||
return 0
|
||||
fi
|
||||
info "查询 GitHub 最新 release..."
|
||||
TARGET_VERSION=$(curl -sf --max-time 20 \
|
||||
"https://api.github.com/repos/${GITHUB_REPO}/releases/latest" \
|
||||
| grep -o '"tag_name"[[:space:]]*:[[:space:]]*"[^"]*"' \
|
||||
| head -1 | sed 's/.*"\([^"]*\)"$/\1/' | sed 's/^v//') || true
|
||||
|
||||
if [ -z "$TARGET_VERSION" ]; then
|
||||
error "无法从 GitHub 获取最新版本(网络问题?)"
|
||||
error "请手动指定: bash upgrade.sh --version 1.37.0"
|
||||
exit 1
|
||||
fi
|
||||
log "GitHub 最新 release: $TARGET_VERSION"
|
||||
}
|
||||
|
||||
# 从 compose 项目推导容器 ID(而非硬编码容器名),停止状态也能拿到
|
||||
compose_container_id() {
|
||||
docker compose ps -a -q vaultwarden 2>/dev/null | head -1 || true
|
||||
}
|
||||
|
||||
# 当前容器实际运行的镜像 ID(sha256:...),拿不到则返回空
|
||||
running_image_id() {
|
||||
local cid
|
||||
cid="$(compose_container_id)"
|
||||
[ -n "$cid" ] || return 0
|
||||
docker inspect "$cid" --format '{{.Image}}' 2>/dev/null || true
|
||||
}
|
||||
|
||||
get_running_githash() {
|
||||
curl -sf --max-time 10 "http://127.0.0.1:${PORT}/api/config" 2>/dev/null \
|
||||
| grep -o '"gitHash"[[:space:]]*:[[:space:]]*"[^"]*"' \
|
||||
| sed 's/.*"\([^"]*\)"$/\1/' || true
|
||||
}
|
||||
|
||||
# =============================================================
|
||||
# 数据指纹
|
||||
# =============================================================
|
||||
# 统计关键表行数,用于升级前后比对。表不存在时输出 "-"
|
||||
snapshot_counts() {
|
||||
local db="$1" t out=""
|
||||
for t in users ciphers folders sends attachments organizations collections; do
|
||||
local n
|
||||
n=$(sqlite3 "$db" "SELECT COUNT(*) FROM $t;" 2>/dev/null || echo "-")
|
||||
out+="${t}=${n} "
|
||||
done
|
||||
echo "${out% }"
|
||||
}
|
||||
|
||||
print_counts() {
|
||||
local label="$1" counts="$2" kv
|
||||
echo " ${label}:"
|
||||
for kv in $counts; do
|
||||
printf " %-16s %s\n" "${kv%%=*}" "${kv##*=}"
|
||||
done
|
||||
}
|
||||
|
||||
# =============================================================
|
||||
# 备份
|
||||
# =============================================================
|
||||
cold_backup() {
|
||||
step "冷备份(服务已停止,快照一致)"
|
||||
|
||||
mkdir -p "$BACKUP_BASE"
|
||||
|
||||
info "执行 WAL checkpoint,将未落盘数据合并进主库..."
|
||||
sqlite3 "$DB_FILE" "PRAGMA wal_checkpoint(TRUNCATE);" >/dev/null
|
||||
log "WAL 已合并"
|
||||
|
||||
info "校验数据库完整性..."
|
||||
local integrity
|
||||
integrity=$(sqlite3 "$DB_FILE" "PRAGMA integrity_check;" 2>&1 | head -1)
|
||||
if [ "$integrity" != "ok" ]; then
|
||||
error "数据库完整性检查未通过: $integrity"
|
||||
error "升级已中止,服务未改动。请先修复数据库。"
|
||||
docker compose up -d
|
||||
exit 1
|
||||
fi
|
||||
log "数据库完整性: ok"
|
||||
|
||||
COUNTS_BEFORE="$(snapshot_counts "$DB_FILE")"
|
||||
print_counts "升级前数据统计" "$COUNTS_BEFORE"
|
||||
|
||||
BACKUP_PATH="${BACKUP_BASE}/pre-upgrade-${TIMESTAMP}.tar.gz"
|
||||
info "打包整个数据目录 → $BACKUP_PATH"
|
||||
tar czf "$BACKUP_PATH" -C "$(dirname "$DATA_DIR")" "$(basename "$DATA_DIR")"
|
||||
|
||||
# 校验备份包。注意:这里先把清单收进变量再用 here-string 匹配,
|
||||
# 不能写成 `tar tzf ... | grep -q`:grep -q 命中即退出会让 tar 收到
|
||||
# SIGPIPE,在 pipefail 下整条管道被判为失败,导致好备份被误判成坏的。
|
||||
info "校验备份包..."
|
||||
local listing
|
||||
listing="$(tar tzf "$BACKUP_PATH")" || { error "备份包损坏,无法列出内容"; exit 1; }
|
||||
grep -q 'db\.sqlite3$' <<<"$listing" || { error "备份包中没有数据库文件"; exit 1; }
|
||||
grep -q 'rsa_key\.pem$' <<<"$listing" || warn "备份包中没有 rsa_key.pem"
|
||||
|
||||
# 同时备份 .env(含 ADMIN_TOKEN)
|
||||
cp .env "${BACKUP_BASE}/env-${TIMESTAMP}.bak"
|
||||
chmod 600 "${BACKUP_BASE}/env-${TIMESTAMP}.bak"
|
||||
|
||||
log "备份完成: $BACKUP_PATH ($(du -h "$BACKUP_PATH" | cut -f1))"
|
||||
}
|
||||
|
||||
# =============================================================
|
||||
# 镜像与容器
|
||||
# =============================================================
|
||||
tag_rollback_image() {
|
||||
step "锚定当前镜像(用于回滚)"
|
||||
|
||||
local image_id
|
||||
image_id="$(running_image_id)"
|
||||
if [ -z "$image_id" ]; then
|
||||
image_id=$(docker image inspect "$CURRENT_IMAGE" --format '{{.Id}}' 2>/dev/null || true)
|
||||
fi
|
||||
[ -n "$image_id" ] || { error "找不到当前运行的镜像,无法建立回滚点"; exit 1; }
|
||||
|
||||
ROLLBACK_TAG="vaultwarden/server:pre-upgrade-${TIMESTAMP}"
|
||||
docker tag "$image_id" "$ROLLBACK_TAG"
|
||||
OLD_IMAGE_REF="$CURRENT_IMAGE"
|
||||
|
||||
log "当前镜像已标记为 $ROLLBACK_TAG"
|
||||
info "镜像 ID: ${image_id#sha256:}"
|
||||
}
|
||||
|
||||
pull_target_image() {
|
||||
step "拉取目标镜像(此步骤不影响运行中的服务)"
|
||||
|
||||
local target="vaultwarden/server:${TARGET_VERSION}"
|
||||
info "docker pull $target ..."
|
||||
if ! docker pull "$target"; then
|
||||
error "镜像拉取失败,服务未做任何改动,可安全重试"
|
||||
exit 1
|
||||
fi
|
||||
log "镜像已就绪: $target"
|
||||
}
|
||||
|
||||
switch_image_in_env() {
|
||||
local new_ref="$1"
|
||||
cp .env ".env.bak-${TIMESTAMP}"
|
||||
chmod 600 ".env.bak-${TIMESTAMP}"
|
||||
if grep -q '^VAULTWARDEN_IMAGE=' .env; then
|
||||
sed -i "s|^VAULTWARDEN_IMAGE=.*|VAULTWARDEN_IMAGE=${new_ref}|" .env
|
||||
else
|
||||
printf '\nVAULTWARDEN_IMAGE=%s\n' "$new_ref" >> .env
|
||||
fi
|
||||
# 必须同步更新导出的变量:load_env 里 `set -a; source .env` 已经把旧值导出到
|
||||
# 本进程环境,而 docker compose 对环境变量的优先级高于 .env 文件 —— 只改文件
|
||||
# 的话 compose 仍会读到旧镜像,表现为「容器重建了但版本没变」。
|
||||
export VAULTWARDEN_IMAGE="$new_ref"
|
||||
info ".env 中 VAULTWARDEN_IMAGE → $new_ref"
|
||||
}
|
||||
|
||||
wait_healthy() {
|
||||
local max_wait="${1:-60}" i
|
||||
info "等待服务就绪(最多 ${max_wait}s)..."
|
||||
for i in $(seq 1 "$max_wait"); do
|
||||
if curl -sf --max-time 3 "http://127.0.0.1:${PORT}/alive" >/dev/null 2>&1; then
|
||||
log "服务已就绪(${i}s)"
|
||||
return 0
|
||||
fi
|
||||
sleep 1
|
||||
done
|
||||
return 1
|
||||
}
|
||||
|
||||
# =============================================================
|
||||
# 验证
|
||||
# =============================================================
|
||||
# 确认容器真的跑在目标镜像上,防止「改了配置但容器没重建」这类静默失败
|
||||
verify_running_image() {
|
||||
local expect="$1" cid actual_ref actual_id expect_id
|
||||
cid="$(compose_container_id)"
|
||||
[ -n "$cid" ] || { error "找不到容器"; return 1; }
|
||||
|
||||
actual_ref=$(docker inspect "$cid" --format '{{.Config.Image}}' 2>/dev/null || true)
|
||||
actual_id=$(docker inspect "$cid" --format '{{.Image}}' 2>/dev/null || true)
|
||||
expect_id=$(docker image inspect "$expect" --format '{{.Id}}' 2>/dev/null || true)
|
||||
|
||||
if [ "$actual_id" != "$expect_id" ]; then
|
||||
error "容器未运行在目标镜像上"
|
||||
error " 期望: $expect ($expect_id)"
|
||||
error " 实际: $actual_ref ($actual_id)"
|
||||
return 1
|
||||
fi
|
||||
log "容器镜像确认: $actual_ref"
|
||||
return 0
|
||||
}
|
||||
|
||||
# $1: strict(默认) —— 要求新版 prelogin 路由存在(升级后)
|
||||
# lenient —— 该路由 404 属正常(回滚到 1.36.0 之前的版本后)
|
||||
verify_endpoints() {
|
||||
local strict="${1:-strict}"
|
||||
step "接口验证"
|
||||
local failed=0
|
||||
|
||||
# 1. /alive
|
||||
if curl -sf --max-time 5 "http://127.0.0.1:${PORT}/alive" >/dev/null; then
|
||||
log "/alive 正常"
|
||||
else
|
||||
error "/alive 无响应"; failed=1
|
||||
fi
|
||||
|
||||
# 2. 新版 prelogin 路由(1.36.0+ 才有,是本次升级的核心目标)
|
||||
local code
|
||||
code=$(curl -s -o /dev/null -w '%{http_code}' --max-time 8 \
|
||||
-X POST -H 'Content-Type: application/json' \
|
||||
-d '{"email":"healthcheck@example.invalid"}' \
|
||||
"http://127.0.0.1:${PORT}/identity/accounts/prelogin/password" || echo "000")
|
||||
if [ "$code" = "200" ]; then
|
||||
log "/identity/accounts/prelogin/password → 200(新版客户端可登录)"
|
||||
elif [ "$strict" = "lenient" ]; then
|
||||
info "/identity/accounts/prelogin/password → $code(该版本无此路由,回滚后属预期)"
|
||||
else
|
||||
error "/identity/accounts/prelogin/password → $code(期望 200)"; failed=1
|
||||
fi
|
||||
|
||||
# 3. 旧版 prelogin 路由(兼容老客户端)
|
||||
code=$(curl -s -o /dev/null -w '%{http_code}' --max-time 8 \
|
||||
-X POST -H 'Content-Type: application/json' \
|
||||
-d '{"email":"healthcheck@example.invalid"}' \
|
||||
"http://127.0.0.1:${PORT}/identity/accounts/prelogin" || echo "000")
|
||||
if [ "$code" = "200" ]; then
|
||||
log "/identity/accounts/prelogin → 200(旧客户端兼容)"
|
||||
else
|
||||
warn "/identity/accounts/prelogin → $code"
|
||||
fi
|
||||
|
||||
# 4. 经 Nginx 的外部访问(失败只告警,属于代理层问题,不触发回滚)
|
||||
if [ -n "$DOMAIN" ]; then
|
||||
code=$(curl -s -o /dev/null -w '%{http_code}' --max-time 10 \
|
||||
"https://${DOMAIN}/api/config" || echo "000")
|
||||
if [ "$code" = "200" ]; then
|
||||
log "https://${DOMAIN}/api/config → 200(Nginx 反代正常)"
|
||||
else
|
||||
warn "https://${DOMAIN}/api/config → $code(检查 Nginx/证书,不影响本次升级判定)"
|
||||
fi
|
||||
fi
|
||||
|
||||
return $failed
|
||||
}
|
||||
|
||||
verify_data_integrity() {
|
||||
step "数据完整性验证"
|
||||
|
||||
local integrity
|
||||
integrity=$(sqlite3 "$DB_FILE" "PRAGMA integrity_check;" 2>&1 | head -1)
|
||||
if [ "$integrity" != "ok" ]; then
|
||||
error "升级后数据库完整性检查失败: $integrity"
|
||||
return 1
|
||||
fi
|
||||
log "数据库完整性: ok"
|
||||
|
||||
local counts_after
|
||||
counts_after="$(snapshot_counts "$DB_FILE")"
|
||||
print_counts "升级后数据统计" "$counts_after"
|
||||
|
||||
# 逐表比对:行数只允许持平或增加,减少视为数据丢失
|
||||
local kv table before after lost=0
|
||||
for kv in $COUNTS_BEFORE; do
|
||||
table="${kv%%=*}"; before="${kv##*=}"
|
||||
after=$(echo "$counts_after" | tr ' ' '\n' | grep "^${table}=" | cut -d= -f2 || true)
|
||||
# 升级前该表就不存在 / 升级后读不到 → 跳过比对
|
||||
if [ "$before" = "-" ] || [ -z "$after" ] || [ "$after" = "-" ]; then
|
||||
continue
|
||||
fi
|
||||
if [ "$after" -lt "$before" ]; then
|
||||
error "表 $table 行数减少: $before → $after"
|
||||
lost=1
|
||||
fi
|
||||
done
|
||||
|
||||
if [ "$lost" -eq 1 ]; then
|
||||
error "检测到数据丢失!"
|
||||
return 1
|
||||
fi
|
||||
log "所有表行数无减少,数据完好"
|
||||
return 0
|
||||
}
|
||||
|
||||
# =============================================================
|
||||
# 回滚
|
||||
# =============================================================
|
||||
save_state() {
|
||||
cat > "$STATE_FILE" <<EOF
|
||||
# Vaultwarden 升级状态记录 —— 由 upgrade.sh 自动生成
|
||||
UPGRADE_TIMESTAMP=${TIMESTAMP}
|
||||
BACKUP_PATH=${BACKUP_PATH}
|
||||
ROLLBACK_TAG=${ROLLBACK_TAG}
|
||||
OLD_IMAGE_REF=${OLD_IMAGE_REF}
|
||||
NEW_IMAGE_REF=vaultwarden/server:${TARGET_VERSION}
|
||||
DATA_DIR=${DATA_DIR}
|
||||
ENV_BACKUP=${SCRIPT_DIR}/.env.bak-${TIMESTAMP}
|
||||
EOF
|
||||
chmod 600 "$STATE_FILE"
|
||||
}
|
||||
|
||||
# 恢复数据目录:旧目录改名保留,绝不 rm
|
||||
restore_data_dir() {
|
||||
local backup_tar="$1" ts="$2"
|
||||
local parent failed_dir
|
||||
parent="$(dirname "$DATA_DIR")"
|
||||
failed_dir="${DATA_DIR}.failed-${ts}"
|
||||
|
||||
if [ -d "$DATA_DIR" ]; then
|
||||
mv "$DATA_DIR" "$failed_dir"
|
||||
warn "升级后的数据目录已保留为: $failed_dir(未删除)"
|
||||
fi
|
||||
tar xzf "$backup_tar" -C "$parent"
|
||||
[ -f "$DB_FILE" ] || { error "恢复失败:$DB_FILE 不存在"; return 1; }
|
||||
log "数据目录已从备份恢复"
|
||||
}
|
||||
|
||||
# 服务已停、但备份尚未就绪时的紧急恢复:此阶段 .env 未改动,直接拉起原版本即可
|
||||
emergency_start_old() {
|
||||
step "紧急恢复"
|
||||
warn "升级在备份阶段中止,数据未被改动,正在拉起原服务..."
|
||||
docker compose up -d || true
|
||||
if wait_healthy 60; then
|
||||
log "原服务已恢复运行,本次升级未造成任何影响"
|
||||
else
|
||||
error "原服务恢复失败!请手动执行: cd $SCRIPT_DIR && docker compose up -d"
|
||||
error "查看日志: docker compose logs --tail 100"
|
||||
fi
|
||||
}
|
||||
|
||||
# 统一的退出兜底:确保任何异常路径都不会把服务丢在停机状态
|
||||
on_exit() {
|
||||
local code=$?
|
||||
trap - EXIT ERR
|
||||
[ "$code" -eq 0 ] && return 0
|
||||
case "$SAFETY_STAGE" in
|
||||
stopped) emergency_start_old ;;
|
||||
armed) do_rollback "脚本异常中止(退出码 $code)" ;;
|
||||
*) : ;; # 尚未动过服务,无需处理
|
||||
esac
|
||||
return 0
|
||||
}
|
||||
|
||||
do_rollback() {
|
||||
trap - ERR EXIT # 先解除,避免回滚过程自身出错导致递归
|
||||
SAFETY_STAGE="done"
|
||||
set +e
|
||||
local reason="$1"
|
||||
step "自动回滚"
|
||||
error "触发原因: $reason"
|
||||
|
||||
docker compose down || true
|
||||
restore_data_dir "$BACKUP_PATH" "$TIMESTAMP" || {
|
||||
error "自动恢复失败!请手动处理:"
|
||||
error " tar xzf $BACKUP_PATH -C $(dirname "$DATA_DIR")"
|
||||
exit 1
|
||||
}
|
||||
switch_image_in_env "$ROLLBACK_TAG"
|
||||
docker compose up -d --force-recreate
|
||||
|
||||
if wait_healthy 60; then
|
||||
log "回滚完成,已恢复到升级前状态"
|
||||
else
|
||||
error "回滚后服务仍未就绪,请检查: docker compose logs --tail 100"
|
||||
fi
|
||||
echo ""
|
||||
warn "升级失败已回滚。备份保留在: $BACKUP_PATH"
|
||||
exit 1
|
||||
}
|
||||
|
||||
manual_rollback() {
|
||||
step "手动回滚"
|
||||
|
||||
[ -f "$STATE_FILE" ] || { error "找不到升级记录 $STATE_FILE,无法自动回滚"; exit 1; }
|
||||
# shellcheck disable=SC1090
|
||||
source "$STATE_FILE"
|
||||
DB_FILE="$DATA_DIR/db.sqlite3" # 状态文件里的 DATA_DIR 可能与 .env 不同,重新推导
|
||||
|
||||
echo ""
|
||||
info "将回滚到:"
|
||||
info " 镜像: $ROLLBACK_TAG"
|
||||
info " 备份: $BACKUP_PATH"
|
||||
info " 时间: $UPGRADE_TIMESTAMP"
|
||||
echo ""
|
||||
|
||||
[ -f "$BACKUP_PATH" ] || { error "备份文件不存在: $BACKUP_PATH"; exit 1; }
|
||||
docker image inspect "$ROLLBACK_TAG" >/dev/null 2>&1 \
|
||||
|| { error "回滚镜像不存在: $ROLLBACK_TAG"; exit 1; }
|
||||
|
||||
warn "回滚会丢弃升级后产生的所有数据变更(新增/修改的密码条目)"
|
||||
confirm "确认回滚?" || { info "已取消"; exit 0; }
|
||||
|
||||
docker compose down || true
|
||||
restore_data_dir "$BACKUP_PATH" "$(date +%Y%m%d_%H%M%S)" || exit 1
|
||||
switch_image_in_env "$ROLLBACK_TAG"
|
||||
docker compose up -d --force-recreate
|
||||
|
||||
if wait_healthy 60; then
|
||||
log "回滚完成"
|
||||
verify_endpoints lenient || true
|
||||
else
|
||||
error "服务未就绪,请检查: docker compose logs --tail 100"
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
# =============================================================
|
||||
# 主流程
|
||||
# =============================================================
|
||||
show_plan() {
|
||||
local githash
|
||||
githash="$(get_running_githash)"
|
||||
|
||||
echo ""
|
||||
echo -e "${BOLD}══════════ 升级计划 ══════════${NC}"
|
||||
echo " 域名: ${DOMAIN:-未配置}"
|
||||
echo " 当前镜像: $CURRENT_IMAGE"
|
||||
echo " 当前 gitHash: ${githash:-未知(服务未响应?)}"
|
||||
echo " 目标版本: vaultwarden/server:${TARGET_VERSION}"
|
||||
echo " 数据目录: $DATA_DIR"
|
||||
echo " 备份目录: $BACKUP_BASE"
|
||||
echo ""
|
||||
echo " 执行步骤:"
|
||||
echo " 1. 锚定当前镜像为回滚点"
|
||||
echo " 2. 拉取新镜像(失败则零影响退出)"
|
||||
echo " 3. 停止容器"
|
||||
echo " 4. WAL checkpoint + 完整性校验 + 冷备份"
|
||||
echo " 5. 切换镜像并启动"
|
||||
echo " 6. 验证接口 + 比对数据行数"
|
||||
echo " 7. 任一验证失败 → 自动回滚"
|
||||
echo -e "${BOLD}═════════════════════════════${NC}"
|
||||
}
|
||||
|
||||
main() {
|
||||
parse_args "$@"
|
||||
|
||||
echo -e "${CYAN}${BOLD}"
|
||||
echo " Vaultwarden 安全升级"
|
||||
echo -e "${NC}"
|
||||
|
||||
load_env
|
||||
|
||||
if [ "$MODE" = "rollback" ]; then
|
||||
[ "$(id -u)" -eq 0 ] || { error "需要 root 权限"; exit 1; }
|
||||
manual_rollback
|
||||
exit 0
|
||||
fi
|
||||
|
||||
preflight
|
||||
resolve_target_version
|
||||
show_plan
|
||||
|
||||
# 已是目标版本?
|
||||
local cur_image_id target_image_id
|
||||
cur_image_id="$(running_image_id)"
|
||||
target_image_id=$(docker image inspect "vaultwarden/server:${TARGET_VERSION}" \
|
||||
--format '{{.Id}}' 2>/dev/null || true)
|
||||
if [ -n "$cur_image_id" ] && [ "$cur_image_id" = "$target_image_id" ]; then
|
||||
echo ""
|
||||
log "当前运行的已经是 ${TARGET_VERSION},无需升级"
|
||||
if [ "$MODE" = "check" ]; then
|
||||
exit 0
|
||||
fi
|
||||
confirm "仍要强制重新部署一次?" || { info "已取消"; exit 0; }
|
||||
fi
|
||||
|
||||
if [ "$MODE" = "check" ]; then
|
||||
echo ""
|
||||
info "--check 模式,未做任何改动"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
confirm "确认开始升级?" || { info "已取消"; exit 0; }
|
||||
|
||||
ensure_sqlite3
|
||||
# 先拉镜像再打回滚标签:拉取失败时服务未动,也不留下无用的标签
|
||||
pull_target_image
|
||||
tag_rollback_image
|
||||
|
||||
# ===== 从这里开始服务会中断,全程由 on_exit 兜底 =====
|
||||
trap on_exit EXIT
|
||||
trap 'exit 1' ERR
|
||||
|
||||
step "停止容器"
|
||||
SAFETY_STAGE="stopped" # 异常退出 → 直接拉起原版本
|
||||
docker compose stop
|
||||
log "容器已停止"
|
||||
|
||||
cold_backup
|
||||
save_state
|
||||
SAFETY_STAGE="armed" # 备份已就位,异常退出 → 完整回滚
|
||||
|
||||
step "启动新版本"
|
||||
switch_image_in_env "vaultwarden/server:${TARGET_VERSION}"
|
||||
# 必须 --force-recreate:容器处于 stopped 状态时,docker compose up -d 只会
|
||||
# 把原容器重新 start,不会因为镜像变更而重建,结果是「升级了但还跑着旧镜像」。
|
||||
docker compose up -d --force-recreate
|
||||
|
||||
if ! wait_healthy 90; then
|
||||
docker compose logs --tail 40 || true
|
||||
do_rollback "服务在 90 秒内未就绪"
|
||||
fi
|
||||
if ! verify_running_image "vaultwarden/server:${TARGET_VERSION}"; then
|
||||
do_rollback "容器未运行在目标镜像上"
|
||||
fi
|
||||
if ! verify_endpoints; then
|
||||
do_rollback "接口验证失败"
|
||||
fi
|
||||
if ! verify_data_integrity; then
|
||||
do_rollback "数据完整性验证失败"
|
||||
fi
|
||||
|
||||
# ===== 成功 =====
|
||||
SAFETY_STAGE="done"
|
||||
trap - EXIT ERR
|
||||
step "升级成功"
|
||||
local new_githash
|
||||
new_githash="$(get_running_githash)"
|
||||
echo ""
|
||||
echo -e "${GREEN}${BOLD}══════════ 完成 ══════════${NC}"
|
||||
echo " 运行版本: vaultwarden/server:${TARGET_VERSION}"
|
||||
echo " gitHash: ${new_githash:-未知}"
|
||||
echo " 备份文件: $BACKUP_PATH"
|
||||
echo " 回滚镜像: $ROLLBACK_TAG"
|
||||
echo ""
|
||||
echo " 如需回滚: bash upgrade.sh --rollback"
|
||||
echo " 查看日志: docker compose logs --tail 50"
|
||||
echo -e "${GREEN}${BOLD}══════════════════════════${NC}"
|
||||
echo ""
|
||||
warn "请登录 Web 端确认数据无误后,再考虑清理旧镜像:"
|
||||
warn " docker image rm $ROLLBACK_TAG"
|
||||
}
|
||||
|
||||
main "$@"
|
||||
+27
-3
@@ -6,11 +6,35 @@
|
||||
# XRAY_MODE=reality
|
||||
|
||||
# 监听端口(默认 443,建议不改)
|
||||
# NAT / 端口转发的机器请填服务商实际映射给你的端口
|
||||
# XRAY_PORT=443
|
||||
|
||||
# Reality 伪装目标(留空则自动选择延迟最低的)
|
||||
# REALITY_DEST=www.microsoft.com
|
||||
# REALITY_SNI=www.microsoft.com
|
||||
# 分享链接中使用的 IP,留空则自动探测出口 IP
|
||||
# NAT / 端口转发的机器上出口 IP 与入站 IP 可能不同,此时必须手动指定
|
||||
# SERVER_IP=1.2.3.4
|
||||
|
||||
# Reality 伪装目标(留空则自动从候选中挑选,见 deploy.sh 的 select_reality_dest)
|
||||
# 手动指定时务必确认新域名满足两个条件,否则客户端会刷 REALITY 报错:
|
||||
# 1) TLS 1.3 且证书链 < 8192 字节(用 xray tls ping <域名> 查「with SNI」那段)
|
||||
# 2) 该域名会被客户端路由判成「直连」(命中 geosite:cn)
|
||||
# 已知不可用:dl.google.com(命中 geosite:google)、www.microsoft.com、www.amazon.com
|
||||
# REALITY_DEST=www.apple.com
|
||||
# REALITY_SNI=www.apple.com
|
||||
|
||||
# ===== 定时重启 =====
|
||||
# 0 = 关闭(默认);N = 每 N 天重启一次 Xray。
|
||||
# 崩溃恢复已由 systemd 的 Restart=always 覆盖(秒级响应),而硬重启会切断
|
||||
# 全部活动连接。仅在确实观察到长期运行后性能退化时才启用。
|
||||
# XRAY_RESTART_EVERY_DAYS=0
|
||||
# XRAY_RESTART_TIME=04:00
|
||||
# XRAY_RESTART_TIMEZONE=Asia/Shanghai
|
||||
#
|
||||
# 改这几项无需完整重新部署,用轻量入口即可(不影响 vless 链接):
|
||||
# bash deploy.sh --restart 7 04:00
|
||||
# bash deploy.sh --restart 0
|
||||
#
|
||||
# 旧字段 XRAY_DAILY_RESTART 已废弃,仅在上面三项都没设时作为兼容回退:
|
||||
# true → 每 1 天,其它 → 关闭。重跑部署后会自动换写成新字段。
|
||||
|
||||
# 备份目录
|
||||
# BACKUP_DIR=/var/backups/xray
|
||||
|
||||
+193
-6
@@ -52,14 +52,20 @@
|
||||
```
|
||||
vps-xray/
|
||||
├── deploy.sh # 一键部署脚本
|
||||
├── backup.sh # 备份脚本
|
||||
├── uninstall.sh # 完全卸载脚本
|
||||
├── backup.sh # 备份脚本
|
||||
├── upload.sh # 上传脚本(仅上传服务端必要文件)
|
||||
├── .env.example # 配置模板
|
||||
├── README.md # 本文件
|
||||
├── vps-xray-optimized.md # Reality 方案详细文档
|
||||
└── vps-xray-fast.md # Fast TCP 方案详细文档
|
||||
├── vps-xray-fast.md # Fast TCP 方案详细文档
|
||||
└── client-config/ # 客户端配置(本地使用,不上传服务器)
|
||||
├── gen-clash.sh # 由节点信息生成 Clash.Meta 配置
|
||||
└── client-macos.md # macOS 客户端指南(模板,可提交)
|
||||
```
|
||||
|
||||
> `client-config/` 下含真实 UUID / 服务器 IP 的节点配置(`*-info*.md`、`*.yaml`)已在根 `.gitignore` 中排除,不要提交到版本库。
|
||||
|
||||
服务器上的文件位置:
|
||||
|
||||
```
|
||||
@@ -73,8 +79,11 @@ vps-xray/
|
||||
|
||||
### 第一步:上传文件到 VPS
|
||||
|
||||
使用 `upload.sh` 只上传服务端必要文件(`deploy.sh`、`uninstall.sh`、`.env.example`),跳过文档和客户端配置:
|
||||
|
||||
```bash
|
||||
scp -r vps-xray/ root@<VPS_IP>:/opt/vps-xray
|
||||
# 在本地 vps-xray/ 目录下执行
|
||||
bash upload.sh <VPS_IP>
|
||||
```
|
||||
|
||||
### 第二步:登录 VPS 执行部署
|
||||
@@ -94,7 +103,15 @@ bash deploy.sh --mode fast
|
||||
- 连接参数(IP、端口、UUID、密钥等)
|
||||
- VLESS 分享链接(可直接导入客户端)
|
||||
|
||||
> **⚠️ 请妥善保存输出的连接信息!密钥仅显示一次。**凭据同时保存在 `/opt/vps-xray/.env` 中。
|
||||
凭据保存在 `/opt/vps-xray/.env`(权限 600)。**信息不会丢** —— 随时可以重新打印:
|
||||
|
||||
```bash
|
||||
cd /opt/vps-xray && bash deploy.sh --show
|
||||
```
|
||||
|
||||
`--show` 只读取 `.env` 输出连接信息和分享链接,不安装、不改配置、不重启服务。
|
||||
|
||||
> 即使 `.env` 被误删,只要 `/usr/local/etc/xray/config.json` 还在,UUID、shortId、SNI 都能从中读出,公钥可用 `xray x25519 -i <privateKey>` 从私钥反推。
|
||||
|
||||
### 第三步:客户端配置
|
||||
|
||||
@@ -107,16 +124,186 @@ bash deploy.sh --mode fast
|
||||
| iOS | Shadowrocket / Streisand |
|
||||
| Android | v2rayNG |
|
||||
|
||||
#### 生成 Clash 配置
|
||||
|
||||
把每台 VPS 的 `bash deploy.sh --show` 输出保存成 `client-config/xxx-info.md`,然后:
|
||||
|
||||
```bash
|
||||
cd client-config
|
||||
bash gen-clash.sh # 交互式勾选要写入的节点
|
||||
bash gen-clash.sh --all # 全部节点
|
||||
```
|
||||
|
||||
脚本解析文件里的 `vless://` 链接,生成 `vpn-vps-xray-optimized.yaml`(多节点自动组成 url-test 自动选择组,原文件会先备份为 `.bak`)。
|
||||
|
||||
> 生成的配置需要 **Clash.Meta / mihomo** 内核(Clash Verge Rev、ClashX Meta、FlClash 等)。原版 Clash / Clash Premium 不支持 VLESS + Reality。
|
||||
|
||||
详细的 Clash Meta / Sing-Box 配置参见 [vps-xray-optimized.md](vps-xray-optimized.md)。
|
||||
|
||||
## 日常运维
|
||||
|
||||
### 查看连接信息 / 分享链接
|
||||
|
||||
```bash
|
||||
cd /opt/vps-xray
|
||||
bash deploy.sh --show
|
||||
```
|
||||
|
||||
只读操作,不会改动任何服务。
|
||||
|
||||
### 更换 Reality 伪装目标
|
||||
|
||||
伪装目标在首次部署时按握手延迟自动挑选,之后固定写入 `.env` 不再变动。后续每次部署会检查它是否仍然可达,不可达时告警但**不会自动更换**——因为更换会改变 SNI,导致所有客户端链接静默失效。
|
||||
|
||||
确认要换时显式执行:
|
||||
|
||||
```bash
|
||||
bash deploy.sh --redetect
|
||||
```
|
||||
|
||||
> ⚠️ 换完后 SNI 变了,**所有客户端都必须用新的分享链接重新配置**。脚本会在更换时打印醒目提示。
|
||||
|
||||
#### 自定义伪装目标时的两个硬条件
|
||||
|
||||
在 `.env` 里手动指定 `REALITY_DEST` / `REALITY_SNI` 前,必须确认新域名同时满足:
|
||||
|
||||
**1. 技术条件** — TLS 1.3,且证书链总长 < 8192 字节。上限是 `xtls/reality` 里的硬编码值,超出会直接握手失败(见 [XTLS/Xray-core#6356](https://github.com/XTLS/Xray-core/issues/6356),`www.microsoft.com` 的 8273 字节即触发)。在服务器上自查:
|
||||
|
||||
```bash
|
||||
xray tls ping www.apple.com # 只看「Pinging with SNI」那一段
|
||||
```
|
||||
|
||||
**2. 客户端路由条件** — 该域名必须会被客户端判成「直连」。
|
||||
|
||||
这条不直观。TUN 模式下客户端偶尔会把「连接本代理服务器」这件事本身也抓进代理(v2rayN 的 sing-box + xray 双核心靠进程嗅探防回环,Windows 上存在竞态)。这种回环连接的去向完全由 SNI 决定:
|
||||
|
||||
| SNI 被判为 | 结果 |
|
||||
|---|---|
|
||||
| 直连 | 连接真的到达本服务器,REALITY 握手成功,**用户无感** |
|
||||
| 走代理 | 进隧道 → 服务器替客户端连了真正的伪装站 → 客户端收到真证书 → 刷屏 `REALITY: received real certificate` |
|
||||
|
||||
实测(v2rayN 默认规则模板,`geosite:google → proxy` 排在 `geosite:cn → direct` 之前):
|
||||
|
||||
| 域名 | 命中规则 | 可用性 |
|
||||
|---|---|---|
|
||||
| `www.apple.com`、`*.apple.com` | `geosite:cn` | ✅ 安全 |
|
||||
| `dl.google.com` | `geosite:google` | ❌ 不安全 |
|
||||
| `www.microsoft.com` | 都不命中 → `final:proxy` | ❌ 不安全 |
|
||||
| `www.amazon.com` | 都不命中 → `final:proxy` | ❌ 不安全 |
|
||||
|
||||
脚本的候选列表因此只保留 Apple 系域名。
|
||||
|
||||
### 定时重启
|
||||
|
||||
默认**关闭**。原因是:
|
||||
|
||||
- 崩溃恢复已由 systemd 的 `Restart=always` + `RestartSec=3` 覆盖(秒级响应),不必等到凌晨
|
||||
- Xray 无已知需要定期重启的缺陷(内存、句柄上限均已配置)
|
||||
- 而硬重启会切断全部活动连接——跨夜下载、备份、CI 都会中断
|
||||
|
||||
即"为假想问题付出真实代价"。仅在确实观察到长期运行后性能退化时才启用。
|
||||
|
||||
需要时用轻量入口调整,**不会重装 Xray、不会重写配置、不会重启服务,现有 vless 链接完全不受影响**:
|
||||
|
||||
```bash
|
||||
bash deploy.sh --restart 7 04:00 # 每 7 天 04:00 重启
|
||||
bash deploy.sh --restart 3 # 每 3 天,时间沿用 .env
|
||||
bash deploy.sh --restart 0 # 关闭
|
||||
bash deploy.sh --restart # 按 .env 现有值重建 timer
|
||||
```
|
||||
|
||||
对应 `.env` 三个字段:
|
||||
|
||||
| 字段 | 含义 | 默认 |
|
||||
|---|---|---|
|
||||
| `XRAY_RESTART_EVERY_DAYS` | 0 = 关闭;N = 每 N 天 | `0` |
|
||||
| `XRAY_RESTART_TIME` | 24 小时制 `HH:MM` | `04:00` |
|
||||
| `XRAY_RESTART_TIMEZONE` | 时区 | `Asia/Shanghai` |
|
||||
|
||||
#### 查看当前配置与重启记录
|
||||
|
||||
**一条命令看全** —— 不带参数的 `--restart` 只按 `.env` 现值重建 timer,不改配置、不重启服务:
|
||||
|
||||
```bash
|
||||
bash deploy.sh --restart
|
||||
```
|
||||
|
||||
输出里包含当前间隔与时间、**上次实际重启**、**下次实际重启**。
|
||||
|
||||
分项查看:
|
||||
|
||||
```bash
|
||||
# 1. 配置值
|
||||
grep '^XRAY_RESTART_' /opt/vps-xray/.env
|
||||
|
||||
# 2. timer 是否启用、下次唤醒时间
|
||||
systemctl list-timers xray-restart.timer
|
||||
|
||||
# 3. 确认时区真的生效(最容易出错的一项)
|
||||
systemctl show xray-restart.timer -p TimersCalendar
|
||||
|
||||
# 4. 上次实际重启的时间戳
|
||||
cat /var/lib/xray/last-restart # Unix 时间戳
|
||||
date -d "@$(cat /var/lib/xray/last-restart)" # 可读格式
|
||||
|
||||
# 5. 守卫每次唤醒后的判断记录(跳过还是执行)
|
||||
journalctl -u xray-restart.service --no-pager -n 50
|
||||
|
||||
# 6. xray 进程最近一次启动的时刻
|
||||
systemctl show xray -p ActiveEnterTimestamp --value
|
||||
```
|
||||
|
||||
第 3 条的输出应形如:
|
||||
|
||||
```
|
||||
{ OnCalendar=*-*-* 04:00:00 Asia/Shanghai ; next_elapse=Fri 2026-08-07 20:00:00 UTC }
|
||||
```
|
||||
|
||||
**里面必须看得到时区。** 若只有 `OnCalendar=*-*-* 04:00:00` 而没有时区,说明时区没生效,systemd 会按服务器的系统时区解析——多数 VPS 是 UTC,那样设的「04:00 北京时间」实际会在**中午 12 点**触发,偏移 8 小时且没有任何提示。
|
||||
|
||||
> 早期实现把时区写成 `[Timer]` 段的 `TimeZone=`,而 `systemd.timer` 根本没有这个 key——写了只会在 journal 里留一行 `Unknown key name 'TimeZone' in section 'Timer', ignoring.` 然后按本地时区解析。排查时可以顺手确认一下:
|
||||
>
|
||||
> ```bash
|
||||
> journalctl -b | grep -i "unknown key name"
|
||||
> systemd-analyze verify /etc/systemd/system/xray-restart.timer
|
||||
> ```
|
||||
|
||||
第 5 条的日志长这样:
|
||||
|
||||
```
|
||||
xray-restart-guard[1234]: 距上次重启不足 7 天(已过 3 天),跳过
|
||||
xray-restart-guard[5678]: 距上次重启已满 7 天,执行重启
|
||||
```
|
||||
|
||||
出现下面这行说明状态文件写不进去(磁盘满或只读挂载),守卫会主动跳过本轮——宁可不重启,也不让保护失效后退化成每天硬重启:
|
||||
|
||||
```
|
||||
xray-restart-guard[...]: 无法写入 /var/lib/xray/last-restart,跳过本轮重启以免退化成每日重启
|
||||
```
|
||||
|
||||
> ⚠️ `list-timers` 的 `NEXT` 是 **timer 下次「唤醒」的时间,不是下次重启的时间**。timer 每天唤醒一次,NEXT 因此恒为次日的 `XRAY_RESTART_TIME`;是否真的重启由守卫按间隔判断。设了「每 7 天」且 5 天前刚重启过时,NEXT 显示明天凌晨,实际还要再等 2 天。想看真实的下次重启时间,用上面那条不带参数的 `bash deploy.sh --restart`。
|
||||
|
||||
#### 为什么不是纯 systemd timer
|
||||
|
||||
`OnCalendar` 表达不了任意 N 天:`*-*-1/7 04:00:00` 是「每月 1、8、15、22、29 号」,**跨月会重置**——29 号到下月 1 号只隔 2~3 天。
|
||||
|
||||
所以实现是 timer 每天到点唤醒,由 `/usr/local/bin/xray-restart-guard` 读 `/var/lib/xray/last-restart`,满 N 天才真正重启。间隔因此是「距上次实际重启」的真实天数,`N=3` 与 `N=7` 同样准确。
|
||||
|
||||
具体怎么查看配置与执行记录,见上面的「查看当前配置与重启记录」。
|
||||
|
||||
#### 已知取舍
|
||||
|
||||
- **间隔锚点是「上次由守卫执行的重启」,不是任何一次 xray 重启。** 完整部署(`bash deploy.sh`)里的 `systemctl restart xray` 不会更新 `/var/lib/xray/last-restart`——设了每 7 天、距上次守卫重启已 6.9 天时跑一次完整部署,次日守卫仍会判定满 7 天而再重启一次。介意的话可在部署后手动执行 `date +%s > /var/lib/xray/last-restart` 把锚点对齐。
|
||||
- **重启失败时时间戳仍已更新**,本轮被跳过,下一轮照常——这是为了避免陷入每天硬重启的循环。但若时间戳**写不进去**(磁盘满、只读挂载),守卫会跳过本轮重启并在日志里报错,宁可不重启也不退化成每日重启。
|
||||
|
||||
> 旧字段 `XRAY_DAILY_RESTART` 已废弃。三个新字段都未设置时它仍作为兼容回退(`true` → 每 1 天,其它 → 关闭),重跑部署后 `.env` 会自动换写成新字段。
|
||||
|
||||
### 查看状态 / 日志
|
||||
|
||||
```bash
|
||||
systemctl status xray
|
||||
journalctl -u xray -f
|
||||
journalctl -u xray --tail 100
|
||||
journalctl -u xray -n 100
|
||||
```
|
||||
|
||||
### 备份
|
||||
@@ -235,7 +422,7 @@ systemctl status xray
|
||||
ss -tlnp | grep 443
|
||||
|
||||
# 查看错误日志
|
||||
journalctl -u xray --tail 50
|
||||
journalctl -u xray -n 50
|
||||
```
|
||||
|
||||
### 速度慢
|
||||
|
||||
@@ -0,0 +1,93 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
# ============================================
|
||||
# Xray 备份脚本
|
||||
# 备份 Xray 配置 / 部署配置 / 网络调优参数
|
||||
# 输出目录默认 /var/backups/xray,自动清理 30 天前的旧备份
|
||||
# ============================================
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
cd "$SCRIPT_DIR"
|
||||
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
RED='\033[0;31m'
|
||||
NC='\033[0m'
|
||||
|
||||
log() { echo -e "${GREEN}[INFO]${NC} $*"; }
|
||||
warn() { echo -e "${YELLOW}[WARN]${NC} $*"; }
|
||||
error() { echo -e "${RED}[ERROR]${NC} $*" >&2; }
|
||||
|
||||
if [ "$(id -u)" -ne 0 ]; then
|
||||
error "请使用 root 用户运行: sudo bash backup.sh"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ -f .env ]; then
|
||||
sed -i 's/\r$//' .env
|
||||
set -a; source .env; set +a
|
||||
fi
|
||||
|
||||
BACKUP_DIR="${BACKUP_DIR:-/var/backups/xray}"
|
||||
KEEP_DAYS="${BACKUP_KEEP_DAYS:-30}"
|
||||
STAMP="$(date '+%Y%m%d%H%M%S')"
|
||||
|
||||
mkdir -p "$BACKUP_DIR"
|
||||
chmod 700 "$BACKUP_DIR"
|
||||
|
||||
backed_up=0
|
||||
|
||||
# ===== 1. Xray 配置 =====
|
||||
# 打包 /usr/local/etc/xray 整个目录,恢复时 tar xzf ... -C /usr/local/etc/
|
||||
if [ -d /usr/local/etc/xray ]; then
|
||||
tar czf "${BACKUP_DIR}/xray_config_${STAMP}.tar.gz" -C /usr/local/etc xray
|
||||
log "Xray 配置已备份: ${BACKUP_DIR}/xray_config_${STAMP}.tar.gz"
|
||||
backed_up=$((backed_up + 1))
|
||||
else
|
||||
warn "/usr/local/etc/xray 不存在,跳过"
|
||||
fi
|
||||
|
||||
# ===== 2. 部署配置(.env + 脚本)=====
|
||||
# .env 含私钥,打包后权限收紧到 600
|
||||
deploy_files=()
|
||||
for f in .env deploy.sh uninstall.sh backup.sh upload.sh .env.example; do
|
||||
[ -f "$f" ] && deploy_files+=("$f")
|
||||
done
|
||||
if [ "${#deploy_files[@]}" -gt 0 ]; then
|
||||
tar czf "${BACKUP_DIR}/deploy_${STAMP}.tar.gz" -C "$SCRIPT_DIR" "${deploy_files[@]}"
|
||||
chmod 600 "${BACKUP_DIR}/deploy_${STAMP}.tar.gz"
|
||||
log "部署配置已备份: ${BACKUP_DIR}/deploy_${STAMP}.tar.gz(权限 600,含私钥)"
|
||||
backed_up=$((backed_up + 1))
|
||||
else
|
||||
warn "未找到可备份的部署文件,跳过"
|
||||
fi
|
||||
|
||||
# ===== 3. 网络调优参数 =====
|
||||
if [ -f /etc/sysctl.d/99-xray-turbo.conf ]; then
|
||||
cp /etc/sysctl.d/99-xray-turbo.conf "${BACKUP_DIR}/sysctl_${STAMP}.conf"
|
||||
log "网络调优参数已备份: ${BACKUP_DIR}/sysctl_${STAMP}.conf"
|
||||
backed_up=$((backed_up + 1))
|
||||
else
|
||||
warn "/etc/sysctl.d/99-xray-turbo.conf 不存在,跳过"
|
||||
fi
|
||||
|
||||
if [ "$backed_up" -eq 0 ]; then
|
||||
error "没有任何内容被备份,请确认 Xray 是否已部署"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# ===== 4. 清理过期备份 =====
|
||||
old_count=$(find "$BACKUP_DIR" -maxdepth 1 -type f \
|
||||
\( -name 'xray_config_*.tar.gz' -o -name 'deploy_*.tar.gz' -o -name 'sysctl_*.conf' \) \
|
||||
-mtime "+${KEEP_DAYS}" | wc -l)
|
||||
if [ "$old_count" -gt 0 ]; then
|
||||
find "$BACKUP_DIR" -maxdepth 1 -type f \
|
||||
\( -name 'xray_config_*.tar.gz' -o -name 'deploy_*.tar.gz' -o -name 'sysctl_*.conf' \) \
|
||||
-mtime "+${KEEP_DAYS}" -delete
|
||||
log "已清理 ${old_count} 个超过 ${KEEP_DAYS} 天的旧备份"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
log "备份完成,目录: ${BACKUP_DIR}"
|
||||
ls -lh "$BACKUP_DIR" | tail -n +2 | tail -10
|
||||
@@ -0,0 +1,31 @@
|
||||
==========================================================
|
||||
Xray VLESS-Reality 部署完成 ✅
|
||||
==========================================================
|
||||
|
||||
IP : 179.255.122.99
|
||||
端口 : 443
|
||||
协议 : VLESS
|
||||
UUID : 9cf72f15-8aa2-45fb-9a5a-1dfbfa2a682c
|
||||
流控 : xtls-rprx-vision
|
||||
传输 : tcp
|
||||
安全 : reality
|
||||
SNI : www.apple.com
|
||||
Fingerprint : chrome
|
||||
PublicKey : Ak0Z19O1-D7Mg_NDDKxs156Dvt34u1ZQ2s7-iChHpSc
|
||||
ShortId : 77a2fa8920fc691b
|
||||
|
||||
==========================================================
|
||||
|
||||
>>> VLESS 分享链接(可直接导入客户端):
|
||||
|
||||
vless://9cf72f15-8aa2-45fb-9a5a-1dfbfa2a682c@179.255.122.99:443?encryption=none&flow=xtls-rprx-vision&security=reality&sni=www.apple.com&fp=chrome&pbk=Ak0Z19O1-D7Mg_NDDKxs156Dvt34u1ZQ2s7-iChHpSc&sid=77a2fa8920fc691b&type=tcp#xray-dimt-1
|
||||
|
||||
==========================================================
|
||||
|
||||
⚠️ 请妥善保存以上信息!
|
||||
配置文件: /usr/local/etc/xray/config.json
|
||||
凭据备份: /opt/vps-xray/.env
|
||||
再次查看: bash deploy.sh --show
|
||||
查看日志: journalctl -u xray -f
|
||||
重启服务: systemctl restart xray
|
||||
==========================================================
|
||||
@@ -0,0 +1,31 @@
|
||||
==========================================================
|
||||
Xray VLESS-Reality 部署完成 ✅
|
||||
==========================================================
|
||||
|
||||
IP : 23.106.155.200
|
||||
端口 : 443
|
||||
协议 : VLESS
|
||||
UUID : 6d960afe-aaa1-4b7d-9f39-8a6ed751570d
|
||||
流控 : xtls-rprx-vision
|
||||
传输 : tcp
|
||||
安全 : reality
|
||||
SNI : www.apple.com
|
||||
Fingerprint : chrome
|
||||
PublicKey : bu5Ctzw3Il3hrJBtM9dbF0V0aU0GKtJvzmsPscoXMHg
|
||||
ShortId : cb1431005cf8c4f2
|
||||
|
||||
==========================================================
|
||||
|
||||
>>> VLESS 分享链接(可直接导入客户端):
|
||||
|
||||
vless://6d960afe-aaa1-4b7d-9f39-8a6ed751570d@23.106.155.200:443?encryption=none&flow=xtls-rprx-vision&security=reality&sni=www.apple.com&fp=chrome&pbk=bu5Ctzw3Il3hrJBtM9dbF0V0aU0GKtJvzmsPscoXMHg&sid=cb1431005cf8c4f2&type=tcp#xray-bandwagon-1
|
||||
|
||||
==========================================================
|
||||
|
||||
⚠️ 请妥善保存以上信息!
|
||||
配置文件: /usr/local/etc/xray/config.json
|
||||
凭据备份: /opt/vps-xray/.env
|
||||
再次查看: bash deploy.sh --show
|
||||
查看日志: journalctl -u xray -f
|
||||
重启服务: systemctl restart xray
|
||||
==========================================================
|
||||
@@ -0,0 +1,341 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
# ============================================
|
||||
# Clash.Meta 配置生成器
|
||||
#
|
||||
# 从 deploy.sh 输出的节点信息文件(*info*.md,内含 vless:// 链接)
|
||||
# 生成 Clash.Meta / mihomo 可用的 YAML 配置。
|
||||
#
|
||||
# 用法:
|
||||
# bash gen-clash.sh # 交互式选择节点文件
|
||||
# bash gen-clash.sh a.md b.md # 直接指定
|
||||
# bash gen-clash.sh -o my.yaml a.md # 指定输出文件
|
||||
# bash gen-clash.sh --all # 使用全部 *info*.md
|
||||
#
|
||||
# 本脚本仅在本地运行,不会上传到 VPS。
|
||||
# ============================================
|
||||
|
||||
# 不切换工作目录:候选文件在脚本所在目录里找,
|
||||
# 但命令行显式传入的路径仍相对调用者的当前目录解析
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
|
||||
GREEN='\033[0;32m'; YELLOW='\033[1;33m'; RED='\033[0;31m'; CYAN='\033[0;36m'; NC='\033[0m'
|
||||
log() { echo -e "${GREEN}[INFO]${NC} $*"; }
|
||||
warn() { echo -e "${YELLOW}[WARN]${NC} $*"; }
|
||||
error() { echo -e "${RED}[ERROR]${NC} $*" >&2; }
|
||||
|
||||
OUTPUT="${SCRIPT_DIR}/vpn-vps-xray-optimized.yaml"
|
||||
USE_ALL=0
|
||||
FILES=()
|
||||
|
||||
usage() {
|
||||
sed -n '4,17p' "${BASH_SOURCE[0]}" | sed 's/^# \?//'
|
||||
exit 0
|
||||
}
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
-o|--output) OUTPUT="${2:?-o 需要一个文件名}"; shift 2 ;;
|
||||
--all) USE_ALL=1; shift ;;
|
||||
-h|--help) usage ;;
|
||||
-*) error "未知参数: $1"; usage ;;
|
||||
*) FILES+=("$1"); shift ;;
|
||||
esac
|
||||
done
|
||||
|
||||
# ===== 收集候选文件 =====
|
||||
mapfile -t CANDIDATES < <(ls -1 "${SCRIPT_DIR}"/*info*.md 2>/dev/null || true)
|
||||
|
||||
if [ "${#FILES[@]}" -eq 0 ]; then
|
||||
if [ "${#CANDIDATES[@]}" -eq 0 ]; then
|
||||
error "当前目录下找不到任何 *info*.md 节点信息文件"
|
||||
error "请先在 VPS 上执行 bash deploy.sh --show,把输出保存为 xxx-info.md 放到这里"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ "$USE_ALL" -eq 1 ]; then
|
||||
FILES=("${CANDIDATES[@]}")
|
||||
else
|
||||
echo ""
|
||||
echo -e "${CYAN}可用的节点信息文件:${NC}"
|
||||
for i in "${!CANDIDATES[@]}"; do
|
||||
f="${CANDIDATES[$i]}"
|
||||
# 顺带把链接里的备注名显示出来,方便辨认
|
||||
tag=$(grep -m1 -o 'vless://[^[:space:]]*' "$f" 2>/dev/null | sed 's/.*#//' || true)
|
||||
printf " %2d) %-45s %s\n" "$((i + 1))" "${f#"${SCRIPT_DIR}"/}" "${tag:+[${tag}]}"
|
||||
done
|
||||
echo ""
|
||||
read -r -p "选择要写入配置的节点(空格分隔序号,回车=全选): " picks
|
||||
if [ -z "${picks// /}" ]; then
|
||||
FILES=("${CANDIDATES[@]}")
|
||||
else
|
||||
for n in $picks; do
|
||||
if ! [[ "$n" =~ ^[0-9]+$ ]] || [ "$n" -lt 1 ] || [ "$n" -gt "${#CANDIDATES[@]}" ]; then
|
||||
error "无效序号: $n"
|
||||
exit 1
|
||||
fi
|
||||
FILES+=("${CANDIDATES[$((n - 1))]}")
|
||||
done
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
# ===== 工具函数 =====
|
||||
urldecode() {
|
||||
local s="${1//+/ }"
|
||||
printf '%b' "${s//%/\\x}"
|
||||
}
|
||||
|
||||
# 从 query string 中取出指定参数
|
||||
qs_get() {
|
||||
local query="$1" key="$2" kv
|
||||
local IFS='&'
|
||||
for kv in $query; do
|
||||
if [ "${kv%%=*}" = "$key" ]; then
|
||||
urldecode "${kv#*=}"
|
||||
return
|
||||
fi
|
||||
done
|
||||
}
|
||||
|
||||
# ===== 解析所有节点 =====
|
||||
NAMES=(); SERVERS=(); PORTS=(); UUIDS=(); SNIS=(); PBKS=(); SIDS=(); FPS=(); FLOWS=(); SECS=()
|
||||
|
||||
declare -A name_seen=()
|
||||
|
||||
for f in "${FILES[@]}"; do
|
||||
[ -f "$f" ] || { error "文件不存在: $f"; exit 1; }
|
||||
|
||||
link=$(grep -m1 -o 'vless://[^[:space:]]*' "$f" 2>/dev/null || true)
|
||||
if [ -z "$link" ]; then
|
||||
warn "跳过 ${f#"${SCRIPT_DIR}"/}:未找到 vless:// 链接"
|
||||
continue
|
||||
fi
|
||||
|
||||
rest="${link#vless://}"
|
||||
|
||||
# 备注名(# 之后)
|
||||
if [[ "$rest" == *"#"* ]]; then
|
||||
name=$(urldecode "${rest##*#}")
|
||||
rest="${rest%%#*}"
|
||||
else
|
||||
name=""
|
||||
fi
|
||||
|
||||
uuid="${rest%%@*}"
|
||||
hpq="${rest#*@}"
|
||||
hostport="${hpq%%\?*}"
|
||||
query=""
|
||||
[[ "$hpq" == *"?"* ]] && query="${hpq#*\?}"
|
||||
|
||||
# 兼容 IPv6 字面量 [::1]:443
|
||||
if [[ "$hostport" == \[* ]]; then
|
||||
server="${hostport%%]*}]"
|
||||
port="${hostport##*]:}"
|
||||
else
|
||||
server="${hostport%%:*}"
|
||||
port="${hostport##*:}"
|
||||
fi
|
||||
|
||||
sec=$(qs_get "$query" security); sec="${sec:-none}"
|
||||
sni=$(qs_get "$query" sni)
|
||||
pbk=$(qs_get "$query" pbk)
|
||||
sid=$(qs_get "$query" sid)
|
||||
fp=$(qs_get "$query" fp); fp="${fp:-chrome}"
|
||||
flow=$(qs_get "$query" flow)
|
||||
net=$(qs_get "$query" type); net="${net:-tcp}"
|
||||
|
||||
if [ -z "$uuid" ] || [ -z "$server" ] || [ -z "$port" ]; then
|
||||
error "${f#"${SCRIPT_DIR}"/} 的链接缺少 uuid/server/port,无法解析"
|
||||
exit 1
|
||||
fi
|
||||
if [ "$sec" = "reality" ] && { [ -z "$pbk" ] || [ -z "$sni" ]; }; then
|
||||
error "${f#"${SCRIPT_DIR}"/} 是 reality 节点但缺少 pbk 或 sni"
|
||||
exit 1
|
||||
fi
|
||||
if [ "$net" != "tcp" ]; then
|
||||
warn "${f#"${SCRIPT_DIR}"/}: transport=${net},本生成器目前只支持 tcp,已跳过"
|
||||
continue
|
||||
fi
|
||||
|
||||
[ -z "$name" ] && name="${server}"
|
||||
# 名称去重,Clash 要求 proxy name 唯一
|
||||
base="$name"; n=2
|
||||
while [ -n "${name_seen[$name]:-}" ]; do
|
||||
name="${base}-${n}"; n=$((n + 1))
|
||||
done
|
||||
name_seen["$name"]=1
|
||||
|
||||
NAMES+=("$name"); SERVERS+=("$server"); PORTS+=("$port"); UUIDS+=("$uuid")
|
||||
SNIS+=("$sni"); PBKS+=("$pbk"); SIDS+=("$sid")
|
||||
FPS+=("$fp"); FLOWS+=("$flow"); SECS+=("$sec")
|
||||
|
||||
log "解析成功: ${name} (${server}:${port}, ${sec})"
|
||||
done
|
||||
|
||||
if [ "${#NAMES[@]}" -eq 0 ]; then
|
||||
error "没有解析出任何可用节点"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# ===== 备份既有输出 =====
|
||||
if [ -f "$OUTPUT" ]; then
|
||||
cp "$OUTPUT" "${OUTPUT}.bak"
|
||||
log "已备份原配置: ${OUTPUT}.bak"
|
||||
fi
|
||||
|
||||
# ===== 生成 YAML =====
|
||||
{
|
||||
cat <<'HEADER'
|
||||
# ============================================================
|
||||
# 由 gen-clash.sh 自动生成,请勿手工编辑(重新生成会覆盖)
|
||||
#
|
||||
# ⚠ 需要 Clash.Meta / mihomo 内核(Clash Verge Rev、ClashX Meta、
|
||||
# FlClash 等)。原版 Clash / Clash Premium 不支持 VLESS + Reality。
|
||||
#
|
||||
# ⚠ 本文件含真实 UUID 与服务器地址,等同凭据,切勿提交到版本库或外传。
|
||||
# ============================================================
|
||||
|
||||
mixed-port: 7890
|
||||
allow-lan: false
|
||||
mode: rule
|
||||
log-level: info
|
||||
|
||||
# fake-ip 下开启嗅探,才能从流量中还原域名,让域名类规则对直连 IP 也生效
|
||||
sniffer:
|
||||
enable: true
|
||||
sniff:
|
||||
HTTP:
|
||||
ports: [80, 8080-8880]
|
||||
override-destination: true
|
||||
TLS:
|
||||
ports: [443, 8443]
|
||||
QUIC:
|
||||
ports: [443, 8443]
|
||||
skip-domain:
|
||||
- "+.push.apple.com"
|
||||
|
||||
dns:
|
||||
enable: true
|
||||
ipv6: false
|
||||
enhanced-mode: fake-ip
|
||||
fake-ip-range: 198.18.0.1/16
|
||||
# 这些域名必须拿到真实 IP,否则局域网发现、时间同步、
|
||||
# 网络连通性检测等会出问题
|
||||
fake-ip-filter:
|
||||
- "*.lan"
|
||||
- "*.local"
|
||||
- "*.localdomain"
|
||||
- "*.home.arpa"
|
||||
- "+.msftconnecttest.com"
|
||||
- "+.msftncsi.com"
|
||||
- "time.*.com"
|
||||
- "ntp.*.com"
|
||||
- "+.pool.ntp.org"
|
||||
- "localhost.ptlogin2.qq.com"
|
||||
# 用于解析下面 fallback 里 DoH 服务器自身的域名
|
||||
default-nameserver:
|
||||
- 223.5.5.5
|
||||
- 119.29.29.29
|
||||
nameserver:
|
||||
- 223.5.5.5
|
||||
- 119.29.29.29
|
||||
# fallback 的意义是绕过投毒,必须用加密 DNS——
|
||||
# 明文 UDP 的 8.8.8.8 在国内同样会被污染,等于没有 fallback
|
||||
fallback:
|
||||
- https://1.1.1.1/dns-query
|
||||
- https://dns.google/dns-query
|
||||
fallback-filter:
|
||||
geoip: true
|
||||
geoip-code: CN
|
||||
ipcidr:
|
||||
- 240.0.0.0/4
|
||||
- 0.0.0.0/32
|
||||
|
||||
proxies:
|
||||
HEADER
|
||||
|
||||
for i in "${!NAMES[@]}"; do
|
||||
echo " - name: \"${NAMES[$i]}\""
|
||||
echo " type: vless"
|
||||
echo " server: ${SERVERS[$i]}"
|
||||
echo " port: ${PORTS[$i]}"
|
||||
echo " uuid: ${UUIDS[$i]}"
|
||||
echo " network: tcp"
|
||||
echo " udp: true"
|
||||
if [ "${SECS[$i]}" = "reality" ]; then
|
||||
echo " tls: true"
|
||||
[ -n "${FLOWS[$i]}" ] && echo " flow: ${FLOWS[$i]}"
|
||||
echo " servername: ${SNIS[$i]}"
|
||||
echo " client-fingerprint: ${FPS[$i]}"
|
||||
echo " reality-opts:"
|
||||
echo " public-key: ${PBKS[$i]}"
|
||||
[ -n "${SIDS[$i]}" ] && echo " short-id: ${SIDS[$i]}"
|
||||
else
|
||||
echo " tls: false"
|
||||
fi
|
||||
echo ""
|
||||
done
|
||||
|
||||
cat <<'GROUPS_HEAD'
|
||||
proxy-groups:
|
||||
- name: "Proxy"
|
||||
type: select
|
||||
proxies:
|
||||
- Auto-Select
|
||||
GROUPS_HEAD
|
||||
|
||||
for n in "${NAMES[@]}"; do echo " - \"${n}\""; done
|
||||
echo " - DIRECT"
|
||||
echo ""
|
||||
|
||||
cat <<'AUTO_HEAD'
|
||||
- name: "Auto-Select"
|
||||
type: url-test
|
||||
url: "http://www.gstatic.com/generate_204"
|
||||
interval: 300
|
||||
tolerance: 50
|
||||
proxies:
|
||||
AUTO_HEAD
|
||||
|
||||
for n in "${NAMES[@]}"; do echo " - \"${n}\""; done
|
||||
echo ""
|
||||
|
||||
echo "rules:"
|
||||
echo " # 代理服务器自身必须直连。否则 TUN / 系统代理会把「连接代理服务器」"
|
||||
echo " # 这件事本身也丢进代理,形成回环,表现为 REALITY 握手拿到真证书。"
|
||||
declare -A srv_seen=()
|
||||
for s in "${SERVERS[@]}"; do
|
||||
[ -n "${srv_seen[$s]:-}" ] && continue
|
||||
srv_seen["$s"]=1
|
||||
if [[ "$s" =~ ^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
|
||||
echo " - IP-CIDR,${s}/32,DIRECT,no-resolve"
|
||||
elif [[ "$s" == \[*\] ]]; then
|
||||
echo " - IP-CIDR6,${s//[\[\]]/}/128,DIRECT,no-resolve"
|
||||
else
|
||||
echo " - DOMAIN,${s},DIRECT"
|
||||
fi
|
||||
done
|
||||
|
||||
cat <<'RULES'
|
||||
# 本机与局域网直连(fake-ip 下不加这条,路由器/NAS 会被丢进代理)
|
||||
- IP-CIDR,127.0.0.0/8,DIRECT,no-resolve
|
||||
- IP-CIDR,10.0.0.0/8,DIRECT,no-resolve
|
||||
- IP-CIDR,172.16.0.0/12,DIRECT,no-resolve
|
||||
- IP-CIDR,192.168.0.0/16,DIRECT,no-resolve
|
||||
- IP-CIDR,169.254.0.0/16,DIRECT,no-resolve
|
||||
# 域名规则放在 IP 规则之前:GEOSITE 直接按域名命中,
|
||||
# 不必先做一次真实 DNS 解析(fake-ip 下 GEOIP 前置会触发额外解析)
|
||||
- GEOSITE,cn,DIRECT
|
||||
- GEOIP,CN,DIRECT
|
||||
# 兜底。MATCH 是终结规则,其后任何规则都不可达
|
||||
- MATCH,Proxy
|
||||
RULES
|
||||
} > "$OUTPUT"
|
||||
|
||||
echo ""
|
||||
log "已生成: ${OUTPUT}(${#NAMES[@]} 个节点)"
|
||||
for n in "${NAMES[@]}"; do echo " - ${n}"; done
|
||||
echo ""
|
||||
warn "该文件含真实凭据,已被根 .gitignore 排除,请勿提交或外传"
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user