diff --git a/script/migrate-db-files-to-oss/.gitignore b/script/migrate-db-files-to-oss/.gitignore
new file mode 100644
index 0000000..ef2f8f7
--- /dev/null
+++ b/script/migrate-db-files-to-oss/.gitignore
@@ -0,0 +1,4 @@
+node_modules/
+config.local.json
+*.local.json
+reports/
diff --git a/script/migrate-db-files-to-oss/README.md b/script/migrate-db-files-to-oss/README.md
new file mode 100644
index 0000000..59fda16
--- /dev/null
+++ b/script/migrate-db-files-to-oss/README.md
@@ -0,0 +1,155 @@
+# 数据库图片迁移到阿里云 OSS
+
+纯 JavaScript / Node.js 工具。默认自动读取项目当前环境的主数据库连接，从数据库识别旧数据库存储和默认阿里云 OSS，直接读取原图上传并校验，然后更新文件表和业务字段中的旧地址。
+
+**通常不需要填写配置，不需要设置数据库密码环境变量，也不需要启动本地后端。** 需要 Node.js 20.19+、MySQL 5.7/8.x 和项目已设置好的默认阿里云 OSS。
+
+## 直接使用
+
+在本目录执行：
+
+```sh
+npm ci --ignore-scripts
+node migrate.mjs init
+```
+
+这会读取代码里的数据库连接，查询存储配置和已有迁移批次，自动生成 `config.local.json`。不上传图片、不更新数据库、不创建迁移表。数据库密码仍从项目 YAML 读取，OSS 密钥仍从存储配置表读取，不重复写入生成文件。
+
+接着只读预检，或开始迁移：
+
+```sh
+node migrate.mjs
+node migrate.mjs apply --limit 50
+node migrate.mjs apply
+```
+
+第一条没有参数，默认只读预检；第二条实际迁移少量图片；第三条继续完成剩余迁移。可跳过 init，预检时也会自动生成配置。中断或失败后重跑相同 apply 命令即可。
+
+预检只读文件元数据，不读取图片二进制内容或计算 SHA-256。图片分组、文件记录和路径拼写由三次批量查询完成，不再逐张反复访问远程数据库。输出包含当前阶段、扫描表名和行数；等待较长查询时每 5 秒显示耗时。`estimatedBytes` 是文件表记录的容量估算，`sizePending` 是缺少容量信息的文件数，实际长度和哈希在 apply 时校验。
+
+上传时的源文件复核也分两步：先只查最新记录编号，再按主键读取该图片的长度和哈希，避免在没有路径索引的排序查询中读取图片大字段。每张图片完成后会显示源读取/复核、OSS 上传/校验、最终地址校验及总耗时，便于区分数据库等待与网络传输。
+
+“更新已校验文件记录”是数据库事务阶段，不是再次上传。脚本每批最多处理 50 张：按主键范围加锁，批量复核源文件最新版本和哈希，再批量更新文件记录、写入逐条回滚日志及迁移状态，一起提交。保留原值比较和事务保护，不需要新增配置或给业务表加索引。该阶段重新计时，并显示每批源校验、记录及日志、提交的耗时。已知源变化或原值冲突会回滚整批，再逐项定位；连接或提交异常则停止，避免重复提交。
+
+按 Ctrl+C 中断后，保留 `config.local.json` 和同一个 runId，重新执行 `node migrate.mjs apply` 即可续跑。已提交记录保留，未提交事务由数据库回滚。处于 `verified` 的已上传文件采用最多 4 张并发，重新读取 OSS 内容和最终访问地址核对哈希，不重新下载数据库图片，也不重新上传；切换时仍在事务里复核源内容。已切换文件也会批量核对文件记录，并并发检查访问地址。校验失败的图片不会用于更新业务引用。更新脚本后，需要停止旧进程再执行，新代码才会生效。
+
+文件总大小不等于网络总传输量：脚本在运行电脑上从数据库读取、上传 OSS，并读取 OSS 及最终访问地址进行校验。macOS 系统代理开关也不代表 Node 默认会使用代理；需要结合进程代理变量、实际连接及系统路由判断。此脚本不修改系统代理设置。
+
+如需回滚：
+
+```sh
+node migrate.mjs rollback
+```
+
+也可以使用 `npm run init`、`npm run plan`、`npm run apply`、`npm run rollback`。生成的配置和报告已忽略，不进入 Git。
+
+## 自动识别的内容
+
+- 环境：显式 `--profile` 优先，其次 `SPRING_PROFILES_ACTIVE`，否则按项目 `application.yaml`。当前源码默认是 dev；生成文件会固定已选环境，避免下一次切换环境后误续跑。
+- 数据库：合并基础 YAML 和对应环境 YAML，读取 dynamic.primary 指定的主数据源，包括地址、端口、数据库、用户名、密码。按需要解析环境变量占位符。
+- 源配置：从仍有未删除 BLOB 的数据库存储配置中识别，不依赖迁移过程中会变化的文件表 config_id。
+- 目标：唯一的默认存储配置，且必须是阿里云 OSS。不能确定时停止，不自动猜选。
+- 历史域名：从旧文件地址提取；OSS Bucket、访问域名和密钥直接使用已有配置。
+- 批次：已有批次优先续跑；首次生成稳定批次名称。保留服务器身份、存储编号和扫描范围，避免换库或换目标后误续跑。
+- 下载接口：根据项目端口和 API 前缀推导。当前源码端口是 48083；默认直接读数据库，不依赖这个接口。
+
+项目 dev/local 的数据库可能是远程地址，不能把环境名 local 当成本机数据库。每次执行会显示实际连接地址和库名。工具读取仓库配置及支持的环境变量，无法知道部署服务器额外的 Docker/命令行覆盖。
+
+多个未完成批次、默认 OSS 改变、数据库服务器改变等情况会明确提示。若固定批次建立后又出现新 BLOB，可显式新建批次：
+
+```sh
+node migrate.mjs apply --run-id db-images-next
+```
+
+新批次名称会保存到自动配置。完成回滚的批次不会自动重新迁移。
+
+## 仅在需要时覆盖
+
+正常不需要复制示例或逐项编辑。`config.example.json` 为 `{}`，表示全部自动识别；`config.manual.example.json` 保留完全手工配置方式，兼容旧用法。
+
+例如切换项目环境：
+
+```sh
+node migrate.mjs init --profile local
+```
+
+同一生成配置绑定原数据库；如果切换后指向不同数据库，会停止，不能把原批次用于另一台服务器。
+
+其他可选设置仍可写入配置文件：`referenceSchemas` 添加同一 MySQL 实例的 IM 库，`imagesOnly: false` 包含非图片，`sourceMode: "http"` 使用原下载接口。手动连接另一数据库时可用 `--config config.manual.local.json`，其中 mysql.passwordEnv 指定的环境变量需要自行提供。
+
+迁移前备份数据库，保持原图片和目标配置稳定。回滚只恢复本批次写入且没有被后来修改的值，不删除原 BLOB 或 OSS 对象。最终清理仍是独立步骤。
+
+## 实际处理流程
+
+1. 按数据库自身的路径比较规则，选取每个 `config_id + path` 最大的未删除内容 ID，与 `DBFileClient.getContent` 一致。大整数 ID 全程使用字符串。
+2. 默认处理 MIME 为 `image/*` 或常见图片扩展名的内容；没有文件表记录但可以识别为图片的 BLOB 也会处理。其他文件计入跳过数量；如确需迁移全部数据库文件，设置 `imagesOnly: false`。无法识别的无扩展名内容需要单独核查。
+3. 建立固定迁移清单，保存原文件记录和旧地址。默认直接读取数据库原始字节；HTTP 模式通过旧下载接口读取。两种方式均与数据库计算的长度、SHA-256 比较。
+4. 使用独立目录 `objectPrefix/旧配置编号/路径哈希/内容哈希.扩展名` 上传；保留原始文件字节，不压缩图片。上传前后均检查对象，不覆盖不同内容。
+5. 从 OSS 重新读取校验，并匿名访问最终将写入数据库的 URL 再次校验。SDK 上传成功但浏览器地址不可访问时，不切换数据库记录。
+6. 本次选中文件全部通过上传校验后，每批最多 50 张用一个事务更新 `infra_file.config_id/path/url/type/size`，保留文件记录 ID 和自动更新时间。逐条回滚日志与文件记录、迁移状态同时提交。
+7. 扫描指定数据库的字符、文本、JSON 字段，按精确地址映射替换单 URL、逗号列表、JSON 值及可识别的富文本地址。每行更新都检查原值，避免覆盖用户刚修改的数据。原生 JSON 的日志保存数据库实际持久化结果，便于正确回滚。
+
+上传图片是逐个进行的；正式迁移按主键分批计算内容哈希，不会一次加载整个 BLOB 表。文件元数据和 URL 映射保存在内存；引用读取每页至少 1000 行，更新仍逐行执行。MySQL 当前会话的单条 SELECT 默认最多执行 60 秒，沿用 `timeoutMs` 配置，不修改服务器全局设置。
+
+## OSS 访问方式
+
+- 默认 `targetUrlMode: "direct"`，业务字段保存 OSS/CDN 稳定地址。地址优先使用 `targetPublicDomain`，其次使用存储配置的 `domain`，最后根据 Bucket 和 OSS 节点生成公网域名。Bucket/CDN 应允许这些图片按该地址读取；脚本不会修改 Bucket 权限。
+- 私有 Bucket 可使用 `targetUrlMode: "backend"`，业务字段保存现有后端下载入口加目标配置编号的地址，后端通过自己的 OSS 凭据读取。`downloadBaseUrl` 此时必须是用户设备能访问的稳定地址，不能填仅运行脚本机器可访问的 localhost。现有接口是 `@PermitAll`，此模式沿用项目已有访问规则，不新增鉴权。
+- 默认 `sourceMode: "database"`，无需启动旧后端。也可设置 `sourceMode: "http"` 使用原下载接口。使用 backend 目标模式时，后端仍须在线。
+
+## 旧地址与迁移范围
+
+修改文件表不会自动改变旧接口的行为：它仍按 URL 中的配置编号直接读取存储。因此脚本始终保留 `infra_file_content` 和原数据库配置，让旧客户端缓存及未替换引用仍有机会继续读取。
+
+过渡期通过文件管理删除 OSS 文件，不会自动删除旧数据库副本。历史副本的删除和最终清理需要关联处理；本脚本不承担清理工作。续跑会重新检查已迁移对象和文件记录，目标已被删除或改动时停止复用对应映射。
+
+旧接口会解码路径一次，并将裸 `+` 当空格。脚本下载时逐段编码；旧 URL 只有在其实际解码结果对应当前文件时才加入映射。裸 `+`、已有 `%HH`、`?/#` 等存在歧义的旧地址不会被静默指向另一张图片，未解决的引用会进入复核统计。
+
+未知查询参数、片段、相对地址和无法识别的文本组合不做猜测替换。JSON 属性名不改。历史域名别名通过 `oldDomains` 补充。默认扫描主数据库，额外数据库只扫描同一个 MySQL 实例且当前账号有权限的 `referenceSchemas`；不同实例的 IM 库需要单独处理。
+
+IM 自己上传到 MinIO 的文件不属于本次 BLOB 搬迁，但 IM 中保存的主库旧图片 URL 可以随引用扫描替换。Redis、客户端本地缓存、其他数据库实例和外部系统不在扫描范围，旧 BLOB 不能在脚本结束后立即清理。
+
+业务引用表需要单列主键；无主键、复合主键表会列入待复核。更新要求 InnoDB。视图、生成列、文件基础表和迁移日志表不参与业务引用替换。`excludeReferenceTables` 可以填写明确不需要扫描的表名或 `库名.表名`；这些主动排除项会在报告中列出。
+
+接口访问日志、接口错误日志、定时任务日志、系统操作日志和登录日志也默认排除：它们不是页面图片引用，迁移不应改写历史记录。具体为 `infra_api_access_log`、`infra_api_error_log`、`infra_job_log`、`system_operate_log`、`system_login_log`，不会泛化排除其他名字包含 log 的业务表。
+
+## 结果、恢复与权限
+
+每次命令在配置文件旁的 `reports/` 写 JSON 报告：
+
+| 状态 | 含义 |
+| --- | --- |
+| `planned` | 数据库预检通过，尚未校验 HTTP/OSS 连通性 |
+| `complete` | 本批文件已切换，已扫描范围未检测到剩余旧路由引用 |
+| `partial` | 限量试迁移、上传失败、文件记录冲突或业务更新冲突，需要续跑/处理 |
+| `needs-review` | 预检/迁移扫描仍发现旧地址或不支持的表；可能包含有意跳过的非图片、未知 URL 变体，不能据此清理源数据 |
+| `rolled_back` / `rollback_partial` | 回滚完成 / 仍有并发修改冲突 |
+
+退出码：0 为本次命令成功；2 为部分完成或需要复核；1 为执行错误。
+失败时只输出简短错误，不输出配置密钥；详细原值、映射及逐项状态保存在主库的三个表：
+
+```text
+infra_file_oss_migration_runs
+infra_file_oss_migration_items
+infra_file_oss_migration_changes
+```
+
+`plan` 不创建这些表、不上传、不更新业务数据；`apply` 首次执行会创建日志表。更新与日志在同一事务中提交，回滚不依赖本地报告。保留这些表及数据库备份直到验收结束。
+
+同一主库的 apply/rollback 使用 MySQL 命名锁避免重复运行。同一 runId 绑定原数据库服务器、存储目标和扫描范围，不能换库或换 Bucket 后继续执行。需要 SELECT/UPDATE、主库创建和写入日志表的权限，以及 OSS 对指定对象的读取/写入权限；不需要 OSS 删除权限。
+
+脚本会开启当前数据库连接的严格模式，并在 SQL 出现截断等警告时回滚，防止较短的业务字段保存损坏 URL。apply 将当前连接的隔离级别设为 REPEATABLE READ，要求源内容表、文件表和日志表为 InnoDB；批次源校验的主键范围锁覆盖可能新增或恢复的较新版本，防止校验期间源内容变化。不修改数据库全局隔离级别。配置文件、依赖目录和报告已通过此目录的 `.gitignore` 排除；回滚日志包含原业务值，应按业务数据库的权限保护。
+
+## 验证
+
+```sh
+npm test
+```
+
+默认运行本地 HTTP、模拟 OSS 和事务接口的测试。真实 MySQL 集成测试默认跳过；仅可指向专用临时 MySQL 容器的端口：
+
+```sh
+MIGRATION_TEST_DB_PORT=临时容器端口 node --test test/integration.test.mjs
+```
+
+集成测试会创建并删除固定的 `codex_oss_migration_test` 测试库，不可指向业务数据库服务器。其 OSS 是内存模拟，HTTP 下载接口由本地测试服务模拟；真实阿里云连接和实际业务页面仍需通过少量试迁移验收。
diff --git a/script/migrate-db-files-to-oss/config.example.json b/script/migrate-db-files-to-oss/config.example.json
new file mode 100644
index 0000000..0967ef4
--- /dev/null
+++ b/script/migrate-db-files-to-oss/config.example.json
@@ -0,0 +1 @@
+{}
diff --git a/script/migrate-db-files-to-oss/config.manual.example.json b/script/migrate-db-files-to-oss/config.manual.example.json
new file mode 100644
index 0000000..598a237
--- /dev/null
+++ b/script/migrate-db-files-to-oss/config.manual.example.json
@@ -0,0 +1,25 @@
+{
+ "runId": "db-images-20261105",
+ "mysql": {
+ "host": "127.0.0.1",
+ "port": 3306,
+ "user": "root",
+ "passwordEnv": "MIGRATION_DB_PASSWORD",
+ "database": "YOUR_DATABASE"
+ },
+ "sourceConfigIds": [],
+ "targetConfigId": null,
+ "sourceMode": "http",
+ "downloadBaseUrl": "http://127.0.0.1:48083/admin-api/infra/file",
+ "oldDomains": [],
+ "targetUrlMode": "direct",
+ "targetPublicDomain": "",
+ "objectPrefix": "migration/db-images",
+ "referenceSchemas": [],
+ "excludeReferenceTables": [],
+ "imagesOnly": true,
+ "batchSize": 100,
+ "timeoutMs": 60000,
+ "maxFileBytes": 67108864,
+ "retries": 3
+}
diff --git a/script/migrate-db-files-to-oss/lib/auto-storage.mjs b/script/migrate-db-files-to-oss/lib/auto-storage.mjs
new file mode 100644
index 0000000..9ecf903
--- /dev/null
+++ b/script/migrate-db-files-to-oss/lib/auto-storage.mjs
@@ -0,0 +1,173 @@
+import { baseUrl, bit, id, normalizeConfig, ossSettings, qualified, sha256 } from './core.mjs';
+
+const RUNS_TABLE = 'infra_file_oss_migration_runs';
+const RESTORED_FIELDS = [
+ 'sourceConfigIds', 'targetConfigId', 'sourceMode', 'downloadBaseUrl', 'oldDomains',
+ 'objectPrefix', 'imagesOnly', 'referenceSchemas', 'excludeReferenceTables', 'targetUrlMode'
+];
+const hasValue = (value, key) => Object.prototype.hasOwnProperty.call(value, key) && value[key] !== undefined;
+const sortedIds = values => [...new Set(values.map(id))].sort((left, right) => BigInt(left) < BigInt(right) ? -1 : BigInt(left) > BigInt(right) ? 1 : 0);
+
+function assertIdentity(expected, current) {
+ if (!expected || expected.server !== current.server || expected.schema !== current.schema) {
+ throw new Error('数据库身份与原迁移配置不一致，请检查数据库连接和批次编号');
+ }
+}
+
+function parseRunSettings(value) {
+ let settings;
+ try {
+ settings = typeof value === 'string' ? JSON.parse(value) : value;
+ } catch {
+ throw new Error('迁移批次的配置记录损坏，不能自动恢复');
+ }
+ if (!settings || typeof settings !== 'object' || !settings.binding || typeof settings.binding !== 'object') {
+ throw new Error('迁移批次缺少原始配置绑定，不能自动恢复');
+ }
+ return settings;
+}
+
+function selectRun(rows, requestedRunId) {
+ if (requestedRunId !== undefined) return rows.find(row => row.run_id === requestedRunId);
+ const unfinished = rows.filter(row => !['complete', 'rolled_back'].includes(row.status));
+ if (unfinished.length > 1) throw new Error('存在多个未完成迁移批次，请使用 --run-id 指定要继续的批次');
+ if (unfinished.length === 1) return unfinished[0];
+ if (rows.length > 1) throw new Error('存在多个历史迁移批次，请使用 --run-id 指定批次，或指定新的批次编号');
+ return rows[0];
+}
+
+function contentId(value) {
+ if ((typeof value === 'number' && !Number.isSafeInteger(value)) || !/^\d+$/.test(String(value))) {
+ throw new Error('迁移批次的内容上界无效，不能安全判断增量数据');
+ }
+ return BigInt(value);
+}
+
+/**
+ * Read-only discovery of storage configuration and a stable migration run.
+ * This function never emits logs, persists configuration, or returns the OSS
+ * credentials read from infra_file_config. Explicit input fields take priority
+ * over a restored binding; loadSettings/runApply validate the final fingerprint.
+ */
+export async function discoverStorageConfig(db, input, { command = 'plan' } = {}) {
+ if (!input?.mysql?.database) throw new Error('缺少本地数据库名称，无法自动识别文件存储');
+ const schema = input.mysql.database;
+ const [[serverRow]] = await db.execute('SELECT @@server_uuid AS server_uuid');
+ if (!serverRow?.server_uuid) throw new Error('无法读取数据库服务器身份');
+ const identity = { server: serverRow.server_uuid, schema };
+ if (hasValue(input, '_databaseIdentity')) assertIdentity(input._databaseIdentity, identity);
+
+ const [runTable] = await db.execute(
+ "SELECT 1 AS present FROM information_schema.TABLES WHERE TABLE_SCHEMA=? AND TABLE_NAME=? AND TABLE_TYPE='BASE TABLE' LIMIT 1",
+ [schema, RUNS_TABLE]
+ );
+ let history = [];
+ if (runTable.length) {
+ [history] = await db.execute(`SELECT run_id,settings,status FROM ${qualified(schema, RUNS_TABLE)}`);
+ }
+ const selected = selectRun(history, input.runId);
+ const settings = selected ? parseRunSettings(selected.settings) : undefined;
+ const config = { ...input, mysql: { ...input.mysql } };
+ if (selected) {
+ assertIdentity(settings.binding, identity);
+ for (const key of RESTORED_FIELDS) {
+ if (!hasValue(input, key) && hasValue(settings.binding, key)) {
+ config[key] = structuredClone(settings.binding[key]);
+ }
+ }
+ if (!hasValue(input, 'targetPublicDomain') && hasValue(settings.binding, 'publicDomain')) {
+ config.targetPublicDomain = settings.binding.publicDomain;
+ }
+ config.runId = selected.run_id;
+ if (command === 'apply' && (selected.status.startsWith('rollback') || selected.status === 'rolled_back')) {
+ throw new Error('该批次已开始或完成回滚，不能继续迁移；请继续 rollback，或使用 --run-id 指定新批次');
+ }
+ } else if (command === 'rollback') {
+ throw new Error('未找到可回滚的历史迁移批次，请使用 --run-id 指定已有批次');
+ }
+
+ config._databaseIdentity = identity;
+ config._autoDiscovery = true;
+ delete config._projectDownloadBaseUrl;
+
+ // Rollback only restores journalled database values. Current storage rows,
+ // default targets and OSS credentials are deliberately irrelevant here.
+ if (command === 'rollback') return normalizeConfig(config);
+
+ if (!hasValue(config, 'sourceMode')) config.sourceMode = 'database';
+ if (!hasValue(config, 'downloadBaseUrl') && input._projectDownloadBaseUrl) {
+ config.downloadBaseUrl = input._projectDownloadBaseUrl;
+ }
+ const [storageRows] = await db.execute(`SELECT id,storage,master,config FROM ${qualified(schema, 'infra_file_config')} WHERE deleted=0`);
+ const byId = new Map(storageRows.map(row => [id(row.id), row]));
+ const masters = storageRows.filter(row => bit(row.master));
+ const requireDefault = !hasValue(config, 'targetConfigId') || input._autoDiscovery === true || Boolean(selected);
+ if (requireDefault && masters.length !== 1) {
+ throw new Error('默认文件存储配置必须唯一，请先在文件配置中设置唯一的阿里云 OSS 默认存储');
+ }
+ if (!hasValue(config, 'targetConfigId')) config.targetConfigId = id(masters[0].id);
+ config.targetConfigId = id(config.targetConfigId);
+ if (requireDefault && (id(masters[0].id) !== config.targetConfigId
+ || (selected && id(masters[0].id) !== id(settings.binding.targetConfigId)))) {
+ throw new Error('当前默认 OSS 与原迁移目标不同，不能自动续跑；请恢复原默认配置或使用 --run-id 建立新批次');
+ }
+ const target = byId.get(config.targetConfigId);
+ if (!target) throw new Error('目标文件存储配置不存在或已删除');
+ try {
+ ossSettings(target, config);
+ } catch {
+ // JSON/URL parsing errors must not include raw configuration or secrets.
+ throw new Error('目标默认存储不是有效的阿里云 OSS 配置，请检查类型、endpoint、Bucket 和访问凭据');
+ }
+
+ if (!hasValue(config, 'sourceConfigIds')) {
+ const databaseIds = sortedIds(storageRows.filter(row => Number(row.storage) === 1).map(row => row.id));
+ let contents = [];
+ if (databaseIds.length) {
+ [contents] = await db.execute(
+ `SELECT DISTINCT config_id FROM ${qualified(schema, 'infra_file_content')} WHERE deleted=0 AND config_id IN (${databaseIds.map(() => '?').join(',')})`,
+ databaseIds
+ );
+ }
+ config.sourceConfigIds = sortedIds(contents.map(row => row.config_id));
+ }
+ if (!Array.isArray(config.sourceConfigIds) || !config.sourceConfigIds.length) {
+ throw new Error('未发现包含未删除内容的数据库存储配置，无需迁移');
+ }
+ config.sourceConfigIds = [...new Set(config.sourceConfigIds.map(id))];
+ for (const sourceId of config.sourceConfigIds) {
+ const source = byId.get(sourceId);
+ if (!source || Number(source.storage) !== 1) throw new Error('原数据库存储配置不存在、已删除或类型发生变化，请检查原迁移绑定');
+ if (bit(source.master)) throw new Error('源配置仍是默认上传配置，请先切换默认配置到阿里云 OSS');
+ }
+
+ if (selected && selected.status !== 'inventory' && command === 'apply') {
+ const sourceIds = settings.binding.sourceConfigIds.map(id);
+ if (!sourceIds.length) throw new Error('历史迁移批次的源配置列表为空，不能安全续跑');
+ const [[maximum]] = await db.execute(
+ `SELECT COALESCE(MAX(id),0) AS max_id FROM ${qualified(schema, 'infra_file_content')} WHERE deleted=0 AND config_id IN (${sourceIds.map(() => '?').join(',')})`,
+ sourceIds
+ );
+ if (contentId(maximum.max_id) > contentId(settings.maxId)) {
+ throw new Error('原批次建立后出现新的数据库文件，请使用 --run-id 指定新批次，避免遗漏新增内容');
+ }
+ }
+
+ if (!hasValue(config, 'oldDomains')) {
+ const [domains] = await db.execute(
+ `SELECT DISTINCT SUBSTRING_INDEX(url,'/admin-api/infra/file/',1) AS domain FROM ${qualified(schema, 'infra_file')} WHERE deleted=0 AND config_id IN (${config.sourceConfigIds.map(() => '?').join(',')}) AND url LIKE '%/admin-api/infra/file/%'`,
+ config.sourceConfigIds
+ );
+ const validDomains = new Set();
+ for (const row of domains) {
+ try { validDomains.add(baseUrl(row.domain)); } catch { /* Ignore malformed and credential-bearing historical URLs. */ }
+ }
+ config.oldDomains = [...validDomains].sort();
+ }
+
+ if (!hasValue(config, 'runId')) {
+ const fingerprint = { ...identity, sourceConfigIds: sortedIds(config.sourceConfigIds), targetConfigId: config.targetConfigId };
+ config.runId = `db-images-${sha256(JSON.stringify(fingerprint)).slice(0, 16)}`;
+ }
+ return normalizeConfig(config);
+}
diff --git a/script/migrate-db-files-to-oss/lib/concurrency.mjs b/script/migrate-db-files-to-oss/lib/concurrency.mjs
new file mode 100644
index 0000000..9e07fa7
--- /dev/null
+++ b/script/migrate-db-files-to-oss/lib/concurrency.mjs
@@ -0,0 +1,30 @@
+/**
+ * Map in input order with bounded parallelism. After the first failure, stop
+ * scheduling work and drain every in-flight worker before rejecting.
+ */
+export async function mapBounded(items, limit, worker) {
+ if (!Number.isSafeInteger(limit) || limit < 1) throw new Error('并发数量必须为正整数');
+ const source = Array.from(items);
+ const results = new Array(source.length);
+ let next = 0;
+ let failed = false;
+ let firstError;
+
+ async function run() {
+ while (!failed && next < source.length) {
+ const index = next++;
+ try {
+ results[index] = await worker(source[index], index);
+ } catch (error) {
+ if (!failed) {
+ failed = true;
+ firstError = error;
+ }
+ }
+ }
+ }
+
+ await Promise.all(Array.from({ length: Math.min(limit, source.length) }, () => run()));
+ if (failed) throw firstError;
+ return results;
+}
diff --git a/script/migrate-db-files-to-oss/lib/core.mjs b/script/migrate-db-files-to-oss/lib/core.mjs
new file mode 100644
index 0000000..96d5f83
--- /dev/null
+++ b/script/migrate-db-files-to-oss/lib/core.mjs
@@ -0,0 +1,174 @@
+import { createHash } from 'node:crypto';
+
+export const sha256 = value => createHash('sha256').update(value).digest('hex');
+export const quote = name => '`' + String(name).replaceAll('`', '``') + '`';
+export const qualified = (schema, table) => `${quote(schema)}.${quote(table)}`;
+export const bit = value => Buffer.isBuffer(value) ? value.some(byte => byte !== 0) : value === true || value === 1 || value === '1';
+export const errorText = error => String(error.code || error.message || error.name || '未知错误').replace(/https?:\/\/\S+/g, '[URL]').slice(0, 300);
+
+export function id(value) {
+ if (typeof value === 'number' && !Number.isSafeInteger(value)) throw new Error('配置编号超出安全整数范围，请使用字符串');
+ if (!/^[1-9]\d*$/.test(String(value))) throw new Error('配置编号必须为正整数，不能使用存储类型代替配置编号');
+ return String(value);
+}
+
+export function baseUrl(value) {
+ const parsed = new URL(value);
+ if (!['http:', 'https:'].includes(parsed.protocol) || parsed.username || parsed.password || parsed.search || parsed.hash) {
+ throw new Error('访问基础地址必须是无账号、查询参数和片段的 HTTP(S) URL');
+ }
+ return parsed.href.replace(/\/+$/, '');
+}
+
+export function encodePath(path) {
+ if (!path || path.startsWith('/') || path.includes('\\') || path.split('/').some(part => ['.', '..', ''].includes(part))) {
+ throw new Error('文件路径为空或含不支持的路径段，需单独处理');
+ }
+ return path.split('/').map(encodeURIComponent).join('/');
+}
+
+export const downloadUrl = (base, configId, path) => `${baseUrl(base)}/${id(configId)}/get/${encodePath(path)}`;
+
+export function normalizeConfig(input) {
+ const config = {
+ sourceMode: 'http', targetUrlMode: 'direct', objectPrefix: 'migration/db-images',
+ imagesOnly: true, batchSize: 100, timeoutMs: 60000, maxFileBytes: 64 * 1024 * 1024,
+ retries: 3, oldDomains: [], excludeReferenceTables: [], ...input
+ };
+ if (!/^[a-zA-Z0-9_-]{1,64}$/.test(config.runId || '')) throw new Error('runId 必须是 1～64 位字母、数字、横线或下划线');
+ if (!config.mysql?.database || !config.mysql?.user) throw new Error('请填写 mysql.database 和 mysql.user');
+ if (!Array.isArray(config.sourceConfigIds) || !config.sourceConfigIds.length) throw new Error('请填写 sourceConfigIds（原数据库存储配置编号）');
+ config.sourceConfigIds = [...new Set(config.sourceConfigIds.map(id))];
+ config.targetConfigId = id(config.targetConfigId);
+ if (config.sourceConfigIds.includes(config.targetConfigId)) throw new Error('源配置和 OSS 目标配置不能相同');
+ if (!['http', 'database'].includes(config.sourceMode)) throw new Error('sourceMode 只能为 http 或 database');
+ if (!['direct', 'backend'].includes(config.targetUrlMode)) throw new Error('targetUrlMode 只能为 direct 或 backend');
+ if (config.sourceMode === 'http' || config.targetUrlMode === 'backend') config.downloadBaseUrl = baseUrl(config.downloadBaseUrl);
+ if (!/^[a-zA-Z0-9][a-zA-Z0-9/_-]{0,119}$/.test(config.objectPrefix) || config.objectPrefix.endsWith('/')) throw new Error('objectPrefix 只能包含字母、数字、横线、下划线和目录分隔符');
+ for (const key of ['batchSize', 'timeoutMs', 'maxFileBytes', 'retries']) {
+ if (!Number.isSafeInteger(config[key]) || config[key] < 1) throw new Error(`${key} 必须为正整数`);
+ }
+ if (config.batchSize > 1000 || config.retries > 10) throw new Error('batchSize 上限为 1000，retries 上限为 10');
+ config.oldDomains = config.oldDomains.map(baseUrl);
+ config.referenceSchemas = [...new Set([config.mysql.database, ...(config.referenceSchemas || [])])];
+ return config;
+}
+
+export function parseStorage(row) {
+ return typeof row.config === 'string' ? JSON.parse(row.config) : row.config;
+}
+
+export function ossSettings(row, config) {
+ if (Number(row.storage) !== 20) throw new Error('目标配置不是 S3/OSS 存储（storage=20）');
+ const value = parseStorage(row);
+ const endpoint = new URL(/^https?:\/\//.test(value.endpoint) ? value.endpoint : `https://${value.endpoint}`);
+ if (!/^oss-[a-z0-9-]+\.aliyuncs\.com$/.test(endpoint.hostname) || endpoint.pathname !== '/' || endpoint.search || endpoint.hash || endpoint.username || endpoint.password) {
+ throw new Error('目标 endpoint 必须为阿里云 OSS 区域/加速节点，不能是其他 S3 服务或 Bucket 域名');
+ }
+ if (!value.bucket || !value.accessKey || !value.accessSecret) throw new Error('OSS 配置缺少 bucket/accessKey/accessSecret');
+ const publicDomain = baseUrl(config.targetPublicDomain || value.domain || `https://${value.bucket}.${endpoint.hostname.replace('-internal.', '.')}`);
+ return { endpoint: endpoint.origin, bucket: value.bucket, accessKeyId: value.accessKey, accessKeySecret: value.accessSecret, publicDomain };
+}
+
+export function objectKey(config, item) {
+ const extension = /\.([a-zA-Z0-9]{1,10})$/.exec(item.path)?.[1]?.toLowerCase();
+ return `${config.objectPrefix}/${item.configId}/${sha256(item.path)}/${item.sha256}${extension ? '.' + extension : ''}`;
+}
+
+export function aliases(item, rows, sourceDomain, config) {
+ const result = new Set(rows.map(row => row.url).filter(Boolean));
+ const domains = [...new Set([sourceDomain, ...config.oldDomains].filter(Boolean).map(baseUrl))];
+ const paths = [...new Set([item.path, ...rows.map(row => row.path)])];
+ const sourcePaths = new Set(paths);
+ const addPathAliases = (prefix, path) => {
+ const encoded = encodePath(path);
+ result.add(prefix + path);
+ result.add(prefix + encoded);
+ // FileController uses form-style decoding: a literal '+' means a space.
+ result.add(prefix + encoded.replaceAll('%20', '+'));
+ };
+ for (const domain of domains) for (const path of paths) {
+ const prefix = `${domain}/admin-api/infra/file/${item.configId}/get/`;
+ addPathAliases(prefix, path);
+ }
+ if (config.downloadBaseUrl) {
+ const prefix = baseUrl(config.downloadBaseUrl) + '/' + id(item.configId) + '/get/';
+ for (const path of paths) addPathAliases(prefix, path);
+ }
+ return [...result].filter(value => {
+ try {
+ // Parse the actual request path: raw '?' / '#' end the path, and the
+ // browser encodes spaces/Unicode before FileController sees requestURI.
+ const parsed = new URL(value, 'https://migration-alias.invalid');
+ const route = parsed.pathname.match(/\/infra\/file\/([^/]+)\/get(?:\/|$)/);
+ if (!route) return true; // Keep pre-existing URLs from other file routes.
+ if (BigInt(decodeURIComponent(route[1])) !== BigInt(item.configId)) return false;
+ const marker = parsed.pathname.indexOf('/get/');
+ if (marker < 0) return false;
+ const path = decodeURIComponent(parsed.pathname.slice(marker + 5).replaceAll('+', ' '));
+ return sourcePaths.has(path);
+ } catch {
+ // Unverifiable aliases remain untouched and are reported by the caller.
+ return false;
+ }
+ });
+}
+
+export function mimeType(path, records) {
+ const stored = records.map(row => row.type).find(type => /^image\//i.test(type || ''));
+ if (stored) return stored;
+ const ext = path.split('.').pop().toLowerCase();
+ const types = { png: 'image/png', jpg: 'image/jpeg', jpeg: 'image/jpeg', gif: 'image/gif', webp: 'image/webp', svg: 'image/svg+xml', bmp: 'image/bmp', ico: 'image/x-icon', tif: 'image/tiff', tiff: 'image/tiff', avif: 'image/avif', heic: 'image/heic', heif: 'image/heif' };
+ return types[ext] || records.find(row => row.type)?.type || 'application/octet-stream';
+}
+
+export async function readHttp(url, { timeoutMs, maxFileBytes }, expectedSize) {
+ const response = await fetch(url, { signal: AbortSignal.timeout(timeoutMs), redirect: 'error' });
+ if (!response.ok) { await response.body?.cancel(); throw new Error(`文件访问返回 HTTP ${response.status}`); }
+ const declared = Number(response.headers.get('content-length'));
+ if (declared > maxFileBytes || (expectedSize !== undefined && declared > expectedSize)) {
+ await response.body?.cancel(); throw new Error('HTTP 文件长度超过限制');
+ }
+ let size = 0;
+ const chunks = [];
+ for await (const chunk of response.body) {
+ size += chunk.length;
+ if (size > maxFileBytes || (expectedSize !== undefined && size > expectedSize)) throw new Error('HTTP 文件长度超过限制');
+ chunks.push(chunk);
+ }
+ return Buffer.concat(chunks);
+}
+
+export function verifyContent(bytes, expected) {
+ if (bytes.length !== Number(expected.size) || sha256(bytes) !== expected.sha256) throw new Error('文件长度或 SHA-256 不一致');
+}
+
+export async function ensureObject(oss, key, bytes, item) {
+ let exists = true;
+ try {
+ const result = await oss.head(key);
+ if (Number(result.res.headers['content-length']) !== Number(item.size)) throw new Error('OSS 同名对象长度不同，禁止覆盖');
+ } catch (error) {
+ if (error.code === 'NoSuchKey' || error.status === 404) exists = false;
+ else throw error;
+ }
+ if (!exists) {
+ try {
+ await oss.put(key, bytes, { headers: { 'Content-Type': item.type, 'x-oss-forbid-overwrite': 'true' } });
+ } catch (error) {
+ if (error.code !== 'FileAlreadyExists' && error.status !== 409) throw error;
+ }
+ }
+ const result = await oss.head(key);
+ if (Number(result.res.headers['content-length']) !== Number(item.size)) throw new Error('OSS 文件长度不一致');
+ verifyContent((await oss.get(key)).content, item);
+}
+
+export async function retry(action, attempts, wait = ms => new Promise(resolve => setTimeout(resolve, ms))) {
+ for (let i = 0; ; i++) {
+ try { return await action(); } catch (error) {
+ if (i + 1 >= attempts) throw error;
+ await wait(Math.min(1000 * 2 ** i, 5000));
+ }
+ }
+}
diff --git a/script/migrate-db-files-to-oss/lib/journal.mjs b/script/migrate-db-files-to-oss/lib/journal.mjs
new file mode 100644
index 0000000..e73f978
--- /dev/null
+++ b/script/migrate-db-files-to-oss/lib/journal.mjs
@@ -0,0 +1,318 @@
+import { qualified, quote } from './core.mjs';
+
+const RUNS = 'infra_file_oss_migration_runs';
+const ITEMS = 'infra_file_oss_migration_items';
+const CHANGES = 'infra_file_oss_migration_changes';
+const metadataCache = new WeakMap();
+const own = (object, key) => Object.prototype.hasOwnProperty.call(object, key);
+
+// Snapshots must preserve BIT/Buffer and 64-bit keys without Number conversion.
+function pack(value) {
+ if (Buffer.isBuffer(value)) return { type: 'buffer', value: value.toString('base64') };
+ if (typeof value === 'bigint') return { type: 'bigint', value: value.toString() };
+ if (value instanceof Date) return { type: 'date', value: value.toISOString() };
+ if (value === undefined) throw new Error('变更快照不能包含 undefined');
+ return { type: 'json', value };
+}
+
+function unpack(value) {
+ if (value.type === 'buffer') return Buffer.from(value.value, 'base64');
+ if (value.type === 'bigint') return BigInt(value.value);
+ if (value.type === 'date') return new Date(value.value);
+ if (value.type === 'json') return value.value;
+ throw new Error('变更日志值类型不受支持');
+}
+
+function snapshot(record) {
+ return JSON.stringify(Object.fromEntries(Object.entries(record).map(([key, value]) => [key, pack(value)])));
+}
+
+function restoreSnapshot(text) {
+ return Object.fromEntries(Object.entries(JSON.parse(text)).map(([key, value]) => [key, unpack(value)]));
+}
+
+function parameter(value) {
+ if (typeof value === 'bigint') return value.toString();
+ if (value !== null && typeof value === 'object' && !Buffer.isBuffer(value) && !(value instanceof Date)) return JSON.stringify(value);
+ return value;
+}
+
+const equal = (left, right) => JSON.stringify(pack(left)) === JSON.stringify(pack(right));
+
+function condition(values) {
+ const entries = Object.entries(values);
+ return {
+ sql: entries.map(([key]) => `CAST(${quote(key)} AS BINARY) <=> CAST(? AS BINARY)`).join(' AND '),
+ params: entries.map(([, value]) => parameter(value))
+ };
+}
+
+async function tableMetadata(connection, schema, table) {
+ let cache = metadataCache.get(connection);
+ if (!cache) { cache = new Map(); metadataCache.set(connection, cache); }
+ const key = JSON.stringify([schema, table]);
+ if (!cache.has(key)) {
+ const [rows] = await connection.execute(
+ `SELECT c.COLUMN_NAME AS name, c.EXTRA AS extra, t.ENGINE AS engine
+ FROM information_schema.COLUMNS c JOIN information_schema.TABLES t
+ ON t.TABLE_SCHEMA = c.TABLE_SCHEMA AND t.TABLE_NAME = c.TABLE_NAME
+ WHERE c.TABLE_SCHEMA = ? AND c.TABLE_NAME = ?`, [schema, table]);
+ if (!rows.length || rows.some(row => String(row.engine).toLowerCase() !== 'innodb')) {
+ throw new Error(`表 ${schema}.${table} 必须为 InnoDB，才能保证更新和日志原子提交`);
+ }
+ cache.set(key, {
+ names: new Set(rows.map(row => row.name)),
+ automatic: rows.filter(row => /on update/i.test(row.extra || '')).map(row => row.name)
+ });
+ }
+ return cache.get(key);
+}
+
+function assertFields(metadata, primaryKey, before, after) {
+ if (own(after, primaryKey)) throw new Error('迁移不能修改主键');
+ for (const name of [primaryKey, ...Object.keys(before), ...Object.keys(after)]) {
+ if (!metadata.names.has(name)) throw new Error(`变更列 ${name} 不存在`);
+ }
+ for (const name of Object.keys(after)) {
+ if (!own(before, name)) throw new Error(`变更列 ${name} 缺少原值快照`);
+ }
+}
+
+function assignments(values, metadata) {
+ const entries = Object.entries(values);
+ return {
+ sql: [
+ ...entries.map(([name]) => `${quote(name)} = ?`),
+ ...metadata.automatic.filter(name => !own(values, name)).map(name => `${quote(name)} = ${quote(name)}`)
+ ].join(', '),
+ params: entries.map(([, value]) => parameter(value))
+ };
+}
+
+export async function ensureJournal(connection, schema) {
+ await connection.execute(`CREATE TABLE IF NOT EXISTS ${qualified(schema, RUNS)} (
+ run_id VARCHAR(64) CHARACTER SET ascii COLLATE ascii_bin NOT NULL PRIMARY KEY,
+ settings LONGTEXT NOT NULL,
+ status VARCHAR(32) NOT NULL DEFAULT 'active',
+ created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
+ updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
+ ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin`);
+ await connection.execute(`CREATE TABLE IF NOT EXISTS ${qualified(schema, ITEMS)} (
+ run_id VARCHAR(64) CHARACTER SET ascii COLLATE ascii_bin NOT NULL,
+ item_key CHAR(64) CHARACTER SET ascii COLLATE ascii_bin NOT NULL,
+ data LONGTEXT NOT NULL,
+ status VARCHAR(32) NOT NULL,
+ error TEXT NULL,
+ updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
+ PRIMARY KEY (run_id, item_key)
+ ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin`);
+ await connection.execute(`CREATE TABLE IF NOT EXISTS ${qualified(schema, CHANGES)} (
+ id BIGINT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY,
+ run_id VARCHAR(64) CHARACTER SET ascii COLLATE ascii_bin NOT NULL,
+ schema_name VARCHAR(64) NOT NULL,
+ table_name VARCHAR(64) NOT NULL,
+ pk_name VARCHAR(64) NOT NULL,
+ pk_value TEXT NOT NULL,
+ before_json LONGTEXT NOT NULL,
+ after_json LONGTEXT NOT NULL,
+ rolled_back TINYINT NOT NULL DEFAULT 0,
+ created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
+ KEY ix_run_id_id (run_id, id)
+ ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin`);
+ for (const table of [RUNS, ITEMS, CHANGES]) await tableMetadata(connection, schema, table);
+}
+
+/** The caller owns BEGIN/COMMIT/ROLLBACK, including the change-journal INSERT. */
+export async function applyChange(connection, journalSchema, runId, change) {
+ const { schema, table, primaryKey, pkValue, before, after } = change;
+ const metadata = await tableMetadata(connection, schema, table);
+ assertFields(metadata, primaryKey, before, after);
+ const changed = Object.fromEntries(Object.entries(after).filter(([name, value]) => !equal(before[name], value)));
+ if (!Object.keys(changed).length) return { changed: false };
+ const update = assignments(changed, metadata);
+ const guard = condition(before);
+ const [result] = await connection.execute(
+ `UPDATE ${qualified(schema, table)} SET ${update.sql}
+ WHERE ${quote(primaryKey)} = ?${guard.sql ? ` AND ${guard.sql}` : ''}`,
+ [...update.params, parameter(pkValue), ...guard.params]);
+ if (result.affectedRows !== 1) {
+ const error = new Error(`数据已变化或不存在：${schema}.${table}，本次更新已停止`);
+ error.code = 'MIGRATION_CAS_CONFLICT';
+ throw error;
+ }
+ if (result.warningStatus > 0) {
+ throw new Error('数据库更新产生警告，可能发生字段截断或类型转换；事务必须回滚');
+ }
+ // Native JSON columns normalize whitespace/key order, and MySQL may convert
+ // numeric request values. Rollback must compare against the bytes/type that
+ // the database actually persisted, not the JavaScript UPDATE parameters.
+ const [storedRows] = await connection.execute(
+ `SELECT ${Object.keys(changed).map(quote).join(', ')} FROM ${qualified(schema, table)}
+ WHERE ${quote(primaryKey)} = ? FOR UPDATE`, [parameter(pkValue)]);
+ if (storedRows.length !== 1 || Object.keys(changed).some(name => !own(storedRows[0], name))) {
+ throw new Error('无法读取更新后的记录，事务必须回滚');
+ }
+ const persisted = Object.fromEntries(Object.keys(changed).map(name => [name, storedRows[0][name]]));
+ const [journal] = await connection.execute(
+ `INSERT INTO ${qualified(journalSchema, CHANGES)}
+ (run_id, schema_name, table_name, pk_name, pk_value, before_json, after_json)
+ VALUES (?, ?, ?, ?, ?, ?, ?)`,
+ [runId, schema, table, primaryKey, JSON.stringify(pack(pkValue)), snapshot(before), snapshot(persisted)]);
+ return { changed: true, journalId: String(journal.insertId) };
+}
+
+function primaryKeyIdentity(value) {
+ if (Buffer.isBuffer(value)) return `buffer:${value.toString('base64')}`;
+ if (!['string', 'number', 'bigint'].includes(typeof value)
+ || (typeof value === 'number' && !Number.isSafeInteger(value))) {
+ throw new Error('批量变更主键必须是字符串、安全整数、BigInt 或 Buffer');
+ }
+ // mysql2 may return integer primary keys as either strings or numbers.
+ return `scalar:${String(value)}`;
+}
+
+/**
+ * Apply one homogeneous batch with one CASE UPDATE, one persisted-value read,
+ * and one journal INSERT. The caller must roll back the entire transaction on
+ * any error: the UPDATE can have matched only part of a concurrently edited
+ * batch before affectedRows reports the conflict.
+ */
+export async function applyChanges(connection, journalSchema, runId, changes) {
+ if (!Array.isArray(changes)) throw new Error('批量变更必须为数组');
+ if (!changes.length) return { changed: 0, skipped: 0 };
+ const { schema, table, primaryKey } = changes[0];
+ const afterFields = Object.keys(changes[0].after).sort();
+ const identities = new Set();
+ for (const change of changes) {
+ if (change.schema !== schema || change.table !== table || change.primaryKey !== primaryKey) {
+ throw new Error('批量变更必须使用相同的数据库、表和主键列');
+ }
+ if (JSON.stringify(Object.keys(change.after).sort()) !== JSON.stringify(afterFields)) {
+ throw new Error('批量变更的目标字段集合必须一致');
+ }
+ const identity = primaryKeyIdentity(change.pkValue);
+ if (identities.has(identity)) throw new Error('批量变更包含重复主键');
+ identities.add(identity);
+ }
+ const metadata = await tableMetadata(connection, schema, table);
+ const active = [];
+ for (const change of changes) {
+ assertFields(metadata, primaryKey, change.before, change.after);
+ const changed = Object.fromEntries(Object.entries(change.after).filter(([name, value]) => !equal(change.before[name], value)));
+ if (Object.keys(changed).length) active.push({ ...change, changed });
+ }
+ const skipped = changes.length - active.length;
+ if (!active.length) return { changed: 0, skipped };
+
+ const fields = afterFields.filter(name => active.some(change => own(change.changed, name)));
+ const updateParams = [];
+ const set = fields.map(name => {
+ const branches = active.filter(change => own(change.changed, name)).map(change => {
+ updateParams.push(parameter(change.pkValue), parameter(change.changed[name]));
+ return 'WHEN ? THEN ?';
+ });
+ return `${quote(name)} = CASE ${quote(primaryKey)} ${branches.join(' ')} ELSE ${quote(name)} END`;
+ });
+ set.push(...metadata.automatic.filter(name => !fields.includes(name)).map(name => `${quote(name)} = ${quote(name)}`));
+ const where = active.map(change => {
+ const guard = condition(change.before);
+ updateParams.push(parameter(change.pkValue), ...guard.params);
+ return `(${quote(primaryKey)} = ?${guard.sql ? ` AND ${guard.sql}` : ''})`;
+ });
+ const [updated] = await connection.execute(
+ `UPDATE ${qualified(schema, table)} SET ${set.join(', ')} WHERE ${where.join(' OR ')}`, updateParams);
+ if (updated.affectedRows !== active.length) {
+ const error = new Error(`批量更新发现数据已变化或不存在：${schema}.${table}，必须回滚整批事务`);
+ error.code = 'MIGRATION_CAS_CONFLICT';
+ throw error;
+ }
+ if (updated.warningStatus > 0) throw new Error('数据库批量更新产生警告，可能发生字段截断或类型转换；事务必须回滚');
+
+ const [storedRows] = await connection.execute(
+ `SELECT ${[primaryKey, ...fields].map(quote).join(', ')} FROM ${qualified(schema, table)}
+ WHERE ${quote(primaryKey)} IN (${active.map(() => '?').join(', ')}) FOR UPDATE`,
+ active.map(change => parameter(change.pkValue)));
+ if (storedRows.length !== active.length) throw new Error('无法读取批量更新后的全部记录，事务必须回滚');
+ const storedByKey = new Map();
+ for (const row of storedRows) {
+ if (![primaryKey, ...fields].every(name => own(row, name))) throw new Error('批量更新读回记录缺少字段，事务必须回滚');
+ const identity = primaryKeyIdentity(row[primaryKey]);
+ if (storedByKey.has(identity)) throw new Error('批量更新读回记录包含重复主键，事务必须回滚');
+ storedByKey.set(identity, row);
+ }
+ const journalParams = [];
+ for (const change of active) {
+ const row = storedByKey.get(primaryKeyIdentity(change.pkValue));
+ if (!row) throw new Error('批量更新读回主键与请求不一致，事务必须回滚');
+ const persisted = Object.fromEntries(Object.keys(change.changed).map(name => [name, row[name]]));
+ journalParams.push(runId, schema, table, primaryKey, JSON.stringify(pack(change.pkValue)), snapshot(change.before), snapshot(persisted));
+ }
+ const [journal] = await connection.execute(
+ `INSERT INTO ${qualified(journalSchema, CHANGES)}
+ (run_id, schema_name, table_name, pk_name, pk_value, before_json, after_json)
+ VALUES ${active.map(() => '(?, ?, ?, ?, ?, ?, ?)').join(', ')}`, journalParams);
+ if (journal.affectedRows !== active.length || journal.warningStatus > 0) {
+ throw new Error('批量回滚日志未完整保存，事务必须回滚');
+ }
+ return { changed: active.length, skipped };
+}
+
+export async function rollbackChanges(connection, journalSchema, runId) {
+ const result = { restored: 0, alreadyRestored: 0, conflicts: [] };
+ let cursor = null;
+ const journal = qualified(journalSchema, CHANGES);
+ while (true) {
+ const [batch] = await connection.execute(
+ `SELECT CAST(id AS CHAR) AS id FROM ${journal}
+ WHERE run_id = ? AND rolled_back = 0${cursor === null ? '' : ' AND id < ?'} ORDER BY id DESC LIMIT 100`,
+ cursor === null ? [runId] : [runId, cursor]);
+ if (!batch.length) break;
+ for (const entry of batch) {
+ cursor = String(entry.id);
+ await connection.beginTransaction();
+ try {
+ const [[row]] = await connection.execute(`SELECT * FROM ${journal} WHERE id = ? AND run_id = ? FOR UPDATE`, [cursor, runId]);
+ if (!row || Number(row.rolled_back) === 1) { await connection.commit(); continue; }
+ const before = restoreSnapshot(row.before_json);
+ const after = restoreSnapshot(row.after_json);
+ const pkValue = unpack(JSON.parse(row.pk_value));
+ const metadata = await tableMetadata(connection, row.schema_name, row.table_name);
+ assertFields(metadata, row.pk_name, before, after);
+ const target = qualified(row.schema_name, row.table_name);
+ const expectedAfter = condition({ ...before, ...after });
+ const [matchesAfter] = await connection.execute(
+ `SELECT 1 AS matched FROM ${target} WHERE ${quote(row.pk_name)} = ? AND ${expectedAfter.sql} FOR UPDATE`,
+ [parameter(pkValue), ...expectedAfter.params]);
+ let alreadyRestored = false;
+ if (!matchesAfter.length) {
+ const expectedBefore = condition(before);
+ const [matchesBefore] = await connection.execute(
+ `SELECT 1 AS matched FROM ${target} WHERE ${quote(row.pk_name)} = ? AND ${expectedBefore.sql} FOR UPDATE`,
+ [parameter(pkValue), ...expectedBefore.params]);
+ if (!matchesBefore.length) {
+ await connection.rollback();
+ result.conflicts.push({ id: cursor, schema: row.schema_name, table: row.table_name, pkValue: typeof pkValue === 'bigint' ? pkValue.toString() : pkValue });
+ continue;
+ }
+ alreadyRestored = true;
+ } else {
+ const values = Object.fromEntries(Object.keys(after).map(name => [name, before[name]]));
+ const update = assignments(values, metadata);
+ const [updated] = await connection.execute(
+ `UPDATE ${target} SET ${update.sql} WHERE ${quote(row.pk_name)} = ? AND ${expectedAfter.sql}`,
+ [...update.params, parameter(pkValue), ...expectedAfter.params]);
+ if (updated.affectedRows !== 1) throw new Error('已锁定的回滚记录发生变化，事务已终止');
+ if (updated.warningStatus > 0) throw new Error('数据库回滚产生警告，事务已终止');
+ }
+ await connection.execute(`UPDATE ${journal} SET rolled_back = 1 WHERE id = ? AND run_id = ?`, [cursor, runId]);
+ await connection.commit();
+ if (alreadyRestored) result.alreadyRestored++;
+ else result.restored++;
+ } catch (error) {
+ await connection.rollback();
+ throw error;
+ }
+ }
+ }
+ return result;
+}
diff --git a/script/migrate-db-files-to-oss/lib/progress.mjs b/script/migrate-db-files-to-oss/lib/progress.mjs
new file mode 100644
index 0000000..7d47cce
--- /dev/null
+++ b/script/migrate-db-files-to-oss/lib/progress.mjs
@@ -0,0 +1,43 @@
+/**
+ * Keep database waits and long scans visible without logging every record.
+ * update() also checks elapsed time when a synchronous loop delays timers.
+ */
+export function createProgress({ log = console.log, intervalMs = 5000, now = Date.now } = {}) {
+ if (!Number.isSafeInteger(intervalMs) || intervalMs < 1) throw new Error('进度输出间隔必须为正整数毫秒');
+ let message = '';
+ let startedAt;
+ let lastOutput;
+ let closed = false;
+
+ function output(force = false) {
+ if (closed || !message) return;
+ const current = now();
+ if (!force && lastOutput !== undefined && current - lastOutput < intervalMs) return;
+ lastOutput = current;
+ const seconds = Math.max(0, Math.floor((current - startedAt) / 1000));
+ log(message + '（已用时 ' + seconds + ' 秒）');
+ }
+
+ const timer = setInterval(() => output(), intervalMs);
+ timer.unref?.();
+
+ return {
+ stage(value) {
+ if (closed) return;
+ message = String(value ?? '').trim();
+ startedAt = now();
+ output(true);
+ },
+ update(value) {
+ if (closed) return;
+ message = String(value ?? '').trim();
+ if (startedAt === undefined) startedAt = now();
+ output();
+ },
+ close() {
+ if (closed) return;
+ closed = true;
+ clearInterval(timer);
+ },
+ };
+}
diff --git a/script/migrate-db-files-to-oss/lib/project-config.mjs b/script/migrate-db-files-to-oss/lib/project-config.mjs
new file mode 100644
index 0000000..5c57a6f
--- /dev/null
+++ b/script/migrate-db-files-to-oss/lib/project-config.mjs
@@ -0,0 +1,223 @@
+import { readFile } from 'node:fs/promises';
+import { resolve } from 'node:path';
+import { isIP } from 'node:net';
+import { parseAllDocuments } from 'yaml';
+
+const own = (object, key) => Object.prototype.hasOwnProperty.call(object, key);
+const record = value => value !== null && typeof value === 'object' && !Array.isArray(value);
+const canonical = key => key.replace(/[-_]/g, '').toLowerCase();
+const environmentName = path => path.replace(/([a-z0-9])([A-Z])/g, '$1_$2').replace(/[.-]/g, '_').toUpperCase();
+
+function merge(target, source) {
+ const result = Object.assign(Object.create(null), target);
+ for (const [key, value] of Object.entries(source)) {
+ result[key] = record(value) && record(result[key]) ? merge(result[key], value) : value;
+ }
+ return result;
+}
+
+function property(root, path) {
+ const parts = path.split('.');
+ let current = root;
+ for (let index = 0; index < parts.length; index++) {
+ if (!record(current)) return undefined;
+ const remainder = parts.slice(index).join('.');
+ if (own(current, remainder)) return current[remainder];
+ const part = parts[index];
+ let key = part;
+ if (!own(current, key)) {
+ const matches = Object.keys(current).filter(candidate => canonical(candidate) === canonical(part));
+ if (matches.length > 1) throw new Error('配置存在多个等价属性名，无法自动选择；请使用明确的迁移配置');
+ if (!matches.length) return undefined;
+ key = matches[0];
+ }
+ current = current[key];
+ }
+ return current;
+}
+
+function resolver(configuration, env) {
+ const raw = name => {
+ if (own(env, name) && env[name] !== undefined) return env[name];
+ const key = environmentName(name);
+ if (own(env, key) && env[key] !== undefined) return env[key];
+ return property(configuration, name);
+ };
+ const scalar = value => {
+ if (!['string', 'number', 'bigint', 'boolean'].includes(typeof value)) {
+ throw new Error('自动配置所需字段必须为单个值，请检查环境配置');
+ }
+ return String(value);
+ };
+ function expand(value, trail = []) {
+ const text = scalar(value);
+ let output = '', cursor = 0;
+ while (cursor < text.length) {
+ const start = text.indexOf('${', cursor);
+ if (start < 0) return output + text.slice(cursor);
+ output += text.slice(cursor, start);
+ let depth = 1, end = start + 2;
+ for (; end < text.length && depth; end++) {
+ if (text.startsWith('${', end)) { depth++; end++; }
+ else if (text[end] === '}') depth--;
+ }
+ if (depth) throw new Error('自动配置所需字段包含未闭合的占位符，请检查环境配置');
+ const expression = text.slice(start + 2, end - 1);
+ let separator = -1, nested = 0;
+ for (let index = 0; index < expression.length; index++) {
+ if (expression.startsWith('${', index)) { nested++; index++; }
+ else if (expression[index] === '}') nested--;
+ else if (expression[index] === ':' && nested === 0) { separator = index; break; }
+ }
+ const name = (separator < 0 ? expression : expression.slice(0, separator)).trim();
+ if (!name || name.includes('${')) throw new Error('自动配置不支持动态占位符名称，请使用明确的迁移配置');
+ if (trail.includes(name) || trail.length >= 32) throw new Error('自动配置所需字段存在循环占位符引用，请检查环境配置');
+ let replacement = raw(name);
+ if (replacement === undefined || replacement === null) {
+ if (separator < 0) throw new Error('自动配置所需占位符未提供，请设置对应环境变量或使用明确的迁移配置');
+ replacement = expression.slice(separator + 1);
+ }
+ output += expand(replacement, [...trail, name]);
+ cursor = end;
+ }
+ return output;
+ }
+ return {
+ raw,
+ field(name, fallback) {
+ const value = raw(name);
+ if (value === undefined || value === null) {
+ if (fallback !== undefined) return expand(fallback);
+ throw new Error('项目配置缺少迁移所需字段，请检查主数据源配置或使用明确的迁移配置');
+ }
+ return expand(value, [name]);
+ },
+ expand,
+ };
+}
+
+async function yamlFile(path) {
+ let text;
+ try { text = await readFile(path, 'utf8'); }
+ catch { throw new Error('无法读取基础或所选环境 YAML，请检查项目路径和 --profile'); }
+ try {
+ const documents = parseAllDocuments(text, { uniqueKeys: true, intAsBigInt: true });
+ let result = Object.create(null);
+ for (const document of documents) {
+ if (document.errors.length || document.warnings.length) throw new Error();
+ const value = document.toJS({ maxAliasCount: 100 });
+ if (value === null) continue;
+ if (!record(value)) throw new Error();
+ if (property(value, 'spring.config.activate.on-profile') !== undefined
+ || typeof property(value, 'spring.profiles') === 'string'
+ || Array.isArray(property(value, 'spring.profiles'))) {
+ throw Object.assign(new Error('自动配置不支持带 profile 条件的 YAML 文档，请使用明确的迁移配置'), { unsupported: true });
+ }
+ result = merge(result, value);
+ }
+ return result;
+ } catch (error) {
+ if (error.unsupported) throw error;
+ throw new Error('YAML 无法安全解析，请检查缩进、重复键和文档结构');
+ }
+}
+
+function singleProfile(value) {
+ if (typeof value !== 'string' || !/^[a-zA-Z0-9][a-zA-Z0-9._-]*$/.test(value.trim())) {
+ throw new Error('自动配置仅支持单个 profile，请使用 --profile dev、local 或 prod 明确选择');
+ }
+ return value.trim();
+}
+
+function assertSelectionSupported(configuration, profile, env) {
+ const resolved = resolver(configuration, env);
+ const imports = resolved.raw('spring.config.import');
+ const includes = resolved.raw('spring.profiles.include');
+ if (imports !== undefined && imports !== null && imports !== '' && (!Array.isArray(imports) || imports.length)) {
+ throw new Error('自动配置不支持 spring.config.import，请使用明确的迁移配置');
+ }
+ if (includes !== undefined && includes !== null && includes !== '' && (!Array.isArray(includes) || includes.length)) {
+ throw new Error('自动配置不支持附加 profiles.include，请使用明确的迁移配置');
+ }
+ const groups = property(configuration, 'spring.profiles.group');
+ if ((record(groups) && own(groups, profile))
+ || resolved.raw('spring.profiles.group.' + profile) !== undefined) {
+ throw new Error('所选 profile 是配置组，无法唯一确定数据源；请使用具体环境或明确的迁移配置');
+ }
+}
+
+function port(value, fallbackMessage) {
+ if (!/^\d+$/.test(value) || Number(value) < 1 || Number(value) > 65535) throw new Error(fallbackMessage);
+ return Number(value);
+}
+
+function mysqlAddress(value) {
+ if (!value.startsWith('jdbc:mysql://')) throw new Error('自动配置仅支持单主机 jdbc:mysql:// URL，请使用明确的迁移配置');
+ try {
+ const url = new URL(value.slice(5));
+ const host = url.hostname.replace(/^\[|\]$/g, '');
+ if (!host || (!isIP(host) && !/^[a-zA-Z0-9.-]+$/.test(host)) || url.username || url.password || url.hash) throw new Error();
+ const parts = url.pathname.split('/');
+ if (parts.length !== 2 || !parts[1]) throw new Error();
+ const database = decodeURIComponent(parts[1]);
+ if (!database || /[/\\\u0000]/.test(database)) throw new Error();
+ return { host, port: url.port ? port(url.port, '') : 3306, database };
+ } catch {
+ throw new Error('MySQL URL 必须指定单个主机、有效端口和数据库，不能含账号或多主机地址');
+ }
+}
+
+function route(value, allowEmpty) {
+ if (value === '' && allowEmpty) return '';
+ if (!value.startsWith('/') || /[?#\\]/.test(value)
+ || value.split('/').some(part => part === '.' || part === '..')) {
+ throw new Error('HTTP context-path 或 API 前缀无效，请使用明确的迁移下载地址');
+ }
+ return value.replace(/\/+$/, '');
+}
+
+/**
+ * Read only the selected project's YAML. This deliberately supports a narrow
+ * Spring configuration subset and never logs credentials or parser source text.
+ */
+export async function loadProjectConfig({ projectRoot, profile, env = process.env }) {
+ if (!projectRoot || typeof projectRoot !== 'string') throw new Error('请指定后端项目根目录');
+ const resourceRoot = resolve(projectRoot, 'yudao-server/src/main/resources');
+ const baseFile = resolve(resourceRoot, 'application.yaml');
+ const base = await yamlFile(baseFile);
+ const baseResolver = resolver(base, env);
+ let selected;
+ if (profile !== undefined) selected = profile;
+ else if (own(env, 'SPRING_PROFILES_ACTIVE') && env.SPRING_PROFILES_ACTIVE !== undefined) selected = env.SPRING_PROFILES_ACTIVE;
+ else selected = baseResolver.raw('spring.profiles.active');
+ if (typeof selected === 'string') selected = baseResolver.expand(selected);
+ const selectedProfile = singleProfile(selected);
+ assertSelectionSupported(base, selectedProfile, env);
+ const selectedFile = resolve(resourceRoot, 'application-' + selectedProfile + '.yaml');
+ const selectedConfiguration = await yamlFile(selectedFile);
+ if (property(selectedConfiguration, 'spring.profiles.active') !== undefined) {
+ throw new Error('所选环境文件再次声明 active profile，自动选择存在歧义；请使用明确的迁移配置');
+ }
+ const configuration = merge(base, selectedConfiguration);
+ assertSelectionSupported(configuration, selectedProfile, env);
+ const selectedResolver = resolver(configuration, env);
+ const primary = selectedResolver.field('spring.datasource.dynamic.primary', 'master');
+ if (!/^[a-zA-Z0-9_-]+$/.test(primary)) throw new Error('主数据源名称无效，请检查 dynamic.primary');
+ const datasource = 'spring.datasource.dynamic.datasource.' + primary + '.';
+ const address = mysqlAddress(selectedResolver.field(datasource + 'url'));
+ const user = selectedResolver.field(datasource + 'username');
+ const password = selectedResolver.field(datasource + 'password');
+ if (!user.trim() || !password.trim()) throw new Error('主数据库用户名或密码缺失，请配置凭据后重试');
+ if (/^\s*ENC\s*\(/i.test(password)) throw new Error('数据库密码仍为 ENC 加密内容，请通过环境变量提供已解密密码');
+ const serverPort = port(selectedResolver.field('server.port', '8080'), 'HTTP 端口无效，无法推导下载地址');
+ const context = route(selectedResolver.field('server.servlet.context-path', ''), true);
+ const prefix = route(selectedResolver.field('yudao.web.admin-api.prefix', '/admin-api'), false);
+ const ssl = selectedResolver.field('server.ssl.enabled', 'false');
+ if (!['true', 'false'].includes(ssl.toLowerCase())) throw new Error('HTTP SSL 配置无效，无法推导下载地址');
+ return {
+ profile: selectedProfile,
+ mysql: { ...address, user, password },
+ downloadBaseUrl: (ssl.toLowerCase() === 'true' ? 'https' : 'http') + '://127.0.0.1:' + serverPort + context + prefix + '/infra/file',
+ sourceFiles: [baseFile, selectedFile],
+ };
+}
diff --git a/script/migrate-db-files-to-oss/lib/references.mjs b/script/migrate-db-files-to-oss/lib/references.mjs
new file mode 100644
index 0000000..88637f3
--- /dev/null
+++ b/script/migrate-db-files-to-oss/lib/references.mjs
@@ -0,0 +1,207 @@
+const DEFAULT_EXCLUDED_TABLES = new Set([
+ 'infra_file', 'infra_file_content', 'infra_file_config',
+ 'infra_api_access_log', 'infra_api_error_log', 'infra_job_log',
+ 'system_operate_log', 'system_login_log',
+]);
+const MIGRATION_TABLE_PREFIX = /^(?:_+)?(?:migration(?:_|$)|migrate(?:_|$)|(?:infra_file_oss|infra_file|file|oss)_migration(?:_|$))/i;
+const URL_RUN = /[a-z][a-z0-9+.-]*:\/\/[^\s"'`<>]+/gi;
+const NEXT_URL = /,(?=[a-z][a-z0-9+.-]*:\/\/)/gi;
+const TRAILING_DELIMITERS = /[,\])}]+$/;
+
+function lookup(value, replacements) {
+ const replacement = replacements.get(value);
+ return typeof replacement === 'string' && replacement !== value
+ ? { value: replacement, changed: true, replacements: 1 }
+ : { value, changed: false, replacements: 0 };
+}
+
+function rewriteUrlPart(part, replacements) {
+ const exact = lookup(part, replacements);
+ if (exact.changed || replacements.has(part)) return exact;
+ // Closing prose/Markdown punctuation is separate from a URL. A dictionary
+ // entry that includes that punctuation always takes precedence.
+ const delimiters = part.match(TRAILING_DELIMITERS)?.[0] ?? '';
+ for (let count = 1; count <= delimiters.length; count += 1) {
+ const prefix = part.slice(0, -count);
+ const result = lookup(prefix, replacements);
+ if (result.changed) return { ...result, value: result.value + part.slice(-count) };
+ }
+ return exact;
+}
+
+function rewriteUrlRun(run, replacements) {
+ const exact = rewriteUrlPart(run, replacements);
+ if (exact.changed || replacements.has(run)) return exact;
+ const parts = run.split(NEXT_URL);
+ if (parts.length === 1) return exact;
+ let count = 0;
+ const rewritten = [];
+ for (let index = 0; index < parts.length; index += 1) {
+ const result = rewriteUrlPart(parts[index], replacements);
+ rewritten.push(result.value);
+ count += result.replacements;
+ // An unknown query/fragment can contain a literal URL. Do not treat its
+ // comma-delimited contents as independent business references.
+ if (!result.changed && !replacements.has(parts[index]) && /[?#]/.test(parts[index])) {
+ rewritten.push(...parts.slice(index + 1));
+ break;
+ }
+ }
+ return { value: rewritten.join(','), changed: count > 0, replacements: count };
+}
+
+function rewriteText(value, replacements) {
+ const exact = lookup(value, replacements);
+ if (exact.changed || replacements.has(value)) return exact;
+
+ // Validate JSON first, then edit only string-value tokens. This preserves
+ // whitespace, key spelling and untouched escapes in serialized JSON.
+ if (/^[\s]*[\[{\"]/.test(value)) {
+ let isJson = false;
+ try {
+ const parsed = JSON.parse(value);
+ isJson = parsed !== null && (typeof parsed === 'object' || typeof parsed === 'string');
+ } catch {
+ // A regular text or HTML field may also begin with a JSON-like character.
+ }
+ if (isJson) {
+ let count = 0;
+ const rewritten = value.replace(/"(?:\\[\s\S]|[^"\\])*"/g, (token, offset) => {
+ if (/^\s*:/.test(value.slice(offset + token.length))) return token;
+ const result = rewriteText(JSON.parse(token), replacements);
+ count += result.replacements;
+ return result.changed ? JSON.stringify(result.value) : token;
+ });
+ return { value: rewritten, changed: count > 0, replacements: count };
+ }
+ }
+
+ let count = 0;
+ const rewritten = value.replace(URL_RUN, (run) => {
+ const result = rewriteUrlRun(run, replacements);
+ count += result.replacements;
+ return result.value;
+ });
+ return { value: rewritten, changed: count > 0, replacements: count };
+}
+
+/**
+ * Replace complete absolute URL references, without URL decoding, prefix
+ * substitution, or query/fragment normalization. JSON values returned as
+ * objects by mysql2 are supported; object keys are never rewritten.
+ * The input is not mutated and replacement values are not processed again.
+ */
+export function rewriteReferences(value, replacements) {
+ if (!(replacements instanceof Map)) throw new TypeError('replacements must be a Map');
+ const active = new WeakSet();
+ function visit(input) {
+ if (typeof input === 'string') return rewriteText(input, replacements);
+ if (input === null || typeof input !== 'object' || input instanceof Date || ArrayBuffer.isView(input)) {
+ return { value: input, changed: false, replacements: 0 };
+ }
+ if (active.has(input)) throw new TypeError('Reference values must not contain cycles');
+ active.add(input);
+ const output = Array.isArray(input) ? [] : {};
+ let count = 0;
+ for (const [key, item] of Object.entries(input)) {
+ const result = visit(item);
+ // defineProperty also handles JSON keys such as "__proto__" safely.
+ Object.defineProperty(output, key, { value: result.value, enumerable: true, writable: true, configurable: true });
+ count += result.replacements;
+ }
+ active.delete(input);
+ return { value: count > 0 ? output : input, changed: count > 0, replacements: count };
+ }
+ return visit(value);
+}
+
+/** Detect remaining legacy URLs in text/JSON values, including nested JSON. */
+export function containsLegacyReference(value, sourceConfigIds) {
+ const markers = [...sourceConfigIds].map((id) => `/infra/file/${String(id)}/get/`);
+ if (!markers.length) return false;
+ const visited = new WeakSet();
+ function visit(input) {
+ if (typeof input === 'string') {
+ if (/^[\s]*[\[{\"]/.test(input)) {
+ try {
+ const parsed = JSON.parse(input);
+ // Parse JSON before examining its text so keys do not count as
+ // references, and \\u002f / nested JSON escapes are decoded properly.
+ return visit(parsed);
+ } catch {
+ // Rich text or malformed JSON still receives the literal text check.
+ }
+ }
+ const normalized = input.replaceAll('\\/', '/');
+ return markers.some((marker) => normalized.includes(marker));
+ }
+ if (input === null || typeof input !== 'object' || input instanceof Date || ArrayBuffer.isView(input)) return false;
+ if (visited.has(input)) return false;
+ visited.add(input);
+ return Object.values(input).some(visit);
+ }
+ return visit(value);
+}
+
+/**
+ * Discover writable text/JSON columns with one primary-key column. All schema
+ * names are bound values, never SQL identifiers. excludedTables accepts table
+ * names or schema.table names in an Array/Set (case-insensitive exclusion).
+ */
+export async function discoverReferenceColumns(connection, schemas, excludedTables = []) {
+ const schemaNames = [...new Set(schemas)];
+ if (schemaNames.some((name) => typeof name !== 'string' || !name.trim())) {
+ throw new TypeError('schemas must contain non-empty schema names');
+ }
+ if (schemaNames.length === 0) return { columns: [], skipped: [] };
+ const excluded = new Set([...excludedTables].map((name) => String(name).toLowerCase()));
+ const placeholders = schemaNames.map(() => '?').join(', ');
+ const [rows] = await connection.execute(`
+ SELECT c.TABLE_SCHEMA AS schemaName, c.TABLE_NAME AS tableName,
+ c.COLUMN_NAME AS columnName, c.DATA_TYPE AS dataType,
+ c.EXTRA AS extra, c.GENERATION_EXPRESSION AS generationExpression,
+ pk.primaryKey, pk.primaryKeyCount
+ FROM information_schema.COLUMNS c
+ JOIN information_schema.TABLES t
+ ON t.TABLE_SCHEMA = c.TABLE_SCHEMA AND t.TABLE_NAME = c.TABLE_NAME
+ LEFT JOIN (
+ SELECT TABLE_SCHEMA, TABLE_NAME, MIN(COLUMN_NAME) AS primaryKey,
+ COUNT(*) AS primaryKeyCount
+ FROM information_schema.KEY_COLUMN_USAGE
+ WHERE CONSTRAINT_NAME = 'PRIMARY' AND TABLE_SCHEMA IN (${placeholders})
+ GROUP BY TABLE_SCHEMA, TABLE_NAME
+ ) pk ON pk.TABLE_SCHEMA = c.TABLE_SCHEMA AND pk.TABLE_NAME = c.TABLE_NAME
+ WHERE c.TABLE_SCHEMA IN (${placeholders}) AND t.TABLE_TYPE = 'BASE TABLE'
+ AND c.DATA_TYPE IN ('char', 'varchar', 'tinytext', 'text', 'mediumtext', 'longtext', 'json')
+ ORDER BY c.TABLE_SCHEMA, c.TABLE_NAME, c.ORDINAL_POSITION
+ `, [...schemaNames, ...schemaNames]);
+
+ const columns = [];
+ const skipped = [];
+ const skippedTables = new Set();
+ for (const row of rows) {
+ const schema = row.schemaName;
+ const table = row.tableName;
+ const lowerTable = table.toLowerCase();
+ const fullName = `${schema}.${table}`;
+ let reason;
+ if (DEFAULT_EXCLUDED_TABLES.has(lowerTable) || MIGRATION_TABLE_PREFIX.test(table)
+ || excluded.has(lowerTable) || excluded.has(fullName.toLowerCase())) {
+ reason = 'excluded table';
+ } else if (Number(row.primaryKeyCount) !== 1) {
+ reason = Number(row.primaryKeyCount) > 1 ? 'composite primary key' : 'no primary key';
+ }
+ if (reason) {
+ if (!skippedTables.has(fullName)) {
+ skipped.push({ schema, table, reason });
+ skippedTables.add(fullName);
+ }
+ continue;
+ }
+ const extra = row.extra ?? '';
+ if (row.columnName === row.primaryKey || /\bauto_increment\b/i.test(extra)
+ || row.generationExpression || /\b(?:VIRTUAL|STORED)\s+GENERATED\b/i.test(extra)) continue;
+ columns.push({ schema, table, column: row.columnName, primaryKey: row.primaryKey, dataType: row.dataType.toLowerCase() });
+ }
+ return { columns, skipped };
+}
diff --git a/script/migrate-db-files-to-oss/lib/source-batch.mjs b/script/migrate-db-files-to-oss/lib/source-batch.mjs
new file mode 100644
index 0000000..8cd67ca
--- /dev/null
+++ b/script/migrate-db-files-to-oss/lib/source-batch.mjs
@@ -0,0 +1,87 @@
+import { id, qualified } from './core.mjs';
+
+function sourceChanged() {
+ return Object.assign(new Error('源图片在任务建立后发生变化或已删除，请检查该批文件'), {
+ code: 'MIGRATION_SOURCE_CHANGED'
+ });
+}
+
+/**
+ * Validate a bounded batch immediately after the caller begins an InnoDB
+ * REPEATABLE READ transaction, BEFORE any consistent read or explicitly created
+ * consistent snapshot. The plain metadata read must establish its snapshot
+ * only after the primary-key range has been locked.
+ *
+ * The first locking read scans from the minimum expected ID through the upper
+ * key range, including deleted rows and potential newer versions. This retains
+ * range/gap protection until commit/rollback, including against undeleting or
+ * changing another row into a newer matching version. Do not restrict this
+ * range lock to only expected IDs, active rows, or matching paths.
+ *
+ * The caller owns transaction boundaries, rollback on failure and batch size.
+ */
+export async function verifySourceBatch(db, config, items) {
+ if (!Array.isArray(items) || items.length === 0) throw new TypeError('源文件校验批次不能为空');
+ if (!config?.mysql?.database) throw new TypeError('源文件校验需要数据库名称');
+ const expected = new Map();
+ for (const item of items) {
+ const contentId = id(item?.contentId);
+ const configId = id(item?.configId);
+ if (expected.has(contentId)) throw new TypeError('源文件校验批次不能包含重复的内容编号');
+ if (typeof item.path !== 'string' || !item.path.length) throw new TypeError('源文件路径必须为非空字符串');
+ if (!Number.isSafeInteger(item.size) || item.size < 0 || typeof item.sha256 !== 'string' || !item.sha256.length) {
+ throw new TypeError('源文件校验需要有效的预期大小和 SHA-256');
+ }
+ expected.set(contentId, { ...item, contentId, configId });
+ }
+ const entries = [...expected.values()];
+ const minimumId = entries.reduce((minimum, item) => BigInt(item.contentId) < BigInt(minimum) ? item.contentId : minimum, entries[0].contentId);
+ const sourceTable = qualified(config.mysql.database, 'infra_file_content');
+ const cases = entries.map(() => 'WHEN config_id=? AND path=? THEN ?').join(' ');
+ const predicates = entries.map(() => '(config_id=? AND path=? AND id>=?)').join(' OR ');
+ const caseParams = entries.flatMap(item => [item.configId, item.path, item.contentId]);
+ const predicateParams = entries.flatMap(item => [item.configId, item.path, item.contentId]);
+
+ // A primary-key-only locking scan avoids the large-row access path triggered
+ // by locking reads that evaluate config/path predicates. Do not filter out
+ // deleted rows: their undelete could otherwise create a new active version.
+ await db.execute(
+ `SELECT id FROM ${sourceTable} WHERE id>=? ORDER BY id FOR UPDATE`,
+ [minimumId]
+ );
+
+ // CASE is evaluated by MySQL using the path column's collation. The returned
+ // expected ID is the only grouping key used in JavaScript. This consistent
+ // read establishes a fresh snapshot while the whole relevant range is held.
+ const [latestRows] = await db.execute(
+ `SELECT id,CASE ${cases} END AS expected_content_id FROM ${sourceTable}
+ WHERE deleted=0 AND id>=? AND (${predicates}) ORDER BY id DESC`,
+ [...caseParams, minimumId, ...predicateParams]
+ );
+ const latest = new Map();
+ for (const row of latestRows) {
+ const expectedId = String(row.expected_content_id);
+ if (!expected.has(expectedId)) throw sourceChanged();
+ const actualId = id(row.id);
+ if (!latest.has(expectedId) || BigInt(actualId) > BigInt(latest.get(expectedId))) latest.set(expectedId, actualId);
+ }
+ for (const contentId of expected.keys()) {
+ if (latest.get(contentId) !== contentId) throw sourceChanged();
+ }
+
+ // The range is already locked and each config/path version is validated.
+ // Digest work now touches only the selected primary keys, not a sorted scan.
+ const [contentRows] = await db.execute(
+ `SELECT id,config_id,path,OCTET_LENGTH(content) AS size,LOWER(SHA2(content,256)) AS sha256
+ FROM ${sourceTable} WHERE deleted=0 AND id IN (${entries.map(() => '?').join(',')}) FOR UPDATE`,
+ entries.map(item => item.contentId)
+ );
+ const verified = new Set();
+ for (const row of contentRows) {
+ const contentId = id(row.id);
+ const item = expected.get(contentId);
+ if (!item || verified.has(contentId) || Number(row.size) !== item.size || row.sha256 !== item.sha256) throw sourceChanged();
+ verified.add(contentId);
+ }
+ if (verified.size !== expected.size) throw sourceChanged();
+}
diff --git a/script/migrate-db-files-to-oss/lib/source-index.mjs b/script/migrate-db-files-to-oss/lib/source-index.mjs
new file mode 100644
index 0000000..1d92d25
--- /dev/null
+++ b/script/migrate-db-files-to-oss/lib/source-index.mjs
@@ -0,0 +1,107 @@
+import { id, qualified } from './core.mjs';
+
+const compareIds = (left, right) => BigInt(left) < BigInt(right) ? -1 : BigInt(left) > BigInt(right) ? 1 : 0;
+
+function boundary(value) {
+ if ((typeof value === 'number' && !Number.isSafeInteger(value)) || !/^\d+$/.test(String(value))) {
+ throw new Error('文件清单上界必须是非负整数；大整数请使用字符串');
+ }
+ return BigInt(value).toString();
+}
+
+function pathText(value) {
+ if (Buffer.isBuffer(value)) return value.toString('utf8');
+ if (typeof value !== 'string') throw new Error('文件路径不是有效的文本值');
+ return value;
+}
+
+function fileMetadata(row) {
+ return {
+ id: id(row.id), config_id: id(row.config_id), path: pathText(row.path),
+ url: row.url, type: row.type, size: row.size
+ };
+}
+
+/**
+ * Build a metadata-only index in three SELECTs. MySQL performs all path
+ * grouping/comparisons with the columns' collation; JavaScript only associates
+ * records by their latest numeric ID. No file bytes or digests are selected.
+ */
+export async function loadSourceIndex(db, config, { maxId, progress } = {}) {
+ if (!config?.mysql?.database || !Array.isArray(config.sourceConfigIds) || !config.sourceConfigIds.length) {
+ throw new Error('文件清单需要数据库名称和至少一个源配置编号');
+ }
+ const sourceIds = [...new Set(config.sourceConfigIds.map(id))].sort(compareIds);
+ const upperBound = maxId === undefined ? undefined : boundary(maxId);
+ const contentTable = qualified(config.mysql.database, 'infra_file_content');
+ const fileTable = qualified(config.mysql.database, 'infra_file');
+ const markers = sourceIds.map(() => '?').join(',');
+ // Keep the bound outside this grouping: a newer active row above the bound
+ // must suppress the whole group, not make an older duplicate current again.
+ const latest = `SELECT config_id,path,MAX(id) AS id FROM ${contentTable}
+ WHERE deleted=0 AND config_id IN (${markers}) GROUP BY config_id,path`;
+ const boundClause = upperBound === undefined ? '' : ' WHERE g.id<=?';
+ const boundParams = upperBound === undefined ? [] : [upperBound];
+
+ progress?.stage?.('读取数据库文件分组（只读取元数据，不计算图片哈希）');
+ const [latestRows] = await db.execute(
+ `SELECT c.id AS content_id,c.config_id,c.path FROM ${contentTable} c
+ JOIN (${latest}) g ON g.id=c.id${boundClause} ORDER BY c.id`,
+ [...sourceIds, ...boundParams]
+ );
+ const byContentId = new Map();
+ let observedMaximum = '0';
+ for (const row of latestRows) {
+ const contentId = id(row.content_id);
+ if (upperBound !== undefined && compareIds(contentId, upperBound) > 0) continue;
+ const group = {
+ contentId, configId: id(row.config_id), path: pathText(row.path), files: [], spellings: []
+ };
+ byContentId.set(contentId, group);
+ if (compareIds(contentId, observedMaximum) > 0) observedMaximum = contentId;
+ }
+
+ progress?.stage?.('批量匹配文件记录并检查缺失内容');
+ const [fileRows] = await db.execute(
+ `SELECT f.id,f.config_id,f.path,f.url,f.type,f.size,g.id AS content_id
+ FROM ${fileTable} f LEFT JOIN (${latest}) g
+ ON g.config_id=f.config_id AND g.path=f.path
+ WHERE f.deleted=0 AND f.config_id IN (${markers}) ORDER BY f.id`,
+ [...sourceIds, ...sourceIds]
+ );
+ const missingFiles = [];
+ for (const row of fileRows) {
+ const file = fileMetadata(row);
+ if (row.content_id === null || row.content_id === undefined) {
+ missingFiles.push(file);
+ continue;
+ }
+ // The join deliberately has no upper bound: a file whose latest content
+ // lies beyond this batch is not a missing-content file.
+ byContentId.get(id(row.content_id))?.files.push(file);
+ }
+
+ progress?.stage?.('批量收集原路径拼写，保留历史图片地址兼容性');
+ const [spellingRows] = await db.execute(
+ `SELECT DISTINCT g.id AS content_id,BINARY c.path AS path
+ FROM ${contentTable} c JOIN (${latest}) g
+ ON g.config_id=c.config_id AND g.path=c.path
+ WHERE c.deleted=0${upperBound === undefined ? '' : ' AND g.id<=?'} ORDER BY content_id`,
+ [...sourceIds, ...boundParams]
+ );
+ const spellingSets = new Map();
+ for (const row of spellingRows) {
+ const contentId = id(row.content_id);
+ if (!byContentId.has(contentId)) continue;
+ if (!spellingSets.has(contentId)) spellingSets.set(contentId, new Set());
+ spellingSets.get(contentId).add(pathText(row.path));
+ }
+ const groups = [...byContentId.values()].sort((left, right) => compareIds(left.contentId, right.contentId));
+ for (const group of groups) {
+ group.files.sort((left, right) => compareIds(left.id, right.id));
+ group.spellings = [...(spellingSets.get(group.contentId) || [])]
+ .sort((left, right) => Buffer.compare(Buffer.from(left, 'utf8'), Buffer.from(right, 'utf8')));
+ }
+ missingFiles.sort((left, right) => compareIds(left.id, right.id));
+ return { maxId: upperBound ?? observedMaximum, groups, missingFiles };
+}
diff --git a/script/migrate-db-files-to-oss/lib/transition.mjs b/script/migrate-db-files-to-oss/lib/transition.mjs
new file mode 100644
index 0000000..0cf4bd4
--- /dev/null
+++ b/script/migrate-db-files-to-oss/lib/transition.mjs
@@ -0,0 +1,88 @@
+import { performance } from 'node:perf_hooks';
+import { qualified, sha256 } from './core.mjs';
+import { applyChanges } from './journal.mjs';
+import { verifySourceBatch } from './source-batch.mjs';
+
+/**
+ * Switch a verified batch in one caller-session REPEATABLE READ transaction.
+ * Source validation must be the first operation after BEGIN: it establishes
+ * the source range lock before the transaction's consistent-read snapshot.
+ */
+export async function switchFiles(db, config, items) {
+ if (!Array.isArray(items)) throw new TypeError('文件切换批次必须为数组');
+ const timings = { sourceMs: 0, writeMs: 0, statusMs: 0, commitMs: 0 };
+ if (!items.length) return timings;
+ const changes = items.flatMap(item => {
+ if (!Array.isArray(item?.files)) throw new TypeError('文件切换项必须包含 files 数组（无文件记录时使用空数组）');
+ return item.files.map(file => ({
+ schema: config.mysql.database, table: 'infra_file', primaryKey: 'id', pkValue: file.id,
+ before: {
+ config_id: file.config_id, path: file.path, url: file.url,
+ type: file.type, size: file.size, deleted: Buffer.from([0])
+ },
+ after: {
+ config_id: config.targetConfigId, path: item.key, url: item.url,
+ type: item.type, size: item.size
+ }
+ }));
+ });
+ const keys = items.map(item => sha256(`${item.configId}\0${item.path}`));
+ if (new Set(keys).size !== keys.length) throw new TypeError('文件切换批次不能包含重复源路径');
+ if (new Set(changes.map(change => String(change.pkValue))).size !== changes.length) {
+ throw new TypeError('文件切换批次不能包含重复文件记录');
+ }
+ const measure = async (name, action) => {
+ const started = performance.now();
+ try { return await action(); }
+ finally { timings[name] += performance.now() - started; }
+ };
+
+ await db.beginTransaction();
+ let committing = false;
+ try {
+ // Do not insert metadata lookups or other consistent reads before this.
+ await measure('sourceMs', () => verifySourceBatch(db, config, items));
+ await measure('writeMs', async () => {
+ // One source path may have many infra_file rows. Bound prepared-statement
+ // size while retaining the same source locks and transaction for all rows.
+ for (let offset = 0; offset < changes.length; offset += 100) {
+ await applyChanges(db, config.mysql.database, config.runId, changes.slice(offset, offset + 100));
+ }
+ });
+ await measure('statusMs', async () => {
+ const [result] = await db.execute(
+ `UPDATE ${qualified(config.mysql.database, 'infra_file_oss_migration_items')}
+ SET status='switched',error=NULL,updated_at=CURRENT_TIMESTAMP
+ WHERE run_id=? AND status='verified' AND item_key IN (${keys.map(() => '?').join(',')})`,
+ [config.runId, ...keys]);
+ if (result.affectedRows !== items.length) {
+ throw Object.assign(new Error('部分迁移项已变化或不再处于 verified 状态，必须回滚整批事务'), {
+ code: 'MIGRATION_CAS_CONFLICT'
+ });
+ }
+ if (result.warningStatus > 0) {
+ throw Object.assign(new Error('批量迁移状态更新产生数据库警告，必须回滚整批事务'), {
+ code: 'MIGRATION_SQL_WARNING'
+ });
+ }
+ });
+ committing = true;
+ await measure('commitMs', () => db.commit());
+ return timings;
+ } catch (error) {
+ try {
+ await db.rollback();
+ } catch (rollbackError) {
+ throw Object.assign(new Error('事务回滚失败，连接状态不确定，禁止降级重试', { cause: error }), {
+ code: 'MIGRATION_ROLLBACK_FAILED', fatal: true, noFallback: true, rollbackError
+ });
+ }
+ if (committing) {
+ // A failed COMMIT acknowledgement does not prove the server rolled back.
+ throw Object.assign(new Error('事务提交结果不确定，禁止降级重试，请重新核对迁移状态', { cause: error }), {
+ code: error.code || 'MIGRATION_COMMIT_FAILED', fatal: true, noFallback: true
+ });
+ }
+ throw error;
+ }
+}
diff --git a/script/migrate-db-files-to-oss/migrate.mjs b/script/migrate-db-files-to-oss/migrate.mjs
new file mode 100644
index 0000000..8301a01
--- /dev/null
+++ b/script/migrate-db-files-to-oss/migrate.mjs
@@ -0,0 +1,582 @@
+#!/usr/bin/env node
+import { readFile, mkdir, writeFile } from 'node:fs/promises';
+import { resolve, dirname } from 'node:path';
+import { fileURLToPath } from 'node:url';
+import { isDeepStrictEqual } from 'node:util';
+import { aliases, bit, downloadUrl, encodePath, ensureObject, errorText, mimeType, normalizeConfig, objectKey, ossSettings, parseStorage, qualified, quote, readHttp, retry, sha256, verifyContent } from './lib/core.mjs';
+import { containsLegacyReference, discoverReferenceColumns, rewriteReferences } from './lib/references.mjs';
+import { ensureJournal, applyChange, rollbackChanges } from './lib/journal.mjs';
+import { loadProjectConfig } from './lib/project-config.mjs';
+import { discoverStorageConfig } from './lib/auto-storage.mjs';
+import { loadSourceIndex } from './lib/source-index.mjs';
+import { createProgress } from './lib/progress.mjs';
+import { switchFiles } from './lib/transition.mjs';
+import { mapBounded } from './lib/concurrency.mjs';
+
+const SCRIPT_DIR = dirname(fileURLToPath(import.meta.url));
+const PROJECT_ROOT = resolve(SCRIPT_DIR, '../..');
+const TRANSITION_BATCH_SIZE = 50;
+const VALIDATION_CONCURRENCY = 4;
+const seconds = milliseconds => (milliseconds / 1000).toFixed(2);
+
+const HELP = `数据库图片迁移到阿里云 OSS（Node.js 20.19+）
+ node migrate.mjs 自动读取项目配置，只读预检
+ node migrate.mjs init 自动生成本机配置，只查询存储配置及已有批次
+ node migrate.mjs apply 执行迁移；可加 --limit 50 先迁移少量
+ node migrate.mjs rollback 回滚当前批次的数据库修改
+可选：--profile dev | --config 文件路径 | --run-id 新批次名称
+默认复用项目当前环境的主数据库和默认 OSS，不需要填写数据库密码或存储编号。
+保留原 BLOB、旧配置和 OSS 对象；rollback 只撤销本批次数据库修改。`;
+
+export function parseArgs(argv) {
+ if (argv.includes('--help') || argv.includes('-h')) return { help: true };
+ const hasCommand = argv.length && !argv[0].startsWith('--');
+ const command = hasCommand ? argv[0] : 'plan';
+ const rest = hasCommand ? argv.slice(1) : argv;
+ if (!['init', 'plan', 'apply', 'rollback'].includes(command)) throw new Error('命令只能为 init、plan、apply 或 rollback');
+ const result = { command, limit: Infinity, configPath: resolve(SCRIPT_DIR, 'config.local.json') };
+ for (let i = 0; i < rest.length; i += 2) {
+ if (!['--config', '--limit', '--profile', '--run-id'].includes(rest[i]) || !rest[i + 1]) throw new Error('未知参数或缺少参数值');
+ if (rest[i] === '--config') { result.configPath = resolve(rest[i + 1]); result.explicitConfig = true; }
+ else if (rest[i] === '--profile') result.profile = rest[i + 1];
+ else if (rest[i] === '--run-id') result.runId = rest[i + 1];
+ else result.limit = Number(rest[i + 1]);
+ }
+ if (result.limit !== Infinity && (!Number.isSafeInteger(result.limit) || result.limit < 1 || command !== 'apply')) throw new Error('--limit 仅用于 apply，且必须为正整数');
+ return result;
+}
+
+export async function prepareInput(args, projectRoot = PROJECT_ROOT) {
+ let raw = {}, exists = false;
+ try { raw = JSON.parse(await readFile(args.configPath, 'utf8')); exists = true; }
+ catch (error) { if (error.code !== 'ENOENT' || args.explicitConfig) throw error; }
+ const placeholder = raw.mysql?.database === 'YOUR_DATABASE';
+ const projectManaged = !raw.mysql || placeholder || raw._autoDiscovery === true;
+ let input;
+ if (projectManaged) {
+ const project = await loadProjectConfig({ projectRoot, profile: args.profile || raw.projectProfile });
+ input = { ...raw, projectProfile: project.profile, mysql: project.mysql,
+ _projectDownloadBaseUrl: project.downloadBaseUrl, _autoDiscovery: true };
+ } else {
+ if (args.profile) throw new Error('手动 mysql 配置不能同时使用 --profile；删除 mysql 项即可复用项目配置');
+ input = { ...raw, mysql: { ...raw.mysql } };
+ }
+ if (args.runId) input.runId = args.runId;
+ if (input.targetConfigId === null) delete input.targetConfigId;
+ if (Array.isArray(input.sourceConfigIds) && input.sourceConfigIds.length === 0) delete input.sourceConfigIds;
+ const password = input.mysql.passwordEnv ? process.env[input.mysql.passwordEnv] : input.mysql.password;
+ if (password === undefined) throw new Error('数据库密码未提供；自动模式无需填写 mysql，手动模式请设置指定环境变量');
+ input.mysql.password = password;
+ return { input, exists, projectManaged, raw };
+}
+
+export async function persistGeneratedConfig(config, args, prepared) {
+ if (!prepared.projectManaged || (prepared.exists && prepared.raw._autoDiscovery !== true)) return;
+ const saved = {
+ projectProfile: config.projectProfile, runId: config.runId,
+ sourceConfigIds: config.sourceConfigIds, targetConfigId: config.targetConfigId,
+ sourceMode: config.sourceMode, oldDomains: config.oldDomains,
+ _autoDiscovery: true, _databaseIdentity: config._databaseIdentity
+ };
+ // Persist discovery/batch choices, but read credentials from the project's
+ // YAML on each invocation. No database or OSS secret is copied here.
+ const defaults = normalizeConfig({ runId: config.runId, mysql: config.mysql, sourceConfigIds: config.sourceConfigIds, targetConfigId: config.targetConfigId, sourceMode: 'database' });
+ for (const key of ['targetUrlMode', 'targetPublicDomain', 'objectPrefix', 'referenceSchemas', 'excludeReferenceTables', 'imagesOnly', 'batchSize', 'timeoutMs', 'maxFileBytes', 'retries']) {
+ if (config[key] !== undefined && JSON.stringify(config[key]) !== JSON.stringify(defaults[key])) saved[key] = config[key];
+ }
+ if (config.sourceMode === 'http' || config.targetUrlMode === 'backend') saved.downloadBaseUrl = config.downloadBaseUrl;
+ const serialized = JSON.stringify(saved, null, 2) + '\n';
+ if (prepared.exists && JSON.stringify(prepared.raw) === JSON.stringify(saved)) return;
+ // Never overwrite a pre-existing manual file; generated files may be
+ // refreshed to pin an explicitly requested new batch.
+ await writeFile(args.configPath, serialized, { mode: 0o600, flag: prepared.exists ? 'w' : 'wx' });
+}
+
+const table = (config, name) => qualified(config.mysql.database, name);
+const runs = config => table(config, 'infra_file_oss_migration_runs');
+const items = config => table(config, 'infra_file_oss_migration_items');
+const parseJson = value => typeof value === 'string' ? JSON.parse(value) : value;
+const itemKey = item => sha256(`${item.configId}\0${item.path}`);
+const samePlannedItem = (left, right) => {
+ const canonical = item => ({ ...item, aliases: [...item.aliases].sort(), files: [...item.files].sort((a, b) => BigInt(a.id) < BigInt(b.id) ? -1 : BigInt(a.id) > BigInt(b.id) ? 1 : 0) });
+ return isDeepStrictEqual(canonical(left), canonical(right));
+};
+
+export async function loadSettings(db, config) {
+ const ids = [...config.sourceConfigIds, config.targetConfigId];
+ const [rows] = await db.execute(`SELECT id, storage, master, config FROM ${table(config, 'infra_file_config')} WHERE deleted=0 AND id IN (${ids.map(() => '?').join(',')})`, ids);
+ const byId = new Map(rows.map(row => [String(row.id), row]));
+ for (const sourceId of config.sourceConfigIds) {
+ const row = byId.get(sourceId);
+ if (!row || Number(row.storage) !== 1) throw new Error(`源配置 ${sourceId} 不存在或不是数据库存储`);
+ if (bit(row.master)) throw new Error(`源配置 ${sourceId} 仍是默认上传配置，请先切换默认配置到 OSS`);
+ }
+ const targetRow = byId.get(config.targetConfigId);
+ if (!targetRow) throw new Error('目标存储配置不存在');
+ const oss = ossSettings(targetRow, config);
+ const [identity] = await db.query('SELECT @@server_uuid AS server_uuid');
+ const safe = {
+ version: 1, server: identity[0].server_uuid, schema: config.mysql.database,
+ sourceConfigIds: config.sourceConfigIds, targetConfigId: config.targetConfigId,
+ sourceDomains: config.sourceConfigIds.map(sourceId => parseStorage(byId.get(sourceId)).domain || ''),
+ sourceMode: config.sourceMode, downloadBaseUrl: config.downloadBaseUrl || '',
+ oldDomains: config.oldDomains, targetUrlMode: config.targetUrlMode,
+ endpoint: oss.endpoint, bucket: oss.bucket, publicDomain: oss.publicDomain,
+ objectPrefix: config.objectPrefix, imagesOnly: config.imagesOnly,
+ referenceSchemas: config.referenceSchemas, excludeReferenceTables: config.excludeReferenceTables
+ };
+ return { byId, oss, safe };
+}
+
+const progressOf = config => config._progress || { stage: message => console.log(message), update() {} };
+
+function sourceItem(group, config, settings) {
+ const type = mimeType(group.path, group.files);
+ if (config.imagesOnly && !type.toLowerCase().startsWith('image/')) return null;
+ encodePath(group.path);
+ const item = { contentId: String(group.contentId), configId: String(group.configId), path: group.path, type, files: group.files };
+ const aliasRows = [...group.files, ...group.spellings.map(path => ({ path }))];
+ item.aliases = aliases(item, aliasRows, parseStorage(settings.byId.get(item.configId)).domain, config);
+ return item;
+}
+
+const missingCount = (index, config) => index.missingFiles.filter(file => !config.imagesOnly || mimeType(file.path, [file]).toLowerCase().startsWith('image/')).length;
+
+export async function sourceCurrent(db, config, item, lock = false) {
+ // Keep large content out of the unindexed path/order lookup. On the actual
+ // database the combined lookup + BLOB projection took seconds per image;
+ // a metadata lookup followed by a primary-key hash read takes milliseconds.
+ // With lock=true the first SELECT retains the original range-lock semantics.
+ const [[latest]] = await db.execute(`SELECT id FROM ${table(config, 'infra_file_content')} WHERE deleted=0 AND config_id=? AND path=? ORDER BY id DESC LIMIT 1${lock ? ' FOR UPDATE' : ''}`, [item.configId, item.path]);
+ if (!latest || String(latest.id) !== String(item.contentId)) throw new Error('源图片在任务建立后发生变化或已删除，请检查该文件');
+ const [[current]] = await db.execute(`SELECT id,OCTET_LENGTH(content) AS size,LOWER(SHA2(content,256)) AS sha256 FROM ${table(config, 'infra_file_content')} WHERE id=? AND config_id=? AND path=? AND deleted=0${lock ? ' FOR UPDATE' : ''}`, [item.contentId, item.configId, item.path]);
+ if (!current || String(current.id) !== String(item.contentId) || current.sha256 !== item.sha256 || Number(current.size) !== item.size) throw new Error('源图片在任务建立后发生变化或已删除，请检查该文件');
+}
+
+export async function* inventory(db, config, settings, maxId, summary, sourceIndex) {
+ const progress = progressOf(config);
+ const index = sourceIndex || await loadSourceIndex(db, config, { maxId, progress });
+ const candidates = [];
+ for (const group of index.groups) {
+ summary.sourceGroups++;
+ const item = sourceItem(group, config, settings);
+ if (item) candidates.push(item); else summary.skippedNonImages++;
+ }
+ progress.stage(`建立迁移清单：按主键读取并校验 ${candidates.length} 个文件的内容`);
+ for (let start = 0; start < candidates.length; start += config.batchSize) {
+ const batch = candidates.slice(start, start + config.batchSize);
+ progress.update(`建立迁移清单：已校验 ${start}/${candidates.length} 个文件，正在读取下一批`);
+ const [rows] = await db.execute(`SELECT id,config_id,path,OCTET_LENGTH(content) AS size,LOWER(SHA2(content,256)) AS sha256 FROM ${table(config, 'infra_file_content')} WHERE deleted=0 AND id IN (${batch.map(() => '?').join(',')})`, batch.map(item => item.contentId));
+ const byId = new Map(rows.map(row => [String(row.id), row]));
+ for (const item of batch) {
+ const row = byId.get(item.contentId);
+ if (!row || String(row.config_id) !== item.configId || row.path !== item.path) throw new Error(`内容记录 ${item.contentId} 在建立清单期间发生变化，请检查后重试`);
+ item.size = Number(row.size);
+ item.sha256 = row.sha256;
+ if (!item.size || item.size > config.maxFileBytes) throw new Error(`内容记录 ${item.contentId} 为空或超过 maxFileBytes`);
+ item.key = objectKey(config, item);
+ item.url = config.targetUrlMode === 'backend' ? downloadUrl(config.downloadBaseUrl, config.targetConfigId, item.key) : `${settings.oss.publicDomain}/${encodePath(item.key)}`;
+ if (item.key.length > 512 || item.url.length > 1024) throw new Error('目标路径或访问地址超出 infra_file 字段长度');
+ summary.candidates++;
+ summary.bytes += item.size;
+ if (!item.files.length) summary.withoutFileRecord++;
+ yield item;
+ }
+ }
+ progress.stage(`迁移清单已建立：${summary.candidates} 个文件，内容校验完成`);
+}
+
+function addMappings(map, item) {
+ for (const oldUrl of item.aliases) {
+ if (map.has(oldUrl) && map.get(oldUrl) !== item.url) throw new Error('同一旧地址对应多个目标，必须先解决数据冲突');
+ map.set(oldUrl, item.url);
+ }
+}
+
+const emptyInventory = () => ({ sourceGroups: 0, candidates: 0, bytes: 0, skippedNonImages: 0, withoutFileRecord: 0 });
+
+async function scanReferences(db, config, map, { apply = false } = {}) {
+ const progress = progressOf(config);
+ progress.stage('检查需要扫描的业务表和图片地址字段');
+ const discovery = await discoverReferenceColumns(db, config.referenceSchemas, config.excludeReferenceTables);
+ const groups = new Map();
+ for (const column of discovery.columns) {
+ const key = JSON.stringify([column.schema, column.table]);
+ if (!groups.has(key)) groups.set(key, { ...column, columns: [] });
+ groups.get(key).columns.push(column.column);
+ }
+ const result = { scannedRows: 0, changedRows: 0, replacements: 0, unresolvedRows: 0, conflicts: 0, skippedTables: discovery.skipped.filter(entry => entry.reason !== 'excluded table'), excludedTables: discovery.skipped.filter(entry => entry.reason === 'excluded table'), tables: [] };
+ let tableNumber = 0;
+ const readBatchSize = Math.max(config.batchSize, 1000);
+ for (const group of groups.values()) {
+ tableNumber++;
+ let scanned = 0;
+ progress.stage(`引用扫描 ${tableNumber}/${groups.size}：${group.schema}.${group.table}`);
+ const stat = { schema: group.schema, table: group.table, changedRows: 0, unresolvedRows: 0 };
+ let cursor;
+ for (;;) {
+ const columns = [group.primaryKey, ...group.columns];
+ const [rows] = await db.query(`SELECT ${columns.map(quote).join(',')} FROM ${qualified(group.schema, group.table)} ${cursor === undefined ? '' : `WHERE ${quote(group.primaryKey)}>?`} ORDER BY ${quote(group.primaryKey)} LIMIT ${readBatchSize}`, cursor === undefined ? [] : [cursor]);
+ if (!rows.length) break;
+ for (const row of rows) {
+ cursor = row[group.primaryKey];
+ result.scannedRows++;
+ scanned++;
+ progress.update(`引用扫描 ${tableNumber}/${groups.size}：${group.schema}.${group.table}，已检查 ${scanned} 行，匹配 ${stat.changedRows} 行`);
+ const before = {}, after = {};
+ let unresolved = false, replacements = 0;
+ for (const column of group.columns) {
+ const rewritten = rewriteReferences(row[column], map);
+ if (rewritten.changed) { before[column] = row[column]; after[column] = rewritten.value; replacements += rewritten.replacements; }
+ if (containsLegacyReference(rewritten.value, config.sourceConfigIds)) unresolved = true;
+ }
+ if (unresolved) { result.unresolvedRows++; stat.unresolvedRows++; }
+ if (!Object.keys(after).length) continue;
+ if (apply) {
+ await db.beginTransaction();
+ try {
+ await applyChange(db, config.mysql.database, config.runId, { schema: group.schema, table: group.table, primaryKey: group.primaryKey, pkValue: cursor, before, after });
+ await db.commit();
+ } catch (error) {
+ await db.rollback();
+ result.conflicts++;
+ console.error(`引用更新未完成 ${group.schema}.${group.table}：${errorText(error)}`);
+ continue;
+ }
+ }
+ result.changedRows++; stat.changedRows++; result.replacements += replacements;
+ }
+ }
+ if (stat.changedRows || stat.unresolvedRows) result.tables.push(stat);
+ console.log(`引用扫描 ${group.schema}.${group.table}：匹配 ${stat.changedRows} 行，仍含旧地址 ${stat.unresolvedRows} 行`);
+ }
+ return result;
+}
+
+async function setItem(db, config, item, status, error = null) {
+ await db.execute(`UPDATE ${items(config)} SET status=?,error=?,updated_at=CURRENT_TIMESTAMP WHERE run_id=? AND item_key=?`, [status, error, config.runId, itemKey(item)]);
+}
+
+async function* savedItems(db, config, statuses = null) {
+ let cursor = '';
+ for (;;) {
+ const [rows] = await db.execute(`SELECT item_key,data,status FROM ${items(config)} WHERE run_id=? AND item_key>? ${statuses ? `AND status IN (${statuses.map(() => '?').join(',')})` : ''} ORDER BY item_key LIMIT ${config.batchSize}`, [config.runId, cursor, ...(statuses || [])]);
+ if (!rows.length) return;
+ for (const row of rows) { cursor = row.item_key; yield { ...parseJson(row.data), status: row.status }; }
+ }
+}
+
+async function migrateItem(db, config, oss, item) {
+ const timings = { sourceMs: 0, ossMs: 0, publicMs: 0, recordMs: 0 };
+ const measure = async (key, label, action) => {
+ progressOf(config).update(`文件 ${item.contentId}：${label}`);
+ const started = performance.now();
+ try { return await action(); } finally { timings[key] += performance.now() - started; }
+ };
+ await measure('sourceMs', '复核源文件版本', () => sourceCurrent(db, config, item));
+ let bytes;
+ if (config.sourceMode === 'http') {
+ bytes = await measure('sourceMs', '读取原图片', () => readHttp(downloadUrl(config.downloadBaseUrl, item.configId, item.path), config, item.size));
+ } else {
+ const [[row]] = await measure('sourceMs', '读取数据库原图片', () => db.execute(`SELECT content FROM ${table(config, 'infra_file_content')} WHERE id=? AND deleted=0`, [item.contentId]));
+ if (!row) throw new Error('源内容已删除');
+ bytes = row.content;
+ }
+ verifyContent(bytes, item);
+ await measure('ossMs', '上传 OSS 并读取校验', () => ensureObject(oss, item.key, bytes, item));
+ // SDK 可读不代表页面可读：按最终将写入业务表的地址匿名访问并再次校验。
+ verifyContent(await measure('publicMs', '校验最终图片访问地址', () => readHttp(item.url, config, item.size)), item);
+ await measure('sourceMs', '复核源图片未发生变化', () => sourceCurrent(db, config, item));
+ await measure('recordMs', '保存迁移进度', () => setItem(db, config, item, 'verified'));
+ return timings;
+}
+
+export async function verifyCheckpoint(oss, config, item) {
+ // A verified checkpoint needs no source download or PUT. Still check the
+ // authenticated object and its final public URL before trusting it again.
+ const head = await oss.head(item.key);
+ if (Number(head.res.headers['content-length']) !== item.size || item.size > config.maxFileBytes) {
+ throw new Error('OSS 文件长度不一致或超过限制');
+ }
+ verifyContent((await oss.get(item.key)).content, item);
+ verifyContent(await readHttp(item.url, config, item.size), item);
+}
+
+export async function verifySwitchedBatch(db, config, batch) {
+ // A resumed run must not reuse a mapping whose object/metadata was deleted
+ // or changed after the earlier successful batch. Read metadata once; only
+ // HTTP validation is concurrent, never transactions on the shared connection.
+ const ids = [...new Set(batch.flatMap(item => item.files.map(file => String(file.id))))];
+ const byId = new Map();
+ for (let start = 0; start < ids.length; start += 1000) {
+ const chunk = ids.slice(start, start + 1000);
+ const [rows] = await db.execute(`SELECT id,config_id,path,url,deleted FROM ${table(config, 'infra_file')} WHERE id IN (${chunk.map(() => '?').join(',')})`, chunk);
+ for (const row of rows) byId.set(String(row.id), row);
+ }
+ return mapBounded(batch, VALIDATION_CONCURRENCY, async item => {
+ try {
+ for (const original of item.files) {
+ const current = byId.get(String(original.id));
+ if (!current || bit(current.deleted) || String(current.config_id) !== config.targetConfigId || current.path !== item.key || current.url !== item.url) {
+ throw new Error('已迁移文件记录后来发生变化或已删除，停止复用该地址映射');
+ }
+ }
+ await retry(async () => verifyContent(await readHttp(item.url, config, item.size), item), config.retries);
+ return { item };
+ } catch (error) {
+ return { item, error };
+ }
+ });
+}
+
+async function* itemBatches(iterable, size) {
+ let batch = [];
+ for await (const item of iterable) {
+ batch.push(item);
+ if (batch.length === size) { yield batch; batch = []; }
+ }
+ if (batch.length) yield batch;
+}
+
+export async function runPlan(db, config, settings) {
+ const progress = progressOf(config);
+ progress.stage('开始只读预检：只读取文件信息，不读取图片内容或计算 SHA-256');
+ const index = await loadSourceIndex(db, config, { progress });
+ const summary = { command: 'plan', ...emptyInventory(), bytes: null, estimatedBytes: 0, sizePending: 0, contentVerified: false, missingContent: missingCount(index, config) };
+ const mappings = new Map();
+ for (const group of index.groups) {
+ summary.sourceGroups++;
+ const item = sourceItem(group, config, settings);
+ if (!item) { summary.skippedNonImages++; continue; }
+ summary.candidates++;
+ const sizes = item.files.map(file => Number(file.size)).filter(size => Number.isSafeInteger(size) && size >= 0);
+ if (sizes.length) summary.estimatedBytes += Math.max(...sizes); else summary.sizePending++;
+ if (!item.files.length) summary.withoutFileRecord++;
+ // Preview markers exist only in memory. Actual object keys require the
+ // verified content hash and are created exclusively during apply.
+ item.url = `migration-preview://${item.configId}/${item.contentId}`;
+ addMappings(mappings, item);
+ }
+ progress.stage(`已识别 ${summary.candidates} 个图片文件；开始检查业务引用（实际内容在迁移时校验）`);
+ summary.references = await scanReferences(db, config, mappings);
+ summary.status = summary.missingContent || summary.references.unresolvedRows || summary.references.skippedTables.length ? 'needs-review' : 'planned';
+ progress.stage('只读预检完成');
+ console.log('预检未读取图片二进制内容、未计算 SHA-256；容量仅为文件表估算。未验证 HTTP/OSS 连通性，也未创建迁移表、上传或修改数据。');
+ return summary;
+}
+
+export async function runApply(db, config, settings, limit = Infinity, createOss = null) {
+ const progress = progressOf(config);
+ progress.stage('读取迁移批次和断点记录');
+ // Batch source checks depend on InnoDB next-key locks and a snapshot first
+ // established AFTER their range lock, even if the server defaults to RC.
+ await db.query('SET SESSION TRANSACTION ISOLATION LEVEL REPEATABLE READ');
+ const [engines] = await db.execute(`SELECT TABLE_NAME AS name,ENGINE AS engine FROM information_schema.TABLES WHERE TABLE_SCHEMA=? AND TABLE_NAME IN ('infra_file_content','infra_file')`, [config.mysql.database]);
+ if (engines.length !== 2 || engines.some(row => String(row.engine).toLowerCase() !== 'innodb')) {
+ throw new Error('infra_file_content 和 infra_file 必须为 InnoDB，才能保证源校验与记录更新的事务保护');
+ }
+ await ensureJournal(db, config.mysql.database);
+ const [[saved]] = await db.execute(`SELECT settings,status FROM ${runs(config)} WHERE run_id=?`, [config.runId]);
+ let runSettings, sourceIndex;
+ if (saved) {
+ runSettings = parseJson(saved.settings);
+ if (JSON.stringify(runSettings.binding) !== JSON.stringify(settings.safe)) throw new Error('同一 runId 的数据库、源/目标或引用范围发生变化，请恢复原配置或使用新 runId');
+ if (saved.status.startsWith('rollback') || saved.status === 'rolled_back') throw new Error('本批次已开始回滚，不能继续迁移，请继续回滚或使用新 runId');
+ } else {
+ sourceIndex = await loadSourceIndex(db, config, { progress });
+ const missing = missingCount(sourceIndex, config);
+ if (missing) throw new Error(`发现 ${missing} 条图片记录没有数据库内容，请先执行 plan 并处理`);
+ runSettings = { binding: settings.safe, maxId: sourceIndex.maxId };
+ await db.execute(`INSERT INTO ${runs(config)} (run_id,settings,status) VALUES (?,?,'inventory')`, [config.runId, JSON.stringify(runSettings)]);
+ }
+ const summary = { command: 'apply', ...emptyInventory(), ...runSettings.inventory, uploaded: 0, revalidated: 0, switched: 0, failed: 0, limited: false };
+ if (!saved || saved.status === 'inventory') {
+ for await (const item of inventory(db, config, settings, runSettings.maxId, summary, sourceIndex)) {
+ const [[existing]] = await db.execute(`SELECT data FROM ${items(config)} WHERE run_id=? AND item_key=?`, [config.runId, itemKey(item)]);
+ if (existing) {
+ if (!samePlannedItem(parseJson(existing.data), item)) throw new Error('建立清单期间源数据发生变化，请检查后建立新批次');
+ } else {
+ await db.execute(`INSERT INTO ${items(config)} (run_id,item_key,data,status) VALUES (?,?,?,'planned')`, [config.runId, itemKey(item), JSON.stringify(item)]);
+ }
+ }
+ runSettings.inventory = Object.fromEntries(Object.keys(emptyInventory()).map(key => [key, summary[key]]));
+ await db.execute(`UPDATE ${runs(config)} SET settings=?,status='active',updated_at=CURRENT_TIMESTAMP WHERE run_id=?`, [JSON.stringify(runSettings), config.runId]);
+ }
+ // 先验证所有映射没有歧义，再发生 OSS 写入。
+ const allMappings = new Map();
+ let initialSwitched = 0, totalItems = 0;
+ progress.stage('检查迁移地址映射是否存在冲突');
+ for await (const item of savedItems(db, config)) {
+ addMappings(allMappings, item);
+ totalItems++;
+ if (item.status === 'switched') initialSwitched++;
+ }
+ const { publicDomain, ...ossOptions } = settings.oss;
+ const oss = createOss ? createOss() : new (await import('ali-oss')).default({ ...ossOptions, timeout: config.timeoutMs });
+ let processed = 0;
+ const selected = [], checkpoints = [];
+ progress.stage('开始上传并校验 OSS 文件');
+ for await (const item of savedItems(db, config, ['planned', 'failed', 'verified'])) {
+ if (processed >= limit) { summary.limited = true; break; }
+ processed++;
+ if (item.status === 'verified') { checkpoints.push(item); continue; }
+ try {
+ const started = performance.now();
+ const timings = await retry(() => migrateItem(db, config, oss, item), config.retries);
+ summary.uploaded++;
+ selected.push(item);
+ console.log(`上传校验通过：内容记录 ${item.contentId}（本次 ${processed}，总计 ${seconds(performance.now() - started)} 秒；源读取/复核 ${seconds(timings.sourceMs)} 秒，OSS 上传/校验 ${seconds(timings.ossMs)} 秒，访问校验 ${seconds(timings.publicMs)} 秒）`);
+ } catch (error) {
+ await setItem(db, config, item, 'failed', errorText(error));
+ summary.failed++;
+ console.error(`上传未完成：内容记录 ${item.contentId}，${errorText(error)}`);
+ }
+ }
+ if (checkpoints.length) {
+ let checked = 0;
+ progress.stage(`复核已上传断点：0/${checkpoints.length}，最多 ${VALIDATION_CONCURRENCY} 张同时校验`);
+ const results = await mapBounded(checkpoints, VALIDATION_CONCURRENCY, async item => {
+ try {
+ await retry(() => verifyCheckpoint(oss, config, item), config.retries);
+ return { item };
+ } catch (error) {
+ return { item, error };
+ } finally {
+ progress.update(`复核已上传断点：${++checked}/${checkpoints.length}`);
+ }
+ });
+ // All network workers have settled before using the database again.
+ for (const { item, error } of results) {
+ if (error) {
+ await setItem(db, config, item, 'failed', errorText(error));
+ summary.failed++;
+ console.error(`已上传断点校验失败：内容记录 ${item.contentId}，${errorText(error)}`);
+ } else { selected.push(item); summary.revalidated++; }
+ }
+ }
+ if (summary.failed) {
+ summary.status = 'partial';
+ const [counts] = await db.execute(`SELECT status,COUNT(*) AS count FROM ${items(config)} WHERE run_id=? GROUP BY status`, [config.runId]);
+ summary.items = Object.fromEntries(counts.map(row => [row.status, Number(row.count)]));
+ await db.execute(`UPDATE ${runs(config)} SET status='partial',updated_at=CURRENT_TIMESTAMP WHERE run_id=?`, [config.runId]);
+ return summary;
+ }
+ const switchProgress = () => `更新已校验文件记录：本次完成 ${summary.switched}/${selected.length}，累计 ${initialSwitched + summary.switched}/${totalItems}`;
+ progress.stage(switchProgress());
+ for (let start = 0; start < selected.length; start += TRANSITION_BATCH_SIZE) {
+ const batch = selected.slice(start, start + TRANSITION_BATCH_SIZE);
+ const started = performance.now();
+ try {
+ const timing = await switchFiles(db, config, batch);
+ summary.switched += batch.length;
+ console.log(`文件记录批次完成：${batch.length} 张，${seconds(performance.now() - started)} 秒（源校验 ${seconds(timing.sourceMs)} 秒，文件及回滚日志 ${seconds(timing.writeMs)} 秒，进度 ${seconds(timing.statusMs)} 秒，提交 ${seconds(timing.commitMs)} 秒）；累计 ${initialSwitched + summary.switched}/${totalItems}`);
+ } catch (error) {
+ // Isolate known data conflicts after the whole batch rolled back. Never
+ // replay a transaction with an ambiguous commit or broken connection.
+ if (error.fatal || !['MIGRATION_CAS_CONFLICT', 'MIGRATION_SOURCE_CHANGED'].includes(error.code)) throw error;
+ console.error(`该批文件存在修改冲突，事务已回滚；逐项定位：${errorText(error)}`);
+ for (const item of batch) {
+ try { await switchFiles(db, config, [item]); summary.switched++; }
+ catch (itemError) {
+ if (itemError.fatal || !['MIGRATION_CAS_CONFLICT', 'MIGRATION_SOURCE_CHANGED'].includes(itemError.code)) throw itemError;
+ summary.failed++;
+ console.error(`文件记录未切换：内容记录 ${item.contentId}，${errorText(itemError)}`);
+ }
+ }
+ }
+ progress.update(switchProgress());
+ }
+ const mappings = new Map();
+ let checkedSwitched = 0;
+ progress.stage(`复核已迁移文件的访问地址：0/${initialSwitched + summary.switched}`);
+ for await (const batch of itemBatches(savedItems(db, config, ['switched']), TRANSITION_BATCH_SIZE)) {
+ for (const { item, error } of await verifySwitchedBatch(db, config, batch)) {
+ if (error) {
+ summary.failed++;
+ console.error(`目标地址不可复用：内容记录 ${item.contentId}，${errorText(error)}`);
+ } else addMappings(mappings, item);
+ }
+ checkedSwitched += batch.length;
+ progress.update(`复核已迁移文件的访问地址：${checkedSwitched}/${initialSwitched + summary.switched}`);
+ }
+ summary.references = await scanReferences(db, config, mappings, { apply: true });
+ const [counts] = await db.execute(`SELECT status,COUNT(*) AS count FROM ${items(config)} WHERE run_id=? GROUP BY status`, [config.runId]);
+ summary.items = Object.fromEntries(counts.map(row => [row.status, Number(row.count)]));
+ const unfinished = counts.some(row => row.status !== 'switched');
+ summary.status = unfinished || summary.failed || summary.references.conflicts ? 'partial'
+ : summary.references.unresolvedRows || summary.references.skippedTables.length ? 'needs-review' : 'complete';
+ await db.execute(`UPDATE ${runs(config)} SET status=?,updated_at=CURRENT_TIMESTAMP WHERE run_id=?`, [summary.status, config.runId]);
+ return summary;
+}
+
+export async function runRollback(db, config) {
+ const [[run]] = await db.execute(`SELECT settings,status FROM ${runs(config)} WHERE run_id=?`, [config.runId]);
+ if (!run) throw new Error('找不到此 runId 的迁移日志');
+ const [[identity]] = await db.query('SELECT @@server_uuid AS server_uuid');
+ const binding = parseJson(run.settings).binding;
+ if (binding.schema !== config.mysql.database || binding.server !== identity.server_uuid) throw new Error('回滚数据库与迁移时不一致');
+ await db.execute(`UPDATE ${runs(config)} SET status='rollback',updated_at=CURRENT_TIMESTAMP WHERE run_id=?`, [config.runId]);
+ const result = await rollbackChanges(db, config.mysql.database, config.runId);
+ const status = result.conflicts.length ? 'rollback_partial' : 'rolled_back';
+ await db.execute(`UPDATE ${runs(config)} SET status=?,updated_at=CURRENT_TIMESTAMP WHERE run_id=?`, [status, config.runId]);
+ return { command: 'rollback', status, ...result };
+}
+
+export async function main(argv = process.argv.slice(2)) {
+ const args = parseArgs(argv);
+ if (args.help) { console.log(HELP); return; }
+ const prepared = await prepareInput(args);
+ const { default: mysql } = await import('mysql2/promise');
+ const { passwordEnv, ...mysqlOptions } = prepared.input.mysql;
+ console.log(`读取${prepared.input.projectProfile ? `项目 ${prepared.input.projectProfile} 环境` : '指定配置'}：MySQL ${mysqlOptions.host || 'localhost'}:${mysqlOptions.port || 3306}/${mysqlOptions.database}`);
+ const db = await mysql.createConnection({ connectTimeout: 10000, ...mysqlOptions, multipleStatements: false, supportBigNumbers: true, bigNumberStrings: true, jsonStrings: true, dateStrings: true });
+ const progress = createProgress();
+ let locked = false;
+ const lockName = `oss-file-migration:${sha256(mysqlOptions.database).slice(0, 40)}`;
+ try {
+ progress.stage('识别旧存储、默认 OSS 和迁移批次');
+ await db.query("SET SESSION sql_mode = CONCAT_WS(',', NULLIF(@@SESSION.sql_mode,''), 'STRICT_ALL_TABLES')");
+ if (['apply', 'rollback'].includes(args.command)) {
+ const [[lock]] = await db.execute('SELECT GET_LOCK(?,0) AS acquired', [lockName]);
+ if (Number(lock.acquired) !== 1) throw new Error('已有文件迁移或回滚任务运行，请等待其结束');
+ locked = true;
+ }
+ const auto = prepared.projectManaged || !prepared.input.runId || !prepared.input.sourceConfigIds?.length || !prepared.input.targetConfigId;
+ const config = normalizeConfig(auto ? await discoverStorageConfig(db, prepared.input, { command: args.command }) : prepared.input);
+ config._progress = progress;
+ await db.query(`SET SESSION max_execution_time=${config.timeoutMs}`);
+ await persistGeneratedConfig(config, args, prepared);
+ if (args.command === 'init') {
+ await loadSettings(db, config);
+ progress.close();
+ console.log(`配置已就绪：旧数据库存储 ${config.sourceConfigIds.join(',')} → 默认 OSS ${config.targetConfigId}；批次 ${config.runId}`);
+ console.log(`配置文件：${args.configPath}；数据库密码和 OSS 密钥未复制到此文件。`);
+ console.log('只读取了项目和数据库配置，未上传图片或修改数据库。');
+ return { status: 'configured', configPath: args.configPath };
+ }
+ let summary;
+ if (args.command === 'rollback') summary = await runRollback(db, config);
+ else {
+ const settings = await loadSettings(db, config);
+ console.log(`源配置 ${config.sourceConfigIds.join(',')} → OSS 配置 ${config.targetConfigId}；批次 ${config.runId}`);
+ summary = args.command === 'plan' ? await runPlan(db, config, settings) : await runApply(db, config, settings, args.limit);
+ }
+ const reportDir = resolve(dirname(args.configPath), 'reports');
+ await mkdir(reportDir, { recursive: true, mode: 0o700 });
+ const reportPath = resolve(reportDir, `${config.runId}-${args.command}-${Date.now()}.json`);
+ await writeFile(reportPath, JSON.stringify(summary, null, 2), { mode: 0o600 });
+ progress.close();
+ console.log(JSON.stringify(summary, null, 2));
+ console.log(`报告：${reportPath}`);
+ if (['partial', 'needs-review', 'rollback_partial'].includes(summary.status)) process.exitCode = 2;
+ return summary;
+ } finally {
+ progress.close();
+ try { if (locked) await db.execute('SELECT RELEASE_LOCK(?)', [lockName]); }
+ finally { await db.end(); }
+ }
+}
+
+if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
+ main().catch(error => { console.error(`执行停止：${errorText(error)}`); process.exitCode = 1; });
+}
diff --git a/script/migrate-db-files-to-oss/package-lock.json b/script/migrate-db-files-to-oss/package-lock.json
new file mode 100644
index 0000000..e299726
--- /dev/null
+++ b/script/migrate-db-files-to-oss/package-lock.json
@@ -0,0 +1,1101 @@
+{
+ "name": "friendship-db-files-to-oss",
+ "lockfileVersion": 3,
+ "requires": true,
+ "packages": {
+ "": {
+ "name": "friendship-db-files-to-oss",
+ "dependencies": {
+ "ali-oss": "6.23.0",
+ "mysql2": "3.15.3",
+ "yaml": "2.8.2"
+ },
+ "engines": {
+ "node": ">=20.19"
+ }
+ },
+ "node_modules/address": {
+ "version": "1.2.2",
+ "resolved": "https://registry.npmmirror.com/address/-/address-1.2.2.tgz",
+ "integrity": "sha512-4B/qKCfeE/ODUaAUpSwfzazo5x29WD4r3vXiWsB7I2mSDAihwEqKO+g8GELZUQSSAo5e1XTYh3ZVfLyxBc12nA==",
+ "license": "MIT",
+ "engines": {
+ "node": ">= 10.0.0"
+ }
+ },
+ "node_modules/agentkeepalive": {
+ "version": "3.5.3",
+ "resolved": "https://registry.npmmirror.com/agentkeepalive/-/agentkeepalive-3.5.3.tgz",
+ "integrity": "sha512-yqXL+k5rr8+ZRpOAntkaaRgWgE5o8ESAj5DyRmVTCSoZxXmqemb9Dd7T4i5UzwuERdLAJUy6XzR9zFVuf0kzkw==",
+ "license": "MIT",
+ "dependencies": {
+ "humanize-ms": "^1.2.1"
+ },
+ "engines": {
+ "node": ">= 4.0.0"
+ }
+ },
+ "node_modules/ali-oss": {
+ "version": "6.23.0",
+ "resolved": "https://registry.npmmirror.com/ali-oss/-/ali-oss-6.23.0.tgz",
+ "integrity": "sha512-FipRmyd16Pr/tEey/YaaQ/24Pc3HEpLM9S1DRakEuXlSLXNIJnu1oJtHM53eVYpvW3dXapSjrip3xylZUTIZVQ==",
+ "license": "MIT",
+ "dependencies": {
+ "address": "^1.2.2",
+ "agentkeepalive": "^3.4.1",
+ "bowser": "^1.6.0",
+ "copy-to": "^2.0.1",
+ "dateformat": "^2.0.0",
+ "debug": "^4.3.4",
+ "destroy": "^1.0.4",
+ "end-or-error": "^1.0.1",
+ "get-ready": "^1.0.0",
+ "humanize-ms": "^1.2.0",
+ "is-type-of": "^1.4.0",
+ "js-base64": "^2.5.2",
+ "jstoxml": "^2.0.0",
+ "lodash": "^4.17.21",
+ "merge-descriptors": "^1.0.1",
+ "mime": "^2.4.5",
+ "platform": "^1.3.1",
+ "pump": "^3.0.0",
+ "qs": "^6.4.0",
+ "sdk-base": "^2.0.1",
+ "stream-http": "2.8.2",
+ "stream-wormhole": "^1.0.4",
+ "urllib": "^2.44.0",
+ "utility": "^1.18.0",
+ "xml2js": "^0.6.2"
+ },
+ "engines": {
+ "node": ">=8"
+ }
+ },
+ "node_modules/any-promise": {
+ "version": "1.3.0",
+ "resolved": "https://registry.npmmirror.com/any-promise/-/any-promise-1.3.0.tgz",
+ "integrity": "sha512-7UvmKalWRt1wgjL1RrGxoSJW/0QZFIegpeGvZG9kjp8vrRu55XTHbwnqq2GpXm9uLbcuhxm3IqX9OB4MZR1b2A==",
+ "license": "MIT"
+ },
+ "node_modules/aws-ssl-profiles": {
+ "version": "1.1.2",
+ "resolved": "https://registry.npmmirror.com/aws-ssl-profiles/-/aws-ssl-profiles-1.1.2.tgz",
+ "integrity": "sha512-NZKeq9AfyQvEeNlN0zSYAaWrmBffJh3IELMZfRpJVWgrpEbtEpnjvzqBPf+mxoI287JohRDoa+/nsfqqiZmF6g==",
+ "license": "MIT",
+ "engines": {
+ "node": ">= 6.0.0"
+ }
+ },
+ "node_modules/bowser": {
+ "version": "1.9.4",
+ "resolved": "https://registry.npmmirror.com/bowser/-/bowser-1.9.4.tgz",
+ "integrity": "sha512-9IdMmj2KjigRq6oWhmwv1W36pDuA4STQZ8q6YO9um+x07xgYNCD3Oou+WP/3L1HNz7iqythGet3/p4wvc8AAwQ==",
+ "license": "MIT"
+ },
+ "node_modules/builtin-status-codes": {
+ "version": "3.0.0",
+ "resolved": "https://registry.npmmirror.com/builtin-status-codes/-/builtin-status-codes-3.0.0.tgz",
+ "integrity": "sha512-HpGFw18DgFWlncDfjTa2rcQ4W88O1mC8e8yZ2AvQY5KDaktSTwo+KRf6nHK6FRI5FyRyb/5T6+TSxfP7QyGsmQ==",
+ "license": "MIT"
+ },
+ "node_modules/call-bind-apply-helpers": {
+ "version": "1.0.2",
+ "resolved": "https://registry.npmmirror.com/call-bind-apply-helpers/-/call-bind-apply-helpers-1.0.2.tgz",
+ "integrity": "sha512-Sp1ablJ0ivDkSzjcaJdxEunN5/XvksFJ2sMBFfq6x0ryhQV/2b/KwFe21cMpmHtPOSij8K99/wSfoEuTObmuMQ==",
+ "license": "MIT",
+ "dependencies": {
+ "es-errors": "^1.3.0",
+ "function-bind": "^1.1.2"
+ },
+ "engines": {
+ "node": ">= 0.4"
+ }
+ },
+ "node_modules/call-bound": {
+ "version": "1.0.4",
+ "resolved": "https://registry.npmmirror.com/call-bound/-/call-bound-1.0.4.tgz",
+ "integrity": "sha512-+ys997U96po4Kx/ABpBCqhA9EuxJaQWDQg7295H4hBphv3IZg0boBKuwYpt4YXp6MZ5AmZQnU/tyMTlRpaSejg==",
+ "license": "MIT",
+ "dependencies": {
+ "call-bind-apply-helpers": "^1.0.2",
+ "get-intrinsic": "^1.3.0"
+ },
+ "engines": {
+ "node": ">= 0.4"
+ },
+ "funding": {
+ "url": "https://github.com/sponsors/ljharb"
+ }
+ },
+ "node_modules/content-type": {
+ "version": "1.0.5",
+ "resolved": "https://registry.npmmirror.com/content-type/-/content-type-1.0.5.tgz",
+ "integrity": "sha512-nTjqfcBFEipKdXCv4YDQWCfmcLZKm81ldF0pAopTvyrFGVbcR6P/VAAd5G7N+0tTr8QqiU0tFadD6FK4NtJwOA==",
+ "license": "MIT",
+ "engines": {
+ "node": ">= 0.6"
+ }
+ },
+ "node_modules/copy-to": {
+ "version": "2.0.1",
+ "resolved": "https://registry.npmmirror.com/copy-to/-/copy-to-2.0.1.tgz",
+ "integrity": "sha512-3DdaFaU/Zf1AnpLiFDeNCD4TOWe3Zl2RZaTzUvWiIk5ERzcCodOE20Vqq4fzCbNoHURFHT4/us/Lfq+S2zyY4w==",
+ "license": "MIT"
+ },
+ "node_modules/core-util-is": {
+ "version": "1.0.3",
+ "resolved": "https://registry.npmmirror.com/core-util-is/-/core-util-is-1.0.3.tgz",
+ "integrity": "sha512-ZQBvi1DcpJ4GDqanjucZ2Hj3wEO5pZDS89BWbkcrvdxksJorwUDDZamX9ldFkp9aw2lmBDLgkObEA4DWNJ9FYQ==",
+ "license": "MIT"
+ },
+ "node_modules/dateformat": {
+ "version": "2.2.0",
+ "resolved": "https://registry.npmmirror.com/dateformat/-/dateformat-2.2.0.tgz",
+ "integrity": "sha512-GODcnWq3YGoTnygPfi02ygEiRxqUxpJwuRHjdhJYuxpcZmDq4rjBiXYmbCCzStxo176ixfLT6i4NPwQooRySnw==",
+ "license": "MIT",
+ "engines": {
+ "node": "*"
+ }
+ },
+ "node_modules/debug": {
+ "version": "4.4.3",
+ "resolved": "https://registry.npmmirror.com/debug/-/debug-4.4.3.tgz",
+ "integrity": "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==",
+ "license": "MIT",
+ "dependencies": {
+ "ms": "^2.1.3"
+ },
+ "engines": {
+ "node": ">=6.0"
+ },
+ "peerDependenciesMeta": {
+ "supports-color": {
+ "optional": true
+ }
+ }
+ },
+ "node_modules/default-user-agent": {
+ "version": "1.0.0",
+ "resolved": "https://registry.npmmirror.com/default-user-agent/-/default-user-agent-1.0.0.tgz",
+ "integrity": "sha512-bDF7bg6OSNcSwFWPu4zYKpVkJZQYVrAANMYB8bc9Szem1D0yKdm4sa/rOCs2aC9+2GMqQ7KnwtZRvDhmLF0dXw==",
+ "license": "MIT",
+ "dependencies": {
+ "os-name": "~1.0.3"
+ },
+ "engines": {
+ "node": ">= 0.10.0"
+ }
+ },
+ "node_modules/denque": {
+ "version": "2.1.0",
+ "resolved": "https://registry.npmmirror.com/denque/-/denque-2.1.0.tgz",
+ "integrity": "sha512-HVQE3AAb/pxF8fQAoiqpvg9i3evqug3hoiwakOyZAwJm+6vZehbkYXZ0l4JxS+I3QxM97v5aaRNhj8v5oBhekw==",
+ "license": "Apache-2.0",
+ "engines": {
+ "node": ">=0.10"
+ }
+ },
+ "node_modules/destroy": {
+ "version": "1.2.0",
+ "resolved": "https://registry.npmmirror.com/destroy/-/destroy-1.2.0.tgz",
+ "integrity": "sha512-2sJGJTaXIIaR1w4iJSNoN0hnMY7Gpc/n8D4qSCJw8QqFWXf7cuAgnEHxBpweaVcPevC2l3KpjYCx3NypQQgaJg==",
+ "license": "MIT",
+ "engines": {
+ "node": ">= 0.8",
+ "npm": "1.2.8000 || >= 1.4.16"
+ }
+ },
+ "node_modules/digest-header": {
+ "version": "1.1.0",
+ "resolved": "https://registry.npmmirror.com/digest-header/-/digest-header-1.1.0.tgz",
+ "integrity": "sha512-glXVh42vz40yZb9Cq2oMOt70FIoWiv+vxNvdKdU8CwjLad25qHM3trLxhl9bVjdr6WaslIXhWpn0NO8T/67Qjg==",
+ "license": "MIT",
+ "engines": {
+ "node": ">= 8.0.0"
+ }
+ },
+ "node_modules/dunder-proto": {
+ "version": "1.0.1",
+ "resolved": "https://registry.npmmirror.com/dunder-proto/-/dunder-proto-1.0.1.tgz",
+ "integrity": "sha512-KIN/nDJBQRcXw0MLVhZE9iQHmG68qAVIBg9CqmUYjmQIhgij9U5MFvrqkUL5FbtyyzZuOeOt0zdeRe4UY7ct+A==",
+ "license": "MIT",
+ "dependencies": {
+ "call-bind-apply-helpers": "^1.0.1",
+ "es-errors": "^1.3.0",
+ "gopd": "^1.2.0"
+ },
+ "engines": {
+ "node": ">= 0.4"
+ }
+ },
+ "node_modules/ee-first": {
+ "version": "1.1.1",
+ "resolved": "https://registry.npmmirror.com/ee-first/-/ee-first-1.1.1.tgz",
+ "integrity": "sha512-WMwm9LhRUo+WUaRN+vRuETqG89IgZphVSNkdFgeb6sS/E4OrDIN7t48CAewSHXc6C8lefD8KKfr5vY61brQlow==",
+ "license": "MIT"
+ },
+ "node_modules/end-of-stream": {
+ "version": "1.4.5",
+ "resolved": "https://registry.npmmirror.com/end-of-stream/-/end-of-stream-1.4.5.tgz",
+ "integrity": "sha512-ooEGc6HP26xXq/N+GCGOT0JKCLDGrq2bQUZrQ7gyrJiZANJ/8YDTxTpQBXGMn+WbIQXNVpyWymm7KYVICQnyOg==",
+ "license": "MIT",
+ "dependencies": {
+ "once": "^1.4.0"
+ }
+ },
+ "node_modules/end-or-error": {
+ "version": "1.0.1",
+ "resolved": "https://registry.npmmirror.com/end-or-error/-/end-or-error-1.0.1.tgz",
+ "integrity": "sha512-OclLMSug+k2A0JKuf494im25ANRBVW8qsjmwbgX7lQ8P82H21PQ1PWkoYwb9y5yMBS69BPlwtzdIFClo3+7kOQ==",
+ "license": "MIT",
+ "engines": {
+ "node": ">= 0.11.14"
+ }
+ },
+ "node_modules/es-define-property": {
+ "version": "1.0.1",
+ "resolved": "https://registry.npmmirror.com/es-define-property/-/es-define-property-1.0.1.tgz",
+ "integrity": "sha512-e3nRfgfUZ4rNGL232gUgX06QNyyez04KdjFrF+LTRoOXmrOgFKDg4BCdsjW8EnT69eqdYGmRpJwiPVYNrCaW3g==",
+ "license": "MIT",
+ "engines": {
+ "node": ">= 0.4"
+ }
+ },
+ "node_modules/es-errors": {
+ "version": "1.3.0",
+ "resolved": "https://registry.npmmirror.com/es-errors/-/es-errors-1.3.0.tgz",
+ "integrity": "sha512-Zf5H2Kxt2xjTvbJvP2ZWLEICxA6j+hAmMzIlypy4xcBg1vKVnx89Wy0GbS+kf5cwCVFFzdCFh2XSCFNULS6csw==",
+ "license": "MIT",
+ "engines": {
+ "node": ">= 0.4"
+ }
+ },
+ "node_modules/es-object-atoms": {
+ "version": "1.1.2",
+ "resolved": "https://registry.npmmirror.com/es-object-atoms/-/es-object-atoms-1.1.2.tgz",
+ "integrity": "sha512-HWcBoN6NileqtSydK2FqHbS/LoDd2pqrnQHLyJzBj4kOp/ky2MWMN694xOfkK8/SnUsW2DH7EfyVlydKCsm1Zw==",
+ "license": "MIT",
+ "dependencies": {
+ "es-errors": "^1.3.0"
+ },
+ "engines": {
+ "node": ">= 0.4"
+ }
+ },
+ "node_modules/escape-html": {
+ "version": "1.0.3",
+ "resolved": "https://registry.npmmirror.com/escape-html/-/escape-html-1.0.3.tgz",
+ "integrity": "sha512-NiSupZ4OeuGwr68lGIeym/ksIZMJodUGOSCZ/FSnTxcrekbvqrgdUxlJOMpijaKZVjAJrWrGs/6Jy8OMuyj9ow==",
+ "license": "MIT"
+ },
+ "node_modules/extend-shallow": {
+ "version": "2.0.1",
+ "resolved": "https://registry.npmmirror.com/extend-shallow/-/extend-shallow-2.0.1.tgz",
+ "integrity": "sha512-zCnTtlxNoAiDc3gqY2aYAWFx7XWWiasuF2K8Me5WbN8otHKTUKBwjPtNpRs/rbUZm7KxWAaNj7P1a/p52GbVug==",
+ "license": "MIT",
+ "dependencies": {
+ "is-extendable": "^0.1.0"
+ },
+ "engines": {
+ "node": ">=0.10.0"
+ }
+ },
+ "node_modules/formstream": {
+ "version": "1.5.2",
+ "resolved": "https://registry.npmmirror.com/formstream/-/formstream-1.5.2.tgz",
+ "integrity": "sha512-NASf0lgxC1AyKNXQIrXTEYkiX99LhCEXTkiGObXAkpBui86a4u8FjH1o2bGb3PpqI3kafC+yw4zWeK6l6VHTgg==",
+ "license": "MIT",
+ "dependencies": {
+ "destroy": "^1.0.4",
+ "mime": "^2.5.2",
+ "node-hex": "^1.0.1",
+ "pause-stream": "~0.0.11"
+ }
+ },
+ "node_modules/function-bind": {
+ "version": "1.1.2",
+ "resolved": "https://registry.npmmirror.com/function-bind/-/function-bind-1.1.2.tgz",
+ "integrity": "sha512-7XHNxH7qX9xG5mIwxkhumTox/MIRNcOgDrxWsMt2pAr23WHp6MrRlN7FBSFpCpr+oVO0F744iUgR82nJMfG2SA==",
+ "license": "MIT",
+ "funding": {
+ "url": "https://github.com/sponsors/ljharb"
+ }
+ },
+ "node_modules/generate-function": {
+ "version": "2.3.1",
+ "resolved": "https://registry.npmmirror.com/generate-function/-/generate-function-2.3.1.tgz",
+ "integrity": "sha512-eeB5GfMNeevm/GRYq20ShmsaGcmI81kIX2K9XQx5miC8KdHaC6Jm0qQ8ZNeGOi7wYB8OsdxKs+Y2oVuTFuVwKQ==",
+ "license": "MIT",
+ "dependencies": {
+ "is-property": "^1.0.2"
+ }
+ },
+ "node_modules/get-intrinsic": {
+ "version": "1.3.0",
+ "resolved": "https://registry.npmmirror.com/get-intrinsic/-/get-intrinsic-1.3.0.tgz",
+ "integrity": "sha512-9fSjSaos/fRIVIp+xSJlE6lfwhES7LNtKaCBIamHsjr2na1BiABJPo0mOjjz8GJDURarmCPGqaiVg5mfjb98CQ==",
+ "license": "MIT",
+ "dependencies": {
+ "call-bind-apply-helpers": "^1.0.2",
+ "es-define-property": "^1.0.1",
+ "es-errors": "^1.3.0",
+ "es-object-atoms": "^1.1.1",
+ "function-bind": "^1.1.2",
+ "get-proto": "^1.0.1",
+ "gopd": "^1.2.0",
+ "has-symbols": "^1.1.0",
+ "hasown": "^2.0.2",
+ "math-intrinsics": "^1.1.0"
+ },
+ "engines": {
+ "node": ">= 0.4"
+ },
+ "funding": {
+ "url": "https://github.com/sponsors/ljharb"
+ }
+ },
+ "node_modules/get-proto": {
+ "version": "1.0.1",
+ "resolved": "https://registry.npmmirror.com/get-proto/-/get-proto-1.0.1.tgz",
+ "integrity": "sha512-sTSfBjoXBp89JvIKIefqw7U2CCebsc74kiY6awiGogKtoSGbgjYE/G/+l9sF3MWFPNc9IcoOC4ODfKHfxFmp0g==",
+ "license": "MIT",
+ "dependencies": {
+ "dunder-proto": "^1.0.1",
+ "es-object-atoms": "^1.0.0"
+ },
+ "engines": {
+ "node": ">= 0.4"
+ }
+ },
+ "node_modules/get-ready": {
+ "version": "1.0.0",
+ "resolved": "https://registry.npmmirror.com/get-ready/-/get-ready-1.0.0.tgz",
+ "integrity": "sha512-mFXCZPJIlcYcth+N8267+mghfYN9h3EhsDa6JSnbA3Wrhh/XFpuowviFcsDeYZtKspQyWyJqfs4O6P8CHeTwzw==",
+ "license": "MIT"
+ },
+ "node_modules/gopd": {
+ "version": "1.2.0",
+ "resolved": "https://registry.npmmirror.com/gopd/-/gopd-1.2.0.tgz",
+ "integrity": "sha512-ZUKRh6/kUFoAiTAtTYPZJ3hw9wNxx+BIBOijnlG9PnrJsCcSjs1wyyD6vJpaYtgnzDrKYRSqf3OO6Rfa93xsRg==",
+ "license": "MIT",
+ "engines": {
+ "node": ">= 0.4"
+ },
+ "funding": {
+ "url": "https://github.com/sponsors/ljharb"
+ }
+ },
+ "node_modules/has-symbols": {
+ "version": "1.1.0",
+ "resolved": "https://registry.npmmirror.com/has-symbols/-/has-symbols-1.1.0.tgz",
+ "integrity": "sha512-1cDNdwJ2Jaohmb3sg4OmKaMBwuC48sYni5HUw2DvsC8LjGTLK9h+eb1X6RyuOHe4hT0ULCW68iomhjUoKUqlPQ==",
+ "license": "MIT",
+ "engines": {
+ "node": ">= 0.4"
+ },
+ "funding": {
+ "url": "https://github.com/sponsors/ljharb"
+ }
+ },
+ "node_modules/hasown": {
+ "version": "2.0.4",
+ "resolved": "https://registry.npmmirror.com/hasown/-/hasown-2.0.4.tgz",
+ "integrity": "sha512-T2UbfbBEF32wiepXIsMlTW9+dDYC6wMh/t/vYA4tuOMKqWz/n3vr1NFSxQiyP+zk2mXsoMA/i/7qV6LKut1t1A==",
+ "license": "MIT",
+ "dependencies": {
+ "function-bind": "^1.1.2"
+ },
+ "engines": {
+ "node": ">= 0.4"
+ }
+ },
+ "node_modules/humanize-ms": {
+ "version": "1.2.1",
+ "resolved": "https://registry.npmmirror.com/humanize-ms/-/humanize-ms-1.2.1.tgz",
+ "integrity": "sha512-Fl70vYtsAFb/C06PTS9dZBo7ihau+Tu/DNCk/OyHhea07S+aeMWpFFkUaXRa8fI+ScZbEI8dfSxwY7gxZ9SAVQ==",
+ "license": "MIT",
+ "dependencies": {
+ "ms": "^2.0.0"
+ }
+ },
+ "node_modules/iconv-lite": {
+ "version": "0.7.3",
+ "resolved": "https://registry.npmmirror.com/iconv-lite/-/iconv-lite-0.7.3.tgz",
+ "integrity": "sha512-IKXpvIzjnC9XTAUbVBcMfGS0EPaIXtW6v+zr+RRp+hqULEpo0owZax6wyRwPOJbWbzjYspQwusTsfVr0ifh4uQ==",
+ "license": "MIT",
+ "dependencies": {
+ "safer-buffer": ">= 2.1.2 < 3.0.0"
+ },
+ "engines": {
+ "node": ">=0.10.0"
+ },
+ "funding": {
+ "type": "opencollective",
+ "url": "https://opencollective.com/express"
+ }
+ },
+ "node_modules/inherits": {
+ "version": "2.0.4",
+ "resolved": "https://registry.npmmirror.com/inherits/-/inherits-2.0.4.tgz",
+ "integrity": "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ==",
+ "license": "ISC"
+ },
+ "node_modules/is-class-hotfix": {
+ "version": "0.0.6",
+ "resolved": "https://registry.npmmirror.com/is-class-hotfix/-/is-class-hotfix-0.0.6.tgz",
+ "integrity": "sha512-0n+pzCC6ICtVr/WXnN2f03TK/3BfXY7me4cjCAqT8TYXEl0+JBRoqBo94JJHXcyDSLUeWbNX8Fvy5g5RJdAstQ==",
+ "license": "MIT"
+ },
+ "node_modules/is-extendable": {
+ "version": "0.1.1",
+ "resolved": "https://registry.npmmirror.com/is-extendable/-/is-extendable-0.1.1.tgz",
+ "integrity": "sha512-5BMULNob1vgFX6EjQw5izWDxrecWK9AM72rugNr0TFldMOi0fj6Jk+zeKIt0xGj4cEfQIJth4w3OKWOJ4f+AFw==",
+ "license": "MIT",
+ "engines": {
+ "node": ">=0.10.0"
+ }
+ },
+ "node_modules/is-property": {
+ "version": "1.0.2",
+ "resolved": "https://registry.npmmirror.com/is-property/-/is-property-1.0.2.tgz",
+ "integrity": "sha512-Ks/IoX00TtClbGQr4TWXemAnktAQvYB7HzcCxDGqEZU6oCmb2INHuOoKxbtR+HFkmYWBKv/dOZtGRiAjDhj92g==",
+ "license": "MIT"
+ },
+ "node_modules/is-type-of": {
+ "version": "1.4.0",
+ "resolved": "https://registry.npmmirror.com/is-type-of/-/is-type-of-1.4.0.tgz",
+ "integrity": "sha512-EddYllaovi5ysMLMEN7yzHEKh8A850cZ7pykrY1aNRQGn/CDjRDE9qEWbIdt7xGEVJmjBXzU/fNnC4ABTm8tEQ==",
+ "license": "MIT",
+ "dependencies": {
+ "core-util-is": "^1.0.2",
+ "is-class-hotfix": "~0.0.6",
+ "isstream": "~0.1.2"
+ }
+ },
+ "node_modules/isarray": {
+ "version": "1.0.0",
+ "resolved": "https://registry.npmmirror.com/isarray/-/isarray-1.0.0.tgz",
+ "integrity": "sha512-VLghIWNM6ELQzo7zwmcg0NmTVyWKYjvIeM83yjp0wRDTmUnrM678fQbcKBo6n2CJEF0szoG//ytg+TKla89ALQ==",
+ "license": "MIT"
+ },
+ "node_modules/isstream": {
+ "version": "0.1.2",
+ "resolved": "https://registry.npmmirror.com/isstream/-/isstream-0.1.2.tgz",
+ "integrity": "sha512-Yljz7ffyPbrLpLngrMtZ7NduUgVvi6wG9RJ9IUcyCd59YQ911PBJphODUcbOVbqYfxe1wuYf/LJ8PauMRwsM/g==",
+ "license": "MIT"
+ },
+ "node_modules/js-base64": {
+ "version": "2.6.4",
+ "resolved": "https://registry.npmmirror.com/js-base64/-/js-base64-2.6.4.tgz",
+ "integrity": "sha512-pZe//GGmwJndub7ZghVHz7vjb2LgC1m8B07Au3eYqeqv9emhESByMXxaEgkUkEqJe87oBbSniGYoQNIBklc7IQ==",
+ "license": "BSD-3-Clause"
+ },
+ "node_modules/jstoxml": {
+ "version": "2.2.9",
+ "resolved": "https://registry.npmmirror.com/jstoxml/-/jstoxml-2.2.9.tgz",
+ "integrity": "sha512-OYWlK0j+roh+eyaMROlNbS5cd5R25Y+IUpdl7cNdB8HNrkgwQzIS7L9MegxOiWNBj9dQhA/yAxiMwCC5mwNoBw==",
+ "license": "MIT"
+ },
+ "node_modules/lodash": {
+ "version": "4.18.1",
+ "resolved": "https://registry.npmmirror.com/lodash/-/lodash-4.18.1.tgz",
+ "integrity": "sha512-dMInicTPVE8d1e5otfwmmjlxkZoUpiVLwyeTdUsi/Caj/gfzzblBcCE5sRHV/AsjuCmxWrte2TNGSYuCeCq+0Q==",
+ "license": "MIT"
+ },
+ "node_modules/long": {
+ "version": "5.3.2",
+ "resolved": "https://registry.npmmirror.com/long/-/long-5.3.2.tgz",
+ "integrity": "sha512-mNAgZ1GmyNhD7AuqnTG3/VQ26o760+ZYBPKjPvugO8+nLbYfX6TVpJPseBvopbdY+qpZ/lKUnmEc1LeZYS3QAA==",
+ "license": "Apache-2.0"
+ },
+ "node_modules/lru.min": {
+ "version": "1.1.5",
+ "resolved": "https://registry.npmmirror.com/lru.min/-/lru.min-1.1.5.tgz",
+ "integrity": "sha512-5J9ysMYUpYIg9RF2vJpy9SinEmSviFSe0GyPpCQ4L5QSkLAgeLXlTAOu2ZwWUU5m+0SBl6gUU1R1ZQB3aKypfA==",
+ "license": "MIT",
+ "engines": {
+ "bun": ">=1.0.0",
+ "deno": ">=1.30.0",
+ "node": ">=8.0.0"
+ },
+ "funding": {
+ "type": "github",
+ "url": "https://github.com/sponsors/wellwelwel"
+ }
+ },
+ "node_modules/math-intrinsics": {
+ "version": "1.1.0",
+ "resolved": "https://registry.npmmirror.com/math-intrinsics/-/math-intrinsics-1.1.0.tgz",
+ "integrity": "sha512-/IXtbwEk5HTPyEwyKX6hGkYXxM9nbj64B+ilVJnC/R6B0pH5G4V3b0pVbL7DBj4tkhBAppbQUlf6F6Xl9LHu1g==",
+ "license": "MIT",
+ "engines": {
+ "node": ">= 0.4"
+ }
+ },
+ "node_modules/merge-descriptors": {
+ "version": "1.0.3",
+ "resolved": "https://registry.npmmirror.com/merge-descriptors/-/merge-descriptors-1.0.3.tgz",
+ "integrity": "sha512-gaNvAS7TZ897/rVaZ0nMtAyxNyi/pdbjbAwUpFQpN70GqnVfOiXpeUUMKRBmzXaSQ8DdTX4/0ms62r2K+hE6mQ==",
+ "license": "MIT",
+ "funding": {
+ "url": "https://github.com/sponsors/sindresorhus"
+ }
+ },
+ "node_modules/mime": {
+ "version": "2.6.0",
+ "resolved": "https://registry.npmmirror.com/mime/-/mime-2.6.0.tgz",
+ "integrity": "sha512-USPkMeET31rOMiarsBNIHZKLGgvKc/LrjofAnBlOttf5ajRvqiRA8QsenbcooctK6d6Ts6aqZXBA+XbkKthiQg==",
+ "license": "MIT",
+ "bin": {
+ "mime": "cli.js"
+ },
+ "engines": {
+ "node": ">=4.0.0"
+ }
+ },
+ "node_modules/minimist": {
+ "version": "1.2.8",
+ "resolved": "https://registry.npmmirror.com/minimist/-/minimist-1.2.8.tgz",
+ "integrity": "sha512-2yyAR8qBkN3YuheJanUpWC5U3bb5osDywNB8RzDVlDwDHbocAJveqqj1u8+SVD7jkWT4yvsHCpWqqWqAxb0zCA==",
+ "license": "MIT",
+ "funding": {
+ "url": "https://github.com/sponsors/ljharb"
+ }
+ },
+ "node_modules/mkdirp": {
+ "version": "0.5.6",
+ "resolved": "https://registry.npmmirror.com/mkdirp/-/mkdirp-0.5.6.tgz",
+ "integrity": "sha512-FP+p8RB8OWpF3YZBCrP5gtADmtXApB5AMLn+vdyA+PyxCjrCs00mjyUozssO33cwDeT3wNGdLxJ5M//YqtHAJw==",
+ "license": "MIT",
+ "dependencies": {
+ "minimist": "^1.2.6"
+ },
+ "bin": {
+ "mkdirp": "bin/cmd.js"
+ }
+ },
+ "node_modules/ms": {
+ "version": "2.1.3",
+ "resolved": "https://registry.npmmirror.com/ms/-/ms-2.1.3.tgz",
+ "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==",
+ "license": "MIT"
+ },
+ "node_modules/mysql2": {
+ "version": "3.15.3",
+ "resolved": "https://registry.npmmirror.com/mysql2/-/mysql2-3.15.3.tgz",
+ "integrity": "sha512-FBrGau0IXmuqg4haEZRBfHNWB5mUARw6hNwPDXXGg0XzVJ50mr/9hb267lvpVMnhZ1FON3qNd4Xfcez1rbFwSg==",
+ "license": "MIT",
+ "dependencies": {
+ "aws-ssl-profiles": "^1.1.1",
+ "denque": "^2.1.0",
+ "generate-function": "^2.3.1",
+ "iconv-lite": "^0.7.0",
+ "long": "^5.2.1",
+ "lru.min": "^1.0.0",
+ "named-placeholders": "^1.1.3",
+ "seq-queue": "^0.0.5",
+ "sqlstring": "^2.3.2"
+ },
+ "engines": {
+ "node": ">= 8.0"
+ }
+ },
+ "node_modules/mz": {
+ "version": "2.7.0",
+ "resolved": "https://registry.npmmirror.com/mz/-/mz-2.7.0.tgz",
+ "integrity": "sha512-z81GNO7nnYMEhrGh9LeymoE4+Yr0Wn5McHIZMK5cfQCl+NDX08sCZgUc9/6MHni9IWuFLm1Z3HTCXu2z9fN62Q==",
+ "license": "MIT",
+ "dependencies": {
+ "any-promise": "^1.0.0",
+ "object-assign": "^4.0.1",
+ "thenify-all": "^1.0.0"
+ }
+ },
+ "node_modules/named-placeholders": {
+ "version": "1.1.6",
+ "resolved": "https://registry.npmmirror.com/named-placeholders/-/named-placeholders-1.1.6.tgz",
+ "integrity": "sha512-Tz09sEL2EEuv5fFowm419c1+a/jSMiBjI9gHxVLrVdbUkkNUUfjsVYs9pVZu5oCon/kmRh9TfLEObFtkVxmY0w==",
+ "license": "MIT",
+ "dependencies": {
+ "lru.min": "^1.1.0"
+ },
+ "engines": {
+ "node": ">=8.0.0"
+ }
+ },
+ "node_modules/node-hex": {
+ "version": "1.0.1",
+ "resolved": "https://registry.npmmirror.com/node-hex/-/node-hex-1.0.1.tgz",
+ "integrity": "sha512-iwpZdvW6Umz12ICmu9IYPRxg0tOLGmU3Tq2tKetejCj3oZd7b2nUXwP3a7QA5M9glWy8wlPS1G3RwM/CdsUbdQ==",
+ "license": "MIT",
+ "engines": {
+ "node": ">=8.0.0"
+ }
+ },
+ "node_modules/object-assign": {
+ "version": "4.1.1",
+ "resolved": "https://registry.npmmirror.com/object-assign/-/object-assign-4.1.1.tgz",
+ "integrity": "sha512-rJgTQnkUnH1sFw8yT6VSU3zD3sWmu6sZhIseY8VX+GRu3P6F7Fu+JNDoXfklElbLJSnc3FUQHVe4cU5hj+BcUg==",
+ "license": "MIT",
+ "engines": {
+ "node": ">=0.10.0"
+ }
+ },
+ "node_modules/object-inspect": {
+ "version": "1.13.4",
+ "resolved": "https://registry.npmmirror.com/object-inspect/-/object-inspect-1.13.4.tgz",
+ "integrity": "sha512-W67iLl4J2EXEGTbfeHCffrjDfitvLANg0UlX3wFUUSTx92KXRFegMHUVgSqE+wvhAbi4WqjGg9czysTV2Epbew==",
+ "license": "MIT",
+ "engines": {
+ "node": ">= 0.4"
+ },
+ "funding": {
+ "url": "https://github.com/sponsors/ljharb"
+ }
+ },
+ "node_modules/once": {
+ "version": "1.4.0",
+ "resolved": "https://registry.npmmirror.com/once/-/once-1.4.0.tgz",
+ "integrity": "sha512-lNaJgI+2Q5URQBkccEKHTQOPaXdUxnZZElQTZY0MFUAuaEqe1E+Nyvgdz/aIyNi6Z9MzO5dv1H8n58/GELp3+w==",
+ "license": "ISC",
+ "dependencies": {
+ "wrappy": "1"
+ }
+ },
+ "node_modules/os-name": {
+ "version": "1.0.3",
+ "resolved": "https://registry.npmmirror.com/os-name/-/os-name-1.0.3.tgz",
+ "integrity": "sha512-f5estLO2KN8vgtTRaILIgEGBoBrMnZ3JQ7W9TMZCnOIGwHe8TRGSpcagnWDo+Dfhd/z08k9Xe75hvciJJ8Qaew==",
+ "license": "MIT",
+ "dependencies": {
+ "osx-release": "^1.0.0",
+ "win-release": "^1.0.0"
+ },
+ "bin": {
+ "os-name": "cli.js"
+ },
+ "engines": {
+ "node": ">=0.10.0"
+ }
+ },
+ "node_modules/osx-release": {
+ "version": "1.1.0",
+ "resolved": "https://registry.npmmirror.com/osx-release/-/osx-release-1.1.0.tgz",
+ "integrity": "sha512-ixCMMwnVxyHFQLQnINhmIpWqXIfS2YOXchwQrk+OFzmo6nDjQ0E4KXAyyUh0T0MZgV4bUhkRrAbVqlE4yLVq4A==",
+ "license": "MIT",
+ "dependencies": {
+ "minimist": "^1.1.0"
+ },
+ "bin": {
+ "osx-release": "cli.js"
+ },
+ "engines": {
+ "node": ">=0.10.0"
+ }
+ },
+ "node_modules/pause-stream": {
+ "version": "0.0.11",
+ "resolved": "https://registry.npmmirror.com/pause-stream/-/pause-stream-0.0.11.tgz",
+ "integrity": "sha512-e3FBlXLmN/D1S+zHzanP4E/4Z60oFAa3O051qt1pxa7DEJWKAyil6upYVXCWadEnuoqa4Pkc9oUx9zsxYeRv8A==",
+ "license": [
+ "MIT",
+ "Apache2"
+ ],
+ "dependencies": {
+ "through": "~2.3"
+ }
+ },
+ "node_modules/platform": {
+ "version": "1.3.6",
+ "resolved": "https://registry.npmmirror.com/platform/-/platform-1.3.6.tgz",
+ "integrity": "sha512-fnWVljUchTro6RiCFvCXBbNhJc2NijN7oIQxbwsyL0buWJPG85v81ehlHI9fXrJsMNgTofEoWIQeClKpgxFLrg==",
+ "license": "MIT"
+ },
+ "node_modules/process-nextick-args": {
+ "version": "2.0.1",
+ "resolved": "https://registry.npmmirror.com/process-nextick-args/-/process-nextick-args-2.0.1.tgz",
+ "integrity": "sha512-3ouUOpQhtgrbOa17J7+uxOTpITYWaGP7/AhoR3+A+/1e9skrzelGi/dXzEYyvbxubEF6Wn2ypscTKiKJFFn1ag==",
+ "license": "MIT"
+ },
+ "node_modules/pump": {
+ "version": "3.0.4",
+ "resolved": "https://registry.npmmirror.com/pump/-/pump-3.0.4.tgz",
+ "integrity": "sha512-VS7sjc6KR7e1ukRFhQSY5LM2uBWAUPiOPa/A3mkKmiMwSmRFUITt0xuj+/lesgnCv+dPIEYlkzrcyXgquIHMcA==",
+ "license": "MIT",
+ "dependencies": {
+ "end-of-stream": "^1.1.0",
+ "once": "^1.3.1"
+ }
+ },
+ "node_modules/qs": {
+ "version": "6.17.1",
+ "resolved": "https://registry.npmmirror.com/qs/-/qs-6.17.1.tgz",
+ "integrity": "sha512-jd3COkaKVgGUW+ZHyEHCG9bH7RoaaiWuMgzsIBjUMVjLAaWGWpoUAVwljVvBHxRu/2qEHnadLGZrkutYUBPQJA==",
+ "license": "BSD-3-Clause",
+ "dependencies": {
+ "es-define-property": "^1.0.1",
+ "side-channel": "^1.1.1"
+ },
+ "engines": {
+ "node": ">=0.6"
+ },
+ "funding": {
+ "url": "https://github.com/sponsors/ljharb"
+ }
+ },
+ "node_modules/readable-stream": {
+ "version": "2.3.8",
+ "resolved": "https://registry.npmmirror.com/readable-stream/-/readable-stream-2.3.8.tgz",
+ "integrity": "sha512-8p0AUk4XODgIewSi0l8Epjs+EVnWiK7NoDIEGU0HhE7+ZyY8D1IMY7odu5lRrFXGg71L15KG8QrPmum45RTtdA==",
+ "license": "MIT",
+ "dependencies": {
+ "core-util-is": "~1.0.0",
+ "inherits": "~2.0.3",
+ "isarray": "~1.0.0",
+ "process-nextick-args": "~2.0.0",
+ "safe-buffer": "~5.1.1",
+ "string_decoder": "~1.1.1",
+ "util-deprecate": "~1.0.1"
+ }
+ },
+ "node_modules/safe-buffer": {
+ "version": "5.1.2",
+ "resolved": "https://registry.npmmirror.com/safe-buffer/-/safe-buffer-5.1.2.tgz",
+ "integrity": "sha512-Gd2UZBJDkXlY7GbJxfsE8/nvKkUEU1G38c1siN6QP6a9PT9MmHB8GnpscSmMJSoF8LOIrt8ud/wPtojys4G6+g==",
+ "license": "MIT"
+ },
+ "node_modules/safer-buffer": {
+ "version": "2.1.2",
+ "resolved": "https://registry.npmmirror.com/safer-buffer/-/safer-buffer-2.1.2.tgz",
+ "integrity": "sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg==",
+ "license": "MIT"
+ },
+ "node_modules/sax": {
+ "version": "1.6.1",
+ "resolved": "https://registry.npmmirror.com/sax/-/sax-1.6.1.tgz",
+ "integrity": "sha512-42tBVwLWnaQvW5zc4HbZrTuWccECCZfBi92FDuwtqxasH+JbPB3/FOKb1m222K42R4WxuxzzMsTswfzgtSu64Q==",
+ "license": "BlueOak-1.0.0",
+ "engines": {
+ "node": ">=11.0.0"
+ }
+ },
+ "node_modules/sdk-base": {
+ "version": "2.0.1",
+ "resolved": "https://registry.npmmirror.com/sdk-base/-/sdk-base-2.0.1.tgz",
+ "integrity": "sha512-eeG26wRwhtwYuKGCDM3LixCaxY27Pa/5lK4rLKhQa7HBjJ3U3Y+f81MMZQRsDw/8SC2Dao/83yJTXJ8aULuN8Q==",
+ "license": "MIT",
+ "dependencies": {
+ "get-ready": "~1.0.0"
+ }
+ },
+ "node_modules/semver": {
+ "version": "5.7.3",
+ "resolved": "https://registry.npmmirror.com/semver/-/semver-5.7.3.tgz",
+ "integrity": "sha512-kM18BPOqTi0+TB68703/GJRpAML5Me+IDQsR1J26YB3EFKKNEyHIVZuWDX7+WKQB3tHwDwgk24BfhdpWUbytvQ==",
+ "license": "ISC",
+ "bin": {
+ "semver": "bin/semver"
+ }
+ },
+ "node_modules/seq-queue": {
+ "version": "0.0.5",
+ "resolved": "https://registry.npmmirror.com/seq-queue/-/seq-queue-0.0.5.tgz",
+ "integrity": "sha512-hr3Wtp/GZIc/6DAGPDcV4/9WoZhjrkXsi5B/07QgX8tsdc6ilr7BFM6PM6rbdAX1kFSDYeZGLipIZZKyQP0O5Q=="
+ },
+ "node_modules/side-channel": {
+ "version": "1.1.1",
+ "resolved": "https://registry.npmmirror.com/side-channel/-/side-channel-1.1.1.tgz",
+ "integrity": "sha512-6x6dK6zJdpTzF4sQeNYxwtvBzf6Eg4GtlesS94HOvTudUeyK2WXAaIfmDgsyslYrRBeFIlsi54AYsFGUuhmvrQ==",
+ "license": "MIT",
+ "dependencies": {
+ "es-errors": "^1.3.0",
+ "object-inspect": "^1.13.4",
+ "side-channel-list": "^1.0.1",
+ "side-channel-map": "^1.0.1",
+ "side-channel-weakmap": "^1.0.2"
+ },
+ "engines": {
+ "node": ">= 0.4"
+ },
+ "funding": {
+ "url": "https://github.com/sponsors/ljharb"
+ }
+ },
+ "node_modules/side-channel-list": {
+ "version": "1.0.1",
+ "resolved": "https://registry.npmmirror.com/side-channel-list/-/side-channel-list-1.0.1.tgz",
+ "integrity": "sha512-mjn/0bi/oUURjc5Xl7IaWi/OJJJumuoJFQJfDDyO46+hBWsfaVM65TBHq2eoZBhzl9EchxOijpkbRC8SVBQU0w==",
+ "license": "MIT",
+ "dependencies": {
+ "es-errors": "^1.3.0",
+ "object-inspect": "^1.13.4"
+ },
+ "engines": {
+ "node": ">= 0.4"
+ },
+ "funding": {
+ "url": "https://github.com/sponsors/ljharb"
+ }
+ },
+ "node_modules/side-channel-map": {
+ "version": "1.0.1",
+ "resolved": "https://registry.npmmirror.com/side-channel-map/-/side-channel-map-1.0.1.tgz",
+ "integrity": "sha512-VCjCNfgMsby3tTdo02nbjtM/ewra6jPHmpThenkTYh8pG9ucZ/1P8So4u4FGBek/BjpOVsDCMoLA/iuBKIFXRA==",
+ "license": "MIT",
+ "dependencies": {
+ "call-bound": "^1.0.2",
+ "es-errors": "^1.3.0",
+ "get-intrinsic": "^1.2.5",
+ "object-inspect": "^1.13.3"
+ },
+ "engines": {
+ "node": ">= 0.4"
+ },
+ "funding": {
+ "url": "https://github.com/sponsors/ljharb"
+ }
+ },
+ "node_modules/side-channel-weakmap": {
+ "version": "1.0.2",
+ "resolved": "https://registry.npmmirror.com/side-channel-weakmap/-/side-channel-weakmap-1.0.2.tgz",
+ "integrity": "sha512-WPS/HvHQTYnHisLo9McqBHOJk2FkHO/tlpvldyrnem4aeQp4hai3gythswg6p01oSoTl58rcpiFAjF2br2Ak2A==",
+ "license": "MIT",
+ "dependencies": {
+ "call-bound": "^1.0.2",
+ "es-errors": "^1.3.0",
+ "get-intrinsic": "^1.2.5",
+ "object-inspect": "^1.13.3",
+ "side-channel-map": "^1.0.1"
+ },
+ "engines": {
+ "node": ">= 0.4"
+ },
+ "funding": {
+ "url": "https://github.com/sponsors/ljharb"
+ }
+ },
+ "node_modules/sqlstring": {
+ "version": "2.3.3",
+ "resolved": "https://registry.npmmirror.com/sqlstring/-/sqlstring-2.3.3.tgz",
+ "integrity": "sha512-qC9iz2FlN7DQl3+wjwn3802RTyjCx7sDvfQEXchwa6CWOx07/WVfh91gBmQ9fahw8snwGEWU3xGzOt4tFyHLxg==",
+ "license": "MIT",
+ "engines": {
+ "node": ">= 0.6"
+ }
+ },
+ "node_modules/statuses": {
+ "version": "1.5.0",
+ "resolved": "https://registry.npmmirror.com/statuses/-/statuses-1.5.0.tgz",
+ "integrity": "sha512-OpZ3zP+jT1PI7I8nemJX4AKmAX070ZkYPVWV/AaKTJl+tXCTGyVdC1a4SL8RUQYEwk/f34ZX8UTykN68FwrqAA==",
+ "license": "MIT",
+ "engines": {
+ "node": ">= 0.6"
+ }
+ },
+ "node_modules/stream-http": {
+ "version": "2.8.2",
+ "resolved": "https://registry.npmmirror.com/stream-http/-/stream-http-2.8.2.tgz",
+ "integrity": "sha512-QllfrBhqF1DPcz46WxKTs6Mz1Bpc+8Qm6vbqOpVav5odAXwbyzwnEczoWqtxrsmlO+cJqtPrp/8gWKWjaKLLlA==",
+ "license": "MIT",
+ "dependencies": {
+ "builtin-status-codes": "^3.0.0",
+ "inherits": "^2.0.1",
+ "readable-stream": "^2.3.6",
+ "to-arraybuffer": "^1.0.0",
+ "xtend": "^4.0.0"
+ }
+ },
+ "node_modules/stream-wormhole": {
+ "version": "1.1.0",
+ "resolved": "https://registry.npmmirror.com/stream-wormhole/-/stream-wormhole-1.1.0.tgz",
+ "integrity": "sha512-gHFfL3px0Kctd6Po0M8TzEvt3De/xu6cnRrjlfYNhwbhLPLwigI2t1nc6jrzNuaYg5C4YF78PPFuQPzRiqn9ew==",
+ "license": "MIT",
+ "engines": {
+ "node": ">=4.0.0"
+ }
+ },
+ "node_modules/string_decoder": {
+ "version": "1.1.1",
+ "resolved": "https://registry.npmmirror.com/string_decoder/-/string_decoder-1.1.1.tgz",
+ "integrity": "sha512-n/ShnvDi6FHbbVfviro+WojiFzv+s8MPMHBczVePfUpDJLwoLT0ht1l4YwBCbi8pJAveEEdnkHyPyTP/mzRfwg==",
+ "license": "MIT",
+ "dependencies": {
+ "safe-buffer": "~5.1.0"
+ }
+ },
+ "node_modules/thenify": {
+ "version": "3.3.1",
+ "resolved": "https://registry.npmmirror.com/thenify/-/thenify-3.3.1.tgz",
+ "integrity": "sha512-RVZSIV5IG10Hk3enotrhvz0T9em6cyHBLkH/YAZuKqd8hRkKhSfCGIcP2KUY0EPxndzANBmNllzWPwak+bheSw==",
+ "license": "MIT",
+ "dependencies": {
+ "any-promise": "^1.0.0"
+ }
+ },
+ "node_modules/thenify-all": {
+ "version": "1.6.0",
+ "resolved": "https://registry.npmmirror.com/thenify-all/-/thenify-all-1.6.0.tgz",
+ "integrity": "sha512-RNxQH/qI8/t3thXJDwcstUO4zeqo64+Uy/+sNVRBx4Xn2OX+OZ9oP+iJnNFqplFra2ZUVeKCSa2oVWi3T4uVmA==",
+ "license": "MIT",
+ "dependencies": {
+ "thenify": ">= 3.1.0 < 4"
+ },
+ "engines": {
+ "node": ">=0.8"
+ }
+ },
+ "node_modules/through": {
+ "version": "2.3.8",
+ "resolved": "https://registry.npmmirror.com/through/-/through-2.3.8.tgz",
+ "integrity": "sha512-w89qg7PI8wAdvX60bMDP+bFoD5Dvhm9oLheFp5O4a2QF0cSBGsBX4qZmadPMvVqlLJBBci+WqGGOAPvcDeNSVg==",
+ "license": "MIT"
+ },
+ "node_modules/to-arraybuffer": {
+ "version": "1.0.1",
+ "resolved": "https://registry.npmmirror.com/to-arraybuffer/-/to-arraybuffer-1.0.1.tgz",
+ "integrity": "sha512-okFlQcoGTi4LQBG/PgSYblw9VOyptsz2KJZqc6qtgGdes8VktzUQkj4BI2blit072iS8VODNcMA+tvnS9dnuMA==",
+ "license": "MIT"
+ },
+ "node_modules/unescape": {
+ "version": "1.0.1",
+ "resolved": "https://registry.npmmirror.com/unescape/-/unescape-1.0.1.tgz",
+ "integrity": "sha512-O0+af1Gs50lyH1nUu3ZyYS1cRh01Q/kUKatTOkSs7jukXE6/NebucDVxyiDsA9AQ4JC1V1jUH9EO8JX2nMDgGQ==",
+ "license": "MIT",
+ "dependencies": {
+ "extend-shallow": "^2.0.1"
+ },
+ "engines": {
+ "node": ">=0.10.0"
+ }
+ },
+ "node_modules/urllib": {
+ "version": "2.44.1",
+ "resolved": "https://registry.npmmirror.com/urllib/-/urllib-2.44.1.tgz",
+ "integrity": "sha512-vreOVvFizoiIz5NK9IYMgUknkriHHBVccn2VFfJhgKz6O2qwm0SgjFk4OpXFRDXpdrTx8EzM1DB0/pejrqXwPA==",
+ "license": "MIT",
+ "dependencies": {
+ "any-promise": "^1.3.0",
+ "content-type": "^1.0.2",
+ "default-user-agent": "^1.0.0",
+ "digest-header": "^1.0.0",
+ "ee-first": "~1.1.1",
+ "formstream": "^1.1.0",
+ "humanize-ms": "^1.2.0",
+ "iconv-lite": "^0.6.3",
+ "pump": "^3.0.0",
+ "qs": "^6.4.0",
+ "statuses": "^1.3.1",
+ "utility": "^1.16.1"
+ },
+ "engines": {
+ "node": ">= 0.10.0"
+ },
+ "peerDependencies": {
+ "proxy-agent": "^5.0.0"
+ },
+ "peerDependenciesMeta": {
+ "proxy-agent": {
+ "optional": true
+ }
+ }
+ },
+ "node_modules/urllib/node_modules/iconv-lite": {
+ "version": "0.6.3",
+ "resolved": "https://registry.npmmirror.com/iconv-lite/-/iconv-lite-0.6.3.tgz",
+ "integrity": "sha512-4fCk79wshMdzMp2rH06qWrJE4iolqLhCUH+OiuIgU++RB0+94NlDL81atO7GX55uUKueo0txHNtvEyI6D7WdMw==",
+ "license": "MIT",
+ "dependencies": {
+ "safer-buffer": ">= 2.1.2 < 3.0.0"
+ },
+ "engines": {
+ "node": ">=0.10.0"
+ }
+ },
+ "node_modules/util-deprecate": {
+ "version": "1.0.2",
+ "resolved": "https://registry.npmmirror.com/util-deprecate/-/util-deprecate-1.0.2.tgz",
+ "integrity": "sha512-EPD5q1uXyFxJpCrLnCc1nHnq3gOa6DZBocAIiI2TaSCA7VCJ1UJDMagCzIkXNsUYfD1daK//LTEQ8xiIbrHtcw==",
+ "license": "MIT"
+ },
+ "node_modules/utility": {
+ "version": "1.18.0",
+ "resolved": "https://registry.npmmirror.com/utility/-/utility-1.18.0.tgz",
+ "integrity": "sha512-PYxZDA+6QtvRvm//++aGdmKG/cI07jNwbROz0Ql+VzFV1+Z0Dy55NI4zZ7RHc9KKpBePNFwoErqIuqQv/cjiTA==",
+ "license": "MIT",
+ "dependencies": {
+ "copy-to": "^2.0.1",
+ "escape-html": "^1.0.3",
+ "mkdirp": "^0.5.1",
+ "mz": "^2.7.0",
+ "unescape": "^1.0.1"
+ },
+ "engines": {
+ "node": ">= 0.12.0"
+ }
+ },
+ "node_modules/win-release": {
+ "version": "1.1.1",
+ "resolved": "https://registry.npmmirror.com/win-release/-/win-release-1.1.1.tgz",
+ "integrity": "sha512-iCRnKVvGxOQdsKhcQId2PXV1vV3J/sDPXKA4Oe9+Eti2nb2ESEsYHRYls/UjoUW3bIc5ZDO8dTH50A/5iVN+bw==",
+ "license": "MIT",
+ "dependencies": {
+ "semver": "^5.0.1"
+ },
+ "engines": {
+ "node": ">=0.10.0"
+ }
+ },
+ "node_modules/wrappy": {
+ "version": "1.0.2",
+ "resolved": "https://registry.npmmirror.com/wrappy/-/wrappy-1.0.2.tgz",
+ "integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==",
+ "license": "ISC"
+ },
+ "node_modules/xml2js": {
+ "version": "0.6.2",
+ "resolved": "https://registry.npmmirror.com/xml2js/-/xml2js-0.6.2.tgz",
+ "integrity": "sha512-T4rieHaC1EXcES0Kxxj4JWgaUQHDk+qwHcYOCFHfiwKz7tOVPLq7Hjq9dM1WCMhylqMEfP7hMcOIChvotiZegA==",
+ "license": "MIT",
+ "dependencies": {
+ "sax": ">=0.6.0",
+ "xmlbuilder": "~11.0.0"
+ },
+ "engines": {
+ "node": ">=4.0.0"
+ }
+ },
+ "node_modules/xmlbuilder": {
+ "version": "11.0.1",
+ "resolved": "https://registry.npmmirror.com/xmlbuilder/-/xmlbuilder-11.0.1.tgz",
+ "integrity": "sha512-fDlsI/kFEx7gLvbecc0/ohLG50fugQp8ryHzMTuW9vSa1GJ0XYWKnhsUx7oie3G98+r56aTQIUB4kht42R3JvA==",
+ "license": "MIT",
+ "engines": {
+ "node": ">=4.0"
+ }
+ },
+ "node_modules/xtend": {
+ "version": "4.0.2",
+ "resolved": "https://registry.npmmirror.com/xtend/-/xtend-4.0.2.tgz",
+ "integrity": "sha512-LKYU1iAXJXUgAXn9URjiu+MWhyUXHsvfp7mcuYm9dSUKK0/CjtrUwFAxD82/mCWbtLsGjFIad0wIsod4zrTAEQ==",
+ "license": "MIT",
+ "engines": {
+ "node": ">=0.4"
+ }
+ },
+ "node_modules/yaml": {
+ "version": "2.8.2",
+ "resolved": "https://registry.npmmirror.com/yaml/-/yaml-2.8.2.tgz",
+ "integrity": "sha512-mplynKqc1C2hTVYxd0PU2xQAc22TI1vShAYGksCCfxbn/dFwnHTNi1bvYsBTkhdUNtGIf5xNOg938rrSSYvS9A==",
+ "license": "ISC",
+ "bin": {
+ "yaml": "bin.mjs"
+ },
+ "engines": {
+ "node": ">= 14.6"
+ },
+ "funding": {
+ "url": "https://github.com/sponsors/eemeli"
+ }
+ }
+ }
+}
diff --git a/script/migrate-db-files-to-oss/package.json b/script/migrate-db-files-to-oss/package.json
new file mode 100644
index 0000000..cf5c70b
--- /dev/null
+++ b/script/migrate-db-files-to-oss/package.json
@@ -0,0 +1,20 @@
+{
+ "name": "friendship-db-files-to-oss",
+ "private": true,
+ "type": "module",
+ "engines": {
+ "node": ">=20.19"
+ },
+ "scripts": {
+ "test": "node --test test/*.test.mjs",
+ "init": "node migrate.mjs init",
+ "plan": "node migrate.mjs",
+ "apply": "node migrate.mjs apply",
+ "rollback": "node migrate.mjs rollback"
+ },
+ "dependencies": {
+ "ali-oss": "6.23.0",
+ "mysql2": "3.15.3",
+ "yaml": "2.8.2"
+ }
+}
diff --git a/script/migrate-db-files-to-oss/test/auto-storage.test.mjs b/script/migrate-db-files-to-oss/test/auto-storage.test.mjs
new file mode 100644
index 0000000..2e42e1a
--- /dev/null
+++ b/script/migrate-db-files-to-oss/test/auto-storage.test.mjs
@@ -0,0 +1,172 @@
+import test from 'node:test';
+import assert from 'node:assert/strict';
+import { discoverStorageConfig } from '../lib/auto-storage.mjs';
+
+const database = 'local_app';
+const server = 'test-server-uuid';
+const input = () => ({ mysql: { database, user: 'local-test', password: 'db-test-only' }, projectProfile: 'local', _projectDownloadBaseUrl: 'http://localhost:48080/admin-api/infra/file' });
+const source = value => ({ id: value, storage: 1, master: Buffer.from([0]), config: JSON.stringify({ domain: 'https://old.example' }) });
+const target = (value = '30', master = 1) => ({ id: value, storage: 20, master: Buffer.from([master]), config: JSON.stringify({ endpoint: 'oss-cn-hangzhou.aliyuncs.com', bucket: 'migration-test', accessKey: 'oss-test-access', accessSecret: 'oss-secret-must-not-escape', domain: 'https://media.example' }) });
+function binding(overrides = {}) {
+ return { server, schema: database, sourceConfigIds: ['5'], targetConfigId: '30', sourceMode: 'database', downloadBaseUrl: 'https://old.example/admin-api/infra/file', oldDomains: ['https://historic.example'], objectPrefix: 'migration/original', imagesOnly: true, referenceSchemas: [database, 'chat'], excludeReferenceTables: ['audit'], targetUrlMode: 'direct', publicDomain: 'https://original-media.example', ...overrides };
+}
+const run = (runId = 'original-run', status = 'partial', overrides = {}) => ({ run_id: runId, status, settings: JSON.stringify({ binding: binding(overrides), maxId: '9007199254740993' }) });
+
+class FakeDb {
+ constructor(options = {}) {
+ this.server = server;
+ this.configs = [source('5'), target()];
+ this.sourceIds = ['5'];
+ this.runs = [];
+ this.domains = ['https://old.example'];
+ this.maximum = '9007199254740993';
+ this.calls = [];
+ Object.assign(this, options);
+ }
+ async execute(sql, params = []) {
+ assert.match(sql.trim(), /^SELECT\b/i, 'discovery must only issue SELECT');
+ this.calls.push({ sql, params });
+ if (sql.includes('@@server_uuid')) return [[{ server_uuid: this.server }]];
+ if (sql.includes('information_schema.TABLES')) {
+ assert.deepEqual(params, [database, 'infra_file_oss_migration_runs']);
+ return [this.runs.length ? [{ present: 1 }] : []];
+ }
+ if (sql.includes('infra_file_oss_migration_runs')) return [this.runs];
+ if (sql.includes('infra_file_config')) return [this.configs];
+ if (sql.includes('MAX(id)')) return [[{ max_id: this.maximum }]];
+ if (sql.includes('infra_file_content')) {
+ assert.match(sql, /deleted=0/);
+ return [this.sourceIds.filter(value => params.includes(value)).map(config_id => ({ config_id }))];
+ }
+ if (sql.includes('SUBSTRING_INDEX')) return [this.domains.map(domain => ({ domain }))];
+ throw new Error(`Unexpected test SQL: ${sql}`);
+ }
+}
+
+test('discovers source BLOB configurations and the unique OSS default using SELECT only', async () => {
+ const db = new FakeDb({ configs: [source('9007199254740995'), target(), source('12'), source('2'), source('999')], sourceIds: ['12', '9007199254740995', '2', '2'] });
+ const result = await discoverStorageConfig(db, input());
+ assert.deepEqual(result.sourceConfigIds, ['2', '12', '9007199254740995']);
+ assert.equal(result.targetConfigId, '30');
+ assert.equal(result.sourceMode, 'database');
+ assert.equal(result.downloadBaseUrl, input()._projectDownloadBaseUrl);
+ assert.equal(result.projectProfile, 'local');
+ assert.equal(result._autoDiscovery, true);
+ assert.deepEqual(result._databaseIdentity, { server, schema: database });
+ assert.match(result.runId, /^db-images-[a-f0-9]{16}$/);
+ assert.equal(JSON.stringify(result).includes('oss-secret-must-not-escape'), false);
+ assert.ok(db.calls.some(call => call.sql.includes('SELECT DISTINCT config_id FROM')));
+});
+
+test('generated runId is independent of source ordering, credentials, domains and remaining file metadata', async () => {
+ const first = await discoverStorageConfig(new FakeDb({ configs: [source('7'), target(), source('5')], sourceIds: ['7', '5'] }), input());
+ const secondInput = input();
+ secondInput.mysql.password = 'rotated-password';
+ const second = await discoverStorageConfig(new FakeDb({ configs: [source('5'), source('7'), target()], sourceIds: ['5', '7'], domains: [] }), secondInput);
+ assert.equal(second.runId, first.runId);
+ assert.deepEqual(second.sourceConfigIds, first.sourceConfigIds);
+});
+
+test('only valid HTTP base domains without credentials are inferred and sorted', async () => {
+ const result = await discoverStorageConfig(new FakeDb({ domains: ['https://z.example/', 'https://a.example/prefix', 'https://z.example', 'https://user:secret@example.com', 'ftp://file.example', 'https://x.example?secret=1', 'not a URL'] }), input());
+ assert.deepEqual(result.oldDomains, ['https://a.example/prefix', 'https://z.example']);
+});
+
+test('requires a unique default and refuses non-Aliyun defaults instead of selecting a different OSS', async () => {
+ await assert.rejects(discoverStorageConfig(new FakeDb({ configs: [source('5'), target('30', 0)] }), input()), /默认.*唯一/);
+ await assert.rejects(discoverStorageConfig(new FakeDb({ configs: [source('5'), target(), target('31')] }), input()), /默认.*唯一/);
+ const invalidDefault = target();
+ invalidDefault.config = JSON.stringify({ endpoint: 'minio.example', accessSecret: 'dont-leak' });
+ await assert.rejects(discoverStorageConfig(new FakeDb({ configs: [source('5'), invalidDefault, target('31', 0)] }), input()), error => /阿里云/.test(error.message) && !error.message.includes('dont-leak'));
+ const malformedDefault = target();
+ malformedDefault.config = '{"secret":"dont-leak", broken';
+ await assert.rejects(discoverStorageConfig(new FakeDb({ configs: [source('5'), malformedDefault] }), input()), error => !error.message.includes('dont-leak'));
+});
+
+test('restores an unfinished run before current metadata can shrink its original scope', async () => {
+ const original = run('original-run', 'partial', { sourceConfigIds: ['7', '5'] });
+ const db = new FakeDb({ runs: [original], configs: [source('5'), source('7'), target()], sourceIds: [], domains: [] });
+ const result = await discoverStorageConfig(db, input(), { command: 'apply' });
+ assert.equal(result.runId, 'original-run');
+ assert.deepEqual(result.sourceConfigIds, ['7', '5'], 'historical order is part of the existing fingerprint');
+ assert.deepEqual(result.oldDomains, ['https://historic.example']);
+ assert.equal(result.objectPrefix, 'migration/original');
+ assert.equal(result.downloadBaseUrl, 'https://old.example/admin-api/infra/file');
+ assert.equal(result.targetPublicDomain, 'https://original-media.example');
+ assert.deepEqual(result.referenceSchemas, [database, 'chat']);
+ assert.ok(!db.calls.some(call => call.sql.includes('SELECT DISTINCT config_id')));
+ assert.ok(!db.calls.some(call => call.sql.includes('SUBSTRING_INDEX')));
+});
+
+test('explicit input fields override restored fields for downstream fingerprint validation', async () => {
+ const result = await discoverStorageConfig(new FakeDb({ runs: [run()] }), {
+ ...input(), oldDomains: [], objectPrefix: 'migration/explicit', imagesOnly: false,
+ targetPublicDomain: 'https://explicit.example', sourceMode: 'http', downloadBaseUrl: 'https://explicit.example/admin-api/infra/file'
+ });
+ assert.deepEqual(result.oldDomains, []);
+ assert.equal(result.objectPrefix, 'migration/explicit');
+ assert.equal(result.imagesOnly, false);
+ assert.equal(result.targetPublicDomain, 'https://explicit.example');
+ assert.equal(result.sourceMode, 'http');
+});
+
+test('automatic resumed inputs and historical runs reject a changed default target', async () => {
+ const options = { configs: [source('5'), target('30', 0), target('31')] };
+ await assert.rejects(discoverStorageConfig(new FakeDb(options), { ...input(), targetConfigId: '30', _autoDiscovery: true }), /原迁移目标不同/);
+ await assert.rejects(discoverStorageConfig(new FakeDb({ ...options, runs: [run()] }), input()), /原迁移目标不同/);
+});
+
+test('rejects cross-server/schema inputs and restored history before accessing storage credentials', async () => {
+ for (const wrong of [{ server: 'other', schema: database }, { server, schema: 'other' }]) {
+ const db = new FakeDb();
+ await assert.rejects(discoverStorageConfig(db, { ...input(), _databaseIdentity: wrong }), /身份.*不一致/);
+ assert.equal(db.calls.length, 1);
+ }
+ const db = new FakeDb({ runs: [run('foreign', 'partial', { server: 'other' })] });
+ await assert.rejects(discoverStorageConfig(db, input()), /身份.*不一致/);
+ assert.ok(!db.calls.some(call => call.sql.includes('infra_file_config')));
+});
+
+test('historical selection refuses ambiguity but honors an explicit existing or new runId', async () => {
+ const unfinished = [run('one'), run('two')];
+ await assert.rejects(discoverStorageConfig(new FakeDb({ runs: unfinished }), input()), /多个未完成.*--run-id/);
+ const chosen = await discoverStorageConfig(new FakeDb({ runs: unfinished }), { ...input(), runId: 'two' });
+ assert.equal(chosen.runId, 'two');
+ const fresh = await discoverStorageConfig(new FakeDb({ runs: unfinished }), { ...input(), runId: 'new-run' });
+ assert.equal(fresh.runId, 'new-run');
+ assert.equal(fresh.objectPrefix, 'migration/db-images');
+ await assert.rejects(discoverStorageConfig(new FakeDb({ runs: [run('one', 'complete'), run('two', 'rolled_back')] }), input()), /多个历史.*--run-id/);
+ const single = await discoverStorageConfig(new FakeDb({ runs: [run('complete-run', 'complete')] }), input());
+ assert.equal(single.runId, 'complete-run');
+ const active = await discoverStorageConfig(new FakeDb({ runs: [run('old', 'complete'), run('active', 'needs-review')] }), input());
+ assert.equal(active.runId, 'active');
+});
+
+test('apply cannot resume a rollback while rollback does not require current storage configs or secrets', async () => {
+ for (const status of ['rollback', 'rollback_partial', 'rolled_back']) {
+ await assert.rejects(discoverStorageConfig(new FakeDb({ runs: [run('old', status)] }), input(), { command: 'apply' }), /回滚.*--run-id/);
+ }
+ const db = new FakeDb({ runs: [run('old', 'rollback_partial')], configs: [] });
+ const result = await discoverStorageConfig(db, input(), { command: 'rollback' });
+ assert.equal(result.runId, 'old');
+ assert.equal(result.targetConfigId, '30');
+ assert.ok(!db.calls.some(call => call.sql.includes('infra_file_config')));
+ await assert.rejects(discoverStorageConfig(new FakeDb(), input(), { command: 'rollback' }), /未找到可回滚/);
+});
+
+test('new content beyond a non-inventory run boundary blocks apply, including bigint boundaries', async () => {
+ const options = { runs: [run()], maximum: '9007199254740994' };
+ await assert.rejects(discoverStorageConfig(new FakeDb(options), input(), { command: 'apply' }), /新增内容/);
+ const planned = await discoverStorageConfig(new FakeDb(options), input(), { command: 'plan' });
+ assert.equal(planned.runId, 'original-run');
+ const inventoryDb = new FakeDb({ ...options, runs: [run('still-building', 'inventory')] });
+ const inventory = await discoverStorageConfig(inventoryDb, input(), { command: 'apply' });
+ assert.equal(inventory.runId, 'still-building');
+ assert.ok(!inventoryDb.calls.some(call => call.sql.includes('MAX(id)')));
+});
+
+test('missing source contents produce a clear no-migration result without creating a run', async () => {
+ const db = new FakeDb({ sourceIds: [] });
+ await assert.rejects(discoverStorageConfig(db, input()), /无需迁移/);
+ assert.ok(db.calls.every(call => /^SELECT\b/i.test(call.sql.trim())));
+});
diff --git a/script/migrate-db-files-to-oss/test/concurrency.test.mjs b/script/migrate-db-files-to-oss/test/concurrency.test.mjs
new file mode 100644
index 0000000..22ff72d
--- /dev/null
+++ b/script/migrate-db-files-to-oss/test/concurrency.test.mjs
@@ -0,0 +1,146 @@
+import test from 'node:test';
+import assert from 'node:assert/strict';
+import { setImmediate } from 'node:timers/promises';
+import { mapBounded } from '../lib/concurrency.mjs';
+
+function deferred() {
+ let resolve, reject;
+ const promise = new Promise((res, rej) => { resolve = res; reject = rej; });
+ return { promise, resolve, reject };
+}
+
+test('bounded workers keep input order when tasks finish out of order', { timeout: 2000 }, async () => {
+ const gates = Array.from({ length: 5 }, deferred);
+ const started = [];
+ let active = 0, maximum = 0;
+ const pending = mapBounded(['a', 'b', 'c', 'd', 'e'], 2, async (value, index) => {
+ started.push(index);
+ maximum = Math.max(maximum, ++active);
+ try { await gates[index].promise; return value.toUpperCase(); }
+ finally { active--; }
+ });
+ assert.deepEqual(started, [0, 1]);
+ gates[1].resolve();
+ await setImmediate();
+ assert.deepEqual(started, [0, 1, 2]);
+ gates[2].resolve();
+ await setImmediate();
+ assert.deepEqual(started, [0, 1, 2, 3]);
+ gates[0].resolve();
+ await setImmediate();
+ assert.deepEqual(started, [0, 1, 2, 3, 4]);
+ gates[4].resolve();
+ gates[3].resolve();
+ assert.deepEqual(await pending, ['A', 'B', 'C', 'D', 'E']);
+ assert.equal(maximum, 2);
+ assert.equal(active, 0);
+});
+
+test('failure stops new work and waits for all running workers before rejecting', { timeout: 2000 }, async () => {
+ const gates = Array.from({ length: 5 }, deferred);
+ const started = [];
+ const finished = [];
+ const failure = new Error('first verification failed');
+ let settled = false;
+ const pending = mapBounded([0, 1, 2, 3, 4], 3, async index => {
+ started.push(index);
+ try { await gates[index].promise; return index; }
+ finally { finished.push(index); }
+ });
+ const outcome = pending.then(
+ value => { settled = true; return { value }; },
+ error => { settled = true; return { error }; },
+ );
+ assert.deepEqual(started, [0, 1, 2]);
+ gates[1].reject(failure);
+ await setImmediate();
+ assert.deepEqual(started, [0, 1, 2]);
+ assert.equal(settled, false);
+ gates[0].resolve();
+ await setImmediate();
+ assert.deepEqual(started, [0, 1, 2]);
+ assert.equal(settled, false);
+ gates[2].resolve();
+ assert.equal((await outcome).error, failure);
+ assert.equal(settled, true);
+ assert.deepEqual(finished.sort(), [0, 1, 2]);
+});
+
+test('additional in-flight rejections are drained while preserving the first failure', { timeout: 2000 }, async () => {
+ const gates = Array.from({ length: 3 }, deferred);
+ const first = new Error('first');
+ const second = new Error('second');
+ let calls = 0, settled = false;
+ const outcome = mapBounded([0, 1, 2, 3], 3, async index => {
+ calls++;
+ await gates[index].promise;
+ return index;
+ }).then(
+ () => { settled = true; assert.fail('must reject'); },
+ error => { settled = true; return error; },
+ );
+ gates[0].reject(first);
+ await setImmediate();
+ gates[1].reject(second);
+ await setImmediate();
+ assert.equal(settled, false);
+ assert.equal(calls, 3);
+ gates[2].resolve();
+ assert.equal(await outcome, first);
+});
+
+test('a synchronous failure stops scheduling immediately', async () => {
+ const failure = new Error('synchronous failure');
+ const started = [];
+ await assert.rejects(mapBounded([0, 1, 2], 3, value => {
+ started.push(value);
+ throw failure;
+ }), error => error === failure);
+ assert.deepEqual(started, [0]);
+});
+
+test('falsy rejection values still reject and never get treated as success', async () => {
+ for (const failure of [undefined, null, false, 0, '']) {
+ let rejected = false;
+ await mapBounded([1], 1, () => Promise.reject(failure)).then(
+ () => assert.fail('must reject'),
+ error => { rejected = true; assert.equal(error, failure); },
+ );
+ assert.equal(rejected, true);
+ }
+});
+
+test('limit one executes serially and accepts synchronous return values', async () => {
+ const order = [];
+ assert.deepEqual(await mapBounded([2, 4, 6], 1, (value, index) => {
+ order.push(index);
+ return value * 2;
+ }), [4, 8, 12]);
+ assert.deepEqual(order, [0, 1, 2]);
+});
+
+test('limits larger than the item count only start existing items', async () => {
+ let calls = 0;
+ const result = await mapBounded(['x', 'y'], 1000, async value => { calls++; return value; });
+ assert.deepEqual(result, ['x', 'y']);
+ assert.equal(calls, 2);
+});
+
+test('empty input returns an empty array without invoking the worker', async () => {
+ assert.deepEqual(await mapBounded([], 4, () => assert.fail('must not run')), []);
+});
+
+test('invalid limits reject even for an empty input', async () => {
+ for (const limit of [0, -1, 1.5, Infinity, NaN, '4', null, undefined, Number.MAX_SAFE_INTEGER + 1]) {
+ await assert.rejects(mapBounded([], limit, () => {}), /正整数/);
+ }
+});
+
+test('input values are snapshotted before workers mutate the original collection', async () => {
+ const input = ['first', 'second', 'third'];
+ const result = await mapBounded(input, 1, async (value, index) => {
+ if (index === 0) input.splice(1, 2, 'changed');
+ return value;
+ });
+ assert.deepEqual(result, ['first', 'second', 'third']);
+});
diff --git a/script/migrate-db-files-to-oss/test/core.test.mjs b/script/migrate-db-files-to-oss/test/core.test.mjs
new file mode 100644
index 0000000..21fa80b
--- /dev/null
+++ b/script/migrate-db-files-to-oss/test/core.test.mjs
@@ -0,0 +1,428 @@
+import test, { mock } from 'node:test';
+import assert from 'node:assert/strict';
+import { createServer } from 'node:http';
+import { once } from 'node:events';
+import {
+ aliases, baseUrl, bit, downloadUrl, encodePath, ensureObject, errorText, id,
+ mimeType, normalizeConfig, objectKey, ossSettings, parseStorage, qualified,
+ quote, readHttp, retry, sha256, verifyContent,
+} from '../lib/core.mjs';
+
+const IMAGE = Buffer.from('89504e470d0a1a0a0001020304050607', 'hex');
+const ITEM = {
+ configId: '9007199254740993', path: '头像/张 三+100%.PNG',
+ size: IMAGE.length, sha256: sha256(IMAGE), type: 'image/png',
+};
+const HTTP_OPTIONS = { timeoutMs: 1000, maxFileBytes: 1024 };
+const sourceBase = 'http://127.0.0.1:48080/admin-api/infra/file';
+const baseConfig = overrides => ({
+ runId: 'unit-test', mysql: { database: 'unit_test', user: 'test-only' },
+ sourceConfigIds: ['1'], targetConfigId: '20', downloadBaseUrl: sourceBase,
+ ...overrides,
+});
+const storageRow = overrides => ({
+ storage: 20,
+ config: {
+ endpoint: 'oss-cn-hangzhou.aliyuncs.com', bucket: 'test-only-bucket',
+ accessKey: 'test-only-access-key', accessSecret: 'test-only-access-secret',
+ ...overrides,
+ },
+});
+const noSuchKey = () => Object.assign(new Error('missing'), { code: 'NoSuchKey', status: 404 });
+const head = bytes => ({ res: { headers: { 'content-length': String(bytes.length) } } });
+const storedObject = bytes => ({
+ head: mock.fn(async () => head(bytes)),
+ get: mock.fn(async () => ({ content: bytes })),
+ put: mock.fn(async () => { throw new Error('unexpected write'); }),
+});
+
+async function localServer(t, handler) {
+ const server = createServer(handler);
+ server.listen(0, '127.0.0.1');
+ await once(server, 'listening');
+ t.after(async () => {
+ const closed = new Promise((resolve, reject) => server.close(error => error ? reject(error) : resolve()));
+ server.closeAllConnections();
+ await closed;
+ });
+ return 'http://127.0.0.1:' + server.address().port;
+}
+
+test('download URL preserves Chinese, space, plus, percent, question mark and fragment characters', () => {
+ const path = '头像/张 三+100%?#.png';
+ const encoded = '%E5%A4%B4%E5%83%8F/%E5%BC%A0%20%E4%B8%89%2B100%25%3F%23.png';
+ assert.equal(encodePath(path), encoded);
+ const url = downloadUrl(sourceBase + '///', ITEM.configId, path);
+ assert.equal(url, sourceBase + '/' + ITEM.configId + '/get/' + encoded);
+ assert.equal(decodeURIComponent(new URL(url).pathname.split('/get/')[1]), path);
+ assert.equal(new URL(url).search, '');
+ assert.equal(new URL(url).hash, '');
+});
+
+test('encoded-looking path text is encoded exactly once without decoding the source path', () => {
+ assert.equal(encodePath('%2F/%20+%.png'), '%252F/%2520%2B%25.png');
+ assert.equal(decodeURIComponent(encodePath('%2F/%20+%.png')), '%2F/%20+%.png');
+});
+
+test('ambiguous or normalized path segments are rejected', () => {
+ for (const path of ['', '/image.png', 'a//b.png', 'a/', './a.png', 'a/../b.png', 'a/./b.png', 'a\\b.png']) {
+ assert.throws(() => encodePath(path), /路径/, path);
+ }
+});
+
+test('URL base validates scheme and rejects credentials, queries and fragments', () => {
+ assert.equal(baseUrl('https://images.example.test/prefix///'), 'https://images.example.test/prefix');
+ for (const url of ['ftp://example.test', 'file:///tmp/image', 'https://user:pass@example.test', 'https://example.test/?key=value', 'https://example.test/#part']) {
+ assert.throws(() => baseUrl(url), /HTTP/);
+ }
+});
+
+test('bigint IDs remain exact strings and unsafe numeric IDs are rejected', () => {
+ assert.equal(id('9007199254740993'), '9007199254740993');
+ assert.equal(id(9007199254740993n), '9007199254740993');
+ assert.equal(id(42), '42');
+ assert.throws(() => id(Number('9007199254740993')), /安全整数/);
+ for (const value of [0, -1, 1.5, '01', '1e3', ' 1', '', null, undefined, true]) {
+ assert.throws(() => id(value), /正整数|安全整数/, String(value));
+ }
+});
+
+test('MySQL BIT values distinguish zero buffers from true values', () => {
+ for (const value of [Buffer.from([0]), Buffer.alloc(0), false, 0, '0', null, undefined]) assert.equal(bit(value), false);
+ for (const value of [Buffer.from([1]), true, 1, '1']) assert.equal(bit(value), true);
+});
+
+test('configuration normalizes IDs and schemas without changing bigint precision', () => {
+ const result = normalizeConfig(baseConfig({
+ sourceConfigIds: [1, '1', '9007199254740993'],
+ referenceSchemas: ['chat', 'unit_test', 'chat'],
+ oldDomains: ['https://old.example.test///'],
+ }));
+ assert.deepEqual(result.sourceConfigIds, ['1', '9007199254740993']);
+ assert.deepEqual(result.referenceSchemas, ['unit_test', 'chat']);
+ assert.deepEqual(result.oldDomains, ['https://old.example.test']);
+ assert.equal(result.downloadBaseUrl, sourceBase);
+});
+
+test('configuration rejects same source and target, invalid modes and unsafe batch limits', () => {
+ for (const overrides of [
+ { targetConfigId: '1' }, { sourceMode: 'unknown' }, { targetUrlMode: 'unknown' },
+ { batchSize: 0 }, { batchSize: 1001 }, { retries: 0 }, { retries: 11 },
+ { maxFileBytes: 1.5 }, { timeoutMs: 0 }, { objectPrefix: '../unsafe' },
+ ]) assert.throws(() => normalizeConfig(baseConfig(overrides)));
+});
+
+test('database source and direct target do not require a download address', () => {
+ const result = normalizeConfig(baseConfig({ sourceMode: 'database', downloadBaseUrl: undefined }));
+ assert.equal(result.sourceMode, 'database');
+});
+
+test('OSS settings use the actual Java configuration field names and support JSON storage', () => {
+ const row = storageRow();
+ row.config = JSON.stringify(row.config);
+ assert.equal(parseStorage(row).accessKey, 'test-only-access-key');
+ assert.deepEqual(ossSettings(row, {}), {
+ endpoint: 'https://oss-cn-hangzhou.aliyuncs.com', bucket: 'test-only-bucket',
+ accessKeyId: 'test-only-access-key', accessKeySecret: 'test-only-access-secret',
+ publicDomain: 'https://test-only-bucket.oss-cn-hangzhou.aliyuncs.com',
+ });
+});
+
+test('OSS internal endpoint gets a public fallback domain and explicit domain takes precedence', () => {
+ const row = storageRow({ endpoint: 'https://oss-cn-hangzhou-internal.aliyuncs.com' });
+ const result = ossSettings(row, {});
+ assert.equal(result.endpoint, 'https://oss-cn-hangzhou-internal.aliyuncs.com');
+ assert.equal(result.publicDomain, 'https://test-only-bucket.oss-cn-hangzhou.aliyuncs.com');
+ assert.equal(ossSettings(storageRow({ domain: 'https://cdn.example.test/' }), {}).publicDomain, 'https://cdn.example.test');
+ assert.equal(ossSettings(row, { targetPublicDomain: 'https://override.example.test/assets/' }).publicDomain, 'https://override.example.test/assets');
+});
+
+test('OSS settings reject wrong storage type, non-OSS services, bucket endpoints and missing fields', () => {
+ assert.throws(() => ossSettings({ ...storageRow(), storage: 1 }, {}), /storage=20/);
+ for (const endpoint of [
+ 'http://127.0.0.1:9000', 's3.amazonaws.com', 'bucket.oss-cn-hangzhou.aliyuncs.com',
+ 'oss-cn-hangzhou.aliyuncs.com.evil.test', 'https://oss-cn-hangzhou.aliyuncs.com/path',
+ 'https://user:pass@oss-cn-hangzhou.aliyuncs.com', 'https://oss-cn-hangzhou.aliyuncs.com/?x=1',
+ ]) assert.throws(() => ossSettings(storageRow({ endpoint }), {}), /endpoint/);
+ for (const field of ['bucket', 'accessKey', 'accessSecret']) {
+ assert.throws(() => ossSettings(storageRow({ [field]: '' }), {}), /缺少/);
+ }
+});
+
+test('object keys separate config, source path and content while keeping a safe extension', () => {
+ const config = { objectPrefix: 'migration/test' };
+ const key = objectKey(config, ITEM);
+ assert.equal(key, 'migration/test/' + ITEM.configId + '/' + sha256(ITEM.path) + '/' + ITEM.sha256 + '.png');
+ assert.notEqual(key, objectKey(config, { ...ITEM, configId: '2' }));
+ assert.notEqual(key, objectKey(config, { ...ITEM, path: 'other.PNG' }));
+ assert.notEqual(key, objectKey(config, { ...ITEM, sha256: 'b'.repeat(64) }));
+});
+
+test('aliases retain unrelated stored URLs and only route variants resolving to the source path', () => {
+ const row = { path: ITEM.path, url: 'https://special.example.test/persisted-image' };
+ const result = aliases(ITEM, [row, row], 'https://old.example.test', {
+ oldDomains: ['https://older.example.test'], downloadBaseUrl: sourceBase,
+ });
+ assert.equal(result.filter(value => value === row.url).length, 1);
+ assert.ok(!result.includes('https://old.example.test/admin-api/infra/file/' + ITEM.configId + '/get/' + ITEM.path));
+ assert.ok(result.includes('https://older.example.test/admin-api/infra/file/' + ITEM.configId + '/get/' + encodePath(ITEM.path)));
+ assert.ok(result.includes(downloadUrl(sourceBase, ITEM.configId, ITEM.path)));
+});
+
+test('plus and space source files cannot claim each other through raw plus aliases', () => {
+ const domain = 'https://old.example.test';
+ const prefix = domain + '/admin-api/infra/file/1/get/';
+ const config = { oldDomains: [], downloadBaseUrl: sourceBase };
+ const plus = aliases({ configId: '1', path: 'a+b.png' }, [
+ { path: 'a+b.png', url: prefix + 'a+b.png' },
+ { path: 'a+b.png', url: prefix + 'a%2Bb.png' },
+ ], domain, config);
+ const space = aliases({ configId: '1', path: 'a b.png' }, [
+ { path: 'a b.png', url: prefix + 'a+b.png' },
+ ], domain, config);
+ assert.ok(!plus.includes(prefix + 'a+b.png'));
+ assert.ok(plus.includes(prefix + 'a%2Bb.png'));
+ assert.ok(space.includes(prefix + 'a+b.png'));
+ assert.ok(space.includes(prefix + 'a b.png'));
+ assert.ok(space.includes(prefix + 'a%20b.png'));
+ assert.ok(!space.includes(prefix + 'a%2Bb.png'));
+ assert.ok(space.includes(sourceBase + '/1/get/a+b.png'));
+ assert.deepEqual(plus.filter(url => space.includes(url)), []);
+});
+
+test('percent-encoded-looking filenames keep their meaning after exactly one route decode', () => {
+ const domain = 'https://old.example.test';
+ const prefix = domain + '/admin-api/infra/file/1/get/';
+ const config = { oldDomains: [] };
+ const literal = aliases({ configId: '1', path: 'a%2Fb.png' }, [
+ { path: 'a%2Fb.png', url: prefix + 'a%2Fb.png' },
+ ], domain, config);
+ const directory = aliases({ configId: '1', path: 'a/b.png' }, [
+ { path: 'a/b.png', url: prefix + 'a%2Fb.png' },
+ ], domain, config);
+ assert.ok(!literal.includes(prefix + 'a%2Fb.png'));
+ assert.ok(literal.includes(prefix + 'a%252Fb.png'));
+ assert.ok(directory.includes(prefix + 'a%2Fb.png'));
+ assert.ok(directory.includes(prefix + 'a/b.png'));
+ assert.deepEqual(literal.filter(url => directory.includes(url)), []);
+});
+
+test('raw query, fragment and unverifiable percent paths are excluded from source aliases', () => {
+ const domain = 'https://old.example.test';
+ const prefix = domain + '/admin-api/infra/file/1/get/';
+ for (const path of ['question?.png', 'fragment#.png', 'literal%.png', 'a%20b.png']) {
+ const result = aliases({ configId: '1', path }, [{ path, url: prefix + path }], domain, { oldDomains: [] });
+ assert.ok(!result.includes(prefix + path), path);
+ assert.ok(result.includes(prefix + encodePath(path)), path);
+ }
+});
+
+test('route validation accepts SQL-equivalent spellings and ignores genuine query parameters', () => {
+ const domain = 'https://old.example.test';
+ const prefix = domain + '/admin-api/infra/file/1/get/';
+ const alternate = 'photo a.JPG';
+ const result = aliases({ configId: '1', path: 'Photo A.jpg' }, [
+ { path: alternate, url: prefix + 'photo+a.JPG?download=1#preview' },
+ { path: alternate, url: domain + '/admin-api/infra/file/2/get/photo+a.JPG' },
+ { path: alternate, url: prefix + 'unrelated.png' },
+ ], domain, { oldDomains: [] });
+ assert.ok(result.includes(prefix + 'photo+a.JPG?download=1#preview'));
+ assert.ok(result.includes(prefix + 'photo+a.JPG'));
+ assert.ok(result.includes(prefix + 'Photo+A.jpg'));
+ assert.ok(!result.includes(domain + '/admin-api/infra/file/2/get/photo+a.JPG'));
+ assert.ok(!result.includes(prefix + 'unrelated.png'));
+});
+
+test('MIME detection prefers stored image type, then extension, then other stored type', () => {
+ assert.equal(mimeType('renamed.jpg', [{ type: 'image/png' }]), 'image/png');
+ assert.equal(mimeType('image.WEBP', []), 'image/webp');
+ assert.equal(mimeType('no-extension', [{ type: 'application/pdf' }]), 'application/pdf');
+ assert.equal(mimeType('no-extension', []), 'application/octet-stream');
+});
+
+test('SQL identifiers are quoted and error URLs are redacted', () => {
+ assert.equal(quote('strange\u0060name'), '\u0060strange\u0060\u0060name\u0060');
+ assert.equal(qualified('schema', 'table'), '\u0060schema\u0060.\u0060table\u0060');
+ const redacted = errorText(new Error('failed https://example.test/?secret=private afterward'));
+ assert.equal(redacted, 'failed [URL] afterward');
+ assert.ok(errorText(new Error('x'.repeat(500))).length <= 300);
+});
+
+test('HTTP download sends the intended encoded route and preserves binary bytes', async t => {
+ let observedUrl;
+ const base = await localServer(t, (request, response) => {
+ observedUrl = request.url;
+ response.writeHead(200, { 'Content-Type': 'image/png', 'Content-Length': IMAGE.length });
+ response.end(IMAGE);
+ });
+ const route = downloadUrl(base + '/admin-api/infra/file', ITEM.configId, ITEM.path);
+ const bytes = await readHttp(route, HTTP_OPTIONS, IMAGE.length);
+ assert.deepEqual(bytes, IMAGE);
+ assert.equal(observedUrl, '/admin-api/infra/file/' + ITEM.configId + '/get/' + encodePath(ITEM.path));
+ assert.doesNotThrow(() => verifyContent(bytes, ITEM));
+});
+
+test('HTTP errors and redirects are rejected rather than uploaded', async t => {
+ const base = await localServer(t, (request, response) => {
+ if (request.url === '/redirect') response.writeHead(302, { Location: '/image' });
+ else response.writeHead(404);
+ response.end();
+ });
+ await assert.rejects(readHttp(base + '/missing', HTTP_OPTIONS), /HTTP 404/);
+ await assert.rejects(readHttp(base + '/redirect', HTTP_OPTIONS));
+});
+
+test('HTTP 200 HTML or JSON with the same byte count fails hash verification', async t => {
+ const base = await localServer(t, (request, response) => {
+ const fake = Buffer.alloc(IMAGE.length, ' ');
+ fake.write(request.url === '/html' ? 'error' : '{"error":true}');
+ response.writeHead(200, { 'Content-Type': request.url === '/html' ? 'text/html' : 'application/json' });
+ response.end(fake);
+ });
+ for (const path of ['/html', '/json']) {
+ const bytes = await readHttp(base + path, HTTP_OPTIONS, IMAGE.length);
+ assert.equal(bytes.length, IMAGE.length);
+ assert.throws(() => verifyContent(bytes, ITEM), /SHA-256/);
+ }
+});
+
+test('short downloads fail expected length validation', async t => {
+ const base = await localServer(t, (_request, response) => response.end(IMAGE.subarray(0, 4)));
+ const bytes = await readHttp(base, HTTP_OPTIONS, IMAGE.length);
+ assert.throws(() => verifyContent(bytes, ITEM), /长度/);
+});
+
+test('HTTP declared length exceeding either cap is rejected', async t => {
+ const base = await localServer(t, (_request, response) => {
+ response.writeHead(200, { 'Content-Length': IMAGE.length });
+ response.end(IMAGE);
+ });
+ await assert.rejects(readHttp(base, { ...HTTP_OPTIONS, maxFileBytes: 4 }), /长度超过限制/);
+ await assert.rejects(readHttp(base, HTTP_OPTIONS, 4), /长度超过限制/);
+});
+
+test('chunked HTTP bodies cannot bypass actual byte limits', async t => {
+ const base = await localServer(t, (_request, response) => {
+ response.writeHead(200, { 'Transfer-Encoding': 'chunked' });
+ response.write(IMAGE.subarray(0, 8));
+ response.end(IMAGE.subarray(8));
+ });
+ await assert.rejects(readHttp(base, { ...HTTP_OPTIONS, maxFileBytes: 4 }), /长度超过限制/);
+ await assert.rejects(readHttp(base, HTTP_OPTIONS, 4), /长度超过限制/);
+});
+
+test('HTTP requests time out when the source never responds', async t => {
+ const base = await localServer(t, () => {});
+ await assert.rejects(readHttp(base, { ...HTTP_OPTIONS, timeoutMs: 30 }));
+});
+
+test('content verification checks both length and hash', () => {
+ assert.doesNotThrow(() => verifyContent(IMAGE, { ...ITEM, size: String(IMAGE.length) }));
+ assert.throws(() => verifyContent(IMAGE, { ...ITEM, size: IMAGE.length + 1 }), /长度/);
+ assert.throws(() => verifyContent(IMAGE, { ...ITEM, sha256: '0'.repeat(64) }), /SHA-256/);
+});
+
+test('existing identical OSS objects are read and verified without upload', async () => {
+ const oss = storedObject(IMAGE);
+ await ensureObject(oss, 'target/key.png', IMAGE, ITEM);
+ assert.equal(oss.put.mock.callCount(), 0);
+ assert.equal(oss.get.mock.callCount(), 1);
+ assert.equal(oss.get.mock.calls[0].arguments[0], 'target/key.png');
+});
+
+test('existing OSS objects with a different length are not overwritten', async () => {
+ const oss = storedObject(Buffer.from('wrong length'));
+ await assert.rejects(ensureObject(oss, 'target/key.png', IMAGE, ITEM), /禁止覆盖/);
+ assert.equal(oss.put.mock.callCount(), 0);
+ assert.equal(oss.get.mock.callCount(), 0);
+});
+
+test('existing same-length OSS objects with different bytes are not overwritten', async () => {
+ const oss = storedObject(Buffer.alloc(IMAGE.length, 1));
+ await assert.rejects(ensureObject(oss, 'target/key.png', IMAGE, ITEM), /SHA-256/);
+ assert.equal(oss.put.mock.callCount(), 0);
+});
+
+test('missing OSS objects upload exact bytes with MIME and overwrite prevention', async () => {
+ let saved;
+ const oss = {
+ head: mock.fn(async () => { if (!saved) throw noSuchKey(); return head(saved); }),
+ put: mock.fn(async (_key, bytes) => { saved = Buffer.from(bytes); }),
+ get: mock.fn(async () => ({ content: saved })),
+ };
+ await ensureObject(oss, 'target/key.png', IMAGE, ITEM);
+ assert.equal(oss.put.mock.callCount(), 1);
+ assert.deepEqual(oss.put.mock.calls[0].arguments, [
+ 'target/key.png', IMAGE, { headers: { 'Content-Type': 'image/png', 'x-oss-forbid-overwrite': 'true' } },
+ ]);
+ assert.deepEqual(saved, IMAGE);
+});
+
+test('OSS HEAD authorization failures do not cause a speculative upload', async () => {
+ const denied = Object.assign(new Error('denied'), { code: 'AccessDenied', status: 403 });
+ const oss = { head: mock.fn(async () => { throw denied; }), put: mock.fn(), get: mock.fn() };
+ await assert.rejects(ensureObject(oss, 'key', IMAGE, ITEM), error => error === denied);
+ assert.equal(oss.put.mock.callCount(), 0);
+});
+
+test('upload competition is accepted only after verifying the winner content', async () => {
+ for (const sameContent of [true, false]) {
+ const winner = sameContent ? IMAGE : Buffer.alloc(IMAGE.length, 2);
+ let appeared = false;
+ const oss = {
+ head: mock.fn(async () => { if (!appeared) throw noSuchKey(); return head(winner); }),
+ put: mock.fn(async () => {
+ appeared = true;
+ throw Object.assign(new Error('another upload won'), { code: 'FileAlreadyExists', status: 409 });
+ }),
+ get: mock.fn(async () => ({ content: winner })),
+ };
+ if (sameContent) await ensureObject(oss, 'key', IMAGE, ITEM);
+ else await assert.rejects(ensureObject(oss, 'key', IMAGE, ITEM), /SHA-256/);
+ assert.equal(oss.put.mock.callCount(), 1);
+ assert.equal(oss.get.mock.callCount(), 1);
+ }
+});
+
+test('failed or corrupted OSS uploads never report success', async () => {
+ const offline = Object.assign(new Error('offline'), { code: 'ConnectionTimeout' });
+ const failed = {
+ head: mock.fn(async () => { throw noSuchKey(); }),
+ put: mock.fn(async () => { throw offline; }), get: mock.fn(),
+ };
+ await assert.rejects(ensureObject(failed, 'key', IMAGE, ITEM), error => error === offline);
+ assert.equal(failed.get.mock.callCount(), 0);
+ let uploaded = false;
+ const corrupt = {
+ head: mock.fn(async () => { if (!uploaded) throw noSuchKey(); return head(IMAGE); }),
+ put: mock.fn(async () => { uploaded = true; }),
+ get: mock.fn(async () => ({ content: Buffer.alloc(IMAGE.length) })),
+ };
+ await assert.rejects(ensureObject(corrupt, 'key', IMAGE, ITEM), /SHA-256/);
+});
+
+test('retry stops on success and uses capped exponential waits', async () => {
+ let calls = 0;
+ const waits = [];
+ const value = await retry(async () => {
+ if (++calls < 7) throw new Error('transient');
+ return 'done';
+ }, 7, async ms => { waits.push(ms); });
+ assert.equal(value, 'done');
+ assert.equal(calls, 7);
+ assert.deepEqual(waits, [1000, 2000, 4000, 5000, 5000, 5000]);
+});
+
+test('retry preserves the last failure and never exceeds the configured attempt count', async () => {
+ let calls = 0;
+ const waits = [];
+ const failure = new Error('permanent');
+ await assert.rejects(retry(async () => { calls++; throw failure; }, 3, async ms => waits.push(ms)), error => error === failure);
+ assert.equal(calls, 3);
+ assert.deepEqual(waits, [1000, 2000]);
+ const first = mock.fn(async () => 'first');
+ const wait = mock.fn();
+ assert.equal(await retry(first, 3, wait), 'first');
+ assert.equal(first.mock.callCount(), 1);
+ assert.equal(wait.mock.callCount(), 0);
+});
diff --git a/script/migrate-db-files-to-oss/test/integration.test.mjs b/script/migrate-db-files-to-oss/test/integration.test.mjs
new file mode 100644
index 0000000..fbe8147
--- /dev/null
+++ b/script/migrate-db-files-to-oss/test/integration.test.mjs
@@ -0,0 +1,241 @@
+import test from 'node:test';
+import assert from 'node:assert/strict';
+import { createServer } from 'node:http';
+import { once } from 'node:events';
+import { encodePath, normalizeConfig } from '../lib/core.mjs';
+import { applyChange } from '../lib/journal.mjs';
+import { discoverStorageConfig } from '../lib/auto-storage.mjs';
+import { loadSourceIndex } from '../lib/source-index.mjs';
+import { loadSettings, runApply, runPlan, runRollback } from '../migrate.mjs';
+
+// Only point this at the disposable test MySQL container: this suite recreates
+// its dedicated codex_oss_migration_test schema, never the application database.
+test('MySQL migration, resume, exact reference replacement and rollback', {
+ skip: !process.env.MIGRATION_TEST_DB_PORT, timeout: 60000
+}, async t => {
+ const { default: mysql } = await import('mysql2/promise');
+ const db = await mysql.createConnection({ host: '127.0.0.1', port: Number(process.env.MIGRATION_TEST_DB_PORT), user: 'root', password: '', supportBigNumbers: true, bigNumberStrings: true, jsonStrings: true, dateStrings: true });
+ const objects = new Map();
+ let putCount = 0, failPut = false, sourceDownloads = 0;
+ const oss = {
+ async head(key) { if (!objects.has(key)) throw Object.assign(new Error('missing'), { code: 'NoSuchKey', status: 404 }); return { res: { headers: { 'content-length': String(objects.get(key).length) } } }; },
+ async get(key) { return { content: Buffer.from(objects.get(key)) }; },
+ async put(key, bytes, options) {
+ assert.equal(options.headers['x-oss-forbid-overwrite'], 'true');
+ if (failPut) throw Object.assign(new Error('test failure'), { code: 'TestOssUnavailable' });
+ assert.ok(!objects.has(key)); objects.set(key, Buffer.from(bytes)); putCount++;
+ }
+ };
+ const server = createServer(async (request, response) => {
+ try {
+ const url = new URL(request.url, 'http://localhost');
+ if (url.pathname.startsWith('/objects/')) {
+ const bytes = objects.get(decodeURIComponent(url.pathname.slice('/objects/'.length)));
+ if (!bytes) { response.writeHead(404).end(); return; }
+ response.end(bytes); return;
+ }
+ const prefix = '/admin-api/infra/file/5/get/';
+ if (url.pathname.startsWith(prefix)) {
+ sourceDownloads++;
+ const path = decodeURIComponent(url.pathname.slice(prefix.length).replaceAll('+', ' '));
+ const [[row]] = await db.execute('SELECT content FROM infra_file_content WHERE config_id=5 AND path=? AND deleted=0 ORDER BY id DESC LIMIT 1', [path]);
+ if (row) response.end(row.content); else response.writeHead(404).end();
+ return;
+ }
+ response.writeHead(404).end();
+ } catch { response.writeHead(500).end(); }
+ });
+ server.listen(0, '127.0.0.1'); await once(server, 'listening');
+ const origin = `http://127.0.0.1:${server.address().port}`;
+ const schema = 'codex_oss_migration_test';
+ let config;
+ const firstPath = '头像/张 三+100%.png';
+ const firstUrl = `${origin}/admin-api/infra/file/5/get/${encodePath(firstPath)}`;
+ const orphanUrl = `${origin}/admin-api/infra/file/5/get/orphan.png`;
+ const caseUrl = `${origin}/admin-api/infra/file/5/get/Case.png`;
+ const originals = [Buffer.from('first current image'), Buffer.from('second orphan image'), Buffer.from('case latest image')];
+ try {
+ await db.query(`DROP DATABASE IF EXISTS ${schema}`);
+ await db.query(`CREATE DATABASE ${schema} CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci`);
+ await db.changeUser({ database: schema });
+ await db.query('CREATE TABLE infra_file_config (id BIGINT PRIMARY KEY,storage INT,master BIT(1),config TEXT,deleted BIT(1) DEFAULT 0) ENGINE=InnoDB');
+ await db.query('CREATE TABLE infra_file_content (id BIGINT PRIMARY KEY,config_id BIGINT,path VARCHAR(512),content MEDIUMBLOB,deleted BIT(1) DEFAULT 0) ENGINE=InnoDB');
+ await db.query('CREATE TABLE infra_file (id BIGINT PRIMARY KEY,config_id BIGINT,path VARCHAR(512),url VARCHAR(1024),type VARCHAR(128),size INT,deleted BIT(1) DEFAULT 0,update_time TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP) ENGINE=InnoDB');
+ await db.query('CREATE TABLE member_user (id BIGINT PRIMARY KEY,avatar VARCHAR(1024),wechat_qr_code VARCHAR(255),extra_json JSON,update_time TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP) ENGINE=InnoDB');
+ await db.query('CREATE TABLE notes (id INT PRIMARY KEY,body TEXT) ENGINE=InnoDB');
+ await db.execute('INSERT INTO infra_file_config VALUES (5,1,0,?,0),(30,20,1,?,0)', [JSON.stringify({ domain: origin }), JSON.stringify({ endpoint: 'oss-cn-hangzhou.aliyuncs.com', bucket: 'test', accessKey: 'test-key', accessSecret: 'test-secret', domain: `${origin}/objects` })]);
+ const records = [
+ ['9007199254740992', 'Case.png', Buffer.from('case old image'), 0],
+ ['9007199254740993', firstPath, Buffer.from('first old image'), 0],
+ ['9007199254740994', firstPath, originals[0], 0],
+ ['9007199254740995', firstPath, Buffer.from('deleted newest'), 1],
+ ['9007199254740996', 'orphan.png', originals[1], 0],
+ ['9007199254740997', 'case.PNG', originals[2], 0]
+ ];
+ for (const [contentId, path, content, deleted] of records) await db.execute('INSERT INTO infra_file_content VALUES (?,5,?,?,?)', [contentId, path, content, deleted]);
+ for (const [fileId, path, url] of [['9007199254741001', firstPath, firstUrl], ['9007199254741002', 'Case.png', caseUrl], ['9007199254741003', 'case.PNG', caseUrl.replace('Case.png', 'case.PNG')]]) {
+ await db.execute("INSERT INTO infra_file (id,config_id,path,url,type,size,update_time) VALUES (?,5,?,?,'image/png',1,'2020-01-01 00:00:00')", [fileId, path, url]);
+ }
+ const payload = JSON.stringify({ a: [firstUrl, orphanUrl], nested: { image: caseUrl }, outside: 'https://outside.example/image.png' });
+ await db.execute("INSERT INTO member_user VALUES (9007199254741101,?,?,?,'2020-01-01 00:00:00')", [firstUrl, orphanUrl, payload]);
+ await db.execute('INSERT INTO notes VALUES (1,?)', [`
${orphanUrl},${caseUrl}`]);
+ config = normalizeConfig({ runId: 'integration', mysql: { database: schema, user: 'root' }, sourceConfigIds: ['5'], targetConfigId: '30', downloadBaseUrl: `${origin}/admin-api/infra/file`, batchSize: 1, retries: 1 });
+ const settings = await loadSettings(db, config);
+
+ await t.test('automatic discovery uses the existing database and unique OSS without configuration writes', async () => {
+ const automatic = await discoverStorageConfig(db, { mysql: config.mysql, _autoDiscovery: true, _projectDownloadBaseUrl: config.downloadBaseUrl });
+ assert.deepEqual(automatic.sourceConfigIds, ['5']); assert.equal(automatic.targetConfigId, '30');
+ assert.equal(automatic.sourceMode, 'database'); assert.ok(automatic.runId.startsWith('db-images-'));
+ assert.ok(automatic.oldDomains.includes(origin)); assert.equal(objects.size, 0);
+ const [tables] = await db.query("SHOW TABLES LIKE 'infra_file_oss_migration_%'"); assert.equal(tables.length, 0);
+ });
+
+ await t.test('plan is read-only and uses latest active content with bigint/collation semantics', async () => {
+ const statements = [];
+ const readOnlyDb = {
+ execute(sql, params) { statements.push(sql); return db.execute(sql, params); },
+ query(sql, params) { statements.push(sql); return db.query(sql, params); }
+ };
+ const plan = await runPlan(readOnlyDb, config, settings);
+ assert.equal(plan.status, 'planned'); assert.equal(plan.candidates, 3); assert.equal(plan.withoutFileRecord, 1);
+ assert.equal(plan.bytes, null); assert.equal(plan.estimatedBytes, 2); assert.equal(plan.sizePending, 1); assert.equal(plan.contentVerified, false);
+ assert.ok(statements.every(sql => /^\s*SELECT/i.test(sql)));
+ assert.ok(statements.every(sql => !/\b(?:SHA2|OCTET_LENGTH|content)\b|NOT EXISTS/i.test(sql)), 'plan must never scan image bytes or use per-file correlated subqueries');
+ assert.equal(plan.references.changedRows, 2); assert.equal(objects.size, 0);
+ const [rows] = await db.query("SHOW TABLES LIKE 'infra_file_oss_migration_%'"); assert.equal(rows.length, 0);
+ });
+
+ await t.test('metadata upper bound excludes newer versions without misclassifying their files as missing', async () => {
+ const index = await loadSourceIndex(db, config, { maxId: '9007199254740996' });
+ assert.equal(index.groups.length, 2); assert.equal(index.missingFiles.length, 0);
+ assert.equal(index.groups[0].path, firstPath); assert.equal(index.groups[1].path, 'orphan.png');
+ const full = await loadSourceIndex(db, config);
+ assert.deepEqual(full.groups[2].spellings.slice().sort(), ['Case.png', 'case.PNG'].sort());
+ assert.equal(full.groups[2].files.length, 2);
+ });
+
+ await t.test('limited trial, resume and repeated apply preserve originals and journals', async () => {
+ const trial = await runApply(db, config, settings, 1, () => oss);
+ assert.equal(trial.status, 'partial'); assert.equal(trial.items.switched, 1); assert.equal(objects.size, 1);
+ const resumedConfig = await discoverStorageConfig(db, { mysql: config.mysql, _autoDiscovery: true }, { command: 'apply' });
+ assert.equal(resumedConfig.runId, 'integration'); assert.deepEqual(resumedConfig.sourceConfigIds, ['5']);
+ assert.equal(resumedConfig.sourceMode, 'http'); assert.equal(resumedConfig.downloadBaseUrl, config.downloadBaseUrl);
+ assert.deepEqual((await loadSettings(db, resumedConfig)).safe, settings.safe);
+ // Simulate interruption after all remaining uploads reached verified but
+ // before their first file transaction. Resume must reuse the checkpoint.
+ const interruptedDb = {
+ execute: (...args) => db.execute(...args), query: (...args) => db.query(...args),
+ beginTransaction: async () => { throw new Error('simulated interruption before transition'); }
+ };
+ await assert.rejects(runApply(interruptedDb, config, settings, Infinity, () => oss), /simulated interruption/);
+ const [[checkpoints]] = await db.query("SELECT COUNT(*) AS n FROM infra_file_oss_migration_items WHERE run_id='integration' AND status='verified'");
+ assert.equal(Number(checkpoints.n), 2);
+ const downloadsBeforeResume = sourceDownloads;
+ const putsBeforeResume = putCount;
+ const done = await runApply(db, config, settings, Infinity, () => oss);
+ assert.equal(done.status, 'complete'); assert.equal(done.items.switched, 3); assert.equal(objects.size, 3); assert.equal(putCount, 3);
+ assert.equal(done.revalidated, 2); assert.equal(done.uploaded, 0);
+ assert.equal(sourceDownloads, downloadsBeforeResume); assert.equal(putCount, putsBeforeResume);
+ const [[member]] = await db.query('SELECT * FROM member_user');
+ assert.ok(member.avatar.startsWith(`${origin}/objects/`)); assert.ok(member.wechat_qr_code.startsWith(`${origin}/objects/`));
+ assert.deepEqual(Object.values(JSON.parse(member.extra_json).nested).map(url => url.startsWith(`${origin}/objects/`)), [true]);
+ assert.equal(JSON.parse(member.extra_json).outside, 'https://outside.example/image.png');
+ const [[counts]] = await db.query('SELECT COUNT(*) AS n FROM infra_file_content'); assert.equal(Number(counts.n), 6);
+ const [files] = await db.query('SELECT * FROM infra_file ORDER BY id');
+ assert.ok(files.every(file => file.config_id === '30'));
+ assert.equal(files[1].path, files[2].path); // case/accent-insensitive source lookup remains equivalent
+ assert.equal(files[0].size, originals[0].length); // stale old metadata is repaired
+ assert.ok(files.every(file => file.update_time === '2020-01-01 00:00:00'));
+ const [[before]] = await db.query('SELECT COUNT(*) AS n FROM infra_file_oss_migration_changes');
+ const repeated = await runApply(db, config, settings, Infinity, () => oss); assert.equal(repeated.status, 'complete');
+ const [[after]] = await db.query('SELECT COUNT(*) AS n FROM infra_file_oss_migration_changes'); assert.equal(before.n, after.n); assert.equal(putCount, 3);
+ const [[entry]] = await db.query("SELECT data FROM infra_file_oss_migration_items WHERE JSON_UNQUOTE(JSON_EXTRACT(data,'$.path')) LIKE '%100%.png'");
+ assert.ok(JSON.parse(entry.data).aliases.includes(firstUrl)); // logs must never rewrite themselves
+ });
+
+ await t.test('resume does not rewrite new references using a deleted OSS object', async () => {
+ const [[entry]] = await db.query("SELECT data FROM infra_file_oss_migration_items WHERE JSON_UNQUOTE(JSON_EXTRACT(data,'$.path')) LIKE '%100%.png'");
+ const item = JSON.parse(entry.data), bytes = objects.get(item.key);
+ objects.delete(item.key);
+ await db.execute('INSERT INTO notes VALUES (2,?)', [firstUrl]);
+ const result = await runApply(db, config, settings, Infinity, () => oss);
+ assert.equal(result.status, 'partial'); assert.equal(result.failed, 1);
+ const [[note]] = await db.query('SELECT body FROM notes WHERE id=2'); assert.equal(note.body, firstUrl);
+ objects.set(item.key, bytes); await db.query('DELETE FROM notes WHERE id=2');
+ });
+
+ await t.test('rollback preserves later user edits and can resume after conflict resolution', async () => {
+ const [[migrated]] = await db.query('SELECT avatar FROM member_user');
+ await db.execute('UPDATE member_user SET avatar=?', ['https://user-new.example/avatar.png']);
+ const rollback = await runRollback(db, config); assert.equal(rollback.status, 'rollback_partial'); assert.ok(rollback.conflicts.length > 0);
+ const [[current]] = await db.query('SELECT avatar FROM member_user'); assert.equal(current.avatar, 'https://user-new.example/avatar.png');
+ await db.execute('UPDATE member_user SET avatar=?', [migrated.avatar]);
+ const resumed = await runRollback(db, config); assert.equal(resumed.status, 'rolled_back');
+ const [[member]] = await db.query('SELECT * FROM member_user'); assert.equal(member.avatar, firstUrl); assert.equal(member.wechat_qr_code, orphanUrl); assert.deepEqual(JSON.parse(member.extra_json), JSON.parse(payload));
+ const [files] = await db.query('SELECT * FROM infra_file'); assert.ok(files.every(file => file.config_id === '5')); assert.equal(objects.size, 3);
+ await assert.rejects(runApply(db, config, settings, Infinity, () => oss), /回滚/);
+ });
+
+ await t.test('failed uploads persist a partial state and never switch database addresses', async () => {
+ const failureConfig = { ...config, runId: 'upload-failure', objectPrefix: 'migration/failure' };
+ const failureSettings = await loadSettings(db, failureConfig); failPut = true;
+ const result = await runApply(db, failureConfig, failureSettings, Infinity, () => oss); failPut = false;
+ assert.equal(result.status, 'partial'); assert.equal(result.items.failed, 3);
+ const [[run]] = await db.query("SELECT status FROM infra_file_oss_migration_runs WHERE run_id='upload-failure'"); assert.equal(run.status, 'partial');
+ const [[member]] = await db.query('SELECT avatar FROM member_user'); assert.equal(member.avatar, firstUrl);
+ });
+
+ await t.test('non-strict MySQL truncation warnings roll back both data and journal', async () => {
+ await db.query('CREATE TABLE short_url (id INT PRIMARY KEY,url VARCHAR(8)) ENGINE=InnoDB');
+ await db.query("INSERT INTO short_url VALUES (1,'old')");
+ const [[mode]] = await db.query('SELECT @@SESSION.sql_mode AS value');
+ await db.query("SET SESSION sql_mode=''"); await db.beginTransaction();
+ await assert.rejects(applyChange(db, schema, 'warning', { schema, table: 'short_url', primaryKey: 'id', pkValue: 1, before: { url: 'old' }, after: { url: 'this-would-be-truncated' } }), /警告/);
+ await db.rollback(); await db.execute('SET SESSION sql_mode=?', [mode.value]);
+ const [[row]] = await db.query('SELECT url FROM short_url'); assert.equal(row.url, 'old');
+ const [[count]] = await db.query("SELECT COUNT(*) AS n FROM infra_file_oss_migration_changes WHERE run_id='warning'"); assert.equal(Number(count.n), 0);
+ });
+
+ await t.test('batch CAS conflict isolates the edited file and commits unrelated verified items', async () => {
+ const conflictConfig = { ...config, runId: 'batch-conflict', objectPrefix: 'migration/conflict' };
+ const conflictSettings = await loadSettings(db, conflictConfig);
+ let changed = false, rollbacks = 0;
+ const concurrentEditDb = {
+ query: (...args) => db.query(...args), execute: (...args) => db.execute(...args),
+ async beginTransaction() {
+ if (!changed) {
+ changed = true;
+ await db.execute('UPDATE infra_file SET url=? WHERE id=?', ['https://user-change.example/new.png', '9007199254741001']);
+ }
+ return db.beginTransaction();
+ },
+ commit: () => db.commit(),
+ rollback() { rollbacks++; return db.rollback(); }
+ };
+ const result = await runApply(concurrentEditDb, conflictConfig, conflictSettings, Infinity, () => oss);
+ assert.equal(result.status, 'partial'); assert.equal(result.failed, 1);
+ assert.equal(result.items.switched, 2); assert.equal(result.items.verified, 1);
+ assert.ok(rollbacks >= 2, 'the batch and the conflicted single item are rolled back');
+ const [[edited]] = await db.query('SELECT config_id,url FROM infra_file WHERE id=9007199254741001');
+ assert.equal(edited.config_id, '5'); assert.equal(edited.url, 'https://user-change.example/new.png');
+ const [[member]] = await db.query('SELECT avatar FROM member_user'); assert.equal(member.avatar, firstUrl);
+ const rollback = await runRollback(db, conflictConfig); assert.equal(rollback.status, 'rolled_back');
+ await db.execute('UPDATE infra_file SET url=? WHERE id=9007199254741001', [firstUrl]);
+ });
+
+ await t.test('database-source mode works and ambiguous raw-plus links require review', async () => {
+ const rawUrl = `${origin}/admin-api/infra/file/5/get/${firstPath}`;
+ await db.execute('INSERT INTO notes VALUES (3,?)', [rawUrl]);
+ const directConfig = { ...config, runId: 'database-mode', sourceMode: 'database' };
+ const directSettings = await loadSettings(db, directConfig);
+ const result = await runApply(db, directConfig, directSettings, Infinity, () => oss);
+ assert.equal(result.status, 'needs-review'); assert.equal(result.items.switched, 3);
+ const [[note]] = await db.query('SELECT body FROM notes WHERE id=3'); assert.equal(note.body, rawUrl);
+ assert.ok(result.references.unresolvedRows > 0);
+ });
+ } finally {
+ server.close(); server.closeAllConnections();
+ await db.query(`DROP DATABASE IF EXISTS ${schema}`);
+ await db.end();
+ }
+});
diff --git a/script/migrate-db-files-to-oss/test/journal-batch.test.mjs b/script/migrate-db-files-to-oss/test/journal-batch.test.mjs
new file mode 100644
index 0000000..316a75d
--- /dev/null
+++ b/script/migrate-db-files-to-oss/test/journal-batch.test.mjs
@@ -0,0 +1,311 @@
+import test from 'node:test';
+import assert from 'node:assert/strict';
+import { applyChanges } from '../lib/journal.mjs';
+
+const clone = value => Buffer.isBuffer(value) ? Buffer.from(value)
+ : Array.isArray(value) ? value.map(clone)
+ : value && typeof value === 'object' ? Object.fromEntries(Object.entries(value).map(([key, item]) => [key, clone(item)])) : value;
+const key = value => Buffer.isBuffer(value) ? `buffer:${value.toString('hex')}` : String(value);
+const same = (left, right) => Buffer.isBuffer(left) || Buffer.isBuffer(right)
+ ? Buffer.isBuffer(left) && Buffer.isBuffer(right) && left.equals(right)
+ : left === null || right === null ? left === right : String(left) === String(right);
+
+const makeRow = (id, suffix) => ({
+ id, config_id: '4', path: `${suffix}.png`, url: `https://old/${suffix}.png`, type: 'image/png', size: 5,
+ deleted: Buffer.from([0]), update_time: 'unchanged', touched: 'unchanged'
+});
+
+// This small SQL test double evaluates the CASE dispatch and every WHERE guard
+// independently, so mixed per-row values and partially matched batches surface.
+class BatchConnection {
+ constructor(rows = [makeRow('9007199254740993', 'a'), makeRow('9007199254740994', 'b')]) {
+ this.rows = Object.fromEntries(rows.map(row => [key(row.id), clone(row)]));
+ this.calls = [];
+ this.journals = [];
+ this.warningStatus = 0;
+ this.failJournal = false;
+ this.journalWarningStatus = 0;
+ this.journalAffectedRows = null;
+ this.engine = 'InnoDB';
+ this.normalize = null;
+ this.omitReadback = false;
+ this.reverseReadback = false;
+ this.nextId = 9007199254741993n;
+ }
+
+ async beginTransaction() {
+ this.calls.push({ transaction: 'begin' });
+ this.saved = { rows: clone(this.rows), journals: clone(this.journals) };
+ }
+ async commit() { this.calls.push({ transaction: 'commit' }); this.saved = undefined; }
+ async rollback() {
+ this.calls.push({ transaction: 'rollback' });
+ if (this.saved) { this.rows = this.saved.rows; this.journals = this.saved.journals; this.saved = undefined; }
+ }
+
+ async execute(sql, params = []) {
+ this.calls.push({ sql, params });
+ if (sql.includes('FROM information_schema.COLUMNS')) {
+ return [[...['id', 'config_id', 'path', 'url', 'type', 'size', 'deleted', 'json_data', 'counter', 'update_time', 'touched']
+ .map(name => ({ name, engine: this.engine, extra: ['update_time', 'touched'].includes(name) ? 'on update CURRENT_TIMESTAMP' : '' }))]];
+ }
+ if (sql.startsWith('UPDATE')) {
+ const [setSql, whereSql] = sql.split(/\bWHERE\b/);
+ const cases = [...setSql.matchAll(/`([^`]+)`\s*=\s*CASE\s+`id`([\s\S]*?)\bEND/g)];
+ assert.ok(cases.length > 0, 'the batch must use a CASE expression for changed columns');
+ let offset = 0;
+ const values = new Map();
+ for (const [, column, branches] of cases) {
+ const count = [...branches.matchAll(/\bWHEN\s+\?\s+THEN\s+\?/g)].length;
+ assert.ok(count > 0);
+ assert.match(branches, new RegExp('ELSE\\s+`' + column + '`'));
+ const entries = new Map();
+ for (let index = 0; index < count; index++) entries.set(key(params[offset++]), params[offset++]);
+ values.set(column, entries);
+ }
+ let affectedRows = 0;
+ const groups = whereSql.trim().replace(/^\(/, '').replace(/\)$/, '').split(/\)\s+OR\s+\(/);
+ for (const group of groups) {
+ assert.match(group, /`id`\s*=\s*\?/);
+ const pk = key(params[offset++]);
+ const guards = [...group.matchAll(/CAST\(`([^`]+)` AS BINARY\)\s*<=>\s*CAST\(\? AS BINARY\)/g)].map(match => match[1]);
+ const expected = guards.map(() => params[offset++]);
+ const row = this.rows[pk];
+ if (!row || !guards.every((name, index) => same(row[name], expected[index]))) continue;
+ for (const [column, entries] of values) if (entries.has(pk)) row[column] = entries.get(pk);
+ if (this.normalize) this.normalize(row, [...values.keys()]);
+ if (!setSql.includes('`update_time` = `update_time`')) row.update_time = 'changed';
+ if (!setSql.includes('`touched` = `touched`')) row.touched = 'changed';
+ affectedRows++;
+ }
+ assert.equal(offset, params.length, 'all prepared UPDATE values must have a matching SQL placeholder');
+ return [{ affectedRows, warningStatus: this.warningStatus }];
+ }
+ if (sql.startsWith('SELECT `')) {
+ assert.match(sql, /\bIN\s*\(/);
+ assert.match(sql, /FOR UPDATE/);
+ const selected = [...sql.split(/\bFROM\b/)[0].matchAll(/`([^`]+)`/g)].map(match => match[1]);
+ const rows = params.map(pk => this.rows[key(pk)]).filter(Boolean)
+ .map(row => Object.fromEntries(selected.map(column => [column, row[column]])));
+ if (this.reverseReadback) rows.reverse();
+ return [this.omitReadback ? rows.slice(1) : rows];
+ }
+ if (sql.startsWith('INSERT INTO')) {
+ if (this.failJournal) throw Object.assign(new Error('injected journal INSERT failure'), { code: 'TEST_JOURNAL_FAILURE' });
+ assert.equal(params.length % 7, 0);
+ const firstId = this.nextId.toString();
+ for (let offset = 0; offset < params.length; offset += 7) {
+ const [run_id, schema_name, table_name, pk_name, pk_value, before_json, after_json] = params.slice(offset, offset + 7);
+ this.journals.push({ id: (this.nextId++).toString(), run_id, schema_name, table_name, pk_name, pk_value, before_json, after_json, rolled_back: 0 });
+ }
+ return [{ affectedRows: this.journalAffectedRows ?? params.length / 7, warningStatus: this.journalWarningStatus, insertId: firstId }];
+ }
+ throw new Error(`Unexpected test SQL: ${sql}`);
+ }
+}
+
+function change(row, suffix = 'migrated') {
+ const { config_id, path, url, type, size, deleted } = row;
+ return {
+ schema: 'app', table: 'infra_file', primaryKey: 'id', pkValue: row.id,
+ before: { config_id, path, url, type, size, deleted },
+ after: { config_id: '20', path: `${suffix}/${path}`, url: `https://oss/${suffix}/${path}`, type, size }
+ };
+}
+
+const changesFor = connection => Object.values(connection.rows).map(row => change(row));
+const dml = connection => connection.calls.filter(call => call.sql && !call.sql.includes('information_schema'));
+
+async function commitBatch(connection, changes) {
+ await connection.beginTransaction();
+ try {
+ const result = await applyChanges(connection, 'app', 'run', changes);
+ await connection.commit();
+ return result;
+ } catch (error) { await connection.rollback(); throw error; }
+}
+
+test('batch uses one CASE update, one readback and one journal insert for distinct bigint rows', async () => {
+ const connection = new BatchConnection();
+ const changes = changesFor(connection);
+ assert.deepEqual(await commitBatch(connection, changes), { changed: 2, skipped: 0 });
+ assert.deepEqual(dml(connection).map(call => call.sql.split(/\s/)[0]), ['UPDATE', 'SELECT', 'INSERT']);
+ assert.equal(connection.rows['9007199254740993'].url, 'https://oss/migrated/a.png');
+ assert.equal(connection.rows['9007199254740994'].url, 'https://oss/migrated/b.png');
+ assert.equal(connection.rows['9007199254740993'].update_time, 'unchanged');
+ assert.equal(connection.rows['9007199254740994'].touched, 'unchanged');
+ assert.equal(connection.journals.length, 2);
+ assert.deepEqual(connection.journals.map(row => JSON.parse(row.pk_value).value), ['9007199254740993', '9007199254740994']);
+ const [update] = dml(connection);
+ assert.match(update.sql, /CAST\(`deleted` AS BINARY\) <=> CAST\(\? AS BINARY\)/);
+ assert.ok(update.params.some(value => Buffer.isBuffer(value) && value.equals(Buffer.from([0]))));
+ assert.ok(!update.sql.includes('https://oss/'), 'URLs must be bound, not interpolated');
+ assert.ok(!update.sql.includes('`deleted` = CASE'), 'before-only deletion guards must not become assignments');
+});
+
+test('table metadata is cached across batches on the same connection', async () => {
+ const connection = new BatchConnection();
+ await commitBatch(connection, changesFor(connection));
+ await commitBatch(connection, Object.values(connection.rows).map(row => change(row, 'second')));
+ assert.equal(connection.calls.filter(call => call.sql?.includes('information_schema')).length, 1);
+ assert.equal(dml(connection).length, 6);
+});
+
+test('empty batches and unchanged rows create no updates or journal entries', async () => {
+ const connection = new BatchConnection();
+ assert.deepEqual(await commitBatch(connection, []), { changed: 0, skipped: 0 });
+ const changes = changesFor(connection);
+ changes.forEach(item => { item.after = Object.fromEntries(Object.keys(item.after).map(name => [name, item.before[name]])); });
+ assert.deepEqual(await commitBatch(connection, changes), { changed: 0, skipped: 2 });
+ assert.equal(dml(connection).length, 0);
+ assert.equal(connection.journals.length, 0);
+});
+
+test('a no-op row is skipped while a different row receives its own values', async () => {
+ const connection = new BatchConnection();
+ const changes = changesFor(connection);
+ changes[1].after = Object.fromEntries(Object.keys(changes[1].after).map(name => [name, changes[1].before[name]]));
+ assert.deepEqual(await commitBatch(connection, changes), { changed: 1, skipped: 1 });
+ assert.equal(connection.rows['9007199254740993'].url, 'https://oss/migrated/a.png');
+ assert.equal(connection.rows['9007199254740994'].url, 'https://old/b.png');
+ assert.equal(connection.journals.length, 1);
+});
+
+test('after field order may differ and row-specific unchanged fields stay unchanged', async () => {
+ const connection = new BatchConnection();
+ const changes = changesFor(connection);
+ changes[0].after.type = 'image/webp';
+ changes[1].after = Object.fromEntries(Object.entries(changes[1].after).reverse());
+ await commitBatch(connection, changes);
+ assert.equal(connection.rows['9007199254740993'].type, 'image/webp');
+ assert.equal(connection.rows['9007199254740994'].type, 'image/png');
+});
+
+test('different schema, table, primary key, or after-field groups are rejected before updates', async () => {
+ for (const mutate of [
+ item => { item.schema = 'other'; },
+ item => { item.table = 'member_user'; },
+ item => { item.primaryKey = 'other_id'; },
+ item => { delete item.after.size; }
+ ]) {
+ const connection = new BatchConnection();
+ const changes = changesFor(connection);
+ mutate(changes[1]);
+ await assert.rejects(commitBatch(connection, changes));
+ assert.equal(dml(connection).length, 0);
+ }
+});
+
+test('duplicate primary keys are rejected even across string and bigint representations', async () => {
+ const connection = new BatchConnection();
+ const changes = changesFor(connection);
+ changes[1].pkValue = BigInt(changes[0].pkValue);
+ await assert.rejects(commitBatch(connection, changes), /主键|重复/);
+ assert.equal(dml(connection).length, 0);
+});
+
+test('primary key updates and missing before snapshots are rejected', async () => {
+ for (const mutate of [
+ item => { item.after.id = '1'; },
+ item => { delete item.before.url; }
+ ]) {
+ const connection = new BatchConnection();
+ const changes = changesFor(connection);
+ changes.forEach(mutate);
+ await assert.rejects(commitBatch(connection, changes));
+ assert.equal(dml(connection).length, 0);
+ }
+});
+
+test('a concurrent case-only edit leaves the entire batch unchanged after caller rollback', async () => {
+ const connection = new BatchConnection();
+ const changes = changesFor(connection);
+ connection.rows['9007199254740994'].url = 'https://old/B.png';
+ const before = clone(connection.rows);
+ await assert.rejects(commitBatch(connection, changes), { code: 'MIGRATION_CAS_CONFLICT' });
+ assert.deepEqual(connection.rows, before);
+ assert.equal(connection.journals.length, 0);
+ assert.equal(dml(connection).length, 1);
+ assert.equal(connection.calls.at(-1).transaction, 'rollback');
+});
+
+test('a deletion guard conflict rolls back otherwise matching rows', async () => {
+ const connection = new BatchConnection();
+ const changes = changesFor(connection);
+ connection.rows['9007199254740994'].deleted = Buffer.from([1]);
+ const before = clone(connection.rows);
+ await assert.rejects(commitBatch(connection, changes), { code: 'MIGRATION_CAS_CONFLICT' });
+ assert.deepEqual(connection.rows, before);
+});
+
+test('journal INSERT failure rolls back all successfully updated rows', async () => {
+ const connection = new BatchConnection();
+ const changes = changesFor(connection);
+ const before = clone(connection.rows);
+ connection.failJournal = true;
+ await assert.rejects(commitBatch(connection, changes), { code: 'TEST_JOURNAL_FAILURE' });
+ assert.deepEqual(connection.rows, before);
+ assert.equal(connection.journals.length, 0);
+ assert.equal(dml(connection).length, 3);
+});
+
+test('MySQL JSON canonicalization and numeric conversion are read back into each journal entry', async () => {
+ const connection = new BatchConnection();
+ connection.reverseReadback = true;
+ for (const row of Object.values(connection.rows)) {
+ row.json_data = JSON.stringify({ url: row.url }).replaceAll('\":', '\": ');
+ row.counter = 1;
+ }
+ const changes = Object.values(connection.rows).map(row => ({
+ schema: 'app', table: 'infra_file', primaryKey: 'id', pkValue: row.id,
+ before: { json_data: row.json_data, counter: 1 },
+ after: { json_data: JSON.stringify({ url: `https://oss/${row.path}` }), counter: '2' }
+ }));
+ connection.normalize = (row, fields) => {
+ if (fields.includes('json_data')) row.json_data = JSON.stringify(JSON.parse(row.json_data)).replaceAll('\":', '\": ');
+ if (fields.includes('counter')) row.counter = Number(row.counter);
+ };
+ await commitBatch(connection, changes);
+ const snapshots = connection.journals.map(row => JSON.parse(row.after_json));
+ assert.equal(snapshots[0].json_data.value, '{"url": "https://oss/a.png"}');
+ assert.equal(snapshots[1].json_data.value, '{"url": "https://oss/b.png"}');
+ assert.equal(snapshots[0].counter.value, 2);
+ assert.equal(snapshots[1].counter.value, 2);
+});
+
+test('UPDATE warnings and incomplete readback each abort the entire batch', async () => {
+ for (const prepare of [
+ connection => { connection.warningStatus = 1; },
+ connection => { connection.omitReadback = true; }
+ ]) {
+ const connection = new BatchConnection();
+ const changes = changesFor(connection);
+ const before = clone(connection.rows);
+ prepare(connection);
+ await assert.rejects(commitBatch(connection, changes));
+ assert.deepEqual(connection.rows, before);
+ assert.equal(connection.journals.length, 0);
+ }
+});
+
+test('incomplete or warned batch journal inserts roll back both updated rows and inserted logs', async () => {
+ for (const prepare of [
+ connection => { connection.journalAffectedRows = 1; },
+ connection => { connection.journalWarningStatus = 1; }
+ ]) {
+ const connection = new BatchConnection();
+ const changes = changesFor(connection);
+ const before = clone(connection.rows);
+ prepare(connection);
+ await assert.rejects(commitBatch(connection, changes), /回滚日志/);
+ assert.deepEqual(connection.rows, before);
+ assert.equal(connection.journals.length, 0);
+ }
+});
+
+test('non-InnoDB targets are rejected before mutation', async () => {
+ const connection = new BatchConnection();
+ connection.engine = 'MyISAM';
+ await assert.rejects(commitBatch(connection, changesFor(connection)), /InnoDB/);
+ assert.equal(dml(connection).length, 0);
+});
diff --git a/script/migrate-db-files-to-oss/test/journal.test.mjs b/script/migrate-db-files-to-oss/test/journal.test.mjs
new file mode 100644
index 0000000..87bf350
--- /dev/null
+++ b/script/migrate-db-files-to-oss/test/journal.test.mjs
@@ -0,0 +1,238 @@
+import test from 'node:test';
+import assert from 'node:assert/strict';
+import { ensureJournal, applyChange, rollbackChanges } from '../lib/journal.mjs';
+
+const clone = value => Buffer.isBuffer(value) ? Buffer.from(value)
+ : Array.isArray(value) ? value.map(clone)
+ : value && typeof value === 'object' ? Object.fromEntries(Object.entries(value).map(([key, item]) => [key, clone(item)])) : value;
+const same = (a, b) => Buffer.isBuffer(a) && Buffer.isBuffer(b) ? a.equals(b) : a === b || (a !== null && b !== null && String(a) === String(b));
+const names = sql => [...sql.matchAll(/CAST\(`((?:``|[^`])+)` AS BINARY\)/g)].map(match => match[1].replaceAll('``', '`'));
+
+class MemoryConnection {
+ constructor() {
+ this.calls = [];
+ this.rows = { '9007199254740993': { id: '9007199254740993', avatar: 'https://old/a', nullable: null, deleted: Buffer.from([0]), update_time: 'unchanged', touched: 'unchanged' } };
+ this.journals = [];
+ this.nextId = 9007199254740993n;
+ this.engine = 'InnoDB';
+ this.failJournal = false;
+ this.normalizeUpdate = null;
+ this.warningStatus = 0;
+ }
+
+ async beginTransaction() { this.calls.push({ transaction: 'begin' }); this.saved = { rows: clone(this.rows), journals: clone(this.journals) }; }
+ async commit() { this.calls.push({ transaction: 'commit' }); this.saved = undefined; }
+ async rollback() { this.calls.push({ transaction: 'rollback' }); if (this.saved) { this.rows = this.saved.rows; this.journals = this.saved.journals; this.saved = undefined; } }
+
+ async execute(sql, params = []) {
+ this.calls.push({ sql, params });
+ if (sql.startsWith('CREATE TABLE')) return [{ affectedRows: 0 }];
+ if (sql.includes('FROM information_schema.COLUMNS')) {
+ return [[...['id', 'avatar', 'nullable', 'deleted', 'update_time', 'touched', 'json_data', 'counter'].map(name => ({ name, engine: this.engine, extra: ['update_time', 'touched'].includes(name) ? 'on update CURRENT_TIMESTAMP' : '' }))]];
+ }
+ if (sql.startsWith('INSERT INTO')) {
+ if (this.failJournal) throw Object.assign(new Error('journal unavailable'), { code: 'TEST_JOURNAL_FAILURE' });
+ const [run_id, schema_name, table_name, pk_name, pk_value, before_json, after_json] = params;
+ const id = (this.nextId++).toString();
+ this.journals.push({ id, run_id, schema_name, table_name, pk_name, pk_value, before_json, after_json, rolled_back: 0 });
+ return [{ insertId: id }];
+ }
+ if (sql.startsWith('SELECT CAST(id AS CHAR)')) {
+ const [runId, cursor] = params;
+ return [this.journals.filter(row => row.run_id === runId && row.rolled_back === 0 && (cursor === undefined || BigInt(row.id) < BigInt(cursor)))
+ .sort((a, b) => BigInt(a.id) > BigInt(b.id) ? -1 : 1).slice(0, 100).map(row => ({ id: row.id }))];
+ }
+ if (sql.startsWith('SELECT *')) return [[this.journals.find(row => row.id === params[0] && row.run_id === params[1])].filter(Boolean)];
+ if (sql.startsWith('SELECT 1 AS matched')) {
+ const row = this.rows[String(params[0])];
+ return [row && names(sql).every((name, index) => same(row[name], params[index + 1])) ? [{ matched: 1 }] : []];
+ }
+ if (sql.startsWith('SELECT `')) {
+ const row = this.rows[String(params[0])];
+ const selected = [...sql.split(' FROM ')[0].matchAll(/`((?:``|[^`])+)`/g)].map(match => match[1]);
+ return [row ? [Object.fromEntries(selected.map(name => [name, row[name]]))] : []];
+ }
+ if (sql.startsWith('UPDATE') && sql.includes('SET rolled_back = 1')) {
+ const row = this.journals.find(row => row.id === params[0] && row.run_id === params[1]);
+ if (row) row.rolled_back = 1;
+ return [{ affectedRows: row ? 1 : 0 }];
+ }
+ if (sql.startsWith('UPDATE')) {
+ const set = sql.split(' SET ')[1].split('WHERE')[0];
+ const changed = [...set.matchAll(/`((?:``|[^`])+)` = \?/g)].map(match => match[1]);
+ const row = this.rows[String(params[changed.length])];
+ if (!row || !names(sql).every((name, index) => same(row[name], params[changed.length + 1 + index]))) return [{ affectedRows: 0 }];
+ changed.forEach((name, index) => { row[name] = params[index]; });
+ if (this.normalizeUpdate) this.normalizeUpdate(row, changed);
+ if (!set.includes('`update_time` = `update_time`')) row.update_time = 'changed';
+ if (!set.includes('`touched` = `touched`')) row.touched = 'changed';
+ return [{ affectedRows: 1, warningStatus: this.warningStatus }];
+ }
+ throw new Error(`Unexpected test query: ${sql}`);
+ }
+}
+
+const change = (before = 'https://old/a', after = 'https://oss/a') => ({
+ schema: 'app', table: 'member_user', primaryKey: 'id', pkValue: '9007199254740993',
+ before: { avatar: before, deleted: Buffer.from([0]), nullable: null }, after: { avatar: after }
+});
+
+async function commitChange(connection, payload = change(), runId = 'run') {
+ await connection.beginTransaction();
+ try {
+ const result = await applyChange(connection, 'app', runId, payload);
+ await connection.commit();
+ return result;
+ } catch (error) { await connection.rollback(); throw error; }
+}
+
+test('journal schema uses InnoDB and independent run/item/change records', async () => {
+ const connection = new MemoryConnection();
+ await ensureJournal(connection, 'app');
+ const creates = connection.calls.filter(call => call.sql?.startsWith('CREATE TABLE'));
+ assert.equal(creates.length, 3);
+ assert.ok(creates.every(call => call.sql.includes('ENGINE=InnoDB')));
+ assert.match(creates[1].sql, /PRIMARY KEY \(run_id, item_key\)/);
+ assert.match(creates[2].sql, /KEY ix_run_id_id \(run_id, id\)/);
+});
+
+test('apply logs original values with binary null-safe guards and lossless bigint IDs', async () => {
+ const connection = new MemoryConnection();
+ const result = await commitChange(connection);
+ assert.equal(result.journalId, '9007199254740993');
+ assert.equal(connection.rows['9007199254740993'].avatar, 'https://oss/a');
+ assert.equal(connection.rows['9007199254740993'].update_time, 'unchanged');
+ assert.equal(connection.rows['9007199254740993'].touched, 'unchanged');
+ const update = connection.calls.find(call => call.sql?.startsWith('UPDATE'));
+ assert.match(update.sql, /CAST\(`avatar` AS BINARY\) <=> CAST\(\? AS BINARY\)/);
+ assert.match(update.sql, /CAST\(`nullable` AS BINARY\) <=> CAST\(\? AS BINARY\)/);
+ assert.equal(update.params[1], '9007199254740993');
+ assert.deepEqual(update.params[3], Buffer.from([0]));
+ assert.equal(update.params[4], null);
+ assert.equal(connection.journals.length, 1);
+});
+
+test('no-op changes do not update rows or create logs', async () => {
+ const connection = new MemoryConnection();
+ assert.deepEqual(await commitChange(connection, change('https://old/a', 'https://old/a')), { changed: false });
+ assert.equal(connection.journals.length, 0);
+ assert.equal(connection.calls.filter(call => call.sql?.startsWith('UPDATE')).length, 0);
+});
+
+test('primary-key changes and missing snapshots are rejected before mutation', async () => {
+ const connection = new MemoryConnection();
+ await assert.rejects(commitChange(connection, { ...change(), after: { id: '1' } }), /不能修改主键/);
+ await assert.rejects(commitChange(connection, { ...change(), before: { deleted: Buffer.from([0]) } }), /缺少原值快照/);
+ assert.equal(connection.journals.length, 0);
+});
+
+test('concurrent edits including case-only changes fail CAS and preserve the row', async () => {
+ const connection = new MemoryConnection();
+ connection.rows['9007199254740993'].avatar = 'https://old/A';
+ await assert.rejects(commitChange(connection), { code: 'MIGRATION_CAS_CONFLICT' });
+ assert.equal(connection.rows['9007199254740993'].avatar, 'https://old/A');
+ assert.equal(connection.journals.length, 0);
+});
+
+test('a journal INSERT failure is rolled back with the row by the caller transaction', async () => {
+ const connection = new MemoryConnection();
+ connection.failJournal = true;
+ await assert.rejects(commitChange(connection), { code: 'TEST_JOURNAL_FAILURE' });
+ assert.equal(connection.rows['9007199254740993'].avatar, 'https://old/a');
+ assert.equal(connection.journals.length, 0);
+ assert.equal(connection.calls.at(-1).transaction, 'rollback');
+});
+
+test('journal reads back native JSON canonicalization and numeric conversions before recording rollback values', async () => {
+ const connection = new MemoryConnection();
+ connection.rows['9007199254740993'].json_data = '{"avatar": "https://old/a"}';
+ connection.rows['9007199254740993'].counter = 1;
+ connection.normalizeUpdate = (row, changed) => {
+ if (changed.includes('json_data')) row.json_data = JSON.stringify(JSON.parse(row.json_data)).replaceAll('\":', '\": ');
+ if (changed.includes('counter')) row.counter = Number(row.counter);
+ };
+ await commitChange(connection, {
+ ...change(),
+ before: { json_data: '{"avatar": "https://old/a"}', counter: 1 },
+ after: { json_data: '{"avatar":"https://oss/a"}', counter: '2' }
+ });
+ const after = JSON.parse(connection.journals[0].after_json);
+ assert.equal(after.json_data.value, '{"avatar": "https://oss/a"}');
+ assert.equal(after.counter.value, 2);
+ const result = await rollbackChanges(connection, 'app', 'run');
+ assert.deepEqual(result, { restored: 1, alreadyRestored: 0, conflicts: [] });
+ assert.equal(connection.rows['9007199254740993'].json_data, '{"avatar": "https://old/a"}');
+ assert.equal(connection.rows['9007199254740993'].counter, 1);
+});
+
+test('silent SQL truncation warnings abort the transaction before recording success', async () => {
+ const connection = new MemoryConnection();
+ connection.warningStatus = 1;
+ await assert.rejects(commitChange(connection), /数据库更新产生警告/);
+ assert.equal(connection.rows['9007199254740993'].avatar, 'https://old/a');
+ assert.equal(connection.journals.length, 0);
+});
+
+test('rollback restores consecutive edits in reverse journal order without losing bigint precision', async () => {
+ const connection = new MemoryConnection();
+ await commitChange(connection);
+ await commitChange(connection, change('https://oss/a', 'https://oss/b'));
+ const result = await rollbackChanges(connection, 'app', 'run');
+ assert.deepEqual(result, { restored: 2, alreadyRestored: 0, conflicts: [] });
+ assert.equal(connection.rows['9007199254740993'].avatar, 'https://old/a');
+ assert.equal(connection.rows['9007199254740993'].update_time, 'unchanged');
+ const reads = connection.calls.filter(call => call.sql?.startsWith('SELECT *'));
+ assert.deepEqual(reads.map(call => call.params[0]), ['9007199254740994', '9007199254740993']);
+ assert.ok(connection.journals.every(row => row.rolled_back === 1));
+ assert.deepEqual(await rollbackChanges(connection, 'app', 'run'), { restored: 0, alreadyRestored: 0, conflicts: [] });
+});
+
+test('rollback detects an already-restored row and only marks the journal', async () => {
+ const connection = new MemoryConnection();
+ await commitChange(connection);
+ connection.rows['9007199254740993'].avatar = 'https://old/a';
+ const result = await rollbackChanges(connection, 'app', 'run');
+ assert.deepEqual(result, { restored: 0, alreadyRestored: 1, conflicts: [] });
+ assert.equal(connection.journals[0].rolled_back, 1);
+});
+
+test('rollback SQL warnings do not mark a truncated restoration as complete', async () => {
+ const connection = new MemoryConnection();
+ await commitChange(connection);
+ connection.warningStatus = 1;
+ await assert.rejects(rollbackChanges(connection, 'app', 'run'), /数据库回滚产生警告/);
+ assert.equal(connection.rows['9007199254740993'].avatar, 'https://oss/a');
+ assert.equal(connection.journals[0].rolled_back, 0);
+});
+
+test('rollback leaves later user edits and deletion guards unchanged', async () => {
+ for (const update of [{ avatar: 'https://user/new' }, { deleted: Buffer.from([1]) }]) {
+ const connection = new MemoryConnection();
+ await commitChange(connection);
+ Object.assign(connection.rows['9007199254740993'], update);
+ const original = clone(connection.rows['9007199254740993']);
+ const result = await rollbackChanges(connection, 'app', 'run');
+ assert.equal(result.restored, 0);
+ assert.equal(result.conflicts.length, 1);
+ assert.equal(result.conflicts[0].pkValue, '9007199254740993');
+ assert.deepEqual(connection.rows['9007199254740993'], original);
+ assert.equal(connection.journals[0].rolled_back, 0);
+ }
+});
+
+test('rollback does not resurrect missing rows or touch another run', async () => {
+ const connection = new MemoryConnection();
+ await commitChange(connection);
+ delete connection.rows['9007199254740993'];
+ assert.deepEqual(await rollbackChanges(connection, 'app', 'another-run'), { restored: 0, alreadyRestored: 0, conflicts: [] });
+ const result = await rollbackChanges(connection, 'app', 'run');
+ assert.equal(result.conflicts.length, 1);
+ assert.equal(Object.keys(connection.rows).length, 0);
+});
+
+test('non-transactional tables are rejected before any update', async () => {
+ const connection = new MemoryConnection();
+ connection.engine = 'MyISAM';
+ await assert.rejects(commitChange(connection), /必须为 InnoDB/);
+ assert.equal(connection.calls.filter(call => call.sql?.startsWith('UPDATE')).length, 0);
+});
diff --git a/script/migrate-db-files-to-oss/test/progress.test.mjs b/script/migrate-db-files-to-oss/test/progress.test.mjs
new file mode 100644
index 0000000..b75c561
--- /dev/null
+++ b/script/migrate-db-files-to-oss/test/progress.test.mjs
@@ -0,0 +1,142 @@
+import test from 'node:test';
+import assert from 'node:assert/strict';
+import { createProgress } from '../lib/progress.mjs';
+
+function fixture(t, intervalMs = 5000) {
+ t.mock.timers.enable({ apis: ['setInterval'] });
+ let time = 0;
+ const lines = [];
+ const progress = createProgress({ log: line => lines.push(line), intervalMs, now: () => time });
+ t.after(() => progress.close());
+ return {
+ progress, lines,
+ setTime(value) { time = value; },
+ tick(value) { time += value; t.mock.timers.tick(value); },
+ };
+}
+
+test('construction stays silent until a stage is available', t => {
+ const { progress, lines, tick } = fixture(t);
+ tick(15000);
+ assert.deepEqual(lines, []);
+ progress.stage('正在等待数据库连接');
+ assert.deepEqual(lines, ['正在等待数据库连接（已用时 0 秒）']);
+});
+
+test('the timer reports a waiting stage at the configured interval', t => {
+ const { progress, lines, tick } = fixture(t);
+ progress.stage('正在统计历史图片');
+ tick(5000);
+ tick(5000);
+ assert.deepEqual(lines, [
+ '正在统计历史图片（已用时 0 秒）',
+ '正在统计历史图片（已用时 5 秒）',
+ '正在统计历史图片（已用时 10 秒）',
+ ]);
+});
+
+test('frequent updates preserve the newest message without printing every record', t => {
+ const { progress, lines, setTime } = fixture(t);
+ progress.stage('正在扫描引用');
+ for (let count = 1; count <= 1000; count++) {
+ setTime(count * 4);
+ progress.update('正在扫描引用，已检查 ' + count + ' 行');
+ }
+ assert.equal(lines.length, 1);
+ setTime(5000);
+ progress.update('正在扫描引用，已检查 1200 行');
+ assert.equal(lines.at(-1), '正在扫描引用，已检查 1200 行（已用时 5 秒）');
+ assert.equal(lines.length, 2);
+});
+
+test('timer output uses the last update message without resetting stage elapsed time', t => {
+ const { progress, lines, tick } = fixture(t);
+ progress.stage('正在扫描引用');
+ tick(2000);
+ progress.update('正在扫描引用，已检查 100 行');
+ tick(3000);
+ assert.deepEqual(lines, [
+ '正在扫描引用（已用时 0 秒）',
+ '正在扫描引用，已检查 100 行（已用时 5 秒）',
+ ]);
+});
+
+test('synchronous updates emit progress even when interval callbacks cannot run', t => {
+ const { progress, lines, setTime } = fixture(t);
+ progress.stage('正在处理清单');
+ setTime(4999);
+ progress.update('正在处理清单 1');
+ assert.equal(lines.length, 1);
+ setTime(5000);
+ progress.update('正在处理清单 2');
+ setTime(10000);
+ progress.update('正在处理清单 3');
+ assert.deepEqual(lines, [
+ '正在处理清单（已用时 0 秒）',
+ '正在处理清单 2（已用时 5 秒）',
+ '正在处理清单 3（已用时 10 秒）',
+ ]);
+});
+
+test('a timer callback immediately after an update does not duplicate output', t => {
+ const { progress, lines, setTime } = fixture(t);
+ progress.stage('正在上传');
+ setTime(5000);
+ progress.update('正在上传，已处理 50 张');
+ t.mock.timers.tick(5000);
+ assert.equal(lines.length, 2);
+});
+
+test('changing stage prints immediately and restarts its elapsed time', t => {
+ const { progress, lines, tick } = fixture(t);
+ progress.stage('正在统计');
+ tick(5000);
+ progress.stage('正在上传');
+ assert.equal(lines.at(-1), '正在上传（已用时 0 秒）');
+ tick(5000);
+ assert.equal(lines.at(-1), '正在上传（已用时 5 秒）');
+});
+
+test('close is idempotent and prevents timer, stage and update output', t => {
+ const { progress, lines, tick } = fixture(t);
+ progress.stage('正在扫描');
+ progress.close();
+ progress.close();
+ tick(15000);
+ progress.stage('已关闭');
+ progress.update('不应输出');
+ assert.deepEqual(lines, ['正在扫描（已用时 0 秒）']);
+});
+
+test('an update can initialize progress when no explicit stage was set', t => {
+ const { progress, lines, tick } = fixture(t);
+ progress.update('正在处理');
+ tick(5000);
+ assert.deepEqual(lines, ['正在处理（已用时 0 秒）', '正在处理（已用时 5 秒）']);
+});
+
+test('the interval is unrefed and its exact handle is cleared', t => {
+ let callback;
+ const handle = { unref: t.mock.fn() };
+ const interval = t.mock.method(globalThis, 'setInterval', (fn, ms) => {
+ callback = fn;
+ assert.equal(ms, 5000);
+ return handle;
+ });
+ const clear = t.mock.method(globalThis, 'clearInterval', () => {});
+ const log = t.mock.fn();
+ const progress = createProgress({ log, now: () => 0 });
+ assert.equal(interval.mock.callCount(), 1);
+ assert.equal(handle.unref.mock.callCount(), 1);
+ assert.equal(log.mock.callCount(), 0);
+ progress.close();
+ assert.deepEqual(clear.mock.calls[0].arguments, [handle]);
+ callback();
+ assert.equal(log.mock.callCount(), 0);
+});
+
+test('invalid intervals fail before starting a timer', () => {
+ for (const intervalMs of [0, -1, 1.5, NaN, Infinity]) {
+ assert.throws(() => createProgress({ intervalMs }), /正整数/);
+ }
+});
diff --git a/script/migrate-db-files-to-oss/test/project-config.test.mjs b/script/migrate-db-files-to-oss/test/project-config.test.mjs
new file mode 100644
index 0000000..760edf7
--- /dev/null
+++ b/script/migrate-db-files-to-oss/test/project-config.test.mjs
@@ -0,0 +1,309 @@
+import test from 'node:test';
+import assert from 'node:assert/strict';
+import { mkdtemp, mkdir, writeFile, rm } from 'node:fs/promises';
+import { tmpdir } from 'node:os';
+import { join, resolve } from 'node:path';
+import { stringify } from 'yaml';
+import { loadProjectConfig } from '../lib/project-config.mjs';
+
+const defaultBase = () => ({ spring: { profiles: { active: 'dev' } } });
+const datasource = overrides => ({
+ url: 'jdbc:mysql://db.example.test:3307/fixture_db?useSSL=false&serverTimezone=Asia%2FShanghai',
+ username: 'fixture-user', password: 'fixture-password', ...overrides,
+});
+const profileDocument = (overrides = {}) => ({
+ spring: { datasource: { dynamic: { datasource: { master: datasource(overrides) } } } },
+ server: { port: 48083 },
+});
+const documents = value => (Array.isArray(value) ? value : [value]).map(document => stringify(document)).join('---\n');
+
+async function fixture(t, { base = defaultBase(), profiles = { dev: profileDocument() } } = {}) {
+ const root = await mkdtemp(join(tmpdir(), 'oss-project-config-test-'));
+ t.after(() => rm(root, { recursive: true, force: true }));
+ const resources = join(root, 'yudao-server/src/main/resources');
+ await mkdir(resources, { recursive: true });
+ await writeFile(join(resources, 'application.yaml'), typeof base === 'string' ? base : documents(base));
+ for (const [name, content] of Object.entries(profiles)) {
+ await writeFile(join(resources, 'application-' + name + '.yaml'), typeof content === 'string' ? content : documents(content));
+ }
+ return root;
+}
+
+test('default profile, master credentials and actual server port come from the project YAML', async t => {
+ const projectRoot = await fixture(t);
+ const result = await loadProjectConfig({ projectRoot, env: {} });
+ assert.equal(result.profile, 'dev');
+ assert.deepEqual(result.mysql, {
+ host: 'db.example.test', port: 3307, database: 'fixture_db',
+ user: 'fixture-user', password: 'fixture-password',
+ });
+ assert.equal(result.downloadBaseUrl, 'http://127.0.0.1:48083/admin-api/infra/file');
+ assert.deepEqual(result.sourceFiles, [
+ resolve(projectRoot, 'yudao-server/src/main/resources/application.yaml'),
+ resolve(projectRoot, 'yudao-server/src/main/resources/application-dev.yaml'),
+ ]);
+});
+
+test('explicit profile overrides environment, which overrides the base active profile', async t => {
+ const projectRoot = await fixture(t, { profiles: {
+ dev: profileDocument({ url: 'jdbc:mysql://dev.example.test/dev_db' }),
+ local: profileDocument({ url: 'jdbc:mysql://local.example.test/local_db' }),
+ prod: profileDocument({ url: 'jdbc:mysql://prod.example.test/prod_db' }),
+ } });
+ const explicit = await loadProjectConfig({ projectRoot, profile: 'local', env: { SPRING_PROFILES_ACTIVE: 'prod' } });
+ const environment = await loadProjectConfig({ projectRoot, env: { SPRING_PROFILES_ACTIVE: 'prod' } });
+ const base = await loadProjectConfig({ projectRoot, env: {} });
+ assert.equal(explicit.mysql.database, 'local_db');
+ assert.equal(environment.mysql.database, 'prod_db');
+ assert.equal(base.mysql.database, 'dev_db');
+});
+
+test('single-profile selection resolves a base property or environment placeholder with a default', async t => {
+ const projectRoot = await fixture(t, {
+ base: { selection: { profile: '${APP_PROFILE:dev}' }, spring: { profiles: { active: '${selection.profile}' } } },
+ profiles: { dev: profileDocument(), local: profileDocument() },
+ });
+ assert.equal((await loadProjectConfig({ projectRoot, env: {} })).profile, 'dev');
+ assert.equal((await loadProjectConfig({ projectRoot, env: { APP_PROFILE: 'local' } })).profile, 'local');
+});
+
+test('explicit profile works even when the base active profile is absent or unusable', async t => {
+ const projectRoot = await fixture(t, { base: { spring: { profiles: { active: ['unused', 'group'] } } } });
+ const result = await loadProjectConfig({ projectRoot, profile: 'dev', env: {} });
+ assert.equal(result.profile, 'dev');
+});
+
+test('multiple, missing and path-traversal profiles fail with actionable selection errors', async t => {
+ for (const active of [['dev', 'prod'], 'dev,prod', 'dev prod', '', '../prod', null, undefined]) {
+ const projectRoot = await fixture(t, { base: { spring: { profiles: { active } } } });
+ await assert.rejects(loadProjectConfig({ projectRoot, env: {} }), /单个 profile.*--profile/);
+ }
+ const projectRoot = await fixture(t);
+ await assert.rejects(loadProjectConfig({ projectRoot, env: { SPRING_PROFILES_ACTIVE: 'dev,prod' } }), /单个 profile/);
+ await assert.rejects(loadProjectConfig({ projectRoot, profile: '../dev', env: {} }), /单个 profile/);
+});
+
+test('multiple unconditional documents merge recursively across base and selected profile', async t => {
+ const projectRoot = await fixture(t, {
+ base: [
+ { spring: { profiles: { active: 'dev' }, datasource: { dynamic: { primary: 'main', datasource: { main: { username: 'base-user', password: 'base-password' } } } } } },
+ { spring: { datasource: { dynamic: { datasource: { main: { url: 'jdbc:mysql://base.example.test/base_db' } } } } }, server: { port: 8081 } },
+ ],
+ profiles: { dev: [
+ { spring: { datasource: { dynamic: { datasource: { main: { url: 'jdbc:mysql://selected.example.test:3308/selected_db' } } } } } },
+ { spring: { datasource: { dynamic: { datasource: { main: { password: 'selected-password' } } } } }, server: { port: 48083 } },
+ ] },
+ });
+ const result = await loadProjectConfig({ projectRoot, env: {} });
+ assert.deepEqual(result.mysql, { host: 'selected.example.test', port: 3308, database: 'selected_db', user: 'base-user', password: 'selected-password' });
+ assert.equal(result.downloadBaseUrl, 'http://127.0.0.1:48083/admin-api/infra/file');
+});
+
+test('environment directly overrides master URL, username and password', async t => {
+ const projectRoot = await fixture(t);
+ const result = await loadProjectConfig({ projectRoot, env: {
+ SPRING_DATASOURCE_DYNAMIC_DATASOURCE_MASTER_URL: 'jdbc:mysql://override.example.test:3310/env_db',
+ SPRING_DATASOURCE_DYNAMIC_DATASOURCE_MASTER_USERNAME: 'env-user',
+ SPRING_DATASOURCE_DYNAMIC_DATASOURCE_MASTER_PASSWORD: ' env:pass#word ',
+ } });
+ assert.deepEqual(result.mysql, { host: 'override.example.test', port: 3310, database: 'env_db', user: 'env-user', password: ' env:pass#word ' });
+});
+
+test('dynamic primary environment override selects the named source instead of master or slave', async t => {
+ const document = profileDocument();
+ document.spring.datasource.dynamic.datasource.reporting = datasource({ url: 'jdbc:mysql://reporting.example.test/reporting_db' });
+ document.spring.datasource.dynamic.datasource.slave = datasource({ url: 'jdbc:mysql://slave.example.test/slave_db' });
+ const projectRoot = await fixture(t, { profiles: { dev: document } });
+ const result = await loadProjectConfig({ projectRoot, env: {
+ SPRING_DATASOURCE_DYNAMIC_PRIMARY: 'reporting',
+ SPRING_DATASOURCE_DYNAMIC_DATASOURCE_REPORTING_PASSWORD: 'reporting-env-password',
+ } });
+ assert.equal(result.mysql.database, 'reporting_db');
+ assert.equal(result.mysql.password, 'reporting-env-password');
+});
+
+test('required fields support property references and nested environment defaults', async t => {
+ const document = profileDocument({
+ url: 'jdbc:mysql://${DB_HOST:${database.host}}:${DB_PORT:${database.port:3306}}/${database.name}',
+ username: '${DB_USER:${database.user}}',
+ password: '${DB_PASSWORD:${database.password}}',
+ });
+ document.database = { host: 'property.example.test', name: 'property_db', user: 'property-user', password: 'property-password' };
+ const projectRoot = await fixture(t, { profiles: { dev: document } });
+ const base = await loadProjectConfig({ projectRoot, env: {} });
+ assert.deepEqual(base.mysql, { host: 'property.example.test', port: 3306, database: 'property_db', user: 'property-user', password: 'property-password' });
+ const overridden = await loadProjectConfig({ projectRoot, env: { DB_HOST: 'env.example.test', DB_PORT: '3309', DB_PASSWORD: 'env-password' } });
+ assert.equal(overridden.mysql.host, 'env.example.test');
+ assert.equal(overridden.mysql.port, 3309);
+ assert.equal(overridden.mysql.password, 'env-password');
+});
+
+test('unrelated unresolved placeholders, service credentials and cycles are never resolved', async t => {
+ const document = profileDocument();
+ document.unrelated = { secret: '${MISSING_SERVICE_SECRET}', a: '${unrelated.b}', b: '${unrelated.a}', encrypted: 'ENC(unrelated-service-secret)' };
+ const projectRoot = await fixture(t, { profiles: { dev: document } });
+ assert.equal((await loadProjectConfig({ projectRoot, env: {} })).mysql.database, 'fixture_db');
+});
+
+test('HTTP context path, admin prefix and SSL settings determine the download URL', async t => {
+ const document = profileDocument();
+ document.server = { port: 8443, servlet: { contextPath: '/backend/' }, ssl: { enabled: true } };
+ document.yudao = { web: { adminApi: { prefix: '/management/' } } };
+ const projectRoot = await fixture(t, { profiles: { dev: document } });
+ const result = await loadProjectConfig({ projectRoot, env: {} });
+ assert.equal(result.downloadBaseUrl, 'https://127.0.0.1:8443/backend/management/infra/file');
+ const overridden = await loadProjectConfig({ projectRoot, env: {
+ SERVER_PORT: '9080', SERVER_SERVLET_CONTEXT_PATH: '/overridden',
+ YUDAO_WEB_ADMIN_API_PREFIX: '/admin', SERVER_SSL_ENABLED: 'false',
+ } });
+ assert.equal(overridden.downloadBaseUrl, 'http://127.0.0.1:9080/overridden/admin/infra/file');
+});
+
+test('Spring defaults apply to missing primary, HTTP port and API prefix', async t => {
+ const document = profileDocument();
+ delete document.server;
+ const projectRoot = await fixture(t, { profiles: { dev: document } });
+ assert.equal((await loadProjectConfig({ projectRoot, env: {} })).downloadBaseUrl, 'http://127.0.0.1:8080/admin-api/infra/file');
+});
+
+test('an explicit root context path does not produce duplicate slashes', async t => {
+ const document = profileDocument();
+ document.server.servlet = { 'context-path': '/' };
+ document.yudao = { web: { 'admin-api': { prefix: '/' } } };
+ const projectRoot = await fixture(t, { profiles: { dev: document } });
+ assert.equal((await loadProjectConfig({ projectRoot, env: {} })).downloadBaseUrl, 'http://127.0.0.1:48083/infra/file');
+});
+
+test('single-host MySQL URL supports omitted ports and IPv6 without changing database names', async t => {
+ for (const [url, expected] of [
+ ['jdbc:mysql://localhost/test_db', { host: 'localhost', port: 3306, database: 'test_db' }],
+ ['jdbc:mysql://[::1]:3309/test_db', { host: '::1', port: 3309, database: 'test_db' }],
+ ['jdbc:mysql://db.example.test/test%20db', { host: 'db.example.test', port: 3306, database: 'test db' }],
+ ]) {
+ const projectRoot = await fixture(t, { profiles: { dev: profileDocument({ url }) } });
+ const result = await loadProjectConfig({ projectRoot, env: {} });
+ assert.deepEqual({ host: result.mysql.host, port: result.mysql.port, database: result.mysql.database }, expected);
+ }
+});
+
+test('ambiguous, unsupported and incomplete JDBC URLs are rejected without echoing their values', async t => {
+ for (const url of [
+ 'jdbc:mysql:loadbalance://first,second/db',
+ 'jdbc:mysql://first,second/db',
+ 'jdbc:mysql://first:3306,second:3306/db',
+ 'jdbc:postgresql://secret-host/secret-database',
+ 'jdbc:mysql://db.example.test',
+ 'jdbc:mysql://db.example.test/a/b',
+ 'jdbc:mysql://db.example.test:65536/db',
+ 'jdbc:mysql://db.example.test:0/db',
+ 'jdbc:mysql://user:sensitive-url-password@db.example.test/db',
+ 'jdbc:mysql://db.example.test/a%2Fb',
+ ]) {
+ const projectRoot = await fixture(t, { profiles: { dev: profileDocument({ url }) } });
+ await assert.rejects(loadProjectConfig({ projectRoot, env: {} }), error => {
+ assert.match(error.message, /MySQL|jdbc:mysql/);
+ assert.ok(!error.message.includes(url));
+ assert.ok(!error.message.includes('sensitive-url-password'));
+ return true;
+ });
+ }
+});
+
+test('missing or still-encrypted passwords are rejected, with direct environment override available', async t => {
+ for (const password of [null, undefined, '', 'ENC(sensitive-encrypted-password)']) {
+ const projectRoot = await fixture(t, { profiles: { dev: profileDocument({ password }) } });
+ await assert.rejects(loadProjectConfig({ projectRoot, env: {} }), error => {
+ assert.match(error.message, /缺少|缺失|ENC/);
+ assert.ok(!error.message.includes('sensitive-encrypted-password'));
+ return true;
+ });
+ assert.equal((await loadProjectConfig({ projectRoot, env: { SPRING_DATASOURCE_DYNAMIC_DATASOURCE_MASTER_PASSWORD: 'provided-password' } })).mysql.password, 'provided-password');
+ }
+});
+
+test('required unresolved, malformed or cyclic placeholders stop safely without revealing secrets', async t => {
+ for (const password of [
+ 'sensitive-prefix-${MISSING_PASSWORD}',
+ 'sensitive-prefix-${MISSING_PASSWORD',
+ '${spring.datasource.dynamic.datasource.master.password}',
+ ]) {
+ const projectRoot = await fixture(t, { profiles: { dev: profileDocument({ password }) } });
+ await assert.rejects(loadProjectConfig({ projectRoot, env: {} }), error => {
+ assert.match(error.message, /占位符/);
+ assert.ok(!error.message.includes('sensitive-prefix'));
+ return true;
+ });
+ }
+});
+
+test('numeric YAML scalar credentials retain bigint precision', async t => {
+ const document = profileDocument({ password: 900719925474099312345n });
+ const projectRoot = await fixture(t, { profiles: { dev: document } });
+ assert.equal((await loadProjectConfig({ projectRoot, env: {} })).mysql.password, '900719925474099312345');
+});
+
+test('config imports, included profiles and a selected profile group require explicit configuration', async t => {
+ for (const base of [
+ { spring: { profiles: { active: 'dev' }, config: { import: 'optional:file:sensitive-location' } } },
+ { spring: { profiles: { active: 'dev', include: ['common'] } } },
+ { spring: { profiles: { active: 'dev', group: { dev: ['common', 'database'] } } } },
+ ]) {
+ const projectRoot = await fixture(t, { base });
+ await assert.rejects(loadProjectConfig({ projectRoot, env: {} }), error => {
+ assert.match(error.message, /import|include|配置组/);
+ assert.ok(!error.message.includes('sensitive-location'));
+ return true;
+ });
+ }
+ const projectRoot = await fixture(t);
+ await assert.rejects(loadProjectConfig({ projectRoot, env: { SPRING_CONFIG_IMPORT: 'optional:file:sensitive-location' } }), /import/);
+ await assert.rejects(loadProjectConfig({ projectRoot, env: { SPRING_PROFILES_GROUP_DEV: 'common,database' } }), /配置组/);
+});
+
+test('an unrelated profile group does not affect an explicitly selected concrete profile', async t => {
+ const projectRoot = await fixture(t, { base: { spring: { profiles: { active: 'dev', group: { production: ['prod', 'shared'] } } } } });
+ assert.equal((await loadProjectConfig({ projectRoot, env: {} })).profile, 'dev');
+});
+
+test('conditional YAML documents and active-profile redefinition are rejected', async t => {
+ for (const content of [
+ [profileDocument(), { spring: { config: { activate: { 'on-profile': 'dev' } } }, server: { port: 8080 } }],
+ [profileDocument(), { spring: { profiles: 'dev' }, server: { port: 8080 } }],
+ [profileDocument(), { spring: { profiles: { active: 'prod' } } }],
+ ]) {
+ const projectRoot = await fixture(t, { profiles: { dev: content } });
+ await assert.rejects(loadProjectConfig({ projectRoot, env: {} }), /条件|歧义/);
+ }
+});
+
+test('missing files and malformed YAML errors do not expose parser source or secrets', async t => {
+ const projectRoot = await fixture(t, { base: 'spring: [sensitive-password' });
+ await assert.rejects(loadProjectConfig({ projectRoot, env: {} }), error => {
+ assert.match(error.message, /YAML/);
+ assert.ok(!error.message.includes('sensitive-password'));
+ return true;
+ });
+ const validRoot = await fixture(t);
+ await assert.rejects(loadProjectConfig({ projectRoot: validRoot, profile: 'missing', env: {} }), /--profile/);
+ await assert.rejects(loadProjectConfig({ projectRoot: '', env: {} }), /项目根目录/);
+});
+
+test('duplicate keys inside one YAML document cannot silently choose credentials', async t => {
+ const projectRoot = await fixture(t, { base: 'spring:\n profiles:\n active: dev\nspring:\n password: sensitive-password\n' });
+ await assert.rejects(loadProjectConfig({ projectRoot, env: {} }), error => {
+ assert.match(error.message, /YAML/);
+ assert.ok(!error.message.includes('sensitive-password'));
+ return true;
+ });
+});
+
+test('invalid HTTP ports and route prefixes fail instead of generating unusable URLs', async t => {
+ const projectRoot = await fixture(t);
+ for (const env of [
+ { SERVER_PORT: '0' }, { SERVER_PORT: '65536' }, { SERVER_PORT: 'not-a-port' },
+ { SERVER_SERVLET_CONTEXT_PATH: 'missing-leading-slash' },
+ { YUDAO_WEB_ADMIN_API_PREFIX: '/admin?query' },
+ { SERVER_SERVLET_CONTEXT_PATH: '/a/../b' },
+ { SERVER_SSL_ENABLED: 'unknown' },
+ ]) await assert.rejects(loadProjectConfig({ projectRoot, env }), /HTTP/);
+});
diff --git a/script/migrate-db-files-to-oss/test/references.test.mjs b/script/migrate-db-files-to-oss/test/references.test.mjs
new file mode 100644
index 0000000..15975f4
--- /dev/null
+++ b/script/migrate-db-files-to-oss/test/references.test.mjs
@@ -0,0 +1,171 @@
+import test from 'node:test';
+import assert from 'node:assert/strict';
+import { containsLegacyReference, discoverReferenceColumns, rewriteReferences } from '../lib/references.mjs';
+
+const oldUrl = 'https://old.example/admin-api/infra/file/1/get/a.png';
+const newUrl = 'https://bucket.example/migrated/a.png';
+const otherUrl = 'https://elsewhere.example/a.png';
+const replacements = new Map([[oldUrl, newUrl]]);
+
+test('replaces an exact URL once and never cascades a destination mapping', () => {
+ assert.deepEqual(rewriteReferences(oldUrl, new Map([[oldUrl, newUrl], [newUrl, otherUrl]])), {
+ value: newUrl, changed: true, replacements: 1
+ });
+ assert.deepEqual(rewriteReferences(null, replacements), { value: null, changed: false, replacements: 0 });
+});
+
+test('replaces comma lists, repeated values and rich text while preserving surrounding text', () => {
+ const input = `${oldUrl},${otherUrl}, ${oldUrl}\n
`;
+ const expected = `${newUrl},${otherUrl}, ${newUrl}\n
`;
+ assert.deepEqual(rewriteReferences(input, replacements), { value: expected, changed: true, replacements: 4 });
+ assert.equal(rewriteReferences(`[image](${oldUrl}),`, replacements).value, `[image](${newUrl}),`);
+});
+
+test('never matches URL prefixes, query extensions, fragments or external embedded URLs', () => {
+ const unchanged = [
+ `${oldUrl}.bak`, `${oldUrl}/thumbnail`, `${oldUrl}?width=200`, `${oldUrl}#preview`,
+ `${oldUrl}%20other`, `${oldUrl}&x=1`, `${oldUrl};v=1`, otherUrl,
+ `https://external.example/?redirect=${oldUrl}`,
+ `https://external.example/?redirect=x,${oldUrl}`,
+ `${oldUrl}?url=x,${oldUrl}`
+ ];
+ for (const value of unchanged) {
+ assert.deepEqual(rewriteReferences(value, replacements), { value, changed: false, replacements: 0 }, value);
+ }
+});
+
+test('only rewrites queries and fragments explicitly present in the dictionary', () => {
+ const url = `${oldUrl}?v=2#original`;
+ assert.equal(rewriteReferences(`
`, new Map([[url, newUrl]])).value, ``);
+ assert.equal(rewriteReferences(url, replacements).value, url);
+});
+
+test('keeps URL encoding literal and supports Unicode paths', () => {
+ const encoded = oldUrl.replace('a.png', '%E5%A4%B4%E5%83%8F%20a.png');
+ const unicode = oldUrl.replace('a.png', '头像.png');
+ const mapping = new Map([[encoded, newUrl], [unicode, `${newUrl}?kind=unicode`]]);
+ assert.equal(rewriteReferences(`
${unicode}`, mapping).replacements, 2);
+ assert.equal(rewriteReferences(encoded.replace('%20', ' '), mapping).changed, false);
+ assert.equal(rewriteReferences(encoded.toLowerCase(), mapping).changed, false);
+});
+
+test('JSON text retains formatting and untouched escapes; keys remain unchanged', () => {
+ const escapedUrl = oldUrl.replaceAll('/', '\\/');
+ const input = `{\n "${oldUrl}": "key stays", "avatar" : "${escapedUrl}",\n "nested": [{"pic":"${oldUrl}"}], "untouched":"\\u4e2d"\n}`;
+ const expected = `{\n "${oldUrl}": "key stays", "avatar" : "${newUrl}",\n "nested": [{"pic":"${newUrl}"}], "untouched":"\\u4e2d"\n}`;
+ assert.deepEqual(rewriteReferences(input, replacements), { value: expected, changed: true, replacements: 2 });
+ assert.equal(rewriteReferences(JSON.stringify(JSON.stringify({ avatar: oldUrl })), replacements).value,
+ JSON.stringify(JSON.stringify({ avatar: newUrl })));
+});
+
+test('parsed JSON is recursively rewritten without changing input or prototype', () => {
+ const input = JSON.parse(`{"__proto__":{"url":"${oldUrl}"},"photos":["${oldUrl}",null,4],"url":"${otherUrl}"}`);
+ const result = rewriteReferences(input, replacements);
+ assert.equal(result.replacements, 2);
+ assert.equal(input.photos[0], oldUrl);
+ assert.equal(result.value.photos[0], newUrl);
+ assert.equal(result.value.__proto__.url, newUrl);
+ assert.equal(Object.getPrototypeOf(result.value), Object.prototype);
+ assert.equal(rewriteReferences({ url: otherUrl }, replacements).changed, false);
+});
+
+test('malformed JSON safely uses literal URL replacement without changing escapes', () => {
+ const input = `{"avatar":"${oldUrl}",}`;
+ assert.equal(rewriteReferences(input, replacements).value, `{"avatar":"${newUrl}",}`);
+});
+
+test('remaining-reference detection finds Unicode-escaped JSON URLs with unknown queries', () => {
+ const withQuery = `${oldUrl}?unmapped=1#preview`;
+ const unicodeJson = JSON.stringify({ avatar: withQuery }).replaceAll('/', '\\u002f');
+ const rewritten = rewriteReferences(unicodeJson, replacements);
+ assert.equal(rewritten.changed, false);
+ assert.equal(containsLegacyReference(rewritten.value, ['1']), true);
+ assert.equal(containsLegacyReference(rewritten.value, ['2']), false);
+});
+
+test('remaining-reference detection recursively handles nested JSON strings and parsed values', () => {
+ const nested = JSON.stringify({ data: JSON.stringify({ photos: [`${oldUrl}?v=2`] }).replaceAll('/', '\\u002f') });
+ assert.equal(containsLegacyReference(nested, new Set([1])), true);
+ assert.equal(containsLegacyReference({ data: [null, { image: oldUrl }] }, [1]), true);
+ assert.equal(containsLegacyReference(JSON.stringify(JSON.stringify(`${oldUrl}#unknown`)), ['1']), true);
+ assert.equal(containsLegacyReference(rewriteReferences(JSON.stringify({ image: oldUrl }), replacements).value, [1]), false);
+});
+
+test('remaining-reference detection handles escaped plain text, exact config IDs, and excludes JSON keys', () => {
+ assert.equal(containsLegacyReference(`}?size=2)
`, [1]), true);
+ assert.equal(containsLegacyReference(`${oldUrl}?query=2`, [11]), false);
+ assert.equal(containsLegacyReference(oldUrl.replace('/file/1/', '/file/11/'), [1]), false);
+ assert.equal(containsLegacyReference(JSON.stringify({ [oldUrl]: 'not a URL value' }), [1]), false);
+ assert.equal(containsLegacyReference({ [oldUrl]: 'not a URL value' }, [1]), false);
+ assert.equal(containsLegacyReference(null, [1]), false);
+ assert.equal(containsLegacyReference(oldUrl, []), false);
+ const cycle = {};
+ cycle.self = cycle;
+ assert.equal(containsLegacyReference(cycle, [1]), false);
+});
+
+test('discovery binds schema names and excludes unsafe or non-content columns', async () => {
+ const row = (tableName, columnName, extra = {}) => ({
+ schemaName: 'app', tableName, columnName, dataType: 'varchar', extra: '',
+ generationExpression: '', primaryKey: 'id', primaryKeyCount: 1, ...extra
+ });
+ const rows = [
+ row('member_user', 'avatar'), row('member_user', 'id'),
+ row('member_user', 'generated', { generationExpression: 'avatar', extra: 'VIRTUAL GENERATED' }),
+ row('member_user', 'stored', { extra: 'STORED GENERATED' }),
+ row('member_user', 'seq', { extra: 'auto_increment' }),
+ row('member_user', 'defaulted', { extra: 'DEFAULT_GENERATED' }),
+ row('member_com_auth', 'auth_extra_json', { dataType: 'json' }),
+ row('infra_file', 'url'), row('infra_file_content', 'path'), row('infra_file_config', 'config'),
+ row('migration_files', 'old_url'), row('infra_file_migration_journal', 'old_url'),
+ row('infra_file_oss_migration_runs', 'settings'), row('infra_file_oss_migration_items', 'data'),
+ row('infra_file_oss_migration_changes', 'before_json'), row('INFRA_FILE_OSS_MIGRATION_CHANGES', 'after_json'),
+ row('no_key', 'a', { primaryKeyCount: null, primaryKey: null }),
+ row('no_key', 'b', { primaryKeyCount: null, primaryKey: null }),
+ row('composite', 'a', { primaryKeyCount: '2' }),
+ row('skip_me', 'a'), row('skip_qualified', 'a'),
+ ];
+ const schemas = ['app', 'odd` schema; DROP TABLE x'];
+ const connection = { async execute(sql, params) {
+ assert.deepEqual(params, [...schemas, ...schemas]);
+ assert.ok(!sql.includes(schemas[1]));
+ assert.match(sql, /t\.TABLE_TYPE = 'BASE TABLE'/);
+ assert.match(sql, /c\.DATA_TYPE IN \('char', 'varchar', 'tinytext', 'text', 'mediumtext', 'longtext', 'json'\)/);
+ return [rows];
+ } };
+ const result = await discoverReferenceColumns(connection, schemas, new Set(['skip_me', 'APP.skip_qualified']));
+ assert.deepEqual(result.columns, [
+ { schema: 'app', table: 'member_user', column: 'avatar', primaryKey: 'id', dataType: 'varchar' },
+ { schema: 'app', table: 'member_user', column: 'defaulted', primaryKey: 'id', dataType: 'varchar' },
+ { schema: 'app', table: 'member_com_auth', column: 'auth_extra_json', primaryKey: 'id', dataType: 'json' }
+ ]);
+ assert.equal(result.skipped.length, 13);
+ for (const table of ['infra_file_oss_migration_runs', 'infra_file_oss_migration_items', 'infra_file_oss_migration_changes', 'INFRA_FILE_OSS_MIGRATION_CHANGES']) {
+ assert.equal(result.skipped.find((item) => item.table === table).reason, 'excluded table');
+ assert.ok(!result.columns.some((item) => item.table === table));
+ }
+ assert.equal(result.skipped.find((item) => item.table === 'no_key').reason, 'no primary key');
+ assert.equal(result.skipped.find((item) => item.table === 'composite').reason, 'composite primary key');
+});
+
+test('discovery makes no connection for an empty schema list and validates names', async () => {
+ assert.deepEqual(await discoverReferenceColumns({}, []), { columns: [], skipped: [] });
+ await assert.rejects(discoverReferenceColumns({}, ['']), /schema names/);
+});
+
+test('discovery explicitly excludes known technical and audit logs while retaining business log tables', async () => {
+ const excluded = [
+ 'infra_api_access_log', 'infra_api_error_log', 'infra_job_log',
+ 'system_operate_log', 'system_login_log', 'SYSTEM_LOGIN_LOG',
+ ];
+ const retained = ['member_activity_log', 'chat_message_log', 'custom_audit_log', 'infra_api_access_log_archive'];
+ const rows = [...excluded, ...retained].flatMap(tableName => ['content', 'details'].map(columnName => ({
+ schemaName: 'app', tableName, columnName, dataType: 'text', extra: '',
+ generationExpression: '', primaryKey: 'id', primaryKeyCount: 1,
+ })));
+ const connection = { async execute() { return [rows]; } };
+ const result = await discoverReferenceColumns(connection, ['app']);
+ assert.deepEqual(result.skipped, excluded.map(table => ({ schema: 'app', table, reason: 'excluded table' })));
+ assert.deepEqual([...new Set(result.columns.map(column => column.table))], retained);
+ assert.equal(result.columns.length, retained.length * 2);
+});
diff --git a/script/migrate-db-files-to-oss/test/runtime-config.test.mjs b/script/migrate-db-files-to-oss/test/runtime-config.test.mjs
new file mode 100644
index 0000000..bd12a1a
--- /dev/null
+++ b/script/migrate-db-files-to-oss/test/runtime-config.test.mjs
@@ -0,0 +1,56 @@
+import test from 'node:test';
+import assert from 'node:assert/strict';
+import { mkdtemp, mkdir, readFile, writeFile, rm } from 'node:fs/promises';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import { normalizeConfig } from '../lib/core.mjs';
+import { parseArgs, prepareInput, persistGeneratedConfig } from '../migrate.mjs';
+
+test('zero arguments default to read-only plan, with optional profile or explicit actions', () => {
+ const args = parseArgs([]);
+ assert.equal(args.command, 'plan'); assert.ok(args.configPath.endsWith('config.local.json'));
+ assert.equal(parseArgs(['--profile', 'dev']).command, 'plan');
+ assert.equal(parseArgs(['init']).command, 'init');
+ assert.equal(parseArgs(['apply', '--limit', '50']).limit, 50);
+ assert.equal(parseArgs(['apply', '--run-id', 'new-batch']).runId, 'new-batch');
+ assert.equal(parseArgs(['--help']).help, true);
+ assert.throws(() => parseArgs(['plan', '--limit', '50']));
+ assert.throws(() => parseArgs(['delete']));
+});
+
+test('automatic configuration inherits project credentials without exporting a password variable', async () => {
+ const root = await mkdtemp(join(tmpdir(), 'oss-runtime-'));
+ try {
+ const resources = join(root, 'yudao-server/src/main/resources'); await mkdir(resources, { recursive: true });
+ await writeFile(join(resources, 'application.yaml'), 'spring:\n profiles:\n active: dev\n');
+ await writeFile(join(resources, 'application-dev.yaml'), `server:\n port: 48083\nspring:\n datasource:\n dynamic:\n primary: master\n datasource:\n master:\n url: jdbc:mysql://127.0.0.1:3306/fixture_db\n username: fixture_user\n password: fixture-secret\n`);
+ const args = { command: 'init', configPath: join(root, 'config.local.json') };
+ const prepared = await prepareInput(args, root);
+ assert.equal(prepared.projectManaged, true); assert.equal(prepared.exists, false);
+ assert.equal(prepared.input.mysql.database, 'fixture_db'); assert.equal(prepared.input.mysql.password, 'fixture-secret');
+ assert.equal(prepared.input.sourceMode, undefined); // defaults must not overwrite a resumed run
+ assert.equal(prepared.input._projectDownloadBaseUrl, 'http://127.0.0.1:48083/admin-api/infra/file');
+ const config = normalizeConfig({ ...prepared.input, sourceMode: 'database', sourceConfigIds: ['5'], targetConfigId: '30', runId: 'auto-test', _databaseIdentity: { server: 'test-uuid', schema: 'fixture_db' } });
+ await persistGeneratedConfig(config, args, prepared);
+ const text = await readFile(args.configPath, 'utf8');
+ assert.ok(!text.includes('fixture-secret')); assert.ok(!text.includes('"mysql"'));
+ const saved = JSON.parse(text); assert.equal(saved.runId, 'auto-test'); assert.equal(saved.projectProfile, 'dev'); assert.equal(saved.targetConfigId, '30');
+ const reloaded = await prepareInput(args, root);
+ assert.equal(reloaded.input.mysql.password, 'fixture-secret'); assert.equal(reloaded.input.runId, 'auto-test');
+ } finally { await rm(root, { recursive: true, force: true }); }
+});
+
+test('manual connection remains supported and is never overwritten with project settings', async () => {
+ const root = await mkdtemp(join(tmpdir(), 'oss-manual-'));
+ try {
+ const path = join(root, 'manual.json');
+ const raw = { mysql: { host: 'manual.example', database: 'manual_db', user: 'manual', password: 'manual-secret' }, runId: 'manual-run', sourceConfigIds: ['5'], targetConfigId: '30', sourceMode: 'database' };
+ await writeFile(path, JSON.stringify(raw));
+ const args = { command: 'init', configPath: path, explicitConfig: true };
+ const prepared = await prepareInput(args, root);
+ assert.equal(prepared.projectManaged, false); assert.equal(prepared.input.mysql.host, 'manual.example');
+ await persistGeneratedConfig(normalizeConfig(prepared.input), args, prepared);
+ assert.deepEqual(JSON.parse(await readFile(path, 'utf8')), raw);
+ await assert.rejects(prepareInput({ ...args, profile: 'prod' }, root), /不能同时使用/);
+ } finally { await rm(root, { recursive: true, force: true }); }
+});
diff --git a/script/migrate-db-files-to-oss/test/source-batch.test.mjs b/script/migrate-db-files-to-oss/test/source-batch.test.mjs
new file mode 100644
index 0000000..d2ec92f
--- /dev/null
+++ b/script/migrate-db-files-to-oss/test/source-batch.test.mjs
@@ -0,0 +1,139 @@
+import test from 'node:test';
+import assert from 'node:assert/strict';
+import { verifySourceBatch } from '../lib/source-batch.mjs';
+
+const config = { mysql: { database: 'source_batch_test' } };
+const first = { contentId: '9007199254740993', configId: '5', path: '相册/Résumé.PNG', size: 1234, sha256: 'a'.repeat(64) };
+const second = { contentId: '9007199254740995', configId: '9007199254740999', path: '头像.png', size: 4321, sha256: 'b'.repeat(64) };
+const latest = item => ({ id: item.contentId, expected_content_id: item.contentId });
+const locked = (...items) => items.map(item => ({ id: item.contentId }));
+const content = (item, overrides = {}) => ({ id: item.contentId, config_id: item.configId, path: item.path, size: String(item.size), sha256: item.sha256, ...overrides });
+const changed = error => error.code === 'MIGRATION_SOURCE_CHANGED';
+
+class FakeDb {
+ constructor(responses) {
+ this.responses = responses;
+ this.calls = [];
+ }
+ async execute(sql, params) {
+ assert.match(sql.trim(), /^SELECT\b/);
+ if (this.calls.length === 1) assert.doesNotMatch(sql, /FOR UPDATE|LOCK IN SHARE MODE/i);
+ else assert.match(sql, /FOR UPDATE$/);
+ assert.doesNotMatch(sql, /\b(?:START TRANSACTION|BEGIN|COMMIT|ROLLBACK)\b/i);
+ this.calls.push({ sql, params });
+ if (this.calls.length > this.responses.length) throw new Error('Unexpected database query');
+ const result = this.responses[this.calls.length - 1];
+ if (result instanceof Error) throw result;
+ return [result];
+ }
+}
+
+test('validates in three SELECTs with a primary-key-only range lock and a separate digest read', async () => {
+ const db = new FakeDb([locked(first, second), [latest(second), latest(first)], [content(first), content(second)]]);
+ await verifySourceBatch(db, config, [second, first]);
+ assert.equal(db.calls.length, 3);
+ const [range, metadata, hashes] = db.calls;
+ assert.match(range.sql, /^SELECT id FROM /);
+ assert.match(range.sql, /WHERE id>=\? ORDER BY id FOR UPDATE$/);
+ assert.doesNotMatch(range.sql, /\b(?:content|deleted|config_id|path|CASE|LIMIT)\b|SHA2|OCTET_LENGTH|id\s+IN\s*\(/i,
+ 'lock every row and gap above the minimum ID, including deleted and unrelated-path rows');
+ assert.deepEqual(range.params, [first.contentId]);
+ assert.match(metadata.sql, /^SELECT id,CASE WHEN config_id=\? AND path=\? THEN \?/);
+ assert.match(metadata.sql, /END AS expected_content_id/);
+ assert.match(metadata.sql, /WHERE deleted=0 AND id>=\? AND \(\(config_id=\? AND path=\? AND id>=\?\) OR/);
+ assert.match(metadata.sql, /ORDER BY id DESC$/);
+ assert.doesNotMatch(metadata.sql, /\bcontent\b|OCTET_LENGTH|SHA2|\bLIMIT\b|id\s+IN\s*\(/i,
+ 'metadata lookup must examine newer IDs without projecting BLOB expressions');
+ assert.deepEqual(metadata.params, [
+ second.configId, second.path, second.contentId,
+ first.configId, first.path, first.contentId,
+ first.contentId,
+ second.configId, second.path, second.contentId,
+ first.configId, first.path, first.contentId
+ ]);
+ assert.match(hashes.sql, /OCTET_LENGTH\(content\) AS size,LOWER\(SHA2\(content,256\)\) AS sha256/);
+ assert.match(hashes.sql, /WHERE deleted=0 AND id IN \(\?,\?\) FOR UPDATE$/);
+ assert.doesNotMatch(hashes.sql, /ORDER BY|\bCASE\b|\bJOIN\b|\bLIMIT\b/i);
+ assert.deepEqual(hashes.params, [second.contentId, first.contentId]);
+});
+
+test('uses server-provided grouping IDs and accepts collation-equivalent case/accent spellings', async () => {
+ const db = new FakeDb([locked(first), [latest(first)], [content(first, { path: '相册/RESUME.png' })]]);
+ await verifySourceBatch(db, config, [first]);
+ assert.deepEqual(db.calls[1].params, [first.configId, first.path, first.contentId, first.contentId, first.configId, first.path, first.contentId]);
+ assert.doesNotMatch(db.calls[1].sql, /BINARY\s+path|LOWER\s*\(path\)/i);
+});
+
+test('rejects a newer active version before performing the batch digest read', async () => {
+ const db = new FakeDb([locked(first, second), [
+ latest(second), latest(first), { id: '9007199254740994', expected_content_id: first.contentId }
+ ]]);
+ await assert.rejects(verifySourceBatch(db, config, [first, second]), changed);
+ assert.equal(db.calls.length, 2);
+});
+
+test('rejects a deleted/missing group or an unexpected server grouping ID before reading hashes', async t => {
+ for (const [label, rows] of [
+ ['missing group', [latest(first)]],
+ ['all deleted', []],
+ ['unexpected group', [latest(first), { id: second.contentId, expected_content_id: '88' }]],
+ ['null CASE result', [latest(first), { id: second.contentId, expected_content_id: null }]]
+ ]) {
+ await t.test(label, async () => {
+ const db = new FakeDb([locked(first, second), rows]);
+ await assert.rejects(verifySourceBatch(db, config, [first, second]), changed);
+ assert.equal(db.calls.length, 2);
+ });
+ }
+});
+
+test('rejects disappearance, size changes, hash changes and wrong IDs in the digest result', async t => {
+ for (const [label, rows] of [
+ ['missing content', [content(first)]],
+ ['wrong size', [content(first), content(second, { size: second.size + 1 })]],
+ ['same size different hash', [content(first), content(second, { sha256: 'c'.repeat(64) })]],
+ ['wrong ID', [content(first), content(second, { id: '9007199254740996' })]],
+ ['duplicate ID', [content(first), content(first)]]
+ ]) {
+ await t.test(label, async () => {
+ const db = new FakeDb([locked(first, second), [latest(first), latest(second)], rows]);
+ await assert.rejects(verifySourceBatch(db, config, [first, second]), changed);
+ assert.equal(db.calls.length, 3);
+ });
+ }
+});
+
+test('preserves unsigned bigint IDs and binds all paths instead of interpolating them', async () => {
+ const large = { ...first, contentId: '18446744073709551615', path: "x.png' OR '1'='1" };
+ const db = new FakeDb([locked(large), [latest(large)], [content(large)]]);
+ await verifySourceBatch(db, { mysql: { database: 'app`db' } }, [large]);
+ for (const call of db.calls) {
+ assert.match(call.sql, /`app``db`\.`infra_file_content`/);
+ assert.ok(!call.sql.includes(large.path));
+ assert.ok(!call.sql.includes(large.contentId));
+ }
+ assert.deepEqual(db.calls[2].params, [large.contentId]);
+});
+
+test('rejects empty batches, duplicate IDs and unsafe numeric IDs before any SQL', async t => {
+ for (const [label, batch, expectedError] of [
+ ['empty batch', [], /不能为空/],
+ ['duplicate content ID', [first, { ...first }], /重复/],
+ ['unsafe content ID', [{ ...first, contentId: Number.MAX_SAFE_INTEGER + 1 }], /安全整数/],
+ ['unsafe config ID', [{ ...first, configId: Number.MAX_SAFE_INTEGER + 1 }], /安全整数/],
+ ['invalid expected size', [{ ...first, size: Number.MAX_SAFE_INTEGER + 1 }], /预期大小/]
+ ]) {
+ await t.test(label, async () => {
+ const db = new FakeDb([]);
+ await assert.rejects(verifySourceBatch(db, config, batch), expectedError);
+ assert.equal(db.calls.length, 0);
+ });
+ }
+});
+
+test('propagates database lock/read errors and leaves rollback to the transaction owner', async () => {
+ const lockFailure = Object.assign(new Error('lock wait timed out'), { code: 'ER_LOCK_WAIT_TIMEOUT' });
+ const db = new FakeDb([lockFailure]);
+ await assert.rejects(verifySourceBatch(db, config, [first]), error => error === lockFailure);
+ assert.equal(db.calls.length, 1);
+});
diff --git a/script/migrate-db-files-to-oss/test/source-current.test.mjs b/script/migrate-db-files-to-oss/test/source-current.test.mjs
new file mode 100644
index 0000000..a9c2654
--- /dev/null
+++ b/script/migrate-db-files-to-oss/test/source-current.test.mjs
@@ -0,0 +1,131 @@
+import test from 'node:test';
+import assert from 'node:assert/strict';
+import { sourceCurrent } from '../migrate.mjs';
+
+const config = { mysql: { database: 'source_current_test' } };
+const item = {
+ contentId: '9007199254740993', configId: '9007199254740995',
+ path: '头像/Résumé + 100%.png', size: 1234, sha256: 'a'.repeat(64)
+};
+const current = overrides => ({ id: item.contentId, size: String(item.size), sha256: item.sha256, ...overrides });
+
+class FakeDb {
+ constructor(responses) {
+ this.responses = responses;
+ this.calls = [];
+ }
+ async execute(sql, params) {
+ assert.match(sql, /^SELECT\b/, 'source validation must not write or manage the caller transaction');
+ this.calls.push({ sql, params });
+ if (this.calls.length > this.responses.length) throw new Error('Unexpected extra database query');
+ const response = this.responses[this.calls.length - 1];
+ if (response instanceof Error) throw response;
+ return [response];
+ }
+}
+
+function assertMetadataLookup(call, expectedItem, locked = false) {
+ assert.match(call.sql, /^SELECT id FROM /);
+ assert.doesNotMatch(call.sql, /\bcontent\b|SHA2\s*\(|OCTET_LENGTH\s*\(/i,
+ 'the unindexed path/order lookup must not project file bytes or content expressions');
+ assert.match(call.sql, /WHERE deleted=0 AND config_id=\? AND path=\? ORDER BY id DESC LIMIT 1/);
+ assert.doesNotMatch(call.sql, /BINARY\s+path|LOWER\s*\(path\)/i, 'preserve native MySQL path collation');
+ assert.deepEqual(call.params, [expectedItem.configId, expectedItem.path]);
+ if (locked) assert.match(call.sql, / FOR UPDATE$/);
+ else assert.doesNotMatch(call.sql, /FOR UPDATE/i);
+}
+
+function assertPrimaryKeyHashRead(call, expectedItem, locked = false) {
+ assert.match(call.sql, /OCTET_LENGTH\(content\) AS size/);
+ assert.match(call.sql, /LOWER\(SHA2\(content,256\)\) AS sha256/);
+ assert.match(call.sql, /WHERE id=\? AND config_id=\? AND path=\? AND deleted=0/);
+ assert.doesNotMatch(call.sql, /ORDER BY|LIMIT|JOIN/i, 'read the digest through a single primary-key lookup');
+ assert.deepEqual(call.params, [expectedItem.contentId, expectedItem.configId, expectedItem.path]);
+ if (locked) assert.match(call.sql, / FOR UPDATE$/);
+ else assert.doesNotMatch(call.sql, /FOR UPDATE/i);
+}
+
+test('looks up the latest ID without BLOB expressions, then validates size and SHA by primary key', async () => {
+ const db = new FakeDb([[{ id: item.contentId }], [current()]]);
+ await sourceCurrent(db, config, item);
+ assert.equal(db.calls.length, 2);
+ assertMetadataLookup(db.calls[0], item);
+ assertPrimaryKeyHashRead(db.calls[1], item);
+});
+
+test('rejects a newer active version before any content/hash query, without rounding bigint IDs', async () => {
+ const db = new FakeDb([[{ id: '9007199254740994' }]]);
+ await assert.rejects(sourceCurrent(db, config, item), /源图片.*变化或已删除/);
+ assert.equal(db.calls.length, 1);
+ assertMetadataLookup(db.calls[0], item);
+});
+
+test('rejects a missing latest record without issuing a content/hash query', async () => {
+ const db = new FakeDb([[]]);
+ await assert.rejects(sourceCurrent(db, config, item), /源图片.*变化或已删除/);
+ assert.equal(db.calls.length, 1);
+ assertMetadataLookup(db.calls[0], item);
+});
+
+test('rejects content disappearing between the metadata and primary-key reads', async () => {
+ const db = new FakeDb([[{ id: item.contentId }], []]);
+ await assert.rejects(sourceCurrent(db, config, item), /源图片.*变化或已删除/);
+ assert.equal(db.calls.length, 2);
+ assertPrimaryKeyHashRead(db.calls[1], item);
+});
+
+test('rejects changed length, changed hash, or an unexpected primary-key row', async t => {
+ for (const [name, row] of [
+ ['changed size', current({ size: item.size + 1 })],
+ ['changed hash with the same size', current({ sha256: 'b'.repeat(64) })],
+ ['wrong content ID', current({ id: '9007199254740994' })],
+ ['missing hash', current({ sha256: null })]
+ ]) {
+ await t.test(name, async () => {
+ const db = new FakeDb([[{ id: item.contentId }], [row]]);
+ await assert.rejects(sourceCurrent(db, config, item), /源图片.*变化或已删除/);
+ assert.equal(db.calls.length, 2);
+ assertPrimaryKeyHashRead(db.calls[1], item);
+ });
+ }
+});
+
+test('retains FOR UPDATE on both validation queries when the caller owns a transaction', async () => {
+ const db = new FakeDb([[{ id: item.contentId }], [current()]]);
+ await sourceCurrent(db, config, item, true);
+ assert.equal(db.calls.length, 2);
+ assertMetadataLookup(db.calls[0], item, true);
+ assertPrimaryKeyHashRead(db.calls[1], item, true);
+});
+
+test('locked validation still stops at the metadata lookup if the latest version changed', async () => {
+ const db = new FakeDb([[{ id: '9007199254740994' }]]);
+ await assert.rejects(sourceCurrent(db, config, item, true), /源图片.*变化或已删除/);
+ assert.equal(db.calls.length, 1);
+ assertMetadataLookup(db.calls[0], item, true);
+});
+
+test('supports unsigned-bigint-sized IDs and binds paths without interpolation or normalization', async () => {
+ const largeItem = { ...item, contentId: '18446744073709551615', path: "case/IMAGE.png' OR 'x'='x" };
+ const db = new FakeDb([[{ id: largeItem.contentId }], [current({ id: largeItem.contentId })]]);
+ await sourceCurrent(db, { mysql: { database: 'app`db' } }, largeItem, false);
+ assertMetadataLookup(db.calls[0], largeItem);
+ assertPrimaryKeyHashRead(db.calls[1], largeItem);
+ for (const call of db.calls) {
+ assert.match(call.sql, /`app``db`\.`infra_file_content`/);
+ assert.ok(!call.sql.includes(largeItem.contentId));
+ assert.ok(!call.sql.includes(largeItem.path));
+ }
+});
+
+test('propagates database failures instead of accepting an unverified source', async () => {
+ for (const responses of [
+ [Object.assign(new Error('metadata lookup timed out'), { code: 'TEST_TIMEOUT' })],
+ [[{ id: item.contentId }], Object.assign(new Error('hash read failed'), { code: 'TEST_READ_FAILURE' })]
+ ]) {
+ const db = new FakeDb(responses);
+ const expected = responses[responses.length - 1];
+ await assert.rejects(sourceCurrent(db, config, item), error => error === expected);
+ assert.equal(db.calls.length, responses.length);
+ }
+});
diff --git a/script/migrate-db-files-to-oss/test/source-index.test.mjs b/script/migrate-db-files-to-oss/test/source-index.test.mjs
new file mode 100644
index 0000000..699e0d3
--- /dev/null
+++ b/script/migrate-db-files-to-oss/test/source-index.test.mjs
@@ -0,0 +1,127 @@
+import test from 'node:test';
+import assert from 'node:assert/strict';
+import { loadSourceIndex } from '../lib/source-index.mjs';
+
+const config = { mysql: { database: 'source_index_test' }, sourceConfigIds: ['9007199254740999', '5'] };
+const makeFile = (value, contentId, overrides = {}) => ({
+ id: value, config_id: '5', path: 'image.png', url: 'https://old.example/image.png',
+ type: 'image/png', size: 3, content_id: contentId, ...overrides
+});
+
+class FakeDb {
+ constructor(batches, stageEvents) {
+ this.batches = batches;
+ this.stageEvents = stageEvents;
+ this.calls = [];
+ }
+ async execute(sql, params) {
+ assert.match(sql.trim(), /^SELECT\b/);
+ assert.doesNotMatch(sql, /\b(?:OCTET_LENGTH|SHA2)\s*\(/i);
+ assert.doesNotMatch(sql, /\bcontent\b|SELECT\s+\*/i, 'only metadata columns may be selected');
+ assert.doesNotMatch(sql, /\b(?:NOT EXISTS|CREATE|INSERT|UPDATE|DELETE|ALTER)\b/i);
+ assert.match(sql, /GROUP BY config_id,path/);
+ assert.match(sql, /MAX\(id\) AS id/);
+ assert.match(sql, /WHERE deleted=0 AND config_id IN \(\?,\?\)/);
+ if (this.stageEvents) assert.equal(this.stageEvents.length, this.calls.length + 1, 'announce each phase before executing it');
+ this.calls.push({ sql, params });
+ if (this.calls.length > 3) throw new Error('Unexpected per-file query');
+ return [this.batches[this.calls.length - 1]];
+ }
+}
+
+test('loads all metadata in exactly three SELECTs and preserves bigint IDs and raw spellings', async () => {
+ const stages = [];
+ const db = new FakeDb([
+ [
+ { content_id: '9007199254740995', config_id: '9007199254740999', path: '头像.png' },
+ { content_id: '9', config_id: '5', path: 'IMAGE.png' }
+ ],
+ [makeFile('9007199254740996', '9'), makeFile('2', '9', { path: 'IMAGE.png' }), makeFile('10', null)],
+ [
+ { content_id: '9', path: Buffer.from('image.png') },
+ { content_id: '9', path: Buffer.from('IMAGE.png') },
+ { content_id: '9', path: Buffer.from('image.png ') },
+ { content_id: '9', path: Buffer.from('image.png') },
+ { content_id: '9007199254740995', path: Buffer.from('头像.png') }
+ ]
+ ], stages);
+ const result = await loadSourceIndex(db, config, { progress: { stage: message => stages.push(message) } });
+ assert.equal(db.calls.length, 3);
+ assert.equal(stages.length, 3);
+ assert.equal(result.maxId, '9007199254740995');
+ assert.deepEqual(result.groups.map(group => group.contentId), ['9', '9007199254740995']);
+ assert.equal(result.groups[0].path, 'IMAGE.png', 'the latest row supplies the canonical raw path');
+ assert.deepEqual(result.groups[0].files.map(file => file.id), ['2', '9007199254740996']);
+ assert.deepEqual(result.groups[0].spellings, ['IMAGE.png', 'image.png', 'image.png ']);
+ assert.equal(result.groups[1].configId, '9007199254740999');
+ assert.deepEqual(result.groups[1].files, [], 'orphan BLOBs remain candidates');
+ assert.deepEqual(result.groups[1].spellings, ['头像.png']);
+ assert.deepEqual(result.missingFiles.map(file => file.id), ['10']);
+ assert.equal(Object.hasOwn(result.missingFiles[0], 'content_id'), false);
+ assert.deepEqual(db.calls[0].params, ['5', '9007199254740999']);
+ assert.deepEqual(db.calls[1].params, ['5', '9007199254740999', '5', '9007199254740999']);
+});
+
+test('upper bound applies after grouping and files with newer content are never marked missing', async () => {
+ const bound = '9007199254740993';
+ const db = new FakeDb([
+ [{ content_id: '8', config_id: '5', path: 'image.png' }],
+ [makeFile('1', '8'), makeFile('2', '9007199254740994'), makeFile('3', null)],
+ [{ content_id: '8', path: 'image.png' }, { content_id: '9007199254740994', path: 'newer.png' }]
+ ]);
+ const result = await loadSourceIndex(db, config, { maxId: bound });
+ assert.equal(result.maxId, bound);
+ assert.deepEqual(result.groups.map(group => group.contentId), ['8']);
+ assert.deepEqual(result.groups[0].files.map(file => file.id), ['1']);
+ assert.deepEqual(result.missingFiles.map(file => file.id), ['3']);
+ assert.match(db.calls[0].sql, /GROUP BY config_id,path\) g ON g.id=c.id WHERE g.id<=\?/);
+ assert.doesNotMatch(db.calls[0].sql, /WHERE deleted=0[^)]*id\s*[<>=]/i);
+ assert.doesNotMatch(db.calls[1].sql, /g\.id<=|id<=/);
+ assert.match(db.calls[2].sql, /WHERE c.deleted=0 AND g.id<=\?/);
+ assert.deepEqual(db.calls[0].params, ['5', '9007199254740999', bound]);
+ assert.deepEqual(db.calls[2].params, ['5', '9007199254740999', bound]);
+});
+
+test('all path associations use server-generated content IDs, never JavaScript case folding', async () => {
+ const db = new FakeDb([
+ [{ content_id: '7', config_id: '5', path: 'Résumé.png' }, { content_id: '8', config_id: '5', path: 'resume.png' }],
+ [makeFile('1', '7', { path: 'RESUME.png' }), makeFile('2', '8', { path: 'resume.png' })],
+ [{ content_id: '7', path: Buffer.from('Résumé.png') }, { content_id: '7', path: Buffer.from('RESUME.png') }, { content_id: '8', path: Buffer.from('resume.png') }]
+ ]);
+ const result = await loadSourceIndex(db, config);
+ assert.equal(result.groups.length, 2);
+ assert.deepEqual(result.groups[0].files.map(file => file.id), ['1']);
+ assert.deepEqual(result.groups[1].files.map(file => file.id), ['2']);
+ assert.match(db.calls[1].sql, /g\.config_id=f\.config_id AND g\.path=f\.path/);
+ assert.match(db.calls[2].sql, /SELECT DISTINCT g.id AS content_id,BINARY c.path AS path/);
+ assert.match(db.calls[2].sql, /g\.config_id=c\.config_id AND g\.path=c\.path/);
+ assert.match(db.calls[1].sql, /WHERE f.deleted=0/);
+ assert.match(db.calls[2].sql, /WHERE c.deleted=0/);
+});
+
+test('empty content results still report missing files and use a zero upper bound', async () => {
+ const db = new FakeDb([[], [makeFile('4', null)], []]);
+ const result = await loadSourceIndex(db, config);
+ assert.equal(db.calls.length, 3);
+ assert.equal(result.maxId, '0');
+ assert.deepEqual(result.groups, []);
+ assert.deepEqual(result.missingFiles.map(file => file.id), ['4']);
+ const bounded = await loadSourceIndex(new FakeDb([[], [], []]), config, { maxId: '0' });
+ assert.equal(bounded.maxId, '0');
+});
+
+test('source IDs are deduplicated and database identifiers are safely quoted', async () => {
+ const db = new FakeDb([[], [], []]);
+ const unusual = { mysql: { database: 'app`db' }, sourceConfigIds: ['9007199254740999', '5', '5'] };
+ await loadSourceIndex(db, unusual);
+ assert.match(db.calls[0].sql, /`app``db`\.`infra_file_content`/);
+ assert.deepEqual(db.calls[0].params, ['5', '9007199254740999']);
+});
+
+test('rejects unsafe numeric boundaries and invalid source configurations without SQL', async () => {
+ const db = new FakeDb([[], [], []]);
+ await assert.rejects(loadSourceIndex(db, config, { maxId: Number.MAX_SAFE_INTEGER + 1 }), /大整数/);
+ await assert.rejects(loadSourceIndex(db, config, { maxId: '-1' }), /非负整数/);
+ await assert.rejects(loadSourceIndex(db, { ...config, sourceConfigIds: [] }), /至少一个/);
+ assert.equal(db.calls.length, 0);
+});
diff --git a/script/migrate-db-files-to-oss/test/target-validation.test.mjs b/script/migrate-db-files-to-oss/test/target-validation.test.mjs
new file mode 100644
index 0000000..2d5ca88
--- /dev/null
+++ b/script/migrate-db-files-to-oss/test/target-validation.test.mjs
@@ -0,0 +1,245 @@
+import test, { mock } from 'node:test';
+import assert from 'node:assert/strict';
+import { createServer } from 'node:http';
+import { once } from 'node:events';
+import { setImmediate } from 'node:timers/promises';
+import { sha256 } from '../lib/core.mjs';
+import { verifyCheckpoint, verifySwitchedBatch } from '../migrate.mjs';
+
+const CONTENT = Buffer.from('89504e470d0a1a0a0011223344556677', 'hex');
+const CORRUPT = Buffer.alloc(CONTENT.length, 1);
+const CONFIG = { mysql: { database: 'target_test' }, targetConfigId: '20', timeoutMs: 2000, maxFileBytes: 1024, retries: 1 };
+const item = (url, files = [{ id: '9007199254740993' }]) => ({
+ contentId: '9007199254740995', configId: '1', path: 'source.png',
+ key: 'migration/object.png', url, size: CONTENT.length, sha256: sha256(CONTENT), files,
+});
+const stored = entry => ({
+ id: entry.files[0].id, config_id: '20', path: entry.key,
+ url: entry.url, deleted: Buffer.from([0]),
+});
+const oss = (bytes = CONTENT) => ({
+ head: mock.fn(async () => ({ res: { headers: { 'content-length': String(bytes.length) } } })),
+ get: mock.fn(async () => ({ content: bytes })),
+ put: mock.fn(async () => { throw new Error('validation must never upload'); }),
+});
+
+async function server(t, handler) {
+ const instance = createServer(handler);
+ instance.listen(0, '127.0.0.1');
+ await once(instance, 'listening');
+ t.after(async () => {
+ const closing = new Promise((resolve, reject) => instance.close(error => error ? reject(error) : resolve()));
+ instance.closeAllConnections();
+ await closing;
+ });
+ return 'http://127.0.0.1:' + instance.address().port;
+}
+
+test('verified checkpoint validates authenticated and public bytes without PUT or source access', async t => {
+ let publicReads = 0;
+ const url = await server(t, (_request, response) => { publicReads++; response.end(CONTENT); });
+ const entry = item(url);
+ const client = oss();
+ await verifyCheckpoint(client, CONFIG, entry);
+ assert.equal(client.head.mock.callCount(), 1);
+ assert.equal(client.get.mock.callCount(), 1);
+ assert.equal(client.put.mock.callCount(), 0);
+ assert.deepEqual(client.head.mock.calls[0].arguments, [entry.key]);
+ assert.deepEqual(client.get.mock.calls[0].arguments, [entry.key]);
+ assert.equal(publicReads, 1);
+});
+
+test('missing checkpoint object is rejected without downloading or creating another object', async () => {
+ const client = oss();
+ const missing = Object.assign(new Error('missing object'), { code: 'NoSuchKey', status: 404 });
+ client.head = mock.fn(async () => { throw missing; });
+ await assert.rejects(verifyCheckpoint(client, CONFIG, item('http://127.0.0.1:1/unused')), error => error === missing);
+ assert.equal(client.get.mock.callCount(), 0);
+ assert.equal(client.put.mock.callCount(), 0);
+});
+
+test('checkpoint HEAD rejects mismatched sizes or configured size limits before GET', async () => {
+ for (const tooLarge of [false, true]) {
+ const client = oss();
+ if (!tooLarge) client.head = mock.fn(async () => ({ res: { headers: { 'content-length': String(CONTENT.length + 1) } } }));
+ await assert.rejects(verifyCheckpoint(client, { ...CONFIG, maxFileBytes: tooLarge ? 1 : CONFIG.maxFileBytes }, item('http://127.0.0.1:1/unused')), /长度/);
+ assert.equal(client.get.mock.callCount(), 0);
+ assert.equal(client.put.mock.callCount(), 0);
+ }
+});
+
+test('same-length authenticated corruption rejects before public validation', async t => {
+ let publicReads = 0;
+ const url = await server(t, (_request, response) => { publicReads++; response.end(CONTENT); });
+ const client = oss(CORRUPT);
+ await assert.rejects(verifyCheckpoint(client, CONFIG, item(url)), /SHA-256/);
+ assert.equal(publicReads, 0);
+ assert.equal(client.put.mock.callCount(), 0);
+});
+
+test('public corruption or access denial prevents reusing an otherwise valid checkpoint', async t => {
+ const url = await server(t, (request, response) => {
+ if (request.url === '/denied') { response.writeHead(403); response.end(); }
+ else response.end(CORRUPT);
+ });
+ for (const [path, message] of [['/corrupt', /SHA-256/], ['/denied', /HTTP 403/]]) {
+ const client = oss();
+ await assert.rejects(verifyCheckpoint(client, CONFIG, item(url + path)), message);
+ assert.equal(client.put.mock.callCount(), 0);
+ }
+});
+
+test('switched batch reads all metadata once with exact bigint IDs and deduplicates repeated records', async t => {
+ const url = await server(t, (_request, response) => response.end(CONTENT));
+ const first = item(url + '/one');
+ const second = { ...item(url + '/two', [{ id: '9007199254740997' }, { id: '9007199254740997' }]), key: 'migration/two.png' };
+ const db = { execute: mock.fn(async (sql, params) => {
+ assert.match(sql, /SELECT id,config_id,path,url,deleted/);
+ assert.match(sql, /WHERE id IN \(\?,\?\)/);
+ assert.deepEqual(params, ['9007199254740993', '9007199254740997']);
+ return [[stored(first), stored(second)]];
+ }) };
+ const results = await verifySwitchedBatch(db, CONFIG, [first, second]);
+ assert.equal(db.execute.mock.callCount(), 1);
+ assert.deepEqual(results, [{ item: first }, { item: second }]);
+});
+
+test('orphaned source content with no file metadata still validates its public target', async t => {
+ let publicReads = 0;
+ const url = await server(t, (_request, response) => { publicReads++; response.end(CONTENT); });
+ const entry = item(url, []);
+ const db = { execute: mock.fn(async () => { throw new Error('orphan must not query file metadata'); }) };
+ assert.deepEqual(await verifySwitchedBatch(db, CONFIG, [entry]), [{ item: entry }]);
+ assert.equal(db.execute.mock.callCount(), 0);
+ assert.equal(publicReads, 1);
+});
+
+test('mutated, deleted and missing file records are returned as errors before public access', async t => {
+ let publicReads = 0;
+ const url = await server(t, (_request, response) => { publicReads++; response.end(CONTENT); });
+ const entry = item(url);
+ for (const changed of [
+ null,
+ { ...stored(entry), config_id: '21' },
+ { ...stored(entry), path: 'some/other/object.png' },
+ { ...stored(entry), url: url + '/edited' },
+ { ...stored(entry), deleted: Buffer.from([1]) },
+ ]) {
+ const db = { execute: mock.fn(async () => [changed ? [changed] : []]) };
+ const results = await verifySwitchedBatch(db, CONFIG, [entry]);
+ assert.equal(results.length, 1);
+ assert.equal(results[0].item, entry);
+ assert.match(results[0].error.message, /发生变化|已删除/);
+ assert.deepEqual(results.filter(result => !result.error), []);
+ }
+ assert.equal(publicReads, 0);
+});
+
+test('a single missing metadata record rejects an item that has multiple original records', async t => {
+ let publicReads = 0;
+ const url = await server(t, (_request, response) => { publicReads++; response.end(CONTENT); });
+ const entry = item(url, [{ id: '9007199254740993' }, { id: '9007199254740997' }]);
+ const db = { execute: mock.fn(async () => [[stored(entry)]]) };
+ const [result] = await verifySwitchedBatch(db, CONFIG, [entry]);
+ assert.ok(result.error);
+ assert.equal(publicReads, 0);
+});
+
+test('switched public corruption excludes only its own mapping and keeps input order', async t => {
+ const url = await server(t, (request, response) => response.end(request.url === '/bad' ? CORRUPT : CONTENT));
+ const bad = item(url + '/bad', []);
+ const good = item(url + '/good', []);
+ const results = await verifySwitchedBatch({}, CONFIG, [bad, good]);
+ assert.equal(results[0].item, bad);
+ assert.match(results[0].error.message, /SHA-256/);
+ assert.deepEqual(results[1], { item: good });
+ assert.deepEqual(results.filter(result => !result.error).map(result => result.item.url), [good.url]);
+});
+
+test('metadata query errors reject before network validation starts', async t => {
+ let publicReads = 0;
+ const url = await server(t, (_request, response) => { publicReads++; response.end(CONTENT); });
+ const failure = new Error('metadata unavailable');
+ const db = { execute: mock.fn(async () => { throw failure; }) };
+ await assert.rejects(verifySwitchedBatch(db, CONFIG, [item(url)]), error => error === failure);
+ assert.equal(publicReads, 0);
+});
+
+test('final validation is capped at four active requests and settles only after every started request', { timeout: 4000 }, async t => {
+ const responses = new Map();
+ let active = 0, maximum = 0, settled = false;
+ const url = await server(t, (request, response) => {
+ active++;
+ maximum = Math.max(maximum, active);
+ response.once('finish', () => { active--; });
+ responses.set(Number(request.url.slice(1)), response);
+ });
+ const batch = Array.from({ length: 6 }, (_, index) => item(url + '/' + index, []));
+ const pending = verifySwitchedBatch({}, CONFIG, batch).then(results => { settled = true; return results; });
+ const until = async predicate => {
+ const deadline = Date.now() + 1500;
+ while (!predicate()) {
+ if (Date.now() > deadline) assert.fail('validation did not reach the expected request state');
+ await setImmediate();
+ }
+ };
+ await until(() => responses.size === 4);
+ assert.equal(active, 4);
+ assert.equal(settled, false);
+ // Per-item failures are collected, so remaining items must still be checked.
+ responses.get(0).end(CORRUPT);
+ responses.get(1).end(CONTENT);
+ await until(() => responses.size === 6);
+ assert.equal(active, 4);
+ assert.equal(settled, false);
+ responses.get(2).end(CONTENT);
+ responses.get(3).end(CONTENT);
+ responses.get(4).end(CONTENT);
+ await setImmediate();
+ assert.equal(settled, false);
+ responses.get(5).end(CONTENT);
+ const results = await pending;
+ assert.equal(maximum, 4);
+ assert.equal(active, 0);
+ assert.equal(settled, true);
+ assert.equal(results.length, 6);
+ assert.match(results[0].error.message, /SHA-256/);
+ assert.deepEqual(results.slice(1), batch.slice(1).map(entry => ({ item: entry })));
+});
+
+test('empty switched batches perform no database or network work', async () => {
+ const db = { execute: mock.fn(async () => { throw new Error('must not query'); }) };
+ assert.deepEqual(await verifySwitchedBatch(db, CONFIG, []), []);
+ assert.equal(db.execute.mock.callCount(), 0);
+});
+
+test('more than 1000 file records are loaded in bounded sequential queries before public validation', async t => {
+ let completedQueries = 0, activeQueries = 0, maximumQueries = 0, publicReads = 0;
+ const url = await server(t, (_request, response) => {
+ publicReads++;
+ assert.equal(completedQueries, 2);
+ assert.equal(activeQueries, 0);
+ response.end(CONTENT);
+ });
+ const files = Array.from({ length: 1001 }, (_, index) => ({ id: String(9007199254740993n + BigInt(index)) }));
+ const entry = item(url, files);
+ const seenIds = [];
+ const querySizes = [];
+ const db = { execute: mock.fn(async (_sql, params) => {
+ maximumQueries = Math.max(maximumQueries, ++activeQueries);
+ assert.ok(params.length <= 1000);
+ assert.equal(publicReads, 0);
+ querySizes.push(params.length);
+ seenIds.push(...params);
+ await setImmediate();
+ completedQueries++;
+ activeQueries--;
+ return [params.map(id => ({ ...stored(entry), id })).reverse()];
+ }) };
+ assert.deepEqual(await verifySwitchedBatch(db, CONFIG, [entry]), [{ item: entry }]);
+ assert.deepEqual(querySizes, [1000, 1]);
+ assert.deepEqual(seenIds, files.map(file => file.id));
+ assert.equal(maximumQueries, 1);
+ assert.equal(db.execute.mock.callCount(), 2);
+ assert.equal(publicReads, 1);
+});
diff --git a/script/migrate-db-files-to-oss/test/transition-integration.test.mjs b/script/migrate-db-files-to-oss/test/transition-integration.test.mjs
new file mode 100644
index 0000000..8ec9a8c
--- /dev/null
+++ b/script/migrate-db-files-to-oss/test/transition-integration.test.mjs
@@ -0,0 +1,213 @@
+import test from 'node:test';
+import assert from 'node:assert/strict';
+import { setTimeout as delay } from 'node:timers/promises';
+import { encodePath, sha256 } from '../lib/core.mjs';
+import { ensureJournal, rollbackChanges } from '../lib/journal.mjs';
+import { verifySourceBatch } from '../lib/source-batch.mjs';
+import { switchFiles } from '../lib/transition.mjs';
+
+// Opt in only against the disposable localhost test server. No host, database,
+// credentials or application configuration is read from the project.
+const SCHEMA = 'codex_oss_transition_test';
+const PORT = process.env.MIGRATION_TEST_DB_PORT;
+const IDS = {
+ older: '9007199254741001', first: '9007199254741003', second: '9007199254741005',
+ orphan: '9007199254741007', deletedNewer: '9007199254741009', newVersion: '9007199254741011'
+};
+const FIRST_PATH = '相册/Résumé.PNG';
+const EQUIVALENT_PATH = '相册/RESUME.png';
+const imageBytes = [Buffer.from([1, 2, 3, 4, 5]), Buffer.from([6, 7, 8, 9]), Buffer.from([10, 11, 12])];
+const changedSource = error => error.code === 'MIGRATION_SOURCE_CHANGED';
+const changedRow = error => error.code === 'MIGRATION_CAS_CONFLICT';
+
+test('MySQL batch transitions preserve source locks, atomic journals and rollback', {
+ skip: !PORT, timeout: 45000
+}, async t => {
+ const port = Number(PORT);
+ assert.ok(Number.isInteger(port) && port > 0 && port <= 65535, 'MIGRATION_TEST_DB_PORT must be a valid disposable localhost MySQL port');
+ const { default: mysql } = await import('mysql2/promise');
+ const connectionOptions = {
+ host: '127.0.0.1', port, user: 'root', password: '', connectTimeout: 3000,
+ supportBigNumbers: true, bigNumberStrings: true, jsonStrings: true, dateStrings: true
+ };
+ const db = await mysql.createConnection(connectionOptions);
+ let contender, observer;
+ try {
+ await db.query(`DROP DATABASE IF EXISTS ${SCHEMA}`);
+ await db.query(`CREATE DATABASE ${SCHEMA} CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci`);
+ await db.changeUser({ database: SCHEMA });
+ await db.query('SET SESSION TRANSACTION ISOLATION LEVEL REPEATABLE READ');
+ await db.query("SET SESSION sql_mode=CONCAT_WS(',',NULLIF(@@SESSION.sql_mode,''),'STRICT_ALL_TABLES')");
+ await db.query('SET SESSION innodb_lock_wait_timeout=5');
+ await db.query(`CREATE TABLE infra_file_content (
+ id BIGINT UNSIGNED NOT NULL PRIMARY KEY,config_id BIGINT UNSIGNED NOT NULL,
+ path VARCHAR(512) NOT NULL,content MEDIUMBLOB NOT NULL,deleted BIT(1) NOT NULL DEFAULT 0
+ ) ENGINE=InnoDB`);
+ await db.query(`CREATE TABLE infra_file (
+ id BIGINT UNSIGNED NOT NULL PRIMARY KEY,config_id BIGINT UNSIGNED NOT NULL,
+ path VARCHAR(512) NOT NULL,url VARCHAR(1024) NOT NULL,type VARCHAR(128),size INT,
+ deleted BIT(1) NOT NULL DEFAULT 0,
+ update_time TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
+ ) ENGINE=InnoDB`);
+ await ensureJournal(db, SCHEMA);
+
+ contender = await mysql.createConnection({ ...connectionOptions, database: SCHEMA });
+ observer = await mysql.createConnection({ ...connectionOptions, database: SCHEMA });
+ await contender.query('SET SESSION TRANSACTION ISOLATION LEVEL REPEATABLE READ');
+ await contender.query('SET SESSION innodb_lock_wait_timeout=5');
+ const [[thread]] = await contender.query('SELECT CONNECTION_ID() AS id');
+
+ const readFiles = async () => {
+ const [rows] = await db.query('SELECT id,config_id,path,url,type,size,deleted,update_time FROM infra_file ORDER BY id');
+ return rows.map(row => ({ ...row }));
+ };
+ const readStatuses = async runId => {
+ const [rows] = await db.execute('SELECT status FROM infra_file_oss_migration_items WHERE run_id=? ORDER BY item_key', [runId]);
+ return rows.map(row => row.status);
+ };
+ const journalCount = async runId => {
+ const [[row]] = await db.execute('SELECT COUNT(*) AS n FROM infra_file_oss_migration_changes WHERE run_id=?', [runId]);
+ return Number(row.n);
+ };
+
+ async function seed(runId) {
+ await db.rollback();
+ for (const table of ['infra_file_oss_migration_changes', 'infra_file_oss_migration_items', 'infra_file_oss_migration_runs', 'infra_file', 'infra_file_content']) {
+ await db.query(`DELETE FROM ${table}`);
+ }
+ const sources = [
+ [IDS.older, EQUIVALENT_PATH, Buffer.from('older version'), 0],
+ [IDS.first, FIRST_PATH, imageBytes[0], 0],
+ [IDS.second, 'second.png', imageBytes[1], 0],
+ [IDS.orphan, 'orphan.png', imageBytes[2], 0],
+ [IDS.deletedNewer, EQUIVALENT_PATH, Buffer.from('deleted newer version'), 1]
+ ];
+ await db.execute(`INSERT INTO infra_file_content(id,config_id,path,content,deleted) VALUES ${sources.map(() => '(?,5,?,?,?)').join(',')}`, sources.flat());
+ const files = [
+ ['9007199254741101', FIRST_PATH], ['9007199254741102', EQUIVALENT_PATH], ['9007199254741103', 'second.png']
+ ];
+ await db.execute(`INSERT INTO infra_file(id,config_id,path,url,type,size,update_time) VALUES ${files.map(() => "(?,5,?,?,'image/png',1,'2020-01-01 00:00:00')").join(',')}`,
+ files.flatMap(([fileId, path]) => [fileId, path, `https://legacy.test/admin-api/infra/file/5/get/${encodePath(path)}`]));
+ const items = [];
+ for (const [index, [contentId, path]] of [[IDS.first, FIRST_PATH], [IDS.second, 'second.png'], [IDS.orphan, 'orphan.png']].entries()) {
+ const [records] = await db.execute('SELECT id,config_id,path,url,type,size FROM infra_file WHERE config_id=5 AND path=? AND deleted=0 ORDER BY id', [path]);
+ const key = `migration/test/${contentId}.png`;
+ items.push({ contentId, configId: '5', path, size: imageBytes[index].length, sha256: sha256(imageBytes[index]), key, url: `https://oss.test/${key}`, type: 'image/png', files: records.map(row => ({ ...row })) });
+ }
+ await db.execute("INSERT INTO infra_file_oss_migration_runs(run_id,settings,status) VALUES (?,?,'active')", [runId, JSON.stringify({ test: true })]);
+ for (const item of items) {
+ await db.execute("INSERT INTO infra_file_oss_migration_items(run_id,item_key,data,status) VALUES (?,?,?,'verified')", [runId, sha256(`${item.configId}\0${item.path}`), JSON.stringify(item)]);
+ }
+ return { config: { mysql: { database: SCHEMA }, runId, targetConfigId: '30' }, items, originals: await readFiles() };
+ }
+
+ await t.test('batch success switches duplicate metadata and orphan progress; journals restore original values', async () => {
+ const fixture = await seed('batch-success');
+ assert.equal(fixture.items[0].files.length, 2, 'native collation includes the case/accent-equivalent metadata');
+ assert.equal(fixture.items[2].files.length, 0, 'orphan content remains a migration item');
+ await switchFiles(db, fixture.config, fixture.items);
+ const files = await readFiles();
+ for (const item of fixture.items) for (const original of item.files) {
+ const stored = files.find(file => file.id === original.id);
+ assert.equal(stored.config_id, '30');
+ assert.equal(stored.path, item.key);
+ assert.equal(stored.url, item.url);
+ assert.equal(stored.size, item.size);
+ assert.equal(stored.update_time, '2020-01-01 00:00:00');
+ }
+ assert.deepEqual(await readStatuses(fixture.config.runId), ['switched', 'switched', 'switched']);
+ assert.equal(await journalCount(fixture.config.runId), 3);
+ const result = await rollbackChanges(db, SCHEMA, fixture.config.runId);
+ assert.equal(result.restored, 3);
+ assert.deepEqual(result.conflicts, []);
+ assert.deepEqual(await readFiles(), fixture.originals);
+ const [[source]] = await db.execute('SELECT content FROM infra_file_content WHERE id=?', [IDS.first]);
+ assert.deepEqual(source.content, imageBytes[0]);
+ });
+
+ await t.test('a business-row CAS conflict rolls back the entire batch and preserves concurrent edits', async () => {
+ const fixture = await seed('cas-conflict');
+ await db.execute('UPDATE infra_file SET url=? WHERE id=?', ['https://user-edit.test/new.png', fixture.items[1].files[0].id]);
+ const beforeAttempt = await readFiles();
+ await assert.rejects(switchFiles(db, fixture.config, fixture.items), changedRow);
+ assert.deepEqual(await readFiles(), beforeAttempt);
+ assert.deepEqual(await readStatuses(fixture.config.runId), ['verified', 'verified', 'verified']);
+ assert.equal(await journalCount(fixture.config.runId), 0);
+ });
+
+ await t.test('a final progress-state CAS conflict rolls back already-written files and journals', async () => {
+ const fixture = await seed('status-conflict');
+ const orphan = fixture.items[2];
+ await db.execute("UPDATE infra_file_oss_migration_items SET status='failed' WHERE run_id=? AND item_key=?", [fixture.config.runId, sha256(`${orphan.configId}\0${orphan.path}`)]);
+ await assert.rejects(switchFiles(db, fixture.config, fixture.items), changedRow);
+ assert.deepEqual(await readFiles(), fixture.originals);
+ assert.deepEqual((await readStatuses(fixture.config.runId)).sort(), ['failed', 'verified', 'verified']);
+ assert.equal(await journalCount(fixture.config.runId), 0);
+ });
+
+ await t.test('source mutations and a newer collation-equivalent version prevent every database switch', async t2 => {
+ for (const kind of ['same-size-hash-change', 'deleted-source', 'newer-equivalent-path']) {
+ await t2.test(kind, async () => {
+ const fixture = await seed(kind);
+ if (kind === 'same-size-hash-change') await db.execute('UPDATE infra_file_content SET content=? WHERE id=?', [Buffer.from([9, 9, 9, 9, 9]), IDS.first]);
+ else if (kind === 'deleted-source') await db.execute('UPDATE infra_file_content SET deleted=1 WHERE id=?', [IDS.first]);
+ else await db.execute('INSERT INTO infra_file_content(id,config_id,path,content) VALUES (?,5,?,?)', [IDS.newVersion, EQUIVALENT_PATH, Buffer.from('new active version')]);
+ await assert.rejects(switchFiles(db, fixture.config, fixture.items), changedSource);
+ assert.deepEqual(await readFiles(), fixture.originals);
+ assert.deepEqual(await readStatuses(fixture.config.runId), ['verified', 'verified', 'verified']);
+ assert.equal(await journalCount(fixture.config.runId), 0);
+ });
+ }
+ });
+
+ async function proveRangeLock(kind, statement, parameters) {
+ const fixture = await seed(`range-${kind}`);
+ let outcome, pending;
+ await db.beginTransaction();
+ try {
+ // Must be the first consistent read after BEGIN; the locking read
+ // inside verification establishes protection before the RR snapshot.
+ await verifySourceBatch(db, fixture.config, [fixture.items[0]]);
+ pending = contender.execute(statement, parameters).then(
+ value => { outcome = { value }; }, error => { outcome = { error }; }
+ );
+ const deadline = Date.now() + 3000;
+ let observedWait = false;
+ while (Date.now() < deadline) {
+ assert.equal(outcome, undefined, 'concurrent source mutation must not complete before commit');
+ const [transactions] = await observer.execute('SELECT trx_state FROM information_schema.innodb_trx WHERE trx_mysql_thread_id=?', [thread.id]);
+ if (transactions.some(transaction => transaction.trx_state === 'LOCK WAIT')) { observedWait = true; break; }
+ await delay(20);
+ }
+ assert.ok(observedWait, 'observe an actual InnoDB lock wait, not merely a delayed promise');
+ assert.equal(outcome, undefined);
+ await db.commit();
+ await pending;
+ if (outcome.error) throw outcome.error;
+ assert.equal(outcome.value[0].affectedRows, 1, 'mutation completes after the source lock is released');
+ } finally {
+ await db.rollback();
+ if (pending) await pending;
+ }
+ // Once the waiting writer commits, the next batch must detect the new
+ // active version rather than continuing with its old snapshot.
+ await db.beginTransaction();
+ try {
+ await assert.rejects(verifySourceBatch(db, fixture.config, [fixture.items[0]]), changedSource);
+ } finally { await db.rollback(); }
+ }
+
+ await t.test('RR range locks block insertion of a new matching version until commit', async () => {
+ await proveRangeLock('insert', 'INSERT INTO infra_file_content(id,config_id,path,content) VALUES (?,5,?,?)', [IDS.newVersion, EQUIVALENT_PATH, Buffer.from('concurrent version')]);
+ });
+ await t.test('RR range locks include deleted rows and block undeleting a newer version until commit', async () => {
+ await proveRangeLock('undelete', 'UPDATE infra_file_content SET deleted=0 WHERE id=?', [IDS.deletedNewer]);
+ });
+ } finally {
+ await db.rollback().catch(() => {});
+ if (contender) await contender.end();
+ if (observer) await observer.end();
+ try { await db.query(`DROP DATABASE IF EXISTS ${SCHEMA}`); }
+ finally { await db.end(); }
+ }
+});
diff --git a/script/migrate-db-files-to-oss/test/transition.test.mjs b/script/migrate-db-files-to-oss/test/transition.test.mjs
new file mode 100644
index 0000000..d76d959
--- /dev/null
+++ b/script/migrate-db-files-to-oss/test/transition.test.mjs
@@ -0,0 +1,251 @@
+import test from 'node:test';
+import assert from 'node:assert/strict';
+import { switchFiles } from '../lib/transition.mjs';
+import { sha256 } from '../lib/core.mjs';
+
+const config = { mysql: { database: 'transition_test' }, runId: 'run', targetConfigId: '20' };
+const itemKey = item => sha256(`${item.configId}\0${item.path}`);
+const clone = value => Buffer.isBuffer(value) ? Buffer.from(value)
+ : Array.isArray(value) ? value.map(clone)
+ : value && typeof value === 'object' ? Object.fromEntries(Object.entries(value).map(([key, item]) => [key, clone(item)])) : value;
+
+function fixture({ orphan = false, count = 1 } = {}) {
+ const item = {
+ contentId: '9007199254740993', configId: '5', path: '头像/原图.png',
+ size: 12, sha256: 'a'.repeat(64), key: 'migration/new.png',
+ url: 'https://oss/new.png', type: 'image/png', files: []
+ };
+ if (!orphan) for (let index = 0; index < count; index++) item.files.push({
+ id: String(9007199254741093n + BigInt(index)), config_id: '5', path: item.path,
+ url: 'https://old/old.png', type: 'image/png', size: 12
+ });
+ return [item];
+}
+
+class TransitionDb {
+ constructor(items) {
+ this.items = items;
+ this.calls = [];
+ this.files = Object.fromEntries(items.flatMap(item => item.files.map(file => [String(file.id), clone(file)])));
+ this.states = Object.fromEntries(items.map(item => [itemKey(item), 'verified']));
+ this.logs = [];
+ this.statusWarning = 0;
+ this.statusAffected = null;
+ this.fileAffected = null;
+ }
+ async beginTransaction() {
+ this.calls.push({ transaction: 'begin' });
+ if (this.beginFailure) throw this.beginFailure;
+ this.saved = clone({ files: this.files, states: this.states, logs: this.logs });
+ }
+ async commit() {
+ this.calls.push({ transaction: 'commit' });
+ if (this.commitFailure) throw this.commitFailure;
+ this.saved = undefined;
+ }
+ async rollback() {
+ this.calls.push({ transaction: 'rollback' });
+ if (this.rollbackFailure) throw this.rollbackFailure;
+ if (this.saved) Object.assign(this, this.saved);
+ this.saved = undefined;
+ }
+ async execute(sql, params = []) {
+ this.calls.push({ sql, params });
+ if (sql.startsWith('SELECT id FROM')) {
+ if (this.sourceFailure) throw this.sourceFailure;
+ return [this.items.map(item => ({ id: item.contentId }))];
+ }
+ if (sql.startsWith('SELECT id,CASE')) {
+ return [this.items.map(item => ({ id: this.newerSource ? String(BigInt(item.contentId) + 1n) : item.contentId, expected_content_id: item.contentId }))];
+ }
+ if (sql.startsWith('SELECT id,config_id,path,OCTET_LENGTH')) {
+ return [this.items.map(item => ({ id: item.contentId, config_id: item.configId, path: item.path, size: item.size, sha256: item.sha256 }))];
+ }
+ if (sql.includes('information_schema.COLUMNS')) {
+ return [[...['id', 'config_id', 'path', 'url', 'type', 'size', 'deleted', 'update_time'].map(name => ({
+ name, extra: name === 'update_time' ? 'on update CURRENT_TIMESTAMP' : '', engine: 'InnoDB'
+ }))]];
+ }
+ if (sql.startsWith('UPDATE') && sql.includes('`infra_file`')) {
+ const [set, where] = sql.split(' WHERE ');
+ let cursor = (set.match(/\?/g) || []).length;
+ const groups = where.split(' OR ');
+ for (const group of groups) {
+ const pk = String(params[cursor]);
+ cursor += (group.match(/\?/g) || []).length;
+ const item = this.items.find(value => value.files.some(file => String(file.id) === pk));
+ assert.ok(item, 'file updates must target the selected batch');
+ this.files[pk] = { ...this.files[pk], config_id: config.targetConfigId, path: item.key, url: item.url, type: item.type, size: item.size };
+ }
+ return [{ affectedRows: this.fileAffected ?? groups.length, warningStatus: 0 }];
+ }
+ if (sql.startsWith('SELECT `')) {
+ return [params.map(pk => this.files[String(pk)])];
+ }
+ if (sql.startsWith('INSERT INTO')) {
+ if (this.journalFailure) throw this.journalFailure;
+ for (let index = 0; index < params.length; index += 7) this.logs.push(params.slice(index, index + 7));
+ return [{ affectedRows: params.length / 7, warningStatus: 0 }];
+ }
+ if (sql.startsWith('UPDATE') && sql.includes('`infra_file_oss_migration_items`')) {
+ assert.match(sql, /WHERE run_id=\? AND status='verified' AND item_key IN \(/);
+ assert.match(sql, /SET status='switched',error=NULL,updated_at=CURRENT_TIMESTAMP/);
+ assert.equal(params[0], config.runId);
+ let affected = 0;
+ for (const key of params.slice(1)) if (this.states[key] === 'verified') { this.states[key] = 'switched'; affected++; }
+ return [{ affectedRows: this.statusAffected ?? affected, warningStatus: this.statusWarning }];
+ }
+ throw new Error(`Unexpected query: ${sql}`);
+ }
+}
+
+const state = db => clone({ files: db.files, states: db.states, logs: db.logs });
+
+test('source range verification is the first operation after BEGIN, then files and guarded item status commit together', async () => {
+ const items = fixture();
+ const db = new TransitionDb(items);
+ const result = await switchFiles(db, config, items);
+ assert.deepEqual(Object.keys(result), ['sourceMs', 'writeMs', 'statusMs', 'commitMs']);
+ assert.ok(Object.values(result).every(value => Number.isFinite(value) && value >= 0));
+ assert.equal(db.calls[0].transaction, 'begin');
+ assert.match(db.calls[1].sql, /^SELECT id FROM .* WHERE id>=\? ORDER BY id FOR UPDATE$/);
+ assert.match(db.calls[2].sql, /^SELECT id,CASE/);
+ assert.match(db.calls[3].sql, /^SELECT id,config_id,path,OCTET_LENGTH/);
+ assert.ok(db.calls[4].sql.includes('information_schema.COLUMNS'));
+ assert.equal(db.calls.at(-1).transaction, 'commit');
+ assert.equal(db.files[items[0].files[0].id].url, items[0].url);
+ assert.equal(db.states[itemKey(items[0])], 'switched');
+ assert.equal(db.logs.length, 1);
+ const statusCall = db.calls.find(call => call.sql?.includes("SET status='switched'"));
+ assert.deepEqual(statusCall.params, [config.runId, itemKey(items[0])]);
+});
+
+test('more than 100 file records use bounded write chunks in one source-locked transaction', async () => {
+ const items = fixture({ count: 101 });
+ const db = new TransitionDb(items);
+ await switchFiles(db, config, items);
+ assert.deepEqual(db.calls.filter(call => call.transaction).map(call => call.transaction), ['begin', 'commit']);
+ assert.equal(db.calls.filter(call => call.sql?.startsWith('SELECT id FROM')).length, 1);
+ const writes = db.calls.filter(call => call.sql?.startsWith('UPDATE') && call.sql.includes('`infra_file`'));
+ assert.equal(writes.length, 2);
+ assert.equal(writes[0].sql.split(' WHERE ')[1].split(' OR ').length, 100);
+ assert.equal(writes[1].sql.split(' WHERE ')[1].split(' OR ').length, 1);
+ assert.equal(db.logs.length, 101);
+ assert.equal(Object.values(db.files).filter(file => file.url === items[0].url).length, 101);
+});
+
+test('orphan content is source-verified and status-switched even without any infra_file rows', async () => {
+ const items = fixture({ orphan: true });
+ const db = new TransitionDb(items);
+ await switchFiles(db, config, items);
+ assert.equal(db.states[itemKey(items[0])], 'switched');
+ assert.equal(db.logs.length, 0);
+ assert.equal(db.calls.filter(call => call.sql).length, 4);
+ assert.equal(db.calls.some(call => call.sql?.includes('information_schema')), false);
+});
+
+test('mixed regular and orphan items both participate in one guarded status update', async () => {
+ const items = fixture();
+ items.push({ ...fixture({ orphan: true })[0], contentId: '9007199254740994', path: 'orphan.png' });
+ const db = new TransitionDb(items);
+ await switchFiles(db, config, items);
+ assert.equal(db.logs.length, 1);
+ assert.deepEqual(Object.values(db.states), ['switched', 'switched']);
+});
+
+test('source errors are propagated unchanged after a successful rollback', async () => {
+ const items = fixture();
+ const db = new TransitionDb(items);
+ const failure = Object.assign(new Error('source changed'), { code: 'MIGRATION_SOURCE_CHANGED' });
+ db.sourceFailure = failure;
+ await assert.rejects(switchFiles(db, config, items), error => error === failure);
+ assert.deepEqual(db.calls.filter(call => call.transaction).map(call => call.transaction), ['begin', 'rollback']);
+ assert.equal(db.calls.some(call => call.sql?.startsWith('UPDATE')), false);
+});
+
+test('a newer source version stops the transaction before any file updates', async () => {
+ const items = fixture();
+ const db = new TransitionDb(items);
+ db.newerSource = true;
+ await assert.rejects(switchFiles(db, config, items), { code: 'MIGRATION_SOURCE_CHANGED' });
+ assert.equal(db.calls.some(call => call.sql?.startsWith('UPDATE')), false);
+ assert.equal(db.calls.at(-1).transaction, 'rollback');
+});
+
+test('file CAS mismatches roll back all file edits and preserve the conflict code', async () => {
+ const items = fixture();
+ const db = new TransitionDb(items);
+ const before = state(db);
+ db.fileAffected = 0;
+ await assert.rejects(switchFiles(db, config, items), { code: 'MIGRATION_CAS_CONFLICT' });
+ assert.deepEqual(state(db), before);
+ assert.equal(db.calls.some(call => call.sql?.includes("SET status='switched'")), false);
+});
+
+test('non-verified item status rolls back previously updated files and journal inserts', async () => {
+ const items = fixture();
+ const db = new TransitionDb(items);
+ db.states[itemKey(items[0])] = 'planned';
+ const before = state(db);
+ await assert.rejects(switchFiles(db, config, items), { code: 'MIGRATION_CAS_CONFLICT' });
+ assert.deepEqual(state(db), before);
+ assert.ok(db.calls.some(call => call.sql?.startsWith('INSERT INTO')));
+});
+
+test('item status SQL warnings roll back files, logs and item statuses', async () => {
+ const items = fixture();
+ const db = new TransitionDb(items);
+ const before = state(db);
+ db.statusWarning = 1;
+ await assert.rejects(switchFiles(db, config, items), { code: 'MIGRATION_SQL_WARNING' });
+ assert.deepEqual(state(db), before);
+});
+
+test('rollback failure is fatal and prohibits fallback while preserving both causes', async () => {
+ const items = fixture();
+ const db = new TransitionDb(items);
+ db.sourceFailure = Object.assign(new Error('source failed'), { code: 'MIGRATION_SOURCE_CHANGED' });
+ db.rollbackFailure = new Error('connection lost');
+ await assert.rejects(switchFiles(db, config, items), error => {
+ assert.equal(error.code, 'MIGRATION_ROLLBACK_FAILED');
+ assert.equal(error.fatal, true);
+ assert.equal(error.noFallback, true);
+ assert.equal(error.cause, db.sourceFailure);
+ assert.equal(error.rollbackError, db.rollbackFailure);
+ return true;
+ });
+});
+
+test('commit acknowledgement failure is fatal even if a rollback call subsequently succeeds', async () => {
+ const items = fixture();
+ const db = new TransitionDb(items);
+ db.commitFailure = Object.assign(new Error('commit transport failure'), { code: 'PROTOCOL_CONNECTION_LOST' });
+ await assert.rejects(switchFiles(db, config, items), error => {
+ assert.equal(error.code, 'PROTOCOL_CONNECTION_LOST');
+ assert.equal(error.fatal, true);
+ assert.equal(error.noFallback, true);
+ assert.equal(error.cause, db.commitFailure);
+ return true;
+ });
+ assert.deepEqual(db.calls.filter(call => call.transaction).map(call => call.transaction), ['begin', 'commit', 'rollback']);
+});
+
+test('BEGIN failure is propagated without performing any reads, writes or rollback', async () => {
+ const items = fixture();
+ const db = new TransitionDb(items);
+ db.beginFailure = new Error('cannot begin');
+ await assert.rejects(switchFiles(db, config, items), error => error === db.beginFailure);
+ assert.deepEqual(db.calls, [{ transaction: 'begin' }]);
+});
+
+test('empty batches are no-ops and malformed or duplicate input is rejected before BEGIN', async () => {
+ const db = new TransitionDb([]);
+ assert.deepEqual(await switchFiles(db, config, []), { sourceMs: 0, writeMs: 0, statusMs: 0, commitMs: 0 });
+ await assert.rejects(switchFiles(db, config, null), /数组/);
+ await assert.rejects(switchFiles(db, config, [{ ...fixture()[0], files: null }]), /files/);
+ await assert.rejects(switchFiles(db, config, [fixture()[0], fixture()[0]]), /重复源路径/);
+ const duplicateFiles = fixture();
+ duplicateFiles[0].files.push({ ...duplicateFiles[0].files[0] });
+ await assert.rejects(switchFiles(db, config, duplicateFiles), /重复文件记录/);
+ assert.equal(db.calls.length, 0);
+});
diff --git a/sql/mysql/recent/20261105_member_recent_cfg_unique.sql b/sql/mysql/recent/20261105_member_recent_cfg_unique.sql
new file mode 100644
index 0000000..83bc010
--- /dev/null
+++ b/sql/mysql/recent/20261105_member_recent_cfg_unique.sql
@@ -0,0 +1,70 @@
+-- 修复重复朋友圈配置，并保证同一用户最多只有一条有效配置。
+-- 适用 MySQL 5.7 / 8.0。执行前选择业务库，并备份 member_recent_cfg。
+-- 先部署兼容重复数据/唯一键冲突的代码；执行期间暂停配置写入（登录初始化、隐私设置、后台编辑）。
+-- DDL 会隐式提交；如中途失败，修复原因后可重跑，不能依靠 ROLLBACK 撤销整个脚本。
+-- 只逻辑删除较旧行，不重建默认配置，不修改保留行的隐私/通知选项。
+-- member_id 引用 member_user 的全局主键，唯一键不额外拼接可能为 NULL 的 tenant_id。
+
+-- 预检：完整展示重复配置，便于核对 home_hide/global_hide/privacy_secure/short_note/is_push 等实际字段。
+-- 保留顺序与服务查询一致：update_time 最新优先，时间相同取 id 最大；NULL 更新时间排在最后。
+SELECT cfg.*
+FROM `member_recent_cfg` cfg
+INNER JOIN (
+ SELECT `member_id`, COUNT(*) AS active_count
+ FROM `member_recent_cfg`
+ WHERE `deleted` = b'0'
+ GROUP BY `member_id`
+ HAVING COUNT(*) > 1
+) duplicated ON duplicated.`member_id` = cfg.`member_id`
+WHERE cfg.`deleted` = b'0'
+ORDER BY cfg.`member_id`, cfg.`update_time` DESC, cfg.`id` DESC;
+
+-- 先物化待删除 id，再更新，避免修改 update_time 影响同一批次的保留顺序。
+-- GROUP BY 阻止 MySQL 5.7/8 合并派生表，避免 1093 自查询更新错误。
+UPDATE `member_recent_cfg` obsolete
+INNER JOIN (
+ SELECT older.`id`
+ FROM `member_recent_cfg` older
+ INNER JOIN `member_recent_cfg` newer
+ ON newer.`member_id` = older.`member_id`
+ AND newer.`deleted` = b'0'
+ AND (
+ newer.`update_time` > older.`update_time`
+ OR (older.`update_time` IS NULL AND newer.`update_time` IS NOT NULL)
+ OR (newer.`update_time` <=> older.`update_time` AND newer.`id` > older.`id`)
+ )
+ WHERE older.`deleted` = b'0'
+ GROUP BY older.`id`
+) duplicate_ids ON duplicate_ids.`id` = obsolete.`id`
+SET obsolete.`deleted` = b'1', obsolete.`update_time` = CURRENT_TIMESTAMP
+WHERE obsolete.`deleted` = b'0';
+SELECT ROW_COUNT() AS logically_deleted_duplicate_configs;
+
+-- 已删除行生成 NULL，保留所有删除历史，同时允许以后重新创建配置。
+SET @cfg_add_column_sql = IF(
+ (SELECT COUNT(*) FROM information_schema.COLUMNS
+ WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'member_recent_cfg' AND COLUMN_NAME = 'active_unique_flag') = 0,
+ 'ALTER TABLE `member_recent_cfg` ADD COLUMN `active_unique_flag` tinyint GENERATED ALWAYS AS (CASE WHEN `deleted` = 0 THEN 1 ELSE NULL END) STORED COMMENT ''有效配置唯一标记，删除记录为NULL''',
+ 'SELECT ''active_unique_flag already exists'' AS migration_step'
+);
+PREPARE cfg_stmt FROM @cfg_add_column_sql;
+EXECUTE cfg_stmt;
+DEALLOCATE PREPARE cfg_stmt;
+
+SET @cfg_add_index_sql = IF(
+ (SELECT COUNT(*) FROM information_schema.STATISTICS
+ WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'member_recent_cfg' AND INDEX_NAME = 'uk_member_recent_cfg_active') = 0,
+ 'ALTER TABLE `member_recent_cfg` ADD UNIQUE KEY `uk_member_recent_cfg_active` (`member_id`, `active_unique_flag`)',
+ 'SELECT ''uk_member_recent_cfg_active already exists'' AS migration_step'
+);
+PREPARE cfg_stmt FROM @cfg_add_index_sql;
+EXECUTE cfg_stmt;
+DEALLOCATE PREPARE cfg_stmt;
+
+-- 验收：应无重复分组；索引 NON_UNIQUE 应为 0。
+SELECT `member_id`, COUNT(*) AS active_count
+FROM `member_recent_cfg`
+WHERE `deleted` = b'0'
+GROUP BY `member_id`
+HAVING COUNT(*) > 1;
+SHOW INDEX FROM `member_recent_cfg` WHERE Key_name = 'uk_member_recent_cfg_active';
diff --git a/sql/mysql/recent/20261105_member_recent_tag_unique.sql b/sql/mysql/recent/20261105_member_recent_tag_unique.sql
new file mode 100644
index 0000000..3928d31
--- /dev/null
+++ b/sql/mysql/recent/20261105_member_recent_tag_unique.sql
@@ -0,0 +1,63 @@
+-- 修复重复朋友圈标签，并保证同一用户、同一类型最多只有一条有效标签。
+-- 适用 MySQL 5.7 / 8.0。执行前选择业务库，并备份 member_recent_tag。
+-- 先部署兼容重复数据/唯一键冲突的代码；执行期间暂停标签写入（登录、后台编辑、认证、VIP及清理任务）。
+-- DDL 会隐式提交；如中途失败，修复原因后可重跑，不能依靠 ROLLBACK 撤销整个脚本。
+-- 仅逻辑删除重复项，保留最小 id；不修改保留行，不物理删除任何记录。
+
+-- member_id 引用 member_user 的全局主键，无需额外拼接租户维度。
+-- 历史标签的 tenant_id 可能为 NULL，将其直接纳入唯一键会让重复有效标签漏过约束。
+SET @tag_group_columns = '`member_id`, `tag_biz_type`';
+
+-- 预检：正常业务类型应为 1~5。NULL 类型不参与本次去重/约束，保留原始异常数据供单独核查。
+SELECT `id`, `member_id`, `tag_biz_type`
+FROM `member_recent_tag`
+WHERE `deleted` = b'0' AND (`tag_biz_type` IS NULL OR `tag_biz_type` NOT IN ('1', '2', '3', '4', '5'));
+
+SET @tag_duplicate_sql = CONCAT(
+ 'SELECT ', @tag_group_columns, ', MIN(`id`) AS keep_id, COUNT(*) AS active_count ',
+ 'FROM `member_recent_tag` WHERE `deleted` = b''0'' AND `tag_biz_type` IS NOT NULL ',
+ 'GROUP BY ', @tag_group_columns, ' HAVING COUNT(*) > 1'
+);
+PREPARE tag_stmt FROM @tag_duplicate_sql;
+EXECUTE tag_stmt;
+DEALLOCATE PREPARE tag_stmt;
+
+-- 逻辑删除同组的其余有效行。GROUP BY 派生表在 MySQL 5.7/8 中物化，避免 1093 自查询更新错误。
+SET @tag_cleanup_sql = CONCAT(
+ 'UPDATE `member_recent_tag` tag INNER JOIN (', @tag_duplicate_sql, ') kept ',
+ 'ON tag.`member_id` = kept.`member_id` AND tag.`tag_biz_type` = kept.`tag_biz_type` ',
+ 'SET tag.`deleted` = b''1'', tag.`update_time` = CURRENT_TIMESTAMP ',
+ 'WHERE tag.`deleted` = b''0'' AND tag.`id` <> kept.keep_id'
+);
+PREPARE tag_stmt FROM @tag_cleanup_sql;
+EXECUTE tag_stmt;
+SELECT ROW_COUNT() AS logically_deleted_duplicate_tags;
+DEALLOCATE PREPARE tag_stmt;
+
+-- 已删除行生成 NULL，利用 MySQL 唯一索引允许多个 NULL 的语义保留全部删除历史。
+-- 不能直接使用 (member_id, tag_biz_type, deleted) 唯一键，否则第二次删除同类型标签会冲突。
+SET @tag_add_column_sql = IF(
+ (SELECT COUNT(*) FROM information_schema.COLUMNS
+ WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'member_recent_tag' AND COLUMN_NAME = 'active_unique_flag') = 0,
+ 'ALTER TABLE `member_recent_tag` ADD COLUMN `active_unique_flag` tinyint GENERATED ALWAYS AS (CASE WHEN `deleted` = 0 THEN 1 ELSE NULL END) STORED COMMENT ''有效标签唯一标记，删除记录为NULL''',
+ 'SELECT ''active_unique_flag already exists'' AS migration_step'
+);
+PREPARE tag_stmt FROM @tag_add_column_sql;
+EXECUTE tag_stmt;
+DEALLOCATE PREPARE tag_stmt;
+
+SET @tag_add_index_sql = IF(
+ (SELECT COUNT(*) FROM information_schema.STATISTICS
+ WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'member_recent_tag' AND INDEX_NAME = 'uk_member_recent_tag_active') = 0,
+ CONCAT('ALTER TABLE `member_recent_tag` ADD UNIQUE KEY `uk_member_recent_tag_active` (', @tag_group_columns, ', `active_unique_flag`)'),
+ 'SELECT ''uk_member_recent_tag_active already exists'' AS migration_step'
+);
+PREPARE tag_stmt FROM @tag_add_index_sql;
+EXECUTE tag_stmt;
+DEALLOCATE PREPARE tag_stmt;
+
+-- 验收：应无重复分组；索引 NON_UNIQUE 应为 0；正常新增标签无需写入生成列。
+PREPARE tag_stmt FROM @tag_duplicate_sql;
+EXECUTE tag_stmt;
+DEALLOCATE PREPARE tag_stmt;
+SHOW INDEX FROM `member_recent_tag` WHERE Key_name = 'uk_member_recent_tag_active';
diff --git a/sql/mysql/recent/member_recent.sql b/sql/mysql/recent/member_recent.sql
index a1731f3..a47ac02 100644
--- a/sql/mysql/recent/member_recent.sql
+++ b/sql/mysql/recent/member_recent.sql
@@ -169,7 +169,9 @@ CREATE TABLE `member_recent_tag`
`update_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '更新时间',
`deleted` bit(1) NOT NULL DEFAULT b'0' COMMENT '是否删除',
`tenant_id` bigint NOT NULL DEFAULT 0 COMMENT '租户编号',
- PRIMARY KEY (`id`) USING BTREE
+ `active_unique_flag` tinyint GENERATED ALWAYS AS (CASE WHEN `deleted` = 0 THEN 1 ELSE NULL END) STORED COMMENT '有效标签唯一标记，删除记录为NULL',
+ PRIMARY KEY (`id`) USING BTREE,
+ UNIQUE KEY `uk_member_recent_tag_active` (`member_id`, `tag_biz_type`, `active_unique_flag`)
) ENGINE = InnoDB AUTO_INCREMENT = 32 CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin COMMENT = '朋友圈用户tag';
-- ----------------------------
@@ -189,7 +191,9 @@ CREATE TABLE `member_recent_cfg`
`update_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '更新时间',
`deleted` bit(1) NOT NULL DEFAULT b'0' COMMENT '是否删除',
`tenant_id` bigint NOT NULL DEFAULT 0 COMMENT '租户编号',
- PRIMARY KEY (`id`) USING BTREE
+ `active_unique_flag` tinyint GENERATED ALWAYS AS (CASE WHEN `deleted` = 0 THEN 1 ELSE NULL END) STORED COMMENT '有效配置唯一标记，删除记录为NULL',
+ PRIMARY KEY (`id`) USING BTREE,
+ UNIQUE KEY `uk_member_recent_cfg_active` (`member_id`, `active_unique_flag`)
) ENGINE = InnoDB AUTO_INCREMENT = 32 CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin COMMENT = '用户朋友圈相关配置';
-- ----------------------------
diff --git a/yudao-module-member/yudao-module-member-biz/src/main/java/cn/iocoder/yudao/module/member/service/user/MemberRecentCfgServiceImpl.java b/yudao-module-member/yudao-module-member-biz/src/main/java/cn/iocoder/yudao/module/member/service/user/MemberRecentCfgServiceImpl.java
index 95a71b3..89d8f70 100644
--- a/yudao-module-member/yudao-module-member-biz/src/main/java/cn/iocoder/yudao/module/member/service/user/MemberRecentCfgServiceImpl.java
+++ b/yudao-module-member/yudao-module-member-biz/src/main/java/cn/iocoder/yudao/module/member/service/user/MemberRecentCfgServiceImpl.java
@@ -11,10 +11,12 @@ import cn.iocoder.yudao.module.member.dal.redis.recent.MemberRecentRedisDAO;
import cn.iocoder.yudao.module.member.enums.MemberErrorCode;
import cn.iocoder.yudao.module.member.service.opentype.OpenTypeService;
import lombok.SneakyThrows;
+import org.springframework.dao.DuplicateKeyException;
import org.springframework.stereotype.Service;
import javax.annotation.Resource;
+import org.springframework.transaction.annotation.Propagation;
import org.springframework.transaction.annotation.Transactional;
import org.springframework.validation.annotation.Validated;
@@ -61,17 +63,23 @@ public class MemberRecentCfgServiceImpl implements MemberRecentCfgService {
* 创建或新增配置
*/
@Override
+ @SneakyThrows
+ @Transactional(propagation = Propagation.NOT_SUPPORTED)
public MemberRecentCfgDO createOrUpdate(MemberRecentCfgReqVo dto) {
- MemberRecentCfgDO detail = null;
- if (ObjectUtil.isNotEmpty(dto.getMemberId())) {
- detail = recentCfgMapper.selectOne(new LambdaQueryWrapperX()
- .eq(MemberRecentCfgDO::getMemberId, dto.getMemberId())
- );
- }
- MemberRecentCfgDO entity = BeanUtil.toBean(dto, MemberRecentCfgDO.class);
- if (ObjectUtil.isEmpty(detail)) {
- recentCfgMapper.insert(entity);
- } else {
+ return memberRecentRedisDAO.lock(dto.getMemberId(), UPDATE_TIMEOUT_MILLIS, () -> {
+ MemberRecentCfgDO detail = selectCurrentConfig(dto.getMemberId(), true);
+ MemberRecentCfgDO entity = BeanUtil.toBean(dto, MemberRecentCfgDO.class);
+ if (detail == null) {
+ try {
+ recentCfgMapper.insert(entity);
+ return entity;
+ } catch (DuplicateKeyException ex) {
+ detail = selectCurrentConfig(dto.getMemberId(), true);
+ if (detail == null) {
+ throw ex;
+ }
+ }
+ }
// 判断是否修改了`全局隐身`和`首页隐身`
if (ObjectUtil.isNotEmpty(dto.getHomeHide())) {
if (dto.getHomeHide()) {
@@ -83,8 +91,8 @@ public class MemberRecentCfgServiceImpl implements MemberRecentCfgService {
}
entity.setId(detail.getId());
recentCfgMapper.updateById(entity);
- }
- return entity;
+ return entity;
+ });
}
/**
@@ -92,26 +100,48 @@ public class MemberRecentCfgServiceImpl implements MemberRecentCfgService {
*/
@Override
@SneakyThrows
- @Transactional(rollbackFor = Exception.class)
+ @Transactional(propagation = Propagation.NOT_SUPPORTED)
public MemberRecentCfgDO getOrCreate(Long memberId) {
- MemberRecentCfgDO detail = recentCfgMapper.selectOne(new LambdaQueryWrapperX()
- .eq(MemberRecentCfgDO::getMemberId, memberId)
- );
+ MemberRecentCfgDO detail = selectCurrentConfig(memberId, false);
if (ObjectUtil.isNotEmpty(detail)) {
return detail;
}
+ // 单条写入自动提交后才解锁；不能让外层事务保留旧快照或在解锁后提交。
return memberRecentRedisDAO.lock(memberId, UPDATE_TIMEOUT_MILLIS, () -> {
+ MemberRecentCfgDO existing = selectCurrentConfig(memberId, true);
+ if (existing != null) {
+ return existing;
+ }
MemberRecentCfgDO insertObj = MemberRecentCfgDO.builder()
.memberId(memberId)
.homeHide(false)
.privacySecure(false)
.shortNote(false)
.build();
- recentCfgMapper.insert(insertObj);
- return insertObj;
+ try {
+ recentCfgMapper.insert(insertObj);
+ return insertObj;
+ } catch (DuplicateKeyException ex) {
+ MemberRecentCfgDO concurrent = selectCurrentConfig(memberId, true);
+ if (concurrent == null) {
+ throw ex;
+ }
+ return concurrent;
+ }
});
}
+ private MemberRecentCfgDO selectCurrentConfig(Long memberId, boolean currentRead) {
+ // 历史重复配置优先保留最近修改的隐私设置，不用默认值覆盖用户选择。
+ // NOT_SUPPORTED 仍可能绑定 MyBatis SqlSession。锁内使用不同 SQL 做当前读，
+ // 避免复用锁外查询缓存的“无记录”；写入仍在当前无事务连接上自动提交。
+ return recentCfgMapper.selectOne(new LambdaQueryWrapperX()
+ .eq(MemberRecentCfgDO::getMemberId, memberId)
+ .orderByDesc(MemberRecentCfgDO::getUpdateTime)
+ .orderByDesc(MemberRecentCfgDO::getId)
+ .last(currentRead ? "LIMIT 1 FOR UPDATE" : "LIMIT 1"));
+ }
+
/**
* 查询首页隐身的用户ids
*/
@@ -134,4 +164,4 @@ public class MemberRecentCfgServiceImpl implements MemberRecentCfgService {
public int del(Long menberId) {
return recentCfgMapper.deleteById(menberId);
}
-}
\ No newline at end of file
+}
diff --git a/yudao-module-member/yudao-module-member-biz/src/main/java/cn/iocoder/yudao/module/member/service/user/MemberRecentTagServiceImp.java b/yudao-module-member/yudao-module-member-biz/src/main/java/cn/iocoder/yudao/module/member/service/user/MemberRecentTagServiceImp.java
index 2a3c08d..276e5ef 100644
--- a/yudao-module-member/yudao-module-member-biz/src/main/java/cn/iocoder/yudao/module/member/service/user/MemberRecentTagServiceImp.java
+++ b/yudao-module-member/yudao-module-member-biz/src/main/java/cn/iocoder/yudao/module/member/service/user/MemberRecentTagServiceImp.java
@@ -3,27 +3,22 @@ package cn.iocoder.yudao.module.member.service.user;
import cn.hutool.core.util.ObjectUtil;
import cn.iocoder.yudao.framework.mybatis.core.query.LambdaQueryWrapperX;
-import cn.iocoder.yudao.framework.tenant.core.context.TenantContextHolder;
import cn.iocoder.yudao.module.member.dal.dataobject.config.MemberConfigDO;
import cn.iocoder.yudao.module.member.dal.dataobject.user.MemberRecentTagDO;
import cn.iocoder.yudao.module.member.dal.dataobject.user.MemberUserDO;
-import cn.iocoder.yudao.module.member.dal.mysql.opentype.OpenTypeMapper;
-import cn.iocoder.yudao.module.member.dal.mysql.opentype.OpenTypeUserMapper;
import cn.iocoder.yudao.module.member.dal.mysql.user.MemberRecentTagMapper;
import cn.iocoder.yudao.module.member.dal.mysql.user.MemberUserMapper;
import cn.iocoder.yudao.module.member.enums.tag.MemberRecentTagEnum;
import cn.iocoder.yudao.module.member.service.config.MemberConfigService;
-import cn.iocoder.yudao.module.member.service.opentype.BO.OpenTypeExtBO;
import cn.iocoder.yudao.module.member.service.opentype.OpenTypeService;
import cn.iocoder.yudao.module.member.service.user.BO.MemberTagExtBO;
import lombok.extern.slf4j.Slf4j;
import org.springframework.context.annotation.Lazy;
-import org.springframework.security.core.parameters.P;
+import org.springframework.dao.DuplicateKeyException;
import org.springframework.stereotype.Service;
import javax.annotation.Resource;
import javax.validation.Valid;
-import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.HashMap;
@@ -54,24 +49,7 @@ public class MemberRecentTagServiceImp implements MemberRecentTagService {
*/
@Override
public void setPersonTag(Long authMemberId, boolean isAddTag) {
- MemberRecentTagDO detail = memberRecentTagMapper.selectOne(new LambdaQueryWrapperX()
- .eq(MemberRecentTagDO::getMemberId, authMemberId)
- .eq(MemberRecentTagDO::getTagBizType, MemberRecentTagEnum.TRUE_PERSON_AUTH.getType())
- );
- MemberRecentTagDO obj = new MemberRecentTagDO();
- obj.setTagBizId(null);
- obj.setMemberId(authMemberId);
- obj.setTagBizType(MemberRecentTagEnum.TRUE_PERSON_AUTH.getType());
- obj.setTagTitle(MemberRecentTagEnum.TRUE_PERSON_AUTH.getDescription());
- if (ObjectUtil.isEmpty(detail)) {
- if (isAddTag) {
- memberRecentTagMapper.insert(obj);
- }
- } else {
- if (!isAddTag) {
- memberRecentTagMapper.deleteById(detail.getId());
- }
- }
+ synchronizeTag(authMemberId, MemberRecentTagEnum.TRUE_PERSON_AUTH, isAddTag);
}
/**
@@ -124,32 +102,7 @@ public class MemberRecentTagServiceImp implements MemberRecentTagService {
// 是否是新人
boolean newPerson = memberUser.isNewPerson(newPersonDays);
- MemberRecentTagDO tag = memberRecentTagMapper.selectOne(new LambdaQueryWrapperX()
- .eq(MemberRecentTagDO::getMemberId, memberId)
- .eq(MemberRecentTagDO::getTagBizType, MemberRecentTagEnum.NEW_PERSON.getType())
- );
-
- // 是新人
- if (newPerson) {
- if (ObjectUtil.isEmpty(tag)) {
- MemberRecentTagDO obj = new MemberRecentTagDO();
- obj.setTagBizId(null);
- obj.setMemberId(memberId);
- obj.setTagBizType(MemberRecentTagEnum.NEW_PERSON.getType());
- obj.setTagTitle(MemberRecentTagEnum.NEW_PERSON.getDescription());
- memberRecentTagMapper.insert(obj);
- return true;
- }
- return false;
- }
- // 不是新人
- else {
- if (ObjectUtil.isNotEmpty(tag)) {
- memberRecentTagMapper.deleteById(tag.getId());
- return true;
- }
- return false;
- }
+ return synchronizeTag(memberId, MemberRecentTagEnum.NEW_PERSON, newPerson);
}
/**
@@ -162,31 +115,40 @@ public class MemberRecentTagServiceImp implements MemberRecentTagService {
if (ObjectUtil.isEmpty(memberUser)) return false;
// 现在是否还是vip
boolean isVip = openTypeService.isUserOpenType(memberId);
- MemberRecentTagDO tag = memberRecentTagMapper.selectOne(new LambdaQueryWrapperX()
+ return synchronizeTag(memberId, MemberRecentTagEnum.VIP, isVip);
+ }
+
+ /**
+ * 同步动态标签。历史重复数据按存在处理，失效时逻辑删除全部匹配记录。
+ */
+ private boolean synchronizeTag(Long memberId, MemberRecentTagEnum tagType, boolean enabled) {
+ if (!enabled) {
+ return memberRecentTagMapper.delete(tagQuery(memberId, tagType)) > 0;
+ }
+ if (memberRecentTagMapper.selectCount(tagQuery(memberId, tagType)) > 0) {
+ return false;
+ }
+ MemberRecentTagDO tag = MemberRecentTagDO.builder()
+ .memberId(memberId)
+ .tagBizType(tagType.getType())
+ .tagTitle(tagType.getDescription())
+ .build();
+ try {
+ return memberRecentTagMapper.insert(tag) > 0;
+ } catch (DuplicateKeyException ex) {
+ // 当前读确认并发插入结果，避免外层 REPEATABLE READ 事务仍读到旧快照。
+ if (ObjectUtil.isNotEmpty(memberRecentTagMapper.selectList(tagQuery(memberId, tagType)
+ .select(MemberRecentTagDO::getId).last("LIMIT 1 FOR UPDATE")))) {
+ return false;
+ }
+ throw ex;
+ }
+ }
+
+ private LambdaQueryWrapperX tagQuery(Long memberId, MemberRecentTagEnum tagType) {
+ return new LambdaQueryWrapperX()
.eq(MemberRecentTagDO::getMemberId, memberId)
- .eq(MemberRecentTagDO::getTagBizType, MemberRecentTagEnum.VIP.getType())
- );
- // 是vip
- if (isVip) {
- if (ObjectUtil.isEmpty(tag)) {
- MemberRecentTagDO obj = new MemberRecentTagDO();
- obj.setTagBizId(null);
- obj.setMemberId(memberId);
- obj.setTagBizType(MemberRecentTagEnum.VIP.getType());
- obj.setTagTitle(MemberRecentTagEnum.VIP.getDescription());
- memberRecentTagMapper.insert(obj);
- return true;
- }
- return false;
- }
- // 不是vip
- else {
- if (ObjectUtil.isNotEmpty(tag)) {
- memberRecentTagMapper.deleteById(tag.getId());
- return true;
- }
- return false;
- }
+ .eq(MemberRecentTagDO::getTagBizType, tagType.getType());
}
/**
diff --git a/yudao-module-member/yudao-module-member-biz/src/main/java/cn/iocoder/yudao/module/member/service/user/MemberUserServiceImpl.java b/yudao-module-member/yudao-module-member-biz/src/main/java/cn/iocoder/yudao/module/member/service/user/MemberUserServiceImpl.java
index d30f830..2a82233 100644
--- a/yudao-module-member/yudao-module-member-biz/src/main/java/cn/iocoder/yudao/module/member/service/user/MemberUserServiceImpl.java
+++ b/yudao-module-member/yudao-module-member-biz/src/main/java/cn/iocoder/yudao/module/member/service/user/MemberUserServiceImpl.java
@@ -442,7 +442,8 @@ public class MemberUserServiceImpl implements MemberUserService {
);
// 添加标签
if (ObjectUtil.isNotEmpty(updateReqVO.getMemberRecentTagList())) {
- List tags = updateReqVO.getMemberRecentTagList().stream().map(v -> {
+ // 历史重复标签或前端重试都可能带来重复类型，保存时按业务类型去重。
+ List tags = updateReqVO.getMemberRecentTagList().stream().distinct().map(v -> {
MemberRecentTagEnum tagEnum = MemberRecentTagEnum.valueOfType(v);
if (ObjectUtil.isEmpty(tagEnum)) return null;
MemberRecentTagDO entity = new MemberRecentTagDO();
diff --git a/yudao-module-member/yudao-module-member-biz/src/test/java/cn/iocoder/yudao/module/member/service/auth/MemberAuthServiceImplSmsRegisterDbTest.java b/yudao-module-member/yudao-module-member-biz/src/test/java/cn/iocoder/yudao/module/member/service/auth/MemberAuthServiceImplSmsRegisterDbTest.java
new file mode 100644
index 0000000..a00f260
--- /dev/null
+++ b/yudao-module-member/yudao-module-member-biz/src/test/java/cn/iocoder/yudao/module/member/service/auth/MemberAuthServiceImplSmsRegisterDbTest.java
@@ -0,0 +1,144 @@
+package cn.iocoder.yudao.module.member.service.auth;
+
+import cn.iocoder.yudao.framework.common.enums.CommonStatusEnum;
+import cn.iocoder.yudao.framework.test.core.ut.BaseDbUnitTest;
+import cn.iocoder.yudao.module.member.client.service.ChatApi;
+import cn.iocoder.yudao.module.member.client.vo.chat.user.ChatLoginReqVO;
+import cn.iocoder.yudao.module.member.client.vo.chat.user.ChatLoginRespVO;
+import cn.iocoder.yudao.module.member.controller.app.auth.vo.AppAuthLoginRespVO;
+import cn.iocoder.yudao.module.member.controller.app.auth.vo.AppAuthRegisterRespVO;
+import cn.iocoder.yudao.module.member.dal.dataobject.config.MemberConfigDO;
+import cn.iocoder.yudao.module.member.dal.dataobject.user.MemberUserDO;
+import cn.iocoder.yudao.module.member.dal.mysql.user.MemberUserMapper;
+import cn.iocoder.yudao.module.member.dal.redis.recent.MemberRecentRedisDAO;
+import cn.iocoder.yudao.module.member.service.config.MemberConfigService;
+import cn.iocoder.yudao.module.member.service.opentype.OpenTypeService;
+import cn.iocoder.yudao.module.member.service.user.MemberRecentCfgService;
+import cn.iocoder.yudao.module.member.service.user.MemberRecentCfgServiceImpl;
+import cn.iocoder.yudao.module.member.service.user.MemberRecentTagService;
+import cn.iocoder.yudao.module.member.service.user.MemberRecentTagServiceImp;
+import cn.iocoder.yudao.module.member.service.user.MemberUserService;
+import cn.iocoder.yudao.module.system.api.logger.LoginLogApi;
+import cn.iocoder.yudao.module.system.api.oauth2.OAuth2TokenApi;
+import cn.iocoder.yudao.module.system.api.oauth2.dto.OAuth2AccessTokenCreateReqDTO;
+import cn.iocoder.yudao.module.system.api.oauth2.dto.OAuth2AccessTokenRespDTO;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.springframework.boot.test.mock.mockito.MockBean;
+import org.springframework.context.annotation.Import;
+import org.springframework.jdbc.core.JdbcTemplate;
+import org.springframework.mock.web.MockHttpServletRequest;
+import org.springframework.test.context.TestPropertySource;
+import org.springframework.test.util.ReflectionTestUtils;
+import org.springframework.web.context.request.RequestContextHolder;
+import org.springframework.web.context.request.ServletRequestAttributes;
+
+import javax.annotation.Resource;
+import javax.sql.DataSource;
+import java.time.LocalDateTime;
+
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.anyInt;
+import static org.mockito.ArgumentMatchers.anyString;
+import static org.mockito.ArgumentMatchers.eq;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.times;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+@Import({MemberRecentTagServiceImp.class, MemberRecentCfgServiceImpl.class})
+@TestPropertySource(properties = {
+ "spring.datasource.url=jdbc:h2:mem:member_sms_register_regression;MODE=MYSQL;DATABASE_TO_UPPER=false;NON_KEYWORDS=value;",
+ "spring.sql.init.schema-locations=classpath:/sql/create_tables.sql,classpath:/sql/member_recent_tag_login_schema.sql,classpath:/sql/member_recent_cfg_schema.sql"
+})
+class MemberAuthServiceImplSmsRegisterDbTest extends BaseDbUnitTest {
+
+ @Resource
+ private MemberRecentTagService memberRecentTagService;
+ @Resource
+ private MemberRecentCfgService memberRecentCfgService;
+ @Resource
+ private DataSource dataSource;
+ @MockBean
+ private MemberUserMapper memberUserMapper;
+ @MockBean
+ private MemberConfigService memberConfigService;
+ @MockBean
+ private OpenTypeService openTypeService;
+ @MockBean
+ private MemberRecentRedisDAO memberRecentRedisDAO;
+
+ private JdbcTemplate jdbcTemplate;
+
+ @BeforeEach
+ void setUp() {
+ jdbcTemplate = new JdbcTemplate(dataSource);
+ MockHttpServletRequest request = new MockHttpServletRequest();
+ request.setRemoteAddr("127.0.0.1");
+ request.addHeader("terminal", "1");
+ RequestContextHolder.setRequestAttributes(new ServletRequestAttributes(request));
+ }
+
+ @AfterEach
+ void tearDown() {
+ RequestContextHolder.resetRequestAttributes();
+ jdbcTemplate.update("DELETE FROM member_recent_tag");
+ jdbcTemplate.update("DELETE FROM member_recent_cfg");
+ }
+
+ @Test
+ void smsRegisterReturnsTokensWhenBothNewPersonTagsAndConfigsContainDuplicates() {
+ Long userId = 740L;
+ String mobile = "18900520049";
+ MemberUserDO user = new MemberUserDO().setId(userId).setMobile(mobile)
+ .setStatus(CommonStatusEnum.ENABLE.getStatus());
+ user.setCreateTime(LocalDateTime.now().minusDays(1));
+ when(memberUserMapper.selectById(userId)).thenReturn(user);
+ MemberConfigDO config = new MemberConfigDO();
+ config.setNewPersonDays(15);
+ when(memberConfigService.getConfig()).thenReturn(config);
+ jdbcTemplate.update("INSERT INTO member_recent_tag (member_id, tag_biz_type, tag_title)"
+ + " VALUES (740, '2', '新人'), (740, '2', '新人')");
+ jdbcTemplate.update("INSERT INTO member_recent_cfg (member_id, home_hide, privacy_secure, update_time)"
+ + " VALUES (740, FALSE, FALSE, '2026-11-01 10:00:00'),"
+ + " (740, TRUE, TRUE, '2026-11-02 10:00:00')");
+
+ MemberUserService userService = mock(MemberUserService.class);
+ OAuth2TokenApi oauth2TokenApi = mock(OAuth2TokenApi.class);
+ ChatApi chatApi = mock(ChatApi.class);
+ when(userService.getUserByMobile(mobile)).thenReturn(user);
+ when(userService.createUserIfAbsent(eq(mobile), anyString(), anyInt())).thenReturn(user);
+ when(userService.syncChatUser(userId)).thenReturn(true);
+ when(oauth2TokenApi.createAccessToken(any(OAuth2AccessTokenCreateReqDTO.class)))
+ .thenReturn(new OAuth2AccessTokenRespDTO().setUserId(userId)
+ .setAccessToken("access-token").setRefreshToken("refresh-token")
+ .setExpiresTime(LocalDateTime.now().plusDays(1)));
+ ChatLoginRespVO chatToken = new ChatLoginRespVO();
+ chatToken.setAccessToken("chat-access-token");
+ chatToken.setRefreshToken("chat-refresh-token");
+ when(chatApi.login(any(ChatLoginReqVO.class))).thenReturn(chatToken);
+ MemberAuthServiceImpl authService = new MemberAuthServiceImpl();
+ ReflectionTestUtils.setField(authService, "userService", userService);
+ ReflectionTestUtils.setField(authService, "memberRecentTagService", memberRecentTagService);
+ ReflectionTestUtils.setField(authService, "memberRecentCfgService", memberRecentCfgService);
+ ReflectionTestUtils.setField(authService, "oauth2TokenApi", oauth2TokenApi);
+ ReflectionTestUtils.setField(authService, "loginLogApi", mock(LoginLogApi.class));
+ ReflectionTestUtils.setField(authService, "chatApi", chatApi);
+ AppAuthRegisterRespVO request = AppAuthRegisterRespVO.builder().mobile(mobile).code("8855").build();
+
+ AppAuthLoginRespVO result = authService.smsRegister(request);
+ AppAuthLoginRespVO retried = authService.smsRegister(request);
+
+ assertThat(result.getUserId()).isEqualTo(userId);
+ assertThat(result.getAccessToken()).isEqualTo("access-token");
+ assertThat(result.getRefreshToken()).isEqualTo("refresh-token");
+ assertThat(result.getChatToken().getAccessToken()).isEqualTo("chat-access-token");
+ assertThat(retried.getAccessToken()).isEqualTo("access-token");
+ assertThat(jdbcTemplate.queryForObject("SELECT COUNT(*) FROM member_recent_tag", Integer.class)).isEqualTo(2);
+ assertThat(jdbcTemplate.queryForObject("SELECT COUNT(*) FROM member_recent_cfg", Integer.class)).isEqualTo(2);
+ assertThat(memberRecentCfgService.getOrCreate(userId).getPrivacySecure()).isTrue();
+ verify(userService, times(2)).updateUserLogin(userId, "127.0.0.1");
+ }
+}
diff --git a/yudao-module-member/yudao-module-member-biz/src/test/java/cn/iocoder/yudao/module/member/service/user/MemberRecentCfgServiceImplDbTest.java b/yudao-module-member/yudao-module-member-biz/src/test/java/cn/iocoder/yudao/module/member/service/user/MemberRecentCfgServiceImplDbTest.java
new file mode 100644
index 0000000..c1e349d
--- /dev/null
+++ b/yudao-module-member/yudao-module-member-biz/src/test/java/cn/iocoder/yudao/module/member/service/user/MemberRecentCfgServiceImplDbTest.java
@@ -0,0 +1,183 @@
+package cn.iocoder.yudao.module.member.service.user;
+
+import cn.iocoder.yudao.framework.test.core.ut.BaseDbUnitTest;
+import cn.iocoder.yudao.module.member.controller.app.user.vo.recentcfg.MemberRecentCfgReqVo;
+import cn.iocoder.yudao.module.member.dal.dataobject.user.MemberRecentCfgDO;
+import cn.iocoder.yudao.module.member.dal.mysql.user.RecentCfgMapper;
+import cn.iocoder.yudao.module.member.dal.redis.recent.MemberRecentRedisDAO;
+import cn.iocoder.yudao.module.member.service.opentype.OpenTypeService;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.springframework.boot.test.mock.mockito.MockBean;
+import org.springframework.context.annotation.Import;
+import org.springframework.jdbc.core.JdbcTemplate;
+import org.springframework.test.context.TestPropertySource;
+import org.springframework.transaction.PlatformTransactionManager;
+import org.springframework.transaction.support.TransactionSynchronizationManager;
+import org.springframework.transaction.support.TransactionTemplate;
+
+import javax.annotation.Resource;
+import javax.sql.DataSource;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.Callable;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.Future;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.locks.ReentrantLock;
+
+import static cn.iocoder.yudao.framework.test.core.util.AssertUtils.assertServiceException;
+import static cn.iocoder.yudao.module.member.enums.MemberErrorCode.USER_UPDATE_CONFIG_ERROR;
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.anyLong;
+import static org.mockito.Mockito.doAnswer;
+import static org.mockito.Mockito.when;
+
+@Import(MemberRecentCfgServiceImpl.class)
+@TestPropertySource(properties = {
+ "spring.datasource.url=jdbc:h2:mem:member_recent_cfg_regression;MODE=MYSQL;DATABASE_TO_UPPER=false;NON_KEYWORDS=value;",
+ "spring.sql.init.schema-locations=classpath:/sql/create_tables.sql,classpath:/sql/member_recent_cfg_schema.sql"
+})
+class MemberRecentCfgServiceImplDbTest extends BaseDbUnitTest {
+
+ @Resource
+ private MemberRecentCfgService memberRecentCfgService;
+ @Resource
+ private RecentCfgMapper recentCfgMapper;
+ @Resource
+ private DataSource dataSource;
+ @Resource
+ private PlatformTransactionManager transactionManager;
+ @MockBean
+ private MemberRecentRedisDAO memberRecentRedisDAO;
+ @MockBean
+ private OpenTypeService openTypeService;
+
+ private JdbcTemplate jdbc;
+
+ @BeforeEach
+ void setUp() throws Exception {
+ jdbc = new JdbcTemplate(dataSource);
+ ReentrantLock lock = new ReentrantLock();
+ when(memberRecentRedisDAO.lock(anyLong(), anyLong(), any())).thenAnswer(invocation -> {
+ lock.lock();
+ try {
+ assertThat(TransactionSynchronizationManager.isActualTransactionActive()).isFalse();
+ return ((Callable) invocation.getArgument(2)).call();
+ } finally {
+ lock.unlock();
+ }
+ });
+ }
+
+ @AfterEach
+ void cleanConfigs() {
+ jdbc.update("DELETE FROM member_recent_cfg");
+ }
+
+ @Test
+ void duplicateConfigsUseLatestPrivacySettingsAndIgnoreDeletedRows() {
+ insertConfig(812L, "2026-11-05 01:21:20", false, false);
+ insertConfig(813L, "2026-11-05 01:21:20", true, false);
+ insertConfig(814L, "2026-11-05 02:00:00", false, true);
+
+ MemberRecentCfgDO config = memberRecentCfgService.getOrCreate(740L);
+
+ assertThat(config.getId()).isEqualTo(813L);
+ assertThat(config.getHomeHide()).isTrue();
+ assertThat(config.getPrivacySecure()).isTrue();
+ assertThat(jdbc.queryForObject("SELECT COUNT(*) FROM member_recent_cfg", Integer.class)).isEqualTo(3);
+ }
+
+ @Test
+ void duplicateConfigsPreferUpdateTimeBeforeId() {
+ insertConfig(812L, "2026-11-05 02:00:00", true, false);
+ insertConfig(813L, "2026-11-05 01:21:20", false, false);
+
+ assertThat(memberRecentCfgService.getOrCreate(740L).getId()).isEqualTo(812L);
+ }
+
+ @Test
+ void updateUsesCurrentDuplicateConfigAndRetainsVipCheck() {
+ insertConfig(812L, "2026-11-05 01:21:20", false, false);
+ insertConfig(813L, "2026-11-05 01:21:20", false, false);
+ MemberRecentCfgReqVo request = request(true);
+
+ assertServiceException(() -> memberRecentCfgService.createOrUpdate(request),
+ USER_UPDATE_CONFIG_ERROR, "全局隐身或首页隐身");
+ assertThat(recentCfgMapper.selectById(813L).getHomeHide()).isFalse();
+
+ when(openTypeService.isUserOpenType()).thenReturn(true);
+ assertThat(memberRecentCfgService.createOrUpdate(request).getId()).isEqualTo(813L);
+ assertThat(recentCfgMapper.selectById(813L).getHomeHide()).isTrue();
+ assertThat(recentCfgMapper.selectById(812L).getHomeHide()).isFalse();
+ }
+
+ @Test
+ void concurrentFirstLoginsCreateOneConfig() throws Exception {
+ // 强制所有请求都完成锁外的“无记录”查询，再允许任一请求进入锁，
+ // 覆盖 MyBatis 会话缓存空结果导致锁内复查失效的实际竞态。
+ CountDownLatch allInitialReadsMissed = new CountDownLatch(8);
+ ReentrantLock lock = new ReentrantLock();
+ doAnswer(invocation -> {
+ allInitialReadsMissed.countDown();
+ assertThat(allInitialReadsMissed.await(10, TimeUnit.SECONDS)).isTrue();
+ lock.lock();
+ try {
+ assertThat(TransactionSynchronizationManager.isActualTransactionActive()).isFalse();
+ return ((Callable) invocation.getArgument(2)).call();
+ } finally {
+ lock.unlock();
+ }
+ }).when(memberRecentRedisDAO).lock(anyLong(), anyLong(), any());
+ ExecutorService executor = Executors.newFixedThreadPool(8);
+ CountDownLatch start = new CountDownLatch(1);
+ List> calls = new ArrayList<>();
+ try {
+ for (int index = 0; index < 8; index++) {
+ calls.add(executor.submit(() -> {
+ start.await();
+ return memberRecentCfgService.getOrCreate(740L);
+ }));
+ }
+ start.countDown();
+ Long configId = calls.get(0).get(10, TimeUnit.SECONDS).getId();
+ for (Future call : calls) {
+ MemberRecentCfgDO config = call.get(10, TimeUnit.SECONDS);
+ assertThat(config.getId()).isEqualTo(configId);
+ assertThat(config.getHomeHide()).isFalse();
+ }
+ assertThat(jdbc.queryForObject("SELECT COUNT(*) FROM member_recent_cfg WHERE member_id=740", Integer.class))
+ .isEqualTo(1);
+ } finally {
+ executor.shutdownNow();
+ }
+ }
+
+ @Test
+ void initializationCommitsBeforeReturningToAnOuterTransaction() {
+ new TransactionTemplate(transactionManager).execute(status -> {
+ assertThat(TransactionSynchronizationManager.isActualTransactionActive()).isTrue();
+ memberRecentCfgService.getOrCreate(740L);
+ assertThat(TransactionSynchronizationManager.isActualTransactionActive()).isTrue();
+ status.setRollbackOnly();
+ return null;
+ });
+
+ assertThat(jdbc.queryForObject("SELECT COUNT(*) FROM member_recent_cfg WHERE member_id=740", Integer.class))
+ .isEqualTo(1);
+ }
+
+ private void insertConfig(Long id, String updateTime, boolean hidden, boolean deleted) {
+ jdbc.update("INSERT INTO member_recent_cfg (id,member_id,home_hide,privacy_secure,update_time,deleted)"
+ + " VALUES (?,740,?,?,?,?)", id, hidden, hidden, updateTime, deleted);
+ }
+
+ private MemberRecentCfgReqVo request(boolean homeHide) {
+ return new MemberRecentCfgReqVo(740L, true, homeHide, false, true, false);
+ }
+}
diff --git a/yudao-module-member/yudao-module-member-biz/src/test/java/cn/iocoder/yudao/module/member/service/user/MemberRecentCfgServiceImplTest.java b/yudao-module-member/yudao-module-member-biz/src/test/java/cn/iocoder/yudao/module/member/service/user/MemberRecentCfgServiceImplTest.java
new file mode 100644
index 0000000..dfd0741
--- /dev/null
+++ b/yudao-module-member/yudao-module-member-biz/src/test/java/cn/iocoder/yudao/module/member/service/user/MemberRecentCfgServiceImplTest.java
@@ -0,0 +1,96 @@
+package cn.iocoder.yudao.module.member.service.user;
+
+import cn.iocoder.yudao.framework.test.core.ut.BaseMockitoUnitTest;
+import cn.iocoder.yudao.module.member.controller.app.user.vo.recentcfg.MemberRecentCfgReqVo;
+import cn.iocoder.yudao.module.member.dal.dataobject.user.MemberRecentCfgDO;
+import cn.iocoder.yudao.module.member.dal.mysql.user.RecentCfgMapper;
+import cn.iocoder.yudao.module.member.dal.redis.recent.MemberRecentRedisDAO;
+import cn.iocoder.yudao.module.member.service.opentype.OpenTypeService;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.mockito.InjectMocks;
+import org.mockito.Mock;
+import org.springframework.dao.DuplicateKeyException;
+
+import java.util.concurrent.Callable;
+
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.anyLong;
+import static org.mockito.Mockito.never;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+class MemberRecentCfgServiceImplTest extends BaseMockitoUnitTest {
+
+ @InjectMocks
+ private MemberRecentCfgServiceImpl service;
+ @Mock
+ private RecentCfgMapper recentCfgMapper;
+ @Mock
+ private MemberRecentRedisDAO memberRecentRedisDAO;
+ @Mock
+ private OpenTypeService openTypeService;
+
+ @BeforeEach
+ void runLockedCallback() throws Exception {
+ when(memberRecentRedisDAO.lock(anyLong(), anyLong(), any())).thenAnswer(invocation ->
+ ((Callable) invocation.getArgument(2)).call());
+ }
+
+ @Test
+ void rechecksAfterAcquiringInitializationLock() {
+ MemberRecentCfgDO existing = existing();
+ when(recentCfgMapper.selectOne(any())).thenReturn(null, existing);
+
+ assertThat(service.getOrCreate(740L)).isSameAs(existing);
+ verify(recentCfgMapper, never()).insert(any(MemberRecentCfgDO.class));
+ }
+
+ @Test
+ void returnsConfigCreatedByAConcurrentWriterOnUniqueConflict() {
+ MemberRecentCfgDO existing = existing();
+ when(recentCfgMapper.selectOne(any())).thenReturn(null, null, existing);
+ when(recentCfgMapper.insert(any(MemberRecentCfgDO.class)))
+ .thenThrow(new DuplicateKeyException("uk_member_recent_cfg_active"));
+
+ assertThat(service.getOrCreate(740L)).isSameAs(existing);
+ }
+
+ @Test
+ void initializationDoesNotHideUnrelatedDuplicateKeyFailures() {
+ DuplicateKeyException failure = new DuplicateKeyException("another constraint");
+ when(recentCfgMapper.insert(any(MemberRecentCfgDO.class))).thenThrow(failure);
+
+ assertThatThrownBy(() -> service.getOrCreate(740L)).isSameAs(failure);
+ }
+
+ @Test
+ void updateAppliesRequestedSettingsAfterConcurrentInitialization() {
+ MemberRecentCfgDO existing = existing();
+ when(recentCfgMapper.selectOne(any())).thenReturn(null, existing);
+ when(recentCfgMapper.insert(any(MemberRecentCfgDO.class)))
+ .thenThrow(new DuplicateKeyException("uk_member_recent_cfg_active"));
+ MemberRecentCfgReqVo request = new MemberRecentCfgReqVo(740L, true, false, false, true, false);
+
+ MemberRecentCfgDO updated = service.createOrUpdate(request);
+
+ assertThat(updated.getId()).isEqualTo(existing.getId());
+ assertThat(updated.getPrivacySecure()).isTrue();
+ verify(recentCfgMapper).updateById(updated);
+ }
+
+ @Test
+ void updateDoesNotHideUnrelatedDuplicateKeyFailures() {
+ DuplicateKeyException failure = new DuplicateKeyException("another constraint");
+ when(recentCfgMapper.insert(any(MemberRecentCfgDO.class))).thenThrow(failure);
+
+ assertThatThrownBy(() -> service.createOrUpdate(
+ new MemberRecentCfgReqVo(740L, true, false, false, true, false))).isSameAs(failure);
+ }
+
+ private MemberRecentCfgDO existing() {
+ return MemberRecentCfgDO.builder().id(813L).memberId(740L).privacySecure(true).build();
+ }
+}
diff --git a/yudao-module-member/yudao-module-member-biz/src/test/java/cn/iocoder/yudao/module/member/service/user/MemberRecentTagServiceImpDbTest.java b/yudao-module-member/yudao-module-member-biz/src/test/java/cn/iocoder/yudao/module/member/service/user/MemberRecentTagServiceImpDbTest.java
new file mode 100644
index 0000000..2a1a133
--- /dev/null
+++ b/yudao-module-member/yudao-module-member-biz/src/test/java/cn/iocoder/yudao/module/member/service/user/MemberRecentTagServiceImpDbTest.java
@@ -0,0 +1,208 @@
+package cn.iocoder.yudao.module.member.service.user;
+
+import cn.iocoder.yudao.framework.test.core.ut.BaseDbUnitTest;
+import cn.iocoder.yudao.module.member.dal.dataobject.config.MemberConfigDO;
+import cn.iocoder.yudao.module.member.dal.dataobject.user.MemberUserDO;
+import cn.iocoder.yudao.module.member.dal.mysql.user.MemberUserMapper;
+import cn.iocoder.yudao.module.member.enums.tag.MemberRecentTagEnum;
+import cn.iocoder.yudao.module.member.service.config.MemberConfigService;
+import cn.iocoder.yudao.module.member.service.opentype.OpenTypeService;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.springframework.boot.test.mock.mockito.MockBean;
+import org.springframework.context.annotation.Import;
+import org.springframework.jdbc.core.JdbcTemplate;
+import org.springframework.test.context.TestPropertySource;
+
+import javax.annotation.Resource;
+import javax.sql.DataSource;
+import java.time.LocalDateTime;
+
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.mockito.Mockito.when;
+
+@Import(MemberRecentTagServiceImp.class)
+@TestPropertySource(properties = {
+ "spring.datasource.url=jdbc:h2:mem:member_recent_tag_login;MODE=MYSQL;DATABASE_TO_UPPER=false;NON_KEYWORDS=value;",
+ "spring.sql.init.schema-locations=classpath:/sql/create_tables.sql,classpath:/sql/member_recent_tag_login_schema.sql"
+})
+class MemberRecentTagServiceImpDbTest extends BaseDbUnitTest {
+
+ private static final Long MEMBER_ID = 740L;
+
+ @Resource
+ private MemberRecentTagServiceImp service;
+ @Resource
+ private DataSource dataSource;
+ @MockBean
+ private MemberUserMapper memberUserMapper;
+ @MockBean
+ private MemberConfigService memberConfigService;
+ @MockBean
+ private OpenTypeService openTypeService;
+
+ private JdbcTemplate jdbcTemplate;
+
+ @BeforeEach
+ void setUp() {
+ jdbcTemplate = new JdbcTemplate(dataSource);
+ }
+
+ @AfterEach
+ void cleanTags() {
+ jdbcTemplate.update("DELETE FROM member_recent_tag");
+ }
+
+ @Test
+ void duplicateNewPersonTagsDoNotBlockAnUnexpiredUser() {
+ mockNewPersonAge(1);
+ insertTag(MEMBER_ID, MemberRecentTagEnum.NEW_PERSON, false);
+ insertTag(MEMBER_ID, MemberRecentTagEnum.NEW_PERSON, false);
+
+ assertThat(service.createOrUpdateNewPersonTag(MEMBER_ID)).isFalse();
+
+ assertThat(activeCount(MEMBER_ID, MemberRecentTagEnum.NEW_PERSON)).isEqualTo(2);
+ assertThat(totalCount()).isEqualTo(2);
+ }
+
+ @Test
+ void expiredNewPersonLogicallyDeletesAllDuplicatesAndPreservesOtherTags() {
+ mockNewPersonAge(30);
+ insertTag(MEMBER_ID, MemberRecentTagEnum.NEW_PERSON, false);
+ insertTag(MEMBER_ID, MemberRecentTagEnum.NEW_PERSON, false);
+ insertTag(MEMBER_ID, MemberRecentTagEnum.NEW_PERSON, true);
+ insertTag(MEMBER_ID, MemberRecentTagEnum.VIP, false);
+ insertTag(741L, MemberRecentTagEnum.NEW_PERSON, false);
+
+ assertThat(service.createOrUpdateNewPersonTag(MEMBER_ID)).isTrue();
+ assertThat(service.createOrUpdateNewPersonTag(MEMBER_ID)).isFalse();
+
+ assertThat(activeCount(MEMBER_ID, MemberRecentTagEnum.NEW_PERSON)).isZero();
+ assertThat(activeCount(MEMBER_ID, MemberRecentTagEnum.VIP)).isEqualTo(1);
+ assertThat(activeCount(741L, MemberRecentTagEnum.NEW_PERSON)).isEqualTo(1);
+ assertThat(totalCount()).isEqualTo(5);
+ assertThat(jdbcTemplate.queryForObject("SELECT COUNT(*) FROM member_recent_tag WHERE deleted = TRUE",
+ Integer.class)).isEqualTo(3);
+ }
+
+ @Test
+ void newPersonTagIsCreatedOnceAndAnExistingTagIsReused() {
+ mockNewPersonAge(1);
+
+ assertThat(service.createOrUpdateNewPersonTag(MEMBER_ID)).isTrue();
+ assertThat(service.createOrUpdateNewPersonTag(MEMBER_ID)).isFalse();
+
+ assertThat(activeCount(MEMBER_ID, MemberRecentTagEnum.NEW_PERSON)).isEqualTo(1);
+ assertThat(jdbcTemplate.queryForObject("SELECT tag_title FROM member_recent_tag", String.class))
+ .isEqualTo(MemberRecentTagEnum.NEW_PERSON.getDescription());
+ }
+
+ @Test
+ void expiredUserWithoutANewPersonTagDoesNotCreateOne() {
+ mockNewPersonAge(30);
+
+ assertThat(service.createOrUpdateNewPersonTag(MEMBER_ID)).isFalse();
+
+ assertThat(totalCount()).isZero();
+ }
+
+ @Test
+ void duplicateVipTagsAreIdempotentAndAllExpireTogether() {
+ mockUser(1);
+ when(openTypeService.isUserOpenType(MEMBER_ID)).thenReturn(true, false);
+ insertTag(MEMBER_ID, MemberRecentTagEnum.VIP, false);
+ insertTag(MEMBER_ID, MemberRecentTagEnum.VIP, false);
+
+ assertThat(service.createOrUpdateVipTag(MEMBER_ID)).isFalse();
+ assertThat(activeCount(MEMBER_ID, MemberRecentTagEnum.VIP)).isEqualTo(2);
+ assertThat(service.createOrUpdateVipTag(MEMBER_ID)).isTrue();
+
+ assertThat(activeCount(MEMBER_ID, MemberRecentTagEnum.VIP)).isZero();
+ assertThat(totalCount()).isEqualTo(2);
+ }
+
+ @Test
+ void vipTagIsCreatedOnce() {
+ mockUser(1);
+ when(openTypeService.isUserOpenType(MEMBER_ID)).thenReturn(true);
+
+ assertThat(service.createOrUpdateVipTag(MEMBER_ID)).isTrue();
+ assertThat(service.createOrUpdateVipTag(MEMBER_ID)).isFalse();
+
+ assertThat(activeCount(MEMBER_ID, MemberRecentTagEnum.VIP)).isEqualTo(1);
+ }
+
+ @Test
+ void personTagCanRevokeAllDuplicatesAndBeGrantedAgainWithoutDeletingHistory() {
+ insertTag(MEMBER_ID, MemberRecentTagEnum.TRUE_PERSON_AUTH, false);
+ insertTag(MEMBER_ID, MemberRecentTagEnum.TRUE_PERSON_AUTH, false);
+ insertTag(MEMBER_ID, MemberRecentTagEnum.NEW_PERSON, false);
+
+ service.setPersonTag(MEMBER_ID, true);
+ assertThat(activeCount(MEMBER_ID, MemberRecentTagEnum.TRUE_PERSON_AUTH)).isEqualTo(2);
+ service.setPersonTag(MEMBER_ID, false);
+ assertThat(activeCount(MEMBER_ID, MemberRecentTagEnum.TRUE_PERSON_AUTH)).isZero();
+ assertThat(totalCount()).isEqualTo(3);
+ service.setPersonTag(MEMBER_ID, true);
+ service.setPersonTag(MEMBER_ID, true);
+
+ assertThat(activeCount(MEMBER_ID, MemberRecentTagEnum.TRUE_PERSON_AUTH)).isEqualTo(1);
+ assertThat(activeCount(MEMBER_ID, MemberRecentTagEnum.NEW_PERSON)).isEqualTo(1);
+ assertThat(totalCount()).isEqualTo(4);
+ }
+
+ @Test
+ void scheduledNewPersonCleanupHandlesDuplicateRowsForTheSameMember() {
+ mockNewPersonAge(30);
+ insertTag(MEMBER_ID, MemberRecentTagEnum.NEW_PERSON, false);
+ insertTag(MEMBER_ID, MemberRecentTagEnum.NEW_PERSON, false);
+
+ assertThat(service.checkNewPersonTagJob()).isEqualTo(1);
+
+ assertThat(activeCount(MEMBER_ID, MemberRecentTagEnum.NEW_PERSON)).isZero();
+ assertThat(totalCount()).isEqualTo(2);
+ }
+
+ @Test
+ void scheduledVipCleanupHandlesDuplicateRowsForTheSameMember() {
+ mockUser(1);
+ when(openTypeService.isUserOpenType(MEMBER_ID)).thenReturn(false);
+ insertTag(MEMBER_ID, MemberRecentTagEnum.VIP, false);
+ insertTag(MEMBER_ID, MemberRecentTagEnum.VIP, false);
+
+ assertThat(service.checkVipTagJob()).isEqualTo(1);
+
+ assertThat(activeCount(MEMBER_ID, MemberRecentTagEnum.VIP)).isZero();
+ assertThat(totalCount()).isEqualTo(2);
+ }
+
+ private void mockNewPersonAge(int days) {
+ mockUser(days);
+ MemberConfigDO config = new MemberConfigDO();
+ config.setNewPersonDays(15);
+ when(memberConfigService.getConfig()).thenReturn(config);
+ }
+
+ private void mockUser(int days) {
+ MemberUserDO user = new MemberUserDO();
+ user.setId(MEMBER_ID);
+ user.setCreateTime(LocalDateTime.now().minusDays(days));
+ when(memberUserMapper.selectById(MEMBER_ID)).thenReturn(user);
+ }
+
+ private void insertTag(Long memberId, MemberRecentTagEnum type, boolean deleted) {
+ jdbcTemplate.update("INSERT INTO member_recent_tag (member_id, tag_biz_type, tag_title, deleted)"
+ + " VALUES (?, ?, ?, ?)", memberId, type.getType(), type.getDescription(), deleted);
+ }
+
+ private int activeCount(Long memberId, MemberRecentTagEnum type) {
+ return jdbcTemplate.queryForObject("SELECT COUNT(*) FROM member_recent_tag"
+ + " WHERE member_id = ? AND tag_biz_type = ? AND deleted = FALSE",
+ Integer.class, memberId, type.getType());
+ }
+
+ private int totalCount() {
+ return jdbcTemplate.queryForObject("SELECT COUNT(*) FROM member_recent_tag", Integer.class);
+ }
+}
diff --git a/yudao-module-member/yudao-module-member-biz/src/test/java/cn/iocoder/yudao/module/member/service/user/MemberRecentTagServiceImpTest.java b/yudao-module-member/yudao-module-member-biz/src/test/java/cn/iocoder/yudao/module/member/service/user/MemberRecentTagServiceImpTest.java
index 651f2b2..39042d7 100644
--- a/yudao-module-member/yudao-module-member-biz/src/test/java/cn/iocoder/yudao/module/member/service/user/MemberRecentTagServiceImpTest.java
+++ b/yudao-module-member/yudao-module-member-biz/src/test/java/cn/iocoder/yudao/module/member/service/user/MemberRecentTagServiceImpTest.java
@@ -5,17 +5,27 @@ import cn.iocoder.yudao.module.member.dal.dataobject.user.MemberRecentTagDO;
import cn.iocoder.yudao.module.member.dal.mysql.user.MemberRecentTagMapper;
import cn.iocoder.yudao.module.member.enums.tag.MemberRecentTagEnum;
import cn.iocoder.yudao.module.member.service.user.BO.MemberTagExtBO;
+import com.baomidou.mybatisplus.core.MybatisConfiguration;
+import com.baomidou.mybatisplus.core.metadata.TableInfoHelper;
+import org.apache.ibatis.builder.MapperBuilderAssistant;
+import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import org.mockito.InjectMocks;
import org.mockito.Mock;
+import org.springframework.dao.DataIntegrityViolationException;
+import org.springframework.dao.DuplicateKeyException;
import java.util.Arrays;
+import java.util.Collections;
import java.util.List;
import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
import static org.assertj.core.groups.Tuple.tuple;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.Mockito.when;
+import static org.mockito.Mockito.never;
+import static org.mockito.Mockito.verify;
class MemberRecentTagServiceImpTest extends BaseMockitoUnitTest {
@@ -25,6 +35,11 @@ class MemberRecentTagServiceImpTest extends BaseMockitoUnitTest {
@Mock
private MemberRecentTagMapper memberRecentTagMapper;
+ @BeforeAll
+ static void initTableInfo() {
+ TableInfoHelper.initTableInfo(new MapperBuilderAssistant(new MybatisConfiguration(), ""), MemberRecentTagDO.class);
+ }
+
@Test
void getUserTagExtListDeduplicatesTagsByMemberAndBizType() {
when(memberRecentTagMapper.selectList(any())).thenReturn(Arrays.asList(
@@ -46,6 +61,44 @@ class MemberRecentTagServiceImpTest extends BaseMockitoUnitTest {
assertThat(result.get(0).getTagTitle()).isEqualTo("VIP用户");
}
+ @Test
+ void concurrentInsertIsIdempotentWhenTheActiveTagNowExists() {
+ when(memberRecentTagMapper.selectCount(any())).thenReturn(0L);
+ when(memberRecentTagMapper.insert(any(MemberRecentTagDO.class)))
+ .thenThrow(new DuplicateKeyException("active member tag already exists"));
+ when(memberRecentTagMapper.selectList(any())).thenReturn(Collections.singletonList(
+ createTag(1L, 740L, MemberRecentTagEnum.TRUE_PERSON_AUTH, "真人认证")));
+
+ memberRecentTagService.setPersonTag(740L, true);
+
+ verify(memberRecentTagMapper).selectCount(any());
+ verify(memberRecentTagMapper).selectList(any());
+ verify(memberRecentTagMapper).insert(any(MemberRecentTagDO.class));
+ }
+
+ @Test
+ void duplicateKeyWithoutTheExpectedActiveTagIsPropagated() {
+ when(memberRecentTagMapper.selectCount(any())).thenReturn(0L);
+ DuplicateKeyException failure = new DuplicateKeyException("unrelated unique constraint");
+ when(memberRecentTagMapper.insert(any(MemberRecentTagDO.class))).thenThrow(failure);
+ when(memberRecentTagMapper.selectList(any())).thenReturn(Collections.emptyList());
+
+ assertThatThrownBy(() -> memberRecentTagService.setPersonTag(740L, true))
+ .isSameAs(failure);
+ }
+
+ @Test
+ void otherDatabaseFailuresAreNotTreatedAsConcurrentSuccess() {
+ when(memberRecentTagMapper.selectCount(any())).thenReturn(0L);
+ DataIntegrityViolationException failure = new DataIntegrityViolationException("invalid tag data");
+ when(memberRecentTagMapper.insert(any(MemberRecentTagDO.class))).thenThrow(failure);
+
+ assertThatThrownBy(() -> memberRecentTagService.setPersonTag(740L, true))
+ .isSameAs(failure);
+ verify(memberRecentTagMapper).selectCount(any());
+ verify(memberRecentTagMapper, never()).selectList(any());
+ }
+
private MemberRecentTagDO createTag(Long id, Long memberId, MemberRecentTagEnum tag, String title) {
return MemberRecentTagDO.builder()
.id(id)
diff --git a/yudao-module-member/yudao-module-member-biz/src/test/java/cn/iocoder/yudao/module/member/service/user/MemberUserServiceImplRecentTagTest.java b/yudao-module-member/yudao-module-member-biz/src/test/java/cn/iocoder/yudao/module/member/service/user/MemberUserServiceImplRecentTagTest.java
new file mode 100644
index 0000000..60a4546
--- /dev/null
+++ b/yudao-module-member/yudao-module-member-biz/src/test/java/cn/iocoder/yudao/module/member/service/user/MemberUserServiceImplRecentTagTest.java
@@ -0,0 +1,77 @@
+package cn.iocoder.yudao.module.member.service.user;
+
+import cn.iocoder.yudao.framework.test.core.ut.BaseMockitoUnitTest;
+import cn.iocoder.yudao.module.member.controller.admin.user.vo.MemberUserUpdateReqVO;
+import cn.iocoder.yudao.module.member.dal.dataobject.user.MemberRecentTagDO;
+import cn.iocoder.yudao.module.member.dal.dataobject.user.MemberUserDO;
+import cn.iocoder.yudao.module.member.dal.mysql.user.MemberRecentTagMapper;
+import cn.iocoder.yudao.module.member.dal.mysql.user.MemberUserMapper;
+import org.junit.jupiter.api.Test;
+import org.mockito.ArgumentCaptor;
+import org.mockito.InjectMocks;
+import org.mockito.Mock;
+
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.anyCollection;
+import static org.mockito.Mockito.never;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.verifyNoInteractions;
+import static org.mockito.Mockito.when;
+
+class MemberUserServiceImplRecentTagTest extends BaseMockitoUnitTest {
+
+ @InjectMocks
+ private MemberUserServiceImpl userService;
+ @Mock
+ private MemberUserMapper memberUserMapper;
+ @Mock
+ private MemberRecentTagMapper memberRecentTagMapper;
+ @Mock
+ private MemberRecentTagService memberRecentTagService;
+
+ @Test
+ @SuppressWarnings({"unchecked", "rawtypes"})
+ void updateUserStoresEachTagTypeOnceWhenRetryPayloadContainsDuplicates() {
+ MemberUserUpdateReqVO request = request();
+ request.setMemberRecentTagList(Arrays.asList("1", "2", "4", "2", "4", "1"));
+
+ userService.updateUser(request);
+
+ ArgumentCaptor> tags = ArgumentCaptor.forClass((Class) Collection.class);
+ verify(memberRecentTagMapper).insert(tags.capture());
+ assertThat(tags.getValue()).extracting(MemberRecentTagDO::getTagBizType)
+ .containsExactly("1", "2", "4");
+ assertThat(tags.getValue()).allSatisfy(tag -> assertThat(tag.getMemberId()).isEqualTo(740L));
+ verify(memberRecentTagService).createOrUpdateNewPersonTag(740L);
+ }
+
+ @Test
+ void updateUserWithoutTagFieldLeavesExistingTagsUntouched() {
+ userService.updateUser(request());
+
+ verifyNoInteractions(memberRecentTagMapper);
+ }
+
+ @Test
+ void updateUserWithEmptyTagListClearsTagsWithoutInserting() {
+ MemberUserUpdateReqVO request = request();
+ request.setMemberRecentTagList(Collections.emptyList());
+
+ userService.updateUser(request);
+
+ verify(memberRecentTagMapper).delete(any());
+ verify(memberRecentTagMapper, never()).insert(anyCollection());
+ }
+
+ private MemberUserUpdateReqVO request() {
+ when(memberUserMapper.selectById(740L)).thenReturn(new MemberUserDO().setId(740L).setStatus(0));
+ MemberUserUpdateReqVO request = new MemberUserUpdateReqVO();
+ request.setId(740L);
+ return request;
+ }
+}
diff --git a/yudao-module-member/yudao-module-member-biz/src/test/resources/sql/member_recent_cfg_schema.sql b/yudao-module-member/yudao-module-member-biz/src/test/resources/sql/member_recent_cfg_schema.sql
new file mode 100644
index 0000000..c8b6aac
--- /dev/null
+++ b/yudao-module-member/yudao-module-member-biz/src/test/resources/sql/member_recent_cfg_schema.sql
@@ -0,0 +1,14 @@
+CREATE TABLE IF NOT EXISTS member_recent_cfg (
+ id bigint NOT NULL GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
+ member_id bigint NOT NULL,
+ is_push bit,
+ home_hide bit,
+ global_hide bit,
+ privacy_secure bit,
+ short_note bit,
+ creator varchar(64) DEFAULT '',
+ create_time datetime NOT NULL DEFAULT CURRENT_TIMESTAMP,
+ updater varchar(64) DEFAULT '',
+ update_time datetime NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
+ deleted bit NOT NULL DEFAULT FALSE
+);
diff --git a/yudao-module-member/yudao-module-member-biz/src/test/resources/sql/member_recent_tag_login_schema.sql b/yudao-module-member/yudao-module-member-biz/src/test/resources/sql/member_recent_tag_login_schema.sql
new file mode 100644
index 0000000..aadd323
--- /dev/null
+++ b/yudao-module-member/yudao-module-member-biz/src/test/resources/sql/member_recent_tag_login_schema.sql
@@ -0,0 +1,13 @@
+CREATE TABLE IF NOT EXISTS member_recent_tag (
+ id bigint NOT NULL GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
+ member_id bigint NOT NULL,
+ tag_biz_type varchar(32),
+ tag_title varchar(64) NOT NULL,
+ tag_biz_id bigint,
+ creator varchar(64) DEFAULT '',
+ create_time datetime NOT NULL DEFAULT CURRENT_TIMESTAMP,
+ updater varchar(64) DEFAULT '',
+ update_time datetime NOT NULL DEFAULT CURRENT_TIMESTAMP,
+ deleted bit NOT NULL DEFAULT FALSE,
+ tenant_id bigint NOT NULL DEFAULT 0
+);