feat: sync account priorities after rate changes
This commit is contained in:
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
+540
@@ -0,0 +1,540 @@
|
||||
# Reasonix project memory
|
||||
|
||||
Notes the user pinned via the `#` prompt prefix. The whole file is
|
||||
loaded into the immutable system prefix every session — keep it terse.
|
||||
|
||||
- 保留自定义页面远程浏览器登录态
|
||||
|
||||
## Summary
|
||||
|
||||
- 根因:自定义页面查看器在启动、切换、卸载时会调用 DELETE /api/browser-sessions/{id},服务端随即关闭 Playwright persistent context。虽然 profile 目录在 data/browser-profiles/page-* 里保留,但每次前端都主动销毁会话,体验上就是“断开后重开”。
|
||||
- 修复目标:普通断开只断 WebSocket,不关闭服务端浏览器;再次打开同一自定义页面时复用同一个 page/profile,从而保留登录态。
|
||||
- 隐私策略:默认保留登录态,并提供显式“清除登录态/重建浏览器”能力。
|
||||
|
||||
## Key Changes
|
||||
|
||||
- 调整 PageViewer.vue 的远程浏览器生命周期:
|
||||
- 启动前不再先 closeRemoteSession(),只 stopRemoteWs(),让后端可复用已有 session。
|
||||
- onBeforeUnmount、active=false、同页重连时只断开 WebSocket 和释放前端截图 blob,不调用后端 DELETE。
|
||||
- 页面 ID 变化时断开当前 WS,再为新页面创建/复用对应 session;不主动销毁旧页面 session。
|
||||
- 保留当前后端 BrowserSessionService.create() 的复用逻辑:
|
||||
- 同一 custom_page_id + origin 已有活跃 session 时直接返回,并调整 viewport。
|
||||
- profile 目录继续使用 BROWSER_PROFILES_DIR,当前 Docker 已挂载到 ./data:/app/data。
|
||||
- 新增显式清理能力:
|
||||
- 后端增加清除 profile 的接口,例如 DELETE /api/browser-sessions/profiles/{custom_page_id},行为是关闭该页面现有 session 并删除对应 page-{id}-{origin} profile 目录。
|
||||
- 前端远程浏览器工具栏增加“清除登录态”按钮,二次确认后调用清理接口,再重新创建 session。
|
||||
- 保留现有 DELETE /api/browser-sessions/{id} 作为“关闭当前会话但不删除 profile”的轻量关闭,不作为普通页面离开的默认动作。
|
||||
|
||||
## Public Interfaces
|
||||
|
||||
- browserSessionsApi 增加 clearProfile(customPageId: number)。
|
||||
- 后端新增一个清理 profile 的管理接口;普通创建、截图、事件、WebSocket 接口保持兼容。
|
||||
- 不改变自定义页面模型、数据库字段或现有 BROWSER_PROFILES_DIR 配置。
|
||||
|
||||
## Test Plan
|
||||
|
||||
- 打开一个远程浏览器自定义页面并登录,离开页面再返回,确认无需重新登录。
|
||||
- 刷新 SmartUp 前端页面后重新进入同一自定义页面,确认服务端仍能复用活跃 session;若服务端重启,则从落盘 profile 恢复登录态。
|
||||
- 点击“清除登录态”,确认 profile 被删除并重新打开后需要重新登录。
|
||||
- 切换两个不同远程浏览器页面,确认各自登录态互不串。
|
||||
- 回归检查远程浏览器的后退、前进、刷新、复制选中文本、一键刷新上游凭证仍可用。
|
||||
|
||||
## Assumptions
|
||||
|
||||
- 本次只处理“自定义页面查看”的远程浏览器登录态,不改“上游认证提取弹窗”的临时 profile 行为。
|
||||
- 登录态保留依赖目标站点 cookie/localStorage 本身有效;如果目标站点设置极短过期或服务端强制失效,SmartUp 无法绕过。
|
||||
- 后续可再加 idle TTL 清理后台浏览器进程,但本次优先解决频繁重开导致的重复登录体验。
|
||||
|
||||
优化
|
||||
- 保留自定义页面远程浏览器登录态
|
||||
|
||||
## Summary
|
||||
|
||||
- 根因:自定义页面查看器在启动、切换、卸载时会调用 DELETE /api/browser-sessions/{id},服务端随即关闭 Playwright persistent context。虽然 profile 目录在 data/browser-profiles/page-* 里保留,但每次前端都主动销毁会话,体验上就是“断开后重开”。
|
||||
- 修复目标:普通断开只断 WebSocket,不关闭服务端浏览器;再次打开同一自定义页面时复用同一个 page/profile,从而保留登录态。
|
||||
- 隐私策略:默认保留登录态,并提供显式“清除登录态/重建浏览器”能力。
|
||||
|
||||
## Key Changes
|
||||
|
||||
- 调整 PageViewer.vue 的远程浏览器生命周期:
|
||||
- 启动前不再先 closeRemoteSession(),只 stopRemoteWs(),让后端可复用已有 session。
|
||||
- onBeforeUnmount、active=false、同页重连时只断开 WebSocket 和释放前端截图 blob,不调用后端 DELETE。
|
||||
- 页面 ID 变化时断开当前 WS,再为新页面创建/复用对应 session;不主动销毁旧页面 session。
|
||||
- 保留当前后端 BrowserSessionService.create() 的复用逻辑:
|
||||
- 同一 custom_page_id + origin 已有活跃 session 时直接返回,并调整 viewport。
|
||||
- profile 目录继续使用 BROWSER_PROFILES_DIR,当前 Docker 已挂载到 ./data:/app/data。
|
||||
- 新增显式清理能力:
|
||||
- 后端增加清除 profile 的接口,例如 DELETE /api/browser-sessions/profiles/{custom_page_id},行为是关闭该页面现有 session 并删除对应 page-{id}-{origin} profile 目录。
|
||||
- 前端远程浏览器工具栏增加“清除登录态”按钮,二次确认后调用清理接口,再重新创建 session。
|
||||
- 保留现有 DELETE /api/browser-sessions/{id} 作为“关闭当前会话但不删除 profile”的轻量关闭,不作为普通页面离开的默认动作。
|
||||
|
||||
## Public Interfaces
|
||||
|
||||
- browserSessionsApi 增加 clearProfile(customPageId: number)。
|
||||
- 后端新增一个清理 profile 的管理接口;普通创建、截图、事件、WebSocket 接口保持兼容。
|
||||
- 不改变自定义页面模型、数据库字段或现有 BROWSER_PROFILES_DIR 配置。
|
||||
|
||||
## Test Plan
|
||||
|
||||
- 打开一个远程浏览器自定义页面并登录,离开页面再返回,确认无需重新登录。
|
||||
- 刷新 SmartUp 前端页面后重新进入同一自定义页面,确认服务端仍能复用活跃 session;若服务端重启,则从落盘 profile 恢复登录态。
|
||||
- 点击“清除登录态”,确认 profile 被删除并重新打开后需要重新登录。
|
||||
- 切换两个不同远程浏览器页面,确认各自登录态互不串。
|
||||
- 回归检查远程浏览器的后退、前进、刷新、复制选中文本、一键刷新上游凭证仍可用。
|
||||
|
||||
## Assumptions
|
||||
|
||||
- 本次只处理“自定义页面查看”的远程浏览器登录态,不改“上游认证提取弹窗”的临时 profile 行为。
|
||||
- 登录态保留依赖目标站点 cookie/localStorage 本身有效;如果目标站点设置极短过期或服务端强制失效,SmartUp 无法绕过。
|
||||
- 后续可再加 idle TTL 清理后台浏览器进程,但本次优先解决频繁重开导致的重复登录体验。
|
||||
开始优化
|
||||
- 远程浏览器复制按钮同步到本机剪贴板
|
||||
|
||||
## Summary
|
||||
|
||||
- 根因:远程浏览器里的网页复制按钮只操作服务端 Chromium 的剪贴板,当前 SmartUp 只把鼠标/键盘事件发到远端,没有把远端 clipboard 同步回本机浏览器。
|
||||
- 修复目标:用户点击远程页面里的复制按钮后,SmartUp 自动尝试读取远端剪贴板并写入本机剪贴板;失败时提供明确提示和手动复制兜底。
|
||||
|
||||
## Key Changes
|
||||
|
||||
- 后端新增远程剪贴板读取能力:
|
||||
- 在 Playwright context 创建后授予当前 origin clipboard-read / clipboard-write 权限。
|
||||
- 增加 GET /api/browser-sessions/{session_id}/clipboard,通过 navigator.clipboard.readText() 读取远端页面剪贴板。
|
||||
- 读取失败时返回可解释错误,不影响现有远程浏览器会话。
|
||||
- 前端 PageViewer.vue 增加自动同步:
|
||||
- 在远程鼠标 mouseup 后短延迟读取远端剪贴板。
|
||||
- 如果读到非空文本且和上次同步内容不同,调用 navigator.clipboard.writeText() 写入本机剪贴板,并提示“已同步到本机剪贴板”。
|
||||
- 保留现有“复制远程选中文本”按钮,作为选中文本场景的手动路径。
|
||||
- 兜底体验:
|
||||
- 如果本机浏览器阻止自动写剪贴板,显示一个可复制文本提示框或通知,让用户手动复制。
|
||||
- 只同步文本,不处理图片/富文本剪贴板。
|
||||
|
||||
## Test Plan
|
||||
|
||||
- 在远程浏览器打开包含 API Key 复制按钮的页面,点击页面内复制图标,回到本机输入框粘贴,确认拿到 sk-...。
|
||||
- 点击普通按钮或空白区域,确认不会反复弹出无意义复制提示。
|
||||
- 点击远程页面后如果 clipboard 为空,确认不报错、不打断操作。
|
||||
- 测试 HTTPS / localhost 下本机 navigator.clipboard.writeText 成功路径。
|
||||
- 测试浏览器禁用剪贴板权限时,确认出现手动复制兜底提示。
|
||||
- 回归 Ctrl+C 选中文本复制、Ctrl+V 粘贴到远程输入框、远程浏览器登录态保留。
|
||||
|
||||
## Assumptions
|
||||
|
||||
- 目标主要是文本类密钥、token、API key。
|
||||
- 自动同步只在用户远程点击后触发,不做持续轮询,避免误复制和资源浪费。
|
||||
- 若目标站点自身复制按钮没有真正写入远端 clipboard,SmartUp 无法凭空知道按钮背后的文本,只能提示未检测到远程剪贴板内容。
|
||||
修复
|
||||
- 远程浏览器登录态持久化与二次访问转圈修复方案
|
||||
|
||||
## Summary
|
||||
|
||||
当前后端已经用 launch_persistent_context(profile_dir) 做了登录态持久化,问题不在“没有持久化”,而在“长期复用活跃 session/page”。第二次访问一直转圈,优先按“复用了不可用的旧 page,前端等不到首帧”处理。
|
||||
|
||||
默认策略改为:登录态靠持久化 profile 保存,页面运行态不长期依赖复用;离开远程浏览器页面时关闭 runtime session,但不删除 profile。再次进入时新建 Chromium context,从 profile 恢复 cookies/localStorage/sessionStorage。
|
||||
|
||||
## Key Changes
|
||||
|
||||
- 前端生命周期调整:
|
||||
- 页面切换、组件卸载、active=false 时调用现有 DELETE /api/browser-sessions/{id} 关闭远程浏览器 runtime session。
|
||||
- 不调用“清除登录态”接口,不删除 profile 目录。
|
||||
- 清空 remoteSession、remoteScreenshotUrl、loading/error/caret 等运行态,避免旧截图和旧 session 干扰。
|
||||
- 后端保持持久化 profile:
|
||||
- close(session_id) 只关闭 context,不删除普通 page-* profile。
|
||||
- 仅 clearProfile(custom_page_id) 删除登录态目录。
|
||||
- auth-capture-* 临时 profile 仍按现有逻辑清理。
|
||||
- 保留同一页面内的短期复用,但加健康兜底:
|
||||
- 如果 create() 命中现有 session,先做轻量健康检查:page 未关闭、能取 state、能在短超时内截图。
|
||||
- 健康检查失败时关闭旧 session,用同一 profile 重新启动 context。
|
||||
- 首帧失败兜底:
|
||||
- 后端 screenshot() 加明确超时,避免 WebSocket push loop 无限等待。
|
||||
- 前端连接 WebSocket 后增加首帧 watchdog,例如 8-10 秒没有收到第一张截图就提示“远程浏览器无响应”,并提供“重建浏览器”按钮。
|
||||
- “重建浏览器”只关闭 runtime session 后重开,不删除登录态。
|
||||
- 文案区分两个动作:
|
||||
- “刷新/重建浏览器”:关闭当前 runtime,用持久化 profile 重开,登录态保留。
|
||||
- “清除登录态”:关闭 runtime 并删除 profile,下次需要重新登录。
|
||||
|
||||
## API / Behavior
|
||||
|
||||
- 不新增必须 API,复用现有:
|
||||
- DELETE /api/browser-sessions/{session_id} = 关闭运行中的浏览器,不清登录态。
|
||||
- DELETE /api/browser-sessions/profiles/{custom_page_id} = 清除登录态。
|
||||
- 如需更清晰,可前端按钮文案改为“重建浏览器”,内部调用 close session 后 startRemoteBrowser()。
|
||||
|
||||
## Test Plan
|
||||
|
||||
- PackyAPI 首次登录成功后,离开页面再回来,应直接进入已登录状态,不要求重新登录。
|
||||
- 关闭浏览器页面、刷新 SmartUp 页面、重新进入远程浏览器,应从持久化 profile 恢复登录态。
|
||||
- 点击“重建浏览器”后,应保留登录态并恢复可操作画面。
|
||||
- 点击“清除登录态”后,应删除 profile,下次进入需要重新登录。
|
||||
- 模拟旧 session 截图失败或 WebSocket 无首帧,前端不能无限转圈,应显示可恢复错误和重建入口。
|
||||
- 多个远程浏览器页面之间切换,profile 不串,登录态按页面隔离。
|
||||
|
||||
## Assumptions
|
||||
|
||||
- 用户更看重“再次访问不用重新登录”和“不要卡死转圈”,不要求保留上一次离开时的精确 DOM/滚动/弹窗运行状态。
|
||||
- profile 目录所在 /app/data/browser-profiles 是持久卷;如果部署环境没有挂载持久卷,需要先修部署卷配置。
|
||||
- 普通关闭 session 不删除 profile,因此用它替代长期活跃复用更可靠。
|
||||
- 二次访问远程浏览器白屏修复方案
|
||||
|
||||
## Summary
|
||||
|
||||
结合 Docker 日志看,第一次进入有完整链路:POST /api/browser-sessions、WebSocket 建连、随后离开时 DELETE /api/browser-sessions/{id}。但第二次白屏期间日志里没有新的 POST /api/browser-sessions,说明问题不是后端 Chromium 已启动但截图失败,而是前端关闭 session 后,重新激活页面时没有重新创建远程浏览器 session。
|
||||
|
||||
根因在 PageViewer.vue 的 props.active watcher:active=false 时关闭了 session;active=true 时只在 remoteSession 仍存在时重连 WebSocket,remoteSession=null 时没有调用 startRemoteBrowser()。
|
||||
|
||||
## Key Changes
|
||||
|
||||
- 修复 props.active watcher:
|
||||
- active=false:继续 stopRemoteWs() + await closeRemoteSession(),保留 profile。
|
||||
- active=true:
|
||||
- 如果 remoteSession.value 存在,按现有逻辑 connectRemoteWs()。
|
||||
- 如果 remoteSession.value 不存在,调用 await startRemoteBrowser(),从持久化 profile 新建 runtime session。
|
||||
- 激活后再 nextTick(() => remoteFrameRef.value?.focus())。
|
||||
- 修复白屏显示状态:
|
||||
- closeRemoteSession() 清空 remoteSession 和截图后,对远程浏览器页面设置 iframeLoading.value = true,避免没有 session、没有截图、也没有 loading 的空白区域。
|
||||
- startRemoteBrowser() 成功建连后继续由首帧 onRemoteImageLoad() 关闭 loading。
|
||||
- 防重复创建:
|
||||
- 复用现有 startRemoteBrowserPromises,确保 active=true、路由 watcher、重建按钮同时触发时不会并发创建多个 session。
|
||||
- 保持 profile 持久化语义:
|
||||
- 普通关闭 session 只释放 context,不删除 page-* profile。
|
||||
- “清除登录态”仍是唯一删除 profile 的入口。
|
||||
|
||||
## Test Plan
|
||||
|
||||
- 首次打开 98 远程浏览器,确认 Docker 日志出现 POST /api/browser-sessions 和 WebSocket。
|
||||
- 切到“网站管理”等普通页面,确认日志出现 DELETE /api/browser-sessions/{id}。
|
||||
- 再次点击 98,应重新出现 POST /api/browser-sessions,页面不白屏,并从 profile 恢复登录态。
|
||||
- 在自定义页面 tab 间切换:旧 tab 关闭 runtime,新 tab 激活时能重新创建 session。
|
||||
- 点击“重建浏览器”:应关闭当前 session 后重新创建,登录态保留。
|
||||
- 点击“清除登录态”:下次进入需要重新登录。
|
||||
- 若首帧 10 秒未到,应显示“远程浏览器无响应”,不能无限白屏或转圈。
|
||||
|
||||
## Assumptions
|
||||
|
||||
- Docker 日志中第二次没有 POST /api/browser-sessions 是本次白屏的主要证据。
|
||||
- 白屏不是目标站点自身渲染白页,而是 SmartUp 前端没有重新启动远程浏览器 session。
|
||||
- 当前持久化 profile 目录 /app/data/browser-profiles 已正确挂载,问题不需要改 Docker volume。
|
||||
- 系统页高亮按钮统一方案
|
||||
|
||||
## Summary
|
||||
|
||||
问题不是所有 primary 都错,而是行内/列表内的频繁操作按钮被做成了实心橙色,和表格、绑定列表这种低强调区域不搭。统一规则:页面级主操作保留橙色 primary;表格行内、列表行内、辅助面板里的操作全部降级为 text/ghost 样式。
|
||||
|
||||
## Key Changes
|
||||
|
||||
- Websites.vue:
|
||||
- “我的网站分组”表格里的“绑定”去掉 type="primary",改为 text 低强调按钮,建议加 class="inline-action".
|
||||
- “分组绑定”列表里的“同步”去掉 type="primary",改为 text 低强调按钮,建议加 class="inline-action".
|
||||
- “分组绑定”面板头部的“新增绑定”如果觉得仍抢眼,改为 text 或 plain,只保留页面顶部“新增网站”为 primary。
|
||||
- 删除按钮继续保留 danger 语义,但通过局部 CSS 保持透明背景,只在 hover 时浅红反馈。
|
||||
- 局部样式:
|
||||
- 新增 .inline-action:小尺寸、透明背景、浅边框/低饱和文字,hover 时轻微橙色底,不使用实心渐变。
|
||||
- 新增 .panel-head .el-button--primary.is-plain 或专用 .panel-action,让面板级新增按钮不等同页面级主 CTA。
|
||||
- 设计层级:
|
||||
- 页面级创建:新增网站、抽屉里的 保存 保留 primary。
|
||||
- 面板级辅助:拉取分组、新增绑定 使用 default/plain/text。
|
||||
- 行内操作:绑定、同步、编辑、删除全部使用 text/ghost。
|
||||
|
||||
## Test Plan
|
||||
|
||||
- 网站管理页不再出现截图里那种橙色实心小按钮。
|
||||
- 页面顶部“新增网站”仍是唯一明显主 CTA。
|
||||
- “绑定”“同步”在 hover 时有反馈,但默认状态不抢视觉。
|
||||
- 删除按钮默认不实心,hover 浅红。
|
||||
- npm --prefix frontend run build 通过。
|
||||
|
||||
## Assumptions
|
||||
|
||||
- 用户要统一的是系统页里的行内高亮按钮,不是取消所有页面级 primary 按钮。
|
||||
- 优化 make up 依赖缓存
|
||||
|
||||
## Summary
|
||||
|
||||
当前 make up 固定执行 docker compose up -d --build,Dockerfile 里又有 npm ci、pip install --no-cache-dir、apt-get install、playwright install chromium。如果 Docker 层缓存失效或 BuildKit cache 不启用,就会反复下载依赖。优化目标是:普通启动不重新构建;需要构建时复用 npm/pip/apt/playwright 缓存。
|
||||
|
||||
## Key Changes
|
||||
|
||||
- 调整 Makefile:
|
||||
- make up 改为只执行 docker compose up -d,不默认 --build
|
||||
- 新增 make build 执行 DOCKER_BUILDKIT=1 docker compose build
|
||||
- 新增 make up-build 执行 DOCKER_BUILDKIT=1 docker compose up -d --build
|
||||
- 保留访问地址输出逻辑
|
||||
- 调整 Dockerfile 依赖安装缓存:
|
||||
- frontend 阶段给 npm ci 加 BuildKit cache mount:--mount=type=cache,target=/root/.npm
|
||||
- backend 阶段取消 pip install --no-cache-dir,改为 BuildKit cache mount:--mount=type=cache,target=/root/.cache/pip
|
||||
- apt-get update/install 加 cache mount:/var/cache/apt、/var/lib/apt/lists
|
||||
- playwright install chromium 加缓存目录,例如 PLAYWRIGHT_BROWSERS_PATH=/ms-playwright 并用 --mount=type=cache,target=/ms-playwright
|
||||
- 保持 Docker 层缓存友好顺序:
|
||||
- frontend/package*.json 先 COPY,再 npm ci
|
||||
- backend/requirements.txt 先 COPY,再 pip install
|
||||
- 源码 COPY 放在依赖安装之后,避免业务代码改动导致依赖层重建
|
||||
- 可选补充:
|
||||
- 在 README 或 Makefile 注释里说明日常使用 make up,依赖变更后用 make up-build
|
||||
|
||||
## Test Plan
|
||||
|
||||
- 第一次运行:
|
||||
- make up-build
|
||||
- 预期会下载依赖并成功启动
|
||||
- 第二次运行:
|
||||
- make up-build
|
||||
- 预期 npm/pip/apt/playwright 依赖明显走缓存,构建时间显著下降
|
||||
- 普通重启:
|
||||
- make down && make up
|
||||
- 预期不触发镜像构建,也不下载依赖
|
||||
- 修改前端/后端业务代码后:
|
||||
- make up-build
|
||||
- 预期只重跑源码 COPY 后的构建步骤,不重新下载 npm/pip 依赖
|
||||
- 修改 frontend/package-lock.json 或 backend/requirements.txt 后:
|
||||
- make up-build
|
||||
- 预期只对应依赖层重新安装
|
||||
|
||||
## Assumptions
|
||||
|
||||
- 本机 Docker 支持 BuildKit cache mount。
|
||||
- 日常 make up 的语义应是“启动服务”,不是“每次强制构建”。
|
||||
- 需要更新镜像时,显式使用 make build 或 make up-build。
|
||||
- 优化网站分组接口错误提示
|
||||
|
||||
## Summary
|
||||
|
||||
当前错误来自 httpx.HTTPStatusError 的原始字符串,包含完整 URL、MDN 链接和两个 fallback endpoint,展示很丑。应在后端 Sub2ApiWebsiteClient 里把常见 HTTP 错误归一化成中文业务错误,前端继续显示 detail 即可。
|
||||
|
||||
## Key Changes
|
||||
|
||||
- 在 backend/app/services/website_client.py 增加错误格式化:
|
||||
- 捕获 httpx.HTTPStatusError
|
||||
- 401 显示:目标网站认证失败,请检查 Admin API Key / JWT 是否正确
|
||||
- 403 显示:目标网站权限不足,请检查当前凭证是否有分组管理权限
|
||||
- 404 显示:目标网站接口不存在,请检查 API Prefix 和分组接口路径
|
||||
- 5xx 显示:目标网站服务异常,请稍后重试
|
||||
- 网络/超时显示:无法连接目标网站 或 目标网站请求超时
|
||||
- 优化 get_groups() fallback 错误聚合:
|
||||
- 如果 /groups 和 /groups/all 都是同一类认证错误,只返回一条友好提示
|
||||
- 不再把完整 URL 和 MDN 链接暴露给用户
|
||||
- 可保留简短接口路径信息,例如:尝试接口:/groups、/groups/all
|
||||
- 保持后端日志可排查:
|
||||
- 用户看到中文友好错误
|
||||
- logger 里仍记录原始异常和 URL,便于排查
|
||||
- 前端 Websites.vue 不需要大改:
|
||||
- 继续使用 e.response?.data?.detail || '拉取分组失败'
|
||||
- 展示出来会变成后端清洗后的中文错误
|
||||
|
||||
## Test Plan
|
||||
|
||||
- 目标网站 Admin API Key 配错:
|
||||
- 点击“拉取分组”或打开导入弹窗
|
||||
- 预期提示:目标网站认证失败,请检查 Admin API Key / JWT 是否正确
|
||||
- 不再出现完整 URL、MDN 链接、/groups/all 原始异常串
|
||||
- API Prefix 配错:
|
||||
- 预期提示接口不存在或路径配置错误
|
||||
- 目标网站关闭或地址不可达:
|
||||
- 预期提示无法连接/请求超时
|
||||
- 正常凭证:
|
||||
- 拉取分组仍正常
|
||||
- 补充单测:
|
||||
- 覆盖 401 双 fallback 聚合
|
||||
- 覆盖 404、超时、网络错误的友好消息
|
||||
|
||||
## Assumptions
|
||||
|
||||
- 这类错误都属于目标 Sub2API 网站调用失败,不应触发 SmartUp 自身退出登录。
|
||||
- 用户界面优先展示可操作建议;原始 HTTP 细节只保留在日志中。
|
||||
- 修复网站管理编辑入口不明显
|
||||
|
||||
## Summary
|
||||
|
||||
编辑功能已经存在,但当前是低对比度 icon-only 圆按钮,和多个操作图标挤在一起,用户很难识别。改为把“编辑”作为显性文字按钮放到操作列第一位,同时保留其他低强调图标按钮。
|
||||
|
||||
## Key Changes
|
||||
|
||||
- 调整网站列表操作列:
|
||||
- 把“编辑”移到第一个
|
||||
- 改成文字按钮:编辑
|
||||
- 可带图标:<Edit /> 编辑
|
||||
- 其他操作继续 icon-only:
|
||||
- 查看分组、连接测试、新增绑定、导入分组、导入账号、删除
|
||||
- 操作列宽度从 232 增加到 280 或改为 min-width="280"
|
||||
- .action-row 间距从 2px 增加到 6px
|
||||
- 给编辑按钮加稳定样式:
|
||||
- 不用 type="primary"
|
||||
- 使用 text/透明背景
|
||||
- hover 时有轻微背景
|
||||
- 不和删除按钮一样用 danger
|
||||
- tooltip 保留:
|
||||
- 编辑按钮仍可 tooltip “编辑网站配置”
|
||||
|
||||
## Test Plan
|
||||
|
||||
- 打开网站管理:
|
||||
- 每行操作列第一个能直接看到“编辑”
|
||||
- 点击“编辑”打开网站抽屉
|
||||
- 可以修改 Base URL、API Prefix、Admin API Key/JWT、超时等字段
|
||||
- 检查视觉:
|
||||
- 编辑按钮不被其他按钮遮挡
|
||||
- 操作按钮不重叠
|
||||
- hover tooltip 正常
|
||||
- 窄屏检查:
|
||||
- 表格横向滚动时操作列仍可用
|
||||
- 构建:
|
||||
- npm --prefix frontend run build
|
||||
|
||||
## Assumptions
|
||||
|
||||
- 问题不是点击事件失效,而是入口不可见/不可识别。
|
||||
- 行内按钮仍遵循之前统一过的低强调风格,不恢复高亮实心按钮。
|
||||
- 修复网站管理操作列按钮被裁切
|
||||
|
||||
## Summary
|
||||
|
||||
截图显示不是层级问题,而是操作列宽度仍不够。当前操作列里放了“编辑”文字按钮 + 6 个圆形图标按钮,width=280 放不下,左侧按钮被裁切,只露出右侧几个图标和 hover 的一部分。需要减少行内按钮数量,把次要操作收进“更多”菜单,保证“编辑”稳定可见。
|
||||
|
||||
## Key Changes
|
||||
|
||||
- 操作列改为固定少量主操作:
|
||||
- 编辑:文字按钮,永远显示,排第一个
|
||||
- 查看分组:文字或图标按钮,永远显示
|
||||
- 更多:下拉菜单
|
||||
- 删除:保留 icon-only danger,或放入更多菜单底部
|
||||
- 将低频操作移入 el-dropdown:
|
||||
- 连接测试
|
||||
- 新增绑定
|
||||
- 导入上游分组
|
||||
- 导入为账号管理账号
|
||||
- 操作列宽度调整为稳定值:
|
||||
- 建议 width="240" 或 min-width="240"
|
||||
- 因为只显示 3-4 个入口,不再需要 280+
|
||||
- .action-row 保持 flex-wrap: nowrap,但不再有 7 个按钮挤在一列
|
||||
- 给 更多 菜单项加清晰文字:
|
||||
- 连接测试
|
||||
- 新增绑定
|
||||
- 导入上游分组
|
||||
- 导入为账号管理账号
|
||||
- 保留现有低强调风格:
|
||||
- 行内不使用实心 primary/success
|
||||
- 删除仍是 danger 低强调 hover
|
||||
|
||||
## Suggested UI
|
||||
|
||||
操作列展示为:
|
||||
|
||||
[编辑] [查看分组] [更多 ▾] [删除]
|
||||
|
||||
更多菜单:
|
||||
|
||||
连接测试
|
||||
新增绑定
|
||||
导入上游分组
|
||||
导入为账号管理账号
|
||||
|
||||
## Test Plan
|
||||
|
||||
- 打开网站管理:
|
||||
- “编辑”在每行操作列左侧完整可见
|
||||
- “查看分组”完整可见
|
||||
- “更多”点击后能看到 4 个次要操作
|
||||
- 删除按钮可见且仍为低强调 danger
|
||||
- 点击验证:
|
||||
- 编辑打开网站抽屉
|
||||
- 查看分组正常加载
|
||||
- 更多里的连接测试、新增绑定、导入分组、导入账号都能触发原函数
|
||||
- 宽屏和窄屏:
|
||||
- 操作列不再裁切左侧按钮
|
||||
- 表格横向滚动时操作列内容仍完整
|
||||
- 构建:
|
||||
- npm --prefix frontend run build
|
||||
|
||||
## Assumptions
|
||||
|
||||
- 这个问题的根因是操作列内容过多导致裁切,不是按钮点击事件失效。
|
||||
- 高频操作是“编辑”和“查看分组”;导入、绑定、测试属于低频操作,可以收进更多菜单。
|
||||
- 上游 SmartUp 分组 Key 唯一化与同步清理
|
||||
|
||||
## Summary
|
||||
|
||||
修复“创建分组 Key 可重复点击导致远端/本地重复”的问题。目标是每个上游分组最多只有一个 SmartUp 前缀 Key,本地数据库也只保留一条;如果远端 Key 被删除,本地同步删除;Key 的本地更新时间与上游分组检测时间保持一致。
|
||||
|
||||
## Key Changes
|
||||
|
||||
- 后端 Key 命名规则改为稳定名称:
|
||||
- 不再在 key_name 里追加时间戳
|
||||
- 固定为:{name_prefix}-{upstream.name}-{group_name}
|
||||
- 默认前缀仍是 SmartUp
|
||||
- 创建前先同步远端已有 Key:
|
||||
- 调用 Sub2API GET /api/v1/keys,用 search={prefix}、group_id={group_id}、status=active 查远端 Key
|
||||
- 匹配规则:同一上游分组下,name 以 SmartUp 或用户填写前缀开头,且名称等于稳定 Key 名
|
||||
- 如果远端已存在:不创建新 Key,只 upsert 本地记录为该远端 Key
|
||||
- 如果远端不存在:创建一个新 Key,并 upsert 本地记录
|
||||
- 本地数据库唯一化:
|
||||
- upstream_generated_keys 增加唯一维度:upstream_id + group_id + key_name
|
||||
- SQLite 迁移时先清理历史重复:
|
||||
- 同一 upstream_id/group_id/key_name 只保留最新一条
|
||||
- 旧的重复记录删除
|
||||
- 之后创建唯一索引,防止并发/重复点击再次插入多条
|
||||
- 同步远端删除:
|
||||
- 在“上游检测成功”时同步该上游的 SmartUp Key 状态
|
||||
- 拉取远端 Key 列表后,如果本地记录的 key_id 在远端不存在,则删除本地记录
|
||||
- 如果上游分组本身在最新检测快照里不存在,也删除该分组对应的本地 SmartUp Key 记录
|
||||
- 检测时间一致:
|
||||
- UpstreamGeneratedKey 增加 updated_at 或 last_checked_at
|
||||
- 每次上游检测成功并同步 Key 后,把相关 Key 的更新时间设置为本次快照 captured_at
|
||||
- 详情抽屉展示这个更新时间,语义为“随上游检测同步于”
|
||||
- 前端防重复点击:
|
||||
- 生成 Key 按钮在 generatingKeys 时禁用,避免连续点击
|
||||
- 创建结果区对“已存在,已复用”和“新创建”做区分
|
||||
- 文案改为“确保每个分组有一个 SmartUp Key”,不再暗示每次都新建
|
||||
- 上游列表操作按钮:
|
||||
- 仍保留“创建分组 Key”
|
||||
- 点击后执行“确保存在/同步”语义,不执行盲目创建
|
||||
|
||||
## API / Client Changes
|
||||
|
||||
- UpstreamClient 增加:
|
||||
- list_api_keys(search, group_id, status, endpoint="/keys")
|
||||
- delete_api_key(key_id, endpoint="/keys/{id}") 可选备用
|
||||
- find_smartup_group_key(group_id, key_name, prefix)
|
||||
- POST /api/upstreams/{id}/keys/generate-by-groups 行为变更:
|
||||
- 从“总是创建”改为“存在则复用,不存在才创建”
|
||||
- 返回 item status:
|
||||
- created:本次新建
|
||||
- exists:远端已存在,本地已同步
|
||||
- failed:失败
|
||||
- GET /api/upstreams/{id}/generated-keys:
|
||||
- 只返回当前本地有效记录
|
||||
- 不再返回远端已删除或分组已不存在的残留记录
|
||||
|
||||
## Test Plan
|
||||
|
||||
- 单测:重复创建
|
||||
- 同一上游同一分组连续调用两次 generate
|
||||
- 预期远端 create 只调用一次
|
||||
- 本地 upstream_generated_keys 只有一条
|
||||
- 第二次返回 exists
|
||||
- 单测:历史重复清理
|
||||
- 预置同一 upstream_id/group_id/key_name 多条记录
|
||||
- 运行迁移/清理函数
|
||||
- 预期只保留最新一条
|
||||
- 单测:远端删除同步
|
||||
- 本地有 Key,远端 GET /keys 不返回该 key_id
|
||||
- 运行上游检测同步
|
||||
- 预期本地记录被删除
|
||||
- 单测:分组删除同步
|
||||
- 最新分组快照不包含某 group_id
|
||||
- 预期该 group_id 的 SmartUp Key 本地记录被删除
|
||||
- 单测:检测时间同步
|
||||
- 上游检测成功写入 snapshot captured_at
|
||||
- 预期相关 Key 的 updated_at/last_checked_at 等于该检测时间
|
||||
- 前端验证:
|
||||
- 快速多次点击创建按钮不会产生多条
|
||||
- 已存在 Key 时显示“已存在,已同步”
|
||||
- 详情抽屉不再显示远端已删除的 Key
|
||||
- 回归:
|
||||
- npm --prefix frontend run build
|
||||
- env PYTHONPATH=backend backend/venv/bin/pytest backend/test_upstream_key_account_import.py backend/test_upstream.py -v
|
||||
|
||||
## Assumptions
|
||||
|
||||
- 只管理 SmartUp 或用户在弹窗中填写的前缀创建的 Key,不清理用户手动创建的普通 Key。
|
||||
- “每个上游分组只能有一个 SmartUp 前缀 Key”按 upstream_id + group_id + stable key_name 判断。
|
||||
- 远端 Sub2API Key 列表接口为 GET /api/v1/keys,删除接口为 DELETE /api/v1/keys/{id}。
|
||||
- 远端 Key 被删除后,本地应物理删除,不保留 deleted 状态记录。
|
||||
@@ -512,13 +512,14 @@ def check_now(uid: int, db: Session = Depends(get_db), _=Depends(get_current_use
|
||||
|
||||
if was_unhealthy:
|
||||
webhook_service.send_status_event(db, u.id, u.name, u.base_url, "upstream_recovered")
|
||||
# 先同步 Key 状态(标记 orphaned),再执行优先级同步(避免未标记的 key 参与计算)
|
||||
from app.services.scheduler import _sync_upstream_keys as _synck
|
||||
_synck(uid, snapshot, new_row.captured_at)
|
||||
|
||||
if changes:
|
||||
webhook_service.send_rate_changed(db, u.id, u.name, u.base_url, changes)
|
||||
website_sync.sync_affected_bindings(db, u.id, changes)
|
||||
|
||||
# 同步 SmartUp Key 状态(使用实际快照入库时间,与定时任务一致)
|
||||
from app.services.scheduler import _sync_upstream_keys as _synck
|
||||
_synck(uid, snapshot, new_row.captured_at)
|
||||
website_sync.sync_account_priorities_for_upstream(db, u.id)
|
||||
|
||||
msg = f"检测成功,{len(groups)} 个分组"
|
||||
if changes:
|
||||
|
||||
@@ -32,7 +32,7 @@ from app.schemas.website import (
|
||||
WebsiteUpdate,
|
||||
)
|
||||
from app.services.website_client import Sub2ApiWebsiteClient
|
||||
from app.services.website_sync import binding_sources, sync_binding
|
||||
from app.services.website_sync import binding_sources, sync_binding, build_rate_priority_map
|
||||
from app.utils.auth import get_current_user
|
||||
|
||||
router = APIRouter(tags=["websites"])
|
||||
@@ -171,24 +171,10 @@ def _numeric_group_id(value: str | None) -> int | None:
|
||||
|
||||
def _build_rate_priority_map(db: Session, upstream_ids: set[int]) -> dict[str, int]:
|
||||
"""根据上游分组倍率构建 group_id → priority 映射。
|
||||
|
||||
遍历所有涉及的上游的最新快照,收集分组的倍率,按倍率升序排列后赋值 priority。
|
||||
倍率最低的 priority=1,次低的 priority=2,以此类推。相同倍率的分组共享同一 priority。
|
||||
|
||||
委托给 website_sync.build_rate_priority_map 避免逻辑重复。
|
||||
"""
|
||||
group_rates: dict[str, float] = {}
|
||||
for uid in upstream_ids:
|
||||
groups = _latest_upstream_groups(db, uid)
|
||||
for g in groups:
|
||||
gid = _source_group_id(g)
|
||||
rate = _source_group_rate(g)
|
||||
if gid:
|
||||
# 同一 group_id 在同个 upstream 内是唯一的;跨 upstream 的相同 group_id
|
||||
# 如果倍率不同则以最后遇到的为准(实际很少冲突)
|
||||
group_rates[gid] = rate
|
||||
# 按倍率排序分配 priority
|
||||
unique_rates = sorted(set(group_rates.values()))
|
||||
rate_to_priority = {rate: idx + 1 for idx, rate in enumerate(unique_rates)}
|
||||
return {gid: rate_to_priority[rate] for gid, rate in group_rates.items()}
|
||||
return build_rate_priority_map(db, upstream_ids)
|
||||
|
||||
|
||||
@router.get("/api/websites", response_model=List[WebsiteResponse])
|
||||
@@ -545,7 +531,7 @@ def import_upstream_keys_as_accounts(
|
||||
exists = c.account_exists(row.imported_account_id)
|
||||
if exists is True:
|
||||
# 自动更新已有账号的 priority(分步导入时全局倍率排序可能已变)
|
||||
new_priority = rate_priority_map.get(row.group_id) if body.auto_priority_by_rate else None
|
||||
new_priority = rate_priority_map.get(f"{row.upstream_id}:{row.group_id}") if body.auto_priority_by_rate else None
|
||||
priority_msg = "已导入过,已跳过"
|
||||
if new_priority is not None:
|
||||
try:
|
||||
@@ -616,7 +602,7 @@ def import_upstream_keys_as_accounts(
|
||||
"group_ids": group_ids,
|
||||
"rate_multiplier": 1,
|
||||
"concurrency": body.concurrency,
|
||||
"priority": rate_priority_map.get(row.group_id, body.priority) if body.auto_priority_by_rate else body.priority,
|
||||
"priority": rate_priority_map.get(f"{row.upstream_id}:{row.group_id}", body.priority) if body.auto_priority_by_rate else body.priority,
|
||||
"notes": f"Imported by SmartUp from upstream key #{row.id}",
|
||||
}
|
||||
try:
|
||||
|
||||
@@ -160,6 +160,7 @@ def _check_upstream(upstream_id: int) -> None:
|
||||
if changes:
|
||||
_notify_rate_changed(upstream_id, upstream.name, upstream.base_url, changes)
|
||||
_sync_website_bindings(upstream_id, changes)
|
||||
_sync_account_priorities(upstream_id)
|
||||
|
||||
if balance_alert_triggered:
|
||||
_notify_balance_low(
|
||||
@@ -284,6 +285,17 @@ def _sync_upstream_keys(upstream_id: int, snapshot: dict[str, Any], captured_at:
|
||||
db.close()
|
||||
|
||||
|
||||
def _sync_account_priorities(upstream_id: int) -> None:
|
||||
"""倍率变更后自动更新已导入下游账号的 priority。"""
|
||||
db = SessionLocal()
|
||||
try:
|
||||
website_sync.sync_account_priorities_for_upstream(db, upstream_id)
|
||||
except Exception:
|
||||
logger.exception("account priority sync failed for upstream %s", upstream_id)
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def _sync_website_bindings(upstream_id: int, changes: list[dict[str, Any]]) -> None:
|
||||
db = SessionLocal()
|
||||
try:
|
||||
|
||||
@@ -16,6 +16,7 @@ from app.utils.dingtalk import (
|
||||
format_dingtalk_website_rate_changed,
|
||||
format_dingtalk_status,
|
||||
format_dingtalk_balance_low,
|
||||
format_dingtalk_priority_changed,
|
||||
)
|
||||
|
||||
|
||||
@@ -223,6 +224,47 @@ def send_balance_low(
|
||||
_log(db, wh, event, generic_payload, "failed", str(exc))
|
||||
|
||||
|
||||
def send_account_priority_changed(
|
||||
db: Session,
|
||||
website_id: int,
|
||||
website_name: str,
|
||||
upstream_id: int,
|
||||
upstream_name: str,
|
||||
updates: list[dict],
|
||||
) -> None:
|
||||
webhooks = (
|
||||
db.query(WebhookConfig)
|
||||
.filter(WebhookConfig.enabled == True)
|
||||
.all()
|
||||
)
|
||||
event = "account_priority_changed"
|
||||
changed_at = _now_iso()
|
||||
success = sum(1 for u in updates if u.get("status") == "success")
|
||||
failed = sum(1 for u in updates if u.get("status") == "failed")
|
||||
skipped = sum(1 for u in updates if u.get("status") == "skipped")
|
||||
generic_payload = {
|
||||
"event": event,
|
||||
"website": {"id": website_id, "name": website_name},
|
||||
"upstream": {"id": upstream_id, "name": upstream_name},
|
||||
"changed_at": changed_at,
|
||||
"updates": updates,
|
||||
"summary": {"total": len(updates), "success": success, "failed": failed, "skipped": skipped},
|
||||
}
|
||||
for wh in webhooks:
|
||||
events = json.loads(wh.events_json or "[]")
|
||||
if event not in events:
|
||||
continue
|
||||
try:
|
||||
if wh.type == "dingtalk":
|
||||
msg = format_dingtalk_priority_changed(website_name, upstream_name, changed_at, updates)
|
||||
resp_text = _send_dingtalk(wh.url, wh.secret, msg)
|
||||
else:
|
||||
resp_text = _send_generic(wh.url, generic_payload)
|
||||
_log(db, wh, event, generic_payload, "success", resp_text)
|
||||
except Exception as exc:
|
||||
_log(db, wh, event, generic_payload, "failed", str(exc))
|
||||
|
||||
|
||||
def send_test_notification(db: Session, webhook: WebhookConfig) -> tuple[bool, str]:
|
||||
payload = {
|
||||
"event": "test",
|
||||
|
||||
@@ -9,6 +9,7 @@ from sqlalchemy.orm import Session
|
||||
|
||||
from app.models.snapshot import UpstreamRateSnapshot
|
||||
from app.models.upstream import Upstream
|
||||
from app.models.upstream_key import UpstreamGeneratedKey
|
||||
from app.models.website import Website, WebsiteGroupBinding, WebsiteSyncLog
|
||||
from app.services.website_client import Sub2ApiWebsiteClient, WebsiteError, calculate_target_rate, decimal_string
|
||||
from app.services import webhook_service
|
||||
@@ -171,6 +172,251 @@ def sync_binding(db: Session, binding: WebsiteGroupBinding, write: bool = True)
|
||||
return _log(db, binding, website, source_rates, "success", message, old_rate, target_rate)
|
||||
|
||||
|
||||
def _snapshot_group_rate(group: dict) -> float:
|
||||
"""从快照分组数据中提取倍率(兼容多个字段名)。"""
|
||||
raw = group.get("rate") or group.get("default_rate") or group.get("rate_multiplier") or 1
|
||||
try:
|
||||
return float(raw)
|
||||
except (TypeError, ValueError):
|
||||
return 1.0
|
||||
|
||||
|
||||
def build_rate_priority_map(db: Session, upstream_ids: set[int]) -> dict[str, int]:
|
||||
"""根据上游分组倍率构建 f"{upstream_id}:{group_id}" → priority 映射。
|
||||
|
||||
使用 (upstream_id, group_id) 复合键避免不同上游的同名分组互相覆盖。
|
||||
遍历所有涉及的上游的最新快照,收集分组的倍率,按倍率升序排列后赋值 priority。
|
||||
倍率最低的 priority=1,次低的 priority=2,以此类推。相同倍率的分组共享同一 priority。
|
||||
"""
|
||||
group_rates: dict[str, float] = {}
|
||||
for uid in upstream_ids:
|
||||
groups = latest_rate_map(db, uid)
|
||||
for gid, g in groups.items():
|
||||
if not isinstance(g, dict):
|
||||
continue
|
||||
rate = _snapshot_group_rate(g)
|
||||
key = f"{uid}:{gid}"
|
||||
group_rates[key] = rate
|
||||
unique_rates = sorted(set(group_rates.values()))
|
||||
rate_to_priority = {rate: idx + 1 for idx, rate in enumerate(unique_rates)}
|
||||
return {key: rate_to_priority[rate] for key, rate in group_rates.items()}
|
||||
|
||||
|
||||
def _priority_result(row, new_priority: int | None, status: str, message: str) -> dict:
|
||||
"""构建统一的优先级同步结果 dict。"""
|
||||
return {
|
||||
"account_id": row.imported_account_id,
|
||||
"group_id": row.group_id,
|
||||
"upstream_id": row.upstream_id,
|
||||
"old_priority": None,
|
||||
"new_priority": new_priority,
|
||||
"status": status,
|
||||
"message": message,
|
||||
}
|
||||
|
||||
|
||||
def _write_priority_sync_log_with_map(
|
||||
db: Session, wid: int, upstream_name: str,
|
||||
results: list[dict], priority_map: dict[str, int],
|
||||
) -> None:
|
||||
"""写入 priority_sync 日志,同时保存账号明细和 priority_map 快照。
|
||||
|
||||
source_rates_json 格式:[{"_meta": "priority_map", "data": {...}}, {"account_id": ..., ...}, ...]
|
||||
兼容 WebsiteSyncLogResponse.source_rates: list[dict] 类型约束。
|
||||
"""
|
||||
log_results: list[dict] = [
|
||||
{"_meta": "priority_map", "data": dict(priority_map)},
|
||||
]
|
||||
log_results.extend(results)
|
||||
success = sum(1 for r in results if r["status"] == "success")
|
||||
failed = sum(1 for r in results if r["status"] == "failed")
|
||||
skipped = sum(1 for r in results if r["status"] == "skipped")
|
||||
parts = []
|
||||
if success:
|
||||
parts.append(f"{success} 个更新成功")
|
||||
if failed:
|
||||
parts.append(f"{failed} 个失败")
|
||||
if skipped:
|
||||
parts.append(f"{skipped} 个跳过")
|
||||
log = WebsiteSyncLog(
|
||||
website_id=wid,
|
||||
binding_id=None,
|
||||
target_group_id="",
|
||||
target_group_name="",
|
||||
algorithm="priority_sync",
|
||||
percent=0,
|
||||
source_rates_json=json.dumps(log_results, ensure_ascii=False, default=str),
|
||||
old_rate=None,
|
||||
new_rate=None,
|
||||
status="failed" if failed else "success",
|
||||
message=f"优先级同步(上游={upstream_name}):{'、'.join(parts)} / 共 {len(results)} 个",
|
||||
)
|
||||
db.add(log)
|
||||
db.commit()
|
||||
|
||||
|
||||
def _try_send_priority_webhook(
|
||||
db: Session, wid: int, website_name: str,
|
||||
upstream_id: int, upstream_name: str,
|
||||
updates: list[dict],
|
||||
) -> None:
|
||||
"""发送 account_priority_changed webhook,失败不抛异常。"""
|
||||
if not updates:
|
||||
return
|
||||
# 如果没传入名称,尝试从 DB 查
|
||||
resolved_name = website_name
|
||||
if not resolved_name:
|
||||
row = db.query(Website.name).filter(Website.id == wid).first()
|
||||
if row:
|
||||
resolved_name = row[0]
|
||||
else:
|
||||
resolved_name = f"网站#{wid}"
|
||||
try:
|
||||
webhook_service.send_account_priority_changed(
|
||||
db,
|
||||
website_id=wid,
|
||||
website_name=resolved_name,
|
||||
upstream_id=upstream_id,
|
||||
upstream_name=upstream_name,
|
||||
updates=updates,
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.warning("account_priority_changed webhook failed for website %s: %s", wid, exc)
|
||||
|
||||
|
||||
def sync_account_priorities_for_upstream(db: Session, upstream_id: int) -> list[dict]:
|
||||
"""上游倍率变化后,自动更新已导入下游账号的 priority。
|
||||
|
||||
查询该上游下所有已导入(非 orphaned)的 Key,按目标网站分组后重新计算全局优先级,
|
||||
并通过 update_account API 推送到下游网站。返回详细结果列表。
|
||||
|
||||
同时写入 WebsiteSyncLog 持久化审计日志,并通过 webhook 发送通知。
|
||||
"""
|
||||
from app.services.website_client import Sub2ApiWebsiteClient as Client
|
||||
|
||||
key_rows = (
|
||||
db.query(UpstreamGeneratedKey)
|
||||
.filter(
|
||||
UpstreamGeneratedKey.upstream_id == upstream_id,
|
||||
UpstreamGeneratedKey.imported_website_id.isnot(None),
|
||||
UpstreamGeneratedKey.imported_account_id.isnot(None),
|
||||
UpstreamGeneratedKey.status != "orphaned",
|
||||
)
|
||||
.all()
|
||||
)
|
||||
if not key_rows:
|
||||
return []
|
||||
|
||||
upstream_name = db.query(Upstream.name).filter(Upstream.id == upstream_id).scalar() or f"#{upstream_id}"
|
||||
|
||||
# 按 imported_website_id 分组
|
||||
website_groups: dict[int, list[UpstreamGeneratedKey]] = {}
|
||||
for row in key_rows:
|
||||
wid = row.imported_website_id
|
||||
if wid not in website_groups:
|
||||
website_groups[wid] = []
|
||||
website_groups[wid].append(row)
|
||||
|
||||
all_results: list[dict] = []
|
||||
|
||||
for wid, rows in website_groups.items():
|
||||
website = db.query(Website).filter(Website.id == wid).first()
|
||||
if not website or not website.enabled:
|
||||
logger.info("skip account priority sync: website %s not found or disabled", wid)
|
||||
site_results = []
|
||||
for row in rows:
|
||||
r = _priority_result(row, None, "failed", "网站不可用")
|
||||
site_results.append(r)
|
||||
all_results.append(r)
|
||||
_write_priority_sync_log_with_map(db, wid, upstream_name, site_results, {})
|
||||
_try_send_priority_webhook(db, wid, "", upstream_id, upstream_name, site_results)
|
||||
continue
|
||||
|
||||
# 查询该网站所有已导入 Key(跨上游),实现全局优先级排序
|
||||
all_website_keys = (
|
||||
db.query(UpstreamGeneratedKey)
|
||||
.filter(
|
||||
UpstreamGeneratedKey.imported_website_id == wid,
|
||||
UpstreamGeneratedKey.imported_account_id.isnot(None),
|
||||
UpstreamGeneratedKey.status != "orphaned",
|
||||
)
|
||||
.all()
|
||||
)
|
||||
all_upstream_ids = {k.upstream_id for k in all_website_keys}
|
||||
try:
|
||||
priority_map = build_rate_priority_map(db, all_upstream_ids)
|
||||
except Exception as exc:
|
||||
logger.warning("build_rate_priority_map failed for website %s: %s", wid, exc)
|
||||
site_results = []
|
||||
for row in all_website_keys:
|
||||
r = _priority_result(row, None, "failed", f"构建优先级映射失败: {exc}")
|
||||
site_results.append(r)
|
||||
all_results.append(r)
|
||||
_write_priority_sync_log_with_map(db, wid, upstream_name, site_results, {})
|
||||
_try_send_priority_webhook(db, wid, "", upstream_id, upstream_name, site_results)
|
||||
continue
|
||||
|
||||
if not priority_map:
|
||||
logger.info("skip account priority sync for website %s: empty priority map", wid)
|
||||
site_results = []
|
||||
for row in all_website_keys:
|
||||
r = _priority_result(row, None, "skipped", "无上游倍率数据")
|
||||
site_results.append(r)
|
||||
all_results.append(r)
|
||||
_write_priority_sync_log_with_map(db, wid, upstream_name, site_results, {})
|
||||
_try_send_priority_webhook(db, wid, "", upstream_id, upstream_name, site_results)
|
||||
continue
|
||||
|
||||
site_results: list[dict] = []
|
||||
try:
|
||||
with Client(
|
||||
base_url=website.base_url,
|
||||
api_prefix=website.api_prefix,
|
||||
auth_type=website.auth_type,
|
||||
auth_config=json.loads(website.auth_config_json or "{}"),
|
||||
timeout=float(website.timeout_seconds),
|
||||
) as client:
|
||||
for row in all_website_keys:
|
||||
account_id = row.imported_account_id
|
||||
if not account_id:
|
||||
continue
|
||||
new_priority = priority_map.get(f"{row.upstream_id}:{row.group_id}")
|
||||
if new_priority is None:
|
||||
site_results.append(
|
||||
_priority_result(row, None, "skipped", "无倍率数据,跳过")
|
||||
)
|
||||
continue
|
||||
try:
|
||||
client.update_account(account_id, {"priority": new_priority})
|
||||
logger.info(
|
||||
"updated priority for account %s (website=%s, upstream=%s, group=%s): %s",
|
||||
account_id, wid, row.upstream_id, row.group_id, new_priority,
|
||||
)
|
||||
site_results.append(
|
||||
_priority_result(row, new_priority, "success", f"优先级已更新为 {new_priority}")
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.warning(
|
||||
"failed to update priority for account %s (website=%s): %s",
|
||||
account_id, wid, exc,
|
||||
)
|
||||
site_results.append(
|
||||
_priority_result(row, new_priority, "failed", str(exc))
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.warning("failed to connect website %s for account priority sync: %s", wid, exc)
|
||||
for row in all_website_keys:
|
||||
site_results.append(
|
||||
_priority_result(row, None, "failed", f"连接网站失败: {exc}")
|
||||
)
|
||||
|
||||
all_results.extend(site_results)
|
||||
_write_priority_sync_log_with_map(db, wid, upstream_name, site_results, priority_map)
|
||||
_try_send_priority_webhook(db, wid, website.name, upstream_id, upstream_name, site_results)
|
||||
|
||||
return all_results
|
||||
|
||||
|
||||
def sync_affected_bindings(db: Session, upstream_id: int, changes: list[dict[str, Any]]) -> None:
|
||||
for binding in get_affected_bindings(db, changes, upstream_id):
|
||||
try:
|
||||
|
||||
@@ -64,6 +64,35 @@ def format_dingtalk_website_rate_changed(
|
||||
}
|
||||
|
||||
|
||||
def format_dingtalk_priority_changed(
|
||||
website_name: str, upstream_name: str, changed_at: str,
|
||||
updates: list[dict],
|
||||
) -> dict[str, Any]:
|
||||
success = sum(1 for u in updates if u.get("status") == "success")
|
||||
failed = sum(1 for u in updates if u.get("status") == "failed")
|
||||
skipped = sum(1 for u in updates if u.get("status") == "skipped")
|
||||
lines = [
|
||||
f"### 🔄 {website_name} 账号优先级变更",
|
||||
"",
|
||||
f"- **触发上游**:{upstream_name}",
|
||||
f"- **时间**:{changed_at}",
|
||||
f"- **摘要**:{success} 更新 / {failed} 失败 / {skipped} 跳过",
|
||||
"",
|
||||
]
|
||||
for u in updates:
|
||||
emoji = {"success": "✅", "failed": "❌", "skipped": "⏭️"}.get(u.get("status", ""), "➖")
|
||||
gid = u.get("group_id", "?")
|
||||
priority = u.get("new_priority", "—")
|
||||
lines.append(f"{emoji} `{gid}` → priority={priority}")
|
||||
return {
|
||||
"msgtype": "markdown",
|
||||
"markdown": {
|
||||
"title": f"{website_name} 账号优先级变更",
|
||||
"text": "\n".join(lines),
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def format_dingtalk_balance_low(
|
||||
upstream_name: str, balance: float, threshold: float, changed_at: str
|
||||
) -> dict[str, Any]:
|
||||
|
||||
@@ -0,0 +1,221 @@
|
||||
import json
|
||||
from datetime import datetime, timezone
|
||||
import pytest
|
||||
from sqlalchemy import create_engine
|
||||
from sqlalchemy.orm import sessionmaker
|
||||
from sqlalchemy.pool import StaticPool
|
||||
|
||||
from app.database import Base
|
||||
from app.models.upstream import Upstream
|
||||
from app.models.website import Website, WebsiteSyncLog
|
||||
from app.models.upstream_key import UpstreamGeneratedKey
|
||||
from app.models.snapshot import UpstreamRateSnapshot
|
||||
from app.services.website_sync import (
|
||||
build_rate_priority_map,
|
||||
sync_account_priorities_for_upstream
|
||||
)
|
||||
from app.services.website_client import Sub2ApiWebsiteClient
|
||||
|
||||
@pytest.fixture()
|
||||
def db_session():
|
||||
engine = create_engine(
|
||||
"sqlite://",
|
||||
connect_args={"check_same_thread": False},
|
||||
poolclass=StaticPool,
|
||||
)
|
||||
Base.metadata.create_all(bind=engine)
|
||||
TestingSessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
|
||||
db = TestingSessionLocal()
|
||||
try:
|
||||
yield db
|
||||
finally:
|
||||
db.close()
|
||||
Base.metadata.drop_all(bind=engine)
|
||||
|
||||
def test_priority_sync_cross_upstream_group(db_session):
|
||||
# Setup 2 upstreams
|
||||
u1 = Upstream(name="U1", base_url="http://u1")
|
||||
u2 = Upstream(name="U2", base_url="http://u2")
|
||||
db_session.add_all([u1, u2])
|
||||
db_session.commit()
|
||||
db_session.refresh(u1)
|
||||
db_session.refresh(u2)
|
||||
|
||||
# Setup snapshots for both with same group ID "VIP" but different rates
|
||||
s1 = UpstreamRateSnapshot(
|
||||
upstream_id=u1.id,
|
||||
snapshot_json=json.dumps({"groups": {"VIP": {"group_name": "VIP", "rate": 1.0}}}),
|
||||
captured_at=datetime.now(timezone.utc)
|
||||
)
|
||||
s2 = UpstreamRateSnapshot(
|
||||
upstream_id=u2.id,
|
||||
snapshot_json=json.dumps({"groups": {"VIP": {"group_name": "VIP", "rate": 2.0}}}),
|
||||
captured_at=datetime.now(timezone.utc)
|
||||
)
|
||||
db_session.add_all([s1, s2])
|
||||
db_session.commit()
|
||||
|
||||
priority_map = build_rate_priority_map(db_session, {u1.id, u2.id})
|
||||
|
||||
assert priority_map[f"{u1.id}:VIP"] == 1
|
||||
assert priority_map[f"{u2.id}:VIP"] == 2
|
||||
assert len(priority_map) == 2
|
||||
|
||||
def test_priority_sync_full_website_update(db_session, monkeypatch):
|
||||
# Setup website and upstreams
|
||||
w = Website(name="W1", base_url="http://w1", enabled=True, auth_config_json="{}", timeout_seconds=30)
|
||||
u1 = Upstream(name="U1", base_url="http://u1")
|
||||
u2 = Upstream(name="U2", base_url="http://u2")
|
||||
db_session.add_all([w, u1, u2])
|
||||
db_session.commit()
|
||||
db_session.refresh(w)
|
||||
db_session.refresh(u1)
|
||||
db_session.refresh(u2)
|
||||
|
||||
# Setup snapshots
|
||||
db_session.add(UpstreamRateSnapshot(
|
||||
upstream_id=u1.id,
|
||||
snapshot_json=json.dumps({"groups": {"G1": {"rate": 1.0}}}),
|
||||
captured_at=datetime.now(timezone.utc)
|
||||
))
|
||||
db_session.add(UpstreamRateSnapshot(
|
||||
upstream_id=u2.id,
|
||||
snapshot_json=json.dumps({"groups": {"G2": {"rate": 2.0}}}),
|
||||
captured_at=datetime.now(timezone.utc)
|
||||
))
|
||||
db_session.commit()
|
||||
|
||||
# Setup keys imported to website
|
||||
k1 = UpstreamGeneratedKey(upstream_id=u1.id, group_id="G1", key_name="K1", key_value="V1",
|
||||
imported_website_id=w.id, imported_account_id="A1")
|
||||
k2 = UpstreamGeneratedKey(upstream_id=u2.id, group_id="G2", key_name="K2", key_value="V2",
|
||||
imported_website_id=w.id, imported_account_id="A2")
|
||||
db_session.add_all([k1, k2])
|
||||
db_session.commit()
|
||||
|
||||
# Mock Sub2ApiWebsiteClient
|
||||
update_calls = []
|
||||
class MockClient:
|
||||
def __init__(self, **kwargs): pass
|
||||
def __enter__(self): return self
|
||||
def __exit__(self, *args): pass
|
||||
def update_account(self, account_id, data):
|
||||
update_calls.append((account_id, data))
|
||||
|
||||
monkeypatch.setattr("app.services.website_sync.Sub2ApiWebsiteClient", MockClient)
|
||||
monkeypatch.setattr("app.services.website_client.Sub2ApiWebsiteClient", MockClient)
|
||||
|
||||
# Trigger sync for U1
|
||||
sync_account_priorities_for_upstream(db_session, u1.id)
|
||||
|
||||
# Verify BOTH A1 and A2 were updated because they belong to the same website
|
||||
assert len(update_calls) == 2
|
||||
account_ids = {c[0] for c in update_calls}
|
||||
assert account_ids == {"A1", "A2"}
|
||||
|
||||
# Priority check: G1(1.0) -> 1, G2(2.0) -> 2
|
||||
for aid, data in update_calls:
|
||||
if aid == "A1": assert data["priority"] == 1
|
||||
if aid == "A2": assert data["priority"] == 2
|
||||
|
||||
def test_priority_sync_log_structure(db_session, monkeypatch):
|
||||
w = Website(name="W1", base_url="http://w1", enabled=True, auth_config_json="{}", timeout_seconds=30)
|
||||
u1 = Upstream(name="U1", base_url="http://u1")
|
||||
db_session.add_all([w, u1])
|
||||
db_session.commit()
|
||||
db_session.refresh(w)
|
||||
db_session.refresh(u1)
|
||||
|
||||
db_session.add(UpstreamRateSnapshot(
|
||||
upstream_id=u1.id,
|
||||
snapshot_json=json.dumps({"groups": {"G1": {"rate": 1.0}}}),
|
||||
captured_at=datetime.now(timezone.utc)
|
||||
))
|
||||
db_session.add(UpstreamGeneratedKey(upstream_id=u1.id, group_id="G1", key_name="K1", key_value="V1",
|
||||
imported_website_id=w.id, imported_account_id="A1"))
|
||||
db_session.commit()
|
||||
|
||||
class MockClient:
|
||||
def __init__(self, **kwargs): pass
|
||||
def __enter__(self): return self
|
||||
def __exit__(self, *args): pass
|
||||
def update_account(self, account_id, data): pass
|
||||
|
||||
monkeypatch.setattr("app.services.website_sync.Sub2ApiWebsiteClient", MockClient)
|
||||
monkeypatch.setattr("app.services.website_client.Sub2ApiWebsiteClient", MockClient)
|
||||
|
||||
sync_account_priorities_for_upstream(db_session, u1.id)
|
||||
|
||||
log = db_session.query(WebsiteSyncLog).filter(WebsiteSyncLog.website_id == w.id).first()
|
||||
assert log is not None
|
||||
assert log.algorithm == "priority_sync"
|
||||
|
||||
data = json.loads(log.source_rates_json)
|
||||
# The first item should be the priority map metadata
|
||||
assert data[0]["_meta"] == "priority_map"
|
||||
assert f"{u1.id}:G1" in data[0]["data"]
|
||||
# The second item should be the account result
|
||||
assert data[1]["account_id"] == "A1"
|
||||
|
||||
def test_import_auto_priority_by_rate(db_session, monkeypatch):
|
||||
from app.routers.websites import import_upstream_keys_as_accounts
|
||||
from app.schemas.website import ImportAccountsRequest
|
||||
|
||||
w = Website(name="W1", base_url="http://w1", enabled=True, auth_config_json="{}",
|
||||
groups_endpoint="/groups", group_update_endpoint="/groups/{id}", timeout_seconds=30)
|
||||
u1 = Upstream(name="U1", base_url="http://u1")
|
||||
u2 = Upstream(name="U2", base_url="http://u2")
|
||||
db_session.add_all([w, u1, u2])
|
||||
db_session.commit()
|
||||
db_session.refresh(w)
|
||||
db_session.refresh(u1)
|
||||
db_session.refresh(u2)
|
||||
|
||||
db_session.add(UpstreamRateSnapshot(
|
||||
upstream_id=u1.id,
|
||||
snapshot_json=json.dumps({"groups": {"G1": {"rate": 2.0}}}),
|
||||
captured_at=datetime.now(timezone.utc)
|
||||
))
|
||||
db_session.add(UpstreamRateSnapshot(
|
||||
upstream_id=u2.id,
|
||||
snapshot_json=json.dumps({"groups": {"G2": {"rate": 1.0}}}),
|
||||
captured_at=datetime.now(timezone.utc)
|
||||
))
|
||||
|
||||
k1 = UpstreamGeneratedKey(upstream_id=u1.id, group_id="G1", group_name="G1", key_name="K1", key_value="V1")
|
||||
k2 = UpstreamGeneratedKey(upstream_id=u2.id, group_id="G2", group_name="G2", key_name="K2", key_value="V2")
|
||||
db_session.add_all([k1, k2])
|
||||
db_session.commit()
|
||||
|
||||
created_accounts = []
|
||||
class MockClient:
|
||||
def __init__(self, **kwargs): pass
|
||||
def __enter__(self): return self
|
||||
def __exit__(self, *args): pass
|
||||
def create_account(self, body):
|
||||
created_accounts.append(body)
|
||||
return {"id": f"remote-{len(created_accounts)}", "name": body["name"]}
|
||||
def extract_id(self, data): return data["id"]
|
||||
def account_exists(self, aid): return False
|
||||
|
||||
monkeypatch.setattr("app.routers.websites._client", lambda website: MockClient())
|
||||
monkeypatch.setattr("app.services.website_client.Sub2ApiWebsiteClient", MockClient)
|
||||
|
||||
req = ImportAccountsRequest(
|
||||
upstream_key_ids=[k1.id, k2.id],
|
||||
target_group_map={},
|
||||
auto_priority_by_rate=True,
|
||||
priority=10,
|
||||
account_name_prefix="test",
|
||||
default_platform="openai"
|
||||
)
|
||||
|
||||
import_upstream_keys_as_accounts(w.id, req, db_session)
|
||||
|
||||
assert len(created_accounts) == 2
|
||||
# G2 has rate 1.0 -> priority 1
|
||||
# G1 has rate 2.0 -> priority 2
|
||||
p1 = next(a["priority"] for a in created_accounts if "G1" in a["name"])
|
||||
p2 = next(a["priority"] for a in created_accounts if "G2" in a["name"])
|
||||
assert p2 == 1
|
||||
assert p1 == 2
|
||||
@@ -0,0 +1,28 @@
|
||||
# SmartUp Brand Spec
|
||||
|
||||
## Identity
|
||||
|
||||
- Product name: `SmartUp`
|
||||
- Product type: API upstream monitoring and webhook notification console
|
||||
- Brand posture: controlled, operational, industrial, high-signal
|
||||
|
||||
## Assets
|
||||
|
||||
- Primary logo mark: [frontend/public/favicon.svg](/home/liumangmang/GiteaRepos/LiuMangMang/SmartUp/frontend/public/favicon.svg)
|
||||
- Logo usage: use the existing brass-toned lightning mark as the product symbol; do not replace it with emoji
|
||||
- Product imagery: none in repository
|
||||
- UI screenshots: none committed; redesign is based on the current Vue codebase
|
||||
|
||||
## Visual System
|
||||
|
||||
- Primary accent: warm brass / ember
|
||||
- Base surfaces: layered graphite / iron neutrals
|
||||
- Status colors: restrained semantic tones with low neon saturation
|
||||
- Display typography: `Alegreya Sans SC`
|
||||
- Body typography: `Noto Sans SC`
|
||||
|
||||
## Constraints
|
||||
|
||||
- Keep the interface suitable for long-duration dashboard use
|
||||
- Prioritize table readability, state clarity, and dense operations over decorative marketing patterns
|
||||
- Avoid emoji-based branding, purple SaaS gradients, and generic dark-devtool styling
|
||||
@@ -0,0 +1,237 @@
|
||||
# Plan: Browser Session Cookie Persistence
|
||||
|
||||
**Generated**: 2026-05-29
|
||||
**Estimated Complexity**: Medium
|
||||
|
||||
## Overview
|
||||
|
||||
Remote browser profiles already live under `browser_profiles_dir` (`/app/data/browser-profiles` by default), so Chromium profile files are expected to survive container restarts when `/app/data` is mounted. The remaining login loss case is mainly session-only cookies that Chromium removes when Playwright closes a persistent context normally.
|
||||
|
||||
Implement a backend-only cookie persistence layer in `BrowserSessionService`:
|
||||
|
||||
- Save all cookies from active persistent browser contexts into a JSON file under the profile directory.
|
||||
- Restore those cookies immediately after `launch_persistent_context(...)` and before `page.goto(...)`.
|
||||
- Keep session-only cookies as session cookies when restoring, instead of rewriting them as long-lived cookies by default.
|
||||
- Exclude ephemeral auth-capture sessions so temporary login extraction profiles keep their current lifecycle.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Current Playwright dependency: `backend/requirements.txt` pins `playwright==1.52.0`.
|
||||
- Playwright `BrowserContext.add_cookies()` accepts cookies with `name`, `value`, and either `url` or `domain` + `path`; it also supports `expires`, `httpOnly`, `secure`, `sameSite`, and `partitionKey`.
|
||||
- No frontend changes are required.
|
||||
|
||||
## Key Design Decisions
|
||||
|
||||
- **Storage location**: `browser-profiles/{profile_key}/session-cookies.json`.
|
||||
- **Scope**: persistent remote-browser sessions only. Do not persist cookies for `auth-capture-*` profiles.
|
||||
- **Save trigger**: after user interaction events, with debounce; before `context.close()` as a final save.
|
||||
- **Restore trigger**: after `launch_persistent_context(...)`, before the first navigation.
|
||||
- **Session cookie behavior**: if `expires` is missing, `None`, or negative, omit `expires` when restoring. This preserves session-cookie behavior inside the new context while still allowing our JSON backup to survive service restarts.
|
||||
- **Security**: treat the JSON file as sensitive. Keep it under the already-private profile directory and write it with owner-readable permissions where practical.
|
||||
|
||||
## Sprint 1: Cookie Persistence Helpers
|
||||
|
||||
**Goal**: Add isolated helper methods without changing session behavior yet.
|
||||
|
||||
**Demo/Validation**:
|
||||
- Unit tests can save, read, normalize, and restore cookie data using fake contexts and a temp profile directory.
|
||||
- Invalid or empty JSON files do not break browser startup.
|
||||
|
||||
### Task 1.1: Add cookie file path helper
|
||||
|
||||
- **Location**: `backend/app/services/browser_session_service.py`
|
||||
- **Description**: Add `_cookies_path(profile_key: str) -> Path`, returning `self._profile_dir(profile_key) / "session-cookies.json"`.
|
||||
- **Dependencies**: None
|
||||
- **Acceptance Criteria**:
|
||||
- Path is profile-local.
|
||||
- Existing `clear_profile()` deletes the cookie JSON automatically because it removes the full profile directory.
|
||||
- **Validation**:
|
||||
- Unit test with a temp `browser_profiles_dir` confirms path location.
|
||||
|
||||
### Task 1.2: Add cookie serialization helpers
|
||||
|
||||
- **Location**: `backend/app/services/browser_session_service.py`
|
||||
- **Description**: Add helpers:
|
||||
- `_normalize_cookie_for_save(cookie: dict[str, Any]) -> dict[str, Any] | None`
|
||||
- `_normalize_cookie_for_restore(cookie: dict[str, Any], now: float) -> dict[str, Any] | None`
|
||||
- **Dependencies**: Task 1.1
|
||||
- **Acceptance Criteria**:
|
||||
- Preserve supported Playwright fields: `name`, `value`, `domain`, `path`, `expires`, `httpOnly`, `secure`, `sameSite`, `partitionKey`.
|
||||
- Drop unsupported or unserializable fields.
|
||||
- Skip cookies missing `name` or `value`.
|
||||
- For restore, skip expired cookies when `expires > 0 and expires <= now`.
|
||||
- For restore, omit `expires` when it is missing, `None`, or negative.
|
||||
- Ensure every restored cookie has either `domain` + `path` or `url`.
|
||||
- **Validation**:
|
||||
- Unit tests for persistent cookies, session cookies, expired cookies, and partitioned cookies.
|
||||
|
||||
### Task 1.3: Add atomic JSON read/write helpers
|
||||
|
||||
- **Location**: `backend/app/services/browser_session_service.py`
|
||||
- **Description**: Add:
|
||||
- `_read_saved_cookies(profile_key: str) -> list[dict[str, Any]]`
|
||||
- `_write_saved_cookies(profile_key: str, cookies: list[dict[str, Any]]) -> None`
|
||||
- **Dependencies**: Tasks 1.1 and 1.2
|
||||
- **Acceptance Criteria**:
|
||||
- JSON schema includes `version`, `profile_key`, `saved_at`, and `cookies`.
|
||||
- Write is atomic via `session-cookies.json.tmp` then `replace(...)`.
|
||||
- Malformed JSON logs a warning and returns an empty cookie list.
|
||||
- Empty cookie list writes a valid file rather than failing.
|
||||
- **Validation**:
|
||||
- Unit tests for normal write/read, corrupted JSON, and atomic replacement.
|
||||
|
||||
## Sprint 2: Restore Cookies On Session Create
|
||||
|
||||
**Goal**: Restore saved cookies before the first page load.
|
||||
|
||||
**Demo/Validation**:
|
||||
- A fake Playwright context receives `add_cookies(...)` before `page.goto(...)`.
|
||||
- Startup continues even if restore fails.
|
||||
|
||||
### Task 2.1: Add restore method
|
||||
|
||||
- **Location**: `backend/app/services/browser_session_service.py`
|
||||
- **Description**: Add `_restore_cookies(session_or_context, profile_key: str) -> None`, using `context.add_cookies(cookies)` when saved cookies exist.
|
||||
- **Dependencies**: Sprint 1
|
||||
- **Acceptance Criteria**:
|
||||
- No-op for missing JSON or empty cookie list.
|
||||
- Logs count of restored cookies at `info` or `debug` level without logging cookie values.
|
||||
- Catches Playwright restore errors and logs them without blocking session creation.
|
||||
- **Validation**:
|
||||
- Unit test fake context records restored cookies.
|
||||
- Unit test invalid cookie list does not raise.
|
||||
|
||||
### Task 2.2: Wire restore into `create()`
|
||||
|
||||
- **Location**: `backend/app/services/browser_session_service.py`
|
||||
- **Description**: Call restore after `launch_persistent_context(...)` and before `page.goto(...)`.
|
||||
- **Dependencies**: Task 2.1
|
||||
- **Acceptance Criteria**:
|
||||
- Applies only to normal persistent sessions.
|
||||
- Existing health-check path for already-open sessions is unchanged.
|
||||
- `page.goto(...)` sees restored cookies on first request.
|
||||
- **Validation**:
|
||||
- Unit test call order with fakes.
|
||||
- Manual test: log into a remote page, restart backend, reopen page, verify logged-in state when server-side session is still valid.
|
||||
|
||||
### Task 2.3: Do not restore for `create_ephemeral()`
|
||||
|
||||
- **Location**: `backend/app/services/browser_session_service.py`
|
||||
- **Description**: Leave auth-capture sessions isolated.
|
||||
- **Dependencies**: Task 2.1
|
||||
- **Acceptance Criteria**:
|
||||
- No cookie JSON is read for `auth-capture-*`.
|
||||
- Existing auth-capture cleanup behavior remains unchanged.
|
||||
- **Validation**:
|
||||
- Unit test or code assertion via fake profile key.
|
||||
|
||||
## Sprint 3: Save Cookies During Activity And Close
|
||||
|
||||
**Goal**: Keep the JSON cache fresh while users interact and before Playwright closes the context.
|
||||
|
||||
**Demo/Validation**:
|
||||
- User interactions cause cookie JSON to appear/update.
|
||||
- Closing a session saves cookies before `context.close()`.
|
||||
|
||||
### Task 3.1: Add save method
|
||||
|
||||
- **Location**: `backend/app/services/browser_session_service.py`
|
||||
- **Description**: Add `_save_cookies(session: BrowserSession, *, force: bool = False) -> None`.
|
||||
- **Dependencies**: Sprint 1
|
||||
- **Acceptance Criteria**:
|
||||
- Calls `await session.context.cookies()`.
|
||||
- Normalizes cookies and writes JSON.
|
||||
- Skips `auth-capture-*` sessions.
|
||||
- Does not log cookie values.
|
||||
- Handles closed contexts or Playwright errors without raising during cleanup.
|
||||
- **Validation**:
|
||||
- Unit test fake context cookies are written.
|
||||
- Unit test auth-capture profile is skipped.
|
||||
|
||||
### Task 3.2: Debounce saves after browser events
|
||||
|
||||
- **Location**: `backend/app/services/browser_session_service.py`
|
||||
- **Description**: After supported `event()` actions complete, call `_save_cookies(session)` with a debounce interval, for example 5 seconds.
|
||||
- **Dependencies**: Task 3.1
|
||||
- **Acceptance Criteria**:
|
||||
- Reuses existing `BrowserSession.last_saved_state_at`.
|
||||
- Saves after meaningful actions including click, type, key, reload, back, forward, resize, and scroll.
|
||||
- Does not save on every screenshot request.
|
||||
- Does not block event responses for long; if cookie reads are fast, inline is acceptable. If they prove slow, use a background task guarded by session lock.
|
||||
- **Validation**:
|
||||
- Unit test repeated events inside debounce produce one write.
|
||||
- Unit test event after debounce writes again.
|
||||
|
||||
### Task 3.3: Force save before close and shutdown
|
||||
|
||||
- **Location**: `backend/app/services/browser_session_service.py`
|
||||
- **Description**: In `close()`, call `_save_cookies(session, force=True)` before `context.close()`.
|
||||
- **Dependencies**: Task 3.1
|
||||
- **Acceptance Criteria**:
|
||||
- Save happens before CDP detach/close when possible.
|
||||
- `shutdown()` benefits automatically because it calls `close()` for each session.
|
||||
- Close still proceeds even if cookie save fails.
|
||||
- **Validation**:
|
||||
- Unit test fake session records save before context close.
|
||||
|
||||
## Sprint 4: Tests And Operational Verification
|
||||
|
||||
**Goal**: Prove the feature works without depending entirely on live websites.
|
||||
|
||||
**Demo/Validation**:
|
||||
- Unit tests pass.
|
||||
- Manual Docker restart test demonstrates retained login for a site whose server-side session remains valid.
|
||||
|
||||
### Task 4.1: Add unit tests
|
||||
|
||||
- **Location**: `backend/test_browser_session_service.py`
|
||||
- **Description**: Extend current fake-based tests for cookie persistence helper behavior.
|
||||
- **Dependencies**: Sprints 1-3
|
||||
- **Acceptance Criteria**:
|
||||
- Tests cover save, restore, malformed JSON, expired cookie skip, session cookie restore, auth-capture skip, close-before-context-close order, and event debounce.
|
||||
- **Validation**:
|
||||
- Run `pytest backend/test_browser_session_service.py`.
|
||||
|
||||
### Task 4.2: Add manual verification checklist
|
||||
|
||||
- **Location**: `browser-session-cookie-persistence-plan.md` or a future PR description
|
||||
- **Description**: Document how to verify in Docker.
|
||||
- **Dependencies**: Implementation complete
|
||||
- **Acceptance Criteria**:
|
||||
- Start Docker deployment with `/app/data` mounted.
|
||||
- Open remote browser page and log in.
|
||||
- Perform an event after login to trigger cookie save.
|
||||
- Confirm `session-cookies.json` exists under the matching profile directory.
|
||||
- Restart backend/container.
|
||||
- Reopen remote browser page and verify login state.
|
||||
- Clear profile and confirm both Chromium profile and cookie JSON are removed.
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
- **Unit tests**: Primary coverage using fake context/page/session objects. This avoids requiring real browser binaries in normal test runs.
|
||||
- **Manual integration test**: Required for confidence because real sites differ in cookie, localStorage, and server-side session behavior.
|
||||
- **Regression checks**:
|
||||
- `pytest backend/test_browser_session_service.py`
|
||||
- Existing backend tests if time allows: `pytest backend`
|
||||
|
||||
## Potential Risks & Gotchas
|
||||
|
||||
- Some sites use server-side session expiry or revocation; restoring cookies cannot bypass that.
|
||||
- Some sites bind sessions to IP, user agent, device fingerprint, or TLS/browser state.
|
||||
- Some login state is stored in `localStorage`, `sessionStorage`, IndexedDB, or service-worker cache. Persistent context already helps with some of this, but this plan only adds explicit cookie backup.
|
||||
- `sessionStorage` is tab-lifetime state and is not covered here. If a target site depends on it heavily, a later phase can add origin-scoped storage backup.
|
||||
- Cookie JSON contains authentication secrets. It must not be committed, logged, or exposed via APIs.
|
||||
- For CHIPS/partitioned cookies, preserve `partitionKey` when Playwright returns it.
|
||||
- Do not rewrite all session cookies to long-lived cookies by default; that changes browser semantics and may create security surprises.
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
- Remove the new helper methods and calls from `BrowserSessionService.create()`, `event()`, and `close()`.
|
||||
- Delete `session-cookies.json` files from affected profile directories if needed.
|
||||
- Existing Chromium persistent profile behavior will continue to work as before.
|
||||
|
||||
## Open Decisions
|
||||
|
||||
- Whether to add a config flag such as `browser_cookie_persistence_enabled: bool = True`. Default can be enabled because this directly addresses the production issue.
|
||||
- Whether to also save `localStorage` through Playwright `storage_state()` in a later phase. Not required for the first implementation.
|
||||
- Whether cookie JSON should be encrypted at rest. For the current Docker single-host deployment, profile-directory isolation is probably sufficient; encryption can be added if this becomes multi-tenant or shared-host.
|
||||
@@ -14,6 +14,7 @@
|
||||
<el-option label="服务异常" value="upstream_unhealthy" />
|
||||
<el-option label="服务恢复" value="upstream_recovered" />
|
||||
<el-option label="余额不足" value="upstream_balance_low" />
|
||||
<el-option label="账号优先级变更" value="account_priority_changed" />
|
||||
<el-option label="测试" value="test" />
|
||||
</el-select>
|
||||
<el-button size="small" text @click="loadList" :loading="tableLoading">刷新</el-button>
|
||||
@@ -119,11 +120,12 @@ const EVENT_LABELS: Record<string, string> = {
|
||||
upstream_unhealthy: '服务异常',
|
||||
upstream_recovered: '服务恢复',
|
||||
upstream_balance_low: '余额不足',
|
||||
account_priority_changed: '账号优先级变更',
|
||||
test: '测试通知',
|
||||
}
|
||||
const eventLabel = (e: string) => EVENT_LABELS[e] || e
|
||||
const eventTagType = (e: string) =>
|
||||
({ upstream_rate_changed: 'primary', website_rate_changed: 'warning', upstream_unhealthy: 'danger', upstream_recovered: 'success', upstream_balance_low: 'warning', test: 'info' }[e] || '')
|
||||
({ upstream_rate_changed: 'primary', website_rate_changed: 'warning', upstream_unhealthy: 'danger', upstream_recovered: 'success', upstream_balance_low: 'warning', account_priority_changed: 'primary', test: 'info' }[e] || '')
|
||||
const toUTC = (t: string) => /[Z+\-]\d*$/.test(t.trim()) ? t : t + 'Z'
|
||||
const fmtTime = (t: string) => dayjs(toUTC(t)).format('MM-DD HH:mm:ss')
|
||||
|
||||
|
||||
@@ -73,6 +73,7 @@
|
||||
<el-checkbox label="upstream_unhealthy">服务异常</el-checkbox>
|
||||
<el-checkbox label="upstream_recovered">服务恢复</el-checkbox>
|
||||
<el-checkbox label="upstream_balance_low">余额不足</el-checkbox>
|
||||
<el-checkbox label="account_priority_changed">账号优先级变更</el-checkbox>
|
||||
</el-checkbox-group>
|
||||
</el-form-item>
|
||||
<el-form-item label="启用">
|
||||
@@ -121,10 +122,11 @@ const EVENT_LABELS: Record<string, string> = {
|
||||
upstream_unhealthy: '服务异常',
|
||||
upstream_recovered: '服务恢复',
|
||||
upstream_balance_low: '余额不足',
|
||||
account_priority_changed: '账号优先级变更',
|
||||
}
|
||||
const eventLabel = (e: string) => EVENT_LABELS[e] || e
|
||||
const eventTagType = (e: string) =>
|
||||
({ upstream_rate_changed: 'primary', website_rate_changed: 'warning', upstream_unhealthy: 'danger', upstream_recovered: 'success', upstream_balance_low: 'warning' }[e] || '')
|
||||
({ upstream_rate_changed: 'primary', website_rate_changed: 'warning', upstream_unhealthy: 'danger', upstream_recovered: 'success', upstream_balance_low: 'warning', account_priority_changed: 'primary' }[e] || '')
|
||||
|
||||
const toUTC = (t: string) => /[Z+\-]\d*$/.test(t.trim()) ? t : t + 'Z'
|
||||
const fmtTime = (t: string) => dayjs(toUTC(t)).format('MM-DD HH:mm')
|
||||
|
||||
@@ -0,0 +1,123 @@
|
||||
# Plan: Remote Browser Autofill Fix
|
||||
|
||||
**Generated**: 2026-05-15
|
||||
**Estimated Complexity**: Medium
|
||||
|
||||
## Overview
|
||||
Remote browser custom pages already store autofill credentials and pass them into `BrowserSessionService.create`, but the service only attempts autofill once immediately after `domcontentloaded`. Modern login pages often render inputs after SPA hydration or behind short async delays, so the first attempt can miss the fields and silently skip.
|
||||
|
||||
Fix the root cause by making remote-browser autofill wait/retry for the login fields for a bounded time, then fill the username/password once both locators are visible. Keep the implementation limited to remote browser mode and avoid exposing stored passwords in API responses or logs.
|
||||
|
||||
## Prerequisites
|
||||
- Existing backend virtualenv/dependencies available for tests.
|
||||
- Playwright APIs remain the same as currently used by `backend/app/services/browser_session_service.py`.
|
||||
- No changes to stored credential schema are required.
|
||||
|
||||
## Sprint 1: Backend Autofill Reliability
|
||||
**Goal**: Make autofill robust when login fields appear shortly after initial page load.
|
||||
**Demo/Validation**:
|
||||
- Unit tests demonstrate delayed locator availability is retried.
|
||||
- Existing autofill behavior remains unchanged when fields are immediately available.
|
||||
- No password value is logged or returned from APIs.
|
||||
|
||||
### Task 1.1: Add focused autofill behavior tests
|
||||
- **Location**: `backend/test_browser_session_service.py`
|
||||
- **Description**: Add async/unit-style tests around `BrowserSessionService._autofill_login` using fake page/locator objects. Cover delayed selector visibility and disabled/missing credential config.
|
||||
- **Dependencies**: None
|
||||
- **Acceptance Criteria**:
|
||||
- Delayed-field test fails against current one-shot implementation.
|
||||
- Test asserts both username and password locators are filled after retry.
|
||||
- Disabled/missing config returns without filling.
|
||||
- Tests do not require launching real Chromium.
|
||||
- **Validation**:
|
||||
- `cd backend && pytest test_browser_session_service.py`
|
||||
|
||||
### Task 1.2: Default-enable autofill when credentials are saved
|
||||
- **Location**: `backend/app/routers/custom_pages.py`
|
||||
- **Description**: When creating or updating a custom page with a non-empty login password and username, default `login_autofill_enabled` to true unless the request explicitly sets it to false. This prevents the common state where credentials are saved but the fill path is disabled.
|
||||
- **Dependencies**: Task 1.1
|
||||
- **Acceptance Criteria**:
|
||||
- New pages with username/password saved execute autofill by default.
|
||||
- Updating an existing page with a new password and username enables autofill unless explicitly disabled.
|
||||
- Existing pages without credentials are not enabled automatically.
|
||||
- Manual switch-off remains respected when explicitly sent.
|
||||
- **Validation**:
|
||||
- Add or extend API/router tests if practical, otherwise verify via focused unit test or direct route behavior.
|
||||
|
||||
### Task 1.3: Implement bounded retry/wait for autofill fields
|
||||
- **Location**: `backend/app/services/browser_session_service.py`
|
||||
- **Description**: Replace the single locator lookup in `_autofill_login` with a bounded retry helper that repeatedly checks the configured/default selectors until both username and password fields are visible or a deadline expires.
|
||||
- **Dependencies**: Task 1.1
|
||||
- **Acceptance Criteria**:
|
||||
- Autofill retries for a short bounded window, e.g. up to 8 seconds with small sleeps.
|
||||
- It fills only after both username and password locators are found.
|
||||
- Existing custom selectors still have highest priority.
|
||||
- Missing username/password config still returns immediately.
|
||||
- Failed/missing fields still log only generic skip information without secrets.
|
||||
- **Validation**:
|
||||
- `cd backend && pytest test_browser_session_service.py`
|
||||
|
||||
### Task 1.4: Preserve reload behavior scope
|
||||
- **Location**: `backend/app/services/browser_session_service.py`
|
||||
- **Description**: Keep reload/back/forward event behavior unchanged unless tests prove it is part of the root cause. The initial fix should target the known missed-on-load path only.
|
||||
- **Dependencies**: Task 1.3
|
||||
- **Acceptance Criteria**:
|
||||
- No unrelated navigation or UI-event behavior changes.
|
||||
- No frontend API contract changes.
|
||||
- **Validation**:
|
||||
- Code review and existing tests.
|
||||
|
||||
## Sprint 2: Verification and Manual Check
|
||||
**Goal**: Confirm the fix is safe and usable in the app.
|
||||
**Demo/Validation**:
|
||||
- Backend tests pass.
|
||||
- Frontend type/build check passes if dependencies are present.
|
||||
- Manual remote-browser check is attempted when the dev stack/browser runtime can be started.
|
||||
|
||||
### Task 2.1: Run backend regression tests
|
||||
- **Location**: `backend/`
|
||||
- **Description**: Run the new focused test and the existing backend tests relevant to service behavior.
|
||||
- **Dependencies**: Sprint 1 complete
|
||||
- **Acceptance Criteria**:
|
||||
- New autofill test passes.
|
||||
- Existing backend tests pass or any unrelated failures are documented.
|
||||
- **Validation**:
|
||||
- `cd backend && pytest test_browser_session_service.py test_upstream.py test_website_client.py test_group_binding_create_sync.py`
|
||||
|
||||
### Task 2.2: Run frontend validation if feasible
|
||||
- **Location**: `frontend/`
|
||||
- **Description**: Run available type/build checks to ensure no accidental frontend impact.
|
||||
- **Dependencies**: Sprint 1 complete
|
||||
- **Acceptance Criteria**:
|
||||
- Frontend build/typecheck passes, or dependency/environment blockers are documented.
|
||||
- **Validation**:
|
||||
- Inspect `frontend/package.json` scripts and run the appropriate check.
|
||||
|
||||
### Task 2.3: Manual remote browser verification
|
||||
- **Location**: App UI, custom page with `access_mode = remote_browser`
|
||||
- **Description**: Start the app stack if feasible, open a custom page with saved credentials, and verify username/password fields populate after the login form appears.
|
||||
- **Dependencies**: Sprint 1 complete
|
||||
- **Acceptance Criteria**:
|
||||
- A remote browser page with delayed login fields gets filled.
|
||||
- If a submit selector is configured, submit still works after fill.
|
||||
- If runtime dependencies are unavailable, document exactly what could not be verified.
|
||||
- **Validation**:
|
||||
- Manual browser check or documented blocker.
|
||||
|
||||
## Testing Strategy
|
||||
- Prefer a focused fake-object unit test for `_autofill_login` so the timing regression is deterministic and does not require Chromium.
|
||||
- Run relevant existing backend pytest files to catch regressions.
|
||||
- Attempt frontend build/typecheck because the user-facing flow crosses frontend/backend even though the code change is expected to be backend-only.
|
||||
- Attempt a manual UI check only after automated validation, and clearly report if Playwright/browser runtime or local services prevent it.
|
||||
|
||||
## Potential Risks & Gotchas
|
||||
- **Autofill switch disabled**: Current DB inspection showed page `id=1` has username/password configured but `login_autofill_enabled=0`; users must enable the switch for autofill to execute.
|
||||
- **Delayed fields inside iframes**: The retry only covers fields in the main page. If the target site renders login inside an iframe, a separate frame-aware enhancement would be needed.
|
||||
- **Login appears only after user action**: Retry cannot fill a modal that is opened only after clicking a login button unless selectors point to already visible inputs after navigation.
|
||||
- **Sites with anti-automation behavior**: Playwright fill may still be blocked by site-specific scripts; this fix handles timing, not hostile automation detection.
|
||||
- **Persistent profile reuse**: If a reused session is already on a non-login page, autofill should not force navigation or overwrite unrelated inputs.
|
||||
- **Selector ambiguity**: Default selectors may match the wrong visible text input on unusual pages. Custom selectors remain the mitigation.
|
||||
|
||||
## Rollback Plan
|
||||
- Revert changes to `backend/app/services/browser_session_service.py` and remove `backend/test_browser_session_service.py`.
|
||||
- Restart the backend to clear any in-memory browser sessions if needed.
|
||||
Reference in New Issue
Block a user