Commit Graph
69 Commits
Author SHA1 Message Date
SmartUp Developer eba209395d feat: persist finance daily summaries 2026-07-03 17:38:54 +08:00
SmartUp Developer 5e99abcbc5 feat: 财务对账页面 — GET /api/finance/daily-summary + Finance.vue 2026-07-03 15:28:06 +08:00
SmartUp Developer 9997b9d56a fix: restore api_key from local key_value when syncing upstream models for apikey accounts 2026-07-02 23:38:57 +08:00
SmartUp Developer a0c96ed5da fix: close platform-check silent-success loophole — raise WebsiteError on None list, missing account, or stale platform after update 2026-07-02 23:28:28 +08:00
SmartUp Developer 2c976f4d48 fix: add active check post account platform update to prevent silent ignore by remote backend 2026-07-02 23:06:38 +08:00
SmartUp Developer fd2755d539 feat: resolve account platforms prioritizing structural snapshot values and correction on existing accounts 2026-07-02 22:58:15 +08:00
SmartUp Developer 8746068cd4 feat: implement priority_weighted_plus_percent pricing algorithm 2026-07-02 20:23:20 +08:00
SmartUp Developer c20c9cfd26 fix: add stream close check, recover regression test assertions, and secure credentials type check 2026-07-02 19:18:10 +08:00
SmartUp Developer ece11d66e6 feat: sync upstream models with real-time streaming updates 2026-07-02 18:11:55 +08:00
liumangmang 852bf84246 fix: 优化同步上游模型,过滤克隆载荷中的敏感 credentials 字段,规避重写导致脱敏密钥丢失/清空的事故风险 2026-07-02 17:05:44 +08:00
liumangmang 316ef24188 fix: 优化同步上游模型的写回载荷,克隆并保留远端返回的所有非敏感 credentials,避免丢失 compact_model_mapping 等其它配置 2026-07-02 17:02:37 +08:00
liumangmang 4645460ff9 fix: 一键同步模型接口同步修复并携带上游 base_url,防止 sub2api 账号 base_url 被清空或置为默认值 2026-07-02 16:56:45 +08:00
liumangmang 3c1fff2491 fix: 优化上游模型同步的写回逻辑,采用增量 JSONB merge,避免脱敏的 credentials 覆盖或丢失原有敏感字段 2026-07-02 15:47:34 +08:00
liumangmang 63cbdf4dca feat: 实现一键同步网站账号上游模型功能 2026-07-02 15:43:00 +08:00
SmartUp Developer c70b68d814 refactor: clean up unused variables in external api logs router 2026-07-02 15:17:05 +08:00
SmartUp Developer ea3fd64686 fix: optimize summary count query and refine row keys for external api logs 2026-07-02 15:07:32 +08:00
SmartUp Developer fe3f9f4a4c feat: 一键整理强制对齐分组逻辑,移除旧托管分组并优化统计消息区分 2026-07-02 10:42:18 +08:00
liumangmang c797aff773 fix: 修正设置并发数无候选时的文案拼写错误 2026-07-02 10:08:30 +08:00
liumangmang 274413bca7 fix: 优化并发数批量设置接口,引入 c.close() 资源管理并增加对 list_accounts() 返回 None 的拦截保护 2026-07-02 10:02:07 +08:00
liumangmang 608ef00920 feat: 实现设置账号并发数功能(支持 bulk-update 与 fallback 逐个更新) 2026-07-02 09:48:59 +08:00
SmartUp DeveloperandClaude Sonnet 4.6 c96d55665d feat: 优化上游认证复用与 wangwang888 分组接口
## 上游认证优化

### 核心改进
- login_password 类型上游优先复用已保存 token,过期后自动 refresh,失败再重新登录
- 新增 ensure_authenticated() 方法替代直接调用 login(),减少不必要的登录请求
- 初始化时从 auth_config 自动加载已保存的 token 和 user_id
- 请求遇到 401 时自动尝试 refresh 或 login 并重试一次

### 实现细节
- UpstreamClient.__init__: 初始化时加载 auth_config 中的 token/new_api_user
- _is_login_password_with_refresh(): 判断是否支持 refresh
- _token_is_expired(): 检查 token 是否过期(提前 60 秒)
- _refresh_login_password_token(): 刷新 login_password 类型的 token
- ensure_authenticated(): 优先复用 token,过期后 refresh,失败再 login
- _send_request(): 401 时针对 login_password 类型先 refresh 再 login 并重试

### 调用点更新
- scheduler.py: _check_upstream, _sync_upstream_keys
- website_sync.py: reconcile_upstream_keys_full
- upstreams.py: list_generated_keys, generate_keys_by_groups, test_all, check_now

## wangwang888 分组接口优化

- 提供数据库更新脚本 update_wangwang888_groups_endpoint.py
- 将 wangwang888 的 groups_endpoint 从 /groups 改为 /groups/all
- 减少不必要的 /api/v1/admin/groups 统计开销
- website_client 已有 fallback 逻辑保证兼容性

## 测试
- 新增 test_upstream_login_password_refresh.py(7 个测试用例)
- 验证 token 复用、refresh、401 重试等逻辑
- 更新 test_upstream_key_sync.py 的 FakeClient 增加 ensure_authenticated
- 所有现有测试保持通过(87 passed)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-02 09:26:19 +08:00
liumangmang 3b23f21ef3 fix: 优化清理失效账号执行响应的 success 标志,对 test_account/delete_account 引入外部 API 日志审计,并对账号 ID 进行安全转义 2026-07-02 00:25:24 +08:00
liumangmang 536c0aa3d4 feat: 实现清理失效账号安全方案(包含预览、SSE测试、安全删除以及本地标记清空) 2026-07-02 00:19:26 +08:00
liumangmang 1ef6188c02 fix: 修正 websites router 中错误的 website.accounts_endpoint 属性调用为默认值,并增加对已存在账号名称回显与默认端点引用的单元测试 2026-07-01 21:15:55 +08:00
liumangmang 94d9550f77 feat: 调整新建/重建账号命名格式为 {上游网站名}-{上游分组名/ID}-{KeyID} 2026-07-01 21:11:37 +08:00
liumangmang 7ae9ad9292 fix: 删除 routers/websites 中残留的死代码函数 _build_rate_priority_map,并为 build_target_group_priority_map 引入严格的分组倍率解析,防止非法/缺失倍率的分组以 1.0 的降级值参与排序 2026-07-01 20:26:23 +08:00
liumangmang dd3649a6ad feat: 修正优先级自动计算算法为分目标分组独立排序,并优化一键分组整理完成后仅触发一次网站级优先级重排 2026-07-01 20:19:54 +08:00
liumangmang ef14382e5f fix: 后端路由 group_id 启用 path 路径匹配转换器,完美兼容带斜杠的远端分组 ID 2026-07-01 17:58:56 +08:00
liumangmang 3172ab18bf fix: 修改目标分组名称后自动同步更新本地绑定表中的名称缓存,并为前端更新分组 URL 中的 groupId 进行 Url 编码 2026-07-01 17:55:53 +08:00
liumangmang 8c2f1c13bb feat: 支持网站目标分组轻量管理(新增、编辑、描述展示),不修改和不提交倍率 2026-07-01 17:47:36 +08:00
liumangmang 8abfa4c5ef feat: 重构接口日志页面与接口监控看板,支持HTTP方法过滤和实时轮询 2026-07-01 15:04:32 +08:00
liumangmang 397a14c978 feat(website): 支持基于现有绑定一键整理分组,支持账号校验补齐、自愈重建与缺失 Key 提示 (含异常保守跳过及名称层级匹配) 2026-07-01 11:21:20 +08:00
liumangmang bf5470caf3 fix(sync): 优先级同步前置过滤远端失效账号防排序污染,移除 locals 隐患,并将自愈结果纳入本地日志与摘要统计 2026-07-01 10:20:13 +08:00
liumangmang efd0a88249 perf(import): 优化账号导入弹窗为秒开体验并支持批量回查校验 2026-07-01 10:00:40 +08:00
liumangmang d92537688a fix(priority): keep readable priority sync details 2026-07-01 09:12:39 +08:00
liumangmang 6065e824e3 fix(upstreams): add db.rollback() on exception to prevent dirty replaced status leak 2026-06-30 17:33:43 +08:00
liumangmang edc00444dc fix(nox-api): address duplicate creation risks, pending upgrading issues, cache warm error handling, test speedups, and frontend display support for pending keys 2026-06-30 17:25:18 +08:00
liumangmang 4fa6a79c5a feat(nox-api): implement batch creation rate limit resilience and pending backfilling 2026-06-30 17:00:26 +08:00
liumangmang 854a640d48 fix: shorten New-API token names 2026-06-30 16:12:46 +08:00
liumangmang c5bb63cf82 feat: support upstream token refresh 2026-06-30 14:34:40 +08:00
SmartUp Developer 2d1dcb8f9f fix: round website rates to two decimals 2026-06-30 11:26:48 +08:00
liumangmang a914d3d222 feat: add Nox-API upstream support 2026-06-30 11:22:38 +08:00
SmartUp Developer b866b387e0 feat: sync upstream keys and reorder priorities 2026-06-03 17:03:11 +08:00
liumangmang a42bcba483 Remove server remote browser support 2026-06-02 19:25:20 +08:00
liumangmang 84148f4a69 feat: support real browser auth import 2026-06-02 13:51:29 +08:00
liumangmang 4f9acdc99c feat(auth-capture): full cookie bundle extraction + richer refresh-auth
Problem: Meow upstream uses Cloudflare, which sets cf_clearance + session
cookies that must all be sent together. The old code only captured a single
session-named cookie via a whitelist, discarding cf_clearance entirely, and
wrote back only 'name=value' instead of the full cookie string.

Changes:

auth_capture_service.py:
  - Add _cookie_matches_hostname(): hostname suffix matching supporting
    dot-prefixed domains (.saki.lat matches api.saki.lat)
  - Add _build_cookie_bundle(): collects ALL cookies matching the current
    page's hostname, returns complete 'name1=v1; name2=v2' string
  - _curate_candidates(): new 'cookie_bundle' candidate type (type=0 in sort,
    highest priority), carries cookie_count + cookie_names in extra fields
  - extract_all(): obtain real-time page URL from session.page.url and pass
    to _curate_candidates so cookie domain filtering is accurate
  - Sort order: cookie_bundle > cookie > bearer_token/api_key > credential
  - Fix bug in original JWT dedup check (was assigning instead of checking)

custom_pages.py:
  - Add logging import + logger
  - _pick_best_candidate(): cookie preferred_auth_type now tries cookie_bundle
    first, then single cookie; bearer/api_key use existing type_map logic
  - RefreshAuthResponse: add optional 'warning' field
  - refresh_auth(): handle ctype='cookie_bundle' same as 'cookie'; always
    write full candidate.value as cookie_string (works for both types)
  - Post-write validation: attempt get_available_groups with new credentials;
    on failure, still commit (lenient mode) but set warning message explaining
    cf_clearance IP-binding as the likely cause; success logs at INFO level

Tests (test_auth_capture.py, 19 cases):
  - _cookie_matches_hostname: exact, dot-prefix subdomain, empty domain,
    different domain, evil-subdomain partial match rejection
  - _build_cookie_bundle: cf_clearance included, cross-domain excluded,
    single cookie, empty value excluded, no cookies
  - _curate_candidates: bundle ranks first, value is full string, bundle
    beats single session cookie, bearer wins when no cookies, empty case,
    cookie_count/cookie_names in extra, session fallback preserved,
    new_api_user propagation to bundle

All 46 tests pass.
2026-06-02 09:32:23 +08:00
liumangmang f17317b13c fix(priority-sync): handle missing rate data and backfill target group on re-import
P1 - Missing rate data now skips account instead of falling back to 1.0:
  In sync_account_priorities_for_upstream(), the rated list now filters
  out accounts whose upstream snapshot has no rate entry for their group_id.
  If after filtering a competitive bucket has fewer than 2 accounts with
  valid rate data, the entire bucket is silently skipped (no update_account
  call, no webhook) rather than treating missing rates as 1.0 and
  potentially triggering spurious notifications.

P2 - Re-importing an existing account now backfills imported_target_group_id:
  In the exists-is-True idempotency branch of import_upstream_keys_as_accounts(),
  if the current request supplies a target_group_id for the account's source group
  and it differs from what is stored, the field is written back and committed.
  This lets operators fix old data by simply re-running the import dialog.

Tests added:
  - test_missing_rate_skips_entire_competitive_group: all accounts in
    competitive group lack snapshot → bucket skipped, no update called
  - test_partial_missing_rate_sufficient_accounts_still_updates: 3 accounts
    in same bucket, 1 missing rate → the 2 with rates still compete normally

All 27 tests pass.
2026-06-01 19:27:35 +08:00
liumangmang e519d1804b fix(priority-sync): narrow account priority update to competitive groups only
Root cause: sync_account_priorities_for_upstream() was doing a global
priority re-rank across ALL imported accounts on a website whenever any
upstream rate changed, triggering spurious account_priority_changed
notifications for accounts in different target groups with no competition.

Fix:
- Add imported_target_group_id / imported_target_group_name to
  UpstreamGeneratedKey (nullable; old data falls back to group_id)
- Writ imported_target_group_id on account import in websites.py
- Rewrite sync_account_priorities_for_upstream():
  * bucket accounts by competition_group = imported_target_group_id or group_id
  * only process buckets with count > 1 (genuine competition)
  * each competitive bucket independently sorted by rate; priority starts at 1
  * single-account groups: completely skipped (no update_account, no notification)
  * no competitive groups at all: early return, no log, no notification
- Remove auto priority update in re-import idempotency path (was also
  incorrect; now fully delegated to sync_account_priorities_for_upstream)
- Fix Sub2ApiWebsiteClient local import in sync fn → use module-level name
  so monkeypatch works correctly in tests

Tests: rewrite test_priority_sync.py
- REMOVED: test_priority_sync_full_website_update (was asserting the buggy behavior)
- NEW: test_no_update_when_different_groups_single_account_each
- NEW: test_same_target_group_two_accounts_updated
- NEW: test_two_target_groups_independent_priority
- NEW: test_old_data_null_target_group_fallback
- NEW: test_single_account_in_mixed_website
- UPDATED: test_priority_sync_log_structure (now requires competitive group)
- KEPT: test_priority_sync_cross_upstream_group, test_import_auto_priority_by_rate

All 25 tests pass (8 priority_sync + 17 existing upstream tests).
2026-06-01 19:13:14 +08:00
liumangmang 871557e4ae feat(upstreams): add batch test-all / check-now-all endpoints
- POST /api/upstreams/test-all: batch connection test for all enabled
  upstreams (no snapshot, no webhook); updates last_status, balance
- POST /api/upstreams/check-now-all: full batch sync (snapshot, diff,
  webhook, key sync, priority sync); mirrors single check-now behavior
- Both routes are registered before /{uid} to avoid path capture
- Skips disabled upstreams (status=skipped); single failure does not
  abort subsequent upstreams (serial execution)
- Returns UpstreamBatchActionResponse with per-item detail and summary

Refactor: extract _test_upstream_core(db, u) and _check_now_core(db, u)
- All four routes (single + batch × 2) now share the same core helpers
- Eliminates duplicate logic and future divergence risk

Frontend:
- Add UpstreamBatchActionResponse / Item / Summary TS types
- Add upstreamsApi.testAll() and upstreamsApi.checkNowAll()
- Add '一键测试' and '一键同步' buttons in Upstreams.vue toolbar
  (order: 一键测试 → 一键同步 → 刷新 → 新增上游)
- Buttons disabled when list is empty or another batch op is running
- On completion: refresh list + ElMessageBox with per-item failure detail
2026-06-01 16:46:42 +08:00
liumangmang a949969c4d perf: remote browser CPU / zombie optimization
- Add tini (init: true in compose) to reap orphan Chromium processes
- Reduce screenshot push frequency (active 0.12→0.20s, idle 0.35→1.00s,
  deep idle 1.00→5.00s, backoff 0.60→2.00s)
- Add 5s timeout to screenshot in WebSocket push loop
- close() now wraps context.close() in asyncio.wait_for(10s)
  with browser.close() fallback on timeout
- Two-phase close logging (closing → closed / close_failed)
- Auth-capture sessions evicted after 10min TTL
- shutdown() with timeout protection and logging
- close_ok correctly tracks success through browser fallback path
2026-06-01 15:47:08 +08:00