Commit Graph
21 Commits
Author SHA1 Message Date
SmartUp Developer e2129d3796 fix: 防止分页漏读和上游异常快照误标 orphaned 2026-07-02 14:21:09 +08:00
SmartUp DeveloperandClaude Sonnet 4.6 c96d55665d feat: 优化上游认证复用与 wangwang888 分组接口
## 上游认证优化

### 核心改进
- login_password 类型上游优先复用已保存 token,过期后自动 refresh,失败再重新登录
- 新增 ensure_authenticated() 方法替代直接调用 login(),减少不必要的登录请求
- 初始化时从 auth_config 自动加载已保存的 token 和 user_id
- 请求遇到 401 时自动尝试 refresh 或 login 并重试一次

### 实现细节
- UpstreamClient.__init__: 初始化时加载 auth_config 中的 token/new_api_user
- _is_login_password_with_refresh(): 判断是否支持 refresh
- _token_is_expired(): 检查 token 是否过期(提前 60 秒)
- _refresh_login_password_token(): 刷新 login_password 类型的 token
- ensure_authenticated(): 优先复用 token,过期后 refresh,失败再 login
- _send_request(): 401 时针对 login_password 类型先 refresh 再 login 并重试

### 调用点更新
- scheduler.py: _check_upstream, _sync_upstream_keys
- website_sync.py: reconcile_upstream_keys_full
- upstreams.py: list_generated_keys, generate_keys_by_groups, test_all, check_now

## wangwang888 分组接口优化

- 提供数据库更新脚本 update_wangwang888_groups_endpoint.py
- 将 wangwang888 的 groups_endpoint 从 /groups 改为 /groups/all
- 减少不必要的 /api/v1/admin/groups 统计开销
- website_client 已有 fallback 逻辑保证兼容性

## 测试
- 新增 test_upstream_login_password_refresh.py(7 个测试用例)
- 验证 token 复用、refresh、401 重试等逻辑
- 更新 test_upstream_key_sync.py 的 FakeClient 增加 ensure_authenticated
- 所有现有测试保持通过(87 passed)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-02 09:26:19 +08:00
liumangmang 94d9550f77 feat: 调整新建/重建账号命名格式为 {上游网站名}-{上游分组名/ID}-{KeyID} 2026-07-01 21:11:37 +08:00
liumangmang 7ae9ad9292 fix: 删除 routers/websites 中残留的死代码函数 _build_rate_priority_map,并为 build_target_group_priority_map 引入严格的分组倍率解析,防止非法/缺失倍率的分组以 1.0 的降级值参与排序 2026-07-01 20:26:23 +08:00
liumangmang dd3649a6ad feat: 修正优先级自动计算算法为分目标分组独立排序,并优化一键分组整理完成后仅触发一次网站级优先级重排 2026-07-01 20:19:54 +08:00
liumangmang 8abfa4c5ef feat: 重构接口日志页面与接口监控看板,支持HTTP方法过滤和实时轮询 2026-07-01 15:04:32 +08:00
liumangmang bf5470caf3 fix(sync): 优先级同步前置过滤远端失效账号防排序污染,移除 locals 隐患,并将自愈结果纳入本地日志与摘要统计 2026-07-01 10:20:13 +08:00
liumangmang d92537688a fix(priority): keep readable priority sync details 2026-07-01 09:12:39 +08:00
liumangmang fe40fcd4c9 feat(priority): group priority changed notifications by target group and fetch remote account details 2026-07-01 09:00:29 +08:00
liumangmang c5bb63cf82 feat: support upstream token refresh 2026-06-30 14:34:40 +08:00
SmartUp Developer 2d1dcb8f9f fix: round website rates to two decimals 2026-06-30 11:26:48 +08:00
SmartUp Developer 9600e4ceba fix: backfill account groups before priority reorder 2026-06-03 18:39:21 +08:00
SmartUp Developer b866b387e0 feat: sync upstream keys and reorder priorities 2026-06-03 17:03:11 +08:00
liumangmang f17317b13c fix(priority-sync): handle missing rate data and backfill target group on re-import
P1 - Missing rate data now skips account instead of falling back to 1.0:
  In sync_account_priorities_for_upstream(), the rated list now filters
  out accounts whose upstream snapshot has no rate entry for their group_id.
  If after filtering a competitive bucket has fewer than 2 accounts with
  valid rate data, the entire bucket is silently skipped (no update_account
  call, no webhook) rather than treating missing rates as 1.0 and
  potentially triggering spurious notifications.

P2 - Re-importing an existing account now backfills imported_target_group_id:
  In the exists-is-True idempotency branch of import_upstream_keys_as_accounts(),
  if the current request supplies a target_group_id for the account's source group
  and it differs from what is stored, the field is written back and committed.
  This lets operators fix old data by simply re-running the import dialog.

Tests added:
  - test_missing_rate_skips_entire_competitive_group: all accounts in
    competitive group lack snapshot → bucket skipped, no update called
  - test_partial_missing_rate_sufficient_accounts_still_updates: 3 accounts
    in same bucket, 1 missing rate → the 2 with rates still compete normally

All 27 tests pass.
2026-06-01 19:27:35 +08:00
liumangmang e519d1804b fix(priority-sync): narrow account priority update to competitive groups only
Root cause: sync_account_priorities_for_upstream() was doing a global
priority re-rank across ALL imported accounts on a website whenever any
upstream rate changed, triggering spurious account_priority_changed
notifications for accounts in different target groups with no competition.

Fix:
- Add imported_target_group_id / imported_target_group_name to
  UpstreamGeneratedKey (nullable; old data falls back to group_id)
- Writ imported_target_group_id on account import in websites.py
- Rewrite sync_account_priorities_for_upstream():
  * bucket accounts by competition_group = imported_target_group_id or group_id
  * only process buckets with count > 1 (genuine competition)
  * each competitive bucket independently sorted by rate; priority starts at 1
  * single-account groups: completely skipped (no update_account, no notification)
  * no competitive groups at all: early return, no log, no notification
- Remove auto priority update in re-import idempotency path (was also
  incorrect; now fully delegated to sync_account_priorities_for_upstream)
- Fix Sub2ApiWebsiteClient local import in sync fn → use module-level name
  so monkeypatch works correctly in tests

Tests: rewrite test_priority_sync.py
- REMOVED: test_priority_sync_full_website_update (was asserting the buggy behavior)
- NEW: test_no_update_when_different_groups_single_account_each
- NEW: test_same_target_group_two_accounts_updated
- NEW: test_two_target_groups_independent_priority
- NEW: test_old_data_null_target_group_fallback
- NEW: test_single_account_in_mixed_website
- UPDATED: test_priority_sync_log_structure (now requires competitive group)
- KEPT: test_priority_sync_cross_upstream_group, test_import_auto_priority_by_rate

All 25 tests pass (8 priority_sync + 17 existing upstream tests).
2026-06-01 19:13:14 +08:00
liumangmang c8ba25f08e feat: live remote key list with auto-upsert and safe group name extraction
- list_generated_keys now fetches live keys from upstream API, merges with
  local DB: remote keys with plaintext values are auto-upserted (by
  group_id+managed_prefix), remote-only keys shown as unimportable
- Use _fetch_remote_managed_prefixes to support custom key prefixes
- Group remote keys by (group_id, prefix), pick latest by key_id
- Extract _remote_group_name helper for safe group name parsing
  (handles dict group field from Meow upstream)
- Frontend excludes orphaned keys from importable list
- Backend import endpoint reconciles upstream before importing
2026-06-01 14:53:40 +08:00
liumangmang bea4344bb3 fix: reconcile upstream keys on list/generate/import to prevent stale key imports
- Extract reconcile_upstream_keys() to website_sync.py (shared scheduler + on-demand)
- Add reconcile_upstream_keys_full() for on-demand reconciliation at three entry points:
  list_generated_keys, generate_keys_by_groups, import_upstream_keys_as_accounts
- Safe on failure: active_group_ids=None / remote_key_ids=None skip cleanup
- Support custom managed_prefix via _fetch_remote_managed_key_ids() helper
- Exclude orphaned keys from frontend importable list
- Remove hardcoded search='SmartUp' from scheduler path
2026-06-01 11:29:37 +08:00
liumangmang 5c20ddc8e6 feat: sync account priorities after rate changes 2026-05-29 17:51:12 +08:00
SmartUp Developer 42d8731ff7 perf: website_sync 批量查询 + custom_pages origin TTL 缓存 2026-05-25 00:52:38 +08:00
SmartUp Developer 2934473770 fix: remove stale _decimal_str ref, add context manager to HTTP clients
- UpstreamClient & Sub2ApiWebsiteClient: add __enter__/__exit__
- Convert all call sites to `with Client(...) as c:` pattern
- Remove unused `upstream_name`/`upstream_base_url` locals in scheduler
- Fix stale _decimal_str→decimal_string in _rate_from_group
2026-05-17 11:29:51 +08:00
liumangmangandClaude Opus 4.7 7adc7c00ab Add remote browser pages and website sync
Enable managed remote browser custom pages with login autofill and add website sync workflows so external admin surfaces can be handled inside SmartUp.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-15 15:43:58 +08:00