LiuMangMang/SmartUp
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
111 Commits 1 Branch 0 Tags
1cb13e878e71ca4e34eb0a846482ac9a3b1ae54f
Commit Graph
37 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
SmartUp Developer 1cb13e878e style: remove trailing whitespace in website_sync.py 2026-07-02 21:45:02 +08:00
SmartUp Developer fdbcca4d95 fix: delay binding.percent update until remote sync succeeds and update UI labels to 自动保底盈利 2026-07-02 21:27:52 +08:00
SmartUp Developer 0e77de31ae feat: implement automatic margin pricing and margin reverse engineering in sync 2026-07-02 21:22:37 +08:00
SmartUp Developer 9011c68b9d fix: use equal weight sharing for remaining 15% and quantize min_rate floor comparison to 2 decimal places 2026-07-02 20:45:40 +08:00
SmartUp Developer 8746068cd4 feat: implement priority_weighted_plus_percent pricing algorithm 2026-07-02 20:23:20 +08:00
SmartUp Developer d07ab62772 test: fix trailing whitespaces and add symmetric compact_model_mapping assertions 2026-07-02 19:48:47 +08:00
SmartUp Developer c20c9cfd26 fix: add stream close check, recover regression test assertions, and secure credentials type check 2026-07-02 19:18:10 +08:00
SmartUp Developer ece11d66e6 feat: sync upstream models with real-time streaming updates 2026-07-02 18:11:55 +08:00
SmartUp Developer 59be00bd11 chore: remove obsolete wangwang888 endpoint script 2026-07-02 16:09:35 +08:00
SmartUp Developer c70b68d814 refactor: clean up unused variables in external api logs router 2026-07-02 15:17:05 +08:00
SmartUp Developer ea3fd64686 fix: optimize summary count query and refine row keys for external api logs 2026-07-02 15:07:32 +08:00
SmartUp Developer e2129d3796 fix: 防止分页漏读和上游异常快照误标 orphaned 2026-07-02 14:21:09 +08:00
SmartUp Developer 00af34c487 fix: Sub2ApiWebsiteClient.list_accounts 支持分页拉取，避免只拿到默认第一页 20 条导致误建重复账号 2026-07-02 11:04:46 +08:00
SmartUp Developer fe3f9f4a4c feat: 一键整理强制对齐分组逻辑，移除旧托管分组并优化统计消息区分 2026-07-02 10:42:18 +08:00
SmartUp DeveloperandClaude Sonnet 4.6 a296525d4e fix: 将 refresh_path 加入 login_password auth_config 白名单 
自定义 refresh 路径会被 normalize_auth_config 丢弃，导致只有默认
/auth/refresh 生效。

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-07-02 09:34:53 +08:00
SmartUp DeveloperandClaude Sonnet 4.6 c96d55665d feat: 优化上游认证复用与 wangwang888 分组接口 
## 上游认证优化

### 核心改进
- login_password 类型上游优先复用已保存 token，过期后自动 refresh，失败再重新登录
- 新增 ensure_authenticated() 方法替代直接调用 login()，减少不必要的登录请求
- 初始化时从 auth_config 自动加载已保存的 token 和 user_id
- 请求遇到 401 时自动尝试 refresh 或 login 并重试一次

### 实现细节
- UpstreamClient.__init__: 初始化时加载 auth_config 中的 token/new_api_user
- _is_login_password_with_refresh(): 判断是否支持 refresh
- _token_is_expired(): 检查 token 是否过期（提前 60 秒）
- _refresh_login_password_token(): 刷新 login_password 类型的 token
- ensure_authenticated(): 优先复用 token，过期后 refresh，失败再 login
- _send_request(): 401 时针对 login_password 类型先 refresh 再 login 并重试

### 调用点更新
- scheduler.py: _check_upstream, _sync_upstream_keys
- website_sync.py: reconcile_upstream_keys_full
- upstreams.py: list_generated_keys, generate_keys_by_groups, test_all, check_now

## wangwang888 分组接口优化

- 提供数据库更新脚本 update_wangwang888_groups_endpoint.py
- 将 wangwang888 的 groups_endpoint 从 /groups 改为 /groups/all
- 减少不必要的 /api/v1/admin/groups 统计开销
- website_client 已有 fallback 逻辑保证兼容性

## 测试
- 新增 test_upstream_login_password_refresh.py（7 个测试用例）
- 验证 token 复用、refresh、401 重试等逻辑
- 更新 test_upstream_key_sync.py 的 FakeClient 增加 ensure_authenticated
- 所有现有测试保持通过（87 passed）

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-07-02 09:26:19 +08:00
SmartUp Developer 2d1dcb8f9f fix: round website rates to two decimals 2026-06-30 11:26:48 +08:00
SmartUp Developer 9600e4ceba fix: backfill account groups before priority reorder 2026-06-03 18:39:21 +08:00
SmartUp Developer b866b387e0 feat: sync upstream keys and reorder priorities 2026-06-03 17:03:11 +08:00
SmartUp Developer e3151a7ea6 chore: untrack runtime data/ directory 
data/ 包含浏览器 profile、SQLite DB、cache 等运行时生成内容，
.gitignore 已显式排除但历史上被误 track。本次 git rm --cached
解除跟踪，工作树文件保留不动。
 2026-05-25 13:47:21 +08:00
SmartUp Developer 42d8731ff7 perf: website_sync 批量查询 + custom_pages origin TTL 缓存 2026-05-25 00:52:38 +08:00
SmartUp Developer 4971263a3a perf: 前端按需引入 Element Plus + Vite 拆包 + template 优化 2026-05-25 00:18:08 +08:00
SmartUp Developer 41a439d830 perf: SQLite WAL + 复合索引 + GZip + scheduler jitter + 构建缓存 2026-05-25 00:08:10 +08:00
SmartUp Developer 3a31d185a4 fix: reuse upstream keys for account import 2026-05-24 23:18:40 +08:00
SmartUp DeveloperandClaude Opus 4.7 4c71148ff9 feat: one-click upstream auth refresh from custom page viewer 
- Add linked_upstream_id to CustomPage model with DB migration
- New POST /api/custom-pages/{pid}/refresh-auth endpoint extracts
  credentials from active remote browser and updates linked upstream
- PageViewer toolbar shows key icon button when page has linked upstream
- CustomPages form adds upstream dropdown for remote_browser pages
- Auth capture extracts New-Api-User from localStorage uid/user/self API
- Upstream client sends New-Api-User header in cookie auth mode
- Fix auth capture dialog: transparent background, field persistence,
  login URL defaults to base_url/login, focus on click for keyboard input
- Fix upstream test ASCII encoding with non-header characters validation

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-19 09:27:14 +08:00
SmartUp Developer 7cb0ff1608 fix: object URL leak, CDP before goto, limit raw secrets in extract 
- AuthCaptureDialog: revokeObjectURL on each frame to prevent memory leak
- CDP Network capture starts before initial page.goto, not after
- /extract defaults to candidates-only; pass ?include_raw=true for full data
 2026-05-18 14:25:56 +08:00
SmartUp Developer c7b33983d6 feat: auth-capture — WS frame stream, drag events, continuous CDP, profile cleanup 
- AuthCaptureDialog: real WebSocket for binary JPEG frame stream (no polling)
- Pointer drag: mousedown/mousemove/mouseup events for slider-captcha
- CDP capture starts at session creation, caches headers in session.captured_headers
- Ephemeral profile dir deleted on session close (shutil.rmtree)
- Candidate types unified: bearer_token / cookie / api_key / credential
- Frontend handleAuthCaptureSelect maps all 4 types to correct form fields
 2026-05-18 14:14:33 +08:00
SmartUp Developer 08c855677a feat: auth capture — interactive browser, CDP header capture, cookie auth 
- AuthCaptureDialog: full WS screenshot stream + mouse/keyboard/scroll events
- Backend auth_capture: CDP Network.requestWillBeSent for Authorization headers
- Candidate scoring: confidence 0-95%, preview (masked), auth_headers section
- Upstream form: add 'Cookie' auth type, handle cookie selection
- UpstreamClient: support auth_type=cookie with Cookie header
- No secrets logged at DEBUG or higher
 2026-05-18 11:44:10 +08:00
SmartUp Developer 4d1237c58f feat: auth capture — remote browser credential extraction 
- BrowserSessionService: add create_ephemeral() for temp sessions
- New auth_capture_service.py: extract cookies, localStorage, sessionStorage from page
- New auth_capture router: POST /sessions, GET /sessions/{id}/extract, DELETE /sessions/{id}
- Frontend AuthCaptureDialog: URL input → browser view → extract → pick candidate
- Upstreams.vue: '提取' button next to Bearer Token field
- No sensitive values logged
 2026-05-17 21:04:36 +08:00
SmartUp Developer c809139470 fix: authStorage key depends on pathname, F5 refresh logs out 
location.pathname changes between /login and /upstreams, causing
different localStorage keys. Use static 'smartup' instead.
 2026-05-17 13:01:53 +08:00
SmartUp Developer 84b714ced3 fix: sync admin password from .env on startup, add host.docker.internal for local sub2api 
- _init_admin now re-hashes password if .env password differs from DB
- docker-compose: add extra_hosts host.docker.internal:host-gateway
  so containers can reach host services via host.docker.internal
 2026-05-17 12:52:09 +08:00
SmartUp Developer 0ad8796285 fix: restore docker-compose env-var pattern, fix .env conflict from shell env 
docker-compose.yml now reads ADMIN_PASSWORD/JWT_SECRET from env or
.env file; the shell had stale ADMIN_PASSWORD=changeme123 which
overrode .env — document this for future runs.
 2026-05-17 12:40:26 +08:00
SmartUp Developer 5c60627fb6 fix: Vite type declaration, non-idempotent retry, hardcoded test IP 
- Add frontend/src/vite-env.d.ts (reference vite/client) to fix vue-tsc build
- Restrict axios-retry to GET/HEAD/OPTIONS only (avoid replaying mutations)
- Convert test_upstream.py to accept URL via CLI args instead of hardcoded IP
 2026-05-17 11:56:49 +08:00
SmartUp Developer 2934473770 fix: remove stale _decimal_str ref, add context manager to HTTP clients 
- UpstreamClient & Sub2ApiWebsiteClient: add __enter__/__exit__
- Convert all call sites to `with Client(...) as c:` pattern
- Remove unused `upstream_name`/`upstream_base_url` locals in scheduler
- Fix stale _decimal_str→decimal_string in _rate_from_group
 2026-05-17 11:29:51 +08:00
SmartUp Developer 8a6ed249be fix: complete remaining 8 optimization items 
- HTTP connection pooling: UpstreamClient & WebsiteClient reuse httpx.Client
- Deduplicate decimal_string into shared app/utils/number.py
- Split scheduler transaction: snapshot write → webhook/website sync in separate sessions
- Remove hardcoded 170.106.100.210 migration from database.py
- Reset consecutive_failures on upstream update
- Healthcheck: install curl, replace python -c with curl -f
- Add .dockerignore to reduce build context
- Frontend: add axios-retry with exponential backoff (5xx/network errors only)
 2026-05-17 11:09:35 +08:00
SmartUp Developer ad16618406 fix: address multiple code audit findings 
- CORS: replace wildcard with explicit origin list from CORS_ORIGINS env
- Auth: enforce strong defaults, JWT blacklist (RevokedToken model), login rate limiting
- Auth: validate password length before bcrypt (72-byte limit)
- Scheduler: single-threaded worker to mitigate SQLite write contention
- Scheduler: graceful shutdown (wait=True)
- Snapshots: add prune_snapshots() with configurable retention count
- Storage: isolate localStorage keys via VITE_APP_KEY prefix
- Config: add cors_origins, login_rate_limit, snapshot_retention_count settings
 2026-05-17 10:52:18 +08:00
SmartUp Developer a42ecf7bcc docs: add optimization-review.md — code audit findings 2026-05-17 04:12:00 +08:00