比较提交
| 作者 | SHA1 | 提交日期 | |
|---|---|---|---|
|
|
df7db3582e |
@@ -18,6 +18,14 @@ WB_ADMIN_PASSWORD=
|
|||||||
# 一个容器一份调度。只有跑多副本时才把除第一份之外的都设成 1。
|
# 一个容器一份调度。只有跑多副本时才把除第一份之外的都设成 1。
|
||||||
WB_DISABLE_SCHEDULER=0
|
WB_DISABLE_SCHEDULER=0
|
||||||
|
|
||||||
|
# ---------- 会话安全 ----------
|
||||||
|
# 会话 Cookie 是否只允许走 HTTPS。
|
||||||
|
# 0 = 关闭(默认,纯 HTTP / 局域网部署的正确值)
|
||||||
|
# 1 = 只在 HTTPS 下发送。**如果你用 http:// 访问却设成 1,会出现
|
||||||
|
# 「登录成功又立刻跳回登录页」**,因为浏览器根本不会回传会话 Cookie。
|
||||||
|
# 只有在前面挂了 HTTPS 反向代理、并且用域名访问时才设为 1。
|
||||||
|
WB_COOKIE_SECURE=0
|
||||||
|
|
||||||
# ---------- 可选:启动时自动导入 ----------
|
# ---------- 可选:启动时自动导入 ----------
|
||||||
# 1 = 尝试从挂载进来的编辑器 settings.json 读取 codebuddyUsage.* 写入数据库
|
# 1 = 尝试从挂载进来的编辑器 settings.json 读取 codebuddyUsage.* 写入数据库
|
||||||
WB_IMPORT_CREDS=0
|
WB_IMPORT_CREDS=0
|
||||||
@@ -26,7 +34,7 @@ WB_IMPORT_XLSX=
|
|||||||
|
|
||||||
# ---------- 镜像名(推送 Gitea 注册表时用)----------
|
# ---------- 镜像名(推送 Gitea 注册表时用)----------
|
||||||
# WB_IMAGE=git.iwali.top/wangchuanli/workbuddy-portal:latest
|
# WB_IMAGE=git.iwali.top/wangchuanli/workbuddy-portal:latest
|
||||||
# WB_IMAGE=git.iwali.top/wangchuanli/workbuddy-portal:1.1.0
|
# WB_IMAGE=git.iwali.top/wangchuanli/workbuddy-portal:1.2.0
|
||||||
|
|
||||||
# ---------- 仅叠加 docker-compose.hostdir.yml 时有效 ----------
|
# ---------- 仅叠加 docker-compose.hostdir.yml 时有效 ----------
|
||||||
# 把数据/日志放到宿主机目录而不是命名卷。**只建议 Linux 宿主机使用**:
|
# 把数据/日志放到宿主机目录而不是命名卷。**只建议 Linux 宿主机使用**:
|
||||||
|
|||||||
@@ -28,14 +28,18 @@ python manage.py serve --port 8848 # 或 docker compose up -d --
|
|||||||
|
|
||||||
## 二、用示例数据开发,别用真实数据
|
## 二、用示例数据开发,别用真实数据
|
||||||
|
|
||||||
`tools/demo_data.py` 会生成一份**完全合成**的数据集(假模型名、假 Prompt、偏斜的积分分布),
|
`tools/demo_data.py` 会生成一份**完全合成**的数据集(假模型名、假 Prompt、假 Cookie 串、
|
||||||
放在 `data/demo/` 下(该目录已在 `.gitignore` 内):
|
偏斜的积分分布),放在 `data/demo/` 下(该目录已在 `.gitignore` 内)。
|
||||||
|
它会造**两个账号**(`admin` 管理员 + `demo` 普通用户),好让你顺手验证多用户隔离:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
python tools/demo_data.py # 默认 data/demo,管理员 admin/admin123
|
python tools/demo_data.py # 默认 data/demo;两个账号口令都是 admin123
|
||||||
WB_DATA_DIR=$PWD/data/demo python manage.py serve --port 8849 --no-scheduler
|
WB_DATA_DIR=$PWD/data/demo python manage.py serve --port 8849 --no-scheduler
|
||||||
```
|
```
|
||||||
|
|
||||||
|
> 目录里的 `usage.sqlite` **在容器里生成**再截图才是对的:宿主机跑会让启动日志印出
|
||||||
|
> `C:\Users\<用户名>\…`,那一行正好会出现在「日志管理」页的截图上。
|
||||||
|
|
||||||
这样你既能有一个「看起来像真的」的界面来调试,也不会把任何真实用量带进仓库或截图。
|
这样你既能有一个「看起来像真的」的界面来调试,也不会把任何真实用量带进仓库或截图。
|
||||||
|
|
||||||
## 三、改动前请先跑一遍验证
|
## 三、改动前请先跑一遍验证
|
||||||
@@ -45,14 +49,17 @@ WB_DATA_DIR=$PWD/data/demo python manage.py serve --port 8849 --no-scheduler
|
|||||||
| # | 命令 | 覆盖什么 | 需要什么 |
|
| # | 命令 | 覆盖什么 | 需要什么 |
|
||||||
|---|---|---|---|
|
|---|---|---|---|
|
||||||
| 1 | `python -m compileall -q workbuddy_portal manage.py tools` | 语法 | — |
|
| 1 | `python -m compileall -q workbuddy_portal manage.py tools` | 语法 | — |
|
||||||
| 2 | `python tools/smoke.py` | **99 项**离线断言:全页面只读渲染、模板残留检测、历史缺陷防回归、静态资源、CSS 类名对账 | 无(用 Flask test_client,不启服务) |
|
| 2 | `python tools/smoke.py` | **165 项**离线断言:全页面只读渲染、模板残留检测、多用户隔离与凭证保密、注册与验证码、历史缺陷防回归、静态资源、CSS 类名对账 | 无(用 Flask test_client,不启服务) |
|
||||||
| 3 | `python tools/check_live.py --base http://127.0.0.1:8848` | **56 项**真实 HTTP 断言,含登录/CSRF/开放重定向 | 一个运行中的服务 |
|
| 3 | `python tools/check_live.py --base http://127.0.0.1:8848` | **83 项**真实 HTTP 断言,含登录/CSRF/开放重定向/验证码/安全响应头 | 一个运行中的服务 |
|
||||||
| 4 | `python tools/shots.py --base http://127.0.0.1:8849` | 登录后逐页截图并收集 `console`/`pageerror` | Playwright + Chromium |
|
| 4 | `python tools/shots.py --base http://127.0.0.1:8849` | 登录后逐页截图并收集 `console`/`pageerror` | Playwright + Chromium |
|
||||||
| 5 | `docker compose up -d --build && docker compose ps` | 容器化路径 | Docker |
|
| 5 | `docker compose up -d --build && docker compose ps` | 容器化路径 | Docker |
|
||||||
|
|
||||||
> `tools/shots.py` 是**最有价值的一层**:项目曾经出过「大屏整页全白」的 bug,
|
> `tools/shots.py` 是**最有价值的一层**:项目曾经出过「大屏整页全白」的 bug,
|
||||||
> 只有它抓到了(`smoke` 与 `check_live` 都放过了)。改前端务必跑。
|
> 只有它抓到了(`smoke` 与 `check_live` 都放过了)。改前端务必跑。
|
||||||
|
|
||||||
|
`check_live.py` 与 `shots.py` 都接受 `--db <路径>`:给了之后它们会直接从库里读验证码答案,
|
||||||
|
从而**自动过掉登录页与注册页的验证码**——不然脚本会被验证码挡在门外。
|
||||||
|
|
||||||
这三支脚本都会打印 `RESULT: ok=N fail=0`,`fail` 不为 0 时退出码是 1,可直接接进 CI。
|
这三支脚本都会打印 `RESULT: ok=N fail=0`,`fail` 不为 0 时退出码是 1,可直接接进 CI。
|
||||||
|
|
||||||
## 四、必须遵守的几条不变量
|
## 四、必须遵守的几条不变量
|
||||||
@@ -72,6 +79,30 @@ WB_DATA_DIR=$PWD/data/demo python manage.py serve --port 8849 --no-scheduler
|
|||||||
6. **`.gitignore` 不支持行尾注释**——规则后跟 `# 注释` 会让整行失效。注释必须单独占一行,
|
6. **`.gitignore` 不支持行尾注释**——规则后跟 `# 注释` 会让整行失效。注释必须单独占一行,
|
||||||
改完用 `git check-ignore -v <file>` 逐条确认命中。
|
改完用 `git check-ignore -v <file>` 逐条确认命中。
|
||||||
|
|
||||||
|
### 多用户相关的四条(v1.2.0 起)
|
||||||
|
|
||||||
|
7. **`uid` 必须是 `conn` 之后的第一个位置参数,且不给默认值。**
|
||||||
|
这是防越权的核心机制:漏传就直接 `TypeError`,而不是静默返回所有人的数据。
|
||||||
|
`query.*` / `collect.*` 全链路都遵循它。新写一个查询函数时请照做,**不要**加 `uid=0` 这种默认值。
|
||||||
|
8. **凭证不参与回落。** `db.NO_FALLBACK_KEYS = {"cookie", "user_agent"}`:
|
||||||
|
个人级没有值时**不许**落回实例级,否则等于拿别人的 Cookie 去采集(串号)。
|
||||||
|
新增任何「账号身份相关」的配置键,都要考虑是否该进这个集合。
|
||||||
|
9. **验证码答案只能放服务端。** 不要图省事塞进 `session`——Flask 的 session 是
|
||||||
|
「签名 + base64」而非加密,客户端能直接解开读到答案。下发给浏览器的只有随机 `captcha_id`;
|
||||||
|
且校验时**先删后判**(一次性)。同理,验证码图不要用 SVG 渲染,那玩意是文本。
|
||||||
|
10. **改主键的迁移必须「删索引 → 改名 → 建新表 → 回填 → 删旧表」。**
|
||||||
|
`ALTER TABLE … RENAME TO` 会**把索引一起带走**,后续 `CREATE INDEX IF NOT EXISTS`
|
||||||
|
就变成空操作,新表会零索引。本项目在迁移前先调 `_drop_all_user_indexes()`,
|
||||||
|
并把 `ALTER TABLE … ADD COLUMN` 放在 `executescript` 之前。
|
||||||
|
|
||||||
|
### 加密与验证码这两块(零依赖约束)
|
||||||
|
|
||||||
|
11. **`crypto.py` 与 `captcha.py` 只能用标准库。** 项目的硬约束是「只要 Flask / waitress / openpyxl」
|
||||||
|
—— 所以 ChaCha20、HMAC、PNG 编码、点阵字模都是手写的。想引 `cryptography` 或 `Pillow`
|
||||||
|
之前先想清楚:这会让「下载即跑」的卖点消失。
|
||||||
|
12. **解密失败必须显式报错,不能「失败就返回原值」。** `crypto.decrypt()` 对非 `v1.` 前缀
|
||||||
|
原样返回(兼容历史明文),但**校验不过就抛 `DecryptError`**。静默降级会让加密形同虚设。
|
||||||
|
|
||||||
## 五、代码风格
|
## 五、代码风格
|
||||||
|
|
||||||
- 遵循 PEP 8;行宽 100。
|
- 遵循 PEP 8;行宽 100。
|
||||||
|
|||||||
@@ -12,8 +12,10 @@
|
|||||||
| 存储 | SQLite(WAL),单文件正本 `data/usage.sqlite` |
|
| 存储 | SQLite(WAL),单文件正本 `data/usage.sqlite` |
|
||||||
| 前端 | 服务端渲染后台 + 独立 ECharts 大屏(离线自带的 `echarts.min.js`) |
|
| 前端 | 服务端渲染后台 + 独立 ECharts 大屏(离线自带的 `echarts.min.js`) |
|
||||||
| 部署 | Docker Compose / 裸机 `waitress`;镜像可推 Gitea 容器注册表 |
|
| 部署 | Docker Compose / 裸机 `waitress`;镜像可推 Gitea 容器注册表 |
|
||||||
| 鉴权 | 全站登录 + CSRF + 角色(管理员 / 普通用户),凭证存库、页面只回掩码 |
|
| 鉴权 | **多用户**(各自的数据与凭证严格隔离)+ 全站登录 + CSRF + 角色(管理员 / 普通) |
|
||||||
| 版本 | v1.1.0 |
|
| 凭证 | Cookie **ChaCha20 + HMAC 静态加密**入库,页面与接口只回掩码 |
|
||||||
|
| 防攻击 | 登录 / 注册**图形验证码**(服务端出题 + 一次性)、失败限速、注册限额 |
|
||||||
|
| 版本 | v1.2.0 |
|
||||||
| **许可证** | **MIT**(第三方组件与再分发资源见 [THIRD-PARTY-NOTICES.md](THIRD-PARTY-NOTICES.md)) |
|
| **许可证** | **MIT**(第三方组件与再分发资源见 [THIRD-PARTY-NOTICES.md](THIRD-PARTY-NOTICES.md)) |
|
||||||
|
|
||||||
**目录**:[核心特性](#核心特性) · [架构](#架构一图) · [快速开始](#快速开始) · [命令一览](#命令一览) ·
|
**目录**:[核心特性](#核心特性) · [架构](#架构一图) · [快速开始](#快速开始) · [命令一览](#命令一览) ·
|
||||||
@@ -25,12 +27,15 @@
|
|||||||
|
|
||||||
| 能力 | 说明 |
|
| 能力 | 说明 |
|
||||||
|---|---|
|
|---|---|
|
||||||
|
| **多用户隔离** | 每个账号只填**自己的** Cookie、收**自己的**数据、看**自己的**日志。`user_id` 是所有查询的第一个条件,且是**必填位置参数**(漏传直接 `TypeError`,不会静默返回全量) |
|
||||||
|
| **凭证加密** | Cookie 以 ChaCha20(RFC 8439)+ HMAC-SHA256 encrypt-then-MAC 密文入库;主密钥单独放在 `data/instance.json`,与 `SECRET_KEY` 分开。升级时会把历史明文自动加密 |
|
||||||
|
| **自助注册 + 验证码** | 开放注册(可关),登录/注册均带**图形验证码**。答案是服务端本地点阵渲染的 PNG,只存库、一次性、5 分钟过期——**不进会话**(Flask 会话是签名不加密的,放进去等于送答案) |
|
||||||
| **增量采集** | 按 `MAX(ts)` 断点续采 + 回退窗口;主键 `ON CONFLICT` 去重,冲突时以「更早的本地时间」为准 |
|
| **增量采集** | 按 `MAX(ts)` 断点续采 + 回退窗口;主键 `ON CONFLICT` 去重,冲突时以「更早的本地时间」为准 |
|
||||||
| **进程内调度** | 每天固定时刻(默认 `09:00,17:00`)由内置线程触发;支持**启动补跑**(程序没开时错过的时刻,开机后在宽限期内补上) |
|
| **进程内调度** | 每天固定时刻(默认 `09:00,17:00`)由内置线程**按账号逐个**触发;支持**启动补跑**(程序没开时错过的时刻,开机后在宽限期内补上) |
|
||||||
| **单写者保证** | 文件锁 `data/collect.lock` 让「调度 / 页面手动触发 / CLI」三处不并发写 SQLite;僵尸锁 30 分钟可抢占 |
|
| **单写者保证** | 文件锁 `data/collect.lock` 让「调度 / 页面手动触发 / CLI」三处不并发写 SQLite;僵尸锁 30 分钟可抢占 |
|
||||||
| **全量存档** | 不随官网导出窗口过期而丢数据;官网 xlsx 丢失约 22% 的 `Prompt`,可用 `fill-prompt` 回补 |
|
| **全量存档** | 不随官网导出窗口过期而丢数据;官网 xlsx 丢失约 22% 的 `Prompt`,可用 `fill-prompt` 回补 |
|
||||||
| **大屏去中间层** | 大屏直接走 `/api`,按当前筛选窗口实时聚合;左侧多取等长一段用于算环比,窗口不变不重复请求 |
|
| **大屏去中间层** | 大屏直接走 `/api`,按当前筛选窗口实时聚合;左侧多取等长一段用于算环比,窗口不变不重复请求 |
|
||||||
| **可观测** | 每次采集落一条 `collect_runs`(含 `[warn]`/`[error]` 逐行原文);另有操作审计与登录审计 |
|
| **可观测** | 每次采集落一条 `collect_runs`(含 `[warn]`/`[error]` 逐行原文);另有操作审计与登录审计,均带账号归属 |
|
||||||
| **一键备份** | 正本就是宿主机上的一个 `.sqlite` 文件,拷走即可;`manage.py vacuum` 回收空闲页 |
|
| **一键备份** | 正本就是宿主机上的一个 `.sqlite` 文件,拷走即可;`manage.py vacuum` 回收空闲页 |
|
||||||
|
|
||||||
---
|
---
|
||||||
@@ -40,32 +45,38 @@
|
|||||||
```
|
```
|
||||||
┌──────────────── workbuddy-portal(单进程)────────────────┐
|
┌──────────────── workbuddy-portal(单进程)────────────────┐
|
||||||
云端用量接口 │ │
|
云端用量接口 │ │
|
||||||
/billing/meter/ │ scheduler.py ──┐ │
|
/billing/meter/ │ scheduler.py ──┐ 按账号逐个判断槽位 │
|
||||||
get-user-request- │ (20s 轮询槽位) │ │
|
get-user-request- │ (20s 轮询槽位) │ │
|
||||||
usage │ ▼ │
|
usage │ ▼ │
|
||||||
▲ │ collect.py ─ 文件锁 collect.lock ─ 去重 upsert ─▶ SQLite │
|
▲ │ collect.py ─ 文件锁 collect.lock ─ 去重 upsert ─▶ SQLite │
|
||||||
│ │ ▲ data/usage.sqlite(WAL) │
|
│ │ ▲ (全部带 user_id) data/usage.sqlite(WAL)│
|
||||||
└───────────┼──────┘ ▲ │
|
└───────────┼──────┘ ▲ │
|
||||||
client.py(urllib)│ │ │
|
client.py(urllib)│ ▲ crypto.py 解密本账号 Cookie │ │
|
||||||
│ query.py(聚合全部下推 SQL) │
|
│ └ captcha.py 出验证码图 │ │
|
||||||
|
│ query.py(uid 为第一个查询条件) │
|
||||||
│ ▲ ▲ │
|
│ ▲ ▲ │
|
||||||
│ web/views.py ──────┘ └──── web/api.py│
|
│ web/views.py ──────┘ └──── web/api.py│
|
||||||
│ (Jinja 后台) (JSON) │
|
│ (Jinja 后台) (JSON) │
|
||||||
└───────────────┬───────────────────────────┬──────────────┘
|
└───────────────┬───────────────────────────┬──────────────┘
|
||||||
▼ ▼
|
▼ ▼
|
||||||
/ /records /tasks /dashboard(ECharts 大屏)
|
/ /records /tasks /config /logs /dashboard(ECharts 大屏)
|
||||||
/config /logs /users
|
/users /profile + 未登录:/login /register
|
||||||
```
|
```
|
||||||
|
|
||||||
四层职责:
|
五层职责:
|
||||||
|
|
||||||
| 层 | 位置 | 说明 |
|
| 层 | 位置 | 说明 |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| 采集 | `workbuddy_portal/collect.py` + `scheduler.py` | 纯 `urllib` 调云端;断点、去重、锁、导入导出 |
|
| 采集 | `workbuddy_portal/collect.py` + `scheduler.py` | 纯 `urllib` 调云端;断点、去重、锁、导入导出,全部按 `uid` 隔离 |
|
||||||
| 存储 | `workbuddy_portal/db.py` + `schema.sql` | SQLite WAL,单写者,运行期配置也在库里(`settings` 表) |
|
| 存储 | `workbuddy_portal/db.py` + `schema.sql` | SQLite WAL,单写者;运行期配置也在库里(`settings` 表,主键 `(user_id, key)`) |
|
||||||
| 聚合 | `workbuddy_portal/query.py` | `daily / dims / top / records / summary / bundle`,全部下推 SQL |
|
| 加固 | `workbuddy_portal/crypto.py` + `captcha.py` | 凭证静态加密(零第三方依赖手写 ChaCha20);验证码用**自写 PNG 编码器**出图 |
|
||||||
|
| 聚合 | `workbuddy_portal/query.py` | `daily / dims / top / records / summary / bundle`,全部下推 SQL,`uid` 是第一个条件 |
|
||||||
| 呈现 | `workbuddy_portal/web/` | Jinja 后台(`views.py`)+ JSON API(`api.py`)+ 静态大屏 |
|
| 呈现 | `workbuddy_portal/web/` | Jinja 后台(`views.py`)+ JSON API(`api.py`)+ 静态大屏 |
|
||||||
|
|
||||||
|
> **为什么验证码不用 SVG、也不用第三方库**:SVG 是文本,答案会明文出现在页面源码里;
|
||||||
|
> 而本项目坚持 `requirements.txt` 只有 Flask / waitress / openpyxl,所以 PNG 编码器
|
||||||
|
> (zlib 压缩 IDAT)与 5×7 点阵字模都是手写的,见 [架构说明](docs/ARCHITECTURE.md#凭证加密与验证码)。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 快速开始
|
## 快速开始
|
||||||
@@ -103,41 +114,59 @@ python manage.py migrate-csv # 可选:把旧版 CSV 存档全量导
|
|||||||
python manage.py serve # 启动,默认 0.0.0.0:8848
|
python manage.py serve # 启动,默认 0.0.0.0:8848
|
||||||
```
|
```
|
||||||
|
|
||||||
|
> 从 v1.1.0 升级上来**不需要手工介入**:`init` 会检测到旧表结构并自动迁移
|
||||||
|
> (历史数据归到首个账号、明文 Cookie 就地加密),全程带审计留痕,可重复执行。
|
||||||
|
|
||||||
### 第一次使用必做三件事
|
### 第一次使用必做三件事
|
||||||
|
|
||||||
1. **改密码**——局域网可访问,默认密码等于没锁门(「配置管理 → 修改密码」)。
|
1. **改密码**——局域网可访问,默认密码等于没锁门(「个人中心」或「配置管理 → 修改密码」)。
|
||||||
2. **填 Cookie**——「配置管理 → 凭证」,否则采集只会记一条 `cookie_expired`。
|
2. **填 Cookie**——「配置管理 → 凭证」,否则采集只会记一条 `cookie_expired`。
|
||||||
获取方式见 [用户手册](docs/USER-GUIDE.md#三获取并填写-cookie)。
|
获取方式见 [用户手册](docs/USER-GUIDE.md#三获取并填写-cookie)。
|
||||||
3. **确认调度时刻**——「任务管理」里把 `09:00,17:00` 改成你的习惯时刻,保存即生效。
|
3. **确认调度时刻**——「任务管理」里把 `09:00,17:00` 改成你的习惯时刻,保存即生效。
|
||||||
|
|
||||||
|
### 想给同事开账号?
|
||||||
|
|
||||||
|
登录页底部有「**自助注册**」入口(管理员可在「配置管理 → 实例级设置」关掉)。
|
||||||
|
注册同样要过验证码,且同一来源每天最多注册 3 个账号(可改)。
|
||||||
|
每个账号登录后填**自己的** Cookie——系统不会、也无法把某人的凭证给别人用。
|
||||||
|
|
||||||
|
> 只想内部开号、不开放注册?管理员在「用户管理」页直接新建即可;
|
||||||
|
> 命令行也行:`python manage.py passwd alice 强密码`(默认普通账号,加 `--role admin` 提权)。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 命令一览
|
## 命令一览
|
||||||
|
|
||||||
统一入口是 `manage.py`(Docker 里同样可用:`docker compose exec portal python manage.py stats`)。
|
统一入口是 `manage.py`(Docker 里同样可用:`docker compose exec portal python manage.py stats`)。
|
||||||
|
|
||||||
|
**多用户下所有涉及数据/凭证的子命令都作用于某一个账号**,用 `-u/--user <用户名>` 指定;
|
||||||
|
不指定则取「管理员优先、其次 id 最小」的那个(所以旧习惯的单账号用法仍然成立)。
|
||||||
|
唯独 `collect` 不带 `-u` 时会**逐个启用账号**跑一遍,与进程内调度线程的行为一致。
|
||||||
|
|
||||||
| 命令 | 作用 |
|
| 命令 | 作用 |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `init` | 初始化数据库(幂等)。`--user` / `--password` 指定首个管理员 |
|
| `init` | 初始化 / 迁移数据库(幂等)。`--user` / `--password` 指定首个管理员 |
|
||||||
| `serve` | 启动 Web。`--host` `--port` `--debug` `--no-scheduler` |
|
| `serve` | 启动 Web。`--host` `--port` `--debug` `--no-scheduler` |
|
||||||
| `collect` | 执行一次增量采集后退出(不想开 Web 时可挂系统计划任务) |
|
| `collect [-u 账号]` | 执行一次增量采集后退出;**不带 `-u` 则所有启用账号各跑一次** |
|
||||||
| `migrate-csv [文件]` | 从旧版 CSV 存档导入(默认自动探测旧项目路径) |
|
| `migrate-csv [文件] [-u 账号]` | 从旧版 CSV 存档导入(默认自动探测旧项目路径),必须说明「算谁的」 |
|
||||||
| `import-xlsx <文件>` | 合入官网「用量明细-导出」的 xlsx |
|
| `import-xlsx <文件> [-u 账号]` | 合入官网「用量明细-导出」的 xlsx |
|
||||||
| `import-creds` | 从 VSCode / Cursor / Trae 的 `settings.json` 读取 `codebuddyUsage.*` 写入数据库 |
|
| `import-creds [-u 账号]` | 从 VSCode / Cursor / Trae 的 `settings.json` 读取 `codebuddyUsage.*` 写入该账号 |
|
||||||
| `fill-prompt` | 回补缺失的 `User Prompt`(官网导出会丢约 22%) |
|
| `fill-prompt [-u 账号]` | 回补缺失的 `User Prompt`(官网导出会丢约 22%) |
|
||||||
| `export-csv [路径]` | 导出与官网 xlsx 同构的 CSV(默认 `data/exports/`) |
|
| `export-csv [路径] [-u 账号]` | 导出 CSV(默认 `data/exports/usage_records_<账号>.csv`,文件名带归属) |
|
||||||
| `vacuum` | `wal_checkpoint(TRUNCATE)` + `VACUUM`,回收空闲页、压缩 WAL |
|
| `vacuum` | `wal_checkpoint(TRUNCATE)` + `VACUUM`,回收空闲页、压缩 WAL |
|
||||||
| `stats` | 存档概况 + 模型维度表 + 最近采集(不联网) |
|
| `stats [-u 账号]` | 先全库概览(每账号多少条 / 多少积分 / Cookie 状态),再给指定账号的维度明细 |
|
||||||
| `status` | 调度开关 / 下次执行 / Cookie 状态 / 最近采集 |
|
| `users` | 列出所有账号:角色、状态、数据量、凭证状态、最近登录 IP |
|
||||||
| `passwd <用户> [新密码]` | 重置或创建登录账号 |
|
| `status` | 逐账号显示调度开关 / 下次执行 / Cookie 状态 / 最近采集 |
|
||||||
|
| `passwd <用户> [新密码]` | 重置或创建账号;`--role admin` 提权,`--activate` 顺手启用 |
|
||||||
|
|
||||||
### 自检工具
|
### 自检工具
|
||||||
|
|
||||||
| 脚本 | 层 | 说明 |
|
| 脚本 | 层 | 说明 |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| `tools/smoke.py` | 离线回归 | `test_client` 对真实库全页面只读渲染,**99 项断言**(历史缺陷防回归 ①~⑭、CSV 列、class↔CSS 对账、静态资源逐个 200),**不需要先起服务** |
|
| `tools/smoke.py` | 离线回归 | `test_client` 对真实库全页面只读渲染,**165 项断言**:历史缺陷防回归 ①~⑭、**多用户隔离 / 凭证保密 / 注册与验证码全链路**、CSV 列、class↔CSS 对账、静态资源逐个 200。**不需要先起服务** |
|
||||||
| `tools/check_live.py` | 真实 HTTP | 对运行中的服务走真实链路(登录 → CSRF → 各页面 → 各 API → 导出 → 安全项),**56 项断言**,基本只读 |
|
| `tools/check_live.py` | 真实 HTTP | 对运行中的服务走真实链路(登录 → CSRF → 各页面 → 各 API → 导出 → 安全项 → **验证码与响应头**),**83 项断言**,基本只读 |
|
||||||
| `tools/shots.py` | 界面实检 | Playwright 登录后逐页截图并收集 console / pageerror,产物在 `data/shots/` |
|
| `tools/shots.py` | 界面实检 | Playwright 登录后逐页截图并收集 console / pageerror,产物在 `data/shots/` |
|
||||||
|
| `tools/demo_data.py` | 示例数据 | 生成**完全合成**的示例库(两个账号,各有自己的数据与假 Cookie),文档截图基于它 |
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
python tools/smoke.py # 离线,随时可跑
|
python tools/smoke.py # 离线,随时可跑
|
||||||
@@ -146,8 +175,11 @@ python tools/check_live.py --base http://127.0.0.1:8849 # 真实 HTTP
|
|||||||
python tools/shots.py --base http://127.0.0.1:8849 --full # 逐页截图
|
python tools/shots.py --base http://127.0.0.1:8849 --full # 逐页截图
|
||||||
```
|
```
|
||||||
|
|
||||||
> `smoke.py` 会写少量 `audit_log` 审计行(被拒的配置写入也留痕),不动业务数据;
|
> `smoke.py` 会写少量 `audit_log` 审计行,并**临时**建两个普通账号用于验证权限边界与注册链路
|
||||||
> `check_live.py` 只读,但登录成功会更新 `users.last_login_at` / `login_count`。
|
> (无论成败都在 `finally` 里删掉),不动任何用量数据;
|
||||||
|
> `check_live.py` / `shots.py` 只读,但登录成功会更新 `users.last_login_at` / `login_count`。
|
||||||
|
> 两者在验证码策略为 `always` 时会**从本地库里取答案**以完成自动登录
|
||||||
|
> ——取的是会话里的 captcha id(答案本身只存在于服务端)。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -155,13 +187,16 @@ python tools/shots.py --base http://127.0.0.1:8849 --full # 逐页截图
|
|||||||
|
|
||||||
| 路径 | 作用 |
|
| 路径 | 作用 |
|
||||||
|---|---|
|
|---|---|
|
||||||
|
| `/login` | **登录**(未登录时的落点):用户名 / 密码 / **图形验证码**,底部有自助注册入口 |
|
||||||
|
| `/register` | **自助注册**:用户名、显示名、邮箱、密码 + 验证码;注册成功直接登录并引导去填自己的 Cookie |
|
||||||
| `/` | **概览**:KPI(含今日 vs 昨日整日)、采集健康度、调度状态、模型 TOP、最近采集 |
|
| `/` | **概览**:KPI(含今日 vs 昨日整日)、采集健康度、调度状态、模型 TOP、最近采集 |
|
||||||
| `/dashboard` | **ECharts 交互大屏**(独立静态页):日历热力图、趋势、维度分布、单笔 TOP,支持区间/维度/指标联动 |
|
| `/dashboard` | **ECharts 交互大屏**(独立静态页):日历热力图、趋势、维度分布、单笔 TOP,支持区间/维度/指标联动 |
|
||||||
| `/records` | **数据明细**:快捷区间、日期/模型/客户端/关键词筛选、排序、分页、展开 Prompt、导出 CSV |
|
| `/records` | **数据明细**:快捷区间、日期/模型/客户端/关键词筛选、排序、分页、展开 Prompt、导出 CSV |
|
||||||
| `/tasks` | **任务管理**:调度开关与时刻、启动补跑、按区间补采、运行历史 |
|
| `/tasks` | **任务管理**:调度开关与时刻、启动补跑、按区间补采、运行历史 |
|
||||||
| `/config` | **配置管理**:Cookie / UA、采集参数、TLS 校验、修改密码、维护动作(回补 Prompt / 导出 / 整理库) |
|
| `/config` | **配置管理**:自己的 Cookie / UA、采集参数、TLS 校验、维护动作;底部是实例级设置区(仅管理员可改) |
|
||||||
| `/logs` | **日志管理**:逐次采集详情(含 `[warn]`/`[error]` 原文)、状态筛选、应用日志、操作审计 |
|
| `/logs` | **日志管理**:**只看得到自己账号的**逐次采集详情(含 `[warn]`/`[error]` 原文)、操作审计;应用日志尾部仅管理员 |
|
||||||
| `/users` | **用户管理**(仅管理员):新建账号、改显示名/权限/密码、删除、用户操作审计 |
|
| `/profile` | **个人中心**:账号概况、我的凭证状态(密文入库)、改密码;点右上角用户名进入 |
|
||||||
|
| `/users` | **用户管理**(仅管理员):新建账号、改显示名/权限/状态/密码、删除、账号操作审计 |
|
||||||
|
|
||||||

|

|
||||||
|
|
||||||
@@ -172,26 +207,31 @@ python tools/shots.py --base http://127.0.0.1:8849 --full # 逐页截图
|
|||||||
## 接口一览
|
## 接口一览
|
||||||
|
|
||||||
全部需要登录(`/api/*` 未登录返回 `401` JSON);写接口另需 CSRF(请求头 `X-CSRF-Token`,
|
全部需要登录(`/api/*` 未登录返回 `401` JSON);写接口另需 CSRF(请求头 `X-CSRF-Token`,
|
||||||
页面已注入 `window.WB_CSRF`)。完整参数说明见 [docs/API.md](docs/API.md)。
|
页面已注入 `window.WB_CSRF`)。
|
||||||
|
**所有数据接口都只返回当前登录账号的数据** —— `user_id` 由会话决定,不接受客户端传入。
|
||||||
|
完整参数说明见 [docs/API.md](docs/API.md)。
|
||||||
|
|
||||||
| 方法 | 路径 | 作用 |
|
| 方法 | 路径 | 作用 |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| GET | `/api/manifest` | 存档总量、日期区间、存活日清单、数据源、健康状态 |
|
| GET | `/api/manifest` | 存档总量、日期区间、存活日清单、数据源、健康状态(含 `cookieChars`/`cookieBroken`) |
|
||||||
| GET | `/api/bundle` | 大屏一次取齐:全量 `daily` + 窗口 `dims`/`top`/`records`/`totals` |
|
| GET | `/api/bundle` | 大屏一次取齐:全量 `daily` + 窗口 `dims`/`top`/`records`/`totals` |
|
||||||
| GET | `/api/summary` | KPI + 环比(前一段不在存档内则不给假数字) |
|
| GET | `/api/summary` | KPI + 环比(前一段不在存档内则不给假数字) |
|
||||||
| GET | `/api/daily` | 逐日聚合(含每日分模型、24 时段) |
|
| GET | `/api/daily` | 逐日聚合(含每日分模型、24 时段) |
|
||||||
| GET | `/api/dims` | 模型 / 客户端 / 时段汇总 |
|
| GET | `/api/dims` | 模型 / 客户端 / 时段汇总 |
|
||||||
| GET | `/api/top` | 单笔消耗榜(唯一带 Prompt 摘要的接口) |
|
| GET | `/api/top` | 单笔消耗榜(唯一带 Prompt 摘要的接口) |
|
||||||
| GET | `/api/records` · `/api/records/<id>` | 明细分页 / 单条详情 |
|
| GET | `/api/records` · `/api/records/<id>` | 明细分页 / 单条详情(`<id>` 也受 `user_id` 约束) |
|
||||||
| GET | `/api/runs` · `/api/runs/<id>` | 采集运行历史 / 单次详情(含逐行日志) |
|
| GET | `/api/runs` · `/api/runs/<id>` | 采集运行历史 / 单次详情(含逐行日志) |
|
||||||
| GET | `/api/status` | 调度状态、下次执行、互斥锁、最近采集 |
|
| GET | `/api/status` | 调度状态、下次执行、互斥锁、最近采集 |
|
||||||
| GET | `/api/audit` | 操作审计分页 + 可选动作清单 |
|
| GET | `/api/audit` | 操作审计分页 + 可选动作清单(管理员看全站,普通账号看自己) |
|
||||||
| POST | `/api/collect` | 手动触发采集(可指定区间补采) |
|
| POST | `/api/collect` | 手动触发采集(可指定区间补采);未配 Cookie 回 `409 no_cookie`,密文解不开回 `409 cookie_broken` |
|
||||||
| POST | `/api/maintenance/<action>` | `fill-prompt` \| `export-csv` \| `vacuum` \| `recount` |
|
| POST | `/api/maintenance/<action>` | `fill-prompt` \| `export-csv` \| `vacuum` \| `recount`(`vacuum` 仅管理员) |
|
||||||
| GET/POST | `/api/settings` | 读 / 写配置(非法值 `400` 并列出全部错误) |
|
| GET/POST | `/api/settings` | 读 / 写配置。读只回**掩码** `cookie_hint`;非管理员写实例级键会被拒(`400` + `denied` 清单) |
|
||||||
|
| POST | `/api/profile` | 改自己的显示名 / 邮箱 |
|
||||||
| POST | `/api/password` | 修改自己的登录密码 |
|
| POST | `/api/password` | 修改自己的登录密码 |
|
||||||
| GET/POST | `/api/users` · `/api/users/<id>` | 用户管理(仅管理员) |
|
| POST | `/api/captcha` | 验证码机制自述(策略、位数、TTL、图片地址),便于排障自检 |
|
||||||
| GET | `/logs/tail` · `/records/export` | 应用日志尾部 / 按筛选流式导出 CSV |
|
| GET/POST | `/api/users` · `/api/users/<id>` · `/api/users/<id>/delete` | 用户管理(仅管理员) |
|
||||||
|
| GET | `/captcha.png?purpose=login\|register` | **图形验证码图片**(唯一无需登录的接口;每次都是新题,带 `no-store`) |
|
||||||
|
| GET | `/logs/tail` · `/records/export` | 应用日志尾部(仅管理员)/ 按筛选流式导出 CSV |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -210,23 +250,27 @@ workbuddy-portal/
|
|||||||
│ └── healthcheck.py 标准库健康检查(免登录页 /login)
|
│ └── healthcheck.py 标准库健康检查(免登录页 /login)
|
||||||
├── docs/ 文档(见下)
|
├── docs/ 文档(见下)
|
||||||
├── tools/
|
├── tools/
|
||||||
│ ├── smoke.py 离线回归(99 项断言)
|
│ ├── smoke.py 离线回归(165 项断言)
|
||||||
│ ├── check_live.py 真实 HTTP 验收(56 项断言)
|
│ ├── check_live.py 真实 HTTP 验收(83 项断言)
|
||||||
│ └── shots.py Playwright 逐页截图 + JS 报错收集
|
│ ├── shots.py Playwright 逐页截图 + JS 报错收集
|
||||||
|
│ └── demo_data.py 生成合成示例库(两个账号)
|
||||||
└── workbuddy_portal/
|
└── workbuddy_portal/
|
||||||
├── __init__.py create_app:配置 / 日志 / 蓝图 / 错误页 / 启动调度
|
├── __init__.py create_app:配置 / 日志 / 蓝图 / 错误页 / 启动调度
|
||||||
├── config.py 路径、项目标识、默认值、写时校验
|
├── config.py 路径、项目标识、默认值、写时校验、密钥管理
|
||||||
├── db.py SQLite 连接、schema、settings 读写、审计
|
├── db.py SQLite 连接、schema、按作用域读写配置、账号、审计
|
||||||
├── schema.sql 表结构
|
├── schema.sql 表结构(多用户布局)
|
||||||
├── security.py 密码哈希、session、CSRF、失败限速、safe_next、角色
|
├── crypto.py 凭证静态加密(手写 ChaCha20 + HMAC-SHA256)
|
||||||
|
├── captcha.py 图形验证码(手写 PNG 编码器 + 点阵字模)
|
||||||
|
├── security.py 密码哈希、会话、CSRF、失败限速、验证码策略、角色、响应头
|
||||||
├── client.py 云端接口(urllib)+ 编辑器凭证读取
|
├── client.py 云端接口(urllib)+ 编辑器凭证读取
|
||||||
├── collect.py 增量采集 / 去重入库 / 互斥锁 / xlsx 导入 / CSV 导出
|
├── collect.py 增量采集 / 去重入库 / 互斥锁 / xlsx 导入 / CSV 导出
|
||||||
├── scheduler.py 进程内调度线程(槽位去重 + 启动补跑)
|
├── scheduler.py 进程内调度线程(按账号遍历 + 槽位去重 + 启动补跑)
|
||||||
├── query.py SQL 聚合层
|
├── query.py SQL 聚合层(uid 必填)
|
||||||
└── web/
|
└── web/
|
||||||
├── views.py 页面路由
|
├── views.py 页面路由(含 /login /register /captcha.png /profile)
|
||||||
├── api.py JSON API
|
├── api.py JSON API
|
||||||
├── templates/ base / login / overview / tasks / config / logs / records / users / error
|
├── templates/ base / login / register / profile / overview / tasks /
|
||||||
|
│ config / logs / records / users / error
|
||||||
└── static/
|
└── static/
|
||||||
├── css/app.css 统一设计令牌
|
├── css/app.css 统一设计令牌
|
||||||
├── js/app.js 带 CSRF 的请求、表单与维护动作绑定
|
├── js/app.js 带 CSRF 的请求、表单与维护动作绑定
|
||||||
@@ -259,12 +303,28 @@ workbuddy-portal/
|
|||||||
局域网可访问 ⇒ 以下每一条都必要:
|
局域网可访问 ⇒ 以下每一条都必要:
|
||||||
|
|
||||||
- **必须改默认密码**;给只读同事发普通账号(`is_admin=0`),不要共用管理员。
|
- **必须改默认密码**;给只读同事发普通账号(`is_admin=0`),不要共用管理员。
|
||||||
- **Cookie 就是账号凭证**:只以掩码回显,存库不外传;默认开启 TLS 证书校验(`ssl_verify=1`),
|
- **数据按账号隔离**:`uid` 是所有查询的必填位置参数(漏传直接报错,不会静默返回全量);
|
||||||
仅在自签 / 企业代理场景临时关闭。
|
`/api/runs/<id>`、`/api/records/<id>` 这类按 id 取的单条接口也带 `user_id` 约束;
|
||||||
|
应用日志尾部仅管理员可看。
|
||||||
|
- **Cookie 静态加密**:ChaCha20 + HMAC-SHA256(encrypt-then-MAC)密文入库,主密钥在
|
||||||
|
`data/instance.json` 的 `cookie_key`(**与 `SECRET_KEY` 分开**,轮换代价不同)。
|
||||||
|
`get_settings()` 把加密键一律置空,要明文只有 `db.get_secret()` 一条路——
|
||||||
|
这样任何「顺手打印全部配置」的代码都带不出凭证。升级时历史明文会被自动加密。
|
||||||
|
- **Cookie 不跨账号回落**:`NO_FALLBACK_KEYS`(`cookie` / `user_agent`)不参与实例级回落,
|
||||||
|
否则新账号会「继承」管理员的凭证,属于最严重的串号越权。
|
||||||
|
- **验证码先于口令校验**:登录时先验验证码再比密码,避免攻击者拿「密码对不对」当信号,
|
||||||
|
在解验证码之前就把字典跑完。答案存服务端 `captchas` 表,**一次性、5 分钟过期、按用途隔离**,
|
||||||
|
下发到浏览器的只有随机 id(Flask 会话是签名不加密的,放答案等于送答案)。
|
||||||
|
- **注册受双重限制**:验证码 + 同 IP 每日配额(默认 3 个,可改;`allow_register=0` 可整体关闭)。
|
||||||
|
- **停用账号立即失效**:`current_user()` 每个请求回查 `users.status`,不必等 12 小时会话过期。
|
||||||
- **CSRF 全站校验**,退出登录也是 `POST`(GET 型退出能被 `<img src="/logout">` 静默触发)。
|
- **CSRF 全站校验**,退出登录也是 `POST`(GET 型退出能被 `<img src="/logout">` 静默触发)。
|
||||||
- **开放重定向防护**:登录跳转的 `next` 只接受站内相对路径,`//evil.com` 这类协议相对 URL 一律回落到 `/`。
|
- **开放重定向防护**:登录跳转的 `next` 只接受站内相对路径,`//evil.com` 这类协议相对 URL 一律回落到 `/`。
|
||||||
- **登录限速**:同 IP 连续失败 5 次锁定 10 分钟;失败计数表有上限与 TTL。
|
- **登录限速**:按 **IP 与用户名两个维度**分别计数,任一维度连续失败 5 次即锁 10 分钟;
|
||||||
- **不进版本库的文件**:`data/instance.json`(含 `secret_key`)、`data/usage.sqlite`、`logs/`、`.env`(含明文密码)。
|
失败计数表有上限与 TTL;验证码出图另有 60 秒 40 张的限速(不设限就是一条廉价的 CPU 放大路径)。
|
||||||
|
- **安全响应头**:CSP(`frame-ancestors 'none'`)、`X-Frame-Options: DENY`、`nosniff`、
|
||||||
|
`Referrer-Policy: same-origin`、COOP;`/api/*` 与 `/captcha*` 带 `Cache-Control: no-store`。
|
||||||
|
- **不进版本库的文件**:`data/instance.json`(含 `secret_key` 与 `cookie_key`)、
|
||||||
|
`data/usage.sqlite`、`logs/`、`.env`(含明文密码)。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -296,8 +356,14 @@ Apache-2.0 许可)——之所以内置而不走 CDN,是为了让大屏在
|
|||||||
|
|
||||||
`docs/images/` 的全部界面截图与 `docs/` 中的 JSON 示例**均为合成数据**:
|
`docs/images/` 的全部界面截图与 `docs/` 中的 JSON 示例**均为合成数据**:
|
||||||
模型名统一为 `demo-*`,客户端为 `vscode` / `webconsole` / `sdk`,Prompt 为通用示例文本,
|
模型名统一为 `demo-*`,客户端为 `vscode` / `webconsole` / `sdk`,Prompt 为通用示例文本,
|
||||||
|
Cookie 是 `deadbeef…` / `cafef00d…` 这类一眼可辨的假串,
|
||||||
审计 IP 取自 RFC 5737 的文档专用网段(`192.0.2.0/24`)。
|
审计 IP 取自 RFC 5737 的文档专用网段(`192.0.2.0/24`)。
|
||||||
生成方式是 `tools/demo_data.py`,所以任何人不需要真实账号就能复现整套文档。
|
生成方式是 `tools/demo_data.py`(会造 `admin` 与 `demo` 两个账号,各有自己的数据),
|
||||||
|
所以任何人不需要真实账号就能复现整套文档。截图由 `tools/shots.py` 逐页重出(11 张,
|
||||||
|
含注册页与个人中心),脚本会读示例库里的验证码答案自动过掉登录。
|
||||||
|
|
||||||
|
> 重出截图时请用**相对路径**起示例服务(`WB_DATA_DIR=data/demo`):用绝对路径会让启动日志
|
||||||
|
> 印出 `C:\Users\<用户名>\…`,而那一行正好会出现在「日志管理」页的截图上。
|
||||||
|
|
||||||
### 致谢
|
### 致谢
|
||||||
|
|
||||||
|
|||||||
@@ -6,12 +6,12 @@
|
|||||||
|
|
||||||
| 版本 | 是否接受安全修复 |
|
| 版本 | 是否接受安全修复 |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `1.1.x`(当前) | ✅ |
|
| `1.2.x`(当前) | ✅ |
|
||||||
| `< 1.1` | ❌ 请先升级 |
|
| `< 1.2` | ❌ 请先升级(1.2.0 修掉了单用户时代「Cookie 明文入库」与「人人都是管理员」两个根本问题) |
|
||||||
|
|
||||||
## 如何报告漏洞
|
## 如何报告漏洞
|
||||||
|
|
||||||
**请不要在公开 Issue 里贴出可直接利用的细节**(含真实 Cookie、`secret_key`、可复现的绕过步骤)。
|
**请不要在公开 Issue 里贴出可直接利用的细节**(含真实 Cookie、`secret_key` / `cookie_key`、可复现的绕过步骤)。
|
||||||
|
|
||||||
请通过以下任一私有渠道联系维护者:
|
请通过以下任一私有渠道联系维护者:
|
||||||
|
|
||||||
@@ -32,36 +32,90 @@
|
|||||||
|
|
||||||
理解这些边界,有助于你判断某个现象是「设计如此」还是「真的漏洞」:
|
理解这些边界,有助于你判断某个现象是「设计如此」还是「真的漏洞」:
|
||||||
|
|
||||||
|
### 身份、会话与权限
|
||||||
|
|
||||||
| 项 | 做法 | 位置 |
|
| 项 | 做法 | 位置 |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| 全站鉴权 | 每个页面都有 `@login_required`,每个 `/api/*` 未登录返回 401 JSON | `workbuddy_portal/security.py`、`web/views.py` |
|
| 全站鉴权 | 每个页面都有 `@login_required`,每个 `/api/*` 未登录返回 401 JSON | `security.login_required`、`web/views.py` |
|
||||||
| CSRF | 所有写请求必须带 `X-CSRF-Token`,页面注入 `window.WB_CSRF`,服务端统一拦截 | `security.check_csrf` |
|
| 角色 | 管理员 / 普通两档;`/users`、`/logs/tail`、`vacuum` 等仅管理员 | `security.admin_required` |
|
||||||
|
| 停用即失效 | `current_user()` **每个请求**回查 `users.status`,不等 12 小时会话过期 | `security.current_user` |
|
||||||
|
| 自锁保护 | 管理员不能停用 / 降权 / 删除自己 | `web/api.py` |
|
||||||
|
| CSRF | 所有写请求必须带 `X-CSRF-Token`,页面注入 `window.WB_CSRF`,服务端统一拦截;退出登录也是 POST | `security.check_csrf` |
|
||||||
| 会话签名 | Flask `secret_key` 由 `data/instance.json` 持有,首次启动随机生成 | `workbuddy_portal/config.py` |
|
| 会话签名 | Flask `secret_key` 由 `data/instance.json` 持有,首次启动随机生成 | `workbuddy_portal/config.py` |
|
||||||
| 口令存储 | 加盐哈希,不存明文 | `security.hash_password` |
|
| 会话 cookie | `HttpOnly` + `SameSite=Lax` + `Path=/`;HTTPS 部署可设 `WB_COOKIE_SECURE=1` 打开 Secure | `workbuddy_portal/__init__.py` |
|
||||||
| 凭据脱敏 | 云端 **Cookie 是账号凭证**:入库后页面与接口**都不回传全文**,只给「N 字符,结尾 …xxxx」 | `config.SECRET_KEYS`、`web/views.py` |
|
| 口令存储 | 加盐哈希(PBKDF2-SHA256),不存明文;强度校验(≥8 位、含两类字符、不得等于用户名) | `security.hash_password` / `password_problem` |
|
||||||
| 开放重定向 | 登录后的 `next` 只允许站内相对路径 | `web/views.py` |
|
| 开放重定向 | 登录后的 `next` 只允许站内相对路径,`//evil.com` 一律回落到 `/` | `security.safe_next` |
|
||||||
| TLS 校验 | 默认开启,**不提供「关掉校验」的快捷开关**(Cookie 不该裸奔) | `settings.ssl_verify` |
|
| 失败限速 | **IP 与用户名两个维度**分别计数,任一维度连续失败 5 次锁 10 分钟;计数表有上限与 TTL | `security.auth_locked` / `note_auth_fail` |
|
||||||
| 容器权限 | 运行层非 root(uid/gid 1000 `app`) | `Dockerfile` |
|
| 响应头 | CSP(`frame-ancestors 'none'`)、`X-Frame-Options: DENY`、`nosniff`、`Referrer-Policy: same-origin`、COOP;`/api/*` 与 `/captcha*` 带 `no-store` | `security.apply_security_headers` |
|
||||||
|
|
||||||
**绝不入库**:`data/instance.json`(含 `secret_key`)、`data/usage.sqlite`、`logs/*`、`.env`。
|
### 多用户数据隔离
|
||||||
|
|
||||||
|
| 项 | 做法 | 位置 |
|
||||||
|
|---|---|---|
|
||||||
|
| 强隔离 | `uid` 是 `conn` 之后的**第一个位置参数且无默认值**;漏传直接 `TypeError`,不会退化成「返回全量」 | `query.py` / `collect.py` / `scheduler.py` |
|
||||||
|
| 按 id 取单条也隔离 | `/api/records/<id>`、`/api/runs/<id>` 的 `WHERE` 都带 `user_id` | `web/api.py` |
|
||||||
|
| 配置作用域 | 三级回落 `个人 → 实例(user_id=0) → DEFAULTS`;`GLOBAL_KEYS` 只有管理员能改 | `db.get_settings`、`config.GLOBAL_KEYS` |
|
||||||
|
| 凭证不回落 | `NO_FALLBACK_KEYS = {cookie, user_agent}` **不参与实例级回落** —— 回落等于新账号继承管理员凭证,是最严重的串号越权 | `db.get_setting` |
|
||||||
|
| 日志隔离 | 采集运行记录按账号下发;`/logs/tail`(应用日志文件)仅管理员 | `web/views.py` |
|
||||||
|
| 导出不互相覆盖 | `/records/export` 与 CLI `export-csv` 的文件名带账号名 | `web/views.py`、`collect.export_csv` |
|
||||||
|
|
||||||
|
### 云端凭证(Cookie)的保密
|
||||||
|
|
||||||
|
| 项 | 做法 | 位置 |
|
||||||
|
|---|---|---|
|
||||||
|
| **静态加密** | ChaCha20(RFC 8439 §2.3)+ HMAC-SHA256 **encrypt-then-MAC**,密文 `v1.<b64salt>.<b64nonce>.<b64ct>.<b64tag>`;手写实现,零第三方依赖 | `workbuddy_portal/crypto.py` |
|
||||||
|
| 密钥分离 | 主密钥 `cookie_key` 与 `SECRET_KEY` **分开键位**存放(两者轮换代价不同:换 `cookie_key` 会让所有已存 Cookie 失效) | `config.encryption_key` / `secret_key` |
|
||||||
|
| 唯一明文出口 | `db.get_secret()` 是取明文的**唯一**通道;`get_settings()` 把 `ENCRYPTED_KEYS` 一律置空,所以「顺手回传全部配置」的代码带不出凭证 | `db.py` |
|
||||||
|
| 只回掩码 | 页面与 `/api/settings` 只给「N 字符,结尾 …xxxx」与 `broken` 标志,`secret_state()` 不返回明文 | `db.secret_state` |
|
||||||
|
| 失败即报错 | `decrypt()` 校验失败**抛 `DecryptError`**,绝不「失败就返回原值」;非 `v1.` 前缀视为历史明文原样返回(下次写入自动升级) | `crypto.decrypt` |
|
||||||
|
| 历史明文清理 | 启动迁移时把 settings 里残留的明文凭证就地加密,并写一条 `encrypt_secrets` 审计 | `db._encrypt_legacy_secrets` |
|
||||||
|
| TLS 校验 | 默认开启,**不提供「关掉校验」的快捷开关**(Cookie 不该裸奔) | `settings.ssl_verify` |
|
||||||
|
|
||||||
|
### 防自动化攻击
|
||||||
|
|
||||||
|
| 项 | 做法 | 位置 |
|
||||||
|
|---|---|---|
|
||||||
|
| 图形验证码 | 手写 PNG 编码器 + 5×7 点阵字模 + 干扰线/噪点;**不用 SVG**(SVG 是文本,答案会明文出现在页面源码里) | `workbuddy_portal/captcha.py` |
|
||||||
|
| 答案不进会话 | 答案只写服务端 `captchas` 表;会话里仅存随机 id —— Flask 会话是「签名不加密」的,放答案等于送答案 | `security.issue_captcha` |
|
||||||
|
| 一次性 | 校验后立即删除,且**先删后判**;5 分钟过期、按 `purpose` 隔离,不能拿注册的题去登登录 | `captcha.verify` |
|
||||||
|
| 先验码后验密 | 登录先校验验证码再比对口令,避免攻击者拿「密码对不对」当提前信号跑完字典 | `web/views.py` |
|
||||||
|
| 出图限速 | 每来源 60 秒最多 40 张(不设限就是一条廉价的 CPU/带宽放大路径) | `security.captcha_fetch_allowed` |
|
||||||
|
| 注册配额 | 同 IP 每日最多注册 N 个(默认 3,可改);`allow_register=0` 可整体关闭 | `security.register_quota` |
|
||||||
|
|
||||||
|
### 其它
|
||||||
|
|
||||||
|
| 项 | 做法 | 位置 |
|
||||||
|
|---|---|---|
|
||||||
|
| 容器权限 | 运行层非 root(uid/gid 1000 `app`) | `Dockerfile` |
|
||||||
|
| 上传体量 | `MAX_CONTENT_LENGTH = 4 MiB` | `workbuddy_portal/__init__.py` |
|
||||||
|
| 不索引 | 页面带 `noindex, nofollow` | `web/templates/base.html` |
|
||||||
|
|
||||||
|
**绝不入库**:`data/instance.json`(含 `secret_key` 与 `cookie_key`)、`data/usage.sqlite`、
|
||||||
|
`logs/*`、`.env`。
|
||||||
`.gitignore` 已覆盖;改动忽略规则后请用 `git check-ignore -v <file>` 逐条复核。
|
`.gitignore` 已覆盖;改动忽略规则后请用 `git check-ignore -v <file>` 逐条复核。
|
||||||
|
注意 `.gitignore` **不支持行尾注释**(`path # 说明` 会让整行变成永不匹配的模式)。
|
||||||
|
|
||||||
## 已知的**非**目标(部署方需自行处理)
|
## 已知的**非**目标(部署方需自行处理)
|
||||||
|
|
||||||
本项目刻意不做下面这些,请按你的环境补齐:
|
本项目刻意不做下面这些,请按你的环境补齐:
|
||||||
|
|
||||||
- **没有多租户与细粒度权限**:登录用户即为管理员,能改配置、能看全部数据。
|
- **没有强制 HTTPS**:请由反向代理(nginx/Caddy)终止 TLS。纯 HTTP 部署时
|
||||||
- **没有限流与账号锁定**:公网暴露前请置于反向代理的 rate limit 之后。
|
**不要**设 `WB_COOKIE_SECURE=1`,否则浏览器不回传会话 cookie(表现为反复被弹回登录页)。
|
||||||
- **没有强制 HTTPS**:请由反向代理(nginx/Caddy)终止 TLS。
|
- **没有 CSRF 之外的重放防护 / 没有 WAF**:公网暴露前请置于反向代理的 rate limit 之后。
|
||||||
- **没有备份机制**:备份策略需要你自己定(见 `docs/DEPLOYMENT.md`)。
|
- **没有备份机制**:备份策略需要你自己定(见 `docs/DEPLOYMENT.md`)。
|
||||||
|
- **没有邮件/短信找回**:邮箱只是联系信息,不参与认证;密码忘掉由管理员重置。
|
||||||
- **不建议直接暴露到公网**:设计前提是局域网或 VPN 内使用。
|
- **不建议直接暴露到公网**:设计前提是局域网或 VPN 内使用。
|
||||||
- **Cookie 的获取方式由使用者负责**:手动从浏览器复制。请勿把该 Cookie 分享给他人,
|
- **Cookie 的获取方式由使用者负责**:手动从浏览器复制、**粘贴给自己的账号**。
|
||||||
它的权限等同于你的账号;轮换后记得在「配置管理」页更新。
|
它的权限等同于你的账号,请勿分享给他人;轮换后记得在「配置管理」页更新。
|
||||||
|
- **`cookie_key` 泄露 = 所有 Cookie 泄露**:`data/instance.json` 的权限应与数据库同级看待。
|
||||||
|
|
||||||
## 部署前的最小检查清单
|
## 部署前的最小检查清单
|
||||||
|
|
||||||
- [ ] 已修改默认管理员口令(`WB_ADMIN_PASSWORD`),不再是 `admin123`
|
- [ ] 已修改默认管理员口令(`WB_ADMIN_PASSWORD`),不再是 `admin123`
|
||||||
- [ ] `data/` 与 `logs/` 目录的权限只对服务账号可读写
|
- [ ] 已确认是否要开放自助注册;开放时按需调小 `register_max_per_ip`
|
||||||
- [ ] 前面有反向代理并启用了 HTTPS
|
- [ ] `data/` 与 `logs/` 目录的权限只对服务账号可读写(内含 `instance.json` 的两个密钥)
|
||||||
|
- [ ] 前面有反向代理并启用了 HTTPS;若是 HTTPS,已设 `WB_COOKIE_SECURE=1`
|
||||||
- [ ] 确认 `data/instance.json` 没有被提交到任何仓库
|
- [ ] 确认 `data/instance.json` 没有被提交到任何仓库
|
||||||
- [ ] 已规划备份(SQLite 库是唯一正本)
|
- [ ] 已规划备份(SQLite 库是唯一正本);备份文件同样受 `cookie_key` 保护,需按机密对待
|
||||||
|
- [ ] 升级到 1.2.0 后登录一次「配置管理」,确认 Cookie 状态为「已配置」而不是
|
||||||
|
「已保存但无法解密」
|
||||||
|
|||||||
@@ -37,6 +37,9 @@ services:
|
|||||||
WB_ADMIN_USER: ${WB_ADMIN_USER:-admin}
|
WB_ADMIN_USER: ${WB_ADMIN_USER:-admin}
|
||||||
WB_ADMIN_PASSWORD: ${WB_ADMIN_PASSWORD:-}
|
WB_ADMIN_PASSWORD: ${WB_ADMIN_PASSWORD:-}
|
||||||
WB_DISABLE_SCHEDULER: ${WB_DISABLE_SCHEDULER:-0}
|
WB_DISABLE_SCHEDULER: ${WB_DISABLE_SCHEDULER:-0}
|
||||||
|
# 会话 Cookie 是否只走 HTTPS。纯 HTTP 部署必须留 0:设成 1 时浏览器
|
||||||
|
# 不会回传会话 Cookie,表现为「登录成功又立刻跳回登录页」。改它要 up -d(环境变量)。
|
||||||
|
WB_COOKIE_SECURE: ${WB_COOKIE_SECURE:-0}
|
||||||
WB_IMPORT_CREDS: ${WB_IMPORT_CREDS:-0}
|
WB_IMPORT_CREDS: ${WB_IMPORT_CREDS:-0}
|
||||||
WB_IMPORT_XLSX: ${WB_IMPORT_XLSX:-}
|
WB_IMPORT_XLSX: ${WB_IMPORT_XLSX:-}
|
||||||
volumes:
|
volumes:
|
||||||
|
|||||||
@@ -6,8 +6,9 @@
|
|||||||
|
|
||||||
| 项 | 说明 |
|
| 项 | 说明 |
|
||||||
|---|---|
|
|---|---|
|
||||||
| 认证 | 全部需要登录。`/api/*` 未登录返回 **401** JSON(页面则跳登录页) |
|
| 认证 | 全部需要登录。`/api/*` 未登录返回 **401** JSON(页面则跳登录页)。唯一例外是 `/captcha.png` |
|
||||||
| CSRF | **写接口**(`POST`)需带 `X-CSRF-Token` 头,或表单域 `_csrf`;缺失 / 错误返回 **400** |
|
| **数据作用域** | **所有数据接口只返回当前登录账号的数据**。`user_id` 由服务端会话决定,**不接受客户端传入** —— 传 `?user_id=1` 会被忽略 |
|
||||||
|
| CSRF | **写接口**(`POST`)需带 `X-CSRF-Token` 头,或表单域 `_csrf`;缺失 / 错误返回 **400**。未登录的 POST 也会先被 CSRF 拦成 400(这比先鉴权更保守) |
|
||||||
| 日期参数 | `from` / `to`,`YYYY-MM-DD`。也容忍 `YYYY/MM/DD`、带时间的写法;起止写反会自动交换 |
|
| 日期参数 | `from` / `to`,`YYYY-MM-DD`。也容忍 `YYYY/MM/DD`、带时间的写法;起止写反会自动交换 |
|
||||||
| 非法参数 | 无法识别时返回 **400** 且带人话说明(如 `参数 from 不是合法日期:abc(正确写法 2026-09-08)`),**不会 500** |
|
| 非法参数 | 无法识别时返回 **400** 且带人话说明(如 `参数 from 不是合法日期:abc(正确写法 2026-09-08)`),**不会 500** |
|
||||||
| 分页 | `page`(默认 1)+ `size`(默认 50,上限 500) |
|
| 分页 | `page`(默认 1)+ `size`(默认 50,上限 500) |
|
||||||
@@ -27,6 +28,8 @@
|
|||||||
| `forbidden` | 403 | 已登录但权限不足 |
|
| `forbidden` | 403 | 已登录但权限不足 |
|
||||||
| `not_found` | 404 | 接口或资源不存在 |
|
| `not_found` | 404 | 接口或资源不存在 |
|
||||||
| `busy` | 409 | 已有采集在跑(单写者约束) |
|
| `busy` | 409 | 已有采集在跑(单写者约束) |
|
||||||
|
| `no_cookie` | 409 | 本账号还没配 Cookie,无法采集 |
|
||||||
|
| `cookie_broken` | 409 | Cookie 密文解不开(`cookie_key` 换过),需重新粘贴 |
|
||||||
| `cookie_expired` | 401 | 云端 Cookie 失效,需去「配置管理」更新 |
|
| `cookie_expired` | 401 | 云端 Cookie 失效,需去「配置管理」更新 |
|
||||||
| `api` | 502 | 云端接口异常 |
|
| `api` | 502 | 云端接口异常 |
|
||||||
| `internal` | 500 | 服务端异常 |
|
| `internal` | 500 | 服务端异常 |
|
||||||
@@ -235,29 +238,82 @@ KPI + 环比。`from`/`to` 缺省时自动取全量区间。
|
|||||||
|
|
||||||
### GET `/api/settings`
|
### GET `/api/settings`
|
||||||
|
|
||||||
读配置。**`cookie` 只回掩码**,绝不回明文;内部簿记键(`slot:*`)不返回。
|
读**当前登录账号的有效配置**。返回的是一个**扁平字典**:配置键 → 值。
|
||||||
|
|
||||||
|
> **凭证只回掩码**:`cookie` 这个键固定为空串,真正的状态在 `cookie_hint` / `cookie_broken`;
|
||||||
|
> 内部簿记键(`slot:*`)根本不返回。
|
||||||
|
> 下例中的数字与尾号都是合成示例数据,不是任何真实实例的值。
|
||||||
|
|
||||||
```json
|
```json
|
||||||
{
|
{
|
||||||
"values": { "page_size": "200", "schedule_times": "09:00,17:00", ... },
|
"api_base": "https://www.workbuddy.cn",
|
||||||
"cookie_hint": "4054 字符,…09db9a660825",
|
"api_path": "/billing/meter/get-user-request-usage",
|
||||||
"num_settings": { "page_size": [20, 1000, "条/页"], ... },
|
"page_size": "200",
|
||||||
"bool_settings": ["schedule_enabled", "catch_up"]
|
"schedule_times": "09:00,17:00",
|
||||||
|
"ssl_verify": "1",
|
||||||
|
"cookie": "",
|
||||||
|
"cookie_hint": "92 字符,…c0ffee",
|
||||||
|
"cookie_broken": false,
|
||||||
|
"user_agent": "Mozilla/5.0 (...)",
|
||||||
|
"_globalKeys": ["allow_register", "api_base", "api_path",
|
||||||
|
"captcha_length", "captcha_policy", "register_max_per_ip"],
|
||||||
|
"_canEditGlobal": true
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
|
| 字段 | 说明 |
|
||||||
|
|---|---|
|
||||||
|
| 各配置键 | 有效值(`个人 → 实例 → DEFAULTS` 三级回落后的结果) |
|
||||||
|
| `cookie` | **恒为空串** —— `db.get_settings()` 统一置空,明文只能经 `db.get_secret()` 取 |
|
||||||
|
| `cookie_hint` | 「N 字符,…尾 4 位」;未配置时为空串 |
|
||||||
|
| `cookie_broken` | `true` 表示密文解不开(`cookie_key` 换过),需重新粘贴 Cookie |
|
||||||
|
| `_globalKeys` | 实例级键清单(所有账号共用一份,只有管理员能改) |
|
||||||
|
| `_canEditGlobal` | 当前账号能否改 `_globalKeys` 里的键 |
|
||||||
|
|
||||||
### GET `/api/users`(管理员)
|
### GET `/api/users`(管理员)
|
||||||
|
|
||||||
```json
|
```json
|
||||||
{ "items": [ { "id": 1, "username": "admin", "display_name": "管理员",
|
{ "items": [ { "id": 1, "username": "admin", "display_name": "管理员", "email": null,
|
||||||
"is_admin": 1, "created_at": "...", "last_login_at": "...", "login_count": 12 } ] }
|
"is_admin": 1, "status": "active", "created_at": "...",
|
||||||
|
"last_login_at": "...", "last_login_ip": "192.0.2.10", "login_count": 12 } ] }
|
||||||
```
|
```
|
||||||
|
|
||||||
> **口令散列永不出现在响应里。**
|
> **口令散列永不出现在响应里**(`_user_public()` 只挑安全字段)。
|
||||||
|
|
||||||
### GET `/logs/tail`
|
### POST `/api/profile`
|
||||||
|
|
||||||
应用日志尾部。
|
改**自己**的显示名与邮箱。
|
||||||
|
|
||||||
|
```json
|
||||||
|
{ "display_name": "新显示名", "email": "me@example.com" }
|
||||||
|
```
|
||||||
|
|
||||||
|
### POST `/api/captcha`
|
||||||
|
|
||||||
|
验证码机制的**自述**,便于排障时自检(不需要猜当前策略是什么)。
|
||||||
|
|
||||||
|
```json
|
||||||
|
{ "policy": "always", "length": 4, "ttl_seconds": 300,
|
||||||
|
"image_url": "/captcha.png",
|
||||||
|
"note": "答案只存在服务端 captchas 表;一次性使用,校验后立即删除。" }
|
||||||
|
```
|
||||||
|
|
||||||
|
### GET `/captcha.png`
|
||||||
|
|
||||||
|
**唯一不需要登录的接口**,返回一张 PNG 图形验证码。
|
||||||
|
|
||||||
|
| 参数 | 默认 | 说明 |
|
||||||
|
|---|---|---|
|
||||||
|
| `purpose` | `login` | `login` \| `register`;其它值一律收敛为 `login`(不会 500) |
|
||||||
|
|
||||||
|
- 响应头带 `Cache-Control: no-store`(缓存旧图会导致「图没变但怎么输都错」);
|
||||||
|
- 每来源 60 秒最多 40 张,超限返回 **429**;
|
||||||
|
- **答案不会出现在响应里,也不会出现在任何页面源码或会话中** ——
|
||||||
|
服务端只把随机 id 写进会话(`cap_login` / `cap_register`),答案留在 `captchas` 表。
|
||||||
|
|
||||||
|
### GET `/logs/tail`(**仅管理员**)
|
||||||
|
|
||||||
|
应用日志尾部。普通账号访问返回 **403**(账号自己的采集日志请用 `/api/runs`)。
|
||||||
|
|
||||||
| 参数 | 默认 | 说明 |
|
| 参数 | 默认 | 说明 |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
@@ -294,20 +350,25 @@ KPI + 环比。`from`/`to` 缺省时自动取全量区间。
|
|||||||
|---|---|
|
|---|---|
|
||||||
| 成功 | `200 {"ok": true, "result": {"message": "新增 11 条,重复 6 条,存档共 944 条", ...}}` |
|
| 成功 | `200 {"ok": true, "result": {"message": "新增 11 条,重复 6 条,存档共 944 条", ...}}` |
|
||||||
| 已有采集在跑 | `409 {"ok": false, "error": "busy", "message": "..."}` |
|
| 已有采集在跑 | `409 {"ok": false, "error": "busy", "message": "..."}` |
|
||||||
|
| 本账号未配 Cookie | `409 {"ok": false, "error": "no_cookie", "message": "..."}` |
|
||||||
|
| Cookie 解不开 | `409 {"ok": false, "error": "cookie_broken", "message": "..."}` |
|
||||||
| Cookie 失效 | `401 {"ok": false, "error": "cookie_expired", "message": "..."}` |
|
| Cookie 失效 | `401 {"ok": false, "error": "cookie_expired", "message": "..."}` |
|
||||||
| 云端异常 | `502 {"ok": false, "error": "api", "message": "..."}` |
|
| 云端异常 | `502 {"ok": false, "error": "api", "message": "..."}` |
|
||||||
| 日期不合法 | `400 {"ok": false, "error": "bad_request", "message": "起始日期不合法:..."}` |
|
| 日期不合法 | `400 {"ok": false, "error": "bad_request", "message": "起始日期不合法:..."}` |
|
||||||
|
|
||||||
|
> 采集只使用**当前账号自己的** Cookie(`collect.load_credentials(conn, uid)`),
|
||||||
|
> 且会写一条带 `user_id` 的 `collect_runs`。多用户下不要并发触发采集(单写者约束)。
|
||||||
|
|
||||||
### POST `/api/maintenance/<action>`
|
### POST `/api/maintenance/<action>`
|
||||||
|
|
||||||
把 CLI 维护动作搬到页面。
|
把 CLI 维护动作搬到页面。
|
||||||
|
|
||||||
| `action` | 作用 |
|
| `action` | 作用 | 权限 |
|
||||||
|---|---|
|
|---|---|---|
|
||||||
| `fill-prompt` | 从云端回补缺失的 Prompt |
|
| `fill-prompt` | 从云端回补缺失的 Prompt | 本人 |
|
||||||
| `export-csv` | 全量导出 CSV 到 `data/exports/` |
|
| `export-csv` | 全量导出 CSV 到 `data/exports/`(文件名带账号名) | 本人 |
|
||||||
| `vacuum` | `wal_checkpoint(TRUNCATE)` + `VACUUM` |
|
| `vacuum` | `wal_checkpoint(TRUNCATE)` + `VACUUM` | **仅管理员** |
|
||||||
| `recount` | 重新统计并返回当前条数 |
|
| `recount` | 重新统计并返回当前条数 | 本人 |
|
||||||
|
|
||||||
未知动作返回 **404**。成功返回 `{"ok": true, "message": "..."}`。
|
未知动作返回 **404**。成功返回 `{"ok": true, "message": "..."}`。
|
||||||
|
|
||||||
@@ -321,12 +382,18 @@ KPI + 环比。`from`/`to` 缺省时自动取全量区间。
|
|||||||
|---|---|
|
|---|---|
|
||||||
| 全部合法 | `200 {"ok": true, "changed": ["page_size"], "ignored": []}` |
|
| 全部合法 | `200 {"ok": true, "changed": ["page_size"], "ignored": []}` |
|
||||||
| 有非法值 | `400 {"ok": false, "error": "invalid", "errors": ["page_size 需在 20 ~ 1000 条/页 之间"]}` |
|
| 有非法值 | `400 {"ok": false, "error": "invalid", "errors": ["page_size 需在 20 ~ 1000 条/页 之间"]}` |
|
||||||
|
| 非管理员改实例级键 | `400 {"ok": false, "error": "invalid", "errors": ["以下为实例级配置,仅管理员可修改:api_base"]}` |
|
||||||
|
|
||||||
要点:
|
要点:
|
||||||
|
|
||||||
- `cookie` **留空 = 不修改**(不会把已有 Cookie 清掉);
|
- `cookie` **留空 = 不修改**(不会把已有 Cookie 清掉);写 `__clear__` 或 `-` 才是清空;
|
||||||
|
- `cookie` 落库前会**自动加密**(`db.set_secret`),写进去的永远不是明文;
|
||||||
|
- **实例级键**(`_globalKeys`:`api_base` / `api_path` / `allow_register` /
|
||||||
|
`register_max_per_ip` / `captcha_policy` / `captcha_length`)非管理员**写不了**
|
||||||
|
—— 否则任意注册用户都能把大家的数据采集指向别的服务器;
|
||||||
- 未知键被忽略并在 `ignored` 里列出,**不会被写成任意键**;
|
- 未知键被忽略并在 `ignored` 里列出,**不会被写成任意键**;
|
||||||
- 内部键(`slot:*`)被忽略;
|
- 内部键(`slot:*`)被忽略;
|
||||||
|
- 改了 `schedule_times` / `schedule_enabled` 会清掉**自己**的槽位标记,新时刻立即生效;
|
||||||
- 每次拒绝都会写一条 `settings_rejected` 审计。
|
- 每次拒绝都会写一条 `settings_rejected` 审计。
|
||||||
|
|
||||||
### POST `/api/password`
|
### POST `/api/password`
|
||||||
@@ -342,22 +409,36 @@ KPI + 环比。`from`/`to` 缺省时自动取全量区间。
|
|||||||
### POST `/api/users`(管理员)
|
### POST `/api/users`(管理员)
|
||||||
|
|
||||||
```json
|
```json
|
||||||
{ "username": "viewer", "display_name": "只读同事", "password": "…", "is_admin": false }
|
{ "username": "viewer", "display_name": "只读同事", "email": "viewer@example.com",
|
||||||
|
"password": "…", "password2": "…", "is_admin": false }
|
||||||
```
|
```
|
||||||
|
|
||||||
|
`is_admin` **默认 false**(多用户系统里「默认给管理员」是最常见的越权起点)。
|
||||||
|
用户名 / 口令强度与自助注册同一套校验。
|
||||||
|
|
||||||
### POST `/api/users/<id>`(管理员)
|
### POST `/api/users/<id>`(管理员)
|
||||||
|
|
||||||
```json
|
```json
|
||||||
{ "display_name": "新名字", "is_admin": true, "password": "可选,重置密码" }
|
{ "display_name": "新名字", "email": "…", "is_admin": true,
|
||||||
|
"status": "active", "password": "可选,重置密码" }
|
||||||
```
|
```
|
||||||
|
|
||||||
|
**自锁护栏**(服务端强制,全部返回 400):
|
||||||
|
|
||||||
|
1. 不能取消自己的管理员身份;
|
||||||
|
2. 不能停用自己的账号;
|
||||||
|
3. 不能把最后一个**启用状态的**管理员降权或停用。
|
||||||
|
|
||||||
### POST `/api/users/<id>/delete`(管理员)
|
### POST `/api/users/<id>/delete`(管理员)
|
||||||
|
|
||||||
删除账号。**三重护栏**(服务端强制):
|
删除账号。护栏:
|
||||||
|
|
||||||
1. 不能取消自己的管理员身份;
|
1. 不能删除自己;
|
||||||
2. 不能删除自己;
|
2. 至少要保留一个账号;
|
||||||
3. 至少保留一个账号。
|
3. 不能删掉最后一个启用状态的管理员。
|
||||||
|
|
||||||
|
请求体可选 `{"keep_data": true}` 保留其用量数据;**默认连同数据与 Cookie 一起删除** ——
|
||||||
|
留下孤儿数据既占空间,也会在有人重新注册同名账号时被看到(`user_id` 复用风险)。
|
||||||
|
|
||||||
违反返回 `400`。
|
违反返回 `400`。
|
||||||
|
|
||||||
|
|||||||
@@ -24,15 +24,17 @@ client.py 纯 urllib 调云端;读编辑器 settings.json 取凭证
|
|||||||
│
|
│
|
||||||
▼
|
▼
|
||||||
collect.py 断点续采 → 文件锁 → 规范化 → 分批 upsert;导入/导出也在这
|
collect.py 断点续采 → 文件锁 → 规范化 → 分批 upsert;导入/导出也在这
|
||||||
│ ▲
|
│ ▲ 所有函数都要求 uid
|
||||||
│ │ scheduler.py 只是「到点调 collect.sync()」
|
│ │ scheduler.py 只是「到点调 collect.sync(conn, uid, ...)」
|
||||||
▼
|
▼
|
||||||
db.py + schema.sql SQLite(WAL),单写者;settings 表兼作运行期配置
|
crypto.py 凭证静态加密(手写 ChaCha20 + HMAC);captcha.py 出验证码图
|
||||||
|
▼
|
||||||
|
db.py + schema.sql SQLite(WAL),单写者;settings 表兼作运行期配置(主键 (user_id,key))
|
||||||
│
|
│
|
||||||
▼
|
▼
|
||||||
query.py 全部聚合下推 SQL:daily / dims / top / records / summary / bundle / manifest
|
query.py 全部聚合下推 SQL:daily / dims / top / records / summary / bundle / manifest
|
||||||
│
|
│ uid 是 WHERE 的第一个条件
|
||||||
├──▶ web/views.py Jinja 后台(7 个页面)
|
├──▶ web/views.py Jinja 后台(含 /login /register /profile + 6 个数据页)
|
||||||
└──▶ web/api.py JSON(大屏 + 页面异步调用)
|
└──▶ web/api.py JSON(大屏 + 页面异步调用)
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -40,38 +42,64 @@ query.py 全部聚合下推 SQL:daily / dims / top / records / summa
|
|||||||
任何一次查询变化都要重新跑生成器。现在聚合全部下推 SQL,页面与接口共享同一个 `query` 层,
|
任何一次查询变化都要重新跑生成器。现在聚合全部下推 SQL,页面与接口共享同一个 `query` 层,
|
||||||
口径不可能不一致。
|
口径不可能不一致。
|
||||||
|
|
||||||
|
**关键点:没有「当前用户」这种隐式全局。** `uid` 必须由调用方一路显式传下去
|
||||||
|
(见 [七、安全模型](#七安全模型)),所以「忘了过滤」在类型层面就写不出来。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 二、数据模型
|
## 二、数据模型
|
||||||
|
|
||||||
```sql
|
```sql
|
||||||
|
-- 多用户布局:所有按账号隔离的表都以 user_id 打头
|
||||||
usage_records(
|
usage_records(
|
||||||
request_id TEXT PRIMARY KEY, -- 云端请求 ID,去重靠它
|
user_id INTEGER NOT NULL DEFAULT 0,
|
||||||
ts TEXT NOT NULL, -- 本地时间戳 'YYYY-MM-DD HH:MM:SS'
|
request_id TEXT NOT NULL, -- 云端请求 ID,去重靠它
|
||||||
day TEXT NOT NULL, -- 派生字段:便于按天聚合与建索引
|
ts TEXT NOT NULL, -- 本地时间戳 'YYYY-MM-DD HH:MM:SS'
|
||||||
|
day TEXT NOT NULL, -- 派生字段:便于按天聚合与建索引
|
||||||
hour INTEGER NOT NULL, -- 派生字段:0-23,供时段分布
|
hour INTEGER NOT NULL, -- 派生字段:0-23,供时段分布
|
||||||
model TEXT, client TEXT,
|
model TEXT NOT NULL DEFAULT '-', client TEXT NOT NULL DEFAULT '-',
|
||||||
credits REAL NOT NULL,
|
credits REAL NOT NULL DEFAULT 0,
|
||||||
prompt TEXT, -- 可截断(max_prompt)
|
prompt TEXT, -- 可截断(max_prompt)
|
||||||
first_seen TEXT, last_seen TEXT,
|
first_seen TEXT NOT NULL, last_seen TEXT NOT NULL,
|
||||||
cloud_ts TEXT -- 云端原始时间,用于漂移检测
|
cloud_ts TEXT, -- 云端原始时间,用于漂移检测
|
||||||
|
PRIMARY KEY (user_id, request_id) -- 复合主键:去重是「按账号」去重
|
||||||
)
|
)
|
||||||
|
|
||||||
collect_runs(
|
collect_runs(
|
||||||
id INTEGER PRIMARY KEY, trigger TEXT, status TEXT,
|
id INTEGER PRIMARY KEY, user_id INTEGER NOT NULL DEFAULT 0,
|
||||||
|
trigger TEXT, status TEXT,
|
||||||
started_at, finished_at, duration_ms,
|
started_at, finished_at, duration_ms,
|
||||||
win_from, win_to, -- 本次扫描窗口
|
win_from, win_to, -- 本次扫描窗口
|
||||||
fetched, added, dup, total, conflicts,
|
fetched, added, dup, total, conflicts,
|
||||||
exit_code, message, detail -- detail 存逐行日志原文
|
exit_code, message, detail -- detail 存逐行日志原文
|
||||||
)
|
)
|
||||||
|
|
||||||
settings(key PRIMARY KEY, value, updated_at) -- cookie / 调度 / 采集参数 / 与 slot:HH:MM 簿记
|
settings(user_id INTEGER NOT NULL DEFAULT 0, key TEXT NOT NULL, value, updated_at,
|
||||||
users(id, username UNIQUE, password_hash, display_name, is_admin, created_at, last_login_at, login_count)
|
PRIMARY KEY (user_id, key))
|
||||||
audit_log(id, at, actor, action, detail, ip)
|
|
||||||
|
users(id, username UNIQUE, password_hash, display_name, email,
|
||||||
|
is_admin, status, register_ip, last_login_ip,
|
||||||
|
created_at, last_login_at, login_count)
|
||||||
|
|
||||||
|
captchas(id TEXT PRIMARY KEY, answer TEXT, purpose TEXT,
|
||||||
|
created_at, expires_at, used_at)
|
||||||
|
|
||||||
|
audit_log(id, user_id, at, actor, action, detail, ip)
|
||||||
```
|
```
|
||||||
|
|
||||||
索引:`day`、`(day,hour)`、`(model,day)`、`(client,day)`、`credits DESC`、`ts`。
|
**`user_id = 0` 的含义**:在 `settings` / `collect_runs` / `audit_log` 里表示
|
||||||
覆盖了「按天」「按天+时段」「模型/客户端 × 天」「单笔 TOP」「时间排序」五类热点查询。
|
**实例级**(所有账号共用,例如 `api_base`、系统迁移审计);在 `usage_records` 里
|
||||||
|
是「尚未归属」的兜底值,正常不会出现。
|
||||||
|
|
||||||
|
索引全部以 `user_id` 打头,覆盖六类热点查询:
|
||||||
|
`(user_id, day)`、`(user_id, day, hour)`、`(user_id, model, day)`、`(user_id, client, day)`、
|
||||||
|
`(user_id, credits DESC)`、`(user_id, ts)`。
|
||||||
|
|
||||||
|
### 为什么要复合主键而不是「加一列 user_id」
|
||||||
|
|
||||||
|
`usage_records` 的主键从 `request_id` 变成 `(user_id, request_id)` 是**语义**变化:
|
||||||
|
两个人可能各自命中同一个云端 `request_id`(同一台机器上的多个浏览器 profile 就会),
|
||||||
|
单列主键会让后写入的人把前一个人的记录覆盖掉。去重必须按账号做。
|
||||||
|
|
||||||
### 为什么 `day`/`hour` 是冗余列
|
### 为什么 `day`/`hour` 是冗余列
|
||||||
|
|
||||||
@@ -84,6 +112,27 @@ audit_log(id, at, actor, action, detail, ip)
|
|||||||
`settings(key='slot:2026-09-14T09:00', value='done')`。这类键用前缀 `slot:` 标记为
|
`settings(key='slot:2026-09-14T09:00', value='done')`。这类键用前缀 `slot:` 标记为
|
||||||
**内部键**:`/api/settings` 读写两侧都过滤掉(`config.is_internal_key()`),
|
**内部键**:`/api/settings` 读写两侧都过滤掉(`config.is_internal_key()`),
|
||||||
用户不会在配置页看到它们,也无法通过接口写入任意键。
|
用户不会在配置页看到它们,也无法通过接口写入任意键。
|
||||||
|
多用户下槽位标记是**按账号**的(`(user_id, 'slot:09:00')`),所以谁改了自己的时刻
|
||||||
|
只影响自己。
|
||||||
|
|
||||||
|
### 升级路径:`PRAGMA user_version`
|
||||||
|
|
||||||
|
`db.init_db()` 用 `PRAGMA user_version` 判断库结构版本(0/1 = 单用户,2 = 多用户)。
|
||||||
|
**主键变了的表不能 `ALTER`**,只能重建,顺序不能变:
|
||||||
|
|
||||||
|
```
|
||||||
|
1. 删掉所有自建索引 ← 关键,见下
|
||||||
|
2. ALTER TABLE ... RENAME TO _v1_xxx
|
||||||
|
3. ALTER TABLE 加新列(只加列的表用这个,代价小得多)
|
||||||
|
4. executescript(schema.sql) ← 此时列齐了,表与索引一次建全
|
||||||
|
5. INSERT ... SELECT 回填(user_id) → DROP TABLE _v1_xxx
|
||||||
|
6. 把历史明文凭证加密 + 写一条 schema_migrate 审计
|
||||||
|
```
|
||||||
|
|
||||||
|
> **为什么第 1 步不能省**:`ALTER TABLE RENAME` 会把索引**一起带走且名字仍被占用**,
|
||||||
|
> 于是随后的 `CREATE INDEX IF NOT EXISTS` 被静默跳过 —— 新表一个索引都没有,
|
||||||
|
> 功能看起来完全正常,查询却慢几百倍。这是最阴的一类迁移 bug。
|
||||||
|
> 第 3 步放在第 4 步之前,则是为了让引用新列的索引一次就建成功。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -240,19 +289,85 @@ records: id c(credits) m(model) cl(client) t(ts) px(prompt)
|
|||||||
| 密码存储 | `pbkdf2:sha256:200000`(Werkzeug 实现) |
|
| 密码存储 | `pbkdf2:sha256:200000`(Werkzeug 实现) |
|
||||||
| 会话 | Flask 签名 cookie `workbuddy_portal_sid`,HttpOnly + SameSite=Lax,12 小时 |
|
| 会话 | Flask 签名 cookie `workbuddy_portal_sid`,HttpOnly + SameSite=Lax,12 小时 |
|
||||||
| 密钥持久化 | `data/instance.json` 的 `secret_key`,重启不踢人 |
|
| 密钥持久化 | `data/instance.json` 的 `secret_key`,重启不踢人 |
|
||||||
| 失败限速 | 同 IP 连续 5 次失败锁定 10 分钟;计数表有上限(4096 个 IP)与 TTL(1 小时) |
|
| 失败限速 | **IP 与用户名双维度**各连续 5 次失败锁定 10 分钟;计数表有上限(8192 key)与 TTL(1 小时) |
|
||||||
|
| 验证码 | 策略 `always`(默认)/ `adaptive` / `off`;**先验码再验密** |
|
||||||
|
| 注册 | 开关 `allow_register` + 同 IP 每日配额 `register_max_per_ip` + 强制验证码 |
|
||||||
|
| 停用即失效 | `current_user()` 每请求回查 `users.status`(缓存在 `flask.g`),不等会话过期 |
|
||||||
|
| 会话固定防护 | `login_session()` 先 `session.clear()`,顺带换掉 CSRF token 与验证码 id |
|
||||||
|
|
||||||
### 授权
|
### 授权与数据隔离
|
||||||
|
|
||||||
`@login_required`(`/api/*` 未登录返回 401 JSON,页面跳登录)+
|
`@login_required`(`/api/*` 未登录返回 401 JSON,页面跳登录)+
|
||||||
`@admin_required`(403)两层。`/users` 与 `/api/users*` 全部要管理员。
|
`@admin_required`(403)两层。`/users`、`/api/users*`、`/logs/tail`、`vacuum` 要管理员。
|
||||||
|
|
||||||
内置护栏(服务端强制,前端只是提前提示):不能取消自己的管理员身份、不能删自己、至少留一个账号。
|
**多租户隔离靠「显式传参」而不是「隐式全局」**,这是本节最重要的一条设计:
|
||||||
|
|
||||||
|
```python
|
||||||
|
# 每个公开函数都把 uid 放在 conn 之后的第一个位置,且**不给默认值**
|
||||||
|
def daily(conn, uid, frm=None, to=None, with_maps=True): ...
|
||||||
|
def totals(conn, uid, frm=None, to=None): ...
|
||||||
|
collect.sync(conn, uid, trigger="manual", ...)
|
||||||
|
scheduler.slots(conn, uid=0)
|
||||||
|
```
|
||||||
|
|
||||||
|
为什么不给默认值?因为一旦写成 `uid=0`,忘记传参就会**静默返回实例级(≈全量)数据**——
|
||||||
|
这种越权不会报错、不会进日志,只会在某天被人发现。给成必填参数后,漏传是 `TypeError`,
|
||||||
|
在第一次跑测试时就炸掉。
|
||||||
|
|
||||||
|
配套的几条:
|
||||||
|
|
||||||
|
| 项 | 做法 |
|
||||||
|
|---|---|
|
||||||
|
| 单条读取也过滤 | `/api/records/<id>`、`/api/runs/<id>` 的 `WHERE` 都带 `user_id` |
|
||||||
|
| 配置作用域 | 三级回落 `个人 → 实例 → DEFAULTS`;`GLOBAL_KEYS` 只有管理员能改 |
|
||||||
|
| **凭证不回落** | `NO_FALLBACK_KEYS = {cookie, user_agent}` 跳过实例级回落 —— 否则新账号会「继承」管理员的 Cookie,这是最严重的串号越权 |
|
||||||
|
| 导出隔离 | CSV 文件名带账号名(多用户下同目录同名会互相覆盖) |
|
||||||
|
| 审计归属 | `audit_log` / `collect_runs` 都带 `user_id`;`/api/audit` 普通账号只看自己 |
|
||||||
|
|
||||||
|
内置护栏(服务端强制,前端只是提前提示):不能取消自己的管理员身份、不能停用自己、
|
||||||
|
不能删自己、至少留一个账号、至少留一个**启用状态的**管理员。
|
||||||
|
|
||||||
|
### 凭证加密与验证码
|
||||||
|
|
||||||
|
这两件事都要求「零第三方依赖」(`requirements.txt` 只有 Flask / waitress / openpyxl),
|
||||||
|
所以都是手写标准库实现。
|
||||||
|
|
||||||
|
**凭证加密(`crypto.py`)** —— 手写 ChaCha20(RFC 8439 §2.3 的块函数)+ HMAC-SHA256
|
||||||
|
**encrypt-then-MAC**,所以不存在「先解密再验签」的填充预言类问题:
|
||||||
|
|
||||||
|
```
|
||||||
|
密文 = "v1." + b64(salt) + "." + b64(nonce) + "." + b64(ciphertext) + "." + b64(tag)
|
||||||
|
子密钥 = HMAC-SHA256(master, salt, "aead-key") / ("aead-mac") ← 密钥分离
|
||||||
|
```
|
||||||
|
|
||||||
|
实现上的三个决定:
|
||||||
|
|
||||||
|
1. **`decrypt()` 失败抛异常,绝不返回原值**。返回原值看似「容错」,
|
||||||
|
实际是把「密钥不匹配」伪装成「Cookie 是个奇怪字符串」,然后把这段垃圾发给云端。
|
||||||
|
现在会明确抛 `db.SecretUnreadable`,页面提示「密文解不开,请重新粘贴」。
|
||||||
|
2. **非 `v1.` 前缀原样返回** —— 专门用来兼容单用户时代存下来的明文;
|
||||||
|
下次写入时自动升级为密文。升级迁移还会主动扫一遍并就地加密。
|
||||||
|
3. **`get_settings()` 把加密键置空**,明文的唯一出口是 `db.get_secret()`。
|
||||||
|
这比「记得别回传 cookie」可靠:写新接口的人即使 `**settings` 一把梭也带不出凭证。
|
||||||
|
|
||||||
|
**图形验证码(`captcha.py`)** —— 手写 PNG 编码器(zlib 压缩 IDAT)+ 5×7 点阵字模
|
||||||
|
+ Bresenham 干扰线 + 逐字符抖动 + 噪点:
|
||||||
|
|
||||||
|
| 决定 | 原因 |
|
||||||
|
|---|---|
|
||||||
|
| 出 PNG 而不是 SVG | SVG 是文本,答案会**明文出现在页面源码里**,等于把答案发给机器人 |
|
||||||
|
| 不用第三方 captcha/Pillow | 保持零第三方依赖;图像只由点阵矩形构成,没必要引入整个图像栈 |
|
||||||
|
| 答案不进会话 | Flask 会话是「签名 + base64,**不加密**」的(客户端可解码读明文),放答案等于送答案。只写一个随机 id |
|
||||||
|
| 先删后判 | `verify()` 先 `DELETE` 再比对,避免并发下同一张图被用两次 |
|
||||||
|
| 按 `purpose` 隔离 | 拿注册的题去登录校验必然失败 |
|
||||||
|
| 出图限速 | 60 秒 40 张 —— 出图要做点阵渲染 + zlib 压缩,不设限就是一条廉价的 CPU/带宽放大路径 |
|
||||||
|
| 登录先验码 | 否则攻击者能拿「密码对不对」当信号,在解验证码之前就把字典跑完 |
|
||||||
|
|
||||||
### CSRF
|
### CSRF
|
||||||
|
|
||||||
`before_request` 统一校验:`X-CSRF-Token` 头或 `_csrf` 表单域。
|
`before_request` 统一校验:`X-CSRF-Token` 头或 `_csrf` 表单域。
|
||||||
**退出登录也走 POST**——GET 型退出能被 `<img src="/logout">` 静默触发。
|
**退出登录也走 POST**——GET 型退出能被 `<img src="/logout">` 静默触发。
|
||||||
|
未登录的 `POST` 也会先被 CSRF 拦成 400(先拦比先鉴权更保守)。
|
||||||
|
|
||||||
### 开放重定向
|
### 开放重定向
|
||||||
|
|
||||||
@@ -298,6 +413,23 @@ page_size = db.get_int(conn, "page_size", 200) # 任何异常都回落默认
|
|||||||
|
|
||||||
采集路径上**不允许出现裸 `int(s.get(...))`**。数值还会按 `NUM_SETTINGS` 的范围再钳一次。
|
采集路径上**不允许出现裸 `int(s.get(...))`**。数值还会按 `NUM_SETTINGS` 的范围再钳一次。
|
||||||
|
|
||||||
|
### 配置的作用域:三级回落与一个例外
|
||||||
|
|
||||||
|
```
|
||||||
|
个人(user_id=n) ──没有──▶ 实例(user_id=0) ──没有──▶ config.DEFAULTS
|
||||||
|
▲
|
||||||
|
└── NO_FALLBACK_KEYS(cookie / user_agent)到此为止,不回落到实例级
|
||||||
|
```
|
||||||
|
|
||||||
|
`GLOBAL_KEYS`(`api_base` / `api_path` / `allow_register` / `register_max_per_ip` /
|
||||||
|
`captcha_policy` / `captcha_length`)在读写两侧都被强制折算到 `user_id = 0`,
|
||||||
|
所以它们天然只有一份,非管理员改不了。
|
||||||
|
|
||||||
|
有个**容易误判**的细节:`NO_FALLBACK_KEYS` 只拦住「实例级那一行」,不拦 `DEFAULTS`。
|
||||||
|
所以一个全新账号读 `user_agent` 拿到的是 `DEFAULTS` 里的**通用 Chrome UA**(非空),
|
||||||
|
而不是空串 —— 这是刻意的(首次采集总得带个 UA)。测试断言要注意:
|
||||||
|
正确的断言是「新账号的 UA ≠ 实例级那一份」,而不是「新账号的 UA 为空」。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 九、已知坑与红线
|
## 九、已知坑与红线
|
||||||
@@ -321,6 +453,20 @@ Flask 在 `full_dispatch_request()` 返回 `app_iter` **之后**就 pop 请求
|
|||||||
现在 `tools/smoke.py` 有专门一节:抓页面里所有 `src`/`href` 资源引用逐个断言 200
|
现在 `tools/smoke.py` 有专门一节:抓页面里所有 `src`/`href` 资源引用逐个断言 200
|
||||||
(断言前先剥掉 HTML 注释,否则注释里的示例路径会被误判)。
|
(断言前先剥掉 HTML 注释,否则注释里的示例路径会被误判)。
|
||||||
|
|
||||||
|
### 母模板里的 `{% set %}` 会静默覆盖子模板的同名变量
|
||||||
|
|
||||||
|
`base.html` 顶层原本写 `{% set me = current_user() %}`,用来渲染右上角的用户名。
|
||||||
|
但 `current_user()` 只回 `{id, username, display_name, is_admin}` 四个键 ——
|
||||||
|
而 `profile.html` 自己也用 `me` 接视图传来的**完整用户行**。
|
||||||
|
|
||||||
|
结果:母模板的 `set` 把子模板的 `me` 顶掉了,`me.created_at` 取不到,
|
||||||
|
个人中心渲染成「账号 admin · 注册于 · 最近登录 未登录」——**不报错、不告警**,
|
||||||
|
页面看起来只是"少了个时间"。
|
||||||
|
|
||||||
|
**规则**:母模板里给全站用的局部变量要**起专门的名字**(现在叫 `cur`),
|
||||||
|
不要复用子模板可能用到的键。`smoke.py` 里有 3 条断言盯着这件事
|
||||||
|
(`注册于` 必须是真实日期、`最近登录` 不能是空占位、`base.html` 不得再出现 `set me = `)。
|
||||||
|
|
||||||
### Jinja 里避开 `dict` 的方法名
|
### Jinja 里避开 `dict` 的方法名
|
||||||
|
|
||||||
模板中 `a.items` / `a.keys` / `a.get` / `a.values` / `a.update` / `a.pop` / `a.copy`
|
模板中 `a.items` / `a.keys` / `a.get` / `a.values` / `a.update` / `a.pop` / `a.copy`
|
||||||
@@ -367,8 +513,8 @@ GMT+8 下 `new Date("2026-08-15T00:00:00")` 的 UTC 时刻是前一天 16:00,
|
|||||||
| 层 | 手段 | 抓什么 |
|
| 层 | 手段 | 抓什么 |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| 1 | 独立聚合对账(直读 CSV 不走 `query.py`) | 口径错、少算。热力图要**逐格**比,历史上出过「同格覆盖少算 84%」 |
|
| 1 | 独立聚合对账(直读 CSV 不走 `query.py`) | 口径错、少算。热力图要**逐格**比,历史上出过「同格覆盖少算 84%」 |
|
||||||
| 2 | `tools/smoke.py`(99 项断言,离线) | 模板残留、历史缺陷防回归 ①~⑭、静态资源 404、class↔CSS 对账 |
|
| 2 | `tools/smoke.py`(**165 项断言**,离线) | 模板残留、历史缺陷防回归 ①~⑭、**多用户隔离 / 凭证保密 / 注册与验证码全链路**、静态资源 404、class↔CSS 对账 |
|
||||||
| 3 | `tools/check_live.py`(56 项断言,真实 HTTP) | `test_client` 覆盖不到的:waitress、端口、cookie 往返、开放重定向、CSRF |
|
| 3 | `tools/check_live.py`(**83 项断言**,真实 HTTP) | `test_client` 覆盖不到的:waitress、端口、cookie 往返、开放重定向、CSRF、验证码、安全响应头 |
|
||||||
| 4 | Node DOM stub + `vm.runInContext` 跑大屏真实脚本 | 「页面聚合 == 独立算出的聚合」、切区间只发一次请求 |
|
| 4 | Node DOM stub + `vm.runInContext` 跑大屏真实脚本 | 「页面聚合 == 独立算出的聚合」、切区间只发一次请求 |
|
||||||
| 5 | `tools/shots.py`(Playwright 截图 + console/pageerror) | **界面层**。本轮最有价值的 bug(大屏全白)只有它抓到 |
|
| 5 | `tools/shots.py`(Playwright 截图 + console/pageerror) | **界面层**。本轮最有价值的 bug(大屏全白)只有它抓到 |
|
||||||
|
|
||||||
@@ -379,4 +525,15 @@ python tools/check_live.py --base http://127.0.0.1:8849 # 3 层
|
|||||||
python tools/shots.py --base http://127.0.0.1:8849 --full # 5 层
|
python tools/shots.py --base http://127.0.0.1:8849 --full # 5 层
|
||||||
```
|
```
|
||||||
|
|
||||||
|
关于第 2 层在**多用户**下的两个约定:
|
||||||
|
|
||||||
|
1. `login(cli, uid)` 只注入 `uid` 不够「假装」成谁 —— `current_user()` 每请求回查 `users`
|
||||||
|
表(为的是停用立即失效),所以 `uname` / `adm` 这些会话键改不了权限。
|
||||||
|
要测非管理员行为,必须**真的**在库里有一个普通账号。
|
||||||
|
`smoke.py` 会临时建一个(随机用户名,`finally` 里删掉),并在注册链路里临时再建一个。
|
||||||
|
2. 「验证码答案没泄漏」这类断言要挑对判据:答案是 4 位随机大写串,
|
||||||
|
直接在页面里搜它只能说明「这次没撞上」。更可靠的是**结构断言** ——
|
||||||
|
会话里只有 id、库里才有答案、同 id 二次校验必失败、图必须由独立接口下发
|
||||||
|
(页面里不出现 `data:image`)。
|
||||||
|
|
||||||
**改动前先读 [九、已知坑与红线](#九已知坑与红线),改完先把第 2 层跑绿。**
|
**改动前先读 [九、已知坑与红线](#九已知坑与红线),改完先把第 2 层跑绿。**
|
||||||
|
|||||||
@@ -8,6 +8,84 @@
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## [1.2.0] — 2026-09-15
|
||||||
|
|
||||||
|
**主题:多用户化 · Cookie 加密 · 开放注册与验证码**
|
||||||
|
|
||||||
|
从单用户版升级到多用户版。**数据不会丢**:`manage.py init` 会自动检测旧表结构并迁移
|
||||||
|
(`PRAGMA user_version` 0 → 2),历史用量归到首个账号、明文 Cookie 就地加密,
|
||||||
|
全程写一条 `schema_migrate` / `encrypt_secrets` 审计,且可重复执行。
|
||||||
|
|
||||||
|
### 新增
|
||||||
|
|
||||||
|
- **多用户与数据隔离**
|
||||||
|
- `users` 表补齐 `email` / `status` / `register_ip` / `last_login_ip`;
|
||||||
|
`settings` 主键改为 `(user_id, key)`,`usage_records` 改为 `(user_id, request_id)`,
|
||||||
|
全部索引以 `user_id` 打头;`collect_runs` / `audit_log` 增加 `user_id`
|
||||||
|
- `query.py` / `collect.py` / `scheduler.py` 全链路把 `uid` 作为 `conn` 之后的
|
||||||
|
**第一个位置参数且无默认值** —— 漏传直接 `TypeError`,不会退化成「返回全量」
|
||||||
|
- 配置三级回落:`个人 → 实例(user_id=0) → config.DEFAULTS`;
|
||||||
|
新增 `NO_FALLBACK_KEYS = {cookie, user_agent}`,凭证**永不回落**(回落即串号越权)
|
||||||
|
- `scheduler.tick()` 遍历启用账号逐个判断槽位;未配 Cookie 的账号自动跳过
|
||||||
|
- 新增 `manage.py users` / `stats -u` / `status`(逐账号)/ 各子命令的 `-u/--user`
|
||||||
|
- **Cookie 静态加密**(`workbuddy_portal/crypto.py`,约 190 行,**零第三方依赖**)
|
||||||
|
- 手写 ChaCha20 块函数(RFC 8439 §2.3)+ HMAC-SHA256 **encrypt-then-MAC**,
|
||||||
|
密文格式 `v1.<b64salt>.<b64nonce>.<b64ct>.<b64tag>`,已用官方测试向量逐字节验证
|
||||||
|
- 主密钥 `cookie_key` 独立存放在 `data/instance.json`(与 `SECRET_KEY` 分开键位)
|
||||||
|
- `db.get_secret()` 是取明文的**唯一**通道;`get_settings()` 把加密键一律置空;
|
||||||
|
`db.secret_state()` 只回 `{set, chars, tail, broken}`,绝不含明文
|
||||||
|
- `decrypt()` 对非 `v1.` 前缀原样返回(兼容历史明文,下次写入自动升级),
|
||||||
|
校验失败**抛异常**而不是「失败就返回原值」;升级时自动把历史明文加密
|
||||||
|
- **开放注册**:`/register` 页 + `POST /api/users`(管理员);开关 `allow_register`、
|
||||||
|
同 IP 每日配额 `register_max_per_ip`;用户名/密码强度校验(保留字黑名单、≥8 位且两类字符)
|
||||||
|
- **图形验证码**(`workbuddy_portal/captcha.py`,约 250 行,零第三方依赖)
|
||||||
|
- **手写 PNG 编码器**(zlib 压缩 IDAT)+ 5×7 点阵字模 + Bresenham 干扰线与噪点。
|
||||||
|
刻意不用 SVG —— SVG 是文本,答案会明文出现在页面源码里
|
||||||
|
- 答案只写服务端 `captchas` 表;会话里仅存随机 id;**一次性、5 分钟过期、按用途隔离**
|
||||||
|
- 策略 `captcha_policy`:`always`(默认)/ `adaptive`(同来源失败 2 次后要求)/ `off`
|
||||||
|
- 登录**先验验证码再比口令**(否则攻击者能拿「密码对不对」当信号提前跑完字典)
|
||||||
|
- **安全加固**
|
||||||
|
- 失败限速改为 **IP + 用户名双维度**,任一超限即锁;新增验证码出图限速(60s/40 张)
|
||||||
|
- `current_user()` 每请求回查 `users.status` ⇒ 停用账号**立即**失效,不必等会话过期
|
||||||
|
- 安全响应头:CSP / `X-Frame-Options` / `nosniff` / `Referrer-Policy` / COOP;
|
||||||
|
`/api/*` 与 `/captcha*` 带 `no-store`
|
||||||
|
- 会话 cookie 显式 `HttpOnly` + `SameSite=Lax` + `Path=/`;`WB_COOKIE_SECURE=1` 可开 Secure
|
||||||
|
- `/logs/tail` 改为**仅管理员**;管理员不能停用/降权/删除自己
|
||||||
|
- **页面**:新增 `/login` 验证码、`/register`、`/profile`(个人中心,点右上角用户名进入);
|
||||||
|
`/config` 增加凭证状态与 `cookie_broken` 告警、实例级设置区;`/users` 增加邮箱/状态列与启停
|
||||||
|
|
||||||
|
### 变更
|
||||||
|
|
||||||
|
- 接口新增:`GET/POST /api/profile`、`POST /api/captcha`(机制自述)、
|
||||||
|
`POST /api/users/<id>/delete`;`GET /api/settings` 回传 `_globalKeys` / `_canEditGlobal`
|
||||||
|
- `/api/collect` 未配 Cookie 回 `409 no_cookie`,密文解不开回 `409 cookie_broken`
|
||||||
|
(不再静默当成「未配置」)
|
||||||
|
- `/records/export` 与 CLI `export-csv` 的默认文件名带账号名(多用户下同名会互相覆盖)
|
||||||
|
- `WB_COOKIE` 环境变量兜底**已移除** —— 它会导致串号
|
||||||
|
|
||||||
|
### 修复
|
||||||
|
|
||||||
|
- `db.get_db()` 在流式响应里被复用导致 `Cannot operate on a closed database`
|
||||||
|
(生成器内部改为自建连接)
|
||||||
|
- `.dockerignore` 的 `__pycache__/` 只匹配上下文根目录,嵌套目录会被打进镜像
|
||||||
|
- 注册成功提示与注册页说明里的 `**强调**` 字面量(HTML 不解析 Markdown)
|
||||||
|
- **`base.html` 顶层的 `{% set me = current_user() %}` 会覆盖子模板传入的同名变量**
|
||||||
|
—— 而 `current_user()` 只含 `id/username/display_name/is_admin`,于是个人中心把
|
||||||
|
`me.created_at` 渲染成空(「注册于 ·」)。局部变量改名 `cur`,`smoke.py` 加 3 条防回归断言
|
||||||
|
- `WB_COOKIE_SECURE` 没有写进 `docker-compose.yml` 的 `environment:`
|
||||||
|
—— 在 `.env` 里设了也不生效,文档里的开关实际是哑的(已补上,并加进 `.env.example`)
|
||||||
|
- 「配置管理 → 修改登录密码」提示写「至少 6 位」,与实际策略(≥8 位 + 两类字符)不符
|
||||||
|
- 「用户管理」删除说明写「可勾选保留」,而页面只有确认框、必删数据,措辞改为与实际一致
|
||||||
|
|
||||||
|
### 升级提示
|
||||||
|
|
||||||
|
- 纯 HTTP 局域网部署**不要**设 `WB_COOKIE_SECURE=1`,否则浏览器不回传会话 cookie,
|
||||||
|
表现为「刚登录完又被弹回登录页」
|
||||||
|
- 迁移后请到「配置管理」确认 Cookie 状态;`secret_state.broken = true` 说明
|
||||||
|
`data/instance.json` 里的 `cookie_key` 与写入时不一致,重新粘贴一次即可
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## [1.1.0] — 2026-09-14
|
## [1.1.0] — 2026-09-14
|
||||||
|
|
||||||
**主题:项目定名 `workbuddy-portal` · 容器化 · 文档体系**
|
**主题:项目定名 `workbuddy-portal` · 容器化 · 文档体系**
|
||||||
|
|||||||
@@ -62,14 +62,17 @@ docker compose logs -f
|
|||||||
| `TZ` | `Asia/Shanghai` | **影响「每日 09:00/17:00」与所有日期口径** |
|
| `TZ` | `Asia/Shanghai` | **影响「每日 09:00/17:00」与所有日期口径** |
|
||||||
| `WB_ADMIN_USER` | `admin` | 首个管理员用户名(只在库为空时生效) |
|
| `WB_ADMIN_USER` | `admin` | 首个管理员用户名(只在库为空时生效) |
|
||||||
| `WB_ADMIN_PASSWORD` | 空 | 首个管理员密码。**留空会用 `admin123`**,务必显式设置 |
|
| `WB_ADMIN_PASSWORD` | 空 | 首个管理员密码。**留空会用 `admin123`**,务必显式设置 |
|
||||||
|
| `WB_COOKIE_SECURE` | `0` | `1` = 会话 Cookie 只走 HTTPS。**纯 HTTP 部署设成 `1` 会导致「登录成功又跳回登录页」**,见 [第九节](#登录成功却立刻又跳回登录页) |
|
||||||
| `WB_DISABLE_SCHEDULER` | `0` | `1` = 不启动调度线程(只跑手动采集) |
|
| `WB_DISABLE_SCHEDULER` | `0` | `1` = 不启动调度线程(只跑手动采集) |
|
||||||
| `WB_IMPORT_CREDS` | `0` | `1` = 启动时尝试从挂载的编辑器配置导入 Cookie |
|
| `WB_IMPORT_CREDS` | `0` | `1` = 启动时尝试从挂载的编辑器配置导入 Cookie |
|
||||||
|
|
||||||
|
> `TZ` 与 `WB_COOKIE_SECURE` 都是**进程环境变量**,`docker compose restart` 不生效,要 `up -d`。
|
||||||
|
|
||||||
### 2.4 数据落点:用命名卷,不用绑定挂载
|
### 2.4 数据落点:用命名卷,不用绑定挂载
|
||||||
|
|
||||||
| 容器内 | 存放位置 | 内容 |
|
| 容器内 | 存放位置 | 内容 |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| `/app/data` | Docker 命名卷 `workbuddy-portal_wb_data` | `usage.sqlite`(正本)、`instance.json`(secret_key)、`exports/` |
|
| `/app/data` | Docker 命名卷 `workbuddy-portal_wb_data` | `usage.sqlite`(正本)、`instance.json`(`secret_key` + `cookie_key`)、`exports/` |
|
||||||
| `/app/logs` | Docker 命名卷 `workbuddy-portal_wb_logs` | `app.log`(滚动 2 MB × 3) |
|
| `/app/logs` | Docker 命名卷 `workbuddy-portal_wb_logs` | `app.log`(滚动 2 MB × 3) |
|
||||||
|
|
||||||
**为什么是命名卷而不是脚本目录里的 `./data`**(这不是随手选的):
|
**为什么是命名卷而不是脚本目录里的 `./data`**(这不是随手选的):
|
||||||
@@ -352,7 +355,7 @@ curl -s -u wangchuanli:TOKEN \
|
|||||||
|---|---|---|
|
|---|---|---|
|
||||||
| `usage.sqlite` | ★★★ | **数据正本**,丢了要重新采集,且官网窗口外的数据永久丢失 |
|
| `usage.sqlite` | ★★★ | **数据正本**,丢了要重新采集,且官网窗口外的数据永久丢失 |
|
||||||
| `usage.sqlite-wal` / `-shm` | ★★★ | WAL 模式下未 checkpoint 的数据在这里,**要一起拷** |
|
| `usage.sqlite-wal` / `-shm` | ★★★ | WAL 模式下未 checkpoint 的数据在这里,**要一起拷** |
|
||||||
| `instance.json` | ★★ | 含 `secret_key`,丢了所有人都要重新登录(数据不受影响) |
|
| `instance.json` | ★★★ | 含 `secret_key`(会话签名)**与 `cookie_key`(各账号 Cookie 的加密主密钥)**。丢了/被替换:所有人要重新登录,**且所有账号存的 Cookie 都会变成「无法解密」,需要各自重填** |
|
||||||
| `exports/*.csv` | ★ | 导出快照,可再生 |
|
| `exports/*.csv` | ★ | 导出快照,可再生 |
|
||||||
| `workbuddy-portal_wb_logs` | ☆ | 排错用,可再生 |
|
| `workbuddy-portal_wb_logs` | ☆ | 排错用,可再生 |
|
||||||
|
|
||||||
@@ -465,10 +468,34 @@ sudo systemctl restart workbuddy-portal
|
|||||||
|
|
||||||
### 升级前
|
### 升级前
|
||||||
|
|
||||||
1. **先备份**(见第六节)——`schema.sql` 用的是 `CREATE TABLE IF NOT EXISTS`,
|
1. **先备份**(见第六节)。备份要**同时包含 `usage.sqlite` 与 `instance.json`** ——
|
||||||
加表加索引是安全的,但改列需要手工迁移,所以备份是唯一保险。
|
后者存着凭证加密主密钥,只备库不备它,恢复后所有 Cookie 都要重填。
|
||||||
2. 看一眼 [CHANGELOG](CHANGELOG.md) 有没有破坏性变更。
|
2. 看一眼 [CHANGELOG](CHANGELOG.md) 有没有破坏性变更。
|
||||||
|
|
||||||
|
### 1.1.0 → 1.2.0(单用户 → 多用户)
|
||||||
|
|
||||||
|
**无需任何手工迁移命令。** 首次用新版启动时会自动完成,日志里能看到:
|
||||||
|
|
||||||
|
| 做了什么 | 效果 |
|
||||||
|
|---|---|
|
||||||
|
| 建 `users` 表、写入首个管理员 | 用 `WB_ADMIN_USER` / `WB_ADMIN_PASSWORD`,或沿用 `admin` / `admin123` |
|
||||||
|
| `settings` / `usage_records` / `collect_runs` / `audit_log` 改为 `(user_id, …)` 复合主键 | 老数据整体归到**第一个账号** |
|
||||||
|
| 明文 Cookie 就地加密 | 日志记一条 `明文凭证已加密:settings[uid=1].cookie` |
|
||||||
|
| 建 `captchas` 表、补索引 | 验证码用 |
|
||||||
|
|
||||||
|
迁移由 `PRAGMA user_version` 驱动,**幂等**:重复启动不会重复执行。
|
||||||
|
校验一下:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose logs portal | grep -i "迁移\|migrat"
|
||||||
|
docker compose exec portal python manage.py users # 账号 / 角色 / 数据量 / 凭证状态
|
||||||
|
docker compose exec portal python manage.py stats # 各账号条数与积分
|
||||||
|
```
|
||||||
|
|
||||||
|
> 升级后请**确认 Cookie 能解密**:登录后打开「配置管理 → 我的云端凭证」,
|
||||||
|
> 正常应显示「已配置 · N 字符,结尾 …xxxx」。若显示「无法解密」,说明 `instance.json`
|
||||||
|
> 不匹配,重新粘贴一次即可。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 八、日常巡检
|
## 八、日常巡检
|
||||||
@@ -505,6 +532,23 @@ docker compose logs --tail=100
|
|||||||
| 端口占用 | 改 `.env` 的 `WB_PORT`,如 `18848:8848` |
|
| 端口占用 | 改 `.env` 的 `WB_PORT`,如 `18848:8848` |
|
||||||
| 宿主机能访问、局域网不能 | `WB_BIND` 是不是被改成 `127.0.0.1` 了;防火墙有没有放行 |
|
| 宿主机能访问、局域网不能 | `WB_BIND` 是不是被改成 `127.0.0.1` 了;防火墙有没有放行 |
|
||||||
|
|
||||||
|
### 登录成功却立刻又跳回登录页
|
||||||
|
|
||||||
|
几乎一定是 `WB_COOKIE_SECURE` 被设成了 `1`,而你在用 **HTTP** 访问。
|
||||||
|
|
||||||
|
会话 Cookie 带 `Secure` 属性后,浏览器只在 HTTPS 下才回传;服务端每次都收不到会话,
|
||||||
|
就判定「未登录」,再把你送回登录页。表现是「密码明明对,页面却停在登录页」,日志里
|
||||||
|
看起来像在反复登录。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# .env
|
||||||
|
WB_COOKIE_SECURE=0
|
||||||
|
docker compose up -d # 环境变量,必须 up -d,restart 不生效
|
||||||
|
```
|
||||||
|
|
||||||
|
只有在前面真的挂了 HTTPS 反向代理、并且用域名访问时,才把它设为 `1`。
|
||||||
|
(另:如果站点前后端域名不同,还要看第四节的反代配置。)
|
||||||
|
|
||||||
### `exec format error` / `no such file or directory`(entrypoint)
|
### `exec format error` / `no such file or directory`(entrypoint)
|
||||||
|
|
||||||
`docker/entrypoint.sh` 被 CRLF 污染了。仓库里有 `.gitattributes` 强制 `*.sh` 为 LF;
|
`docker/entrypoint.sh` 被 CRLF 污染了。仓库里有 `.gitattributes` 强制 `*.sh` 为 LF;
|
||||||
@@ -599,26 +643,35 @@ echo "WB_DISABLE_SCHEDULER=1" >> .env && docker compose up -d
|
|||||||
## 十、配置项速查
|
## 十、配置项速查
|
||||||
|
|
||||||
调度与采集参数都在数据库里,**改完立即生效、不用重启**(页面「任务管理 / 配置管理」可改,
|
调度与采集参数都在数据库里,**改完立即生效、不用重启**(页面「任务管理 / 配置管理」可改,
|
||||||
也可以直接改表):
|
也可以直接改表)。表的主键是 `(user_id, key)`:`user_id=0` 表示**实例级**(所有账号共用,
|
||||||
|
仅管理员可改),其余是**个人级**(每个账号一份,互不可见)。
|
||||||
|
|
||||||
| 键 | 默认 | 说明 |
|
| 键 | 默认 | 作用域 | 说明 |
|
||||||
|---|---|---|
|
|---|---|---|---|
|
||||||
| `schedule_enabled` | `1` | 调度总开关 |
|
| `api_base` / `api_path` | 官方地址 | **实例级** | 接口地址(走镜像/代理时改) |
|
||||||
| `schedule_times` | `09:00,17:00` | 每日时刻,逗号分隔,本地时区 |
|
| `allow_register` | `1` | **实例级** | 是否开放自助注册 |
|
||||||
| `catch_up` | `1` | 启动补跑开关 |
|
| `register_max_per_ip` | `3` | **实例级** | 同一 IP 每日注册上限(1~50) |
|
||||||
| `catch_up_grace_hours` | `12` | 补跑宽限期(小时) |
|
| `captcha_policy` | `always` | **实例级** | `always` / `adaptive` / `off` |
|
||||||
| `page_size` | `200` | 采集单页条数(20~1000) |
|
| `captcha_length` | `4` | **实例级** | 验证码位数(4~6) |
|
||||||
| `rewind_minutes` | `2` | 断点回退分钟数(0~120) |
|
| `cookie` | 空 | 个人级 | 账号凭证,**密文入库**;页面只回「长度 + 结尾 4 位」 |
|
||||||
| `drift_tolerance_minutes` | `5` | 云端时间漂移告警阈值(0~720) |
|
| `user_agent` | Chrome UA | 个人级 | 与 Cookie 同源更稳。**`cookie` 与 `user_agent` 不参与实例级回落**(回落 = 串号越权) |
|
||||||
| `max_prompt` | `2048` | Prompt 入库截断长度,0 = 不截断 |
|
| `schedule_enabled` | `1` | 个人级 | 调度总开关 |
|
||||||
| `verify_days` | `0` | 采集后整日校验天数(0~90) |
|
| `schedule_times` | `09:00,17:00` | 个人级 | 每日时刻,逗号分隔,本地时区 |
|
||||||
| `timeout` | `30` | HTTP 超时秒数(5~300) |
|
| `catch_up` | `1` | 个人级 | 启动补跑开关 |
|
||||||
| `ssl_verify` | `1` | 校验云端 HTTPS 证书 |
|
| `catch_up_grace_hours` | `12` | 个人级 | 补跑宽限期(小时) |
|
||||||
| `api_base` / `api_path` | 官方地址 | 接口地址(走镜像/代理时改) |
|
| `page_size` | `200` | 个人级 | 采集单页条数(20~1000) |
|
||||||
| `cookie` | 空 | 账号凭证(页面只回掩码) |
|
| `rewind_minutes` | `2` | 个人级 | 断点回退分钟数(0~120) |
|
||||||
| `user_agent` | Chrome UA | 与 Cookie 同源更稳 |
|
| `drift_tolerance_minutes` | `5` | 个人级 | 云端时间漂移告警阈值(0~720) |
|
||||||
|
| `max_prompt` | `2048` | 个人级 | Prompt 入库截断长度,0 = 不截断 |
|
||||||
|
| `verify_days` | `0` | 个人级 | 采集后整日校验天数(0~90) |
|
||||||
|
| `timeout` | `30` | 个人级 | HTTP 超时秒数(5~300) |
|
||||||
|
| `ssl_verify` | `1` | 个人级 | 校验云端 HTTPS 证书 |
|
||||||
|
|
||||||
**写错的值会在保存时被拒绝**并给出原因,不会污染配置。
|
**读配置时有三级回落**:个人级 → 实例级 → 代码里的 `DEFAULTS`。
|
||||||
|
所以实例级的值只是「默认值」,任何账号都可以用自己的值覆盖它(`cookie` / `user_agent` 例外)。
|
||||||
|
|
||||||
|
**写错的值会在保存时被拒绝**并给出原因,不会污染配置。未知键也会被拒——
|
||||||
|
接口不能用来往 `settings` 表里塞任意键。
|
||||||
|
|
||||||
环境变量(启动期,改了要重建容器):
|
环境变量(启动期,改了要重建容器):
|
||||||
|
|
||||||
@@ -627,6 +680,7 @@ echo "WB_DISABLE_SCHEDULER=1" >> .env && docker compose up -d
|
|||||||
| `TZ` | 时区,影响所有日期口径 |
|
| `TZ` | 时区,影响所有日期口径 |
|
||||||
| `WB_HOST` / `WB_PORT` | 容器内监听地址 / 端口 |
|
| `WB_HOST` / `WB_PORT` | 容器内监听地址 / 端口 |
|
||||||
| `WB_DATA_DIR` / `WB_LOG_DIR` / `WB_DB` | 数据 / 日志 / 库文件路径覆盖 |
|
| `WB_DATA_DIR` / `WB_LOG_DIR` / `WB_DB` | 数据 / 日志 / 库文件路径覆盖 |
|
||||||
|
| `WB_COOKIE_SECURE` | `1` = 会话 Cookie 只走 HTTPS(纯 HTTP 部署必须留 `0`) |
|
||||||
| `WB_DISABLE_SCHEDULER` | `1` = 不启动调度线程 |
|
| `WB_DISABLE_SCHEDULER` | `1` = 不启动调度线程 |
|
||||||
| `WB_ADMIN_USER` / `WB_ADMIN_PASSWORD` | 首个管理员(仅库为空时生效) |
|
| `WB_ADMIN_USER` / `WB_ADMIN_PASSWORD` | 首个管理员(仅库为空时生效) |
|
||||||
| `WB_IMPORT_CREDS` / `WB_IMPORT_XLSX` | 启动时自动导入 |
|
| `WB_IMPORT_CREDS` / `WB_IMPORT_XLSX` | 启动时自动导入 |
|
||||||
|
|||||||
@@ -21,6 +21,22 @@ Error response from daemon: Ports are not available: exposing port TCP 0.0.0.0:8
|
|||||||
2. 服务器防火墙有没有放行该端口;
|
2. 服务器防火墙有没有放行该端口;
|
||||||
3. `docker compose ps` 的 `PORTS` 是不是 `0.0.0.0:8848->8848/tcp`。
|
3. `docker compose ps` 的 `PORTS` 是不是 `0.0.0.0:8848->8848/tcp`。
|
||||||
|
|
||||||
|
### Q:登录成功却立刻又跳回登录页(循环)
|
||||||
|
|
||||||
|
99% 是 `WB_COOKIE_SECURE` 被开成了 `1`,而你在用 **HTTP** 访问。
|
||||||
|
|
||||||
|
会话 Cookie 加了 `Secure` 属性后,浏览器**只在 HTTPS 下才回传它**——于是服务端每次收到
|
||||||
|
请求都看不到会话,判定未登录,再把你送回登录页。日志里看起来是「一直在登录」。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# .env 里改回 0(纯局域网 HTTP 部署的正确值),然后重建容器
|
||||||
|
WB_COOKIE_SECURE=0
|
||||||
|
docker compose up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
> 只有在前面真的挂了 HTTPS 反向代理、并且用域名访问时,才把它设为 `1`。
|
||||||
|
> 注意 `TZ`、`WB_COOKIE_SECURE` 都是**环境变量**,`docker compose restart` 不生效,要 `up -d`。
|
||||||
|
|
||||||
### Q:容器 `unhealthy` 但 `Up`
|
### Q:容器 `unhealthy` 但 `Up`
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
@@ -89,10 +105,26 @@ python -c "p='docker/entrypoint.sh';d=open(p,'rb').read();open(p,'wb').write(d.r
|
|||||||
### Q:采集报 `cookie_expired` / `unauthorized`
|
### Q:采集报 `cookie_expired` / `unauthorized`
|
||||||
|
|
||||||
Cookie 过期。重新获取(见 [用户手册 3.2](USER-GUIDE.md#32-拿-cookie-的两种办法)),
|
Cookie 过期。重新获取(见 [用户手册 3.2](USER-GUIDE.md#32-拿-cookie-的两种办法)),
|
||||||
填进「配置管理 → 凭证」,保存后按区间补采。
|
填进「配置管理 → **我的云端凭证**」,保存后按区间补采。
|
||||||
|
|
||||||
> Cookie 通常是浏览器会话级,**关掉浏览器可能就失效**。从已登录浏览器复制时勾选「保持登录」。
|
> Cookie 通常是浏览器会话级,**关掉浏览器可能就失效**。从已登录浏览器复制时勾选「保持登录」。
|
||||||
|
|
||||||
|
### Q:采集被跳过,日志写 `no_cookie`
|
||||||
|
|
||||||
|
这个账号**还没配 Cookie**。多用户下每个账号要各自配一次——系统**不会**拿别人的 Cookie
|
||||||
|
替你采集(那会把两个人的数据混在一起)。到「配置管理 → 我的云端凭证」粘贴一份即可。
|
||||||
|
|
||||||
|
### Q:日志写 `cookie_broken` / 页面显示「无法解密」
|
||||||
|
|
||||||
|
数据库里的 Cookie 密文,用当前实例主密钥解不开了。通常是 `data/instance.json`
|
||||||
|
(存着 `cookie_key`)被删、被替换,或从别的机器拷了库过来。
|
||||||
|
|
||||||
|
**动作**:重新粘贴一次该账号的 Cookie,历史数据不受影响。
|
||||||
|
**预防**:备份时把 `data/instance.json` 和数据一起备份,别在容器之间混用。
|
||||||
|
|
||||||
|
> 这是**静态加密的正确行为**——密钥换了就该解不开;如果它「解不开也照样能用」,
|
||||||
|
> 那说明根本没加密。
|
||||||
|
|
||||||
### Q:采集成功但「新增 0 条」
|
### Q:采集成功但「新增 0 条」
|
||||||
|
|
||||||
大概率正常。看那一次的 `抓取` 条数:
|
大概率正常。看那一次的 `抓取` 条数:
|
||||||
@@ -195,6 +227,39 @@ docker compose exec portal python -c "from workbuddy_portal import db; print(db.
|
|||||||
|
|
||||||
## 五、账号与权限
|
## 五、账号与权限
|
||||||
|
|
||||||
|
### Q:怎么开放/关闭自助注册
|
||||||
|
|
||||||
|
「配置管理 → 实例级设置 → 开放自助注册」(**仅管理员可见**)。
|
||||||
|
打开后登录页会出现「自助注册」链接,任何人填表即可建号;关掉后只能由管理员在
|
||||||
|
「用户管理 → 新建账号」里建。默认是**开放**。
|
||||||
|
|
||||||
|
### Q:注册被拒 / 提示来源已达上限
|
||||||
|
|
||||||
|
同一个 IP 每天默认只能注册 3 个账号(`register_max_per_ip`,范围 1 ~ 50)。
|
||||||
|
这条限制是防批量刷号用的;换个来源,或由管理员调大。
|
||||||
|
|
||||||
|
### Q:验证码一直不对
|
||||||
|
|
||||||
|
按这个顺序排查:
|
||||||
|
|
||||||
|
| 现象 | 原因 |
|
||||||
|
|---|---|
|
||||||
|
| 刚才还能用,第二次就错 | 验证码**一次性**,用一次即废;输错也要重新取图 |
|
||||||
|
| 输得慢一点就错 | 有效期 **5 分钟**,过期即失效 |
|
||||||
|
| 拿登录页的码去注册 | 两个 `purpose` 的验证码**互不通用** |
|
||||||
|
| 明明对了还是被拒 | 该来源已被锁定(连续失败 5 次 → 锁 10 分钟) |
|
||||||
|
|
||||||
|
**动作**:点验证码图片换一张重来。想少费眼力,让管理员把「验证码位数」保持在 4 位。
|
||||||
|
|
||||||
|
> 验证码答案只存在服务端 `captchas` 表:下发到浏览器的是一个随机 `captcha_id`,
|
||||||
|
> 校验后无论成败都立刻删除。所以**在网页源码里搜不到答案**,抓图也拿不到复用的码。
|
||||||
|
|
||||||
|
### Q:验证码能关掉吗
|
||||||
|
|
||||||
|
「配置管理 → 实例级设置 → 验证码策略」有 `always` / `adaptive` / `off` 三档。
|
||||||
|
`adaptive` 只在同一来源**连续失败 2 次后**才要验证码,对天天登录的人更友好。
|
||||||
|
`off` 会显著放大撞库与批量注册的风险,**只有在前面已有可信网关时才考虑**。
|
||||||
|
|
||||||
### Q:忘记管理员密码
|
### Q:忘记管理员密码
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
@@ -207,20 +272,58 @@ docker compose exec portal python manage.py passwd admin 新密码
|
|||||||
|
|
||||||
不传新密码时会用默认的 `admin123`——**别这么干**。
|
不传新密码时会用默认的 `admin123`——**别这么干**。
|
||||||
|
|
||||||
### Q:怎么给同事开只读账号
|
账号被停用了要顺便恢复启用,加 `--activate`:
|
||||||
|
|
||||||
「用户管理 → 新建账号」,**不要勾**「管理员」。
|
```bash
|
||||||
普通用户能看所有页面、能导出、能触发采集,但看不到「用户管理」且访问 `/users` 返回 403。
|
python manage.py passwd admin 新密码 --activate
|
||||||
|
```
|
||||||
|
|
||||||
|
想看现在有哪些账号、各自角色/状态/数据量/凭证状态:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python manage.py users
|
||||||
|
```
|
||||||
|
|
||||||
|
### Q:怎么给同事开账号
|
||||||
|
|
||||||
|
两种都行:
|
||||||
|
|
||||||
|
1. 让同事**自助注册**(需管理员开放注册);
|
||||||
|
2. 「用户管理 → 新建账号」,权限选**普通**(默认就是普通,管理员要显式选)。
|
||||||
|
|
||||||
|
普通用户能看概览/大屏/明细/任务/配置/日志,能改**自己的**凭证与采集参数、能触发采集与导出;
|
||||||
|
但看不到「用户管理」(访问 `/users` 返回 403),也改不了实例级设置(输入框置灰,接口也会拒)。
|
||||||
|
|
||||||
|
> 建完账号记得告诉同事:**要自己配一份自己的 Cookie**,否则采集不会跑(日志里是 `no_cookie`)。
|
||||||
|
|
||||||
|
### Q:管理员能看到别人的数据吗
|
||||||
|
|
||||||
|
**看不到。** 用户管理页只显示每个账号的记录条数与积分合计,点不进内容;
|
||||||
|
任何页面上 Cookie 都只回显「长度 + 结尾 4 位」。采集也只用本人凭证。
|
||||||
|
|
||||||
|
所以「把两个人的数据合起来看」要各自导出 CSV 再到外部合并——这是刻意的边界,不是缺陷。
|
||||||
|
|
||||||
|
### Q:误操作了别人账号 / 删错了人
|
||||||
|
|
||||||
|
- **停用**是可逆的:数据与 Cookie 都保留,随时可以再启用;
|
||||||
|
- **删除**不可逆:会连同该账号的用量数据与 Cookie 一起删。只能靠备份恢复
|
||||||
|
(见 [六、运维 · 备份](#q备份怎么做最稳))。
|
||||||
|
|
||||||
|
每次账号操作都会写 `audit_log`,在「用户管理 → 账号操作审计」里能查到谁在什么时候动的。
|
||||||
|
|
||||||
### Q:不小心把自己降级 / 删掉自己了
|
### Q:不小心把自己降级 / 删掉自己了
|
||||||
|
|
||||||
做不到。服务端有三条护栏:不能取消自己的管理员身份、不能删除自己、至少保留一个账号。
|
做不到。服务端有四条护栏:不能取消自己的管理员身份、不能停用自己、不能删除自己、
|
||||||
|
不能删掉最后一个启用的管理员。
|
||||||
|
|
||||||
### Q:所有人被踢下线了
|
### Q:所有人被踢下线了
|
||||||
|
|
||||||
`SECRET_KEY` 变了。它存在 `data/instance.json`。这个文件丢了/被删了就会重新生成,
|
`SECRET_KEY` 变了。它存在 `data/instance.json`。这个文件丢了/被删了就会重新生成,
|
||||||
所有会话失效(**数据不受影响**)。恢复办法:从备份里找回 `instance.json`,或让大家重新登录。
|
所有会话失效(**数据不受影响**)。恢复办法:从备份里找回 `instance.json`,或让大家重新登录。
|
||||||
|
|
||||||
|
> 同一个文件里还有 `cookie_key`(凭证加密主密钥),它变了会让**所有账号的 Cookie 都要重填**。
|
||||||
|
> 备份数据库时务必把 `instance.json` 一起备份。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 六、运维
|
## 六、运维
|
||||||
@@ -256,15 +359,38 @@ docker run --rm -v workbuddy-portal_wb_data:/data:ro -v "$PWD/backup":/backup \
|
|||||||
docker compose exec portal python manage.py vacuum
|
docker compose exec portal python manage.py vacuum
|
||||||
```
|
```
|
||||||
|
|
||||||
或在「配置管理 → 维护动作 → 整理数据库」点一下。
|
或在「配置管理 → 维护动作 → 整理数据库」点一下(**这个按钮仅管理员可见**,
|
||||||
|
因为它动的是整库,不只你的数据)。
|
||||||
作用是 `wal_checkpoint(TRUNCATE)` + `VACUUM`,回收删除后的空闲页并压缩 WAL。
|
作用是 `wal_checkpoint(TRUNCATE)` + `VACUUM`,回收删除后的空闲页并压缩 WAL。
|
||||||
|
|
||||||
|
### Q:从 1.1.0 升级到 1.2.0 要做什么
|
||||||
|
|
||||||
|
**手工动作:零。** 首次启动新版本时会自动迁移:
|
||||||
|
|
||||||
|
1. 老数据整体归到**第一个账号**(也就是原来的那个唯一账号);
|
||||||
|
2. 原来明文存的 Cookie **就地加密**,日志里会记一条
|
||||||
|
`明文凭证已加密:settings[uid=1].cookie`;
|
||||||
|
3. 建 `captchas` 表、给各表补 `user_id` 列与索引。
|
||||||
|
|
||||||
|
看迁移结果:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose logs portal | grep -i migrate
|
||||||
|
docker compose exec portal python manage.py users
|
||||||
|
docker compose exec portal python manage.py stats
|
||||||
|
```
|
||||||
|
|
||||||
|
**升级前务必备份**(含 `instance.json`):迁移会改主键与索引,虽然实现了回滚失败即中止,
|
||||||
|
但备份永远是第一道保险。
|
||||||
|
|
||||||
### Q:升级会不会丢数据
|
### Q:升级会不会丢数据
|
||||||
|
|
||||||
不会。数据在 Docker 命名卷 `workbuddy-portal_wb_data` 里(对应容器内 `/app/data`),
|
不会。数据在 Docker 命名卷 `workbuddy-portal_wb_data` 里(对应容器内 `/app/data`),
|
||||||
`docker compose up -d --build` 只重建容器,不碰卷。
|
`docker compose up -d --build` 只重建容器,不碰卷。
|
||||||
但**升级前依然要备份**(见上一条):`schema.sql` 用 `CREATE TABLE IF NOT EXISTS`,
|
但**升级前依然要备份**(见上一条)。
|
||||||
加表加索引安全,**改列需要手工迁移**。
|
|
||||||
|
> 迁移用 `PRAGMA user_version` 记录版本,**幂等**:重复启动不会重复迁移。
|
||||||
|
> 改列这种操作现在也由 `init_db()` 自动完成,不再是「需要手工迁移」。
|
||||||
|
|
||||||
### Q:日志在哪、怎么滚动
|
### Q:日志在哪、怎么滚动
|
||||||
|
|
||||||
@@ -276,7 +402,9 @@ docker compose exec portal python manage.py vacuum
|
|||||||
|
|
||||||
### Q:想改采集的接口地址(走镜像/代理)
|
### Q:想改采集的接口地址(走镜像/代理)
|
||||||
|
|
||||||
「配置管理」里改 `api_base` 与 `api_path`。改了之后记得同步确认 Cookie 是该域下的有效凭证。
|
「配置管理 → 采集参数」里改 `api_base` 与 `api_path`。这两个是**实例级**键,
|
||||||
|
只有管理员能改(普通账号看到的是置灰的输入框,接口层面也会拒绝)。
|
||||||
|
改了之后记得同步确认 Cookie 是该域下的有效凭证。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -287,14 +415,32 @@ docker compose exec portal python manage.py vacuum
|
|||||||
**五层,前两层必须跑绿**:
|
**五层,前两层必须跑绿**:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
python tools/smoke.py # 离线回归 99 项
|
python tools/smoke.py # 离线回归 165 项(不需要起服务)
|
||||||
python manage.py serve --port 8849 --no-scheduler # 另开终端
|
python tools/demo_data.py # 可选:造一份合成示例库
|
||||||
python tools/check_live.py --base http://127.0.0.1:8849 # 真实 HTTP 56 项
|
python manage.py serve --port 8849 --no-scheduler # 另开终端
|
||||||
python tools/shots.py --base http://127.0.0.1:8849 --full # 界面截图 + JS 报错
|
python tools/check_live.py --base http://127.0.0.1:8849 # 真实 HTTP 83 项
|
||||||
|
python tools/shots.py --base http://127.0.0.1:8849 --full # 界面截图 + JS 报错
|
||||||
```
|
```
|
||||||
|
|
||||||
|
两个新增参数值得一提:
|
||||||
|
|
||||||
|
- `check_live.py --db <路径>`:让它直接读库里的验证码答案,从而**自动过验证码**登录;
|
||||||
|
- `shots.py --db <路径>`:同上,截图脚本自动解开登录页与注册页。
|
||||||
|
|
||||||
详见 [架构说明 · 验证体系](ARCHITECTURE.md#十验证体系)。
|
详见 [架构说明 · 验证体系](ARCHITECTURE.md#十验证体系)。
|
||||||
|
|
||||||
|
### Q:改了多用户的代码,怎么确认没越权
|
||||||
|
|
||||||
|
三个低成本自查:
|
||||||
|
|
||||||
|
1. 在 `smoke.py` 的隔离小节里加一条断言 —— 调用 `query.*` 时**故意漏掉 `uid`**,
|
||||||
|
期望它抛 `TypeError`(本项目把 `uid` 设计成「`conn` 之后的第一个位置参数、无默认值」,
|
||||||
|
漏传就炸,不会静默返回全量);
|
||||||
|
2. 临时建一个普通账号,把 `WB_DISABLE_SCHEDULER` 之类放一边,直接访问 `/users` 与
|
||||||
|
`POST /api/settings` 写实例级键,都应该是 403;
|
||||||
|
3. 写一个**哨兵值**(如 `SMOKE-SENTINEL-UA`)到实例级 `user_agent`,
|
||||||
|
断言新账号读不到它——这一招能抓到「落回落到别人配置上」的越权。
|
||||||
|
|
||||||
### Q:`ModuleNotFoundError: No module named 'flask'`
|
### Q:`ModuleNotFoundError: No module named 'flask'`
|
||||||
|
|
||||||
选错解释器了。依赖装在项目的 venv 或托管环境里:
|
选错解释器了。依赖装在项目的 venv 或托管环境里:
|
||||||
|
|||||||
@@ -1,17 +1,18 @@
|
|||||||
# WorkBuddy Portal 用户使用手册
|
# WorkBuddy Portal 用户使用手册
|
||||||
|
|
||||||
> 面向**使用者**(不是开发者)。读完这份就能独立完成日常操作:
|
> 面向**使用者**(不是开发者)。读完这份就能独立完成日常操作:
|
||||||
> 登录 → 看用量 → 配置采集 → 查明细 → 导数据 → 处理常见异常。
|
> 注册 / 登录 → 配好自己的凭证 → 看用量 → 查明细 → 导数据 → 处理常见异常。
|
||||||
|
|
||||||
> **关于配图**:本文所有截图都用 `tools/demo_data.py` 生成的**合成示例数据**渲染
|
> **关于配图**:本文所有截图都用 `tools/demo_data.py` 生成的**合成示例数据**渲染
|
||||||
> ——模型名统一是 `demo-*`,客户端为 `vscode`/`webconsole`/`sdk`,Prompt 是通用示例文本。
|
> ——模型名统一是 `demo-*`,客户端为 `vscode`/`webconsole`/`sdk`,Prompt 是通用示例文本,
|
||||||
|
> Cookie 是**假串**(`wb_demo_session=…`),账号是 `admin` 与 `demo` 两个。
|
||||||
> 所以你可以照着重现出几乎一样的界面,也不必担心文档里夹带真实账号信息。
|
> 所以你可以照着重现出几乎一样的界面,也不必担心文档里夹带真实账号信息。
|
||||||
> 想自己搭一份这样的环境:`python tools/demo_data.py` 然后按输出的提示起服务即可。
|
> 想自己搭一份这样的环境:`python tools/demo_data.py` 然后按输出的提示起服务即可。
|
||||||
|
|
||||||
**目录**
|
**目录**
|
||||||
|
|
||||||
- [一、这个系统是做什么的](#一这个系统是做什么的)
|
- [一、这个系统是做什么的](#一这个系统是做什么的)
|
||||||
- [二、登录与账号](#二登录与账号)
|
- [二、登录、注册与账号](#二登录注册与账号)
|
||||||
- [三、获取并填写 Cookie](#三获取并填写-cookie)
|
- [三、获取并填写 Cookie](#三获取并填写-cookie)
|
||||||
- [四、概览页:一眼看清家底](#四概览页一眼看清家底)
|
- [四、概览页:一眼看清家底](#四概览页一眼看清家底)
|
||||||
- [五、用量大屏:交互式分析](#五用量大屏交互式分析)
|
- [五、用量大屏:交互式分析](#五用量大屏交互式分析)
|
||||||
@@ -35,7 +36,7 @@
|
|||||||
一次典型的日常是:
|
一次典型的日常是:
|
||||||
|
|
||||||
```
|
```
|
||||||
每天 09:00 / 17:00 系统自动采集(你什么都不用做)
|
每个账号按自己配的时刻自动采集(默认 09:00 / 17:00,你什么都不用做)
|
||||||
↓
|
↓
|
||||||
你想看看进度 → 打开「概览」看今天用了多少
|
你想看看进度 → 打开「概览」看今天用了多少
|
||||||
想深挖 → 打开「用量大屏」按模型/客户端/时段切
|
想深挖 → 打开「用量大屏」按模型/客户端/时段切
|
||||||
@@ -45,7 +46,9 @@
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 二、登录与账号
|
## 二、登录、注册与账号
|
||||||
|
|
||||||
|
### 2.1 登录
|
||||||
|
|
||||||
打开 `http://<部署机器IP>:8848`,会看到登录页。
|
打开 `http://<部署机器IP>:8848`,会看到登录页。
|
||||||
|
|
||||||
@@ -55,34 +58,111 @@
|
|||||||
|---|---|
|
|---|---|
|
||||||
| 默认账号 | `admin` / `admin123`(**只有数据库里一个账号都没有时**才会创建) |
|
| 默认账号 | `admin` / `admin123`(**只有数据库里一个账号都没有时**才会创建) |
|
||||||
| 登录保持 | 12 小时 |
|
| 登录保持 | 12 小时 |
|
||||||
| 失败限制 | 同一 IP 连续错 5 次,锁定 10 分钟 |
|
| 验证码 | 默认**始终要求**,4 位,不区分大小写,5 分钟内有效、只能用一次 |
|
||||||
|
| 失败限制 | 同一 IP、或同一用户名连续错 5 次,锁定 10 分钟 |
|
||||||
| 退出 | 右上角「退出」(走 POST,防被恶意链接静默触发) |
|
| 退出 | 右上角「退出」(走 POST,防被恶意链接静默触发) |
|
||||||
|
|
||||||
> ⚠️ **首次部署请立刻改密码**:系统是给局域网访问的,默认密码等于没锁门。
|
**关于验证码**:
|
||||||
> 改法:「配置管理 → 修改密码」,或命令行 `python manage.py passwd admin 新密码`。
|
|
||||||
|
|
||||||
### 权限差别
|
- 图上只有数字与大写字母,并且**去掉了容易看错的 `0 O 1 I L`**;
|
||||||
|
- 看不清就**点图片换一张**,不消耗任何额度;
|
||||||
|
- 一张验证码**用完即废**:输错要换新的,登录用过之后也不能再拿去注册;
|
||||||
|
- 答案只存在服务器数据库里,浏览器拿到的只是一个随机编号——**在网页源码里搜不到答案**;
|
||||||
|
- 被锁定期间,即使验证码填对也会被拒,等 10 分钟或换一个来源。
|
||||||
|
|
||||||
|
> ⚠️ **首次部署请立刻改密码**:系统是给局域网访问的,默认密码等于没锁门。
|
||||||
|
> 改法:「个人中心 → 修改登录密码」,或命令行 `python manage.py passwd admin 新密码`。
|
||||||
|
|
||||||
|
### 2.2 自助注册
|
||||||
|
|
||||||
|
登录页底部有「**自助注册**」入口(地址是 `/register`)。管理员也可以把这个入口关掉。
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
| 字段 | 要求 |
|
||||||
|
|---|---|
|
||||||
|
| 用户名 | 3~32 位,字母或数字开头,可含 `_` `.` `-`;**这是登录名,注册后不可改** |
|
||||||
|
| 显示名 | 选填,留空则与用户名相同 |
|
||||||
|
| 邮箱 | 选填,便于日后找回 |
|
||||||
|
| 密码 | 至少 8 位,且含大写字母 / 小写字母 / 数字 / 符号中的**至少两类** |
|
||||||
|
| 验证码 | 与登录页同款:5 分钟有效、一次性 |
|
||||||
|
|
||||||
|
批量注册被三道闸门挡着:
|
||||||
|
|
||||||
|
1. **图形验证码** —— 每次提交都要重新过一遍;
|
||||||
|
2. **来源限额** —— 同一个 IP 每天最多注册 3 个账号(管理员可调);
|
||||||
|
3. **总开关** —— 管理员可以随时关闭注册入口。
|
||||||
|
|
||||||
|
> 注册成功后**不会**自动帮你配好采集。你要粘贴的是**你自己账号**的 Cookie,
|
||||||
|
> 见 [第三章](#三获取并填写-cookie)。在那之前,概览页只会提示「未配置凭证」。
|
||||||
|
|
||||||
|
### 2.3 个人中心
|
||||||
|
|
||||||
|
点右上角**你自己的名字**,进入个人中心(`/profile`)。
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
| 区块 | 能做什么 |
|
||||||
|
|---|---|
|
||||||
|
| 四张卡片 | 我的记录数 / 我的积分 / 采集次数 / 我的 Cookie 状态 |
|
||||||
|
| 修改资料 | 改显示名、邮箱(用户名只读) |
|
||||||
|
| 修改登录密码 | 需要原密码;改完当前会话仍然有效 |
|
||||||
|
| 我的采集凭证 | 是否已配置、多少字符、结尾 4 位、最后更新时间、当前调度时刻 |
|
||||||
|
|
||||||
|
> 卡片上的「我的积分」只统计**归属你本人的数据**,别人账号的记录不会算进来。
|
||||||
|
|
||||||
|
### 2.4 你的数据边界
|
||||||
|
|
||||||
|
这是多用户版最要紧的一条:**每个账号只看得到、也只影响自己的数据。**
|
||||||
|
|
||||||
|
| 是「你的」 | 是「共用的」 |
|
||||||
|
|---|---|
|
||||||
|
| Cookie 与 User-Agent | 接口基址 / 接口路径 |
|
||||||
|
| 采集参数(分页、超时、截断…) | 是否开放自助注册、注册限额 |
|
||||||
|
| 调度开关与每日时刻 | 验证码策略与位数 |
|
||||||
|
| 用量记录、采集历史、导出的 CSV | 数据库文件本身 |
|
||||||
|
|
||||||
|
两点值得记牢:
|
||||||
|
|
||||||
|
- **管理员也看不到你的 Cookie 和用量明细。** 用户管理页只显示每个账号的记录条数与积分合计,
|
||||||
|
点不进去看内容;Cookie 在页面上永远只回显「长度 + 结尾 4 位」。
|
||||||
|
- **采集只使用本人的凭证。** 系统不会拿别人的 Cookie 去替你采集(那会串号),
|
||||||
|
所以每个账号都必须各自配一次 Cookie。
|
||||||
|
|
||||||
|
### 2.5 权限差别
|
||||||
|
|
||||||
| 能力 | 管理员 | 普通用户 |
|
| 能力 | 管理员 | 普通用户 |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| 看概览 / 大屏 / 明细 / 任务 / 配置 / 日志 | ✅ | ✅ |
|
| 概览 / 大屏 / 明细 / 任务 / 配置 / 日志 / 个人中心 | ✅ | ✅ |
|
||||||
| 手动触发采集、补采、改配置 | ✅ | ✅ |
|
| 改**自己**的采集参数、调度时刻、Cookie | ✅ | ✅ |
|
||||||
| 导出 CSV | ✅ | ✅ |
|
| 手动采集、按区间补采 | ✅(只动自己的数据) | ✅(只动自己的数据) |
|
||||||
| **用户管理**(建号 / 改权限 / 删号) | ✅ | ❌(导航里不显示,直接访问返回 403) |
|
| 导出 CSV | ✅(只有自己的) | ✅(只有自己的) |
|
||||||
|
| 整理数据库(VACUUM,整库操作) | ✅ | ❌ |
|
||||||
|
| 改**实例级**设置(接口地址、开放注册、验证码策略、注册限额) | ✅ | ❌(输入框置灰) |
|
||||||
|
| 应用日志尾部 | ✅ | ❌(接口 403,页面上该区块为空) |
|
||||||
|
| **用户管理**(建号 / 停用 / 删号 / 改权限) | ✅ | ❌(导航里不显示,直接访问返回 403) |
|
||||||
|
|
||||||
> 给只读同事发普通账号即可,没必要共用管理员。
|
> 给同事发普通账号即可,没必要共用管理员——管理员是能停用别人账号的角色。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 三、获取并填写 Cookie
|
## 三、获取并填写 Cookie
|
||||||
|
|
||||||
**没有 Cookie,采集一定失败。** 这是首次部署唯一的必要手工步骤。
|
**没有 Cookie,采集一定失败。** 这是每个账号**各自**要做一次的手工步骤。
|
||||||
|
|
||||||
### 3.1 为什么要 Cookie
|
### 3.1 为什么要 Cookie,以及它怎么被保管
|
||||||
|
|
||||||
采集是直接调账号的用量接口,云端用 Cookie 认人。Cookie 是账号凭证,所以它:
|
采集是直接调账号的用量接口,云端靠 Cookie 认人。Cookie 等于账号凭证,所以系统对它:
|
||||||
- 存在数据库里,页面上**只回显掩码**(如 `a1b2…f9`);
|
|
||||||
- 不会被任何接口以明文返回。
|
- **加密后入库**:落库前用 ChaCha20 + HMAC-SHA256 加密(密钥在 `data/instance.json`),
|
||||||
|
数据库文件被拷走也读不出明文;
|
||||||
|
- **永不回传明文**:页面与接口只回显「多少字符、结尾 4 位」,形如 `1238 字符,结尾 …c0ffe`;
|
||||||
|
- **只属于你**:存在你的账号名下,别人(包括管理员)看不到、也拿不到;
|
||||||
|
- **和 User-Agent 绑在一起**:两者必须取自**同一次浏览器请求**,否则云端会认为是另一个客户端。
|
||||||
|
|
||||||
|
> 「配置管理 → 我的云端凭证」里如果出现 **无法解密** 的红字提示,说明实例主密钥被换过
|
||||||
|
> (`data/instance.json` 被删或被替换),重新粘贴一次即可。详见
|
||||||
|
> [十二、常见问题](#cookie-显示无法解密)。
|
||||||
|
|
||||||
### 3.2 拿 Cookie 的两种办法
|
### 3.2 拿 Cookie 的两种办法
|
||||||
|
|
||||||
@@ -91,12 +171,17 @@
|
|||||||
如果你平时用 VSCode / Cursor / Trae 登录过 WorkBuddy,Cookie 已经在本机设置里:
|
如果你平时用 VSCode / Cursor / Trae 登录过 WorkBuddy,Cookie 已经在本机设置里:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
python manage.py import-creds
|
python manage.py import-creds # 不指定 -u 时给「管理员」账号导入
|
||||||
|
python manage.py import-creds -u alice # 想导给谁就写谁的用户名
|
||||||
```
|
```
|
||||||
|
|
||||||
它会去读编辑器 `settings.json` 里的 `codebuddyUsage.*` 字段,写进数据库。
|
它会去读编辑器 `settings.json` 里的 `codebuddyUsage.*` 字段,写进数据库。
|
||||||
Docker 部署时对应 `WB_IMPORT_CREDS=1`(需要把设置文件挂进容器)。
|
Docker 部署时对应 `WB_IMPORT_CREDS=1`(需要把设置文件挂进容器)。
|
||||||
|
|
||||||
|
> ⚠️ 导入的是**运行这条命令的那台机器上、那个编辑器账号**的 Cookie。
|
||||||
|
> 如果 A 同事的机器上跑这条命令去给 B 同事的账号导入,采到的就是 A 的数据——
|
||||||
|
> 所以更稳的做法是让每个人自己登录网页、粘贴自己的 Cookie。
|
||||||
|
|
||||||
**办法 B:手工复制(一定可行)**
|
**办法 B:手工复制(一定可行)**
|
||||||
|
|
||||||
1. 浏览器打开并登录 WorkBuddy 官网;
|
1. 浏览器打开并登录 WorkBuddy 官网;
|
||||||
@@ -116,6 +201,8 @@ Docker 部署时对应 `WB_IMPORT_CREDS=1`(需要把设置文件挂进容器
|
|||||||
|---|---|---|
|
|---|---|---|
|
||||||
| `新增 N 条` 或 `无新增(已是最新)` | ✅ 正常 | — |
|
| `新增 N 条` 或 `无新增(已是最新)` | ✅ 正常 | — |
|
||||||
| `cookie_expired` / `401` / `403` | Cookie 过期了 | 重新执行 3.2 |
|
| `cookie_expired` / `401` / `403` | Cookie 过期了 | 重新执行 3.2 |
|
||||||
|
| `no_cookie`(采集被跳过) | 这个账号**还没配** Cookie | 按 3.2 填一份 |
|
||||||
|
| `cookie_broken` | 密文解不开(实例主密钥被换过) | 重新粘贴一次,见 [十二](#cookie-显示无法解密) |
|
||||||
| `TLS` / `SSLError` | 证书校验失败 | 见「十二、常见问题」 |
|
| `TLS` / `SSLError` | 证书校验失败 | 见「十二、常见问题」 |
|
||||||
|
|
||||||
---
|
---
|
||||||
@@ -228,6 +315,10 @@ Docker 部署时对应 `WB_IMPORT_CREDS=1`(需要把设置文件挂进容器
|
|||||||
> 调度线程在 Web 进程内,所以「关掉 Web」等于「关掉调度」。
|
> 调度线程在 Web 进程内,所以「关掉 Web」等于「关掉调度」。
|
||||||
> 如果偶尔忘了开机,靠「启动补跑」把错过的时刻补回来。
|
> 如果偶尔忘了开机,靠「启动补跑」把错过的时刻补回来。
|
||||||
|
|
||||||
|
> **调度是按账号配置的**:你在这里改开关与时刻,只影响**你自己**的采集。
|
||||||
|
> 到达时刻时,系统会逐个账号跑——没配 Cookie 的账号会被跳过并在日志里记一条
|
||||||
|
> `no_cookie`,不会影响别人。别人也可以在别的时间点采,互不干扰。
|
||||||
|
|
||||||
### 7.2 手动采集
|
### 7.2 手动采集
|
||||||
|
|
||||||
- **立即采集一次**:按断点续采,最常用的按钮。
|
- **立即采集一次**:按断点续采,最常用的按钮。
|
||||||
@@ -249,19 +340,23 @@ Docker 部署时对应 `WB_IMPORT_CREDS=1`(需要把设置文件挂进容器
|
|||||||
|
|
||||||

|

|
||||||
|
|
||||||
### 8.1 凭证
|
### 8.1 我的云端凭证
|
||||||
|
|
||||||
| 字段 | 说明 |
|
| 字段 | 说明 |
|
||||||
|---|---|
|
|---|---|
|
||||||
| Cookie | 采集用的账号凭证。**只回显掩码**;留空保存 = 不修改(不会被清空) |
|
| Cookie | **你本人账号**的凭证,密文入库。留空保存 = 不修改;填一个 `-` = 清空已保存的 Cookie |
|
||||||
| User-Agent | 与拿 Cookie 的浏览器保持一致更稳 |
|
| User-Agent | 与拿 Cookie 的浏览器保持一致更稳(**两者必须取自同一次请求**) |
|
||||||
|
|
||||||
|
保存后页面上只显示 `当前 Cookie:1238 字符,结尾 …c0ffe(2026-09-15 10:22 更新)`——
|
||||||
|
页面上、接口里都拿不到明文。若显示「**无法解密**」的红字横幅,说明实例主密钥被换过,
|
||||||
|
重新粘贴一次即可。
|
||||||
|
|
||||||
### 8.2 采集参数
|
### 8.2 采集参数
|
||||||
|
|
||||||
| 参数 | 默认 | 范围 | 说明 |
|
| 参数 | 默认 | 范围 | 说明 |
|
||||||
|---|---|---|---|
|
|---|---|---|---|
|
||||||
| `api_base` | `https://www.workbuddy.cn` | — | 接口基址(镜像 / 代理时改) |
|
| `api_base` | `https://www.workbuddy.cn` | — | 接口基址(镜像 / 代理时改)。**实例级,普通账号只读** |
|
||||||
| `api_path` | `/billing/meter/get-user-request-usage` | — | 接口路径 |
|
| `api_path` | `/billing/meter/get-user-request-usage` | — | 接口路径。**实例级,普通账号只读** |
|
||||||
| `page_size` | 200 | 20 ~ 1000 | 单页条数。调大能减少请求次数,但单次更慢 |
|
| `page_size` | 200 | 20 ~ 1000 | 单页条数。调大能减少请求次数,但单次更慢 |
|
||||||
| `rewind_minutes` | 2 | 0 ~ 120 | 断点回退分钟数。避免云端写入延迟导致漏数据 |
|
| `rewind_minutes` | 2 | 0 ~ 120 | 断点回退分钟数。避免云端写入延迟导致漏数据 |
|
||||||
| `drift_tolerance_minutes` | 5 | 0 ~ 720 | 云端时间比本地早超过该值才告警 |
|
| `drift_tolerance_minutes` | 5 | 0 ~ 720 | 云端时间比本地早超过该值才告警 |
|
||||||
@@ -273,19 +368,40 @@ Docker 部署时对应 `WB_IMPORT_CREDS=1`(需要把设置文件挂进容器
|
|||||||
> **写错的值会被当场拒绝**并提示原因,不会污染配置(历史版本会因为一个手滑的数字
|
> **写错的值会被当场拒绝**并提示原因,不会污染配置(历史版本会因为一个手滑的数字
|
||||||
> 让采集整个跑不起来)。范围外的数、非数字都会在保存时被拦下。
|
> 让采集整个跑不起来)。范围外的数、非数字都会在保存时被拦下。
|
||||||
|
|
||||||
|
> 上表里除 `api_base` / `api_path` 外,**其余都是「你自己的」配置**——改它只影响你这个账号的
|
||||||
|
> 采集行为,不影响别人。`schedule_times` 这类调度项同理:每个人可以定自己的采集时刻。
|
||||||
|
|
||||||
### 8.3 维护动作
|
### 8.3 维护动作
|
||||||
|
|
||||||
| 按钮 | 作用 | 何时用 |
|
| 按钮 | 作用 | 何时用 | 谁能用 |
|
||||||
|---|---|---|
|
|---|---|---|---|
|
||||||
| 补全 Prompt | 把缺失的 `Prompt` 从云端回补 | 从官网 xlsx 导入过数据后(xlsx 丢约 22%) |
|
| 补全 Prompt | 把缺失的 `Prompt` 从云端回补 | 从官网 xlsx 导入过数据后(xlsx 丢约 22%) | 所有人(只补自己的) |
|
||||||
| 导出全量 CSV | 全量导出到 `data/exports/` | 归档 / 交接 |
|
| 导出我的 CSV | 导出**你自己的**全量数据到 `data/exports/` | 归档 / 交接 | 所有人 |
|
||||||
| 整理数据库 | `wal_checkpoint` + `VACUUM` | 删过数据后回收空间,或 WAL 文件偏大时 |
|
| 整理数据库 | `wal_checkpoint` + `VACUUM`(**整库操作**) | 删过数据后回收空间,或 WAL 文件偏大时 | **仅管理员** |
|
||||||
|
|
||||||
这些动作**耗时且会占用写权限**,所以有二次确认。执行期间不要重复点击。
|
这些动作**耗时且会占用写权限**,所以有二次确认。执行期间不要重复点击。
|
||||||
|
|
||||||
### 8.4 修改密码
|
### 8.4 修改密码
|
||||||
|
|
||||||
填「当前密码 / 新密码 / 确认新密码」。改完当前会话仍然有效,其他会话需要重新登录。
|
填「当前密码 / 新密码 / 确认新密码」。改完当前会话仍然有效,其他会话需要重新登录。
|
||||||
|
(同样的表单在「个人中心」也有一份。)
|
||||||
|
|
||||||
|
### 8.5 实例级设置(仅管理员可见)
|
||||||
|
|
||||||
|
页面最下方这一块,**只有管理员看得到**,改动对**所有账号**生效:
|
||||||
|
|
||||||
|
| 设置 | 默认 | 说明 |
|
||||||
|
|---|---|---|
|
||||||
|
| 开放自助注册 | 允许 | 关掉后登录页不再显示「自助注册」,只能由管理员建号 |
|
||||||
|
| 同 IP 每日注册上限 | 3 | 防止一个来源批量刷号;范围 1 ~ 50 |
|
||||||
|
| 验证码策略 | 始终要求 | `始终要求` / `仅连续失败 2 次后要求` / `关闭` |
|
||||||
|
| 验证码位数 | 4 | 4 ~ 6 位。位数越多越难被自动识别,也越考验眼力 |
|
||||||
|
|
||||||
|
> **验证码策略怎么选**:默认的「始终要求」最安全;「仅连续失败后要求」对天天登录的人更友好,
|
||||||
|
> 但会给机器人留出 2 次免验证码的尝试机会。**「关闭」只有在前面已经有可信网关时才考虑。**
|
||||||
|
>
|
||||||
|
> 验证码的答案只存在服务端 `captchas` 表里,5 分钟过期、用一次就删——
|
||||||
|
> 所以它不会随会话 Cookie 泄漏出去。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -302,11 +418,14 @@ Docker 部署时对应 `WB_IMPORT_CREDS=1`(需要把设置文件挂进容器
|
|||||||
Web 进程自身的日志(启动、异常栈、调度动作)。默认展示尾部若干行。
|
Web 进程自身的日志(启动、异常栈、调度动作)。默认展示尾部若干行。
|
||||||
|
|
||||||
**3. 操作审计**
|
**3. 操作审计**
|
||||||
谁在什么时候做了什么:登录、登录失败、改配置、触发采集、导出、建号删号……
|
谁在什么时候做了什么:登录、登录失败、改配置、触发采集、导出、建号删号、注册……
|
||||||
可按**动作**筛选,支持翻页。
|
可按**动作**筛选,支持翻页。
|
||||||
|
|
||||||
> 排错顺序建议:操作审计(有没有人动过) → 采集历史(采集本身成不成功) → 应用日志(程序有没有异常)。
|
> 排错顺序建议:操作审计(有没有人动过) → 采集历史(采集本身成不成功) → 应用日志(程序有没有异常)。
|
||||||
|
|
||||||
|
> **多用户下你看到的范围**:「采集运行历史」与「操作审计」只有你**自己的**记录;
|
||||||
|
> 「应用日志尾部」是整机日志,**仅管理员可见**(普通账号看到的是空区块,接口返回 403)。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 十、用户管理页(仅管理员)
|
## 十、用户管理页(仅管理员)
|
||||||
@@ -315,17 +434,25 @@ Web 进程自身的日志(启动、异常栈、调度动作)。默认展示
|
|||||||
|
|
||||||
| 操作 | 说明 |
|
| 操作 | 说明 |
|
||||||
|---|---|
|
|---|---|
|
||||||
| 新建账号 | 填用户名 / 显示名 / 密码,可勾选管理员 |
|
| 新建账号 | 填用户名 / 显示名 / 邮箱 / 密码 / 权限(**默认普通账号**) |
|
||||||
| 改显示名 | 行内直接改,保存即生效 |
|
| 改显示名 | 行内直接改,点该行「保存」生效 |
|
||||||
| 改权限 | 管理员 ↔ 普通用户 |
|
| 改权限 | 管理员 ↔ 普通 |
|
||||||
|
| 改状态 | 启用 ↔ 停用(**停用立即生效**,不必等会话过期) |
|
||||||
| 改密码 | 给忘了密码的同事重置 |
|
| 改密码 | 给忘了密码的同事重置 |
|
||||||
| 删除 | 删除账号 |
|
| 删除 | **不可逆**,会连同该账号的用量数据与 Cookie 一起删除 |
|
||||||
|
|
||||||
内置三条护栏(前端和后端都拦):
|
列表还给出每个账号的**记录条数 / 积分合计 / 最后登录时间与 IP**——但**看不到内容**:
|
||||||
|
管理员能看到的只是「有多少」,看不到「是什么」,也看不到任何人的 Cookie。
|
||||||
|
|
||||||
|
内置四条护栏(前端置灰 + 后端再拦一次):
|
||||||
|
|
||||||
1. **不能取消自己的管理员身份**(防止把自己锁在门外);
|
1. **不能取消自己的管理员身份**(防止把自己锁在门外);
|
||||||
2. **不能删除自己**;
|
2. **不能停用自己**;
|
||||||
3. **至少要保留一个账号**(防止系统变成没人能登录)。
|
3. **不能删除自己**;
|
||||||
|
4. **不能删掉最后一个启用的管理员**(防止系统变成没人能管)。
|
||||||
|
|
||||||
|
页面底部是「**账号操作审计**」:最近 20 条账号相关动作,含**注册**与**登录失败**记录——
|
||||||
|
想知道有没有人在撞你的密码,看这里。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -333,24 +460,74 @@ Web 进程自身的日志(启动、异常栈、调度动作)。默认展示
|
|||||||
|
|
||||||
| 我想… | 怎么做 |
|
| 我想… | 怎么做 |
|
||||||
|---|---|
|
|---|---|
|
||||||
| 立刻采集一次 | 任务管理 → 立即采集一次 |
|
| 自己注册一个账号 | 登录页 → **自助注册**(需管理员开放注册) |
|
||||||
|
| 立刻采集一次 | 任务管理 → 立即采集一次(只采我自己的) |
|
||||||
| 回补某几天的数据 | 任务管理 → 按区间补采,填起止日期 |
|
| 回补某几天的数据 | 任务管理 → 按区间补采,填起止日期 |
|
||||||
| 换 Cookie | 配置管理 → 凭证 → 粘贴新 Cookie → 保存 → 回补最近几天 |
|
| 换我自己的 Cookie | 配置管理 → 我的云端凭证 → 粘贴新 Cookie → 保存 → 回补最近几天 |
|
||||||
|
| 改我的显示名 / 邮箱 / 密码 | 右上角**点自己的名字** → 个人中心 |
|
||||||
|
| 看不清验证码 | **点验证码图片**换一张 |
|
||||||
| 导出某段时间的数据给别人 | 数据明细 → 选日期 → 导出 CSV |
|
| 导出某段时间的数据给别人 | 数据明细 → 选日期 → 导出 CSV |
|
||||||
| 导出全量存档 | 配置管理 → 维护动作 → 导出全量 CSV |
|
| 导出我的全量存档 | 配置管理 → 维护动作 → 导出我的 CSV |
|
||||||
| 找出最贵的请求 | 用量大屏 → 单笔 TOP |
|
| 找出最贵的请求 | 用量大屏 → 单笔 TOP |
|
||||||
| 看某条请求的完整 Prompt | 数据明细 → 该行「展开」 |
|
| 看某条请求的完整 Prompt | 数据明细 → 该行「展开」 |
|
||||||
| 给同事开只读账号 | 用户管理 → 新建账号,**不勾**管理员 |
|
| 给同事开账号 | 用户管理 → 新建账号,权限选**普通**(或让同事自助注册) |
|
||||||
| 同事忘记密码 | 用户管理 → 该行「改密码」 |
|
| 同事忘记密码 | 用户管理 → 该行「改密」 |
|
||||||
| 把数据备份走 | 让运维按 [部署指南 6.2](DEPLOYMENT.md#62-备份) 备份命名卷,或在「配置管理」导出全量 CSV |
|
| 临时封掉某个账号 | 用户管理 → 该行「停用」(数据与 Cookie 保留) |
|
||||||
|
| 拒绝别人自助注册 | 配置管理 → 实例级设置 → 开放自助注册 → 关闭 |
|
||||||
|
| 把数据备份走 | 让运维按 [部署指南 6.2](DEPLOYMENT.md#62-备份) 备份命名卷,或在「配置管理」导出 CSV |
|
||||||
| 关掉自动采集 | 任务管理 → 关「启用调度」 |
|
| 关掉自动采集 | 任务管理 → 关「启用调度」 |
|
||||||
| 改采集时刻 | 任务管理 → 每日时刻,如 `08:30,12:30,18:00` → 保存 |
|
| 改采集时刻 | 任务管理 → 每日时刻,如 `08:30,12:30,18:00` → 保存 |
|
||||||
| 系统变慢了 | 配置管理 → 整理数据库;再不行看「十二」 |
|
| 系统变慢了 | 让**管理员**做「配置管理 → 整理数据库」;再不行看「十二」 |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 十二、常见问题
|
## 十二、常见问题
|
||||||
|
|
||||||
|
### 验证码看不清
|
||||||
|
|
||||||
|
点验证码图片**换一张**,不限次数、不消耗额度。图上刻意去掉了 `0 O 1 I L` 这几个易混字符,
|
||||||
|
只剩数字与不含它们的字母。也可以让管理员把「验证码位数」调成 4 位。
|
||||||
|
|
||||||
|
### 验证码明明填对了,还是提示错误
|
||||||
|
|
||||||
|
三种可能,按顺序排查:
|
||||||
|
|
||||||
|
1. **这张图已经用过了** —— 验证码是**一次性**的,输错一次、或登录成功之后,它立刻作废,
|
||||||
|
必须点图片重新取一张;
|
||||||
|
2. **超过了 5 分钟** —— 有效期只有 5 分钟,慢慢来的话会过期,换一张即可;
|
||||||
|
3. **跨了页面** —— 登录页取到的图不能拿去注册页用(两边的验证码是分开的)。
|
||||||
|
|
||||||
|
> 另外:如果这个来源已被锁定(连续失败 5 次),即使验证码正确也会被拒;等 10 分钟再试。
|
||||||
|
|
||||||
|
### 登录页看不到「自助注册」
|
||||||
|
|
||||||
|
说明管理员把注册关掉了。两条路:请管理员在「配置管理 → 实例级设置」里打开,
|
||||||
|
或直接请管理员在「用户管理」里给你建一个账号。
|
||||||
|
|
||||||
|
### 注册被拒,说来源已达上限
|
||||||
|
|
||||||
|
同一个 IP 每天默认最多注册 3 个账号。换个网络,或请管理员把
|
||||||
|
「同 IP 每日注册上限」调大(范围 1 ~ 50)。
|
||||||
|
|
||||||
|
### 「Cookie 显示无法解密」
|
||||||
|
|
||||||
|
「配置管理 → 我的云端凭证」出现红字横幅,或卡片上写着 **无法解密**:这是说数据库里
|
||||||
|
存的 Cookie 密文,用当前的实例主密钥解不开了。常见原因是 `data/instance.json`
|
||||||
|
(里面存着 `cookie_key`)被删除、被替换,或者从别的机器拷了一份数据库过来。
|
||||||
|
|
||||||
|
**影响**:这个账号的采集会失败,日志里是 `cookie_broken`。
|
||||||
|
|
||||||
|
**怎么办**:重新粘贴一次这个账号的 Cookie 即可,历史数据不受影响。
|
||||||
|
**怎么避免**:`data/instance.json` 里存着会话签名密钥和加密主密钥——备份数据库时
|
||||||
|
**把它一起备份**,并且不要在容器之间混用。
|
||||||
|
|
||||||
|
### 我能不能看别人的用量
|
||||||
|
|
||||||
|
不能,管理员也不能。「用户管理」页只显示每个账号的记录条数与积分合计,看不到内容。
|
||||||
|
这是设计如此:Cookie 是账号级凭证,让它跨账号可见等于把别人的账号交出去。
|
||||||
|
|
||||||
|
如果确实需要合并统计,正确做法是让每个人各自导出 CSV,再在外部合并。
|
||||||
|
|
||||||
### 采集报 `cookie_expired` / `unauthorized`
|
### 采集报 `cookie_expired` / `unauthorized`
|
||||||
|
|
||||||
Cookie 过期。重新按 [3.2](#32-拿-cookie-的两种办法) 拿一份新 Cookie 填进去。
|
Cookie 过期。重新按 [3.2](#32-拿-cookie-的两种办法) 拿一份新 Cookie 填进去。
|
||||||
@@ -391,13 +568,27 @@ Docker 部署请确认 `TZ=Asia/Shanghai`;裸机部署确认系统时区。
|
|||||||
在的。调度在**服务端进程**里,和浏览器无关。要停就去「任务管理」关调度开关,
|
在的。调度在**服务端进程**里,和浏览器无关。要停就去「任务管理」关调度开关,
|
||||||
或停掉服务。
|
或停掉服务。
|
||||||
|
|
||||||
### 忘记管理员密码
|
### 忘记密码
|
||||||
|
|
||||||
在部署机器上执行:
|
**你自己的密码忘了**:网页上没法自助重置(没有邮件通道),找管理员在
|
||||||
|
「用户管理 → 该行『改密』」给你设一个新的。
|
||||||
|
|
||||||
|
**管理员密码忘了**(或者被自己停用了),到部署机器上执行:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
python manage.py passwd admin 新密码 # 裸机
|
python manage.py passwd admin 新密码 # 裸机
|
||||||
docker compose exec portal python manage.py passwd admin 新密码 # Docker
|
docker compose exec portal python manage.py passwd admin 新密码 # Docker
|
||||||
|
|
||||||
|
# 顺便把被停用的账号恢复启用
|
||||||
|
python manage.py passwd admin 新密码 --activate
|
||||||
|
# 需要新建一个管理员
|
||||||
|
python manage.py passwd alice 密码 --role admin
|
||||||
|
```
|
||||||
|
|
||||||
|
想看现在都有哪些账号、各自什么角色与状态,用:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python manage.py users
|
||||||
```
|
```
|
||||||
|
|
||||||
### 数据会丢吗
|
### 数据会丢吗
|
||||||
@@ -417,4 +608,4 @@ docker compose exec portal python manage.py passwd admin 新密码 # Docker
|
|||||||
---
|
---
|
||||||
|
|
||||||
更多技术细节见 [架构与设计说明](ARCHITECTURE.md)、[部署与运维指南](DEPLOYMENT.md)、
|
更多技术细节见 [架构与设计说明](ARCHITECTURE.md)、[部署与运维指南](DEPLOYMENT.md)、
|
||||||
[接口参考](API.md)。
|
[接口参考](API.md)、[安全说明](../SECURITY.md)。
|
||||||
|
|||||||
|
之前 宽度: | 高度: | 大小: 93 KiB 之后 宽度: | 高度: | 大小: 336 KiB |
|
之前 宽度: | 高度: | 大小: 169 KiB 之后 宽度: | 高度: | 大小: 468 KiB |
|
之前 宽度: | 高度: | 大小: 478 KiB 之后 宽度: | 高度: | 大小: 535 KiB |
|
之前 宽度: | 高度: | 大小: 269 KiB 之后 宽度: | 高度: | 大小: 418 KiB |
|
之前 宽度: | 高度: | 大小: 140 KiB 之后 宽度: | 高度: | 大小: 382 KiB |
|
之前 宽度: | 高度: | 大小: 271 KiB 之后 宽度: | 高度: | 大小: 487 KiB |
|
之前 宽度: | 高度: | 大小: 98 KiB 之后 宽度: | 高度: | 大小: 458 KiB |
|
之前 宽度: | 高度: | 大小: 386 KiB 之后 宽度: | 高度: | 大小: 476 KiB |
|
之前 宽度: | 高度: | 大小: 386 KiB 之后 宽度: | 高度: | 大小: 476 KiB |
|
之后 宽度: | 高度: | 大小: 339 KiB |
|
之后 宽度: | 高度: | 大小: 386 KiB |
@@ -7,41 +7,104 @@
|
|||||||
|
|
||||||
采集 / 存储 / 呈现三件事都由本项目承担,不再依赖外部计划任务或自动化。
|
采集 / 存储 / 呈现三件事都由本项目承担,不再依赖外部计划任务或自动化。
|
||||||
|
|
||||||
|
多用户说明:所有涉及「数据」或「凭证」的子命令都作用于**某一个账号**。
|
||||||
|
用 `-u/--user <用户名>` 指定;不指定时取「管理员优先、其次 id 最小」的那个
|
||||||
|
(老库升级后数据都在首个账号名下,所以不指定也能沿用旧习惯)。
|
||||||
|
唯独 `collect` 不带 `-u` 时会**逐个账号**跑一遍,与进程内调度线程的行为一致。
|
||||||
|
|
||||||
常用:
|
常用:
|
||||||
python manage.py init 初始化数据库(建表 + 默认配置 + 管理员)
|
python manage.py init 初始化数据库(建表 + 默认配置 + 管理员)
|
||||||
python manage.py serve 启动 Web(0.0.0.0:8848,进程内含调度线程)
|
python manage.py serve 启动 Web(0.0.0.0:8848,进程内含调度线程)
|
||||||
python manage.py serve --port 9000 --debug 开发模式(reloader 下调度只启动一份)
|
python manage.py serve --port 9000 --debug 开发模式(reloader 下调度只启动一份)
|
||||||
python manage.py collect 执行一次增量采集并退出(可用于外部计划任务)
|
python manage.py collect 为**所有已启用账号**各跑一次增量采集
|
||||||
|
python manage.py collect -u alice 只为 alice 采集
|
||||||
python manage.py migrate-csv [文件] 从旧版 CSV 存档导入(默认自动探测路径)
|
python manage.py migrate-csv [文件] 从旧版 CSV 存档导入(默认自动探测路径)
|
||||||
python manage.py import-xlsx <文件> 从官网导出的 xlsx 合入
|
python manage.py import-xlsx <文件> 从官网导出的 xlsx 合入
|
||||||
python manage.py fill-prompt 补全缺失的 User Prompt
|
python manage.py fill-prompt 补全缺失的 User Prompt
|
||||||
python manage.py export-csv [路径] 导出与官网同构的 CSV
|
python manage.py export-csv [路径] 导出与官网同构的 CSV(文件名带账号名)
|
||||||
python manage.py stats 只看存档概况,不联网
|
python manage.py stats [-u 账号] 看存档概况,不联网
|
||||||
python manage.py passwd <用户名> [新密码] 重置登录密码
|
python manage.py users 列出所有账号及其数据量 / 凭证状态
|
||||||
python manage.py status 查看调度与最近采集状态
|
python manage.py passwd <用户名> [新密码] 重置密码;账号不存在则创建
|
||||||
|
python manage.py passwd <用户名> --role admin 新建或提权为管理员
|
||||||
|
python manage.py status 查看各账号的调度与最近采集状态
|
||||||
|
python manage.py vacuum 整理数据库(checkpoint + VACUUM)
|
||||||
"""
|
"""
|
||||||
import argparse
|
import argparse
|
||||||
import json
|
|
||||||
import os
|
import os
|
||||||
import sys
|
import sys
|
||||||
|
|
||||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||||
|
|
||||||
from workbuddy_portal import client, collect, config, db, query, scheduler # noqa: E402
|
from workbuddy_portal import client, collect, config, db, query, scheduler # noqa: E402
|
||||||
|
from workbuddy_portal import security # noqa: E402
|
||||||
|
|
||||||
|
|
||||||
def _p(*a):
|
def _p(*a):
|
||||||
print(*a)
|
print(*a)
|
||||||
|
|
||||||
|
|
||||||
|
def _warn(*a):
|
||||||
|
print(*a)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------- 账号解析 ----------------
|
||||||
|
def _default_uid(conn):
|
||||||
|
"""没显式指定 `-u` 时的目标账号:管理员优先,其次 id 最小。
|
||||||
|
|
||||||
|
老库升级后全部数据都归到首个账号,所以这个默认值正好等价于旧行为;
|
||||||
|
全空库(只有实例级配置)返回 0,即「实例作用域」。
|
||||||
|
"""
|
||||||
|
row = conn.execute("SELECT id FROM users ORDER BY is_admin DESC, id LIMIT 1").fetchone()
|
||||||
|
return row["id"] if row else 0
|
||||||
|
|
||||||
|
|
||||||
|
def _resolve_uid(conn, name):
|
||||||
|
"""把 `-u` 的取值(用户名或数字 id)解析成 uid;解析不到返回 None。"""
|
||||||
|
if name is None or name == "":
|
||||||
|
return _default_uid(conn)
|
||||||
|
row = db.user_by_name(conn, str(name))
|
||||||
|
if row is None and str(name).isdigit():
|
||||||
|
row = db.user_by_id(conn, int(name))
|
||||||
|
if row is None:
|
||||||
|
_p("[error] 没有这个账号:%s(用 manage.py users 查看)" % name)
|
||||||
|
return None
|
||||||
|
return row["id"]
|
||||||
|
|
||||||
|
|
||||||
|
def _uid_or_fail(conn, name):
|
||||||
|
"""解析失败时返回 None,调用方自行 return 2。"""
|
||||||
|
uid = _resolve_uid(conn, name)
|
||||||
|
if uid is None:
|
||||||
|
return None
|
||||||
|
if uid == 0:
|
||||||
|
_warn("[warn] 库里还没有任何账号,本次按“实例作用域”执行(先跑 manage.py init)")
|
||||||
|
return uid
|
||||||
|
|
||||||
|
|
||||||
|
def _ua_of(conn, uid):
|
||||||
|
"""人话描述某个账号的 Cookie 状态(只看密文可解性,不碰明文)。"""
|
||||||
|
st = db.secret_state(conn, "cookie", uid)
|
||||||
|
if st["broken"]:
|
||||||
|
return "损坏(密钥换过,需重新粘贴)"
|
||||||
|
if not st["set"]:
|
||||||
|
return "未配置"
|
||||||
|
return "已配置 %d 字符" % st["chars"]
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------- 初始化 / 服务 ----------------
|
||||||
def cmd_init(args):
|
def cmd_init(args):
|
||||||
db.init_db(admin_user=args.user, admin_password=args.password)
|
db.init_db(admin_user=args.user, admin_password=args.password)
|
||||||
conn = db.connect()
|
conn = db.connect()
|
||||||
try:
|
try:
|
||||||
n = query.totals(conn)
|
uid = _default_uid(conn)
|
||||||
|
n = query.totals(conn, uid)
|
||||||
_p("数据库已就绪:%s" % config.SQLITE_PATH)
|
_p("数据库已就绪:%s" % config.SQLITE_PATH)
|
||||||
_p(" 存档 %d 条 / %.2f 积分 / %d 个活跃日" % (n["records"], n["credits"], n["days"]))
|
_p(" 账号数:%d" % db.user_count(conn))
|
||||||
_p(" 管理员:%s" % args.user)
|
_p(" 主账号存档 %d 条 / %.2f 积分 / %d 个活跃日" % (n["records"], n["credits"], n["days"]))
|
||||||
|
_p(" 操作账号:%s" % args.user)
|
||||||
|
st = db.secret_state(conn, "cookie", uid)
|
||||||
|
if st["broken"]:
|
||||||
|
_p(" [warn] Cookie 密文无法解开(cookie_key 与写入时不一致),请登录后重新粘贴")
|
||||||
finally:
|
finally:
|
||||||
conn.close()
|
conn.close()
|
||||||
|
|
||||||
@@ -63,19 +126,58 @@ def cmd_serve(args):
|
|||||||
app.run(host=host, port=port, threaded=True)
|
app.run(host=host, port=port, threaded=True)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------- 采集 / 导入 / 导出 ----------------
|
||||||
def cmd_collect(args):
|
def cmd_collect(args):
|
||||||
|
"""不带 -u 时逐个已启用账号采集;带 -u 时只采一个。"""
|
||||||
db.init_db(create_admin=False)
|
db.init_db(create_admin=False)
|
||||||
|
conn = db.connect()
|
||||||
try:
|
try:
|
||||||
r = collect.run_sync(trigger="cli")
|
if args.user:
|
||||||
except collect.Busy as e:
|
uid = _resolve_uid(conn, args.user)
|
||||||
_p("[busy] %s" % e)
|
if uid is None:
|
||||||
return 1
|
return 2
|
||||||
except collect.ApiError as e:
|
targets = [uid]
|
||||||
_p("[error] %s" % e)
|
else:
|
||||||
return 3 if e.cookie_expired else 5
|
targets = [r["id"] for r in db.active_users(conn)]
|
||||||
for line in r["lines"]:
|
if not targets:
|
||||||
_p(line)
|
_p("[error] 没有任何启用中的账号")
|
||||||
return 0
|
return 2
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
rc = 0
|
||||||
|
for uid in targets:
|
||||||
|
conn = db.connect()
|
||||||
|
try:
|
||||||
|
row = db.user_by_id(conn, uid)
|
||||||
|
who = row["username"] if row else "uid=%s" % uid
|
||||||
|
st = db.secret_state(conn, "cookie", uid)
|
||||||
|
if not st["set"] or st["broken"]:
|
||||||
|
_p("— %s:跳过(Cookie %s)" % (who, _ua_of(conn, uid)))
|
||||||
|
continue
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
_p("— %s:" % who)
|
||||||
|
try:
|
||||||
|
r = collect.run_sync(trigger="cli", uid=uid)
|
||||||
|
except collect.Busy as e:
|
||||||
|
_p(" [busy] %s" % e)
|
||||||
|
rc = rc or 1
|
||||||
|
continue
|
||||||
|
except collect.NotReady as e:
|
||||||
|
_p(" [skip] %s" % e)
|
||||||
|
continue
|
||||||
|
except db.SecretUnreadable as e:
|
||||||
|
_p(" [error] %s(请重新粘贴 Cookie)" % e)
|
||||||
|
rc = rc or 4
|
||||||
|
continue
|
||||||
|
except collect.ApiError as e:
|
||||||
|
_p(" [error] %s" % e)
|
||||||
|
rc = rc or (3 if e.cookie_expired else 5)
|
||||||
|
continue
|
||||||
|
for line in r["lines"]:
|
||||||
|
_p(" " + line)
|
||||||
|
return rc
|
||||||
|
|
||||||
|
|
||||||
def cmd_migrate_csv(args):
|
def cmd_migrate_csv(args):
|
||||||
@@ -91,10 +193,13 @@ def cmd_migrate_csv(args):
|
|||||||
return 2
|
return 2
|
||||||
conn = db.connect()
|
conn = db.connect()
|
||||||
try:
|
try:
|
||||||
collect.migrate_from_csv(conn, path, log=_p)
|
uid = _uid_or_fail(conn, args.user)
|
||||||
n = query.totals(conn)
|
if uid is None:
|
||||||
_p("当前存档:%d 条 / %.2f 积分 / %s ~ %s" % (n["records"], n["credits"],
|
return 2
|
||||||
n["firstDay"], n["lastDay"]))
|
collect.migrate_from_csv(conn, uid, path, log=_p)
|
||||||
|
n = query.totals(conn, uid)
|
||||||
|
_p("账号 uid=%s 当前存档:%d 条 / %.2f 积分 / %s ~ %s"
|
||||||
|
% (uid, n["records"], n["credits"], n["firstDay"], n["lastDay"]))
|
||||||
finally:
|
finally:
|
||||||
conn.close()
|
conn.close()
|
||||||
return 0
|
return 0
|
||||||
@@ -104,49 +209,91 @@ def cmd_import_xlsx(args):
|
|||||||
db.init_db(create_admin=False)
|
db.init_db(create_admin=False)
|
||||||
conn = db.connect()
|
conn = db.connect()
|
||||||
try:
|
try:
|
||||||
collect.import_xlsx(conn, args.path, log=_p)
|
uid = _uid_or_fail(conn, args.user)
|
||||||
|
if uid is None:
|
||||||
|
return 2
|
||||||
|
collect.import_xlsx(conn, uid, args.path, log=_p)
|
||||||
finally:
|
finally:
|
||||||
conn.close()
|
conn.close()
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
def cmd_fill_prompt(args):
|
def cmd_fill_prompt(args):
|
||||||
db.init_db(create_admin=False)
|
db.init_db(create_admin=False)
|
||||||
conn = db.connect()
|
conn = db.connect()
|
||||||
try:
|
try:
|
||||||
collect.fill_prompt(conn, log=_p)
|
uid = _uid_or_fail(conn, args.user)
|
||||||
|
if uid is None:
|
||||||
|
return 2
|
||||||
|
collect.fill_prompt(conn, uid, log=_p)
|
||||||
finally:
|
finally:
|
||||||
conn.close()
|
conn.close()
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
def cmd_export_csv(args):
|
def cmd_export_csv(args):
|
||||||
db.init_db(create_admin=False)
|
db.init_db(create_admin=False)
|
||||||
conn = db.connect()
|
conn = db.connect()
|
||||||
try:
|
try:
|
||||||
path, n = collect.export_csv(conn, args.path)
|
uid = _uid_or_fail(conn, args.user)
|
||||||
|
if uid is None:
|
||||||
|
return 2
|
||||||
|
row = db.user_by_id(conn, uid)
|
||||||
|
path, n = collect.export_csv(conn, uid, args.path,
|
||||||
|
username=(row["username"] if row else None))
|
||||||
_p("已导出 %d 条 -> %s" % (n, path))
|
_p("已导出 %d 条 -> %s" % (n, path))
|
||||||
finally:
|
finally:
|
||||||
conn.close()
|
conn.close()
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------- 统计 ----------------
|
||||||
def cmd_stats(args):
|
def cmd_stats(args):
|
||||||
db.init_db(create_admin=False)
|
db.init_db(create_admin=False)
|
||||||
conn = db.connect()
|
conn = db.connect()
|
||||||
try:
|
try:
|
||||||
t = query.totals(conn)
|
# 先给一张全局概览:多用户下最常问的就是「一共多少、谁占多少」
|
||||||
|
rows = conn.execute(
|
||||||
|
"SELECT u.id, u.username, u.display_name, u.is_admin, u.status,"
|
||||||
|
" COUNT(r.request_id) AS records, COALESCE(SUM(r.credits),0) AS credits"
|
||||||
|
" FROM users u LEFT JOIN usage_records r ON r.user_id=u.id"
|
||||||
|
" GROUP BY u.id ORDER BY records DESC, u.id").fetchall()
|
||||||
|
total = conn.execute("SELECT COUNT(*) AS c, COALESCE(SUM(credits),0) AS s"
|
||||||
|
" FROM usage_records").fetchone()
|
||||||
|
_p("全库存档:%d 条 / %.2f 积分 / %d 个账号"
|
||||||
|
% (total["c"], total["s"], db.user_count(conn)))
|
||||||
|
if rows:
|
||||||
|
_p("")
|
||||||
|
_p("%-4s %-16s %-10s %-6s %-8s %8s %12s" %
|
||||||
|
("id", "用户名", "角色", "状态", "Cookie", "调用", "积分"))
|
||||||
|
for r in rows:
|
||||||
|
_p("%-4d %-16s %-10s %-6s %-8s %8d %12.2f" %
|
||||||
|
(r["id"], r["username"], "管理员" if r["is_admin"] else "普通",
|
||||||
|
"启用" if r["status"] == "active" else "停用",
|
||||||
|
_ua_of(conn, r["id"]), r["records"], r["credits"]))
|
||||||
|
|
||||||
|
uid = _resolve_uid(conn, args.user)
|
||||||
|
if uid is None:
|
||||||
|
return 2
|
||||||
|
t = query.totals(conn, uid)
|
||||||
if not t["records"]:
|
if not t["records"]:
|
||||||
_p("存档为空,先跑 python manage.py migrate-csv 或 manage.py collect")
|
_p("")
|
||||||
return
|
_p("(uid=%s 没有数据;换个 -u,或先跑 manage.py migrate-csv / collect)" % uid)
|
||||||
|
return 0
|
||||||
|
_p("")
|
||||||
|
_p("== uid=%s 明细 ==" % uid)
|
||||||
_p("存档:%d 条 / %.2f 积分 / %d 个活跃日(%s ~ %s)"
|
_p("存档:%d 条 / %.2f 积分 / %d 个活跃日(%s ~ %s)"
|
||||||
% (t["records"], t["credits"], t["days"], t["firstDay"], t["lastDay"]))
|
% (t["records"], t["credits"], t["days"], t["firstDay"], t["lastDay"]))
|
||||||
_p("计费调用 %d · 免费调用 %d · 模型 %d · 客户端 %d"
|
_p("计费调用 %d · 免费调用 %d · 模型 %d · 客户端 %d"
|
||||||
% (t["billableCalls"], t["freeCalls"], t["models"], t["clients"]))
|
% (t["billableCalls"], t["freeCalls"], t["models"], t["clients"]))
|
||||||
_p("")
|
_p("")
|
||||||
_p("%-24s %8s %12s %10s %8s" % ("模型", "调用", "积分", "单次均价", "免费占比"))
|
_p("%-24s %8s %12s %10s %8s" % ("模型", "调用", "积分", "单次均价", "免费占比"))
|
||||||
for m in query.dims(conn)["model"]:
|
for m in query.dims(conn, uid)["model"]:
|
||||||
_p("%-24s %8d %12.2f %10.2f %7.0f%%"
|
_p("%-24s %8d %12.2f %10.2f %7.0f%%"
|
||||||
% (m["name"], m["calls"], m["credits"], m["avgPerCall"], m["freeRate"] * 100))
|
% (m["name"], m["calls"], m["credits"], m["avgPerCall"], m["freeRate"] * 100))
|
||||||
runs = conn.execute("SELECT id,trigger,status,started_at,added,dup,total,message"
|
runs = conn.execute("SELECT id,trigger,status,started_at,added,dup,total,message"
|
||||||
" FROM collect_runs ORDER BY id DESC LIMIT 5").fetchall()
|
" FROM collect_runs WHERE user_id=? ORDER BY id DESC LIMIT 5",
|
||||||
|
(uid,)).fetchall()
|
||||||
if runs:
|
if runs:
|
||||||
_p("")
|
_p("")
|
||||||
_p("最近采集:")
|
_p("最近采集:")
|
||||||
@@ -156,13 +303,49 @@ def cmd_stats(args):
|
|||||||
r["total"], r["message"] or ""))
|
r["total"], r["message"] or ""))
|
||||||
finally:
|
finally:
|
||||||
conn.close()
|
conn.close()
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_users(args):
|
||||||
|
"""列出账号:角色 / 状态 / 数据量 / 凭证状态 / 最近登录。"""
|
||||||
|
db.init_db(create_admin=False)
|
||||||
|
conn = db.connect()
|
||||||
|
try:
|
||||||
|
rows = conn.execute(
|
||||||
|
"SELECT u.*, COUNT(r.request_id) AS records,"
|
||||||
|
" COALESCE(SUM(r.credits),0) AS credits,"
|
||||||
|
" MAX(r.day) AS last_day"
|
||||||
|
" FROM users u LEFT JOIN usage_records r ON r.user_id=u.id"
|
||||||
|
" GROUP BY u.id ORDER BY u.id").fetchall()
|
||||||
|
if not rows:
|
||||||
|
_p("还没有任何账号。跑 manage.py init 建管理员,或让用户自助注册。")
|
||||||
|
return 0
|
||||||
|
_p("%-4s %-16s %-12s %-6s %-6s %10s %8s %12s %s" %
|
||||||
|
("id", "用户名", "显示名", "角色", "状态", "Cookie", "调用", "积分", "最近登录 IP"))
|
||||||
|
for r in rows:
|
||||||
|
_p("%-4d %-16s %-12s %-6s %-6s %10s %8d %12.2f %s" %
|
||||||
|
(r["id"], r["username"], r["display_name"] or "",
|
||||||
|
"管理员" if r["is_admin"] else "普通",
|
||||||
|
"启用" if r["status"] == "active" else "停用",
|
||||||
|
_ua_of(conn, r["id"]), r["records"], r["credits"],
|
||||||
|
r["last_login_ip"] or "—"))
|
||||||
|
_p("")
|
||||||
|
_p("提示:cookie_key 或 secret_key 可在 data/instance.json 里找到,"
|
||||||
|
"二者权限等同管理员口令,切勿随仓库分发。")
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------- 凭证 / 口令 ----------------
|
||||||
def cmd_import_creds(args):
|
def cmd_import_creds(args):
|
||||||
"""把 VSCode 设置里的 cookie / userAgent 接管进数据库(一次性迁移用)。"""
|
"""把 VSCode 设置里的 cookie / userAgent 接管进数据库(一次性迁移用)。"""
|
||||||
db.init_db(create_admin=False)
|
db.init_db(create_admin=False)
|
||||||
conn = db.connect()
|
conn = db.connect()
|
||||||
try:
|
try:
|
||||||
|
uid = _uid_or_fail(conn, args.user)
|
||||||
|
if uid is None:
|
||||||
|
return 2
|
||||||
found = client.read_vscode_creds()
|
found = client.read_vscode_creds()
|
||||||
if not found:
|
if not found:
|
||||||
_p("[error] 没找到 VSCode 系编辑器的 settings.json")
|
_p("[error] 没找到 VSCode 系编辑器的 settings.json")
|
||||||
@@ -176,11 +359,15 @@ def cmd_import_creds(args):
|
|||||||
_p("[error] 这些文件里都没有 codebuddyUsage.cookie,请到「配置管理」页手工粘贴")
|
_p("[error] 这些文件里都没有 codebuddyUsage.cookie,请到「配置管理」页手工粘贴")
|
||||||
return 2
|
return 2
|
||||||
cookie, ua = hit
|
cookie, ua = hit
|
||||||
db.set_setting(conn, "cookie", cookie)
|
# 一律走 set_secret(内部就是 set_setting),值在落库前完成加密
|
||||||
|
db.set_secret(conn, "cookie", cookie, uid)
|
||||||
if ua:
|
if ua:
|
||||||
db.set_setting(conn, "user_agent", ua)
|
db.set_setting(conn, "user_agent", ua, uid)
|
||||||
db.audit(conn, "import_creds", "cli", "从 VSCode 设置导入凭证(%d 字符)" % len(cookie), "127.0.0.1")
|
row = db.user_by_id(conn, uid)
|
||||||
_p("已导入 Cookie(%d 字符)与 User-Agent(%s)" % (len(cookie), "有" if ua else "无"))
|
db.audit(conn, "import_creds", (row["username"] if row else "cli"),
|
||||||
|
"从 VSCode 设置导入凭证(%d 字符)" % len(cookie), "127.0.0.1", uid)
|
||||||
|
_p("已把 Cookie(%d 字符)与 User-Agent(%s)写入账号 uid=%s"
|
||||||
|
% (len(cookie), "有" if ua else "无", uid))
|
||||||
finally:
|
finally:
|
||||||
conn.close()
|
conn.close()
|
||||||
return 0
|
return 0
|
||||||
@@ -188,41 +375,83 @@ def cmd_import_creds(args):
|
|||||||
|
|
||||||
def cmd_passwd(args):
|
def cmd_passwd(args):
|
||||||
db.init_db(create_admin=False)
|
db.init_db(create_admin=False)
|
||||||
from workbuddy_portal.security import hash_password
|
|
||||||
conn = db.connect()
|
conn = db.connect()
|
||||||
try:
|
try:
|
||||||
row = conn.execute("SELECT id FROM users WHERE username=?", (args.user,)).fetchone()
|
row = db.user_by_name(conn, args.user)
|
||||||
pwd = args.password or "admin123"
|
pwd = args.password
|
||||||
|
is_admin = 1 if args.role == "admin" else 0
|
||||||
if row:
|
if row:
|
||||||
conn.execute("UPDATE users SET password_hash=? WHERE id=?", (hash_password(pwd), row["id"]))
|
sets, vals = [], []
|
||||||
_p("已重置 %s 的密码" % args.user)
|
if pwd:
|
||||||
|
sets.append("password_hash=?")
|
||||||
|
vals.append(security.hash_password(pwd))
|
||||||
|
if args.role:
|
||||||
|
sets.append("is_admin=?")
|
||||||
|
vals.append(is_admin)
|
||||||
|
if args.activate:
|
||||||
|
sets.append("status='active'")
|
||||||
|
if not sets:
|
||||||
|
_p("没给新密码也没给 --role,什么都没改")
|
||||||
|
return 0
|
||||||
|
vals.append(row["id"])
|
||||||
|
conn.execute("UPDATE users SET %s WHERE id=?" % ",".join(sets), vals)
|
||||||
|
_p("已更新账号 %s(%s)" % (args.user, ",".join(
|
||||||
|
x.split("=")[0] for x in sets)))
|
||||||
else:
|
else:
|
||||||
conn.execute("INSERT INTO users(username,password_hash,display_name,is_admin,created_at)"
|
if not pwd:
|
||||||
" VALUES(?,?,?,1,?)", (args.user, hash_password(pwd), args.user, db.now_str()))
|
_p("[error] 新账号必须给出密码")
|
||||||
_p("已创建用户 %s" % args.user)
|
return 2
|
||||||
_p("新密码:%s" % pwd)
|
conn.execute(
|
||||||
|
"INSERT INTO users(username,password_hash,display_name,is_admin,status,"
|
||||||
|
" created_at) VALUES(?,?,?,?,'active',?)",
|
||||||
|
(args.user, security.hash_password(pwd), args.user, is_admin, db.now_str()))
|
||||||
|
uid = conn.execute("SELECT id FROM users WHERE username=?",
|
||||||
|
(args.user,)).fetchone()["id"]
|
||||||
|
db.audit(conn, "user_create", "cli", "命令行创建账号 %s" % args.user,
|
||||||
|
"127.0.0.1", uid)
|
||||||
|
_p("已创建账号 %s(uid=%s,%s)"
|
||||||
|
% (args.user, uid, "管理员" if is_admin else "普通"))
|
||||||
|
if pwd:
|
||||||
|
_p("密码:%s" % pwd)
|
||||||
finally:
|
finally:
|
||||||
conn.close()
|
conn.close()
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------- 状态 / 维护 ----------------
|
||||||
def cmd_status(args):
|
def cmd_status(args):
|
||||||
db.init_db(create_admin=False)
|
db.init_db(create_admin=False)
|
||||||
conn = db.connect()
|
conn = db.connect()
|
||||||
try:
|
try:
|
||||||
_p("服务器时间:%s" % db.now_str())
|
_p("服务器时间:%s" % db.now_str())
|
||||||
_p("调度开关:%s" % ("启用" if db.get_bool(conn, "schedule_enabled", True) else "停用"))
|
|
||||||
_p("每日时刻:%s" % (", ".join(scheduler.slots(conn)) or "—"))
|
|
||||||
nxt = scheduler.next_run_at(conn)
|
|
||||||
_p("下次执行:%s" % (nxt.strftime("%Y-%m-%d %H:%M:%S") if nxt else "—"))
|
|
||||||
_p("Cookie:%s" % ("已配置" if (db.get_setting(conn, "cookie") or "").strip() else "未配置"))
|
|
||||||
_p("互斥锁:%s" % ("存在(有采集在跑)" if os.path.exists(collect.LOCK_PATH) else "不存在"))
|
_p("互斥锁:%s" % ("存在(有采集在跑)" if os.path.exists(collect.LOCK_PATH) else "不存在"))
|
||||||
last = conn.execute("SELECT * FROM collect_runs ORDER BY id DESC LIMIT 1").fetchone()
|
_p("调度总开关(实例级):%s"
|
||||||
if last:
|
% ("启用" if db.get_bool(conn, "schedule_enabled", True) else "停用"))
|
||||||
_p("最近采集:#%d %s %s %s" % (last["id"], last["started_at"], last["status"],
|
users = db.active_users(conn)
|
||||||
last["message"] or ""))
|
if not users:
|
||||||
else:
|
_p("(没有任何启用中的账号。先 manage.py init 建管理员,"
|
||||||
_p("最近采集:无")
|
"或在「用户管理」页启用一个账号)")
|
||||||
_p("(注意:调度线程只在 manage.py serve 进程内运行)")
|
for u in users:
|
||||||
|
uid = u["id"]
|
||||||
|
_p("")
|
||||||
|
_p("== uid=%d %s%s ==" % (uid, u["username"],
|
||||||
|
"(管理员)" if u["is_admin"] else ""))
|
||||||
|
_p(" 调度:%s / 每日 %s" %
|
||||||
|
("启用" if db.get_bool(conn, "schedule_enabled", True, uid) else "停用",
|
||||||
|
", ".join(scheduler.slots(conn, uid)) or "—"))
|
||||||
|
nxt = scheduler.next_run_at(conn, uid)
|
||||||
|
_p(" 下次执行:%s" % (nxt.strftime("%Y-%m-%d %H:%M:%S") if nxt else "—"))
|
||||||
|
_p(" Cookie:%s" % _ua_of(conn, uid))
|
||||||
|
last = conn.execute("SELECT * FROM collect_runs WHERE user_id=?"
|
||||||
|
" ORDER BY id DESC LIMIT 1", (uid,)).fetchone()
|
||||||
|
if last:
|
||||||
|
_p(" 最近采集:#%d %s %s %s" % (last["id"], last["started_at"],
|
||||||
|
last["status"], last["message"] or ""))
|
||||||
|
else:
|
||||||
|
_p(" 最近采集:无")
|
||||||
|
_p("")
|
||||||
|
_p("(注意:调度线程只在 manage.py serve 进程内运行;"
|
||||||
|
"多实例部署时其余实例要设 WB_DISABLE_SCHEDULER=1)")
|
||||||
finally:
|
finally:
|
||||||
conn.close()
|
conn.close()
|
||||||
|
|
||||||
@@ -238,7 +467,8 @@ def cmd_vacuum(args):
|
|||||||
conn.execute("PRAGMA optimize")
|
conn.execute("PRAGMA optimize")
|
||||||
after = os.path.getsize(config.SQLITE_PATH) if os.path.exists(config.SQLITE_PATH) else 0
|
after = os.path.getsize(config.SQLITE_PATH) if os.path.exists(config.SQLITE_PATH) else 0
|
||||||
_p("数据库整理完成:%s → %s(%+d 字节)" % (_human(before), _human(after), after - before))
|
_p("数据库整理完成:%s → %s(%+d 字节)" % (_human(before), _human(after), after - before))
|
||||||
_p("存档 %d 条记录" % collect.record_count(conn))
|
n = conn.execute("SELECT COUNT(*) FROM usage_records").fetchone()[0]
|
||||||
|
_p("全库存档 %d 条记录" % n)
|
||||||
finally:
|
finally:
|
||||||
conn.close()
|
conn.close()
|
||||||
return 0
|
return 0
|
||||||
@@ -251,6 +481,11 @@ def _human(n):
|
|||||||
n /= 1024.0
|
n /= 1024.0
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------- 参数表 ----------------
|
||||||
|
def _add_user_opt(p, help_text="作用账号(用户名或 uid),默认取管理员 / 最小 id"):
|
||||||
|
p.add_argument("-u", "--user", default=None, help=help_text)
|
||||||
|
|
||||||
|
|
||||||
def main():
|
def main():
|
||||||
ap = argparse.ArgumentParser(description="WorkBuddy Portal(workbuddy-portal)",
|
ap = argparse.ArgumentParser(description="WorkBuddy Portal(workbuddy-portal)",
|
||||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||||
@@ -258,7 +493,7 @@ def main():
|
|||||||
sub = ap.add_subparsers(dest="cmd")
|
sub = ap.add_subparsers(dest="cmd")
|
||||||
|
|
||||||
s = sub.add_parser("init", help="初始化数据库")
|
s = sub.add_parser("init", help="初始化数据库")
|
||||||
s.add_argument("--user", default="admin")
|
s.add_argument("--user", default="admin", help="首个管理员用户名(仅库为空时生效)")
|
||||||
s.add_argument("--password", default=None)
|
s.add_argument("--password", default=None)
|
||||||
s.set_defaults(func=cmd_init)
|
s.set_defaults(func=cmd_init)
|
||||||
|
|
||||||
@@ -269,36 +504,49 @@ def main():
|
|||||||
s.add_argument("--no-scheduler", action="store_true", help="不启动进程内调度线程")
|
s.add_argument("--no-scheduler", action="store_true", help="不启动进程内调度线程")
|
||||||
s.set_defaults(func=cmd_serve)
|
s.set_defaults(func=cmd_serve)
|
||||||
|
|
||||||
s = sub.add_parser("collect", help="执行一次增量采集")
|
s = sub.add_parser("collect", help="执行一次增量采集(默认所有启用账号)")
|
||||||
|
_add_user_opt(s, "只采这一个账号;不传则逐个启用账号采集")
|
||||||
s.set_defaults(func=cmd_collect)
|
s.set_defaults(func=cmd_collect)
|
||||||
|
|
||||||
s = sub.add_parser("migrate-csv", help="从旧版 CSV 导入")
|
s = sub.add_parser("migrate-csv", help="从旧版 CSV 导入")
|
||||||
s.add_argument("path", nargs="?")
|
s.add_argument("path", nargs="?")
|
||||||
|
_add_user_opt(s, "这份老存档算谁的")
|
||||||
s.set_defaults(func=cmd_migrate_csv)
|
s.set_defaults(func=cmd_migrate_csv)
|
||||||
|
|
||||||
s = sub.add_parser("import-xlsx", help="从官网 xlsx 导入")
|
s = sub.add_parser("import-xlsx", help="从官网 xlsx 导入")
|
||||||
s.add_argument("path")
|
s.add_argument("path")
|
||||||
|
_add_user_opt(s)
|
||||||
s.set_defaults(func=cmd_import_xlsx)
|
s.set_defaults(func=cmd_import_xlsx)
|
||||||
|
|
||||||
s = sub.add_parser("fill-prompt", help="补全缺失的 User Prompt")
|
s = sub.add_parser("fill-prompt", help="补全缺失的 User Prompt")
|
||||||
|
_add_user_opt(s)
|
||||||
s.set_defaults(func=cmd_fill_prompt)
|
s.set_defaults(func=cmd_fill_prompt)
|
||||||
|
|
||||||
s = sub.add_parser("export-csv", help="导出 CSV")
|
s = sub.add_parser("export-csv", help="导出 CSV")
|
||||||
s.add_argument("path", nargs="?")
|
s.add_argument("path", nargs="?")
|
||||||
|
_add_user_opt(s)
|
||||||
s.set_defaults(func=cmd_export_csv)
|
s.set_defaults(func=cmd_export_csv)
|
||||||
|
|
||||||
s = sub.add_parser("stats", help="存档概况")
|
s = sub.add_parser("stats", help="存档概况(先全库概览,再给指定账号明细)")
|
||||||
|
_add_user_opt(s)
|
||||||
s.set_defaults(func=cmd_stats)
|
s.set_defaults(func=cmd_stats)
|
||||||
|
|
||||||
|
s = sub.add_parser("users", help="列出所有账号及其数据量 / 凭证状态")
|
||||||
|
s.set_defaults(func=cmd_users)
|
||||||
|
|
||||||
s = sub.add_parser("import-creds", help="从 VSCode 设置导入 cookie / UA 到数据库")
|
s = sub.add_parser("import-creds", help="从 VSCode 设置导入 cookie / UA 到数据库")
|
||||||
|
_add_user_opt(s)
|
||||||
s.set_defaults(func=cmd_import_creds)
|
s.set_defaults(func=cmd_import_creds)
|
||||||
|
|
||||||
s = sub.add_parser("passwd", help="重置 / 创建登录账号")
|
s = sub.add_parser("passwd", help="重置 / 创建登录账号")
|
||||||
s.add_argument("user")
|
s.add_argument("user")
|
||||||
s.add_argument("password", nargs="?")
|
s.add_argument("password", nargs="?")
|
||||||
|
s.add_argument("--role", choices=["admin", "user"], default=None,
|
||||||
|
help="不提则保持原角色;新建时默认普通账号")
|
||||||
|
s.add_argument("--activate", action="store_true", help="顺便把状态改回启用")
|
||||||
s.set_defaults(func=cmd_passwd)
|
s.set_defaults(func=cmd_passwd)
|
||||||
|
|
||||||
s = sub.add_parser("status", help="调度与最近采集状态")
|
s = sub.add_parser("status", help="各账号的调度与最近采集状态")
|
||||||
s.set_defaults(func=cmd_status)
|
s.set_defaults(func=cmd_status)
|
||||||
|
|
||||||
s = sub.add_parser("vacuum", help="整理数据库(checkpoint + VACUUM)")
|
s = sub.add_parser("vacuum", help="整理数据库(checkpoint + VACUUM)")
|
||||||
|
|||||||
@@ -21,13 +21,17 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import argparse
|
import argparse
|
||||||
|
import base64
|
||||||
import http.cookiejar
|
import http.cookiejar
|
||||||
import json
|
import json
|
||||||
|
import os
|
||||||
import re
|
import re
|
||||||
|
import sqlite3
|
||||||
import sys
|
import sys
|
||||||
import urllib.error
|
import urllib.error
|
||||||
import urllib.parse
|
import urllib.parse
|
||||||
import urllib.request
|
import urllib.request
|
||||||
|
import zlib
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
|
|
||||||
OK = 0
|
OK = 0
|
||||||
@@ -40,6 +44,31 @@ def _d(s: str):
|
|||||||
return datetime.strptime(s, "%Y-%m-%d")
|
return datetime.strptime(s, "%Y-%m-%d")
|
||||||
|
|
||||||
|
|
||||||
|
def decode_session(cj) -> dict:
|
||||||
|
"""从 Flask 会话 cookie 里解出那份**未加密**的载荷。
|
||||||
|
|
||||||
|
Flask 的会话是「签名 + base64,**不加密**」的 —— 也就是说持有 cookie 的人
|
||||||
|
就能读到里面的内容。本项目因此把验证码答案放在服务端 captchas 表里,
|
||||||
|
会话里只留一个随机 id;本函数存在的意义就是取出那个 id,
|
||||||
|
好让自动化验收能跨过验证码这一关(顺便也验证了「答案不在会话里」)。
|
||||||
|
"""
|
||||||
|
for c in cj:
|
||||||
|
if not c.name.startswith("workbuddy_portal_sid"):
|
||||||
|
continue
|
||||||
|
seg = urllib.parse.unquote(c.value).split(".")[0]
|
||||||
|
seg += "=" * (-len(seg) % 4)
|
||||||
|
try:
|
||||||
|
raw = base64.urlsafe_b64decode(seg)
|
||||||
|
try:
|
||||||
|
raw = zlib.decompress(raw) # 某些版本的 itsdangerous 会压
|
||||||
|
except zlib.error:
|
||||||
|
pass
|
||||||
|
return json.loads(raw.decode("utf-8"))
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
return {}
|
||||||
|
return {}
|
||||||
|
|
||||||
|
|
||||||
def chk(name: str, cond: bool, extra: str = "") -> None:
|
def chk(name: str, cond: bool, extra: str = "") -> None:
|
||||||
global OK, FAIL
|
global OK, FAIL
|
||||||
if cond:
|
if cond:
|
||||||
@@ -59,9 +88,10 @@ class _NoRedirect(urllib.request.HTTPRedirectHandler):
|
|||||||
|
|
||||||
|
|
||||||
class Live:
|
class Live:
|
||||||
def __init__(self, base: str, timeout: int = 20):
|
def __init__(self, base: str, timeout: int = 20, db_path: str | None = None):
|
||||||
self.base = base.rstrip("/")
|
self.base = base.rstrip("/")
|
||||||
self.timeout = timeout
|
self.timeout = timeout
|
||||||
|
self.db_path = db_path
|
||||||
# 关键:显式清空代理,否则本机代理会把 127.0.0.1 也拦成 502
|
# 关键:显式清空代理,否则本机代理会把 127.0.0.1 也拦成 502
|
||||||
self.cj = http.cookiejar.CookieJar()
|
self.cj = http.cookiejar.CookieJar()
|
||||||
self.op = urllib.request.build_opener(
|
self.op = urllib.request.build_opener(
|
||||||
@@ -114,6 +144,62 @@ class Live:
|
|||||||
st, body = self.get(path)
|
st, body = self.get(path)
|
||||||
return json.loads(body) if st == 200 else {}
|
return json.loads(body) if st == 200 else {}
|
||||||
|
|
||||||
|
def raw(self, path: str):
|
||||||
|
"""返回 (status, headers, bytes)——验证码/响应头这类要原始字节的场景用。"""
|
||||||
|
try:
|
||||||
|
r = self.op.open(urllib.request.Request(self.base + path), timeout=self.timeout)
|
||||||
|
return r.status, r.headers, r.read()
|
||||||
|
except urllib.error.HTTPError as e:
|
||||||
|
return e.code, e.headers, e.read()
|
||||||
|
|
||||||
|
def form_csrf(self, path: str) -> str:
|
||||||
|
"""取某个页面里的 CSRF 隐藏域(该页面必须与当前会话同源)。"""
|
||||||
|
_, html = self.get(path)
|
||||||
|
m = re.search(r'name="_csrf"\s+value="([^"]+)"', html)
|
||||||
|
return m.group(1) if m else ""
|
||||||
|
|
||||||
|
# ---- 验证码辅助(仅验收脚本用)----
|
||||||
|
def solve_captcha(self, purpose: str):
|
||||||
|
"""取一张图 -> 从会话里读 id -> 从本地库里取答案。返回 (答案, 会话载荷)。"""
|
||||||
|
self.raw("/captcha.png?purpose=" + purpose)
|
||||||
|
sess = decode_session(self.cj)
|
||||||
|
cid = sess.get("cap_" + purpose)
|
||||||
|
if not cid or not self.db_path or not os.path.exists(self.db_path):
|
||||||
|
return None, sess
|
||||||
|
try:
|
||||||
|
con = sqlite3.connect(self.db_path)
|
||||||
|
try:
|
||||||
|
row = con.execute("SELECT answer FROM captchas WHERE id=?", (cid,)).fetchone()
|
||||||
|
finally:
|
||||||
|
con.close()
|
||||||
|
except sqlite3.Error:
|
||||||
|
return None, sess
|
||||||
|
return (row[0] if row else None), sess
|
||||||
|
|
||||||
|
def login(self, user: str, pwd: str, nxt: str = "", follow: bool = True):
|
||||||
|
"""完整登录(验证码策略为 always 时自动解)。
|
||||||
|
|
||||||
|
follow=False 时返回原始 (status, Location),用于验证跳转目标是否安全。
|
||||||
|
返回 (status, location, need_captcha, session_payload)。
|
||||||
|
"""
|
||||||
|
html = self.get("/login")[1]
|
||||||
|
need_cap = 'name="captcha"' in html
|
||||||
|
m = re.search(r'name="_csrf"\s+value="([^"]+)"', html)
|
||||||
|
data = {"username": user, "password": pwd, "_csrf": m.group(1) if m else ""}
|
||||||
|
if nxt:
|
||||||
|
data["next"] = nxt
|
||||||
|
sess = {}
|
||||||
|
if need_cap:
|
||||||
|
ans, sess = self.solve_captcha("login")
|
||||||
|
if ans is None:
|
||||||
|
return None, None, True, sess
|
||||||
|
data["captcha"] = ans
|
||||||
|
if follow:
|
||||||
|
st, _ = self.post("/login", data)
|
||||||
|
return st, None, need_cap, sess
|
||||||
|
st, loc = self.post_raw("/login", data)
|
||||||
|
return st, loc, need_cap, sess
|
||||||
|
|
||||||
|
|
||||||
def run(L: Live, user: str, pwd: str, frm: str, to: str) -> None:
|
def run(L: Live, user: str, pwd: str, frm: str, to: str) -> None:
|
||||||
print("== 1. 未登录访问受保护资源 ==")
|
print("== 1. 未登录访问受保护资源 ==")
|
||||||
@@ -123,13 +209,15 @@ def run(L: Live, user: str, pwd: str, frm: str, to: str) -> None:
|
|||||||
st, _ = L.get(p)
|
st, _ = L.get(p)
|
||||||
chk("GET %-14s 未登录=401" % p, st == 401, "status=%s" % st)
|
chk("GET %-14s 未登录=401" % p, st == 401, "status=%s" % st)
|
||||||
|
|
||||||
print("== 2. 登录(含 CSRF) ==")
|
print("== 2. 登录(含 CSRF;验证码策略为 always 时自动解) ==")
|
||||||
st, html = L.get("/login")
|
st, html = L.get("/login")
|
||||||
m = re.search(r'name="_csrf"\s+value="([^"]+)"', html)
|
chk("登录页含 CSRF 隐藏域", bool(re.search(r'name="_csrf"\s+value="([^"]+)"', html)))
|
||||||
chk("登录页含 CSRF 隐藏域", bool(m))
|
st, _, need_cap, sess = L.login(user, pwd)
|
||||||
st, _ = L.post("/login", {"username": user, "password": pwd,
|
|
||||||
"_csrf": m.group(1) if m else ""})
|
|
||||||
chk("登录成功", st in (200, 302), "status=%s" % st)
|
chk("登录成功", st in (200, 302), "status=%s" % st)
|
||||||
|
if need_cap:
|
||||||
|
# 会话里只应有 id,不该有答案本身
|
||||||
|
chk("会话里只存验证码 id(不是答案)", bool(sess.get("cap_login")),
|
||||||
|
"cap_login=%s" % (sess.get("cap_login") or "无"))
|
||||||
st, html = L.get("/")
|
st, html = L.get("/")
|
||||||
chk("登录后 GET / 到概览", st == 200 and "概览" in html, "len=%d" % len(html))
|
chk("登录后 GET / 到概览", st == 200 and "概览" in html, "len=%d" % len(html))
|
||||||
|
|
||||||
@@ -212,11 +300,25 @@ def run(L: Live, user: str, pwd: str, frm: str, to: str) -> None:
|
|||||||
|
|
||||||
print("== 7. 凭据不外泄 ==")
|
print("== 7. 凭据不外泄 ==")
|
||||||
stj = L.jget("/api/settings")
|
stj = L.jget("/api/settings")
|
||||||
chk("settings 无 cookie 明文字段", "cookie" not in stj, "keys=%s" % list(stj.keys()))
|
# 契约:settings 里 cookie 这个键**必须为空**(db.get_settings 统一置空),
|
||||||
|
# 真正的状态只通过 cookie_hint / cookie_broken 这两个派生字段暴露。
|
||||||
|
chk("settings 里 cookie 字段为空串",
|
||||||
|
"cookie" in stj and not str(stj.get("cookie") or "").strip(),
|
||||||
|
"cookie=%r" % stj.get("cookie"))
|
||||||
|
chk("settings 用 cookie_hint/cookie_broken 代替明文",
|
||||||
|
"cookie_hint" in stj and "cookie_broken" in stj)
|
||||||
chk("settings 仅回 cookie_hint 掩码",
|
chk("settings 仅回 cookie_hint 掩码",
|
||||||
bool(stj.get("cookie_hint")) and len(str(stj.get("cookie_hint"))) < 200,
|
bool(stj.get("cookie_hint")) and len(str(stj.get("cookie_hint"))) < 200,
|
||||||
"hint=%s" % stj.get("cookie_hint"))
|
"hint=%s" % stj.get("cookie_hint"))
|
||||||
|
chk("settings 回传实例级键清单", isinstance(stj.get("_globalKeys"), list)
|
||||||
|
and bool(stj.get("_globalKeys")), "%s" % stj.get("_globalKeys"))
|
||||||
|
chk("settings 标明能否改实例级配置", stj.get("_canEditGlobal") is True)
|
||||||
chk("配置页 HTML 不含 cookie 明文", "eyJ" not in L.get("/config")[1])
|
chk("配置页 HTML 不含 cookie 明文", "eyJ" not in L.get("/config")[1])
|
||||||
|
# 密文形态:v1.<b64salt>.<b64nonce>.<b64ct>.<b64tag>,恰好用正则判定,
|
||||||
|
# 免得把版本号 "v1.2.0" 当成泄漏(这两者前缀撞车)
|
||||||
|
cipher_re = re.compile(r"v1\.[A-Za-z0-9+/=]{8,}\.[A-Za-z0-9+/=]{8,}\.")
|
||||||
|
for p in ("/config", "/profile", "/"):
|
||||||
|
chk("%-9s HTML 里没有 Cookie 密文" % p, not cipher_re.search(L.get(p)[1]))
|
||||||
|
|
||||||
print("== 8. 错误处理 ==")
|
print("== 8. 错误处理 ==")
|
||||||
for p in ("/api/nope", "/nope"):
|
for p in ("/api/nope", "/nope"):
|
||||||
@@ -254,20 +356,13 @@ def run(L: Live, user: str, pwd: str, frm: str, to: str) -> None:
|
|||||||
"api=%s csv=%s" % (first_id, (lines[1][:40] if len(lines) > 1 else None)))
|
"api=%s csv=%s" % (first_id, (lines[1][:40] if len(lines) > 1 else None)))
|
||||||
|
|
||||||
print("== 10. 安全:开放重定向与凭证外泄 ==")
|
print("== 10. 安全:开放重定向与凭证外泄 ==")
|
||||||
L2 = Live(L.base) # 全新会话,避免已登录被直跳
|
L2 = Live(L.base, L.timeout, L.db_path) # 全新会话,避免已登录被直跳
|
||||||
st, html = L2.get("/login")
|
st, loc, _, _ = L2.login(user, pwd, nxt="//evil.com", follow=False)
|
||||||
m = re.search(r'name="_csrf"\s+value="([^"]+)"', html)
|
|
||||||
csrf = m.group(1) if m else ""
|
|
||||||
st, loc = L2.post_raw("/login", {"username": user, "password": pwd,
|
|
||||||
"_csrf": csrf, "next": "//evil.com"})
|
|
||||||
chk("next=//evil.com 被拒(不出现协议相对跳转)",
|
chk("next=//evil.com 被拒(不出现协议相对跳转)",
|
||||||
st == 302 and "evil.com" not in (loc or "") and not (loc or "").startswith("//"),
|
st == 302 and "evil.com" not in (loc or "") and not (loc or "").startswith("//"),
|
||||||
"status=%s Location=%s" % (st, loc))
|
"status=%s Location=%s" % (st, loc))
|
||||||
L3 = Live(L.base)
|
L3 = Live(L.base, L.timeout, L.db_path)
|
||||||
st, html = L3.get("/login")
|
st, loc, _, _ = L3.login(user, pwd, nxt="/records", follow=False)
|
||||||
m = re.search(r'name="_csrf"\s+value="([^"]+)"', html)
|
|
||||||
st, loc = L3.post_raw("/login", {"username": user, "password": pwd,
|
|
||||||
"_csrf": m.group(1) if m else "", "next": "/records"})
|
|
||||||
chk("next=/records 站内路径正常放行", st == 302 and loc == "/records",
|
chk("next=/records 站内路径正常放行", st == 302 and loc == "/records",
|
||||||
"status=%s Location=%s" % (st, loc))
|
"status=%s Location=%s" % (st, loc))
|
||||||
st, loc = L3.post_raw("/login", {"username": user, "password": pwd, "_csrf": "wrong"})
|
st, loc = L3.post_raw("/login", {"username": user, "password": pwd, "_csrf": "wrong"})
|
||||||
@@ -277,6 +372,50 @@ def run(L: Live, user: str, pwd: str, frm: str, to: str) -> None:
|
|||||||
st, html = L.get("/")
|
st, html = L.get("/")
|
||||||
chk("GET /logout 后仍处于登录态", st == 200 and "概览" in html, "status=%s" % st)
|
chk("GET /logout 后仍处于登录态", st == 200 and "概览" in html, "status=%s" % st)
|
||||||
|
|
||||||
|
print("== 11. 多用户:注册入口 / 验证码 / 安全响应头 ==")
|
||||||
|
L4 = Live(L.base, L.timeout, L.db_path) # 全新未登录会话
|
||||||
|
st, html = L4.get("/register")
|
||||||
|
chk("GET /register 可达", st == 200 and "注册" in html, "status=%s" % st)
|
||||||
|
chk("注册页带验证码图", "capimg" in html and "/captcha.png" in html)
|
||||||
|
chk("注册页带 CSRF 隐藏域", bool(L4.form_csrf("/register")))
|
||||||
|
st, html = L4.get("/login")
|
||||||
|
chk("登录页带验证码图", "capimg" in html and 'name="captcha"' in html)
|
||||||
|
chk("登录页带自助注册链接", "/register" in html)
|
||||||
|
|
||||||
|
# 出图:真实字节 + 禁缓存 + 确实每次都不一样
|
||||||
|
shots = {}
|
||||||
|
for purpose in ("login", "register"):
|
||||||
|
code, hdr, data = L4.raw("/captcha.png?purpose=" + purpose)
|
||||||
|
chk("GET /captcha.png?purpose=%-8s 出 PNG" % purpose,
|
||||||
|
code == 200 and data[:4] == b"\x89PNG" and len(data) > 200,
|
||||||
|
"status=%s len=%d" % (code, len(data)))
|
||||||
|
chk(" └ 禁缓存 no-store", "no-store" in (hdr.get("Cache-Control") or ""))
|
||||||
|
chk(" └ 类型 image/png", (hdr.get("Content-Type") or "").startswith("image/png"))
|
||||||
|
shots[purpose] = data
|
||||||
|
_, _, again = L4.raw("/captcha.png?purpose=login")
|
||||||
|
chk("两次取图内容不同(不是一张静态图)", again != shots["login"])
|
||||||
|
chk("login 与 register 的图互不相同", shots["login"] != shots["register"])
|
||||||
|
# 图必须由服务端单独下发,不能把答案内联进页面
|
||||||
|
st, html = L4.get("/login")
|
||||||
|
chk("登录页没有内联 data: 图片(答案不走页面源码)",
|
||||||
|
"data:image" not in html and "base64," not in html)
|
||||||
|
|
||||||
|
# 安全响应头
|
||||||
|
code, hdr, _ = L4.raw("/login")
|
||||||
|
for name, want in (("X-Content-Type-Options", "nosniff"),
|
||||||
|
("X-Frame-Options", "DENY"),
|
||||||
|
("Referrer-Policy", "same-origin")):
|
||||||
|
chk("响应头 %-24s" % name, (hdr.get(name) or "") == want, "=%s" % hdr.get(name))
|
||||||
|
chk("响应头含 CSP 且 frame-ancestors 'none'",
|
||||||
|
"frame-ancestors 'none'" in (hdr.get("Content-Security-Policy") or ""))
|
||||||
|
code, hdr, _ = L4.raw("/captcha.png?purpose=login")
|
||||||
|
chk("/captcha 路径带 no-store", "no-store" in (hdr.get("Cache-Control") or ""))
|
||||||
|
|
||||||
|
# 路径穿越式 purpose 必须被收敛到已知用途,而不是 500
|
||||||
|
code, _, data = L4.raw("/captcha.png?purpose=../../etc/passwd")
|
||||||
|
chk("非法 purpose 不报 500", code == 200 and data[:4] == b"\x89PNG",
|
||||||
|
"status=%s" % code)
|
||||||
|
|
||||||
|
|
||||||
def main() -> int:
|
def main() -> int:
|
||||||
ap = argparse.ArgumentParser(description="对运行中的用量门户做端到端验收")
|
ap = argparse.ArgumentParser(description="对运行中的用量门户做端到端验收")
|
||||||
@@ -285,11 +424,17 @@ def main() -> int:
|
|||||||
ap.add_argument("-p", "--password", default="admin123", help="登录密码")
|
ap.add_argument("-p", "--password", default="admin123", help="登录密码")
|
||||||
ap.add_argument("--from", dest="frm", default="2026-09-08", help="验收窗口起")
|
ap.add_argument("--from", dest="frm", default="2026-09-08", help="验收窗口起")
|
||||||
ap.add_argument("--to", dest="to", default="2026-09-14", help="验收窗口止")
|
ap.add_argument("--to", dest="to", default="2026-09-14", help="验收窗口止")
|
||||||
|
ap.add_argument("--db", default=None,
|
||||||
|
help="SQLite 路径(默认 <repo>/data/usage.sqlite)。"
|
||||||
|
"验证码策略为 always 时用它取答案以完成自动登录;"
|
||||||
|
"指向不存在的文件则跳过需要验证码的登录")
|
||||||
ap.add_argument("--timeout", type=int, default=20)
|
ap.add_argument("--timeout", type=int, default=20)
|
||||||
a = ap.parse_args()
|
a = ap.parse_args()
|
||||||
|
|
||||||
print("目标:%s 窗口:%s ~ %s\n" % (a.base, a.frm, a.to))
|
db_path = a.db or os.path.join(
|
||||||
run(Live(a.base, a.timeout), a.user, a.password, a.frm, a.to)
|
os.path.dirname(os.path.dirname(os.path.abspath(__file__))), "data", "usage.sqlite")
|
||||||
|
print("目标:%s 窗口:%s ~ %s\n验证码答案源:%s\n" % (a.base, a.frm, a.to, db_path))
|
||||||
|
run(Live(a.base, a.timeout, db_path), a.user, a.password, a.frm, a.to)
|
||||||
print("\nRESULT: ok=%d fail=%d" % (OK, FAIL))
|
print("\nRESULT: ok=%d fail=%d" % (OK, FAIL))
|
||||||
if FAILS:
|
if FAILS:
|
||||||
print("失败项:%s" % "、".join(FAILS))
|
print("失败项:%s" % "、".join(FAILS))
|
||||||
|
|||||||
@@ -14,18 +14,21 @@
|
|||||||
--------
|
--------
|
||||||
| 表 | 说明 |
|
| 表 | 说明 |
|
||||||
|---|---|
|
|---|---|
|
||||||
|
| `users` | 明示例两个账号:`admin`(管理员)与 `demo`(普通账号),各有自己的数据 |
|
||||||
| `usage_records` | 约 900 条合成记录,跨 30 天,含假模型名 / 假 Prompt / 偏斜的积分分布 |
|
| `usage_records` | 约 900 条合成记录,跨 30 天,含假模型名 / 假 Prompt / 偏斜的积分分布 |
|
||||||
| `collect_runs` | 14 条采集历史,含 ok / warn / error 三种状态 |
|
| `collect_runs` | 采集历史,含 ok / warn / error 三种状态 |
|
||||||
| `settings` | 走项目默认值(`config.DEFAULTS`),**不写入任何凭据** |
|
| `settings` | 走项目默认值(`config.DEFAULTS`),凭证只写**一眼可辨的假值** |
|
||||||
| `audit_log` | 30 条操作审计 |
|
| `audit_log` | 操作审计(按账号归属) |
|
||||||
| `users` | 由 `db.init_db()` 建一个管理员 |
|
|
||||||
|
刻意造两个账号,是因为「数据按账号隔离」在多用户版里是最该被截进文档的性质:
|
||||||
|
只有一个账号的话,用户管理页和「我的数据」列都看不出区别。
|
||||||
|
|
||||||
用法
|
用法
|
||||||
----
|
----
|
||||||
# 默认写到 data/demo/(该目录在 .gitignore 内,不会误提交)
|
# 默认写到 data/demo/(该目录在 .gitignore 内,不会误提交)
|
||||||
python tools/demo_data.py
|
python tools/demo_data.py
|
||||||
|
|
||||||
# 指定目录与管理员口令,然后起服务看效果
|
# 指定目录与口令,然后起服务看效果
|
||||||
python tools/demo_data.py --out data/demo --admin-password demo123
|
python tools/demo_data.py --out data/demo --admin-password demo123
|
||||||
WB_DATA_DIR=$PWD/data/demo python manage.py serve --port 8849 --no-scheduler
|
WB_DATA_DIR=$PWD/data/demo python manage.py serve --port 8849 --no-scheduler
|
||||||
|
|
||||||
@@ -61,7 +64,19 @@ CLIENTS = [("vscode", 74), ("webconsole", 18), ("sdk", 8)]
|
|||||||
|
|
||||||
# 写进 settings 的假 Cookie。刻意用重复串,一眼就能看出不是真凭据;
|
# 写进 settings 的假 Cookie。刻意用重复串,一眼就能看出不是真凭据;
|
||||||
# 作用只是让概览页的健康指示灯是绿的(首装状态是「缺 Cookie」告警)。
|
# 作用只是让概览页的健康指示灯是绿的(首装状态是「缺 Cookie」告警)。
|
||||||
|
# 两个账号给不同的值,这样「各自持有自己的凭证」在截图里看得出来。
|
||||||
DEMO_COOKIE = "wb_demo_session=" + "deadbeef" * 15
|
DEMO_COOKIE = "wb_demo_session=" + "deadbeef" * 15
|
||||||
|
DEMO_COOKIE_2 = "wb_demo_session=" + "cafef00d" * 15
|
||||||
|
|
||||||
|
# 第二个账号(普通用户)用的模型:刻意与管理员**不重名**,
|
||||||
|
# 这样「按账号隔离」在按模型的图上立刻可见。
|
||||||
|
MODELS_2 = [
|
||||||
|
("demo-lite", 30, 0.55),
|
||||||
|
("demo-nano", 26, 0.14),
|
||||||
|
("demo-flash", 22, 0.32),
|
||||||
|
("demo-vision", 12, 5.90),
|
||||||
|
("demo-pro", 10, 2.05),
|
||||||
|
]
|
||||||
|
|
||||||
PROMPTS_SHORT = [
|
PROMPTS_SHORT = [
|
||||||
"帮我解释一下这段代码的作用",
|
"帮我解释一下这段代码的作用",
|
||||||
@@ -144,51 +159,74 @@ def build(out_dir: str, days: int, seed: int, admin_password: str,
|
|||||||
os.environ.setdefault("WB_LOG_DIR", os.path.join(out_dir, "logs"))
|
os.environ.setdefault("WB_LOG_DIR", os.path.join(out_dir, "logs"))
|
||||||
os.environ["WB_DB"] = db_file
|
os.environ["WB_DB"] = db_file
|
||||||
|
|
||||||
from workbuddy_portal import db # noqa: E402
|
from workbuddy_portal import db, security # noqa: E402
|
||||||
|
|
||||||
conn = db.connect()
|
conn = db.connect()
|
||||||
try:
|
try:
|
||||||
db.init_db(conn, create_admin=True, admin_user=admin_user,
|
db.init_db(conn, create_admin=True, admin_user=admin_user,
|
||||||
admin_password=admin_password)
|
admin_password=admin_password)
|
||||||
|
|
||||||
# 写入一个**明显是假值**的 Cookie:让概览页的健康状态显示为「正常」
|
# ---------- 账号 ----------
|
||||||
|
admin_row = db.user_by_name(conn, admin_user)
|
||||||
|
admin_uid = admin_row["id"]
|
||||||
|
demo_user = "demo"
|
||||||
|
if not db.user_by_name(conn, demo_user):
|
||||||
|
conn.execute(
|
||||||
|
"INSERT INTO users(username,password_hash,display_name,email,is_admin,"
|
||||||
|
"status,created_at) VALUES(?,?,?,?,0,'active',?)",
|
||||||
|
(demo_user, security.hash_password(admin_password), "演示账号",
|
||||||
|
"demo@example.invalid", db.now_str()))
|
||||||
|
db_row = db.user_by_name(conn, demo_user)
|
||||||
|
demo_uid = db_row["id"]
|
||||||
|
|
||||||
|
# 写入**明显是假值**的 Cookie:让概览页的健康状态显示为「正常」
|
||||||
# 而不是首装的「缺 Cookie」告警——演示与截图应当呈现「配置完成」后的样子。
|
# 而不是首装的「缺 Cookie」告警——演示与截图应当呈现「配置完成」后的样子。
|
||||||
# 这个值不会被任何真实服务接受,也不含任何真实凭据。
|
# 经 set_secret 落库 = 真的走一遍加密,所以示例库里也是密文。
|
||||||
db.set_setting(conn, "cookie", DEMO_COOKIE)
|
db.set_secret(conn, "cookie", DEMO_COOKIE, admin_uid)
|
||||||
|
db.set_secret(conn, "cookie", DEMO_COOKIE_2, demo_uid)
|
||||||
|
# 两个账号各有一套调度时刻,界面上能看出「每人可改自己的」
|
||||||
|
db.set_setting(conn, "schedule_times", "09:00,17:00", admin_uid)
|
||||||
|
db.set_setting(conn, "schedule_times", "08:30,20:00", demo_uid)
|
||||||
|
|
||||||
rng = random.Random(seed)
|
rng = random.Random(seed)
|
||||||
now = datetime.now().replace(second=0, microsecond=0)
|
now = datetime.now().replace(second=0, microsecond=0)
|
||||||
today0 = now.replace(hour=0, minute=0, second=0)
|
today0 = now.replace(hour=0, minute=0, second=0)
|
||||||
model_pairs = [(m, w) for m, w, _ in MODELS]
|
model_pairs = [(m, w) for m, w, _ in MODELS]
|
||||||
medians = {m: md for m, _, md in MODELS}
|
medians = {m: md for m, _, md in MODELS}
|
||||||
|
model_pairs_2 = [(m, w) for m, w, _ in MODELS_2]
|
||||||
|
medians_2 = {m: md for m, _, md in MODELS_2}
|
||||||
client_pairs = list(CLIENTS)
|
client_pairs = list(CLIENTS)
|
||||||
|
|
||||||
# ---------- usage_records ----------
|
# ---------- usage_records(两个账号各生成一份)----------
|
||||||
rows = []
|
def _gen_records(uid, pairs, med, lo=14, hi=46):
|
||||||
for d in range(days - 1, -1, -1):
|
rows = []
|
||||||
day0 = today0 - timedelta(days=d)
|
for d in range(days - 1, -1, -1):
|
||||||
for _ in range(rng.randint(14, 46)):
|
day0 = today0 - timedelta(days=d)
|
||||||
# 工作时间加权:9-19 点更密
|
for _ in range(rng.randint(lo, hi)):
|
||||||
hour = _weighted(rng, [(h, 6 if 9 <= h <= 19 else 1) for h in range(24)])
|
# 工作时间加权:9-19 点更密
|
||||||
minute = rng.randint(0, 59)
|
hour = _weighted(rng, [(h, 6 if 9 <= h <= 19 else 1) for h in range(24)])
|
||||||
sec = rng.randint(0, 59)
|
ts = day0 + timedelta(hours=hour, minutes=rng.randint(0, 59),
|
||||||
ts = day0 + timedelta(hours=hour, minutes=minute, seconds=sec)
|
seconds=rng.randint(0, 59))
|
||||||
if ts > now:
|
if ts > now:
|
||||||
continue
|
continue
|
||||||
model = _weighted(rng, model_pairs)
|
model = _weighted(rng, pairs)
|
||||||
client = _weighted(rng, client_pairs)
|
client = _weighted(rng, client_pairs)
|
||||||
rid = "req-%s" % "".join(rng.choice("0123456789abcdef") for _ in range(16))
|
rid = "req-%s" % "".join(rng.choice("0123456789abcdef") for _ in range(16))
|
||||||
stamp = ts.strftime("%Y-%m-%d %H:%M:%S")
|
stamp = ts.strftime("%Y-%m-%d %H:%M:%S")
|
||||||
rows.append((
|
rows.append((
|
||||||
rid, stamp, ts.strftime("%Y-%m-%d"), hour, model, client,
|
uid, rid, stamp, ts.strftime("%Y-%m-%d"), hour, model, client,
|
||||||
_credits(rng, medians[model]), _prompt(rng),
|
_credits(rng, med[model]), _prompt(rng), stamp, stamp, stamp,
|
||||||
stamp, stamp, stamp,
|
))
|
||||||
))
|
return rows
|
||||||
|
|
||||||
|
rows = _gen_records(admin_uid, model_pairs, medians)
|
||||||
|
# 普通账号只给大约三分之二的量:列表里一眼能分出主次
|
||||||
|
rows += _gen_records(demo_uid, model_pairs_2, medians_2, lo=9, hi=31)
|
||||||
conn.execute("BEGIN")
|
conn.execute("BEGIN")
|
||||||
conn.executemany(
|
conn.executemany(
|
||||||
"INSERT OR REPLACE INTO usage_records"
|
"INSERT OR REPLACE INTO usage_records"
|
||||||
"(request_id,ts,day,hour,model,client,credits,prompt,"
|
"(user_id,request_id,ts,day,hour,model,client,credits,prompt,"
|
||||||
" first_seen,last_seen,cloud_ts) VALUES(?,?,?,?,?,?,?,?,?,?,?)", rows)
|
" first_seen,last_seen,cloud_ts) VALUES(?,?,?,?,?,?,?,?,?,?,?,?)", rows)
|
||||||
conn.execute("COMMIT")
|
conn.execute("COMMIT")
|
||||||
|
|
||||||
# ---------- collect_runs ----------
|
# ---------- collect_runs ----------
|
||||||
@@ -212,7 +250,7 @@ def build(out_dir: str, days: int, seed: int, admin_password: str,
|
|||||||
fetched = dup = added = 0
|
fetched = dup = added = 0
|
||||||
trigger = "schedule" if i % 3 else "manual"
|
trigger = "schedule" if i % 3 else "manual"
|
||||||
runs.append((
|
runs.append((
|
||||||
trigger, status, started.strftime("%Y-%m-%d %H:%M:%S"),
|
admin_uid, trigger, status, started.strftime("%Y-%m-%d %H:%M:%S"),
|
||||||
(started + timedelta(milliseconds=rng.randint(180, 1400))
|
(started + timedelta(milliseconds=rng.randint(180, 1400))
|
||||||
).strftime("%Y-%m-%d %H:%M:%S"),
|
).strftime("%Y-%m-%d %H:%M:%S"),
|
||||||
rng.randint(180, 1400),
|
rng.randint(180, 1400),
|
||||||
@@ -221,11 +259,20 @@ def build(out_dir: str, days: int, seed: int, admin_password: str,
|
|||||||
fetched, added, dup, total, 0, exit_code, msg,
|
fetched, added, dup, total, 0, exit_code, msg,
|
||||||
"[%s] %s" % (status, msg),
|
"[%s] %s" % (status, msg),
|
||||||
))
|
))
|
||||||
|
# 普通账号也给两条,让「日志只显示自己的」在截图里成立
|
||||||
|
for k, (st, msg) in enumerate((("ok", "新增 6 条,重复 4 条,存档共 192 条"),
|
||||||
|
("ok", "新增 3 条,重复 5 条,存档共 186 条"))):
|
||||||
|
at = now - timedelta(hours=5 + k * 9)
|
||||||
|
runs.append((demo_uid, "schedule", st, at.strftime("%Y-%m-%d %H:%M:%S"),
|
||||||
|
at.strftime("%Y-%m-%d %H:%M:%S"), 420,
|
||||||
|
(at - timedelta(days=1)).strftime("%Y-%m-%d %H:%M:%S"),
|
||||||
|
at.strftime("%Y-%m-%d %H:%M:%S"), 10 - k, 6 - k * 3, 4, 192, 0, 0,
|
||||||
|
msg, "[%s] %s" % (st, msg)))
|
||||||
conn.execute("BEGIN")
|
conn.execute("BEGIN")
|
||||||
conn.executemany(
|
conn.executemany(
|
||||||
"INSERT INTO collect_runs(trigger,status,started_at,finished_at,duration_ms,"
|
"INSERT INTO collect_runs(user_id,trigger,status,started_at,finished_at,duration_ms,"
|
||||||
"win_from,win_to,fetched,added,dup,total,conflicts,exit_code,message,detail)"
|
"win_from,win_to,fetched,added,dup,total,conflicts,exit_code,message,detail)"
|
||||||
" VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)", runs)
|
" VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)", runs)
|
||||||
conn.execute("COMMIT")
|
conn.execute("COMMIT")
|
||||||
|
|
||||||
# ---------- audit_log ----------
|
# ---------- audit_log ----------
|
||||||
@@ -235,18 +282,21 @@ def build(out_dir: str, days: int, seed: int, admin_password: str,
|
|||||||
actions = [
|
actions = [
|
||||||
("login", "登录成功", "192.0.2.10"),
|
("login", "登录成功", "192.0.2.10"),
|
||||||
("login", "登录成功", "192.0.2.10"),
|
("login", "登录成功", "192.0.2.10"),
|
||||||
("settings", "修改:调度时刻 09:00,17:00", "192.0.2.10"),
|
("settings", "修改:schedule_times", "192.0.2.10"),
|
||||||
("settings", "修改:分页大小 200", "192.0.2.10"),
|
("settings", "修改:page_size", "192.0.2.10"),
|
||||||
("maintenance.count", "存档当前 %d 条记录" % total, "192.0.2.10"),
|
("maintenance.count", "存档当前 %d 条记录" % total, "192.0.2.10"),
|
||||||
("settings_rejected", "参数错误:page_size 必须是数字(条/页)", "192.0.2.10"),
|
("settings_rejected", "参数错误:page_size 必须是数字(条/页)", "192.0.2.10"),
|
||||||
]
|
]
|
||||||
for i in range(30):
|
for i in range(30):
|
||||||
act, detail, ip = actions[i % len(actions)]
|
act, detail, ip = actions[i % len(actions)]
|
||||||
at = now - timedelta(hours=i * 3 + rng.randint(0, 2))
|
at = now - timedelta(hours=i * 3 + rng.randint(0, 2))
|
||||||
audits.append((at.strftime("%Y-%m-%d %H:%M:%S"), admin_user, act, detail, ip))
|
uid = admin_uid if i % 3 else demo_uid
|
||||||
|
actor = admin_user if i % 3 else demo_user
|
||||||
|
audits.append((uid, at.strftime("%Y-%m-%d %H:%M:%S"), actor, act, detail, ip))
|
||||||
conn.execute("BEGIN")
|
conn.execute("BEGIN")
|
||||||
conn.executemany(
|
conn.executemany(
|
||||||
"INSERT INTO audit_log(at,actor,action,detail,ip) VALUES(?,?,?,?,?)", audits)
|
"INSERT INTO audit_log(user_id,at,actor,action,detail,ip)"
|
||||||
|
" VALUES(?,?,?,?,?,?)", audits)
|
||||||
conn.execute("COMMIT")
|
conn.execute("COMMIT")
|
||||||
|
|
||||||
# 统计一下,便于打印
|
# 统计一下,便于打印
|
||||||
@@ -255,7 +305,13 @@ def build(out_dir: str, days: int, seed: int, admin_password: str,
|
|||||||
d = conn.execute("SELECT COUNT(DISTINCT day) FROM usage_records").fetchone()[0]
|
d = conn.execute("SELECT COUNT(DISTINCT day) FROM usage_records").fetchone()[0]
|
||||||
print("示例库:%s" % db_file)
|
print("示例库:%s" % db_file)
|
||||||
print(" 记录 %d 条 / 积分 %s / 覆盖 %d 天" % (n, c, d))
|
print(" 记录 %d 条 / 积分 %s / 覆盖 %d 天" % (n, c, d))
|
||||||
print(" 管理员 %s,口令 %s" % (admin_user, admin_password))
|
for r in conn.execute(
|
||||||
|
"SELECT u.id,u.username,u.is_admin,"
|
||||||
|
" (SELECT COUNT(*) FROM usage_records x WHERE x.user_id=u.id) AS n"
|
||||||
|
" FROM users u ORDER BY u.id"):
|
||||||
|
print(" 账号 #%s %-8s %-6s %d 条"
|
||||||
|
% (r["id"], r["username"], "管理员" if r["is_admin"] else "普通", r["n"]))
|
||||||
|
print(" 口令都是 %s(仅供本地演示)" % admin_password)
|
||||||
print(" 该目录在 .gitignore 内,不会被提交")
|
print(" 该目录在 .gitignore 内,不会被提交")
|
||||||
finally:
|
finally:
|
||||||
conn.close()
|
conn.close()
|
||||||
|
|||||||
@@ -6,33 +6,47 @@
|
|||||||
"""界面实检:登录后逐页截图,用来目视确认「统一美化」是否真的落地。
|
"""界面实检:登录后逐页截图,用来目视确认「统一美化」是否真的落地。
|
||||||
|
|
||||||
用法:
|
用法:
|
||||||
python manage.py serve --port 8849 --no-scheduler # 另开一个终端
|
WB_DATA_DIR=$PWD/data/demo python manage.py serve --port 8849 --no-scheduler
|
||||||
python tools/shots.py --base http://127.0.0.1:8849
|
python tools/shots.py --base http://127.0.0.1:8849 --db data/demo/usage.sqlite
|
||||||
python tools/shots.py --full # 整页长图(默认只截首屏)
|
python tools/shots.py --full # 整页长图(默认只截首屏)
|
||||||
|
|
||||||
产物:data/shots/*.png(已被 .gitignore 之外的目录,可直接删)。
|
产物:data/shots/*.png(该目录在 .gitignore 内,可直接删)。
|
||||||
|
|
||||||
为什么不用 headless chrome 直出:本项目的页面都要登录态,
|
为什么不用 headless chrome 直出:本项目的页面都要登录态,
|
||||||
`--screenshot` 无法注入会话 Cookie,所以必须用 Playwright 走一次真实登录。
|
`--screenshot` 无法注入会话 Cookie,所以必须用 Playwright 走一次真实登录。
|
||||||
|
|
||||||
|
验证码:默认策略是 always,所以本脚本会**从本地库里取答案**(取的是会话里的
|
||||||
|
captcha id,答案只存在于服务端),这样自动化能跨过验证码这一关。
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import argparse
|
import argparse
|
||||||
|
import base64
|
||||||
|
import json
|
||||||
import os
|
import os
|
||||||
|
import sqlite3
|
||||||
import sys
|
import sys
|
||||||
|
import urllib.parse
|
||||||
|
import zlib
|
||||||
|
|
||||||
BASE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
BASE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||||
sys.path.insert(0, BASE)
|
sys.path.insert(0, BASE)
|
||||||
|
|
||||||
PAGES = [
|
# 文件名刻意与原有编号保持一致(docs/USER-GUIDE.md 里就是按这些名字引用的),
|
||||||
("login", "/login", "登录页"),
|
# 新增页面排在后面,避免为了两张新图去改一堆文档链接。
|
||||||
("overview", "/", "概览"),
|
CAPTURES = [
|
||||||
("records", "/records", "数据明细"),
|
# (文件名, 路径, 标签, 是否需登录)
|
||||||
("tasks", "/tasks", "任务管理"),
|
("00-login.png", "/login", "登录页 (含图形验证码)", False),
|
||||||
("config", "/config", "配置管理"),
|
("09-register.png", "/register", "注册页", False),
|
||||||
("logs", "/logs", "日志管理"),
|
("01-overview.png", "/", "概览", True),
|
||||||
("users", "/users", "用户管理"),
|
("02-records.png", "/records", "数据明细", True),
|
||||||
("dashboard", "/dashboard", "用量大屏"),
|
("03-tasks.png", "/tasks", "任务管理", True),
|
||||||
|
("04-config.png", "/config", "配置管理", True),
|
||||||
|
("05-logs.png", "/logs", "日志管理", True),
|
||||||
|
("06-users.png", "/users", "用户管理", True),
|
||||||
|
("07-dashboard.png", "/dashboard", "用量大屏", True),
|
||||||
|
("10-profile.png", "/profile", "个人中心", True),
|
||||||
]
|
]
|
||||||
|
|
||||||
|
|
||||||
@@ -61,18 +75,59 @@ def _find_browser() -> str | None:
|
|||||||
return None
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _session_payload(ctx) -> dict:
|
||||||
|
"""解出 Flask 会话 cookie 里的载荷,只为拿 captcha 的 id。
|
||||||
|
|
||||||
|
Flask 会话是「签名 + base64,不加密」的,所以这里面能读出内容 ——
|
||||||
|
正因如此,验证码答案绝不能放进去(本项目只放一个随机 id)。
|
||||||
|
"""
|
||||||
|
for c in ctx.cookies():
|
||||||
|
if not c.get("name", "").startswith("workbuddy_portal_sid"):
|
||||||
|
continue
|
||||||
|
seg = urllib.parse.unquote(c.get("value", "")).split(".")[0]
|
||||||
|
seg += "=" * (-len(seg) % 4)
|
||||||
|
try:
|
||||||
|
raw = base64.urlsafe_b64decode(seg)
|
||||||
|
try:
|
||||||
|
raw = zlib.decompress(raw)
|
||||||
|
except zlib.error:
|
||||||
|
pass
|
||||||
|
return json.loads(raw.decode("utf-8"))
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
return {}
|
||||||
|
return {}
|
||||||
|
|
||||||
|
|
||||||
|
def _captcha_answer(ctx, db_path: str, purpose: str) -> str | None:
|
||||||
|
cid = _session_payload(ctx).get("cap_" + purpose)
|
||||||
|
if not cid or not db_path or not os.path.exists(db_path):
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
con = sqlite3.connect(db_path)
|
||||||
|
try:
|
||||||
|
row = con.execute("SELECT answer FROM captchas WHERE id=?", (cid,)).fetchone()
|
||||||
|
finally:
|
||||||
|
con.close()
|
||||||
|
except sqlite3.Error:
|
||||||
|
return None
|
||||||
|
return row[0] if row else None
|
||||||
|
|
||||||
|
|
||||||
def main() -> int:
|
def main() -> int:
|
||||||
ap = argparse.ArgumentParser()
|
ap = argparse.ArgumentParser()
|
||||||
ap.add_argument("--base", default="http://127.0.0.1:8849")
|
ap.add_argument("--base", default="http://127.0.0.1:8849")
|
||||||
ap.add_argument("-u", "--user", default="admin")
|
ap.add_argument("-u", "--user", default="admin")
|
||||||
ap.add_argument("-p", "--password", default="admin123")
|
ap.add_argument("-p", "--password", default="admin123")
|
||||||
ap.add_argument("--out", default=os.path.join(BASE, "data", "shots"))
|
ap.add_argument("--out", default=os.path.join(BASE, "data", "shots"))
|
||||||
|
ap.add_argument("--db", default=None,
|
||||||
|
help="SQLite 路径(默认 <repo>/data/usage.sqlite),用于取验证码答案")
|
||||||
ap.add_argument("--full", action="store_true", help="截整页长图")
|
ap.add_argument("--full", action="store_true", help="截整页长图")
|
||||||
ap.add_argument("--browser", default="", help="显式指定 chrome/msedge 可执行文件")
|
ap.add_argument("--browser", default="", help="显式指定 chrome/msedge 可执行文件")
|
||||||
ap.add_argument("--width", type=int, default=1440)
|
ap.add_argument("--width", type=int, default=1440)
|
||||||
ap.add_argument("--height", type=int, default=900)
|
ap.add_argument("--height", type=int, default=900)
|
||||||
a = ap.parse_args()
|
a = ap.parse_args()
|
||||||
|
|
||||||
|
db_path = a.db or os.path.join(BASE, "data", "usage.sqlite")
|
||||||
from playwright.sync_api import sync_playwright
|
from playwright.sync_api import sync_playwright
|
||||||
|
|
||||||
exe = a.browser or _find_browser()
|
exe = a.browser or _find_browser()
|
||||||
@@ -93,14 +148,35 @@ def main() -> int:
|
|||||||
page.on("console", lambda m: errors.append(m.text) if m.type == "error" else None)
|
page.on("console", lambda m: errors.append(m.text) if m.type == "error" else None)
|
||||||
page.on("pageerror", lambda e: errors.append(str(e)))
|
page.on("pageerror", lambda e: errors.append(str(e)))
|
||||||
|
|
||||||
# 1) 先截未登录的登录页
|
def shoot(name, path, label):
|
||||||
page.goto(a.base + "/login", wait_until="networkidle")
|
errors.clear()
|
||||||
page.screenshot(path=os.path.join(a.out, "00-login.png"), full_page=a.full)
|
page.goto(a.base + path, wait_until="networkidle")
|
||||||
print("[ok] 00-login.png")
|
page.wait_for_timeout(900) # 等 ECharts / 表格渲染稳下来
|
||||||
|
page.screenshot(path=os.path.join(a.out, name), full_page=a.full)
|
||||||
|
js_err = [e for e in errors if "favicon" not in e.lower()]
|
||||||
|
if js_err:
|
||||||
|
problems.append("%s: %s" % (label, js_err[:3]))
|
||||||
|
print("[ok] %-22s %s%s" % (name, label,
|
||||||
|
"" if not js_err else " [JS错误] " + " | ".join(js_err[:3])))
|
||||||
|
|
||||||
# 2) 登录
|
# 1) 未登录的两页
|
||||||
|
for name, path, label, need_auth in CAPTURES:
|
||||||
|
if need_auth:
|
||||||
|
break
|
||||||
|
shoot(name, path, label)
|
||||||
|
|
||||||
|
# 2) 登录(策略为 always 时自动解验证码)
|
||||||
|
page.goto(a.base + "/login", wait_until="networkidle")
|
||||||
page.fill('input[name=username]', a.user)
|
page.fill('input[name=username]', a.user)
|
||||||
page.fill('input[name=password]', a.password)
|
page.fill('input[name=password]', a.password)
|
||||||
|
if page.query_selector('input[name=captcha]'):
|
||||||
|
ans = _captcha_answer(ctx, db_path, "login")
|
||||||
|
if not ans:
|
||||||
|
print("[FAIL] 需要验证码但取不到答案(--db 是否指向本实例的库?):%s" % db_path)
|
||||||
|
br.close()
|
||||||
|
return 1
|
||||||
|
page.fill('input[name=captcha]', ans)
|
||||||
|
print("[ok] 已用库里的答案通过验证码")
|
||||||
page.click('button[type=submit]')
|
page.click('button[type=submit]')
|
||||||
page.wait_for_load_state("networkidle")
|
page.wait_for_load_state("networkidle")
|
||||||
if "/login" in page.url:
|
if "/login" in page.url:
|
||||||
@@ -108,18 +184,11 @@ def main() -> int:
|
|||||||
br.close()
|
br.close()
|
||||||
return 1
|
return 1
|
||||||
|
|
||||||
# 3) 逐页截图
|
# 3) 登录后的页面
|
||||||
for i, (slug, path, label) in enumerate(PAGES[1:], start=1):
|
for name, path, label, need_auth in CAPTURES:
|
||||||
errors.clear()
|
if not need_auth:
|
||||||
page.goto(a.base + path, wait_until="networkidle")
|
continue
|
||||||
page.wait_for_timeout(900) # 等 ECharts / 表格渲染稳下来
|
shoot(name, path, label)
|
||||||
page.screenshot(path=os.path.join(a.out, "%02d-%s.png" % (i, slug)),
|
|
||||||
full_page=a.full)
|
|
||||||
js_err = [e for e in errors if "favicon" not in e.lower()]
|
|
||||||
flag = "" if not js_err else " [JS错误] " + " | ".join(js_err[:3])
|
|
||||||
if js_err:
|
|
||||||
problems.append("%s: %s" % (label, js_err[:3]))
|
|
||||||
print("[ok] %02d-%s.png %s%s" % (i, slug, label, flag))
|
|
||||||
|
|
||||||
# 4) 大屏页再点几个交互,确认控件联动不炸
|
# 4) 大屏页再点几个交互,确认控件联动不炸
|
||||||
page.goto(a.base + "/dashboard", wait_until="networkidle")
|
page.goto(a.base + "/dashboard", wait_until="networkidle")
|
||||||
|
|||||||
@@ -13,9 +13,14 @@
|
|||||||
覆盖内容:
|
覆盖内容:
|
||||||
1. 全页面渲染(含 /users,需管理员身份)——模板报错会直接暴露成 500
|
1. 全页面渲染(含 /users,需管理员身份)——模板报错会直接暴露成 500
|
||||||
2. 模板未渲染残留(HTML 里出现 {{ / {% 说明有变量名写错)
|
2. 模板未渲染残留(HTML 里出现 {{ / {% 说明有变量名写错)
|
||||||
3. 历史缺陷防回归(见下 REGRESSIONS)
|
3. 历史缺陷防回归(见 4. 的 ①~⑭)
|
||||||
4. CSV 导出可被标准 csv 解析、列数一致
|
4. 多用户:数据隔离 / 凭证保密 / 注册与验证码 / 权限边界
|
||||||
5. 页面 HTML 里的 class 与 app.css 的选择器做差集(抓类名拼写错误)
|
5. CSV 导出可被标准 csv 解析、列数一致
|
||||||
|
6. 页面 HTML 里的 class 与 app.css 的选择器做差集(抓类名拼写错误)
|
||||||
|
|
||||||
|
写库说明:会写少量 audit_log 行;另外会**临时**建两个普通账号
|
||||||
|
(一个用来验权限边界,一个用来走完整注册链路),无论成功失败都在 finally 里删掉。
|
||||||
|
不会改动任何用量数据。
|
||||||
|
|
||||||
用法:
|
用法:
|
||||||
cd workbuddy-portal
|
cd workbuddy-portal
|
||||||
@@ -28,7 +33,9 @@ import csv
|
|||||||
import io
|
import io
|
||||||
import json
|
import json
|
||||||
import os
|
import os
|
||||||
|
import random
|
||||||
import re
|
import re
|
||||||
|
import string
|
||||||
import sys
|
import sys
|
||||||
|
|
||||||
BASE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
BASE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||||
@@ -57,14 +64,19 @@ def note(msg: str) -> None:
|
|||||||
print(" [note] %s" % msg)
|
print(" [note] %s" % msg)
|
||||||
|
|
||||||
|
|
||||||
def login(cli, admin=True):
|
def login(cli, uid: int, csrf: str = "smoke-csrf-token"):
|
||||||
"""注入会话绕过登录:GET 不触发 CSRF,因此可直接测页面渲染。"""
|
"""注入会话绕过登录:GET 不触发 CSRF,因此可直接测页面渲染。
|
||||||
|
|
||||||
|
只有 `uid` 是真正生效的键 —— `security.current_user()` 每请求回查
|
||||||
|
users 表(这样做是为了「停用账号立即失效」),所以 `uname/dname/adm`
|
||||||
|
只是写给自己看的标记,改不了权限。
|
||||||
|
"""
|
||||||
with cli.session_transaction() as s:
|
with cli.session_transaction() as s:
|
||||||
s["uid"] = 1
|
s["uid"] = uid
|
||||||
s["uname"] = "admin" if admin else "viewer"
|
s["uname"] = "smoke-%s" % uid
|
||||||
s["dname"] = "管理员" if admin else "只读账号"
|
s["dname"] = "smoke"
|
||||||
s["adm"] = 1 if admin else 0
|
s["adm"] = 0
|
||||||
s["_csrf"] = "smoke-csrf-token"
|
s["_csrf"] = csrf
|
||||||
|
|
||||||
|
|
||||||
def page(cli, path, method="GET", **kw):
|
def page(cli, path, method="GET", **kw):
|
||||||
@@ -72,36 +84,59 @@ def page(cli, path, method="GET", **kw):
|
|||||||
return r.status_code, r.get_data(as_text=True)
|
return r.status_code, r.get_data(as_text=True)
|
||||||
|
|
||||||
|
|
||||||
|
def _rand(n=8):
|
||||||
|
return "".join(random.choice(string.ascii_lowercase) for _ in range(n))
|
||||||
|
|
||||||
|
|
||||||
def run() -> None:
|
def run() -> None:
|
||||||
from workbuddy_portal import create_app, db, query
|
from workbuddy_portal import captcha, config, create_app, crypto, db, query, security
|
||||||
|
|
||||||
print("== 0. 构建应用 ==")
|
print("== 0. 构建应用 ==")
|
||||||
app = create_app(start_scheduler=False, do_init_db=False)
|
app = create_app(start_scheduler=False, do_init_db=False)
|
||||||
app.config["WTF_CSRF_ENABLED"] = False
|
app.config["WTF_CSRF_ENABLED"] = False
|
||||||
n_routes = len([r for r in app.url_map.iter_rules()])
|
n_routes = len([r for r in app.url_map.iter_rules()])
|
||||||
chk("create_app 成功", app is not None)
|
chk("create_app 成功", app is not None)
|
||||||
chk("路由数量 >= 35", n_routes >= 35, "routes=%d" % n_routes)
|
chk("路由数量 >= 40", n_routes >= 40, "routes=%d" % n_routes)
|
||||||
|
|
||||||
|
# 真实库里的账号:管理员必须有,普通账号按需临时造
|
||||||
|
conn = db.connect()
|
||||||
|
admin_row = conn.execute("SELECT id FROM users WHERE is_admin=1 AND status='active'"
|
||||||
|
" ORDER BY id LIMIT 1").fetchone()
|
||||||
|
if admin_row is None:
|
||||||
|
print("\n[FATAL] 库里没有启用的管理员账号,先跑 python manage.py init")
|
||||||
|
return
|
||||||
|
ADMIN = admin_row["id"]
|
||||||
|
|
||||||
# ---------------- 1. 未登录 ----------------
|
# ---------------- 1. 未登录 ----------------
|
||||||
print("== 1. 未登录:受保护页应跳登录、API 应 401 ==")
|
print("== 1. 未登录:受保护页应跳登录、API 应 401 ==")
|
||||||
with app.test_client() as cli:
|
with app.test_client() as cli:
|
||||||
for p in ("/", "/records", "/tasks", "/config", "/logs", "/users"):
|
for p in ("/", "/records", "/tasks", "/config", "/logs", "/users", "/profile"):
|
||||||
st, _ = page(cli, p)
|
st, _ = page(cli, p)
|
||||||
chk("GET %-10s 未登录=302" % p, st == 302, "status=%s" % st)
|
chk("GET %-10s 未登录=302" % p, st == 302, "status=%s" % st)
|
||||||
for p in ("/api/summary", "/api/users", "/api/settings", "/api/audit"):
|
for p in ("/api/summary", "/api/users", "/api/settings", "/api/audit"):
|
||||||
st, _ = page(cli, p)
|
st, _ = page(cli, p)
|
||||||
chk("GET %-14s 未登录=401" % p, st == 401, "status=%s" % st)
|
chk("GET %-14s 未登录=401" % p, st == 401, "status=%s" % st)
|
||||||
|
# 只认 POST 的接口:GET 应当 405(而不是落到 401 或被 GET 直接执行)
|
||||||
|
# 未登录的 POST 会被 before_request 里的 CSRF 先拦下(400)——
|
||||||
|
# 这比先鉴权更好:没有会话就不该被允许碰任何写接口。
|
||||||
|
for p in ("/api/profile", "/api/captcha"):
|
||||||
|
st, _ = page(cli, p)
|
||||||
|
chk("GET %-14s 未登录=405" % p, st == 405, "status=%s" % st)
|
||||||
|
st, _ = page(cli, p, method="POST")
|
||||||
|
chk("POST %-13s 无 CSRF=400" % p, st == 400, "status=%s" % st)
|
||||||
st, html = page(cli, "/login")
|
st, html = page(cli, "/login")
|
||||||
chk("登录页含 CSRF 隐藏域", 'name="_csrf"' in html)
|
chk("登录页含 CSRF 隐藏域", 'name="_csrf"' in html)
|
||||||
|
chk("登录页含验证码图", "capimg" in html and 'name="captcha"' in html)
|
||||||
|
chk("登录页含自助注册入口", "/register" in html)
|
||||||
|
|
||||||
# ---------------- 2. 管理员:全页面渲染 ----------------
|
# ---------------- 2. 管理员:全页面渲染 ----------------
|
||||||
print("== 2. 管理员:全页面渲染 ==")
|
print("== 2. 管理员:全页面渲染 ==")
|
||||||
with app.test_client() as cli:
|
with app.test_client() as cli:
|
||||||
login(cli, admin=True)
|
login(cli, ADMIN)
|
||||||
pages = [
|
pages = [
|
||||||
("/", "概览"), ("/records", "数据明细"), ("/tasks", "任务管理"),
|
("/", "概览"), ("/records", "数据明细"), ("/tasks", "任务管理"),
|
||||||
("/config", "配置管理"), ("/logs", "日志管理"), ("/users", "用户管理"),
|
("/config", "配置管理"), ("/logs", "日志管理"), ("/users", "用户管理"),
|
||||||
("/dashboard", "<html"),
|
("/profile", "个人中心"), ("/dashboard", "<html"),
|
||||||
]
|
]
|
||||||
for p, kw in pages:
|
for p, kw in pages:
|
||||||
st, html = page(cli, p)
|
st, html = page(cli, p)
|
||||||
@@ -112,24 +147,44 @@ def run() -> None:
|
|||||||
chk(" └ 含导航栏", "topbar" in html or p == "/dashboard")
|
chk(" └ 含导航栏", "topbar" in html or p == "/dashboard")
|
||||||
|
|
||||||
st, html = page(cli, "/users")
|
st, html = page(cli, "/users")
|
||||||
chk("用户管理页列出账号", 'data-uid=' in html, "含行内编辑按钮")
|
chk("用户管理页列出账号", 'data-uid=' in html)
|
||||||
chk("用户管理页含新建表单", 'id="formNewUser"' in html)
|
chk("用户管理页含新建表单", 'id="formNewUser"' in html)
|
||||||
chk("用户管理页含审计表", "用户操作审计" in html)
|
chk("用户管理页含账号操作审计", "账号操作审计" in html)
|
||||||
|
chk("用户管理页含状态列", "status" in html and "停用" in html)
|
||||||
|
chk("用户管理页含邮箱列", "邮箱" in html)
|
||||||
|
|
||||||
# 配置页的维护按钮 + 大屏回后台入口
|
|
||||||
st, cfg = page(cli, "/config")
|
st, cfg = page(cli, "/config")
|
||||||
chk("配置页含维护按钮组", cfg.count("data-maint=") >= 3, "n=%d" % cfg.count("data-maint="))
|
chk("配置页含维护按钮组", cfg.count("data-maint=") >= 3, "n=%d" % cfg.count("data-maint="))
|
||||||
chk("配置页含 TLS 校验下拉", 'name="ssl_verify"' in cfg)
|
chk("配置页含 TLS 校验下拉", 'name="ssl_verify"' in cfg)
|
||||||
|
chk("配置页含实例级设置区", "仅管理员可改" in cfg)
|
||||||
|
chk("配置页显示凭证状态而非明文", "cookie_hint" in cfg or "字符" in cfg)
|
||||||
st, rec = page(cli, "/records")
|
st, rec = page(cli, "/records")
|
||||||
chk("明细页含快捷区间", 'data-range="today"' in rec and 'data-range="30d"' in rec)
|
chk("明细页含快捷区间", 'data-range="today"' in rec and 'data-range="30d"' in rec)
|
||||||
chk("明细页表格包在 .tablewrap", "tablewrap" in rec)
|
chk("明细页表格包在 .tablewrap", "tablewrap" in rec)
|
||||||
|
st, pf = page(cli, "/profile")
|
||||||
|
chk("个人中心含改密表单", "/api/password" in pf or "password" in pf)
|
||||||
|
chk("个人中心说明凭证归属本人", "本人凭证" in pf or "我的 Cookie" in pf)
|
||||||
|
# 防回归:base.html 里曾用 {% set me = current_user() %},把子模板的 me
|
||||||
|
# 覆盖掉了 —— current_user() 只有 id/username/display_name/is_admin,
|
||||||
|
# 于是「注册于」渲染成空。变量已改名 cur,这里把两处都盯住。
|
||||||
|
lead = re.search(r'<p class="lead">(.*?)</p>', pf, re.S)
|
||||||
|
lead_txt = " ".join(lead.group(1).split()) if lead else ""
|
||||||
|
chk("个人中心「注册于」有真实时间",
|
||||||
|
bool(re.search(r"注册于\s+\d{4}-\d{2}-\d{2}", lead_txt)), lead_txt[:80])
|
||||||
|
chk("个人中心「最近登录」不是空占位",
|
||||||
|
bool(re.search(r"最近登录\s+\S", lead_txt)), lead_txt[:80])
|
||||||
|
chk("base.html 未再用 me 作局部变量(防覆盖子模板)",
|
||||||
|
"set me = " not in open(os.path.join(
|
||||||
|
BASE, "workbuddy_portal", "web", "templates", "base.html"),
|
||||||
|
encoding="utf-8").read())
|
||||||
|
|
||||||
# ---------------- 2b. 静态资源引用可解析 ----------------
|
# ---------------- 2b. 静态资源引用可解析 ----------------
|
||||||
print("== 2b. 页面引用的静态资源全部可达 ==")
|
print("== 2b. 页面引用的静态资源全部可达 ==")
|
||||||
asset_re = re.compile(r"\.(?:js|css|svg|png|jpe?g|gif|webp|ico|woff2?)(?:\?|$)", re.I)
|
asset_re = re.compile(r"\.(?:js|css|svg|png|jpe?g|gif|webp|ico|woff2?)(?:\?|$)", re.I)
|
||||||
with app.test_client() as cli:
|
with app.test_client() as cli:
|
||||||
login(cli, admin=True)
|
login(cli, ADMIN)
|
||||||
for p in ("/", "/records", "/tasks", "/config", "/logs", "/users", "/dashboard"):
|
for p in ("/", "/records", "/tasks", "/config", "/logs", "/users",
|
||||||
|
"/profile", "/dashboard"):
|
||||||
_, html = page(cli, p)
|
_, html = page(cli, p)
|
||||||
# 先剥掉 HTML 注释:注释里常写示例路径(src="vendor/x.js"),
|
# 先剥掉 HTML 注释:注释里常写示例路径(src="vendor/x.js"),
|
||||||
# 不剥会把示例当真实引用误报。
|
# 不剥会把示例当真实引用误报。
|
||||||
@@ -150,25 +205,205 @@ def run() -> None:
|
|||||||
chk("%-11s 资源引用全部 200" % p, not bad,
|
chk("%-11s 资源引用全部 200" % p, not bad,
|
||||||
("坏引用=%s" % bad) if bad else "%d 个引用" % n)
|
("坏引用=%s" % bad) if bad else "%d 个引用" % n)
|
||||||
|
|
||||||
# ---------------- 3. 非管理员:权限边界 ----------------
|
# ---------------- 3. 多用户:隔离 / 保密 / 注册与验证码 ----------------
|
||||||
print("== 3. 非管理员:/users 必须 403,导航不出现该入口 ==")
|
print("== 3. 多用户:数据隔离 / 凭证保密 / 注册与验证码 ==")
|
||||||
with app.test_client() as cli:
|
viewer = "smoke_v_%s" % _rand()
|
||||||
login(cli, admin=False)
|
regged = "smoke_r_%s" % _rand()
|
||||||
st, html = page(cli, "/users")
|
created: list[str] = [viewer]
|
||||||
chk("GET /users 非管理员=403", st == 403, "status=%s" % st)
|
saved_ua_inst = None
|
||||||
st, _ = page(cli, "/api/users")
|
|
||||||
chk("GET /api/users 非管理员=403", st == 403, "status=%s" % st)
|
|
||||||
st, _ = page(cli, "/api/users")
|
|
||||||
st, html = page(cli, "/")
|
|
||||||
chk("概览导航不含「用户管理」", "用户管理" not in html)
|
|
||||||
for p in ("/", "/records", "/tasks", "/logs"):
|
|
||||||
st, _ = page(cli, p)
|
|
||||||
chk("GET %-10s 非管理员=200" % p, st == 200, "status=%s" % st)
|
|
||||||
|
|
||||||
# ---------------- 4. 历史缺陷防回归 ----------------
|
def _mk_user(name):
|
||||||
print("== 4. 历史缺陷防回归 ==")
|
conn.execute("INSERT INTO users(username,password_hash,display_name,is_admin,"
|
||||||
|
"status,created_at) VALUES(?,?,?,0,'active',?)",
|
||||||
|
(name, security.hash_password("Smoke-Pass1"), "冒烟账号", db.now_str()))
|
||||||
|
return conn.execute("SELECT id FROM users WHERE username=?", (name,)).fetchone()["id"]
|
||||||
|
|
||||||
|
try:
|
||||||
|
VIEWER = _mk_user(viewer)
|
||||||
|
|
||||||
|
# ① 凭证密文入库
|
||||||
|
row = conn.execute("SELECT value FROM settings WHERE key='cookie' AND user_id=?",
|
||||||
|
(ADMIN,)).fetchone()
|
||||||
|
if row and row["value"]:
|
||||||
|
chk("① Cookie 以密文入库(v1. 前缀)", crypto.is_encrypted(row["value"]),
|
||||||
|
"head=%s" % row["value"][:12])
|
||||||
|
chk("① 密文不含明文片段",
|
||||||
|
crypto.is_encrypted(row["value"]) and ";" not in row["value"][:4])
|
||||||
|
plain = db.get_secret(conn, "cookie", ADMIN)
|
||||||
|
chk("① 解回来长度合理(>100 字符)", len(plain) > 100, "chars=%d" % len(plain))
|
||||||
|
else:
|
||||||
|
note("库里没有 Cookie,跳过密文断言")
|
||||||
|
|
||||||
|
# ② 凭证绝不跨账号回落
|
||||||
|
chk("② NO_FALLBACK_KEYS 含 cookie/user_agent",
|
||||||
|
{"cookie", "user_agent"} <= db.NO_FALLBACK_KEYS)
|
||||||
|
chk("② 新账号读不到别人的 Cookie", db.get_secret(conn, "cookie", VIEWER) == "")
|
||||||
|
# User-Agent 本身不是秘密,新账号拿到 DEFAULTS 里的**通用** UA 是对的;
|
||||||
|
# 要守住的是「不能继承别人存下来的那一份」。用一个哨兵值把这点钉死:
|
||||||
|
sentinel = "SMOKE-SENTINEL-UA/%s" % _rand()
|
||||||
|
saved_ua_inst = db.get_setting(conn, "user_agent", "", 0)
|
||||||
|
db.set_setting(conn, "user_agent", sentinel, 0) # 写实例级
|
||||||
|
chk("② 实例级放哨兵后,新账号仍看不到它",
|
||||||
|
db.get_setting(conn, "user_agent", "", VIEWER) != sentinel,
|
||||||
|
"new=%s…" % (db.get_setting(conn, "user_agent", "", VIEWER) or "")[:22])
|
||||||
|
chk("② 哨兵在实例级确实生效(证明上面的断言不是在空跑)",
|
||||||
|
db.get_setting(conn, "user_agent", "", 0) == sentinel)
|
||||||
|
db.set_setting(conn, "user_agent", saved_ua_inst, 0) # 还原
|
||||||
|
chk("② 已还原实例级 UA", db.get_setting(conn, "user_agent", "", 0) == saved_ua_inst)
|
||||||
|
# 普通配置应当能回落到实例级(否则每个新账号都拿到空配置)
|
||||||
|
chk("② 普通配置仍回落实例级",
|
||||||
|
db.get_setting(conn, "page_size", None, VIEWER) ==
|
||||||
|
db.get_setting(conn, "page_size", None, 0))
|
||||||
|
|
||||||
|
# ③ /api/settings 只给掩码,绝不给明文
|
||||||
|
with app.test_client() as cli:
|
||||||
|
login(cli, ADMIN)
|
||||||
|
st, body = page(cli, "/api/settings")
|
||||||
|
j = json.loads(body)
|
||||||
|
chk("③ /api/settings 200", st == 200, "status=%s" % st)
|
||||||
|
plain = db.get_secret(conn, "cookie", ADMIN)
|
||||||
|
chk("③ 响应体不含 Cookie 明文", not plain or plain not in body)
|
||||||
|
chk("③ cookie 字段被置空", not (j.get("cookie") or "").strip())
|
||||||
|
chk("③ 只给 cookie_hint 掩码", "cookie_hint" in j and "cookie_broken" in j)
|
||||||
|
chk("③ 标注实例级键清单", isinstance(j.get("_globalKeys"), list) and j["_globalKeys"])
|
||||||
|
chk("③ 管理员 _canEditGlobal=True", j.get("_canEditGlobal") is True)
|
||||||
|
chk("③ 无 slot:* 内部键", "slot:" not in body)
|
||||||
|
|
||||||
|
# ④ 验证码:不落 session、一次性、出图禁缓存
|
||||||
|
with app.test_client() as cli:
|
||||||
|
r = cli.get("/captcha.png?purpose=login")
|
||||||
|
chk("④ /captcha.png=200", r.status_code == 200, "status=%s" % r.status_code)
|
||||||
|
chk("④ 是 PNG 字节流",
|
||||||
|
r.headers.get("Content-Type", "").startswith("image/png")
|
||||||
|
and r.get_data()[:8] == b"\x89PNG\r\n\x1a\n")
|
||||||
|
chk("④ 出图禁缓存", "no-store" in r.headers.get("Cache-Control", ""))
|
||||||
|
with cli.session_transaction() as s:
|
||||||
|
cid = s.get("cap_login")
|
||||||
|
chk("④ 会话里只存验证码 id", bool(cid), "id=%s" % (cid or "无"))
|
||||||
|
ans = conn.execute("SELECT answer,purpose FROM captchas WHERE id=?",
|
||||||
|
(cid,)).fetchone() if cid else None
|
||||||
|
chk("④ 答案只存在服务端 captchas 表",
|
||||||
|
ans is not None and len(ans["answer"]) >= 4 and ans["purpose"] == "login")
|
||||||
|
if ans:
|
||||||
|
st, html = page(cli, "/login")
|
||||||
|
chk("④ 页面 HTML 里搜不到答案", ans["answer"] not in html)
|
||||||
|
chk("④ 页面 JS 里也搜不到会话密钥", cid not in html)
|
||||||
|
# 一次性:同一个 id 用两次,第二次必须失败
|
||||||
|
if ans:
|
||||||
|
chk("④ 首次校验通过",
|
||||||
|
captcha.verify(conn, cid, ans["answer"], "login"))
|
||||||
|
chk("④ 同 id 二次校验失败(已消费)",
|
||||||
|
not captcha.verify(conn, cid, ans["answer"], "login"))
|
||||||
|
chk("④ 消费后记录已删除",
|
||||||
|
conn.execute("SELECT COUNT(*) FROM captchas WHERE id=?",
|
||||||
|
(cid,)).fetchone()[0] == 0)
|
||||||
|
|
||||||
|
# ⑤ 自助注册全链路(取答案 -> POST /register -> 账号可用)
|
||||||
|
with app.test_client() as cli:
|
||||||
|
cli.get("/register")
|
||||||
|
# 验证码图是浏览器去取的,test_client 不会自动加载 <img>,
|
||||||
|
# 所以这里显式打一次 —— 这一步正是「注册页有没有发挑战」的验证
|
||||||
|
r = cli.get("/captcha.png?purpose=register")
|
||||||
|
chk("⑤ 注册页的验证码接口可用", r.status_code == 200
|
||||||
|
and r.get_data()[:4] == b"\x89PNG", "status=%s" % r.status_code)
|
||||||
|
with cli.session_transaction() as s:
|
||||||
|
cid = s.get("cap_register")
|
||||||
|
# 这个客户端没有走 login() 注入固定 token,所以要取真实值;
|
||||||
|
# 顺手也证明了 /register 的 CSRF 校验确实在生效
|
||||||
|
csrf = s.get("_csrf")
|
||||||
|
a2 = conn.execute("SELECT answer,purpose FROM captchas WHERE id=?",
|
||||||
|
(cid,)).fetchone() if cid else None
|
||||||
|
chk("⑤ 注册用的挑战落在 register 用途下",
|
||||||
|
a2 is not None and a2["purpose"] == "register")
|
||||||
|
if a2:
|
||||||
|
st, _ = page(cli, "/register", method="POST", data={
|
||||||
|
"username": regged, "display_name": "冒烟注册", "email": "",
|
||||||
|
"password": "Smoke-Pass1", "password2": "Smoke-Pass1",
|
||||||
|
"captcha": a2["answer"]},
|
||||||
|
headers={"X-CSRF-Token": csrf or ""})
|
||||||
|
chk("⑤ 注册成功=302", st == 302, "status=%s" % st)
|
||||||
|
created.append(regged)
|
||||||
|
u = conn.execute("SELECT id,is_admin,status,display_name,last_login_ip"
|
||||||
|
" FROM users WHERE username=?", (regged,)).fetchone()
|
||||||
|
chk("⑤ 建出的是普通账号",
|
||||||
|
u is not None and u["is_admin"] == 0 and u["status"] == "active")
|
||||||
|
chk("⑤ 注册即登录(会话已建立)",
|
||||||
|
u is not None and u["id"] == db.user_by_name(conn, regged)["id"])
|
||||||
|
# 缺 CSRF 必须 400
|
||||||
|
st, _ = page(cli, "/register", method="POST", data={"username": "x" * 3})
|
||||||
|
chk("⑤ 注册缺 CSRF=400", st == 400, "status=%s" % st)
|
||||||
|
|
||||||
|
# ⑥ 权限边界:普通账号改不了实例级配置
|
||||||
|
with app.test_client() as cli:
|
||||||
|
login(cli, VIEWER)
|
||||||
|
st, j = page(cli, "/api/settings")
|
||||||
|
chk("⑥ 非管理员 _canEditGlobal=False", json.loads(j).get("_canEditGlobal") is False)
|
||||||
|
evil = "http://evil.invalid"
|
||||||
|
st, _ = page(cli, "/api/settings", method="POST", json={"api_base": evil},
|
||||||
|
headers={"X-CSRF-Token": "smoke-csrf-token"})
|
||||||
|
chk("⑥ 非管理员改实例级配置=400", st == 400, "status=%s" % st)
|
||||||
|
chk("⑥ 且确实没写进去",
|
||||||
|
db.get_setting(conn, "api_base", "", VIEWER) != evil
|
||||||
|
and db.get_setting(conn, "api_base", "", 0) != evil)
|
||||||
|
|
||||||
|
# ⑦ 数据隔离:所有查询函数都必须显式带 uid
|
||||||
|
try:
|
||||||
|
query.daily(conn)
|
||||||
|
chk("⑦ query.daily 漏传 uid 会报错", False, "居然没报错")
|
||||||
|
except TypeError:
|
||||||
|
chk("⑦ query.daily 漏传 uid 会报错", True)
|
||||||
|
d_admin = query.daily(conn, ADMIN)
|
||||||
|
d_viewer = query.daily(conn, VIEWER)
|
||||||
|
chk("⑦ 不同账号的 daily 互不相同",
|
||||||
|
not d_admin or d_viewer != d_admin or len(d_viewer) == 0)
|
||||||
|
chk("⑦ 新账号 totals 为空", query.totals(conn, VIEWER)["records"] == 0)
|
||||||
|
t_admin = query.totals(conn, ADMIN)
|
||||||
|
chk("⑦ 管理员 totals 有数据", t_admin["records"] > 0, "records=%d" % t_admin["records"])
|
||||||
|
chk("⑦ totals(uid=0) 不含任何人的数据",
|
||||||
|
query.totals(conn, 0)["records"] == 0)
|
||||||
|
finally:
|
||||||
|
# 哨兵 UA 一定要还原(否则下次真采集会带着测试字符串发出去)
|
||||||
|
if saved_ua_inst is not None:
|
||||||
|
db.set_setting(conn, "user_agent", saved_ua_inst, 0)
|
||||||
|
for name in created:
|
||||||
|
conn.execute("DELETE FROM users WHERE username=?", (name,))
|
||||||
|
conn.execute("DELETE FROM settings WHERE user_id NOT IN (SELECT id FROM users)")
|
||||||
|
conn.execute("DELETE FROM usage_records WHERE user_id NOT IN (SELECT id FROM users)")
|
||||||
|
|
||||||
|
# 确认清理干净
|
||||||
|
left = conn.execute("SELECT COUNT(*) FROM users WHERE username LIKE 'smoke\\_%' ESCAPE '\\'"
|
||||||
|
).fetchone()[0]
|
||||||
|
chk("3. 临时账号已清理", left == 0, "残留=%d" % left)
|
||||||
|
|
||||||
|
# ---------------- 4. 普通账号的权限边界 ----------------
|
||||||
|
print("== 4. 非管理员:/users 必须 403,导航不出现该入口 ==")
|
||||||
|
viewer2 = "smoke_w_%s" % _rand()
|
||||||
|
try:
|
||||||
|
V2 = _mk_user(viewer2)
|
||||||
|
with app.test_client() as cli:
|
||||||
|
login(cli, V2)
|
||||||
|
st, html = page(cli, "/users")
|
||||||
|
chk("GET /users 非管理员=403", st == 403, "status=%s" % st)
|
||||||
|
st, _ = page(cli, "/api/users")
|
||||||
|
chk("GET /api/users 非管理员=403", st == 403, "status=%s" % st)
|
||||||
|
st, html = page(cli, "/")
|
||||||
|
chk("概览导航不含「用户管理」", "用户管理" not in html)
|
||||||
|
chk("普通账号导航含「个人中心」入口", 'class="who"' in html)
|
||||||
|
for p in ("/", "/records", "/tasks", "/logs", "/config", "/profile"):
|
||||||
|
st, _ = page(cli, p)
|
||||||
|
chk("GET %-10s 非管理员=200" % p, st == 200, "status=%s" % st)
|
||||||
|
# 日志尾部是管理员专属
|
||||||
|
st, _ = page(cli, "/logs/tail?lines=10")
|
||||||
|
chk("GET /logs/tail 非管理员=403", st == 403, "status=%s" % st)
|
||||||
|
finally:
|
||||||
|
conn.execute("DELETE FROM users WHERE username=?", (viewer2,))
|
||||||
|
conn.execute("DELETE FROM settings WHERE user_id NOT IN (SELECT id FROM users)")
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
# ---------------- 5. 历史缺陷防回归 ----------------
|
||||||
|
print("== 5. 历史缺陷防回归 ==")
|
||||||
with app.test_client() as cli:
|
with app.test_client() as cli:
|
||||||
login(cli, admin=True)
|
login(cli, ADMIN)
|
||||||
# ① 非法日期曾 500
|
# ① 非法日期曾 500
|
||||||
st, body = page(cli, "/api/summary?from=abc&to=def")
|
st, body = page(cli, "/api/summary?from=abc&to=def")
|
||||||
chk("① /api/summary 非法日期=400", st == 400, "status=%s" % st)
|
chk("① /api/summary 非法日期=400", st == 400, "status=%s" % st)
|
||||||
@@ -222,8 +457,8 @@ def run() -> None:
|
|||||||
st, body = page(cli, "/api/maintenance/recount", method="POST", json={},
|
st, body = page(cli, "/api/maintenance/recount", method="POST", json={},
|
||||||
headers={"X-CSRF-Token": "smoke-csrf-token"})
|
headers={"X-CSRF-Token": "smoke-csrf-token"})
|
||||||
chk("⑩ recount=200", st == 200, "status=%s body=%s" % (st, body[:90]))
|
chk("⑩ recount=200", st == 200, "status=%s body=%s" % (st, body[:90]))
|
||||||
# ⑪ 非管理员调用户管理 API
|
# ⑪ 管理员不能删自己
|
||||||
st, _ = page(cli, "/api/users/1/delete", method="POST", json={},
|
st, _ = page(cli, "/api/users/%d/delete" % ADMIN, method="POST", json={},
|
||||||
headers={"X-CSRF-Token": "smoke-csrf-token"})
|
headers={"X-CSRF-Token": "smoke-csrf-token"})
|
||||||
chk("⑪ 删除自己=400(不允许)", st == 400, "status=%s" % st)
|
chk("⑪ 删除自己=400(不允许)", st == 400, "status=%s" % st)
|
||||||
# ⑫ CSRF 缺失必须 400
|
# ⑫ CSRF 缺失必须 400
|
||||||
@@ -251,10 +486,10 @@ def run() -> None:
|
|||||||
chk("⑭ 审计筛选结果不含其他动作", not others and bool(tags),
|
chk("⑭ 审计筛选结果不含其他动作", not others and bool(tags),
|
||||||
"命中=%d 混入=%s" % (len(tags), others))
|
"命中=%d 混入=%s" % (len(tags), others))
|
||||||
|
|
||||||
# ---------------- 5. 数据自洽 ----------------
|
# ---------------- 6. 数据自洽 ----------------
|
||||||
print("== 5. 数据自洽(只读) ==")
|
print("== 6. 数据自洽(只读) ==")
|
||||||
with app.test_client() as cli:
|
with app.test_client() as cli:
|
||||||
login(cli, admin=True)
|
login(cli, ADMIN)
|
||||||
mf = json.loads(page(cli, "/api/manifest")[1])
|
mf = json.loads(page(cli, "/api/manifest")[1])
|
||||||
src = (mf.get("sources") or [{}])[0]
|
src = (mf.get("sources") or [{}])[0]
|
||||||
chk("manifest 存档条数 == 数据源条数",
|
chk("manifest 存档条数 == 数据源条数",
|
||||||
@@ -267,29 +502,32 @@ def run() -> None:
|
|||||||
chk("summary 全量 credits 自洽",
|
chk("summary 全量 credits 自洽",
|
||||||
abs(float(sm.get("credits", 0)) - float(mf["totals"]["credits"])) < 0.005,
|
abs(float(sm.get("credits", 0)) - float(mf["totals"]["credits"])) < 0.005,
|
||||||
"%s vs %s" % (sm.get("credits"), mf["totals"]["credits"]))
|
"%s vs %s" % (sm.get("credits"), mf["totals"]["credits"]))
|
||||||
d = query.daily(db.get_db())
|
d = query.daily(db.get_db(), ADMIN)
|
||||||
chk("daily 逐日积分求和 == 存档总额",
|
chk("daily 逐日积分求和 == 存档总额",
|
||||||
abs(round(sum(float(x["c"]) for x in d), 2)
|
abs(round(sum(float(x["c"]) for x in d), 2)
|
||||||
- round(float(mf["totals"]["credits"]), 2)) < 0.005)
|
- round(float(mf["totals"]["credits"]), 2)) < 0.005)
|
||||||
chk("daily 逐日 h[24] 求和 == 当日积分",
|
chk("daily 逐日 h[24] 求和 == 当日积分",
|
||||||
all(abs(round(sum(x["h"]), 2) - round(x["c"], 2)) < 0.005 for x in d))
|
all(abs(round(sum(x["h"]), 2) - round(x["c"], 2)) < 0.005 for x in d))
|
||||||
note("存档 %s 条 / %s 积分 / %d 天" % (mf["totals"]["records"],
|
note("管理员存档 %s 条 / %s 积分 / %d 天" % (mf["totals"]["records"],
|
||||||
mf["totals"]["credits"], len(d)))
|
mf["totals"]["credits"], len(d)))
|
||||||
|
|
||||||
# ---------------- 6. class 名与 CSS 选择器对账 ----------------
|
# ---------------- 7. class 名与 CSS 选择器对账 ----------------
|
||||||
print("== 6. 页面 class 与 app.css 选择器对账 ==")
|
print("== 7. 页面 class 与 app.css 选择器对账 ==")
|
||||||
css = open(os.path.join(BASE, "workbuddy_portal", "web", "static", "css", "app.css"),
|
css = open(os.path.join(BASE, "workbuddy_portal", "web", "static", "css", "app.css"),
|
||||||
encoding="utf-8").read()
|
encoding="utf-8").read()
|
||||||
css_classes = set(re.findall(r"\.([A-Za-z][\w-]*)", css))
|
css_classes = set(re.findall(r"\.([A-Za-z][\w-]*)", css))
|
||||||
|
anon = ("/login", "/register")
|
||||||
|
auth = ("/", "/records", "/tasks", "/config", "/logs", "/users", "/profile")
|
||||||
|
used: set[str] = set()
|
||||||
|
for p in anon:
|
||||||
|
with app.test_client() as c2:
|
||||||
|
html = page(c2, p)[1]
|
||||||
|
for m in re.findall(r'class="([^"]*)"', html):
|
||||||
|
used.update(t for t in m.split() if t)
|
||||||
with app.test_client() as cli:
|
with app.test_client() as cli:
|
||||||
login(cli, admin=True)
|
login(cli, ADMIN)
|
||||||
used: set[str] = set()
|
for p in auth:
|
||||||
for p in ("/", "/records", "/tasks", "/config", "/logs", "/users", "/login"):
|
html = page(cli, p)[1]
|
||||||
if p == "/login":
|
|
||||||
with app.test_client() as c2:
|
|
||||||
html = page(c2, p)[1]
|
|
||||||
else:
|
|
||||||
html = page(cli, p)[1]
|
|
||||||
for m in re.findall(r'class="([^"]*)"', html):
|
for m in re.findall(r'class="([^"]*)"', html):
|
||||||
used.update(t for t in m.split() if t)
|
used.update(t for t in m.split() if t)
|
||||||
# 允许的无样式类:JS 钩子、第三方/语义标记
|
# 允许的无样式类:JS 钩子、第三方/语义标记
|
||||||
|
|||||||
@@ -25,7 +25,7 @@ from flask import Flask, jsonify, render_template, request
|
|||||||
|
|
||||||
from . import config, db, security
|
from . import config, db, security
|
||||||
|
|
||||||
__version__ = "1.1.0"
|
__version__ = "1.2.0"
|
||||||
PROJECT_NAME = config.PROJECT_NAME
|
PROJECT_NAME = config.PROJECT_NAME
|
||||||
|
|
||||||
|
|
||||||
@@ -51,8 +51,14 @@ def create_app(start_scheduler=True, do_init_db=True, **overrides):
|
|||||||
app.config.update(
|
app.config.update(
|
||||||
SECRET_KEY=config.secret_key(),
|
SECRET_KEY=config.secret_key(),
|
||||||
PERMANENT_SESSION_LIFETIME=timedelta(hours=config.SESSION_HOURS),
|
PERMANENT_SESSION_LIFETIME=timedelta(hours=config.SESSION_HOURS),
|
||||||
|
# ---- 会话 Cookie 加固 ----
|
||||||
|
# HttpOnly:JS 读不到(XSS 也别想直接偷走会话)
|
||||||
SESSION_COOKIE_HTTPONLY=True,
|
SESSION_COOKIE_HTTPONLY=True,
|
||||||
SESSION_COOKIE_SAMESITE="Lax",
|
SESSION_COOKIE_SAMESITE="Lax",
|
||||||
|
# Secure:仅 HTTPS 下发。纯局域网 HTTP 部署必须留 0,否则浏览器
|
||||||
|
# 根本不会回传 Cookie,表现为「刚登录完又被弹回登录页」。
|
||||||
|
SESSION_COOKIE_SECURE=config.COOKIE_SECURE,
|
||||||
|
SESSION_COOKIE_PATH="/",
|
||||||
SESSION_COOKIE_NAME="workbuddy_portal_sid",
|
SESSION_COOKIE_NAME="workbuddy_portal_sid",
|
||||||
MAX_CONTENT_LENGTH=4 * 1024 * 1024,
|
MAX_CONTENT_LENGTH=4 * 1024 * 1024,
|
||||||
TEMPLATES_AUTO_RELOAD=True,
|
TEMPLATES_AUTO_RELOAD=True,
|
||||||
|
|||||||
@@ -0,0 +1,236 @@
|
|||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
# SPDX-License-Identifier: MIT
|
||||||
|
# Copyright (c) 2026 Wang Chuanli
|
||||||
|
|
||||||
|
"""图形验证码(自托管,零第三方依赖)。
|
||||||
|
|
||||||
|
设计要点
|
||||||
|
--------
|
||||||
|
1. **答案只存在服务端**。下发到浏览器的是一个随机 `captcha_id`,它本身
|
||||||
|
不含任何信息。之所以不把答案放进 Flask session:Flask 的 session 是
|
||||||
|
**签名而非加密**的(base64 + HMAC),客户端 base64 解开就能读到明文——
|
||||||
|
把答案放进去等于把答案直接送给机器人。
|
||||||
|
2. **一次性**。校验时无论成功失败都立刻删除该 `captcha_id`,
|
||||||
|
防止「一个码刷一万个用户名」的撞库变体。
|
||||||
|
3. **光栅图,不是 SVG**。SVG 是文本,答案会以明文出现在页面源码或 DOM 里,
|
||||||
|
必须用位图。这里手写 PNG 编码器(zlib 是标准库),点阵字模自带。
|
||||||
|
4. **字符集避开易混字符**(0/O、1/I/L),降低正常用户输错概率。
|
||||||
|
|
||||||
|
字模
|
||||||
|
----
|
||||||
|
5×7 点阵,`#` 为前景。渲染时按整数倍放大并逐字符抖动,
|
||||||
|
再叠噪点与干扰线,普通 OCR 与「按色块切分」都会被破坏。
|
||||||
|
"""
|
||||||
|
import hmac
|
||||||
|
import os
|
||||||
|
import random
|
||||||
|
import secrets
|
||||||
|
import struct
|
||||||
|
import zlib
|
||||||
|
from datetime import datetime, timedelta
|
||||||
|
|
||||||
|
ALPHABET = "23456789ABCDEFGHJKMNPQRSTUVWXYZ" # 去掉 0 O 1 I L
|
||||||
|
|
||||||
|
_FONT = {
|
||||||
|
"2": (".###.", "#...#", "....#", "...#.", "..#..", ".#...", "#####"),
|
||||||
|
"3": ("####.", "....#", "....#", ".###.", "....#", "....#", "####."),
|
||||||
|
"4": ("...#.", "..##.", ".#.#.", "#..#.", "#####", "...#.", "...#."),
|
||||||
|
"5": ("#####", "#....", "####.", "....#", "....#", "#...#", ".###."),
|
||||||
|
"6": ("..##.", ".#...", "#....", "####.", "#...#", "#...#", ".###."),
|
||||||
|
"7": ("#####", "....#", "...#.", "..#..", ".#...", ".#...", ".#..."),
|
||||||
|
"8": (".###.", "#...#", "#...#", ".###.", "#...#", "#...#", ".###."),
|
||||||
|
"9": (".###.", "#...#", "#...#", ".####", "....#", "...#.", ".##.."),
|
||||||
|
"A": ("..#..", ".#.#.", "#...#", "#...#", "#####", "#...#", "#...#"),
|
||||||
|
"B": ("####.", "#...#", "#...#", "####.", "#...#", "#...#", "####."),
|
||||||
|
"C": (".###.", "#...#", "#....", "#....", "#....", "#...#", ".###."),
|
||||||
|
"D": ("###..", "#..#.", "#...#", "#...#", "#...#", "#..#.", "###.."),
|
||||||
|
"E": ("#####", "#....", "#....", "####.", "#....", "#....", "#####"),
|
||||||
|
"F": ("#####", "#....", "#....", "####.", "#....", "#....", "#...."),
|
||||||
|
"G": (".###.", "#...#", "#....", "#.###", "#...#", "#...#", ".###."),
|
||||||
|
"H": ("#...#", "#...#", "#...#", "#####", "#...#", "#...#", "#...#"),
|
||||||
|
"J": ("..###", "...#.", "...#.", "...#.", "...#.", "#..#.", ".##.."),
|
||||||
|
"K": ("#...#", "#..#.", "#.#..", "##...", "#.#..", "#..#.", "#...#"),
|
||||||
|
"M": ("#...#", "##.##", "#.#.#", "#...#", "#...#", "#...#", "#...#"),
|
||||||
|
"N": ("#...#", "##..#", "#.#.#", "#..##", "#...#", "#...#", "#...#"),
|
||||||
|
"P": ("####.", "#...#", "#...#", "####.", "#....", "#....", "#...."),
|
||||||
|
"Q": (".###.", "#...#", "#...#", "#...#", "#.#.#", "#..#.", ".##.#"),
|
||||||
|
"R": ("####.", "#...#", "#...#", "####.", "#.#..", "#..#.", "#...#"),
|
||||||
|
"S": (".####", "#....", "#....", ".###.", "....#", "....#", "####."),
|
||||||
|
"T": ("#####", "..#..", "..#..", "..#..", "..#..", "..#..", "..#.."),
|
||||||
|
"U": ("#...#", "#...#", "#...#", "#...#", "#...#", "#...#", ".###."),
|
||||||
|
"V": ("#...#", "#...#", "#...#", "#...#", "#...#", ".#.#.", "..#.."),
|
||||||
|
"W": ("#...#", "#...#", "#...#", "#...#", "#.#.#", "##.##", "#...#"),
|
||||||
|
"X": ("#...#", "#...#", ".#.#.", "..#..", ".#.#.", "#...#", "#...#"),
|
||||||
|
"Y": ("#...#", "#...#", ".#.#.", "..#..", "..#..", "..#..", "..#.."),
|
||||||
|
"Z": ("#####", "....#", "...#.", "..#..", ".#...", "#....", "#####"),
|
||||||
|
}
|
||||||
|
|
||||||
|
GLYPH_W, GLYPH_H = 5, 7
|
||||||
|
|
||||||
|
# 一次性验证码有效期(秒)。太短用户来不及看,太长给暴力破解留窗口。
|
||||||
|
TTL_SECONDS = 300
|
||||||
|
# 保留已过期记录多久后清理(仅用于体积控制,不影响安全性)
|
||||||
|
PURGE_AFTER_SECONDS = 3600
|
||||||
|
|
||||||
|
|
||||||
|
def random_code(length=4):
|
||||||
|
return "".join(secrets.choice(ALPHABET) for _ in range(length))
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------- PNG 编码(手写,无依赖) ----------------
|
||||||
|
def _chunk(tag, data):
|
||||||
|
return (struct.pack(">I", len(data)) + tag + data
|
||||||
|
+ struct.pack(">I", zlib.crc32(tag + data) & 0xFFFFFFFF))
|
||||||
|
|
||||||
|
|
||||||
|
def encode_png(width, height, rgb):
|
||||||
|
"""把 RGB 字节串编码成 PNG(8 位真彩,无 alpha)。
|
||||||
|
|
||||||
|
rgb 长度必须是 width*height*3。每行前面加一个 filter 字节 0(None),
|
||||||
|
这是 PNG 对「一行一张扫描线」的强制要求。
|
||||||
|
"""
|
||||||
|
stride = width * 3
|
||||||
|
raw = bytearray()
|
||||||
|
for y in range(height):
|
||||||
|
raw.append(0)
|
||||||
|
raw += rgb[y * stride:(y + 1) * stride]
|
||||||
|
ihdr = struct.pack(">IIBBBBB", width, height, 8, 2, 0, 0, 0)
|
||||||
|
return (b"\x89PNG\r\n\x1a\n"
|
||||||
|
+ _chunk(b"IHDR", ihdr)
|
||||||
|
+ _chunk(b"IDAT", zlib.compress(bytes(raw), 9))
|
||||||
|
+ _chunk(b"IEND", b""))
|
||||||
|
|
||||||
|
|
||||||
|
class _Canvas:
|
||||||
|
"""极小的 RGB 画布。坐标越界自动丢弃,省得每处调用都判边界。"""
|
||||||
|
|
||||||
|
def __init__(self, w, h, bg):
|
||||||
|
self.w, self.h = w, h
|
||||||
|
self.buf = bytearray(bg * (w * h))
|
||||||
|
|
||||||
|
def dot(self, x, y, color):
|
||||||
|
if 0 <= x < self.w and 0 <= y < self.h:
|
||||||
|
i = (y * self.w + x) * 3
|
||||||
|
self.buf[i:i + 3] = bytes(color)
|
||||||
|
|
||||||
|
def rect(self, x, y, w, h, color):
|
||||||
|
for dy in range(h):
|
||||||
|
for dx in range(w):
|
||||||
|
self.dot(x + dx, y + dy, color)
|
||||||
|
|
||||||
|
def line(self, x0, y0, x1, y1, color):
|
||||||
|
"""Bresenham 直线。"""
|
||||||
|
dx, dy = abs(x1 - x0), -abs(y1 - y0)
|
||||||
|
sx = 1 if x0 < x1 else -1
|
||||||
|
sy = 1 if y0 < y1 else -1
|
||||||
|
err = dx + dy
|
||||||
|
while True:
|
||||||
|
self.dot(x0, y0, color)
|
||||||
|
if x0 == x1 and y0 == y1:
|
||||||
|
return
|
||||||
|
e2 = 2 * err
|
||||||
|
if e2 >= dy:
|
||||||
|
err += dy
|
||||||
|
x0 += sx
|
||||||
|
if e2 <= dx:
|
||||||
|
err += dx
|
||||||
|
y0 += sy
|
||||||
|
|
||||||
|
def bytes(self):
|
||||||
|
return bytes(self.buf)
|
||||||
|
|
||||||
|
|
||||||
|
def render(code, width=150, height=56, scale=5, rng=None):
|
||||||
|
"""把验证码渲染成 PNG 字节串。
|
||||||
|
|
||||||
|
刻意不做「清晰排版」而是加抖动/噪点/干扰线:这是防机器识别的核心,
|
||||||
|
可读性靠放大字符(scale=5 即 25×35 像素)来补偿。
|
||||||
|
"""
|
||||||
|
rng = rng or random.SystemRandom()
|
||||||
|
n = len(code)
|
||||||
|
gap = 7
|
||||||
|
text_w = n * GLYPH_W * scale + (n - 1) * gap
|
||||||
|
if text_w + 16 > width:
|
||||||
|
width = text_w + 16
|
||||||
|
x0 = max(4, (width - text_w) // 2)
|
||||||
|
y0 = max(3, (height - GLYPH_H * scale) // 2)
|
||||||
|
|
||||||
|
# 背景取浅色,前景取深色 —— 深色底+浅字在缩略图上更容易糊,
|
||||||
|
# 而且打印/截图后对比度更差。
|
||||||
|
bg = tuple(rng.randint(238, 252) for _ in range(3))
|
||||||
|
cv = _Canvas(width, height, bg)
|
||||||
|
|
||||||
|
# 1) 干扰线(先画,压在字下面,不遮挡主体)
|
||||||
|
for _ in range(4):
|
||||||
|
cv.line(rng.randint(0, width - 1), rng.randint(0, height - 1),
|
||||||
|
rng.randint(0, width - 1), rng.randint(0, height - 1),
|
||||||
|
tuple(rng.randint(150, 205) for _ in range(3)))
|
||||||
|
|
||||||
|
# 2) 字符本体:逐字符随机取色 + 整数抖动,破坏固定网格切分
|
||||||
|
for i, ch in enumerate(code):
|
||||||
|
glyph = _FONT.get(ch)
|
||||||
|
if glyph is None:
|
||||||
|
continue
|
||||||
|
color = tuple(rng.randint(20, 105) for _ in range(3))
|
||||||
|
gx = x0 + i * (GLYPH_W * scale + gap) + rng.randint(-1, 1)
|
||||||
|
gy = y0 + rng.randint(-2, 2)
|
||||||
|
for row, bits in enumerate(glyph):
|
||||||
|
for col, bit in enumerate(bits):
|
||||||
|
if bit == "#":
|
||||||
|
cv.rect(gx + col * scale, gy + row * scale, scale, scale, color)
|
||||||
|
|
||||||
|
# 3) 前景噪点:少量深色点会让「按连通域找字符」变得不可靠
|
||||||
|
for _ in range(46):
|
||||||
|
cv.dot(rng.randint(0, width - 1), rng.randint(0, height - 1),
|
||||||
|
tuple(rng.randint(90, 190) for _ in range(3)))
|
||||||
|
|
||||||
|
# 4) 压在字上的细斜线:这是最有效的反 OCR 手段,但别太密,否则人也认不出
|
||||||
|
for _ in range(3):
|
||||||
|
y = rng.randint(2, height - 3)
|
||||||
|
cv.line(0, y, width - 1, y + rng.randint(-9, 9),
|
||||||
|
tuple(rng.randint(120, 175) for _ in range(3)))
|
||||||
|
|
||||||
|
return encode_png(width, height, cv.bytes())
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------- 挑战的存储与校验(SQLite) ----------------
|
||||||
|
def _fmt(dt):
|
||||||
|
return dt.strftime("%Y-%m-%d %H:%M:%S")
|
||||||
|
|
||||||
|
|
||||||
|
def purge(conn, now=None):
|
||||||
|
"""清掉过期的挑战。每次新建时顺手调用(有 expires_at 索引,代价很小)。"""
|
||||||
|
now = now or datetime.now()
|
||||||
|
cut = _fmt(now - timedelta(seconds=PURGE_AFTER_SECONDS))
|
||||||
|
conn.execute("DELETE FROM captchas WHERE expires_at < ?", (cut,))
|
||||||
|
|
||||||
|
|
||||||
|
def create(conn, purpose, length=4, ttl=TTL_SECONDS, now=None):
|
||||||
|
"""新建一个挑战,返回 (captcha_id, code)。code 只应交给渲染函数,不要下发。"""
|
||||||
|
now = now or datetime.now()
|
||||||
|
code = random_code(length)
|
||||||
|
cid = secrets.token_urlsafe(24)
|
||||||
|
purge(conn, now)
|
||||||
|
conn.execute(
|
||||||
|
"INSERT INTO captchas(id,answer,purpose,created_at,expires_at) VALUES(?,?,?,?,?)",
|
||||||
|
(cid, code, purpose, _fmt(now), _fmt(now + timedelta(seconds=ttl))))
|
||||||
|
return cid, code
|
||||||
|
|
||||||
|
|
||||||
|
def verify(conn, captcha_id, answer, purpose, now=None):
|
||||||
|
"""校验并**立即作废**该挑战。返回 True/False。
|
||||||
|
|
||||||
|
永远不区分「过期」「不存在」「答案错」——对外只回一句人话,
|
||||||
|
避免把「这个 id 存在但答错了」这类信息透露给攻击者。
|
||||||
|
"""
|
||||||
|
if not captcha_id or answer is None:
|
||||||
|
return False
|
||||||
|
row = conn.execute("SELECT * FROM captchas WHERE id=?", (captcha_id,)).fetchone()
|
||||||
|
# 先删后判:无论结果如何都不允许第二次使用同一个 id
|
||||||
|
conn.execute("DELETE FROM captchas WHERE id=?", (captcha_id,))
|
||||||
|
if row is None or row["purpose"] != purpose:
|
||||||
|
return False
|
||||||
|
now = now or datetime.now()
|
||||||
|
if row["expires_at"] < _fmt(now):
|
||||||
|
return False
|
||||||
|
return hmac.compare_digest(str(row["answer"]), str(answer).strip().upper())
|
||||||
@@ -5,10 +5,19 @@
|
|||||||
"""采集主流程:云端增量 -> SQLite(去重、断点、漂移校验、运行记录)。
|
"""采集主流程:云端增量 -> SQLite(去重、断点、漂移校验、运行记录)。
|
||||||
|
|
||||||
与原 fetch_usage.py 的差别:
|
与原 fetch_usage.py 的差别:
|
||||||
* 存档正本从 CSV 换成 SQLite,去重由 `ON CONFLICT(request_id)` 承担
|
* 存档正本从 CSV 换成 SQLite,去重由 `ON CONFLICT(user_id, request_id)` 承担
|
||||||
* 断点由 `SELECT MAX(ts)` 承担,不再需要全量读入内存
|
* 断点由 `SELECT MAX(ts) WHERE user_id=?` 承担,不再需要全量读入内存
|
||||||
* 每次运行落一条 collect_runs 记录,页面据此展示任务历史与日志
|
* 每次运行落一条 collect_runs 记录,页面据此展示任务历史与日志
|
||||||
* 采集互斥用文件锁,保证「单写者」——SQLite 只允许一个写进程
|
* 采集互斥用文件锁,保证「单写者」——SQLite 只允许一个写进程
|
||||||
|
|
||||||
|
**多用户约定(最重要)**
|
||||||
|
所有写入函数都要求显式传入 `uid`,且 `uid` 是 `conn` 之后的第一个位置参数、
|
||||||
|
没有默认值。采集**只使用该账号自己保存的 Cookie**:
|
||||||
|
|
||||||
|
* 明文的 Cookie 从来只存在于内存里(库里是 crypto.encrypt 后的密文)
|
||||||
|
* 不再支持 `WB_COOKIE` 环境变量作为采集凭证 —— 那会让所有人共用一份凭证,
|
||||||
|
一旦生效就是「A 的采集把数据写进 B 的账号」这种串号事故。
|
||||||
|
环境变量只保留给 `manage.py import-creds` 做一次性导入。
|
||||||
"""
|
"""
|
||||||
import csv
|
import csv
|
||||||
import os
|
import os
|
||||||
@@ -27,8 +36,19 @@ class Busy(Exception):
|
|||||||
"""已有采集在跑。"""
|
"""已有采集在跑。"""
|
||||||
|
|
||||||
|
|
||||||
|
class NotReady(Exception):
|
||||||
|
"""该账号还没配好凭证 —— 不是错误,只是「没什么可做的」。"""
|
||||||
|
|
||||||
|
|
||||||
# ---------------- 互斥锁 ----------------
|
# ---------------- 互斥锁 ----------------
|
||||||
class _Lock:
|
class _Lock:
|
||||||
|
"""全局单写者锁。
|
||||||
|
|
||||||
|
刻意**不做成按用户加锁**:SQLite 同一时刻只允许一个写事务,
|
||||||
|
按用户并行反而会在 busy_timeout 上互相拖死。串行跑完所有人的采集,
|
||||||
|
总耗时与并发差别很小(每个人一天也就拉一次)。
|
||||||
|
"""
|
||||||
|
|
||||||
def __init__(self, path=LOCK_PATH):
|
def __init__(self, path=LOCK_PATH):
|
||||||
self.path = path
|
self.path = path
|
||||||
self.fd = None
|
self.fd = None
|
||||||
@@ -65,18 +85,17 @@ class _Lock:
|
|||||||
|
|
||||||
|
|
||||||
# ---------------- 运行记录 ----------------
|
# ---------------- 运行记录 ----------------
|
||||||
def start_run(conn, trigger):
|
def start_run(conn, uid, trigger):
|
||||||
cur = conn.execute("INSERT INTO collect_runs(trigger,status,started_at) VALUES(?,?,?)",
|
cur = conn.execute("INSERT INTO collect_runs(user_id,trigger,status,started_at)"
|
||||||
(trigger, "running", db.now_str()))
|
" VALUES(?,?,?,?)", (uid, trigger, "running", db.now_str()))
|
||||||
return cur.lastrowid
|
return cur.lastrowid
|
||||||
|
|
||||||
|
|
||||||
def finish_run(conn, run_id, status, **kw):
|
def finish_run(conn, run_id, status, **kw):
|
||||||
fields = ["finished_at", "duration_ms", "win_from", "win_to", "fetched",
|
|
||||||
"added", "dup", "total", "conflicts", "exit_code", "message", "detail"]
|
|
||||||
sets = ["status=?", "finished_at=?"]
|
sets = ["status=?", "finished_at=?"]
|
||||||
vals = [status, db.now_str()]
|
vals = [status, db.now_str()]
|
||||||
for f in fields[1:]:
|
for f in ("duration_ms", "win_from", "win_to", "fetched", "added", "dup",
|
||||||
|
"total", "conflicts", "exit_code", "message", "detail"):
|
||||||
if f in kw:
|
if f in kw:
|
||||||
sets.append("%s=?" % f)
|
sets.append("%s=?" % f)
|
||||||
vals.append(kw[f])
|
vals.append(kw[f])
|
||||||
@@ -86,10 +105,11 @@ def finish_run(conn, run_id, status, **kw):
|
|||||||
|
|
||||||
# ---------------- 入库 ----------------
|
# ---------------- 入库 ----------------
|
||||||
_UPSERT = """
|
_UPSERT = """
|
||||||
INSERT INTO usage_records(request_id,ts,day,hour,model,client,credits,prompt,
|
INSERT INTO usage_records(user_id,request_id,ts,day,hour,model,client,credits,prompt,
|
||||||
first_seen,last_seen,cloud_ts)
|
first_seen,last_seen,cloud_ts)
|
||||||
VALUES(:request_id,:ts,:day,:hour,:model,:client,:credits,:prompt,:first_seen,:last_seen,:cloud_ts)
|
VALUES(:user_id,:request_id,:ts,:day,:hour,:model,:client,:credits,:prompt,
|
||||||
ON CONFLICT(request_id) DO UPDATE SET
|
:first_seen,:last_seen,:cloud_ts)
|
||||||
|
ON CONFLICT(user_id,request_id) DO UPDATE SET
|
||||||
last_seen = excluded.last_seen,
|
last_seen = excluded.last_seen,
|
||||||
cloud_ts = excluded.cloud_ts,
|
cloud_ts = excluded.cloud_ts,
|
||||||
ts = CASE WHEN excluded.ts <> '' AND excluded.ts < usage_records.ts
|
ts = CASE WHEN excluded.ts <> '' AND excluded.ts < usage_records.ts
|
||||||
@@ -107,10 +127,11 @@ ON CONFLICT(request_id) DO UPDATE SET
|
|||||||
"""
|
"""
|
||||||
|
|
||||||
|
|
||||||
def _row_dict(n):
|
def _row_dict(n, uid):
|
||||||
ts = (n.get("ts") or "").strip()[:19]
|
ts = (n.get("ts") or "").strip()[:19]
|
||||||
hh = ts[11:13]
|
hh = ts[11:13]
|
||||||
return {
|
return {
|
||||||
|
"user_id": uid,
|
||||||
"request_id": n["request_id"],
|
"request_id": n["request_id"],
|
||||||
"ts": ts,
|
"ts": ts,
|
||||||
"day": ts[:10],
|
"day": ts[:10],
|
||||||
@@ -125,8 +146,8 @@ def _row_dict(n):
|
|||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
def upsert(conn, normalized, drift_tolerance=5, log=None):
|
def upsert(conn, uid, normalized, drift_tolerance=5, log=None):
|
||||||
"""按 request_id 去重写入。返回 (added, dup, conflicts) 与逐行警告。
|
"""按 (user_id, request_id) 去重写入。返回 (added, dup, conflicts) 与逐行警告。
|
||||||
|
|
||||||
每 200 条一个事务(连接是 autocommit,不显式 BEGIN 的话每条 INSERT 都要
|
每 200 条一个事务(连接是 autocommit,不显式 BEGIN 的话每条 INSERT 都要
|
||||||
单独 fsync)。upsert 本身幂等,所以按块提交是安全的。
|
单独 fsync)。upsert 本身幂等,所以按块提交是安全的。
|
||||||
@@ -135,7 +156,7 @@ def upsert(conn, normalized, drift_tolerance=5, log=None):
|
|||||||
for n in normalized:
|
for n in normalized:
|
||||||
if not n.get("request_id") or not n.get("ts"):
|
if not n.get("request_id") or not n.get("ts"):
|
||||||
continue
|
continue
|
||||||
recs.append(_row_dict(n))
|
recs.append(_row_dict(n, uid))
|
||||||
added = dup = 0
|
added = dup = 0
|
||||||
conflicts = []
|
conflicts = []
|
||||||
for i in range(0, len(recs), 200):
|
for i in range(0, len(recs), 200):
|
||||||
@@ -146,8 +167,9 @@ def upsert(conn, normalized, drift_tolerance=5, log=None):
|
|||||||
if own_tx:
|
if own_tx:
|
||||||
conn.execute("BEGIN")
|
conn.execute("BEGIN")
|
||||||
try:
|
try:
|
||||||
old = {x["request_id"]: x["ts"] for x in
|
old = {x["request_id"]: x["ts"] for x in conn.execute(
|
||||||
conn.execute("SELECT request_id,ts FROM usage_records WHERE request_id IN (%s)" % ph, ids)}
|
"SELECT request_id,ts FROM usage_records WHERE user_id=? AND request_id IN (%s)"
|
||||||
|
% ph, [uid] + ids)}
|
||||||
for r in chunk:
|
for r in chunk:
|
||||||
prev = old.get(r["request_id"])
|
prev = old.get(r["request_id"])
|
||||||
if prev is None:
|
if prev is None:
|
||||||
@@ -178,18 +200,33 @@ def upsert(conn, normalized, drift_tolerance=5, log=None):
|
|||||||
return added, dup, conflicts
|
return added, dup, conflicts
|
||||||
|
|
||||||
|
|
||||||
def record_count(conn):
|
def record_count(conn, uid):
|
||||||
return conn.execute("SELECT COUNT(*) FROM usage_records").fetchone()[0]
|
return conn.execute("SELECT COUNT(*) FROM usage_records WHERE user_id=?",
|
||||||
|
(uid or 0,)).fetchone()[0]
|
||||||
|
|
||||||
|
|
||||||
def last_ts(conn):
|
def last_ts(conn, uid):
|
||||||
return conn.execute("SELECT MAX(ts) FROM usage_records").fetchone()[0]
|
return conn.execute("SELECT MAX(ts) FROM usage_records WHERE user_id=?",
|
||||||
|
(uid or 0,)).fetchone()[0]
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------- 凭证读取(解密) ----------------
|
||||||
|
def load_credentials(conn, uid):
|
||||||
|
"""取该账号的 (cookie, user_agent)。
|
||||||
|
|
||||||
|
Cookie 从库里读出来是密文,由 db.get_secret 解密;解不开会抛
|
||||||
|
db.SecretUnreadable(多半是 instance.json 里的 cookie_key 被换过),
|
||||||
|
这时应当明确告诉用户「重新粘贴 Cookie」,而不是当成「未配置」静默跳过。
|
||||||
|
"""
|
||||||
|
cookie = db.get_secret(conn, "cookie", uid).strip()
|
||||||
|
ua = (db.get_setting(conn, "user_agent", "", uid) or "").strip()
|
||||||
|
return cookie, ua
|
||||||
|
|
||||||
|
|
||||||
# ---------------- 主同步 ----------------
|
# ---------------- 主同步 ----------------
|
||||||
def sync(conn, trigger="manual", from_dt=None, to_dt=None, verify_days=None,
|
def sync(conn, uid, trigger="manual", from_dt=None, to_dt=None, verify_days=None,
|
||||||
do_write=True, log=None):
|
do_write=True, log=None):
|
||||||
"""增量同步。
|
"""增量同步(仅限 uid 这个账号)。
|
||||||
|
|
||||||
trigger: manual | schedule | cli | startup(写进 collect_runs 便于区分来源)
|
trigger: manual | schedule | cli | startup(写进 collect_runs 便于区分来源)
|
||||||
返回 result dict;异常时抛出 ApiError(调用方决定如何展示)。
|
返回 result dict;异常时抛出 ApiError(调用方决定如何展示)。
|
||||||
@@ -201,40 +238,39 @@ def sync(conn, trigger="manual", from_dt=None, to_dt=None, verify_days=None,
|
|||||||
if log:
|
if log:
|
||||||
log(msg)
|
log(msg)
|
||||||
|
|
||||||
s = db.get_settings(conn)
|
s = db.get_settings(conn, uid=uid)
|
||||||
cookie = (s.get("cookie") or "").strip() or os.environ.get("WB_COOKIE", "").strip()
|
cookie, ua = load_credentials(conn, uid)
|
||||||
ua = (s.get("user_agent") or "").strip() or os.environ.get("WB_UA", "").strip()
|
|
||||||
api_base = s.get("api_base") or config.API_BASE
|
api_base = s.get("api_base") or config.API_BASE
|
||||||
api_path = s.get("api_path") or config.API_PATH
|
api_path = s.get("api_path") or config.API_PATH
|
||||||
# 一律走 db.get_int/get_float:settings 表的值由后台页面自由输入,
|
# 一律走 db.get_int/get_float:settings 表的值由后台页面自由输入,
|
||||||
# 直接 int() 会让一个手滑的字符把整条采集链路打断(历史 bug)。
|
# 直接 int() 会让一个手滑的字符把整条采集链路打断(历史 bug)。
|
||||||
page_size = db.get_int(conn, "page_size", 200)
|
page_size = db.get_int(conn, "page_size", 200, uid)
|
||||||
rewind = db.get_int(conn, "rewind_minutes", 2)
|
rewind = db.get_int(conn, "rewind_minutes", 2, uid)
|
||||||
drift = db.get_int(conn, "drift_tolerance_minutes", 5)
|
drift = db.get_int(conn, "drift_tolerance_minutes", 5, uid)
|
||||||
max_prompt = db.get_int(conn, "max_prompt", 0)
|
max_prompt = db.get_int(conn, "max_prompt", 0, uid)
|
||||||
timeout = db.get_int(conn, "timeout", 30)
|
timeout = db.get_int(conn, "timeout", 30, uid)
|
||||||
ssl_verify = db.get_bool(conn, "ssl_verify", True)
|
ssl_verify = db.get_bool(conn, "ssl_verify", True, uid)
|
||||||
if verify_days is None:
|
if verify_days is None:
|
||||||
verify_days = db.get_int(conn, "verify_days", 0)
|
verify_days = db.get_int(conn, "verify_days", 0, uid)
|
||||||
# 夹到合法区间,避免历史脏数据(如超大的 page_size)把云端打爆
|
# 夹到合法区间,避免历史脏数据(如超大的 page_size)把云端打爆
|
||||||
lo, hi, _ = config.NUM_SETTINGS["page_size"]
|
lo, hi, _ = config.NUM_SETTINGS["page_size"]
|
||||||
page_size = max(lo, min(hi, page_size))
|
page_size = max(lo, min(hi, page_size))
|
||||||
|
|
||||||
run_id = start_run(conn, trigger) if do_write else None
|
run_id = start_run(conn, uid, trigger) if do_write else None
|
||||||
t0 = time.time()
|
t0 = time.time()
|
||||||
base = {"win_from": None, "win_to": None, "fetched": 0,
|
base = {"win_from": None, "win_to": None, "fetched": 0,
|
||||||
"added": 0, "dup": 0, "conflicts": 0}
|
"added": 0, "dup": 0, "conflicts": 0}
|
||||||
|
|
||||||
if not cookie:
|
if not cookie:
|
||||||
msg = "未配置 Cookie,请到「配置管理」页粘贴,或设置环境变量 WB_COOKIE"
|
msg = "未配置 Cookie,请到「配置管理」页粘贴自己账号的 Cookie"
|
||||||
_log("[error] " + msg)
|
_log("[error] " + msg)
|
||||||
if run_id:
|
if run_id:
|
||||||
finish_run(conn, run_id, "error", exit_code=2, message=msg,
|
finish_run(conn, run_id, "error", exit_code=2, message=msg,
|
||||||
duration_ms=int((time.time() - t0) * 1000), detail="\n".join(lines), **base)
|
duration_ms=int((time.time() - t0) * 1000), detail="\n".join(lines), **base)
|
||||||
raise ApiError(msg)
|
raise NotReady(msg)
|
||||||
|
|
||||||
total_before = record_count(conn)
|
total_before = record_count(conn, uid)
|
||||||
tail = last_ts(conn)
|
tail = last_ts(conn, uid)
|
||||||
now = datetime.now()
|
now = datetime.now()
|
||||||
_log("存档:%d 条%s" % (total_before, (",最后记录 " + tail) if tail else "(空)"))
|
_log("存档:%d 条%s" % (total_before, (",最后记录 " + tail) if tail else "(空)"))
|
||||||
|
|
||||||
@@ -270,15 +306,17 @@ def sync(conn, trigger="manual", from_dt=None, to_dt=None, verify_days=None,
|
|||||||
new_rows = [client.normalize(r, max_prompt=max_prompt) for r in raw]
|
new_rows = [client.normalize(r, max_prompt=max_prompt) for r in raw]
|
||||||
_log("云端返回:%d 条" % len(raw))
|
_log("云端返回:%d 条" % len(raw))
|
||||||
|
|
||||||
added, dup, conflicts = upsert(conn, new_rows, drift_tolerance=drift, log=_log)
|
added, dup, conflicts = upsert(conn, uid, new_rows, drift_tolerance=drift, log=_log)
|
||||||
|
|
||||||
# 整日完整性校验(默认关闭;用于排查缺记录)
|
# 整日完整性校验(默认关闭;用于排查缺记录)
|
||||||
if verify_days > 0:
|
if verify_days > 0:
|
||||||
days = [r["day"] for r in conn.execute(
|
days = [r["day"] for r in conn.execute(
|
||||||
"SELECT DISTINCT day FROM usage_records ORDER BY day DESC LIMIT ?", (verify_days,))]
|
"SELECT DISTINCT day FROM usage_records WHERE user_id=? ORDER BY day DESC LIMIT ?",
|
||||||
|
(uid, verify_days))]
|
||||||
_log("完整性校验:最近 %d 天" % len(days))
|
_log("完整性校验:最近 %d 天" % len(days))
|
||||||
for d in sorted(days):
|
for d in sorted(days):
|
||||||
local = conn.execute("SELECT COUNT(*) FROM usage_records WHERE day=?", (d,)).fetchone()[0]
|
local = conn.execute("SELECT COUNT(*) FROM usage_records WHERE user_id=? AND day=?",
|
||||||
|
(uid, d)).fetchone()[0]
|
||||||
d0 = datetime.strptime(d, "%Y-%m-%d")
|
d0 = datetime.strptime(d, "%Y-%m-%d")
|
||||||
try:
|
try:
|
||||||
raw2, t2 = client.fetch_range(d0, d0.replace(hour=23, minute=59, second=59),
|
raw2, t2 = client.fetch_range(d0, d0.replace(hour=23, minute=59, second=59),
|
||||||
@@ -290,14 +328,15 @@ def sync(conn, trigger="manual", from_dt=None, to_dt=None, verify_days=None,
|
|||||||
continue
|
continue
|
||||||
cloud = t2.get(d, 0)
|
cloud = t2.get(d, 0)
|
||||||
if local < cloud:
|
if local < cloud:
|
||||||
a2, _, _ = upsert(conn, [client.normalize(r, max_prompt=max_prompt) for r in raw2],
|
a2, _, _ = upsert(conn, uid,
|
||||||
|
[client.normalize(r, max_prompt=max_prompt) for r in raw2],
|
||||||
drift_tolerance=drift)
|
drift_tolerance=drift)
|
||||||
added += a2
|
added += a2
|
||||||
_log(" %s:云端 %d / 本地 %d → 补入 %d 条" % (d, cloud, local, a2))
|
_log(" %s:云端 %d / 本地 %d → 补入 %d 条" % (d, cloud, local, a2))
|
||||||
else:
|
else:
|
||||||
_log(" %s:云端 %d / 本地 %d OK" % (d, cloud, local))
|
_log(" %s:云端 %d / 本地 %d OK" % (d, cloud, local))
|
||||||
|
|
||||||
total_after = record_count(conn)
|
total_after = record_count(conn, uid)
|
||||||
status = "warn" if conflicts else "ok"
|
status = "warn" if conflicts else "ok"
|
||||||
msg = "新增 %d 条,重复 %d 条,存档共 %d 条" % (added, dup, total_after)
|
msg = "新增 %d 条,重复 %d 条,存档共 %d 条" % (added, dup, total_after)
|
||||||
_log("RESULT: added=%d dup=%d total=%d" % (added, dup, total_after))
|
_log("RESULT: added=%d dup=%d total=%d" % (added, dup, total_after))
|
||||||
@@ -313,27 +352,31 @@ def sync(conn, trigger="manual", from_dt=None, to_dt=None, verify_days=None,
|
|||||||
"total": total_after, "conflicts": len(conflicts),
|
"total": total_after, "conflicts": len(conflicts),
|
||||||
"win_from": start.strftime("%Y-%m-%d %H:%M:%S"),
|
"win_from": start.strftime("%Y-%m-%d %H:%M:%S"),
|
||||||
"win_to": end.strftime("%Y-%m-%d %H:%M:%S"),
|
"win_to": end.strftime("%Y-%m-%d %H:%M:%S"),
|
||||||
"message": msg, "lines": lines, "run_id": run_id}
|
"message": msg, "lines": lines, "run_id": run_id, "uid": uid}
|
||||||
|
|
||||||
|
|
||||||
def run_sync(trigger="manual", **kw):
|
def run_sync(trigger="manual", **kw):
|
||||||
"""带锁的同步入口(供 CLI / 调度器 / 页面手动触发共用)。"""
|
"""带锁的同步入口(供 CLI / 调度器 / 页面手动触发共用)。
|
||||||
|
|
||||||
|
必须显式给出 `uid=...`;漏传会由 sync() 直接报 TypeError,
|
||||||
|
不会退化成「用某个默认账号去采集」。
|
||||||
|
"""
|
||||||
with _Lock():
|
with _Lock():
|
||||||
conn = db.thread_conn()
|
conn = db.thread_conn()
|
||||||
return sync(conn, trigger=trigger, **kw)
|
return sync(conn, trigger=trigger, **kw)
|
||||||
|
|
||||||
|
|
||||||
# ---------------- 补全 / 导入 / 导出 ----------------
|
# ---------------- 补全 / 导入 / 导出 ----------------
|
||||||
def fill_prompt(conn, log=print):
|
def fill_prompt(conn, uid, log=print):
|
||||||
"""补全缺失的 User Prompt(官网导出的 xlsx 会丢约 22%,云端仍保留)。"""
|
"""补全该账号缺失的 User Prompt(官网导出的 xlsx 会丢约 22%,云端仍保留)。"""
|
||||||
s = db.get_settings(conn)
|
s = db.get_settings(conn, uid=uid)
|
||||||
cookie = (s.get("cookie") or "").strip() or os.environ.get("WB_COOKIE", "").strip()
|
cookie, ua = load_credentials(conn, uid)
|
||||||
ua = (s.get("user_agent") or "").strip()
|
max_prompt = db.get_int(conn, "max_prompt", 0, uid)
|
||||||
max_prompt = db.get_int(conn, "max_prompt", 0)
|
|
||||||
if not cookie:
|
if not cookie:
|
||||||
raise ApiError("未配置 Cookie")
|
raise NotReady("未配置 Cookie")
|
||||||
todo = conn.execute("SELECT request_id, day FROM usage_records "
|
todo = conn.execute("SELECT request_id, day FROM usage_records "
|
||||||
"WHERE COALESCE(prompt,'')='' ORDER BY day").fetchall()
|
"WHERE user_id=? AND COALESCE(prompt,'')='' ORDER BY day",
|
||||||
|
(uid,)).fetchall()
|
||||||
if not todo:
|
if not todo:
|
||||||
log("没有缺失的 User Prompt")
|
log("没有缺失的 User Prompt")
|
||||||
return 0
|
return 0
|
||||||
@@ -345,9 +388,9 @@ def fill_prompt(conn, log=print):
|
|||||||
raw, _ = client.fetch_range(d0, d0.replace(hour=23, minute=59, second=59),
|
raw, _ = client.fetch_range(d0, d0.replace(hour=23, minute=59, second=59),
|
||||||
cookie, ua, s.get("api_base") or config.API_BASE,
|
cookie, ua, s.get("api_base") or config.API_BASE,
|
||||||
s.get("api_path") or config.API_PATH,
|
s.get("api_path") or config.API_PATH,
|
||||||
page_size=db.get_int(conn, "page_size", 200),
|
page_size=db.get_int(conn, "page_size", 200, uid),
|
||||||
timeout=db.get_int(conn, "timeout", 30),
|
timeout=db.get_int(conn, "timeout", 30, uid),
|
||||||
ssl_verify=db.get_bool(conn, "ssl_verify", True))
|
ssl_verify=db.get_bool(conn, "ssl_verify", True, uid))
|
||||||
for r in raw:
|
for r in raw:
|
||||||
rid = (r.get("requestId") or "").strip()
|
rid = (r.get("requestId") or "").strip()
|
||||||
if rid:
|
if rid:
|
||||||
@@ -357,21 +400,22 @@ def fill_prompt(conn, log=print):
|
|||||||
for row in todo:
|
for row in todo:
|
||||||
p = pool.get(row["request_id"], "")
|
p = pool.get(row["request_id"], "")
|
||||||
if p:
|
if p:
|
||||||
conn.execute("UPDATE usage_records SET prompt=? WHERE request_id=?", (p, row["request_id"]))
|
conn.execute("UPDATE usage_records SET prompt=? WHERE user_id=? AND request_id=?",
|
||||||
|
(p, uid, row["request_id"]))
|
||||||
n += 1
|
n += 1
|
||||||
left = conn.execute("SELECT COUNT(*) FROM usage_records WHERE COALESCE(prompt,'')=''").fetchone()[0]
|
left = conn.execute("SELECT COUNT(*) FROM usage_records WHERE user_id=?"
|
||||||
|
" AND COALESCE(prompt,'')=''", (uid,)).fetchone()[0]
|
||||||
log("补全 %d 条,仍为空 %d 条" % (n, left))
|
log("补全 %d 条,仍为空 %d 条" % (n, left))
|
||||||
return n
|
return n
|
||||||
|
|
||||||
|
|
||||||
def import_xlsx(conn, path, log=print):
|
def import_xlsx(conn, uid, path, log=print):
|
||||||
"""从官网「用量明细 - 导出」的 xlsx 合入(按 request_id 去重)。"""
|
"""从官网「用量明细 - 导出」的 xlsx 合入(按 request_id 去重)。"""
|
||||||
try:
|
try:
|
||||||
import openpyxl
|
import openpyxl
|
||||||
except ImportError:
|
except ImportError:
|
||||||
raise ApiError("需要 openpyxl:pip install openpyxl")
|
raise ApiError("需要 openpyxl:pip install openpyxl")
|
||||||
s = db.get_settings(conn)
|
max_prompt = db.get_int(conn, "max_prompt", 0, uid)
|
||||||
max_prompt = db.get_int(conn, "max_prompt", 0)
|
|
||||||
wb = openpyxl.load_workbook(path, read_only=True, data_only=True)
|
wb = openpyxl.load_workbook(path, read_only=True, data_only=True)
|
||||||
it = wb.worksheets[0].iter_rows(values_only=True)
|
it = wb.worksheets[0].iter_rows(values_only=True)
|
||||||
header = [str(c).strip() if c is not None else "" for c in next(it)]
|
header = [str(c).strip() if c is not None else "" for c in next(it)]
|
||||||
@@ -401,17 +445,24 @@ def import_xlsx(conn, path, log=print):
|
|||||||
rows.append({"request_id": rid, "ts": t, "credits": round(cr, 2), "prompt": px,
|
rows.append({"request_id": rid, "ts": t, "credits": round(cr, 2), "prompt": px,
|
||||||
"model": str(row[i_m] or "-").strip() or "-",
|
"model": str(row[i_m] or "-").strip() or "-",
|
||||||
"client": str(row[i_cl] or "-").strip() or "-"})
|
"client": str(row[i_cl] or "-").strip() or "-"})
|
||||||
added, dup, _ = upsert(conn, rows)
|
added, dup, _ = upsert(conn, uid, rows)
|
||||||
log("[xlsx] 读取 %d 条,去重后新增 %d 条,存档共 %d 条" % (len(rows), added, record_count(conn)))
|
log("[xlsx] 读取 %d 条,去重后新增 %d 条,该账号存档共 %d 条"
|
||||||
|
% (len(rows), added, record_count(conn, uid)))
|
||||||
return added
|
return added
|
||||||
|
|
||||||
|
|
||||||
def export_csv(conn, path=None):
|
def export_csv(conn, uid, path=None, username=None):
|
||||||
"""导出与旧存档 / 官网 xlsx 完全同构的 CSV(备份与对端交换用)。"""
|
"""导出与旧存档 / 官网 xlsx 完全同构的 CSV(备份与对端交换用)。
|
||||||
path = path or os.path.join(config.EXPORT_DIR, "usage_records.csv")
|
|
||||||
|
文件名带账号名:多用户下所有人导出到同一个目录,
|
||||||
|
不带归属就会互相覆盖。
|
||||||
|
"""
|
||||||
|
if path is None:
|
||||||
|
tag = username or ("u%s" % uid)
|
||||||
|
path = os.path.join(config.EXPORT_DIR, "usage_records_%s.csv" % tag)
|
||||||
os.makedirs(os.path.dirname(path), exist_ok=True)
|
os.makedirs(os.path.dirname(path), exist_ok=True)
|
||||||
rows = conn.execute("SELECT request_id,credits,prompt,model,client,ts FROM usage_records "
|
rows = conn.execute("SELECT request_id,credits,prompt,model,client,ts FROM usage_records "
|
||||||
"ORDER BY ts, request_id")
|
"WHERE user_id=? ORDER BY ts, request_id", (uid,))
|
||||||
n = 0
|
n = 0
|
||||||
with open(path, "w", encoding="utf-8-sig", newline="") as f:
|
with open(path, "w", encoding="utf-8-sig", newline="") as f:
|
||||||
w = csv.writer(f)
|
w = csv.writer(f)
|
||||||
@@ -423,8 +474,12 @@ def export_csv(conn, path=None):
|
|||||||
return path, n
|
return path, n
|
||||||
|
|
||||||
|
|
||||||
def migrate_from_csv(conn, path, log=print):
|
def migrate_from_csv(conn, uid, path, log=print):
|
||||||
"""把旧版 data/usage_records.csv 全量导入 SQLite(幂等,可重复执行)。"""
|
"""把旧版 data/usage_records.csv 全量导入 SQLite(幂等,可重复执行)。
|
||||||
|
|
||||||
|
导入的数据归属 `uid` 指定的账号 —— 老存档是单用户的,
|
||||||
|
必须由调用方明确「这份数据算谁的」。
|
||||||
|
"""
|
||||||
if not os.path.exists(path):
|
if not os.path.exists(path):
|
||||||
raise FileNotFoundError(path)
|
raise FileNotFoundError(path)
|
||||||
with open(path, "rb") as fb:
|
with open(path, "rb") as fb:
|
||||||
@@ -445,8 +500,8 @@ def migrate_from_csv(conn, path, log=print):
|
|||||||
"prompt": " ".join(str(r.get("User Prompt") or "").split()),
|
"prompt": " ".join(str(r.get("User Prompt") or "").split()),
|
||||||
"model": (r.get("模型") or "-").strip() or "-",
|
"model": (r.get("模型") or "-").strip() or "-",
|
||||||
"client": (r.get("客户端") or "-").strip() or "-"})
|
"client": (r.get("客户端") or "-").strip() or "-"})
|
||||||
before = record_count(conn)
|
before = record_count(conn, uid)
|
||||||
added, dup, _ = upsert(conn, recs)
|
added, dup, _ = upsert(conn, uid, recs)
|
||||||
log("[migrate] 源文件 %d 条 → 新增 %d / 已存在 %d,入库前 %d 条,现共 %d 条"
|
log("[migrate] 源文件 %d 条 → 新增 %d / 已存在 %d,入库前 %d 条,现共 %d 条"
|
||||||
% (len(recs), added, dup, before, record_count(conn)))
|
% (len(recs), added, dup, before, record_count(conn, uid)))
|
||||||
return added
|
return added
|
||||||
|
|||||||
@@ -39,9 +39,18 @@ API_BASE = "https://www.workbuddy.cn"
|
|||||||
API_PATH = "/billing/meter/get-user-request-usage"
|
API_PATH = "/billing/meter/get-user-request-usage"
|
||||||
|
|
||||||
# ---------------- 采集参数默认值(可被 settings 表覆盖)----------------
|
# ---------------- 采集参数默认值(可被 settings 表覆盖)----------------
|
||||||
|
# settings 表是 (user_id, key) 复合主键:user_id=0 表示**实例级**,
|
||||||
|
# 其余表示**个人级**(每个账号一份,互不可见)。见下方的 GLOBAL_KEYS。
|
||||||
DEFAULTS = {
|
DEFAULTS = {
|
||||||
|
# ---- 实例级:连接的是哪个云端 ----
|
||||||
"api_base": API_BASE,
|
"api_base": API_BASE,
|
||||||
"api_path": API_PATH,
|
"api_path": API_PATH,
|
||||||
|
# ---- 实例级:开放注册与防攻击策略 ----
|
||||||
|
"allow_register": "1", # 是否开放自助注册
|
||||||
|
"register_max_per_ip": "3", # 同一 IP 每天最多注册几个账号
|
||||||
|
"captcha_policy": "always", # always | adaptive | off(见 CAPTCHA_POLICIES)
|
||||||
|
"captcha_length": "4", # 验证码字符数 4~6
|
||||||
|
# ---- 个人级:采集参数 ----
|
||||||
"page_size": "200",
|
"page_size": "200",
|
||||||
"rewind_minutes": "2", # 断点回退分钟数
|
"rewind_minutes": "2", # 断点回退分钟数
|
||||||
"drift_tolerance_minutes": "5", # 云端比本地早超过该值才告警
|
"drift_tolerance_minutes": "5", # 云端比本地早超过该值才告警
|
||||||
@@ -49,19 +58,36 @@ DEFAULTS = {
|
|||||||
"verify_days": "0", # 每次采集后做整日完整性校验的天数
|
"verify_days": "0", # 每次采集后做整日完整性校验的天数
|
||||||
"timeout": "30",
|
"timeout": "30",
|
||||||
"ssl_verify": "1", # 校验云端 HTTPS 证书(cookie 是凭证,不该裸奔)
|
"ssl_verify": "1", # 校验云端 HTTPS 证书(cookie 是凭证,不该裸奔)
|
||||||
# 调度
|
# ---- 个人级:调度 ----
|
||||||
"schedule_enabled": "1",
|
"schedule_enabled": "1",
|
||||||
"schedule_times": "09:00,17:00", # 每天固定时刻(逗号分隔,本地时区)
|
"schedule_times": "09:00,17:00", # 每天固定时刻(逗号分隔,本地时区)
|
||||||
"catch_up": "1", # 启动时补跑当天已错过且未执行的槽位
|
"catch_up": "1", # 启动时补跑当天已错过且未执行的槽位
|
||||||
"catch_up_grace_hours": "12", # 超过该小时数就不再补跑
|
"catch_up_grace_hours": "12", # 超过该小时数就不再补跑
|
||||||
# 凭证
|
# ---- 个人级:凭证(每个账号自己的,Cookie 静态加密后入库)----
|
||||||
"cookie": "",
|
"cookie": "",
|
||||||
"user_agent": ("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
|
"user_agent": ("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
|
||||||
"(KHTML, like Gecko) Chrome/153.0.0.0 Safari/537.36"),
|
"(KHTML, like Gecko) Chrome/153.0.0.0 Safari/537.36"),
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# 实例级配置:所有账号共用一份,只有管理员能改。
|
||||||
|
# 其余键(采集参数 / 调度 / 凭证)都是个人级 —— 这正是「多用户」的核心:
|
||||||
|
# 每个人填自己的 Cookie、收自己的数据、定自己的采集时刻。
|
||||||
|
GLOBAL_KEYS = {
|
||||||
|
"api_base", "api_path",
|
||||||
|
"allow_register", "register_max_per_ip", "captcha_policy", "captcha_length",
|
||||||
|
}
|
||||||
|
|
||||||
|
# 验证码策略
|
||||||
|
CAPTCHA_POLICIES = {
|
||||||
|
"always": "始终要求(默认,最安全)",
|
||||||
|
"adaptive": "仅在同一来源连续失败 2 次后要求",
|
||||||
|
"off": "关闭(仅当前面有可信网关做鉴权时才考虑)",
|
||||||
|
}
|
||||||
|
|
||||||
# 页面展示用:哪些键属于「敏感」,在界面上做掩码
|
# 页面展示用:哪些键属于「敏感」,在界面上做掩码
|
||||||
SECRET_KEYS = {"cookie"}
|
SECRET_KEYS = {"cookie"}
|
||||||
|
# 需要静态加密后再入库的键(明文只存在于内存与请求体里)
|
||||||
|
ENCRYPTED_KEYS = {"cookie"}
|
||||||
|
|
||||||
# 内部簿记键前缀:调度槽位标记等,**不属于用户可配置项**,
|
# 内部簿记键前缀:调度槽位标记等,**不属于用户可配置项**,
|
||||||
# 不在 /api/settings 里回传,也不允许通过接口写入。
|
# 不在 /api/settings 里回传,也不允许通过接口写入。
|
||||||
@@ -72,6 +98,11 @@ def is_internal_key(key):
|
|||||||
return any(str(key).startswith(p) for p in INTERNAL_PREFIXES)
|
return any(str(key).startswith(p) for p in INTERNAL_PREFIXES)
|
||||||
|
|
||||||
|
|
||||||
|
def is_global_key(key):
|
||||||
|
"""实例级键:所有账号共用一份,只有管理员可写。"""
|
||||||
|
return key in GLOBAL_KEYS
|
||||||
|
|
||||||
|
|
||||||
# ---------------- 设置项校验表 ----------------
|
# ---------------- 设置项校验表 ----------------
|
||||||
# 这些键必须能安全地转成数字:后台页面是自由文本框,用户敲错一个字符
|
# 这些键必须能安全地转成数字:后台页面是自由文本框,用户敲错一个字符
|
||||||
# 就会让采集在 int() 处抛 ValueError(历史 bug),所以写入时校验、读取时兜底。
|
# 就会让采集在 int() 处抛 ValueError(历史 bug),所以写入时校验、读取时兜底。
|
||||||
@@ -84,8 +115,10 @@ NUM_SETTINGS = {
|
|||||||
"verify_days": (0, 90, "天"),
|
"verify_days": (0, 90, "天"),
|
||||||
"timeout": (5, 300, "秒"),
|
"timeout": (5, 300, "秒"),
|
||||||
"catch_up_grace_hours": (1, 168, "小时"),
|
"catch_up_grace_hours": (1, 168, "小时"),
|
||||||
|
"captcha_length": (4, 6, "个字符"),
|
||||||
|
"register_max_per_ip": (1, 50, "个/天"),
|
||||||
}
|
}
|
||||||
BOOL_SETTINGS = {"schedule_enabled", "catch_up"}
|
BOOL_SETTINGS = {"schedule_enabled", "catch_up", "allow_register"}
|
||||||
|
|
||||||
_TRUE = ("1", "true", "yes", "on", "是", "启用")
|
_TRUE = ("1", "true", "yes", "on", "是", "启用")
|
||||||
|
|
||||||
@@ -127,6 +160,12 @@ def normalize_setting(key, raw):
|
|||||||
return None, "每日时刻格式不对,正确写法如 09:00,17:00"
|
return None, "每日时刻格式不对,正确写法如 09:00,17:00"
|
||||||
return ",".join(parsed), None
|
return ",".join(parsed), None
|
||||||
|
|
||||||
|
if key == "captcha_policy":
|
||||||
|
v = str(raw).strip().lower()
|
||||||
|
if v not in CAPTCHA_POLICIES:
|
||||||
|
return None, "验证码策略只能是 %s" % " / ".join(sorted(CAPTCHA_POLICIES))
|
||||||
|
return v, None
|
||||||
|
|
||||||
if key in ("api_base", "api_path"):
|
if key in ("api_base", "api_path"):
|
||||||
v = str(raw).strip()
|
v = str(raw).strip()
|
||||||
if not v:
|
if not v:
|
||||||
@@ -136,6 +175,7 @@ def normalize_setting(key, raw):
|
|||||||
return v, None
|
return v, None
|
||||||
|
|
||||||
if key == "cookie":
|
if key == "cookie":
|
||||||
|
# 明文原样返回,由 db.set_setting 负责加密后再落库
|
||||||
return str(raw).strip(), None
|
return str(raw).strip(), None
|
||||||
|
|
||||||
return str(raw).strip(), None
|
return str(raw).strip(), None
|
||||||
@@ -144,29 +184,64 @@ def normalize_setting(key, raw):
|
|||||||
DEFAULT_HOST = "0.0.0.0" # 局域网可访问
|
DEFAULT_HOST = "0.0.0.0" # 局域网可访问
|
||||||
DEFAULT_PORT = 8848
|
DEFAULT_PORT = 8848
|
||||||
SESSION_HOURS = 12
|
SESSION_HOURS = 12
|
||||||
MAX_LOGIN_FAILS = 5 # 同 IP 连续失败次数
|
MAX_LOGIN_FAILS = 5 # 同 IP / 同用户名连续失败次数
|
||||||
LOGIN_LOCK_MINUTES = 10
|
LOGIN_LOCK_MINUTES = 10
|
||||||
|
|
||||||
|
# ---------------- 账号与口令策略 ----------------
|
||||||
|
USERNAME_RE = r"^[A-Za-z0-9][A-Za-z0-9_.\-]{2,31}$" # 3~32 位,字母开头
|
||||||
|
PASSWORD_MIN = 8
|
||||||
|
PASSWORD_MAX = 128
|
||||||
|
# 开启注册后,未配置 Cookie 的新账号在概览页会被提示「去配置」——
|
||||||
|
# 采集只使用**本人**的 Cookie,绝不复用别人的(否则会串号)。
|
||||||
|
PROFILE_EMAIL_MAX = 128
|
||||||
|
|
||||||
|
# 会话 Cookie 是否只走 HTTPS。纯局域网 HTTP 部署必须留 0,否则浏览器不发送,
|
||||||
|
# 表现为「登录成功但立刻又跳回登录页」,极难排查。
|
||||||
|
COOKIE_SECURE = os.environ.get("WB_COOKIE_SECURE", "0").strip() in ("1", "true", "yes", "on")
|
||||||
|
|
||||||
|
|
||||||
def ensure_dirs():
|
def ensure_dirs():
|
||||||
for d in (DATA_DIR, LOG_DIR, EXPORT_DIR):
|
for d in (DATA_DIR, LOG_DIR, EXPORT_DIR):
|
||||||
os.makedirs(d, exist_ok=True)
|
os.makedirs(d, exist_ok=True)
|
||||||
|
|
||||||
|
|
||||||
|
def _instance_read():
|
||||||
|
"""读 data/instance.json(不存在或损坏都当空字典,不让启动因此失败)。"""
|
||||||
|
ensure_dirs()
|
||||||
|
if not os.path.exists(INSTANCE_FILE):
|
||||||
|
return {}
|
||||||
|
try:
|
||||||
|
with open(INSTANCE_FILE, "r", encoding="utf-8") as f:
|
||||||
|
return json.load(f) or {}
|
||||||
|
except (OSError, ValueError):
|
||||||
|
return {}
|
||||||
|
|
||||||
|
|
||||||
|
def _instance_init(key, maker):
|
||||||
|
"""取 instance.json 里的 key,没有就生成并持久化。"""
|
||||||
|
data = _instance_read()
|
||||||
|
val = data.get(key)
|
||||||
|
if not val:
|
||||||
|
val = maker()
|
||||||
|
data[key] = val
|
||||||
|
try:
|
||||||
|
with open(INSTANCE_FILE, "w", encoding="utf-8") as f:
|
||||||
|
json.dump(data, f, ensure_ascii=False, indent=2)
|
||||||
|
except OSError:
|
||||||
|
pass # 只读文件系统时退化为「本次进程内有效」
|
||||||
|
return val
|
||||||
|
|
||||||
|
|
||||||
def secret_key():
|
def secret_key():
|
||||||
"""SECRET_KEY 持久化在 data/instance.json,避免每次重启把登录态全踢掉。"""
|
"""SECRET_KEY 持久化在 data/instance.json,避免每次重启把登录态全踢掉。"""
|
||||||
ensure_dirs()
|
return _instance_init("secret_key", lambda: secrets.token_hex(32))
|
||||||
data = {}
|
|
||||||
if os.path.exists(INSTANCE_FILE):
|
|
||||||
try:
|
def encryption_key():
|
||||||
with open(INSTANCE_FILE, "r", encoding="utf-8") as f:
|
"""Cookie 静态加密的主密钥(32 字节)。
|
||||||
data = json.load(f) or {}
|
|
||||||
except (OSError, ValueError):
|
与 SECRET_KEY **分开**存放:两者轮换的代价完全不同 —— 换 SECRET_KEY
|
||||||
data = {}
|
只是让所有人重新登录,换这把会让已存的 Cookie 全部解不开。
|
||||||
key = data.get("secret_key")
|
所以混用同一个值会让「想轮换其中一个」变成一件危险的事。
|
||||||
if not key:
|
"""
|
||||||
key = secrets.token_hex(32)
|
return bytes.fromhex(_instance_init("cookie_key", lambda: secrets.token_hex(32)))
|
||||||
data["secret_key"] = key
|
|
||||||
with open(INSTANCE_FILE, "w", encoding="utf-8") as f:
|
|
||||||
json.dump(data, f, ensure_ascii=False, indent=2)
|
|
||||||
return key
|
|
||||||
|
|||||||
@@ -0,0 +1,177 @@
|
|||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
# SPDX-License-Identifier: MIT
|
||||||
|
# Copyright (c) 2026 Wang Chuanli
|
||||||
|
|
||||||
|
"""对称加密 —— 给「数据库里的 Cookie」做静态加密。
|
||||||
|
|
||||||
|
为什么要自己写而不用 `cryptography` / `pycryptodome`
|
||||||
|
----------------------------------------------------
|
||||||
|
本项目刻意保持零第三方依赖(`requirements.txt` 只有 Flask / waitress / openpyxl),
|
||||||
|
而这里需要的原语只有两个,都能在 RFC 里逐行对照实现:
|
||||||
|
|
||||||
|
* **ChaCha20** 流密码(RFC 8439 §2.3)—— 加密
|
||||||
|
* **HMAC-SHA256**(RFC 2104)—— 认证,采用 **encrypt-then-MAC**
|
||||||
|
|
||||||
|
明文密钥不是口令而是 32 字节随机数,所以派生不需要慢速 KDF
|
||||||
|
(PBKDF2/scrypt 是为「低熵口令」设计的),用 HMAC 做一次密钥分离即可。
|
||||||
|
|
||||||
|
密文格式
|
||||||
|
--------
|
||||||
|
v1.<b64(salt)>.<b64(nonce)>.<b64(ciphertext)>.<b64(tag)>
|
||||||
|
|
||||||
|
* salt —— 16 字节随机,用于把主密钥分离成 enc/mac 两把子密钥
|
||||||
|
* nonce —— 12 字节随机,每次加密都重新生成(绝不复用)
|
||||||
|
* tag —— HMAC(mac_key, nonce || ciphertext) 的 SHA-256
|
||||||
|
|
||||||
|
**不做压缩**:Cookie 是几百到几千字节的高熵串,压缩比接近 1,
|
||||||
|
反而会引入 CRIME 类侧信道,不值得。
|
||||||
|
|
||||||
|
向后兼容
|
||||||
|
--------
|
||||||
|
`decrypt()` 遇到不是 `v1.` 开头的值会**原样返回**,这样从旧版本
|
||||||
|
(Cookie 明文存在 settings 表)升级过来不会立刻炸;下次写入时自然
|
||||||
|
会被改写为密文(见 `db.set_secret`)。
|
||||||
|
"""
|
||||||
|
|
||||||
|
import base64
|
||||||
|
import hashlib
|
||||||
|
import hmac
|
||||||
|
import secrets
|
||||||
|
import struct
|
||||||
|
|
||||||
|
PREFIX = "v1."
|
||||||
|
|
||||||
|
# RFC 8439 §2.3 的常数:"expand 32-byte k"
|
||||||
|
_CONST = b"expand 32-byte k"
|
||||||
|
_MASK = 0xFFFFFFFF
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------- ChaCha20 ----------------
|
||||||
|
def _rotl32(v, c):
|
||||||
|
return ((v << c) & _MASK) | (v >> (32 - c))
|
||||||
|
|
||||||
|
|
||||||
|
def _quarter_round(s, a, b, c, d):
|
||||||
|
"""RFC 8439 §2.1。就地修改 s。"""
|
||||||
|
s[a] = (s[a] + s[b]) & _MASK
|
||||||
|
s[d] = _rotl32(s[d] ^ s[a], 16)
|
||||||
|
s[c] = (s[c] + s[d]) & _MASK
|
||||||
|
s[b] = _rotl32(s[b] ^ s[c], 12)
|
||||||
|
s[a] = (s[a] + s[b]) & _MASK
|
||||||
|
s[d] = _rotl32(s[d] ^ s[a], 8)
|
||||||
|
s[c] = (s[c] + s[d]) & _MASK
|
||||||
|
s[b] = _rotl32(s[b] ^ s[c], 7)
|
||||||
|
|
||||||
|
|
||||||
|
def chacha20_block(key32, counter, nonce12):
|
||||||
|
"""产出一个 64 字节的块(RFC 8439 §2.3.2)。"""
|
||||||
|
st = (list(struct.unpack("<4I", _CONST))
|
||||||
|
+ list(struct.unpack("<8I", key32))
|
||||||
|
+ [counter & _MASK]
|
||||||
|
+ list(struct.unpack("<3I", nonce12)))
|
||||||
|
w = list(st)
|
||||||
|
for _ in range(10): # 10 组 = 20 轮
|
||||||
|
_quarter_round(w, 0, 4, 8, 12)
|
||||||
|
_quarter_round(w, 1, 5, 9, 13)
|
||||||
|
_quarter_round(w, 2, 6, 10, 14)
|
||||||
|
_quarter_round(w, 3, 7, 11, 15)
|
||||||
|
_quarter_round(w, 0, 5, 10, 15)
|
||||||
|
_quarter_round(w, 1, 6, 11, 12)
|
||||||
|
_quarter_round(w, 2, 7, 8, 13)
|
||||||
|
_quarter_round(w, 3, 4, 9, 14)
|
||||||
|
return struct.pack("<16I", *[(w[i] + st[i]) & _MASK for i in range(16)])
|
||||||
|
|
||||||
|
|
||||||
|
def _keystream(key32, nonce12, n):
|
||||||
|
"""按需生成 n 字节密钥流。counter 从 1 开始(0 号块留给 Poly1305 用,这里不用)。"""
|
||||||
|
out = bytearray()
|
||||||
|
counter = 1
|
||||||
|
while len(out) < n:
|
||||||
|
out += chacha20_block(key32, counter, nonce12)
|
||||||
|
counter += 1
|
||||||
|
return bytes(out[:n])
|
||||||
|
|
||||||
|
|
||||||
|
def _xor(a, b):
|
||||||
|
return bytes(x ^ y for x, y in zip(a, b))
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------- 密钥分离 ----------------
|
||||||
|
def _derive(master, salt, label):
|
||||||
|
"""HMAC 做一次密钥分离:主密钥是高熵随机数,一次 HMAC 足够。"""
|
||||||
|
return hmac.new(master, salt + label, hashlib.sha256).digest()
|
||||||
|
|
||||||
|
|
||||||
|
def _b64(raw):
|
||||||
|
return base64.urlsafe_b64encode(raw).decode("ascii").rstrip("=")
|
||||||
|
|
||||||
|
|
||||||
|
def _unb64(text):
|
||||||
|
pad = "=" * (-len(text) % 4)
|
||||||
|
return base64.urlsafe_b64decode(text + pad)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------- 对外接口 ----------------
|
||||||
|
def is_encrypted(value):
|
||||||
|
return isinstance(value, str) and value.startswith(PREFIX)
|
||||||
|
|
||||||
|
|
||||||
|
def encrypt(plaintext, master):
|
||||||
|
"""加密任意字符串;空值原样返回(不产生「有密文的空值」这种歧义状态)。
|
||||||
|
|
||||||
|
master: 32 字节主密钥(bytes)。返回可直接存库的 ASCII 字符串。
|
||||||
|
"""
|
||||||
|
if plaintext is None or plaintext == "":
|
||||||
|
return ""
|
||||||
|
if not isinstance(master, (bytes, bytearray)) or len(master) != 32:
|
||||||
|
raise ValueError("主密钥必须是 32 字节")
|
||||||
|
data = plaintext.encode("utf-8") if isinstance(plaintext, str) else bytes(plaintext)
|
||||||
|
salt = secrets.token_bytes(16)
|
||||||
|
nonce = secrets.token_bytes(12)
|
||||||
|
enc_key = _derive(bytes(master), salt, b"enc")
|
||||||
|
mac_key = _derive(bytes(master), salt, b"mac")
|
||||||
|
ct = _xor(data, _keystream(enc_key, nonce, len(data)))
|
||||||
|
tag = hmac.new(mac_key, nonce + ct, hashlib.sha256).digest()
|
||||||
|
return PREFIX + ".".join((_b64(salt), _b64(nonce), _b64(ct), _b64(tag)))
|
||||||
|
|
||||||
|
|
||||||
|
class DecryptError(ValueError):
|
||||||
|
"""密文被篡改、格式损坏或密钥不对。"""
|
||||||
|
|
||||||
|
|
||||||
|
def decrypt(token, master):
|
||||||
|
"""解密。
|
||||||
|
|
||||||
|
* 非密文(历史明文、空串)原样返回,便于平滑升级
|
||||||
|
* 密文校验失败抛 DecryptError —— **绝不**「失败就返回原值」,
|
||||||
|
否则一次篡改会被静默当成合法明文用下去
|
||||||
|
"""
|
||||||
|
if not token or not isinstance(token, str):
|
||||||
|
return ""
|
||||||
|
if not token.startswith(PREFIX):
|
||||||
|
return token # 兼容旧的明文存储
|
||||||
|
if not isinstance(master, (bytes, bytearray)) or len(master) != 32:
|
||||||
|
raise DecryptError("主密钥必须是 32 字节")
|
||||||
|
parts = token[len(PREFIX):].split(".")
|
||||||
|
if len(parts) != 4:
|
||||||
|
raise DecryptError("密文格式不正确")
|
||||||
|
try:
|
||||||
|
salt, nonce, ct, tag = (_unb64(p) for p in parts)
|
||||||
|
except (ValueError, TypeError) as e:
|
||||||
|
raise DecryptError("密文 base64 解码失败:%s" % e)
|
||||||
|
if len(salt) != 16 or len(nonce) != 12 or len(tag) != 32:
|
||||||
|
raise DecryptError("密文长度不合法")
|
||||||
|
mac_key = _derive(bytes(master), salt, b"mac")
|
||||||
|
want = hmac.new(mac_key, nonce + ct, hashlib.sha256).digest()
|
||||||
|
# 先比 MAC 再解密:认证失败时不接触密文,避免 padding/解析类侧信道
|
||||||
|
if not hmac.compare_digest(want, tag):
|
||||||
|
raise DecryptError("完整性校验失败(密文被篡改或主密钥已更换)")
|
||||||
|
enc_key = _derive(bytes(master), salt, b"enc")
|
||||||
|
return _xor(ct, _keystream(enc_key, nonce, len(ct))).decode("utf-8")
|
||||||
|
|
||||||
|
|
||||||
|
def fingerprint(plaintext):
|
||||||
|
"""值指纹:用于「是否换过」的判断,不能反推原文。"""
|
||||||
|
if not plaintext:
|
||||||
|
return ""
|
||||||
|
return hashlib.sha256(plaintext.encode("utf-8")).hexdigest()[:16]
|
||||||
@@ -2,7 +2,7 @@
|
|||||||
# SPDX-License-Identifier: MIT
|
# SPDX-License-Identifier: MIT
|
||||||
# Copyright (c) 2026 Wang Chuanli
|
# Copyright (c) 2026 Wang Chuanli
|
||||||
|
|
||||||
"""SQLite 访问层。
|
"""SQLite 访问层(多用户版)。
|
||||||
|
|
||||||
并发约定(重要):
|
并发约定(重要):
|
||||||
* WAL 模式 —— 采集写入期间页面查询不会被 `database is locked` 挡住
|
* WAL 模式 —— 采集写入期间页面查询不会被 `database is locked` 挡住
|
||||||
@@ -10,18 +10,39 @@
|
|||||||
(由 scheduler / CLI 共享的 collect.lock 保证)
|
(由 scheduler / CLI 共享的 collect.lock 保证)
|
||||||
* busy_timeout=8s —— 偶发并发时等待而不是立刻报错
|
* busy_timeout=8s —— 偶发并发时等待而不是立刻报错
|
||||||
* 每个线程独立连接(sqlite3 默认禁止跨线程复用连接)
|
* 每个线程独立连接(sqlite3 默认禁止跨线程复用连接)
|
||||||
|
|
||||||
|
多用户约定(改代码前务必先读):
|
||||||
|
* `user_id = 0` 在 settings / collect_runs / audit_log 里表示**实例级**;
|
||||||
|
usage_records 里 0 是「历史遗留数据尚未归属」的兜底值,正常不会出现。
|
||||||
|
* `get_settings()` 会把 `ENCRYPTED_KEYS`(Cookie)**一律置空**;
|
||||||
|
要拿明文只有 `get_secret()` 一条路。这样任何「顺手打印一下全部配置」
|
||||||
|
的代码都不可能把凭证带出去。
|
||||||
|
* `NO_FALLBACK_KEYS`(Cookie / User-Agent)**不参与实例级回退**:
|
||||||
|
Cookie 是账号凭证,回落等于串号,是最严重的一类越权。
|
||||||
"""
|
"""
|
||||||
import os
|
import os
|
||||||
import sqlite3
|
import sqlite3
|
||||||
import threading
|
import threading
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
|
|
||||||
from . import config
|
from . import config, crypto
|
||||||
|
|
||||||
_local = threading.local()
|
_local = threading.local()
|
||||||
_init_lock = threading.Lock()
|
_init_lock = threading.Lock()
|
||||||
_initialized = False
|
_initialized = False
|
||||||
|
|
||||||
|
# 库结构版本。写在 PRAGMA user_version 里,用来判断是否需要迁移。
|
||||||
|
# 1 -> 单用户布局(settings 以 key 为主键,usage_records 以 request_id 为主键)
|
||||||
|
# 2 -> 多用户布局(见 schema.sql 顶部说明)
|
||||||
|
DB_SCHEMA_VERSION = 2
|
||||||
|
|
||||||
|
# 这些键即使个人作用域没有值,也**不**回落到实例级
|
||||||
|
NO_FALLBACK_KEYS = {"cookie", "user_agent"}
|
||||||
|
|
||||||
|
|
||||||
|
class SecretUnreadable(Exception):
|
||||||
|
"""密文解不开 —— 通常是 data/instance.json 里的 cookie_key 被换过。"""
|
||||||
|
|
||||||
|
|
||||||
def now_str():
|
def now_str():
|
||||||
return datetime.now().strftime("%Y-%m-%d %H:%M:%S")
|
return datetime.now().strftime("%Y-%m-%d %H:%M:%S")
|
||||||
@@ -58,87 +79,331 @@ def close_thread_conn():
|
|||||||
_local.conn = None
|
_local.conn = None
|
||||||
|
|
||||||
|
|
||||||
# ---------------- 初始化 ----------------
|
def _schema_sql():
|
||||||
|
with open(os.path.join(os.path.dirname(os.path.abspath(__file__)), "schema.sql"),
|
||||||
|
"r", encoding="utf-8") as f:
|
||||||
|
return f.read()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------- 初始化 / 迁移 ----------------
|
||||||
|
def _table_cols(conn, table):
|
||||||
|
return {r["name"] for r in conn.execute("PRAGMA table_info(%s)" % table)}
|
||||||
|
|
||||||
|
|
||||||
|
def _has_table(conn, name):
|
||||||
|
return bool(conn.execute(
|
||||||
|
"SELECT 1 FROM sqlite_master WHERE type='table' AND name=?", (name,)).fetchone())
|
||||||
|
|
||||||
|
|
||||||
|
def _first_owner_uid(conn):
|
||||||
|
"""历史数据归谁:优先第一个管理员,其次第一个账号。"""
|
||||||
|
row = conn.execute("SELECT id FROM users WHERE is_admin=1 ORDER BY id LIMIT 1").fetchone()
|
||||||
|
if row:
|
||||||
|
return row["id"]
|
||||||
|
row = conn.execute("SELECT id FROM users ORDER BY id LIMIT 1").fetchone()
|
||||||
|
return row["id"] if row else 0
|
||||||
|
|
||||||
|
|
||||||
|
def _drop_all_user_indexes(conn):
|
||||||
|
"""删掉本项目自建的全部索引(idx_*)。
|
||||||
|
|
||||||
|
必须先删:`ALTER TABLE ... RENAME TO` 会**把索引一起带走**(名字仍指向
|
||||||
|
改名后的表),于是后面 `CREATE INDEX IF NOT EXISTS` 会被当成「已存在」
|
||||||
|
静默跳过,最终新表上一个索引都没有 —— 表面完全正常,只是慢几百倍。
|
||||||
|
`sqlite_autoindex_*` 是主键/唯一约束的隐式索引,不能动,靠前缀过滤掉。
|
||||||
|
"""
|
||||||
|
names = [r["name"] for r in conn.execute(
|
||||||
|
"SELECT name FROM sqlite_master WHERE type='index' AND name LIKE 'idx_%'")]
|
||||||
|
for n in names:
|
||||||
|
conn.execute('DROP INDEX IF EXISTS "%s"' % n)
|
||||||
|
return names
|
||||||
|
|
||||||
|
|
||||||
|
def _migrate(conn):
|
||||||
|
"""把老库升到 DB_SCHEMA_VERSION。幂等,返回迁移说明列表。
|
||||||
|
|
||||||
|
顺序不能变:
|
||||||
|
1. 删索引(否则改名会把索引名占住,新表建不出索引)
|
||||||
|
2. 改名主键变了的表(settings / usage_records)
|
||||||
|
3. ALTER 加列(collect_runs / audit_log / users)
|
||||||
|
4. 跑 schema.sql —— 此时所有列都齐了,表与索引一次建全
|
||||||
|
5. 回填数据、删掉 _v1_ 旧表
|
||||||
|
"""
|
||||||
|
ver = conn.execute("PRAGMA user_version").fetchone()[0]
|
||||||
|
if ver >= DB_SCHEMA_VERSION:
|
||||||
|
return []
|
||||||
|
done = []
|
||||||
|
ts = now_str()
|
||||||
|
owner = _first_owner_uid(conn)
|
||||||
|
|
||||||
|
tset = _table_cols(conn, "settings")
|
||||||
|
tur = _table_cols(conn, "usage_records")
|
||||||
|
rebuild_settings = "user_id" not in tset
|
||||||
|
rebuild_records = "user_id" not in tur
|
||||||
|
|
||||||
|
_drop_all_user_indexes(conn)
|
||||||
|
if rebuild_settings:
|
||||||
|
conn.execute("ALTER TABLE settings RENAME TO _v1_settings")
|
||||||
|
if rebuild_records:
|
||||||
|
conn.execute("ALTER TABLE usage_records RENAME TO _v1_usage_records")
|
||||||
|
|
||||||
|
# ---- 只加列的表用 ALTER,代价小得多。放在建表之前,好让 schema.sql
|
||||||
|
# 里的 CREATE INDEX 一次就成功(索引引用了这些新列)----
|
||||||
|
if "user_id" not in _table_cols(conn, "collect_runs"):
|
||||||
|
conn.execute("ALTER TABLE collect_runs ADD COLUMN user_id INTEGER NOT NULL DEFAULT 0")
|
||||||
|
conn.execute("UPDATE collect_runs SET user_id=?", (owner,))
|
||||||
|
done.append("collect_runs 增加 user_id")
|
||||||
|
if "user_id" not in _table_cols(conn, "audit_log"):
|
||||||
|
conn.execute("ALTER TABLE audit_log ADD COLUMN user_id INTEGER NOT NULL DEFAULT 0")
|
||||||
|
done.append("audit_log 增加 user_id")
|
||||||
|
tusers = _table_cols(conn, "users")
|
||||||
|
for col, ddl in (("email", "TEXT"),
|
||||||
|
("status", "TEXT NOT NULL DEFAULT 'active'"),
|
||||||
|
("register_ip", "TEXT"),
|
||||||
|
("last_login_ip", "TEXT")):
|
||||||
|
if col not in tusers:
|
||||||
|
conn.execute("ALTER TABLE users ADD COLUMN %s %s" % (col, ddl))
|
||||||
|
done.append("users 增加 %s" % col)
|
||||||
|
conn.execute("UPDATE users SET status='active' WHERE status IS NULL OR status=''")
|
||||||
|
|
||||||
|
conn.executescript(_schema_sql()) # 建出新表 + 全部索引
|
||||||
|
|
||||||
|
if rebuild_settings:
|
||||||
|
# 老布局里 cookie / user_agent 是实例级的 —— 留在实例级等于
|
||||||
|
# 「所有人共用管理员的凭证」,必须归到 owner 名下,且之后永不回落。
|
||||||
|
conn.execute("INSERT INTO settings(user_id,key,value,updated_at)"
|
||||||
|
" SELECT 0,key,value,updated_at FROM _v1_settings"
|
||||||
|
" WHERE key NOT IN ('cookie','user_agent')")
|
||||||
|
conn.execute("INSERT INTO settings(user_id,key,value,updated_at)"
|
||||||
|
" SELECT ?,key,value,updated_at FROM _v1_settings"
|
||||||
|
" WHERE key IN ('cookie','user_agent') AND value IS NOT NULL AND value <> ''",
|
||||||
|
(owner,))
|
||||||
|
conn.execute("DROP TABLE _v1_settings")
|
||||||
|
done.append("settings 改为 (user_id,key) 复合主键:旧值归实例级,"
|
||||||
|
"Cookie/UA 已归属账号 #%d" % owner)
|
||||||
|
|
||||||
|
if rebuild_records:
|
||||||
|
conn.execute("INSERT INTO usage_records(user_id,request_id,ts,day,hour,model,client,"
|
||||||
|
"credits,prompt,first_seen,last_seen,cloud_ts)"
|
||||||
|
" SELECT ?,request_id,ts,day,hour,model,client,credits,prompt,"
|
||||||
|
"first_seen,last_seen,cloud_ts FROM _v1_usage_records", (owner,))
|
||||||
|
conn.execute("DROP TABLE _v1_usage_records")
|
||||||
|
done.append("usage_records 增加 user_id,主键改为 (user_id, request_id),"
|
||||||
|
"历史数据归属账号 #%d" % owner)
|
||||||
|
|
||||||
|
conn.execute("UPDATE users SET is_admin=1 WHERE id=?", (owner,))
|
||||||
|
conn.execute("PRAGMA user_version=%d" % DB_SCHEMA_VERSION)
|
||||||
|
if done:
|
||||||
|
conn.execute("INSERT INTO audit_log(user_id,at,actor,action,detail,ip)"
|
||||||
|
" VALUES(0,?,?,?,?,?)",
|
||||||
|
(ts, "system", "schema_migrate", ";".join(done)[:500], "127.0.0.1"))
|
||||||
|
return done
|
||||||
|
|
||||||
|
|
||||||
|
def _encrypt_legacy_secrets(conn):
|
||||||
|
"""把历史**明文**凭证就地加密。幂等,可反复执行。
|
||||||
|
|
||||||
|
老版本把 Cookie 直接明文写进 settings 表。升级后即便功能正常,
|
||||||
|
「库里躺着一段明文凭证」本身就是风险:备份文件、磁盘镜像、
|
||||||
|
误提交、排障时的一次 dump 都会把它带出去。
|
||||||
|
这里对 `config.ENCRYPTED_KEYS` 里所有**非 v1. 前缀**的值加密一次;
|
||||||
|
已加密的值会因前缀判定被跳过,所以每次启动跑一遍是安全的。
|
||||||
|
"""
|
||||||
|
if not config.ENCRYPTED_KEYS:
|
||||||
|
return []
|
||||||
|
key = config.encryption_key()
|
||||||
|
done = []
|
||||||
|
for r in conn.execute("SELECT user_id,key,value FROM settings").fetchall():
|
||||||
|
if r["key"] not in config.ENCRYPTED_KEYS:
|
||||||
|
continue
|
||||||
|
raw = r["value"]
|
||||||
|
if not raw or crypto.is_encrypted(raw):
|
||||||
|
continue
|
||||||
|
conn.execute("UPDATE settings SET value=?,updated_at=? WHERE user_id=? AND key=?",
|
||||||
|
(crypto.encrypt(raw, key), now_str(), r["user_id"], r["key"]))
|
||||||
|
done.append("settings[uid=%s].%s" % (r["user_id"], r["key"]))
|
||||||
|
return done
|
||||||
|
|
||||||
|
|
||||||
def init_db(conn=None, create_admin=True, admin_user="admin", admin_password=None):
|
def init_db(conn=None, create_admin=True, admin_user="admin", admin_password=None):
|
||||||
"""建表 + 灌默认配置。可重复执行(幂等)。"""
|
"""建表 / 迁移 / 灌默认配置。可重复执行(幂等)。返回迁移说明列表。"""
|
||||||
global _initialized
|
global _initialized
|
||||||
own = conn is None
|
own = conn is None
|
||||||
conn = conn or connect()
|
conn = conn or connect()
|
||||||
try:
|
try:
|
||||||
with open(os.path.join(os.path.dirname(os.path.abspath(__file__)), "schema.sql"),
|
if not _has_table(conn, "users"):
|
||||||
"r", encoding="utf-8") as f:
|
# 全新库:schema.sql 一次到位(避免走迁移路径去 ALTER 不存在的表)
|
||||||
conn.executescript(f.read())
|
conn.executescript(_schema_sql())
|
||||||
# 默认配置(不覆盖已有值)
|
conn.execute("PRAGMA user_version=%d" % DB_SCHEMA_VERSION)
|
||||||
|
migrated = []
|
||||||
|
else:
|
||||||
|
migrated = _migrate(conn)
|
||||||
|
# 无论如何再跑一次:幂等补齐(例如后续版本新增了表/索引,
|
||||||
|
# 而老库的 user_version 已经是最新,就不会走 _migrate 了)
|
||||||
|
conn.executescript(_schema_sql())
|
||||||
ts = now_str()
|
ts = now_str()
|
||||||
|
# 默认配置灌在**实例级**(user_id=0)。个人作用域不预置行,
|
||||||
|
# 读取时按「个人 -> 实例 -> DEFAULTS」三级回落,语义更清楚。
|
||||||
for k, v in config.DEFAULTS.items():
|
for k, v in config.DEFAULTS.items():
|
||||||
conn.execute("INSERT OR IGNORE INTO settings(key,value,updated_at) VALUES(?,?,?)",
|
conn.execute("INSERT OR IGNORE INTO settings(user_id,key,value,updated_at)"
|
||||||
(k, v, ts))
|
" VALUES(0,?,?,?)", (k, v, ts))
|
||||||
|
# 顺手把历史明文凭证加密(幂等;新库无事可做)
|
||||||
|
enc = _encrypt_legacy_secrets(conn)
|
||||||
|
if enc:
|
||||||
|
migrated.append("明文凭证已加密:%s" % ", ".join(enc))
|
||||||
|
conn.execute("INSERT INTO audit_log(user_id,at,actor,action,detail,ip)"
|
||||||
|
" VALUES(0,?,?,?,?,?)",
|
||||||
|
(ts, "system", "encrypt_secrets",
|
||||||
|
"明文凭证已加密:%s" % ", ".join(enc)[:400], "127.0.0.1"))
|
||||||
if create_admin:
|
if create_admin:
|
||||||
n = conn.execute("SELECT COUNT(*) FROM users").fetchone()[0]
|
n = conn.execute("SELECT COUNT(*) FROM users").fetchone()[0]
|
||||||
if n == 0:
|
if n == 0:
|
||||||
from .security import hash_password
|
from .security import hash_password
|
||||||
pwd = admin_password or "admin123"
|
pwd = admin_password or "admin123"
|
||||||
conn.execute(
|
conn.execute(
|
||||||
"INSERT INTO users(username,password_hash,display_name,is_admin,created_at)"
|
"INSERT INTO users(username,password_hash,display_name,is_admin,status,"
|
||||||
" VALUES(?,?,?,1,?)", (admin_user, hash_password(pwd), "管理员", ts))
|
" created_at) VALUES(?,?,?,1,'active',?)",
|
||||||
|
(admin_user, hash_password(pwd), "管理员", ts))
|
||||||
_initialized = True
|
_initialized = True
|
||||||
|
return migrated
|
||||||
finally:
|
finally:
|
||||||
if own:
|
if own:
|
||||||
conn.close()
|
conn.close()
|
||||||
|
|
||||||
|
|
||||||
# ---------------- 配置读写 ----------------
|
# ---------------- 配置读写(按作用域) ----------------
|
||||||
def get_setting(conn, key, default=None):
|
def get_setting(conn, key, default=None, uid=0):
|
||||||
row = conn.execute("SELECT value FROM settings WHERE key=?", (key,)).fetchone()
|
"""取单个配置。
|
||||||
if row is None or row["value"] is None:
|
|
||||||
return config.DEFAULTS.get(key, default)
|
**加密键一律返回空串**:想拿 Cookie 明文只能用 get_secret(),
|
||||||
return row["value"]
|
避免任何「顺手读一下配置」的代码把凭证带进日志或响应体。
|
||||||
|
"""
|
||||||
|
if key in config.ENCRYPTED_KEYS:
|
||||||
|
return ""
|
||||||
|
uid = 0 if config.is_global_key(key) else (uid or 0)
|
||||||
|
row = conn.execute("SELECT value FROM settings WHERE user_id=? AND key=?",
|
||||||
|
(uid, key)).fetchone()
|
||||||
|
if row is not None and row["value"] is not None:
|
||||||
|
return row["value"]
|
||||||
|
if uid and key not in NO_FALLBACK_KEYS:
|
||||||
|
row = conn.execute("SELECT value FROM settings WHERE user_id=0 AND key=?",
|
||||||
|
(key,)).fetchone()
|
||||||
|
if row is not None and row["value"] is not None:
|
||||||
|
return row["value"]
|
||||||
|
return config.DEFAULTS.get(key, default)
|
||||||
|
|
||||||
|
|
||||||
def get_settings(conn, keys=None):
|
def get_settings(conn, keys=None, uid=0):
|
||||||
rows = conn.execute("SELECT key,value FROM settings").fetchall()
|
"""取该账号的**有效配置**(DEFAULTS -> 实例级 -> 个人级 三级合并)。
|
||||||
got = {r["key"]: r["value"] for r in rows}
|
|
||||||
|
Cookie 等加密键固定为空串,页面/接口可以直接整体回传。
|
||||||
|
"""
|
||||||
out = dict(config.DEFAULTS)
|
out = dict(config.DEFAULTS)
|
||||||
out.update(got)
|
for r in conn.execute("SELECT key,value FROM settings WHERE user_id=0"):
|
||||||
|
out[r["key"]] = r["value"]
|
||||||
|
if uid:
|
||||||
|
for r in conn.execute("SELECT key,value FROM settings WHERE user_id=?", (uid,)):
|
||||||
|
if config.is_global_key(r["key"]):
|
||||||
|
continue # 个人作用域里不该有全局键,有也不认
|
||||||
|
out[r["key"]] = r["value"]
|
||||||
|
for k in config.ENCRYPTED_KEYS:
|
||||||
|
out[k] = ""
|
||||||
if keys:
|
if keys:
|
||||||
return {k: out.get(k) for k in keys}
|
return {k: out.get(k) for k in keys}
|
||||||
return out
|
return out
|
||||||
|
|
||||||
|
|
||||||
def set_setting(conn, key, value):
|
def set_setting(conn, key, value, uid=0):
|
||||||
conn.execute("INSERT INTO settings(key,value,updated_at) VALUES(?,?,?) "
|
"""写单个配置。全局键强制落到 user_id=0;加密键自动加密后落库。"""
|
||||||
"ON CONFLICT(key) DO UPDATE SET value=excluded.value, updated_at=excluded.updated_at",
|
uid = 0 if config.is_global_key(key) else (uid or 0)
|
||||||
(key, "" if value is None else str(value), now_str()))
|
text = "" if value is None else str(value)
|
||||||
|
if key in config.ENCRYPTED_KEYS and text:
|
||||||
|
text = crypto.encrypt(text, config.encryption_key())
|
||||||
|
conn.execute("INSERT INTO settings(user_id,key,value,updated_at) VALUES(?,?,?,?) "
|
||||||
|
"ON CONFLICT(user_id,key) DO UPDATE SET value=excluded.value,"
|
||||||
|
" updated_at=excluded.updated_at", (uid, key, text, now_str()))
|
||||||
|
|
||||||
|
|
||||||
def set_settings(conn, pairs):
|
def set_settings(conn, pairs, uid=0):
|
||||||
for k, v in pairs.items():
|
for k, v in pairs.items():
|
||||||
set_setting(conn, k, v)
|
set_setting(conn, k, v, uid)
|
||||||
|
|
||||||
|
|
||||||
def get_int(conn, key, default=0):
|
def get_int(conn, key, default=0, uid=0):
|
||||||
try:
|
try:
|
||||||
return int(float(get_setting(conn, key, default)))
|
return int(float(get_setting(conn, key, default, uid)))
|
||||||
except (TypeError, ValueError):
|
except (TypeError, ValueError):
|
||||||
return default
|
return default
|
||||||
|
|
||||||
|
|
||||||
def get_float(conn, key, default=0.0):
|
def get_float(conn, key, default=0.0, uid=0):
|
||||||
try:
|
try:
|
||||||
return float(get_setting(conn, key, default))
|
return float(get_setting(conn, key, default, uid))
|
||||||
except (TypeError, ValueError):
|
except (TypeError, ValueError):
|
||||||
return default
|
return default
|
||||||
|
|
||||||
|
|
||||||
def get_bool(conn, key, default=False):
|
def get_bool(conn, key, default=False, uid=0):
|
||||||
v = str(get_setting(conn, key, "1" if default else "0")).strip().lower()
|
v = str(get_setting(conn, key, "1" if default else "0", uid)).strip().lower()
|
||||||
return v in ("1", "true", "yes", "on", "是")
|
return v in ("1", "true", "yes", "on", "是")
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------- 凭证(加密存储) ----------------
|
||||||
|
def get_secret(conn, key, uid=0):
|
||||||
|
"""取凭证明文。仅在真正要用它对外发请求时调用。"""
|
||||||
|
row = conn.execute("SELECT value FROM settings WHERE user_id=? AND key=?",
|
||||||
|
(uid or 0, key)).fetchone()
|
||||||
|
if row is None or not row["value"]:
|
||||||
|
return ""
|
||||||
|
try:
|
||||||
|
return crypto.decrypt(row["value"], config.encryption_key())
|
||||||
|
except crypto.DecryptError as e:
|
||||||
|
raise SecretUnreadable("%s 无法解密:%s" % (key, e))
|
||||||
|
|
||||||
|
|
||||||
|
def set_secret(conn, key, value, uid=0):
|
||||||
|
set_setting(conn, key, value, uid)
|
||||||
|
|
||||||
|
|
||||||
|
def secret_state(conn, key, uid=0):
|
||||||
|
"""给界面用的凭证状态:只回「有没有 / 多少字符 / 尾部 4 位」,绝不含明文。"""
|
||||||
|
row = conn.execute("SELECT value FROM settings WHERE user_id=? AND key=?",
|
||||||
|
(uid or 0, key)).fetchone()
|
||||||
|
if row is None or not row["value"]:
|
||||||
|
return {"set": False, "chars": 0, "tail": "", "broken": False, "at": ""}
|
||||||
|
try:
|
||||||
|
plain = crypto.decrypt(row["value"], config.encryption_key())
|
||||||
|
except crypto.DecryptError:
|
||||||
|
return {"set": True, "chars": 0, "tail": "", "broken": True, "at": ""}
|
||||||
|
at = conn.execute("SELECT updated_at FROM settings WHERE user_id=? AND key=?",
|
||||||
|
(uid or 0, key)).fetchone()
|
||||||
|
return {"set": bool(plain), "chars": len(plain),
|
||||||
|
"tail": plain[-4:] if len(plain) >= 4 else "",
|
||||||
|
"broken": False, "at": (at["updated_at"] if at else "") or ""}
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------- 账号 ----------------
|
||||||
|
def user_by_id(conn, uid):
|
||||||
|
return conn.execute("SELECT * FROM users WHERE id=?", (uid,)).fetchone()
|
||||||
|
|
||||||
|
|
||||||
|
def user_by_name(conn, username):
|
||||||
|
return conn.execute("SELECT * FROM users WHERE username=?", (username,)).fetchone()
|
||||||
|
|
||||||
|
|
||||||
|
def active_users(conn):
|
||||||
|
"""启用状态的账号(调度器按人遍历)。"""
|
||||||
|
return conn.execute("SELECT * FROM users WHERE status='active' ORDER BY id").fetchall()
|
||||||
|
|
||||||
|
|
||||||
|
def user_count(conn):
|
||||||
|
return conn.execute("SELECT COUNT(*) FROM users").fetchone()[0]
|
||||||
|
|
||||||
|
|
||||||
# ---------------- 审计 ----------------
|
# ---------------- 审计 ----------------
|
||||||
def audit(conn, action, actor=None, detail=None, ip=None):
|
def audit(conn, action, actor=None, detail=None, ip=None, uid=0):
|
||||||
conn.execute("INSERT INTO audit_log(at,actor,action,detail,ip) VALUES(?,?,?,?,?)",
|
conn.execute("INSERT INTO audit_log(user_id,at,actor,action,detail,ip)"
|
||||||
(now_str(), actor, action, detail, ip))
|
" VALUES(?,?,?,?,?,?)", (uid or 0, now_str(), actor, action, detail, ip))
|
||||||
|
|
||||||
|
|
||||||
# ---------------- Flask 集成 ----------------
|
# ---------------- Flask 集成 ----------------
|
||||||
|
|||||||
@@ -6,12 +6,21 @@
|
|||||||
|
|
||||||
返回结构刻意与旧版 dashboard/data/*.json 的字段保持一致(d/c/k/fc/bc/m/h、
|
返回结构刻意与旧版 dashboard/data/*.json 的字段保持一致(d/c/k/fc/bc/m/h、
|
||||||
id/c/m/cl/t/px …),这样 ECharts 大屏的渲染代码一行都不用改,只换数据来源。
|
id/c/m/cl/t/px …),这样 ECharts 大屏的渲染代码一行都不用改,只换数据来源。
|
||||||
|
|
||||||
|
**多用户约定(最重要)**
|
||||||
|
所有公开函数都要求显式传入 `uid`(归属账号),且 `uid` 是 `conn` 之后的
|
||||||
|
第一个位置参数、**没有默认值**。这是有意设计的:
|
||||||
|
|
||||||
|
忘记传 uid 会直接 TypeError,而不是静默把「全部人的数据」算进去。
|
||||||
|
|
||||||
|
聚合层内部一律通过 `_where(..., uid)` 把 `user_id = ?` 拼进 WHERE,
|
||||||
|
所以任何一条 SQL 都不可能跨账号取数。
|
||||||
"""
|
"""
|
||||||
from datetime import datetime, timedelta
|
from datetime import datetime, timedelta
|
||||||
|
|
||||||
from . import config, db
|
from . import config, db
|
||||||
|
|
||||||
SCHEMA_VERSION = 4
|
SCHEMA_VERSION = 5 # /api/manifest 里对外的结构版本(多用户改版)
|
||||||
TOP_EXCERPT_LEN = 400
|
TOP_EXCERPT_LEN = 400
|
||||||
DEFAULT_TOP_N = 200
|
DEFAULT_TOP_N = 200
|
||||||
# 大屏页一次最多下发多少条窗口明细(页面要拿它在浏览器里算窗口 TOP / 散点)。
|
# 大屏页一次最多下发多少条窗口明细(页面要拿它在浏览器里算窗口 TOP / 散点)。
|
||||||
@@ -46,8 +55,9 @@ def norm_window(frm=None, to=None):
|
|||||||
return f, t
|
return f, t
|
||||||
|
|
||||||
|
|
||||||
def _where(frm=None, to=None, model=None, client=None, q=None):
|
def _where(frm=None, to=None, model=None, client=None, q=None, uid=0):
|
||||||
w, p = [], []
|
"""拼 WHERE。**user_id 永远在第一个条件上**,任何调用方都绕不过去。"""
|
||||||
|
w, p = ["user_id = ?"], [uid or 0]
|
||||||
if frm:
|
if frm:
|
||||||
w.append("day >= ?")
|
w.append("day >= ?")
|
||||||
p.append(frm)
|
p.append(frm)
|
||||||
@@ -63,7 +73,7 @@ def _where(frm=None, to=None, model=None, client=None, q=None):
|
|||||||
if q:
|
if q:
|
||||||
w.append("(prompt LIKE ? OR request_id LIKE ?)")
|
w.append("(prompt LIKE ? OR request_id LIKE ?)")
|
||||||
p += ["%" + q + "%", "%" + q + "%"]
|
p += ["%" + q + "%", "%" + q + "%"]
|
||||||
return ("WHERE " + " AND ".join(w)) if w else "", p
|
return "WHERE " + " AND ".join(w), p
|
||||||
|
|
||||||
|
|
||||||
def _excerpt(s, n):
|
def _excerpt(s, n):
|
||||||
@@ -74,8 +84,8 @@ def _excerpt(s, n):
|
|||||||
|
|
||||||
|
|
||||||
# ---------------- 逐日聚合 ----------------
|
# ---------------- 逐日聚合 ----------------
|
||||||
def daily(conn, frm=None, to=None, with_maps=True):
|
def daily(conn, uid, frm=None, to=None, with_maps=True):
|
||||||
w, p = _where(frm, to)
|
w, p = _where(frm, to, uid=uid)
|
||||||
days = {}
|
days = {}
|
||||||
for r in conn.execute(
|
for r in conn.execute(
|
||||||
"SELECT day d, COUNT(*) k, ROUND(SUM(credits),2) c,"
|
"SELECT day d, COUNT(*) k, ROUND(SUM(credits),2) c,"
|
||||||
@@ -101,8 +111,8 @@ def daily(conn, frm=None, to=None, with_maps=True):
|
|||||||
|
|
||||||
|
|
||||||
# ---------------- 维度汇总 ----------------
|
# ---------------- 维度汇总 ----------------
|
||||||
def _dim(conn, col, frm=None, to=None):
|
def _dim(conn, uid, col, frm=None, to=None):
|
||||||
w, p = _where(frm, to)
|
w, p = _where(frm, to, uid=uid)
|
||||||
rows = conn.execute(
|
rows = conn.execute(
|
||||||
"SELECT %s name, COUNT(*) calls, ROUND(SUM(credits),2) credits,"
|
"SELECT %s name, COUNT(*) calls, ROUND(SUM(credits),2) credits,"
|
||||||
" SUM(CASE WHEN credits<=0 THEN 1 ELSE 0 END) freeCalls,"
|
" SUM(CASE WHEN credits<=0 THEN 1 ELSE 0 END) freeCalls,"
|
||||||
@@ -125,8 +135,8 @@ def _dim(conn, col, frm=None, to=None):
|
|||||||
return out
|
return out
|
||||||
|
|
||||||
|
|
||||||
def dims(conn, frm=None, to=None):
|
def dims(conn, uid, frm=None, to=None):
|
||||||
hours = {int(r["name"]): r for r in _dim(conn, "printf('%02d',hour)", frm, to)}
|
hours = {int(r["name"]): r for r in _dim(conn, uid, "printf('%02d',hour)", frm, to)}
|
||||||
hlist = []
|
hlist = []
|
||||||
for i in range(24):
|
for i in range(24):
|
||||||
h = "%02d" % i
|
h = "%02d" % i
|
||||||
@@ -135,14 +145,14 @@ def dims(conn, frm=None, to=None):
|
|||||||
"lastDay": "", "avgPerCall": 0.0, "freeRate": 0.0}))
|
"lastDay": "", "avgPerCall": 0.0, "freeRate": 0.0}))
|
||||||
for i, o in enumerate(hlist):
|
for i, o in enumerate(hlist):
|
||||||
o["name"] = "%02d" % i
|
o["name"] = "%02d" % i
|
||||||
return {"model": _dim(conn, "model", frm, to),
|
return {"model": _dim(conn, uid, "model", frm, to),
|
||||||
"client": _dim(conn, "client", frm, to),
|
"client": _dim(conn, uid, "client", frm, to),
|
||||||
"hour": hlist}
|
"hour": hlist}
|
||||||
|
|
||||||
|
|
||||||
# ---------------- 单笔榜 ----------------
|
# ---------------- 单笔榜 ----------------
|
||||||
def top(conn, frm=None, to=None, n=DEFAULT_TOP_N):
|
def top(conn, uid, frm=None, to=None, n=DEFAULT_TOP_N):
|
||||||
w, p = _where(frm, to)
|
w, p = _where(frm, to, uid=uid)
|
||||||
items = []
|
items = []
|
||||||
for i, r in enumerate(conn.execute(
|
for i, r in enumerate(conn.execute(
|
||||||
"SELECT request_id, credits, model, client, ts, prompt FROM usage_records %s"
|
"SELECT request_id, credits, model, client, ts, prompt FROM usage_records %s"
|
||||||
@@ -154,8 +164,8 @@ def top(conn, frm=None, to=None, n=DEFAULT_TOP_N):
|
|||||||
|
|
||||||
|
|
||||||
# ---------------- 明细(窗口内精简记录,不带 prompt 全文)----------------
|
# ---------------- 明细(窗口内精简记录,不带 prompt 全文)----------------
|
||||||
def records(conn, frm=None, to=None, excerpt=96, limit=0, offset=0, newest_first=False):
|
def records(conn, uid, frm=None, to=None, excerpt=96, limit=0, offset=0, newest_first=False):
|
||||||
w, p = _where(frm, to)
|
w, p = _where(frm, to, uid=uid)
|
||||||
order = "ORDER BY ts DESC, request_id DESC" if newest_first else "ORDER BY ts, request_id"
|
order = "ORDER BY ts DESC, request_id DESC" if newest_first else "ORDER BY ts, request_id"
|
||||||
sql = ("SELECT request_id, credits, model, client, ts,"
|
sql = ("SELECT request_id, credits, model, client, ts,"
|
||||||
" substr(replace(replace(COALESCE(prompt,''),char(10),' '),char(13),' '),1,?) px"
|
" substr(replace(replace(COALESCE(prompt,''),char(10),' '),char(13),' '),1,?) px"
|
||||||
@@ -168,14 +178,15 @@ def records(conn, frm=None, to=None, excerpt=96, limit=0, offset=0, newest_first
|
|||||||
"cl": r["client"], "t": r["ts"], "px": (r["px"] or "")} for r in conn.execute(sql, args)]
|
"cl": r["client"], "t": r["ts"], "px": (r["px"] or "")} for r in conn.execute(sql, args)]
|
||||||
|
|
||||||
|
|
||||||
def records_page(conn, frm=None, to=None, model=None, client=None, q=None,
|
def records_page(conn, uid, frm=None, to=None, model=None, client=None, q=None,
|
||||||
page=1, size=50, order="ts_desc", with_prompt=True):
|
page=1, size=50, order="ts_desc", with_prompt=True):
|
||||||
frm, to = norm_window(frm, to)
|
frm, to = norm_window(frm, to)
|
||||||
size = max(1, min(int(size or 50), MAX_PAGE_SIZE))
|
size = max(1, min(int(size or 50), MAX_PAGE_SIZE))
|
||||||
page = max(1, int(page or 1))
|
page = max(1, int(page or 1))
|
||||||
w, p = _where(frm, to, model, client, q)
|
w, p = _where(frm, to, model, client, q, uid=uid)
|
||||||
total = conn.execute("SELECT COUNT(*) FROM usage_records %s" % w, p).fetchone()[0]
|
total = conn.execute("SELECT COUNT(*) FROM usage_records %s" % w, p).fetchone()[0]
|
||||||
agg = conn.execute("SELECT ROUND(COALESCE(SUM(credits),0),2) c FROM usage_records %s" % w, p).fetchone()
|
agg = conn.execute("SELECT ROUND(COALESCE(SUM(credits),0),2) c FROM usage_records %s"
|
||||||
|
% w, p).fetchone()
|
||||||
orders = {"ts_desc": "ts DESC, request_id", "ts": "ts, request_id",
|
orders = {"ts_desc": "ts DESC, request_id", "ts": "ts, request_id",
|
||||||
"credits_desc": "credits DESC, ts DESC", "credits": "credits, ts"}
|
"credits_desc": "credits DESC, ts DESC", "credits": "credits, ts"}
|
||||||
ob = orders.get(order, orders["ts_desc"])
|
ob = orders.get(order, orders["ts_desc"])
|
||||||
@@ -195,11 +206,11 @@ def records_page(conn, frm=None, to=None, model=None, client=None, q=None,
|
|||||||
"pages": max(1, (total + size - 1) // size), "items": items}
|
"pages": max(1, (total + size - 1) // size), "items": items}
|
||||||
|
|
||||||
|
|
||||||
def iter_records(conn, frm=None, to=None, model=None, client=None, q=None,
|
def iter_records(conn, uid, frm=None, to=None, model=None, client=None, q=None,
|
||||||
order="ts_desc", with_prompt=True, batch=1000):
|
order="ts_desc", with_prompt=True, batch=1000):
|
||||||
"""流式产出明细(给导出用):不把整个结果集读进内存。"""
|
"""流式产出明细(给导出用):不把整个结果集读进内存。"""
|
||||||
frm, to = norm_window(frm, to)
|
frm, to = norm_window(frm, to)
|
||||||
w, p = _where(frm, to, model, client, q)
|
w, p = _where(frm, to, model, client, q, uid=uid)
|
||||||
orders = {"ts_desc": "ts DESC, request_id", "ts": "ts, request_id",
|
orders = {"ts_desc": "ts DESC, request_id", "ts": "ts, request_id",
|
||||||
"credits_desc": "credits DESC, ts DESC", "credits": "credits, ts"}
|
"credits_desc": "credits DESC, ts DESC", "credits": "credits, ts"}
|
||||||
ob = orders.get(order, orders["ts_desc"])
|
ob = orders.get(order, orders["ts_desc"])
|
||||||
@@ -219,13 +230,14 @@ def iter_records(conn, frm=None, to=None, model=None, client=None, q=None,
|
|||||||
|
|
||||||
|
|
||||||
# ---------------- 全局元信息 ----------------
|
# ---------------- 全局元信息 ----------------
|
||||||
def months(conn):
|
def months(conn, uid=0):
|
||||||
return [r[0] for r in conn.execute(
|
return [r[0] for r in conn.execute(
|
||||||
"SELECT DISTINCT substr(day,1,7) m FROM usage_records ORDER BY m")]
|
"SELECT DISTINCT substr(day,1,7) m FROM usage_records WHERE user_id=? ORDER BY m",
|
||||||
|
(uid or 0,))]
|
||||||
|
|
||||||
|
|
||||||
def totals(conn, frm=None, to=None):
|
def totals(conn, uid, frm=None, to=None):
|
||||||
w, p = _where(frm, to)
|
w, p = _where(frm, to, uid=uid)
|
||||||
r = conn.execute(
|
r = conn.execute(
|
||||||
"SELECT COUNT(*) n, ROUND(COALESCE(SUM(credits),0),2) c,"
|
"SELECT COUNT(*) n, ROUND(COALESCE(SUM(credits),0),2) c,"
|
||||||
" SUM(CASE WHEN credits<=0 THEN 1 ELSE 0 END) fc,"
|
" SUM(CASE WHEN credits<=0 THEN 1 ELSE 0 END) fc,"
|
||||||
@@ -242,51 +254,60 @@ def totals(conn, frm=None, to=None):
|
|||||||
"first": r["t0"] or "", "last": r["t1"] or ""}
|
"first": r["t0"] or "", "last": r["t1"] or ""}
|
||||||
|
|
||||||
|
|
||||||
def day_list(conn):
|
def day_list(conn, uid=0):
|
||||||
return [r[0] for r in conn.execute("SELECT DISTINCT day FROM usage_records ORDER BY day")]
|
return [r[0] for r in conn.execute(
|
||||||
|
"SELECT DISTINCT day FROM usage_records WHERE user_id=? ORDER BY day", (uid or 0,))]
|
||||||
|
|
||||||
|
|
||||||
def manifest(conn):
|
def manifest(conn, uid):
|
||||||
t = totals(conn)
|
"""该账号的数据清单 + 采集健康状态(凭证是否已配置 = 本人是否配了 Cookie)。"""
|
||||||
|
t = totals(conn, uid)
|
||||||
db_bytes = conn.execute("PRAGMA page_count").fetchone()[0] * \
|
db_bytes = conn.execute("PRAGMA page_count").fetchone()[0] * \
|
||||||
conn.execute("PRAGMA page_size").fetchone()[0]
|
conn.execute("PRAGMA page_size").fetchone()[0]
|
||||||
runs = conn.execute("SELECT COUNT(*) FROM collect_runs").fetchone()[0]
|
runs = conn.execute("SELECT COUNT(*) FROM collect_runs WHERE user_id=?", (uid,)).fetchone()[0]
|
||||||
last_run = conn.execute("SELECT * FROM collect_runs ORDER BY id DESC LIMIT 1").fetchone()
|
last_run = conn.execute("SELECT * FROM collect_runs WHERE user_id=?"
|
||||||
health = db.get_setting(conn, "cookie", "")
|
" ORDER BY id DESC LIMIT 1", (uid,)).fetchone()
|
||||||
mons = months(conn) # 只算一次(原来在返回体里调了两遍)
|
cred = db.secret_state(conn, "cookie", uid)
|
||||||
|
mons = months(conn, uid) # 只算一次(原来在返回体里调了两遍)
|
||||||
return {
|
return {
|
||||||
"schema": SCHEMA_VERSION,
|
"schema": SCHEMA_VERSION,
|
||||||
"generated": db.now_str(),
|
"generated": db.now_str(),
|
||||||
"archive": "data/usage.sqlite",
|
"archive": "data/usage.sqlite",
|
||||||
"producer": "workbuddy-portal(Flask + SQLite)",
|
"producer": "workbuddy-portal(Flask + SQLite)",
|
||||||
"note": "数据正本为 SQLite 表 usage_records;daily/dims/top 均为 SQL 实时聚合结果。",
|
"note": "数据正本为 SQLite 表 usage_records;daily/dims/top 均为 SQL 实时聚合结果,"
|
||||||
|
"且只统计当前登录账号的归属数据。",
|
||||||
"totals": {"records": t["records"], "credits": t["credits"], "calls": t["calls"],
|
"totals": {"records": t["records"], "credits": t["credits"], "calls": t["calls"],
|
||||||
"freeCalls": t["freeCalls"], "billableCalls": t["billableCalls"],
|
"freeCalls": t["freeCalls"], "billableCalls": t["billableCalls"],
|
||||||
"days": day_list(conn), "months": mons,
|
"days": day_list(conn, uid), "months": mons,
|
||||||
"models": t["models"], "clients": t["clients"],
|
"models": t["models"], "clients": t["clients"],
|
||||||
"first": t["first"], "last": t["last"],
|
"first": t["first"], "last": t["last"],
|
||||||
"topCredits": (conn.execute("SELECT COALESCE(MAX(credits),0) FROM usage_records")
|
"topCredits": (conn.execute(
|
||||||
.fetchone()[0] or 0.0)},
|
"SELECT COALESCE(MAX(credits),0) FROM usage_records WHERE user_id=?",
|
||||||
|
(uid,)).fetchone()[0] or 0.0)},
|
||||||
"months": mons,
|
"months": mons,
|
||||||
"sources": [
|
"sources": [
|
||||||
{"path": "usage_records", "role": "明细正本(SQLite 表)", "count": t["records"],
|
{"path": "usage_records", "role": "明细正本(SQLite 表,按账号隔离)",
|
||||||
"bytes": db_bytes},
|
"count": t["records"], "bytes": db_bytes},
|
||||||
{"path": "daily 聚合视图", "role": "逐日聚合(SQL GROUP BY day)", "count": t["days"], "bytes": 0},
|
{"path": "daily 聚合视图", "role": "逐日聚合(SQL GROUP BY day)",
|
||||||
|
"count": t["days"], "bytes": 0},
|
||||||
{"path": "dims 聚合视图", "role": "模型/客户端/时段汇总(SQL GROUP BY)",
|
{"path": "dims 聚合视图", "role": "模型/客户端/时段汇总(SQL GROUP BY)",
|
||||||
"count": t["models"] + t["clients"] + 24, "bytes": 0},
|
"count": t["models"] + t["clients"] + 24, "bytes": 0},
|
||||||
{"path": "top 查询", "role": "单笔消耗榜(ORDER BY credits DESC)", "count": DEFAULT_TOP_N, "bytes": 0},
|
{"path": "top 查询", "role": "单笔消耗榜(ORDER BY credits DESC)",
|
||||||
{"path": "collect_runs", "role": "采集运行历史", "count": runs, "bytes": 0},
|
"count": DEFAULT_TOP_N, "bytes": 0},
|
||||||
|
{"path": "collect_runs", "role": "采集运行历史(本账号)", "count": runs, "bytes": 0},
|
||||||
],
|
],
|
||||||
"focusDay": (last_run["win_to"] or "")[:10] if last_run else "",
|
"focusDay": (last_run["win_to"] or "")[:10] if last_run else "",
|
||||||
"health": {"cookie": bool(health and health.strip()),
|
"health": {"cookie": cred["set"] and not cred["broken"],
|
||||||
|
"cookieChars": cred["chars"],
|
||||||
|
"cookieBroken": cred["broken"],
|
||||||
"lastRunAt": last_run["started_at"] if last_run else "",
|
"lastRunAt": last_run["started_at"] if last_run else "",
|
||||||
"lastRunStatus": last_run["status"] if last_run else ""},
|
"lastRunStatus": last_run["status"] if last_run else ""},
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
def bundle(conn, frm=None, to=None, top_n=DEFAULT_TOP_N, excerpt=140,
|
def bundle(conn, uid, frm=None, to=None, top_n=DEFAULT_TOP_N, excerpt=140,
|
||||||
records_cap=BUNDLE_RECORDS_CAP):
|
records_cap=BUNDLE_RECORDS_CAP):
|
||||||
"""大屏页一次请求拿齐所需数据。
|
"""大屏页一次请求拿齐所需数据(全部限定在当前账号内)。
|
||||||
|
|
||||||
窗口裁剪:records(明细)、dims(维度)、totals(KPI)随 frm/to 变化。
|
窗口裁剪:records(明细)、dims(维度)、totals(KPI)随 frm/to 变化。
|
||||||
刻意不裁剪:daily(全量逐日,供日历与日期轴,体量小)、top(全局 TOP 榜)。
|
刻意不裁剪:daily(全量逐日,供日历与日期轴,体量小)、top(全局 TOP 榜)。
|
||||||
@@ -295,16 +316,16 @@ def bundle(conn, frm=None, to=None, top_n=DEFAULT_TOP_N, excerpt=140,
|
|||||||
避免存档长大后「全部」区间把整包明细都压到浏览器。
|
避免存档长大后「全部」区间把整包明细都压到浏览器。
|
||||||
"""
|
"""
|
||||||
frm, to = norm_window(frm, to)
|
frm, to = norm_window(frm, to)
|
||||||
tot = totals(conn, frm, to)
|
tot = totals(conn, uid, frm, to)
|
||||||
# 只有真的会超限时才改成「取最近 N 条」,避免改变现有正常路径的行为
|
# 只有真的会超限时才改成「取最近 N 条」,避免改变现有正常路径的行为
|
||||||
truncated = tot["records"] > records_cap
|
truncated = tot["records"] > records_cap
|
||||||
recs = records(conn, frm, to, excerpt=excerpt,
|
recs = records(conn, uid, frm, to, excerpt=excerpt,
|
||||||
limit=records_cap if truncated else 0, newest_first=truncated)
|
limit=records_cap if truncated else 0, newest_first=truncated)
|
||||||
return {
|
return {
|
||||||
"manifest": manifest(conn),
|
"manifest": manifest(conn, uid),
|
||||||
"daily": daily(conn), # 全量逐日(体量小,供日历与日期轴)
|
"daily": daily(conn, uid), # 全量逐日(体量小,供日历与日期轴)
|
||||||
"dims": dims(conn, frm, to), # 窗口内维度
|
"dims": dims(conn, uid, frm, to), # 窗口内维度
|
||||||
"top": top(conn, None, None, top_n)["items"], # 全局 TOP 榜(对应「全局 TOP200」视图)
|
"top": top(conn, uid, None, None, top_n)["items"], # 该账号全局 TOP 榜
|
||||||
"records": recs,
|
"records": recs,
|
||||||
"recordsTotal": tot["records"],
|
"recordsTotal": tot["records"],
|
||||||
"recordsCap": records_cap,
|
"recordsCap": records_cap,
|
||||||
@@ -315,7 +336,7 @@ def bundle(conn, frm=None, to=None, top_n=DEFAULT_TOP_N, excerpt=140,
|
|||||||
|
|
||||||
|
|
||||||
# ---------------- 环比 ----------------
|
# ---------------- 环比 ----------------
|
||||||
def summary(conn, frm, to):
|
def summary(conn, uid, frm, to):
|
||||||
"""KPI + 环比。前一段必须完整落在存档范围内,否则不给假数字。
|
"""KPI + 环比。前一段必须完整落在存档范围内,否则不给假数字。
|
||||||
|
|
||||||
frm/to 会先归一化(容错 '2026-09-08 12:00:00'、'2026/09/08' 等写法),
|
frm/to 会先归一化(容错 '2026-09-08 12:00:00'、'2026/09/08' 等写法),
|
||||||
@@ -324,18 +345,19 @@ def summary(conn, frm, to):
|
|||||||
frm, to = norm_window(frm, to)
|
frm, to = norm_window(frm, to)
|
||||||
if not frm or not to:
|
if not frm or not to:
|
||||||
# 无法识别的日期:退化成全量口径,不抛异常(API 层会先校验并返回 400)
|
# 无法识别的日期:退化成全量口径,不抛异常(API 层会先校验并返回 400)
|
||||||
t = totals(conn)
|
t = totals(conn, uid)
|
||||||
frm, to = t["firstDay"], t["lastDay"]
|
frm, to = t["firstDay"], t["lastDay"]
|
||||||
if not frm or not to:
|
if not frm or not to:
|
||||||
frm = to = datetime.now().strftime("%Y-%m-%d")
|
frm = to = datetime.now().strftime("%Y-%m-%d")
|
||||||
cur = totals(conn, frm, to)
|
cur = totals(conn, uid, frm, to)
|
||||||
days = (datetime.strptime(to, "%Y-%m-%d") - datetime.strptime(frm, "%Y-%m-%d")).days + 1
|
days = (datetime.strptime(to, "%Y-%m-%d") - datetime.strptime(frm, "%Y-%m-%d")).days + 1
|
||||||
p_to = (datetime.strptime(frm, "%Y-%m-%d") - timedelta(days=1)).strftime("%Y-%m-%d")
|
p_to = (datetime.strptime(frm, "%Y-%m-%d") - timedelta(days=1)).strftime("%Y-%m-%d")
|
||||||
p_frm = (datetime.strptime(p_to, "%Y-%m-%d") - timedelta(days=days - 1)).strftime("%Y-%m-%d")
|
p_frm = (datetime.strptime(p_to, "%Y-%m-%d") - timedelta(days=days - 1)).strftime("%Y-%m-%d")
|
||||||
first_day = conn.execute("SELECT MIN(day) FROM usage_records").fetchone()[0]
|
first_day = conn.execute("SELECT MIN(day) FROM usage_records WHERE user_id=?",
|
||||||
|
(uid or 0,)).fetchone()[0]
|
||||||
prev = None
|
prev = None
|
||||||
if first_day and p_frm >= first_day:
|
if first_day and p_frm >= first_day:
|
||||||
prev = totals(conn, p_frm, p_to)
|
prev = totals(conn, uid, p_frm, p_to)
|
||||||
out = dict(cur)
|
out = dict(cur)
|
||||||
out["window"] = {"from": frm, "to": to, "days": days}
|
out["window"] = {"from": frm, "to": to, "days": days}
|
||||||
out["avgPerCall"] = round(cur["credits"] / cur["calls"], 4) if cur["calls"] else 0.0
|
out["avgPerCall"] = round(cur["credits"] / cur["calls"], 4) if cur["calls"] else 0.0
|
||||||
@@ -350,7 +372,8 @@ def summary(conn, frm, to):
|
|||||||
out["delta"] = None
|
out["delta"] = None
|
||||||
# 残日:最后一天不是完整的一天
|
# 残日:最后一天不是完整的一天
|
||||||
if to == datetime.now().strftime("%Y-%m-%d"):
|
if to == datetime.now().strftime("%Y-%m-%d"):
|
||||||
row = conn.execute("SELECT MAX(ts) FROM usage_records WHERE day=?", (to,)).fetchone()
|
row = conn.execute("SELECT MAX(ts) FROM usage_records WHERE user_id=? AND day=?",
|
||||||
|
(uid or 0, to)).fetchone()
|
||||||
if row and row[0]:
|
if row and row[0]:
|
||||||
out["partial"] = {"date": to, "hhmm": row[0][11:16]}
|
out["partial"] = {"date": to, "hhmm": row[0][11:16]}
|
||||||
return out
|
return out
|
||||||
|
|||||||
@@ -9,6 +9,13 @@
|
|||||||
* 需要「启动补跑」(程序没开的时候错过了时刻,开机后要补上)
|
* 需要「启动补跑」(程序没开的时候错过了时刻,开机后要补上)
|
||||||
* 需要和 CLI 共享同一把文件锁,避免两处同时采集
|
* 需要和 CLI 共享同一把文件锁,避免两处同时采集
|
||||||
|
|
||||||
|
多用户
|
||||||
|
------
|
||||||
|
调度配置(开关 / 时刻 / 补跑 / slot:* 簿记)都是**个人级**设置,
|
||||||
|
所以 tick 会遍历所有启用状态的账号,各自判断有没有到期槽位。
|
||||||
|
好处是「A 想 9 点采、B 想 21 点采」互不影响;代价是串行执行 ——
|
||||||
|
这是刻意的,SQLite 单写者不允许并发采集。
|
||||||
|
|
||||||
单实例保证:
|
单实例保证:
|
||||||
* Flask 的 reloader 会 fork 两个进程 → 只在 WERKZEUG_RUN_MAIN 里启动
|
* Flask 的 reloader 会 fork 两个进程 → 只在 WERKZEUG_RUN_MAIN 里启动
|
||||||
* 多进程部署时用环境变量 WB_DISABLE_SCHEDULER=1 关掉除一个之外的所有实例
|
* 多进程部署时用环境变量 WB_DISABLE_SCHEDULER=1 关掉除一个之外的所有实例
|
||||||
@@ -22,7 +29,7 @@ from datetime import datetime, timedelta
|
|||||||
from . import collect, db
|
from . import collect, db
|
||||||
|
|
||||||
log = logging.getLogger("wb.scheduler")
|
log = logging.getLogger("wb.scheduler")
|
||||||
SLOT_PREFIX = "slot:" # settings 键:slot:09:00 -> 最近执行的日期
|
SLOT_PREFIX = "slot:" # settings 键:slot:09:00 -> 最近执行的日期(按 user_id 存)
|
||||||
|
|
||||||
|
|
||||||
def parse_times(raw):
|
def parse_times(raw):
|
||||||
@@ -50,25 +57,25 @@ def parse_times(raw):
|
|||||||
_parse_times = parse_times
|
_parse_times = parse_times
|
||||||
|
|
||||||
|
|
||||||
def slots(conn):
|
def slots(conn, uid=0):
|
||||||
return parse_times(db.get_setting(conn, "schedule_times"))
|
return parse_times(db.get_setting(conn, "schedule_times", uid=uid))
|
||||||
|
|
||||||
|
|
||||||
def last_run_of_slot(conn, slot):
|
def last_run_of_slot(conn, uid, slot):
|
||||||
return db.get_setting(conn, SLOT_PREFIX + slot, "")
|
return db.get_setting(conn, SLOT_PREFIX + slot, "", uid)
|
||||||
|
|
||||||
|
|
||||||
def mark_slot(conn, slot, day):
|
def mark_slot(conn, uid, slot, day):
|
||||||
db.set_setting(conn, SLOT_PREFIX + slot, day)
|
db.set_setting(conn, SLOT_PREFIX + slot, day, uid)
|
||||||
|
|
||||||
|
|
||||||
def next_run_at(conn, now=None):
|
def next_run_at(conn, uid=0, now=None):
|
||||||
"""下一次计划执行时间(仅按配置推算,不含补跑)。"""
|
"""下一次计划执行时间(仅按配置推算,不含补跑)。"""
|
||||||
if not db.get_bool(conn, "schedule_enabled", True):
|
if not db.get_bool(conn, "schedule_enabled", True, uid):
|
||||||
return None
|
return None
|
||||||
now = now or datetime.now()
|
now = now or datetime.now()
|
||||||
best = None
|
best = None
|
||||||
for s in slots(conn):
|
for s in slots(conn, uid):
|
||||||
hh, mm = map(int, s.split(":"))
|
hh, mm = map(int, s.split(":"))
|
||||||
cand = now.replace(hour=hh, minute=mm, second=0, microsecond=0)
|
cand = now.replace(hour=hh, minute=mm, second=0, microsecond=0)
|
||||||
if cand <= now:
|
if cand <= now:
|
||||||
@@ -78,24 +85,24 @@ def next_run_at(conn, now=None):
|
|||||||
return best
|
return best
|
||||||
|
|
||||||
|
|
||||||
def due_slots(conn, now=None):
|
def due_slots(conn, uid=0, now=None):
|
||||||
"""返回此刻应当执行的槽位列表(含启动补跑)。"""
|
"""返回此刻应当执行的槽位列表(含启动补跑)。"""
|
||||||
if not db.get_bool(conn, "schedule_enabled", True):
|
if not db.get_bool(conn, "schedule_enabled", True, uid):
|
||||||
return []
|
return []
|
||||||
now = now or datetime.now()
|
now = now or datetime.now()
|
||||||
today = now.strftime("%Y-%m-%d")
|
today = now.strftime("%Y-%m-%d")
|
||||||
# 用 get_int 兜底:catch_up_grace_hours 在后台是自由文本框,
|
# 用 get_int 兜底:catch_up_grace_hours 在后台是自由文本框,
|
||||||
# 历史上填成 "12h" 会让这里 int() 抛 ValueError,把 /tasks 打成 500。
|
# 历史上填成 "12h" 会让这里 int() 抛 ValueError,把 /tasks 打成 500。
|
||||||
grace_hours = db.get_int(conn, "catch_up_grace_hours", 12)
|
grace_hours = db.get_int(conn, "catch_up_grace_hours", 12, uid)
|
||||||
grace = timedelta(hours=max(1, grace_hours))
|
grace = timedelta(hours=max(1, grace_hours))
|
||||||
catch_up = db.get_bool(conn, "catch_up", True)
|
catch_up = db.get_bool(conn, "catch_up", True, uid)
|
||||||
out = []
|
out = []
|
||||||
for s in slots(conn):
|
for s in slots(conn, uid):
|
||||||
hh, mm = map(int, s.split(":"))
|
hh, mm = map(int, s.split(":"))
|
||||||
when = now.replace(hour=hh, minute=mm, second=0, microsecond=0)
|
when = now.replace(hour=hh, minute=mm, second=0, microsecond=0)
|
||||||
if when > now:
|
if when > now:
|
||||||
continue # 还没到点
|
continue # 还没到点
|
||||||
if last_run_of_slot(conn, s) == today:
|
if last_run_of_slot(conn, uid, s) == today:
|
||||||
continue # 今天这个槽位已跑过
|
continue # 今天这个槽位已跑过
|
||||||
if when < now - grace and catch_up:
|
if when < now - grace and catch_up:
|
||||||
continue # 错过太久,不补(避免开机狂刷)
|
continue # 错过太久,不补(避免开机狂刷)
|
||||||
@@ -143,19 +150,32 @@ class Scheduler:
|
|||||||
conn = db.thread_conn()
|
conn = db.thread_conn()
|
||||||
now = now or datetime.now()
|
now = now or datetime.now()
|
||||||
today = now.strftime("%Y-%m-%d")
|
today = now.strftime("%Y-%m-%d")
|
||||||
for slot in due_slots(conn, now):
|
for u in db.active_users(conn):
|
||||||
scheduled = now.replace(hour=int(slot[:2]), minute=int(slot[3:]),
|
uid = u["id"]
|
||||||
second=0, microsecond=0)
|
|
||||||
trigger = "startup" if now - scheduled > timedelta(minutes=5) else "schedule"
|
|
||||||
log.info("触发采集:槽位 %s(%s)", slot, trigger)
|
|
||||||
mark_slot(conn, slot, today) # 先占位,避免采集失败被无限重试打爆云端
|
|
||||||
try:
|
try:
|
||||||
r = collect.run_sync(trigger=trigger)
|
pending = due_slots(conn, uid, now)
|
||||||
log.info("采集完成:%s", r["message"])
|
except Exception as e: # 单个账号配置坏了不能拖垮其他人
|
||||||
except collect.Busy as e:
|
log.error("账号 #%s(%s) 读取调度配置失败:%s", uid, u["username"], e)
|
||||||
log.warning("跳过(%s)", e)
|
continue
|
||||||
except Exception as e:
|
for slot in pending:
|
||||||
log.error("采集失败:%s", e)
|
scheduled = now.replace(hour=int(slot[:2]), minute=int(slot[3:]),
|
||||||
|
second=0, microsecond=0)
|
||||||
|
trigger = "startup" if now - scheduled > timedelta(minutes=5) else "schedule"
|
||||||
|
log.info("触发采集:账号 %s 槽位 %s(%s)", u["username"], slot, trigger)
|
||||||
|
# 先占位,避免采集失败被无限重试打爆云端
|
||||||
|
mark_slot(conn, uid, slot, today)
|
||||||
|
if not db.secret_state(conn, "cookie", uid)["set"]:
|
||||||
|
log.info("跳过:账号 %s 还没配置自己的 Cookie", u["username"])
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
r = collect.run_sync(trigger=trigger, uid=uid)
|
||||||
|
log.info("采集完成:%s → %s", u["username"], r["message"])
|
||||||
|
except collect.Busy as e:
|
||||||
|
log.warning("跳过(%s)", e)
|
||||||
|
except db.SecretUnreadable as e:
|
||||||
|
log.error("账号 %s 的 Cookie 解不开:%s", u["username"], e)
|
||||||
|
except Exception as e:
|
||||||
|
log.error("账号 %s 采集失败:%s", u["username"], e)
|
||||||
return True
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -1,15 +1,24 @@
|
|||||||
-- WorkBuddy Portal —— SQLite 表结构
|
-- WorkBuddy Portal —— SQLite 表结构
|
||||||
-- 设计要点:
|
-- 设计要点:
|
||||||
-- * usage_records 是唯一正本,request_id 为主键,去重靠 ON CONFLICT,不再依赖内存比对
|
-- * 多用户:usage_records / collect_runs / audit_log 都带 user_id;
|
||||||
|
-- 每个账号只看得到自己的数据,管理员也不越过这条线(见 docs/ARCHITECTURE.md)
|
||||||
|
-- * settings 是 (user_id, key) 复合主键:user_id=0 为实例级,其余为个人级
|
||||||
|
-- * usage_records 主键是 (user_id, request_id):去重按「人 + 请求」,
|
||||||
|
-- 不同账号拿到相同 requestId 时互不覆盖
|
||||||
-- * ts 存「本地保留的最早开始时间」;cloud_ts 存云端最近一次返回的时间(观察长请求前移)
|
-- * ts 存「本地保留的最早开始时间」;cloud_ts 存云端最近一次返回的时间(观察长请求前移)
|
||||||
-- * day / hour 是冗余列,配合索引让区间扫描与 GROUP BY 都能走索引
|
-- * day / hour 是冗余列,配合索引让区间扫描与 GROUP BY 都能走索引
|
||||||
-- * prompt 单独存一列且默认不参与任何列表接口(占传输量约 80%)
|
-- * prompt 单独存一列且默认不参与任何列表接口(占传输量约 80%)
|
||||||
|
--
|
||||||
|
-- 升级:本文件是 DDL 的唯一来源。db._migrate() 用「改名旧表 -> 重跑本文件 ->
|
||||||
|
-- 回填数据 -> 删旧表 -> 再跑一次本文件补索引」的方式做在线迁移。
|
||||||
|
|
||||||
PRAGMA journal_mode = WAL;
|
PRAGMA journal_mode = WAL;
|
||||||
PRAGMA synchronous = NORMAL;
|
PRAGMA synchronous = NORMAL;
|
||||||
|
|
||||||
|
-- ---------------- 采集正本 ----------------
|
||||||
CREATE TABLE IF NOT EXISTS usage_records (
|
CREATE TABLE IF NOT EXISTS usage_records (
|
||||||
request_id TEXT PRIMARY KEY,
|
user_id INTEGER NOT NULL DEFAULT 0, -- 归属账号(usage_records.user_id -> users.id)
|
||||||
|
request_id TEXT NOT NULL,
|
||||||
ts TEXT NOT NULL, -- 'YYYY-MM-DD HH:MM:SS'
|
ts TEXT NOT NULL, -- 'YYYY-MM-DD HH:MM:SS'
|
||||||
day TEXT NOT NULL, -- 'YYYY-MM-DD'
|
day TEXT NOT NULL, -- 'YYYY-MM-DD'
|
||||||
hour INTEGER NOT NULL, -- 0..23
|
hour INTEGER NOT NULL, -- 0..23
|
||||||
@@ -19,19 +28,23 @@ CREATE TABLE IF NOT EXISTS usage_records (
|
|||||||
prompt TEXT,
|
prompt TEXT,
|
||||||
first_seen TEXT NOT NULL, -- 本地首次入库时间
|
first_seen TEXT NOT NULL, -- 本地首次入库时间
|
||||||
last_seen TEXT NOT NULL, -- 本地最近一次见到的时间
|
last_seen TEXT NOT NULL, -- 本地最近一次见到的时间
|
||||||
cloud_ts TEXT -- 云端最近一次返回的 requestTime
|
cloud_ts TEXT, -- 云端最近一次返回的 requestTime
|
||||||
|
PRIMARY KEY (user_id, request_id)
|
||||||
);
|
);
|
||||||
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_ur_day ON usage_records(day);
|
-- 索引一律以 user_id 打头:所有查询都带「归属人」这个条件,
|
||||||
CREATE INDEX IF NOT EXISTS idx_ur_day_hour ON usage_records(day, hour);
|
-- 少了它会退化成全表扫描(多用户下这是最容易踩的性能坑)。
|
||||||
CREATE INDEX IF NOT EXISTS idx_ur_model_day ON usage_records(model, day);
|
CREATE INDEX IF NOT EXISTS idx_ur_day ON usage_records(user_id, day);
|
||||||
CREATE INDEX IF NOT EXISTS idx_ur_client_day ON usage_records(client, day);
|
CREATE INDEX IF NOT EXISTS idx_ur_day_hour ON usage_records(user_id, day, hour);
|
||||||
CREATE INDEX IF NOT EXISTS idx_ur_credits ON usage_records(credits DESC);
|
CREATE INDEX IF NOT EXISTS idx_ur_model_day ON usage_records(user_id, model, day);
|
||||||
CREATE INDEX IF NOT EXISTS idx_ur_ts ON usage_records(ts);
|
CREATE INDEX IF NOT EXISTS idx_ur_client_day ON usage_records(user_id, client, day);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_ur_credits ON usage_records(user_id, credits DESC);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_ur_ts ON usage_records(user_id, ts);
|
||||||
|
|
||||||
-- 采集运行历史(任务管理 + 日志管理的正本)
|
-- ---------------- 采集运行历史 ----------------
|
||||||
CREATE TABLE IF NOT EXISTS collect_runs (
|
CREATE TABLE IF NOT EXISTS collect_runs (
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
user_id INTEGER NOT NULL DEFAULT 0, -- 哪次「谁」的采集
|
||||||
trigger TEXT NOT NULL, -- manual | schedule | cli | startup
|
trigger TEXT NOT NULL, -- manual | schedule | cli | startup
|
||||||
status TEXT NOT NULL, -- running | ok | warn | error
|
status TEXT NOT NULL, -- running | ok | warn | error
|
||||||
started_at TEXT NOT NULL,
|
started_at TEXT NOT NULL,
|
||||||
@@ -42,7 +55,7 @@ CREATE TABLE IF NOT EXISTS collect_runs (
|
|||||||
fetched INTEGER DEFAULT 0, -- 云端返回条数
|
fetched INTEGER DEFAULT 0, -- 云端返回条数
|
||||||
added INTEGER DEFAULT 0,
|
added INTEGER DEFAULT 0,
|
||||||
dup INTEGER DEFAULT 0,
|
dup INTEGER DEFAULT 0,
|
||||||
total INTEGER DEFAULT 0, -- 入库后总条数
|
total INTEGER DEFAULT 0, -- 入库后该账号总条数
|
||||||
conflicts INTEGER DEFAULT 0,
|
conflicts INTEGER DEFAULT 0,
|
||||||
exit_code INTEGER,
|
exit_code INTEGER,
|
||||||
message TEXT, -- 一句话结论
|
message TEXT, -- 一句话结论
|
||||||
@@ -50,34 +63,59 @@ CREATE TABLE IF NOT EXISTS collect_runs (
|
|||||||
);
|
);
|
||||||
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_runs_started ON collect_runs(started_at DESC);
|
CREATE INDEX IF NOT EXISTS idx_runs_started ON collect_runs(started_at DESC);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_runs_user ON collect_runs(user_id, id DESC);
|
||||||
|
|
||||||
-- 键值配置:cookie / user_agent / 调度时刻 / 采集参数 / 调度槽位去重标记
|
-- ---------------- 键值配置 ----------------
|
||||||
|
-- user_id = 0 : 实例级(接口基址、注册开关、验证码策略)
|
||||||
|
-- user_id > 0 : 个人级(自己填的 Cookie / UA、采集参数、调度时刻、slot:* 簿记)
|
||||||
CREATE TABLE IF NOT EXISTS settings (
|
CREATE TABLE IF NOT EXISTS settings (
|
||||||
key TEXT PRIMARY KEY,
|
user_id INTEGER NOT NULL DEFAULT 0,
|
||||||
|
key TEXT NOT NULL,
|
||||||
value TEXT,
|
value TEXT,
|
||||||
updated_at TEXT
|
updated_at TEXT,
|
||||||
|
PRIMARY KEY (user_id, key)
|
||||||
);
|
);
|
||||||
|
|
||||||
-- 后台登录账号(局域网访问必须)
|
-- ---------------- 账号 ----------------
|
||||||
CREATE TABLE IF NOT EXISTS users (
|
CREATE TABLE IF NOT EXISTS users (
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
username TEXT UNIQUE NOT NULL,
|
username TEXT UNIQUE NOT NULL,
|
||||||
password_hash TEXT NOT NULL,
|
password_hash TEXT NOT NULL,
|
||||||
display_name TEXT,
|
display_name TEXT,
|
||||||
is_admin INTEGER NOT NULL DEFAULT 1,
|
email TEXT,
|
||||||
|
is_admin INTEGER NOT NULL DEFAULT 0,
|
||||||
|
status TEXT NOT NULL DEFAULT 'active', -- active | disabled
|
||||||
created_at TEXT,
|
created_at TEXT,
|
||||||
|
register_ip TEXT, -- 自助注册来源,用于每日限额
|
||||||
last_login_at TEXT,
|
last_login_at TEXT,
|
||||||
|
last_login_ip TEXT,
|
||||||
login_count INTEGER NOT NULL DEFAULT 0
|
login_count INTEGER NOT NULL DEFAULT 0
|
||||||
);
|
);
|
||||||
|
|
||||||
-- 操作审计(登录、改配置、手动触发等)
|
-- ---------------- 操作审计 ----------------
|
||||||
|
-- 刻意**不记录任何凭证**:detail 里只写「改了哪些键」,不写键的值。
|
||||||
CREATE TABLE IF NOT EXISTS audit_log (
|
CREATE TABLE IF NOT EXISTS audit_log (
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
at TEXT NOT NULL,
|
user_id INTEGER NOT NULL DEFAULT 0, -- 归属账号(0 = 系统 / CLI / 未登录事件)
|
||||||
actor TEXT,
|
at TEXT NOT NULL,
|
||||||
action TEXT NOT NULL,
|
actor TEXT,
|
||||||
detail TEXT,
|
action TEXT NOT NULL,
|
||||||
ip TEXT
|
detail TEXT,
|
||||||
|
ip TEXT
|
||||||
);
|
);
|
||||||
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_audit_at ON audit_log(at DESC);
|
CREATE INDEX IF NOT EXISTS idx_audit_at ON audit_log(at DESC);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_audit_user ON audit_log(user_id, id DESC);
|
||||||
|
|
||||||
|
-- ---------------- 图形验证码 ----------------
|
||||||
|
-- 答案只在服务端存在。下发到浏览器的只是 id,且用一次即删。
|
||||||
|
CREATE TABLE IF NOT EXISTS captchas (
|
||||||
|
id TEXT PRIMARY KEY, -- 随机 token(下发给客户端)
|
||||||
|
answer TEXT NOT NULL, -- 正确答案(绝不下发)
|
||||||
|
purpose TEXT NOT NULL, -- login | register
|
||||||
|
created_at TEXT NOT NULL,
|
||||||
|
expires_at TEXT NOT NULL,
|
||||||
|
used_at TEXT
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_captcha_expires ON captchas(expires_at);
|
||||||
|
|||||||
@@ -2,117 +2,211 @@
|
|||||||
# SPDX-License-Identifier: MIT
|
# SPDX-License-Identifier: MIT
|
||||||
# Copyright (c) 2026 Wang Chuanli
|
# Copyright (c) 2026 Wang Chuanli
|
||||||
|
|
||||||
"""密码哈希、登录装饰器、CSRF。
|
"""鉴权、口令策略、图形验证码、限速、CSRF、安全响应头。
|
||||||
|
|
||||||
局域网可访问 ⇒ 必须有鉴权。这里用 Werkzeug 自带的 PBKDF2,不引第三方依赖。
|
多用户下的三条红线
|
||||||
|
------------------
|
||||||
|
1. **Cookie(账号凭证)是按 user_id 分作用域存的**,读取一律走
|
||||||
|
`db.get_secret(conn, "cookie", uid)`;`db.get_settings()` 会把凭证置空,
|
||||||
|
所以「顺手把配置回传给前端」这类代码不可能把它带出去。
|
||||||
|
2. **禁用/删除账号立刻失效**:`current_user()` 每个请求回查一次
|
||||||
|
users.status,不靠会话过期来兜底(默认会话 12 小时,太久了)。
|
||||||
|
3. **失败限速按「来源 IP」和「用户名」双维度计数**:只按 IP 挡不住
|
||||||
|
「一批肉鸡轮流撞同一个账号」,只按用户名又会让一个 IP 无限注册。
|
||||||
"""
|
"""
|
||||||
import functools
|
import functools
|
||||||
import hmac
|
import hmac
|
||||||
|
import re
|
||||||
import secrets
|
import secrets
|
||||||
import time
|
import time
|
||||||
|
|
||||||
from flask import (current_app, flash, jsonify, redirect, render_template, request,
|
from flask import (current_app, flash, g, jsonify, redirect, render_template,
|
||||||
session, url_for)
|
request, session, url_for)
|
||||||
from werkzeug.security import check_password_hash, generate_password_hash
|
|
||||||
|
|
||||||
from . import config, db
|
from . import captcha, config, db
|
||||||
|
|
||||||
# 简易失败计数(内存即可:单进程部署,重启清零可接受)
|
# ---------------- 失败计数(内存即可) ----------------
|
||||||
_fails = {} # ip -> [count, first_ts]
|
# 单进程部署(见 README 的部署约束),重启清零可接受;
|
||||||
_FAILS_MAX_IPS = 4096 # 上限,防止大量来源 IP 把字典撑爆
|
# 真正的防爆破靠「验证码 + 双维度限速」两道,而不是靠计数持久化。
|
||||||
_FAILS_TTL = 3600 # 超过 1 小时无更新的条目会被清理
|
_fails = {} # key -> [count, last_ts]
|
||||||
|
_FAILS_MAX_KEYS = 8192 # 上限,防止海量来源把字典撑爆
|
||||||
|
_FAILS_TTL = 3600 # 超过 1 小时无更新即清理
|
||||||
|
|
||||||
|
CAPTCHA_SESSION_PREFIX = "cap_"
|
||||||
|
|
||||||
|
|
||||||
def _prune_fails(now=None):
|
def _prune_fails(now=None):
|
||||||
"""清掉过期条目;条目数超上限时按时间淘汰最旧的。"""
|
|
||||||
now = now or time.time()
|
now = now or time.time()
|
||||||
dead = [ip for ip, c in _fails.items() if now - c[1] > _FAILS_TTL]
|
dead = [k for k, c in _fails.items() if now - c[1] > _FAILS_TTL]
|
||||||
for ip in dead:
|
for k in dead:
|
||||||
_fails.pop(ip, None)
|
_fails.pop(k, None)
|
||||||
if len(_fails) > _FAILS_MAX_IPS:
|
if len(_fails) > _FAILS_MAX_KEYS:
|
||||||
for ip, _ in sorted(_fails.items(), key=lambda kv: kv[1][1])[:len(_fails) - _FAILS_MAX_IPS]:
|
for k, _ in sorted(_fails.items(), key=lambda kv: kv[1][1])[:len(_fails) - _FAILS_MAX_KEYS]:
|
||||||
_fails.pop(ip, None)
|
_fails.pop(k, None)
|
||||||
|
|
||||||
|
|
||||||
|
def _ip_key(ip):
|
||||||
|
return "ip:" + (ip or "")
|
||||||
|
|
||||||
|
|
||||||
|
def _user_key(username):
|
||||||
|
return "user:" + (username or "").strip().lower()
|
||||||
|
|
||||||
|
|
||||||
|
def note_fail(key):
|
||||||
|
now = time.time()
|
||||||
|
_prune_fails(now)
|
||||||
|
c = _fails.get(key)
|
||||||
|
if c is None or now - c[1] > config.LOGIN_LOCK_MINUTES * 60:
|
||||||
|
_fails[key] = [1, now]
|
||||||
|
return 1
|
||||||
|
c[0] += 1
|
||||||
|
c[1] = now
|
||||||
|
return c[0]
|
||||||
|
|
||||||
|
|
||||||
|
def is_locked(key):
|
||||||
|
c = _fails.get(key)
|
||||||
|
if not c or c[0] < config.MAX_LOGIN_FAILS:
|
||||||
|
return False
|
||||||
|
return time.time() - c[1] <= config.LOGIN_LOCK_MINUTES * 60
|
||||||
|
|
||||||
|
|
||||||
|
def clear_fail(key):
|
||||||
|
_fails.pop(key, None)
|
||||||
|
|
||||||
|
|
||||||
|
def lock_left(key):
|
||||||
|
c = _fails.get(key)
|
||||||
|
if not c:
|
||||||
|
return 0
|
||||||
|
return max(0, int(config.LOGIN_LOCK_MINUTES * 60 - (time.time() - c[1])))
|
||||||
|
|
||||||
|
|
||||||
|
def fail_count(key):
|
||||||
|
c = _fails.get(key)
|
||||||
|
return c[0] if c else 0
|
||||||
|
|
||||||
|
|
||||||
|
def auth_locked(ip, username=""):
|
||||||
|
"""返回还需锁定的秒数(0 = 未锁)。IP 与用户名任一超限即锁。"""
|
||||||
|
return max(lock_left(_ip_key(ip)), lock_left(_user_key(username)))
|
||||||
|
|
||||||
|
|
||||||
|
def note_auth_fail(ip, username=""):
|
||||||
|
n1 = note_fail(_ip_key(ip))
|
||||||
|
n2 = note_fail(_user_key(username)) if username else 0
|
||||||
|
return max(n1, n2)
|
||||||
|
|
||||||
|
|
||||||
|
def clear_auth_fail(ip, username=""):
|
||||||
|
clear_fail(_ip_key(ip))
|
||||||
|
if username:
|
||||||
|
clear_fail(_user_key(username))
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------- 口令 / 用户名策略 ----------------
|
||||||
|
_USERNAME_RE = re.compile(config.USERNAME_RE)
|
||||||
|
|
||||||
|
|
||||||
def hash_password(p):
|
def hash_password(p):
|
||||||
|
from werkzeug.security import generate_password_hash
|
||||||
return generate_password_hash(p, method="pbkdf2:sha256:200000")
|
return generate_password_hash(p, method="pbkdf2:sha256:200000")
|
||||||
|
|
||||||
|
|
||||||
def verify_password(hashed, p):
|
def verify_password(hashed, p):
|
||||||
|
from werkzeug.security import check_password_hash
|
||||||
try:
|
try:
|
||||||
return check_password_hash(hashed, p)
|
return check_password_hash(hashed, p)
|
||||||
except (ValueError, TypeError):
|
except (ValueError, TypeError):
|
||||||
return False
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def username_problem(name):
|
||||||
|
"""校验用户名。开放注册后这是第一个入口,必须收紧。"""
|
||||||
|
name = (name or "").strip()
|
||||||
|
if not name:
|
||||||
|
return "用户名必填"
|
||||||
|
if not _USERNAME_RE.match(name):
|
||||||
|
return "用户名需 3~32 位,以字母或数字开头,只能用字母、数字、下划线、点、连字符"
|
||||||
|
if name.lower() in ("admin", "administrator", "root", "system", "guest", "null"):
|
||||||
|
return "该用户名为系统保留字,请换一个"
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def password_problem(new, new2=None, username=None):
|
||||||
|
"""口令强度:8 位以上,且至少包含两类字符。
|
||||||
|
|
||||||
|
比原来的「只要 6 位」严格——因为现在任何人都能自助注册,
|
||||||
|
弱口令直接决定了整个实例的抗爆破能力。
|
||||||
|
"""
|
||||||
|
new = new or ""
|
||||||
|
if len(new) < config.PASSWORD_MIN:
|
||||||
|
return "密码至少 %d 位" % config.PASSWORD_MIN
|
||||||
|
if len(new) > config.PASSWORD_MAX:
|
||||||
|
return "密码过长(上限 %d 位)" % config.PASSWORD_MAX
|
||||||
|
classes = sum(bool(re.search(p, new)) for p in
|
||||||
|
(r"[a-z]", r"[A-Z]", r"[0-9]", r"[^A-Za-z0-9]"))
|
||||||
|
if classes < 2:
|
||||||
|
return "密码需包含大写字母、小写字母、数字、符号中的至少两类"
|
||||||
|
if new2 is not None and new2 != new:
|
||||||
|
return "两次输入的新密码不一致"
|
||||||
|
if username and new.lower() == str(username).lower():
|
||||||
|
return "密码不能与用户名相同"
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
# 兼容旧名(原来的 api.py 内部函数)
|
||||||
|
_check_password = password_problem
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------- 登录 ----------------
|
||||||
def login_ok(conn, username, password):
|
def login_ok(conn, username, password):
|
||||||
|
"""校验口令。返回 (user_row, error_message)。
|
||||||
|
|
||||||
|
停用账号与口令错误返回**同一句话**,避免探测哪些用户名存在
|
||||||
|
(不过自助注册本身就暴露了用户名唯一性,这里只是不打额外的广告)。
|
||||||
|
"""
|
||||||
|
username = (username or "").strip()
|
||||||
row = conn.execute("SELECT * FROM users WHERE username=?", (username,)).fetchone()
|
row = conn.execute("SELECT * FROM users WHERE username=?", (username,)).fetchone()
|
||||||
if row is None or not verify_password(row["password_hash"], password):
|
if row is None or not verify_password(row["password_hash"], password):
|
||||||
return None
|
return None, "用户名或密码不正确"
|
||||||
|
if (row["status"] or "active") != "active":
|
||||||
|
return None, "该账号已被停用,请联系管理员"
|
||||||
conn.execute("UPDATE users SET last_login_at=?, login_count=login_count+1 WHERE id=?",
|
conn.execute("UPDATE users SET last_login_at=?, login_count=login_count+1 WHERE id=?",
|
||||||
(db.now_str(), row["id"]))
|
(db.now_str(), row["id"]))
|
||||||
return row
|
return row, None
|
||||||
|
|
||||||
|
|
||||||
# ---------------- 跳转目标白名单(防开放重定向) ----------------
|
|
||||||
def safe_next(target, fallback="/"):
|
|
||||||
"""只允许站内相对路径。
|
|
||||||
|
|
||||||
`//evil.com`、`/\\evil.com`、`https://evil.com` 都必须拒绝:
|
|
||||||
`//` 开头是协议相对 URL,浏览器会把 `//evil.com` 当成外站跳转。
|
|
||||||
"""
|
|
||||||
if not target:
|
|
||||||
return fallback
|
|
||||||
t = str(target).strip()
|
|
||||||
if not t.startswith("/"):
|
|
||||||
return fallback
|
|
||||||
if t.startswith("//") or t.startswith("/\\") or "\\" in t:
|
|
||||||
return fallback
|
|
||||||
# 去重斜杠后仍以 // 开头的(如 "/\t/evil")一并拒绝
|
|
||||||
if t.lstrip("/").startswith("//"):
|
|
||||||
return fallback
|
|
||||||
if "\r" in t or "\n" in t:
|
|
||||||
return fallback
|
|
||||||
return t
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------- 登录失败限速 ----------------
|
|
||||||
def note_fail(ip):
|
|
||||||
now = time.time()
|
|
||||||
_prune_fails(now)
|
|
||||||
c = _fails.get(ip)
|
|
||||||
if c is None or now - c[1] > config.LOGIN_LOCK_MINUTES * 60:
|
|
||||||
_fails[ip] = [1, now]
|
|
||||||
return 1
|
|
||||||
c[0] += 1
|
|
||||||
return c[0]
|
|
||||||
|
|
||||||
|
|
||||||
def is_locked(ip):
|
|
||||||
c = _fails.get(ip)
|
|
||||||
if not c or c[0] < config.MAX_LOGIN_FAILS:
|
|
||||||
return False
|
|
||||||
return time.time() - c[1] <= config.LOGIN_LOCK_MINUTES * 60
|
|
||||||
|
|
||||||
|
|
||||||
def clear_fail(ip):
|
|
||||||
_fails.pop(ip, None)
|
|
||||||
|
|
||||||
|
|
||||||
def lock_left(ip):
|
|
||||||
c = _fails.get(ip)
|
|
||||||
if not c:
|
|
||||||
return 0
|
|
||||||
return max(0, int(config.LOGIN_LOCK_MINUTES * 60 - (time.time() - c[1])))
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------- 会话 ----------------
|
# ---------------- 会话 ----------------
|
||||||
def current_user():
|
def current_user():
|
||||||
|
"""当前登录用户(dict)或 None。
|
||||||
|
|
||||||
|
每个请求回查一次 users 表:账号被停用/删除后**立刻**失效,
|
||||||
|
而不是等 12 小时会话自然过期。结果缓存在 flask.g 里,一次请求只查一次。
|
||||||
|
"""
|
||||||
|
if "wb_user" in g:
|
||||||
|
return g.wb_user
|
||||||
uid = session.get("uid")
|
uid = session.get("uid")
|
||||||
if not uid:
|
user = None
|
||||||
return None
|
if uid:
|
||||||
return {"id": uid, "username": session.get("uname"), "display_name": session.get("dname"),
|
try:
|
||||||
"is_admin": bool(session.get("adm", 1))}
|
row = db.get_db().execute(
|
||||||
|
"SELECT id,username,display_name,is_admin,status FROM users WHERE id=?",
|
||||||
|
(uid,)).fetchone()
|
||||||
|
except Exception: # noqa: BLE001 (无请求上下文等)
|
||||||
|
row = None
|
||||||
|
if row is None or (row["status"] or "active") != "active":
|
||||||
|
session.clear()
|
||||||
|
else:
|
||||||
|
user = {"id": row["id"], "username": row["username"],
|
||||||
|
"display_name": row["display_name"] or row["username"],
|
||||||
|
"is_admin": bool(row["is_admin"])}
|
||||||
|
# 个人信息(显示名)改过之后立即生效,不必重新登录
|
||||||
|
session["dname"] = user["display_name"]
|
||||||
|
session["adm"] = 1 if user["is_admin"] else 0
|
||||||
|
g.wb_user = user
|
||||||
|
return user
|
||||||
|
|
||||||
|
|
||||||
def is_admin():
|
def is_admin():
|
||||||
@@ -121,14 +215,17 @@ def is_admin():
|
|||||||
|
|
||||||
|
|
||||||
def login_session(user):
|
def login_session(user):
|
||||||
|
"""建立登录会话。
|
||||||
|
|
||||||
|
`session.clear()` 是必须的:既清掉前一次的残留,
|
||||||
|
也顺带换掉 CSRF token 与验证码 id —— 这正是防「会话固定」的做法。
|
||||||
|
"""
|
||||||
session.clear()
|
session.clear()
|
||||||
session["uid"] = user["id"]
|
session["uid"] = user["id"]
|
||||||
session["uname"] = user["username"]
|
session["uname"] = user["username"]
|
||||||
session["dname"] = user["display_name"] or user["username"]
|
session["dname"] = user["display_name"] or user["username"]
|
||||||
try:
|
session["adm"] = 1 if user["is_admin"] else 0
|
||||||
session["adm"] = 1 if user["is_admin"] else 0
|
session["login_at"] = db.now_str()
|
||||||
except (KeyError, IndexError, TypeError):
|
|
||||||
session["adm"] = 1
|
|
||||||
session.permanent = True
|
session.permanent = True
|
||||||
|
|
||||||
|
|
||||||
@@ -167,6 +264,100 @@ def admin_required(fn):
|
|||||||
return wrapper
|
return wrapper
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------- 跳转目标白名单(防开放重定向) ----------------
|
||||||
|
def safe_next(target, fallback="/"):
|
||||||
|
"""只允许站内相对路径。
|
||||||
|
|
||||||
|
`//evil.com`、`/\\evil.com`、`https://evil.com` 都必须拒绝:
|
||||||
|
`//` 开头是协议相对 URL,浏览器会把 `//evil.com` 当成外站跳转。
|
||||||
|
"""
|
||||||
|
if not target:
|
||||||
|
return fallback
|
||||||
|
t = str(target).strip()
|
||||||
|
if not t.startswith("/"):
|
||||||
|
return fallback
|
||||||
|
if t.startswith("//") or t.startswith("/\\") or "\\" in t:
|
||||||
|
return fallback
|
||||||
|
if t.lstrip("/").startswith("//"):
|
||||||
|
return fallback
|
||||||
|
if "\r" in t or "\n" in t:
|
||||||
|
return fallback
|
||||||
|
return t
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------- 图形验证码 ----------------
|
||||||
|
def captcha_required(conn, ip, username=""):
|
||||||
|
"""按 captcha_policy 决定本次是否需要验证码。"""
|
||||||
|
policy = (db.get_setting(conn, "captcha_policy", "always") or "always").strip().lower()
|
||||||
|
if policy == "off":
|
||||||
|
return False
|
||||||
|
if policy == "adaptive":
|
||||||
|
# 「自适应」= 这个来源出过问题才要求,日常登录不打扰
|
||||||
|
return (fail_count(_ip_key(ip)) >= 2
|
||||||
|
or (username and fail_count(_user_key(username)) >= 2))
|
||||||
|
return True # always(默认)
|
||||||
|
|
||||||
|
|
||||||
|
def issue_captcha(conn, purpose):
|
||||||
|
"""新建验证码并把 id 记进会话,返回 PNG 字节。答案绝不离开服务端。"""
|
||||||
|
try:
|
||||||
|
length = int(db.get_setting(conn, "captcha_length", 4) or 4)
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
length = 4
|
||||||
|
length = max(4, min(6, length))
|
||||||
|
cid, code = captcha.create(conn, purpose, length=length)
|
||||||
|
session[CAPTCHA_SESSION_PREFIX + purpose] = cid
|
||||||
|
return captcha.render(code, width=150 if length <= 4 else 150 + (length - 4) * 32)
|
||||||
|
|
||||||
|
|
||||||
|
def consume_captcha(conn, purpose, answer):
|
||||||
|
"""校验并作废本次验证码。会话里的 id 一并丢掉,逼迫下次换一张新图。"""
|
||||||
|
cid = session.pop(CAPTCHA_SESSION_PREFIX + purpose, None)
|
||||||
|
return captcha.verify(conn, cid, (answer or "").strip().upper(), purpose)
|
||||||
|
|
||||||
|
|
||||||
|
# 验证码出图限速:出图本身要做点阵渲染 + zlib,不设限就是一条廉价的
|
||||||
|
# CPU/带宽放大路径(有人拿它当免费的图片生成器刷)。
|
||||||
|
_cap_fetch = {} # ip -> [count, window_started_at]
|
||||||
|
_CAP_FETCH_MAX = 40 # 每窗口最多出图张数
|
||||||
|
_CAP_FETCH_WINDOW = 60 # 窗口长度(秒)
|
||||||
|
|
||||||
|
|
||||||
|
def captcha_fetch_allowed(ip):
|
||||||
|
now = time.time()
|
||||||
|
cur = _cap_fetch.get(ip)
|
||||||
|
if cur is None or now - cur[1] > _CAP_FETCH_WINDOW:
|
||||||
|
if len(_cap_fetch) > _FAILS_MAX_KEYS:
|
||||||
|
_cap_fetch.clear()
|
||||||
|
_cap_fetch[ip] = [1, now]
|
||||||
|
return True
|
||||||
|
cur[0] += 1
|
||||||
|
return cur[0] <= _CAP_FETCH_MAX
|
||||||
|
|
||||||
|
|
||||||
|
def audit_login_fail(conn, username, detail, ip):
|
||||||
|
"""登录失败审计。
|
||||||
|
|
||||||
|
`user_id` 留 0:此时还不能确定是谁(可能是有人在撞别人的账号),
|
||||||
|
但 `actor` 记下被尝试的用户名,便于事后按人名检索。
|
||||||
|
"""
|
||||||
|
db.audit(conn, "login_failed", username or "-", detail, ip, 0)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------- 注册开关与配额 ----------------
|
||||||
|
def register_allowed(conn):
|
||||||
|
return db.get_bool(conn, "allow_register", True)
|
||||||
|
|
||||||
|
|
||||||
|
def register_quota(conn, ip):
|
||||||
|
"""同一 IP 当天的注册配额。返回 (是否允许, 已注册数, 上限)。"""
|
||||||
|
limit = db.get_int(conn, "register_max_per_ip", 3)
|
||||||
|
today = db.now_str()[:10]
|
||||||
|
n = conn.execute("SELECT COUNT(*) FROM users WHERE register_ip=?"
|
||||||
|
" AND substr(COALESCE(created_at,''),1,10)=?", (ip, today)).fetchone()[0]
|
||||||
|
return n < limit, n, limit
|
||||||
|
|
||||||
|
|
||||||
# ---------------- CSRF ----------------
|
# ---------------- CSRF ----------------
|
||||||
def csrf_token():
|
def csrf_token():
|
||||||
t = session.get("_csrf")
|
t = session.get("_csrf")
|
||||||
@@ -182,11 +373,39 @@ def check_csrf():
|
|||||||
sent = request.form.get("_csrf") or request.headers.get("X-CSRF-Token") or ""
|
sent = request.form.get("_csrf") or request.headers.get("X-CSRF-Token") or ""
|
||||||
if not sent or not hmac.compare_digest(sent, session.get("_csrf", "")):
|
if not sent or not hmac.compare_digest(sent, session.get("_csrf", "")):
|
||||||
if wants_json():
|
if wants_json():
|
||||||
return jsonify({"ok": False, "error": "csrf", "message": "CSRF 校验失败,请刷新页面"}), 400
|
return jsonify({"ok": False, "error": "csrf",
|
||||||
|
"message": "CSRF 校验失败,请刷新页面"}), 400
|
||||||
return "CSRF 校验失败,请刷新页面后重试", 400
|
return "CSRF 校验失败,请刷新页面后重试", 400
|
||||||
return None
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------- 安全响应头 ----------------
|
||||||
|
# 这些头是「纵深防御」:本项目的输出都过了 Jinja 自动转义 + app.js 手动转义,
|
||||||
|
# 但多一层 nosniff / frame-ancestors 能让「某处漏转义」不至于直接变成可利用的 XSS。
|
||||||
|
CSP = ("default-src 'self'; "
|
||||||
|
"img-src 'self' data:; "
|
||||||
|
"style-src 'self' 'unsafe-inline'; "
|
||||||
|
"script-src 'self' 'unsafe-inline'; "
|
||||||
|
"connect-src 'self'; "
|
||||||
|
"font-src 'self' data:; "
|
||||||
|
"object-src 'none'; "
|
||||||
|
"base-uri 'self'; "
|
||||||
|
"form-action 'self'; "
|
||||||
|
"frame-ancestors 'none'")
|
||||||
|
|
||||||
|
|
||||||
|
def apply_security_headers(resp):
|
||||||
|
resp.headers.setdefault("X-Content-Type-Options", "nosniff")
|
||||||
|
resp.headers.setdefault("X-Frame-Options", "DENY")
|
||||||
|
# 不让站内 URL(可能含 next=、run= 等参数)随外链 referer 泄漏出去
|
||||||
|
resp.headers.setdefault("Referrer-Policy", "same-origin")
|
||||||
|
resp.headers.setdefault("Content-Security-Policy", CSP)
|
||||||
|
resp.headers.setdefault("Cross-Origin-Opener-Policy", "same-origin")
|
||||||
|
if request.path.startswith("/api/") or request.path.startswith("/captcha"):
|
||||||
|
resp.headers.setdefault("Cache-Control", "no-store")
|
||||||
|
return resp
|
||||||
|
|
||||||
|
|
||||||
def init_app(app):
|
def init_app(app):
|
||||||
app.jinja_env.globals["csrf_token"] = csrf_token
|
app.jinja_env.globals["csrf_token"] = csrf_token
|
||||||
app.jinja_env.globals["current_user"] = current_user
|
app.jinja_env.globals["current_user"] = current_user
|
||||||
@@ -194,3 +413,7 @@ def init_app(app):
|
|||||||
@app.before_request
|
@app.before_request
|
||||||
def _guard():
|
def _guard():
|
||||||
return check_csrf()
|
return check_csrf()
|
||||||
|
|
||||||
|
@app.after_request
|
||||||
|
def _headers(resp):
|
||||||
|
return apply_security_headers(resp)
|
||||||
|
|||||||
@@ -9,23 +9,39 @@
|
|||||||
* 参数 from/to 为 'YYYY-MM-DD';缺省则不限(即全量)
|
* 参数 from/to 为 'YYYY-MM-DD';缺省则不限(即全量)
|
||||||
* 列表类接口默认不返回 prompt 全文(占传输量约 80%),只有 /api/top 与
|
* 列表类接口默认不返回 prompt 全文(占传输量约 80%),只有 /api/top 与
|
||||||
/api/records/<request_id> 会带
|
/api/records/<request_id> 会带
|
||||||
|
|
||||||
|
**多用户约定**
|
||||||
|
每个接口都只操作 `current_user()["id"]` 那份数据。查询函数要求显式传 uid,
|
||||||
|
所以这里漏传会直接 TypeError(而不是静默返回全量)。
|
||||||
|
`/api/settings` 是唯一的例外:它会回传实例级配置供非管理员只读展示,
|
||||||
|
但**拒绝**非管理员写入实例级键。
|
||||||
"""
|
"""
|
||||||
import os
|
import os
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
|
|
||||||
from flask import Blueprint, jsonify, request
|
from flask import Blueprint, jsonify, request
|
||||||
|
|
||||||
from .. import collect, config, db, query, scheduler
|
from .. import collect, config, db, query, scheduler, security
|
||||||
from ..security import admin_required, current_user, login_required
|
from ..security import admin_required, current_user, is_admin, login_required
|
||||||
|
|
||||||
bp = Blueprint("api", __name__, url_prefix="/api")
|
bp = Blueprint("api", __name__, url_prefix="/api")
|
||||||
|
|
||||||
|
|
||||||
|
def _uid():
|
||||||
|
u = current_user()
|
||||||
|
return u["id"] if u else 0
|
||||||
|
|
||||||
|
|
||||||
def _arg(name, default=None):
|
def _arg(name, default=None):
|
||||||
v = request.args.get(name)
|
v = request.args.get(name)
|
||||||
return v if v not in (None, "") else default
|
return v if v not in (None, "") else default
|
||||||
|
|
||||||
|
|
||||||
|
def _json_body():
|
||||||
|
body = request.get_json(silent=True)
|
||||||
|
return body if isinstance(body, dict) else {}
|
||||||
|
|
||||||
|
|
||||||
class BadParam(ValueError):
|
class BadParam(ValueError):
|
||||||
"""查询参数不合法 -> 由 __init__ 的 ValueError 处理器统一转成 400。"""
|
"""查询参数不合法 -> 由 __init__ 的 ValueError 处理器统一转成 400。"""
|
||||||
|
|
||||||
@@ -60,7 +76,7 @@ def _int(name, default, lo=1, hi=2000):
|
|||||||
@bp.get("/manifest")
|
@bp.get("/manifest")
|
||||||
@login_required
|
@login_required
|
||||||
def api_manifest():
|
def api_manifest():
|
||||||
return jsonify(query.manifest(db.get_db()))
|
return jsonify(query.manifest(db.get_db(), _uid()))
|
||||||
|
|
||||||
|
|
||||||
@bp.get("/bundle")
|
@bp.get("/bundle")
|
||||||
@@ -68,31 +84,33 @@ def api_manifest():
|
|||||||
def api_bundle():
|
def api_bundle():
|
||||||
"""大屏页一次拿齐:全量 daily + 窗口 dims/top/records。"""
|
"""大屏页一次拿齐:全量 daily + 窗口 dims/top/records。"""
|
||||||
frm, to = _win()
|
frm, to = _win()
|
||||||
return jsonify(query.bundle(db.get_db(), frm, to, top_n=_int("topN", query.DEFAULT_TOP_N, 1, 1000)))
|
return jsonify(query.bundle(db.get_db(), _uid(), frm, to,
|
||||||
|
top_n=_int("topN", query.DEFAULT_TOP_N, 1, 1000)))
|
||||||
|
|
||||||
|
|
||||||
@bp.get("/summary")
|
@bp.get("/summary")
|
||||||
@login_required
|
@login_required
|
||||||
def api_summary():
|
def api_summary():
|
||||||
conn = db.get_db()
|
conn = db.get_db()
|
||||||
|
uid = _uid()
|
||||||
frm, to = _win()
|
frm, to = _win()
|
||||||
if not frm or not to:
|
if not frm or not to:
|
||||||
t = query.totals(conn)
|
t = query.totals(conn, uid)
|
||||||
frm, to = t["firstDay"], t["lastDay"]
|
frm, to = t["firstDay"], t["lastDay"]
|
||||||
return jsonify(query.summary(conn, frm, to))
|
return jsonify(query.summary(conn, uid, frm, to))
|
||||||
|
|
||||||
|
|
||||||
@bp.get("/daily")
|
@bp.get("/daily")
|
||||||
@login_required
|
@login_required
|
||||||
def api_daily():
|
def api_daily():
|
||||||
return jsonify({"days": query.daily(db.get_db(), *_win())})
|
return jsonify({"days": query.daily(db.get_db(), _uid(), *_win())})
|
||||||
|
|
||||||
|
|
||||||
@bp.get("/dims")
|
@bp.get("/dims")
|
||||||
@login_required
|
@login_required
|
||||||
def api_dims():
|
def api_dims():
|
||||||
conn = db.get_db()
|
conn = db.get_db()
|
||||||
d = query.dims(conn, *_win())
|
d = query.dims(conn, _uid(), *_win())
|
||||||
dim = _arg("dim")
|
dim = _arg("dim")
|
||||||
if dim in d:
|
if dim in d:
|
||||||
return jsonify({dim: d[dim]})
|
return jsonify({dim: d[dim]})
|
||||||
@@ -103,17 +121,18 @@ def api_dims():
|
|||||||
@login_required
|
@login_required
|
||||||
def api_top():
|
def api_top():
|
||||||
conn = db.get_db()
|
conn = db.get_db()
|
||||||
return jsonify(query.top(conn, *_win(), n=_int("n", 50, 1, 1000)))
|
return jsonify(query.top(conn, _uid(), *_win(), n=_int("n", 50, 1, 1000)))
|
||||||
|
|
||||||
|
|
||||||
@bp.get("/records")
|
@bp.get("/records")
|
||||||
@login_required
|
@login_required
|
||||||
def api_records():
|
def api_records():
|
||||||
conn = db.get_db()
|
conn = db.get_db()
|
||||||
|
uid = _uid()
|
||||||
frm, to = _win()
|
frm, to = _win()
|
||||||
page = _int("page", 1, 1, 100000)
|
page = _int("page", 1, 1, 100000)
|
||||||
size = _int("size", 50, 1, 500)
|
size = _int("size", 50, 1, 500)
|
||||||
r = query.records_page(conn, frm, to, model=_arg("model"), client=_arg("client"),
|
r = query.records_page(conn, uid, frm, to, model=_arg("model"), client=_arg("client"),
|
||||||
q=_arg("q"), page=page, size=size, order=_arg("order", "ts_desc"),
|
q=_arg("q"), page=page, size=size, order=_arg("order", "ts_desc"),
|
||||||
with_prompt=False if _arg("lean") == "1" else True)
|
with_prompt=False if _arg("lean") == "1" else True)
|
||||||
return jsonify(r)
|
return jsonify(r)
|
||||||
@@ -122,8 +141,9 @@ def api_records():
|
|||||||
@bp.get("/records/<request_id>")
|
@bp.get("/records/<request_id>")
|
||||||
@login_required
|
@login_required
|
||||||
def api_record(request_id):
|
def api_record(request_id):
|
||||||
row = db.get_db().execute(
|
# user_id 必须进 WHERE:否则改一个 URL 就能读到别人的 Prompt 全文
|
||||||
"SELECT * FROM usage_records WHERE request_id=?", (request_id,)).fetchone()
|
row = db.get_db().execute("SELECT * FROM usage_records WHERE user_id=? AND request_id=?",
|
||||||
|
(_uid(), request_id)).fetchone()
|
||||||
if row is None:
|
if row is None:
|
||||||
return jsonify({"ok": False, "message": "记录不存在"}), 404
|
return jsonify({"ok": False, "message": "记录不存在"}), 404
|
||||||
return jsonify(dict(row))
|
return jsonify(dict(row))
|
||||||
@@ -135,14 +155,16 @@ def api_runs():
|
|||||||
conn = db.get_db()
|
conn = db.get_db()
|
||||||
rows = conn.execute("SELECT id,trigger,status,started_at,finished_at,duration_ms,win_from,"
|
rows = conn.execute("SELECT id,trigger,status,started_at,finished_at,duration_ms,win_from,"
|
||||||
"win_to,fetched,added,dup,total,conflicts,exit_code,message"
|
"win_to,fetched,added,dup,total,conflicts,exit_code,message"
|
||||||
" FROM collect_runs ORDER BY id DESC LIMIT ?", (_int("limit", 50, 1, 500),))
|
" FROM collect_runs WHERE user_id=? ORDER BY id DESC LIMIT ?",
|
||||||
|
(_uid(), _int("limit", 50, 1, 500)))
|
||||||
return jsonify({"items": [dict(r) for r in rows]})
|
return jsonify({"items": [dict(r) for r in rows]})
|
||||||
|
|
||||||
|
|
||||||
@bp.get("/runs/<int:run_id>")
|
@bp.get("/runs/<int:run_id>")
|
||||||
@login_required
|
@login_required
|
||||||
def api_run(run_id):
|
def api_run(run_id):
|
||||||
row = db.get_db().execute("SELECT * FROM collect_runs WHERE id=?", (run_id,)).fetchone()
|
row = db.get_db().execute("SELECT * FROM collect_runs WHERE id=? AND user_id=?",
|
||||||
|
(run_id, _uid())).fetchone()
|
||||||
if row is None:
|
if row is None:
|
||||||
return jsonify({"ok": False, "message": "运行记录不存在"}), 404
|
return jsonify({"ok": False, "message": "运行记录不存在"}), 404
|
||||||
return jsonify(dict(row))
|
return jsonify(dict(row))
|
||||||
@@ -152,23 +174,30 @@ def api_run(run_id):
|
|||||||
@login_required
|
@login_required
|
||||||
def api_status():
|
def api_status():
|
||||||
conn = db.get_db()
|
conn = db.get_db()
|
||||||
|
uid = _uid()
|
||||||
sch = scheduler.get_scheduler()
|
sch = scheduler.get_scheduler()
|
||||||
nxt = scheduler.next_run_at(conn)
|
nxt = scheduler.next_run_at(conn, uid)
|
||||||
last = conn.execute("SELECT * FROM collect_runs ORDER BY id DESC LIMIT 1").fetchone()
|
last = conn.execute("SELECT * FROM collect_runs WHERE user_id=? ORDER BY id DESC LIMIT 1",
|
||||||
running = conn.execute("SELECT COUNT(*) FROM collect_runs WHERE status='running'").fetchone()[0]
|
(uid,)).fetchone()
|
||||||
|
running = conn.execute("SELECT COUNT(*) FROM collect_runs WHERE user_id=? AND status='running'",
|
||||||
|
(uid,)).fetchone()[0]
|
||||||
|
cred = db.secret_state(conn, "cookie", uid)
|
||||||
return jsonify({
|
return jsonify({
|
||||||
"server_time": db.now_str(),
|
"server_time": db.now_str(),
|
||||||
"scheduler": {
|
"scheduler": {
|
||||||
"running": sch.running,
|
"running": sch.running,
|
||||||
"enabled": db.get_bool(conn, "schedule_enabled", True),
|
"enabled": db.get_bool(conn, "schedule_enabled", True, uid),
|
||||||
"times": scheduler.slots(conn),
|
"times": scheduler.slots(conn, uid),
|
||||||
"next_run": nxt.strftime("%Y-%m-%d %H:%M:%S") if nxt else None,
|
"next_run": nxt.strftime("%Y-%m-%d %H:%M:%S") if nxt else None,
|
||||||
"catch_up": db.get_bool(conn, "catch_up", True),
|
"catch_up": db.get_bool(conn, "catch_up", True, uid),
|
||||||
"lockfile": os.path.exists(collect.LOCK_PATH),
|
"lockfile": os.path.exists(collect.LOCK_PATH),
|
||||||
},
|
},
|
||||||
"running_runs": running,
|
"running_runs": running,
|
||||||
"last_run": dict(last) if last else None,
|
"last_run": dict(last) if last else None,
|
||||||
"cookie_set": bool((db.get_setting(conn, "cookie") or "").strip()),
|
# 只回「有没有配」与字符数,绝不回凭证内容
|
||||||
|
"cookie_set": bool(cred["set"] and not cred["broken"]),
|
||||||
|
"cookie_chars": cred["chars"],
|
||||||
|
"cookie_broken": cred["broken"],
|
||||||
})
|
})
|
||||||
|
|
||||||
|
|
||||||
@@ -176,9 +205,8 @@ def api_status():
|
|||||||
@login_required
|
@login_required
|
||||||
def api_collect():
|
def api_collect():
|
||||||
"""手动触发一次采集(后台线程之外同步执行,页面等待结果)。"""
|
"""手动触发一次采集(后台线程之外同步执行,页面等待结果)。"""
|
||||||
body = request.get_json(silent=True) or {}
|
u = current_user()
|
||||||
if not isinstance(body, dict):
|
body = _json_body()
|
||||||
return jsonify({"ok": False, "message": "请求体必须是对象"}), 400
|
|
||||||
frm, to = body.get("from"), body.get("to")
|
frm, to = body.get("from"), body.get("to")
|
||||||
try:
|
try:
|
||||||
kw = {}
|
kw = {}
|
||||||
@@ -194,40 +222,55 @@ def api_collect():
|
|||||||
kw["to_dt"] = datetime.strptime(d, "%Y-%m-%d").replace(hour=23, minute=59, second=59)
|
kw["to_dt"] = datetime.strptime(d, "%Y-%m-%d").replace(hour=23, minute=59, second=59)
|
||||||
if kw.get("from_dt") and kw.get("to_dt") and kw["from_dt"] > kw["to_dt"]:
|
if kw.get("from_dt") and kw.get("to_dt") and kw["from_dt"] > kw["to_dt"]:
|
||||||
raise BadParam("起始日期不能晚于结束日期")
|
raise BadParam("起始日期不能晚于结束日期")
|
||||||
r = collect.run_sync(trigger="manual", **kw)
|
r = collect.run_sync(trigger="manual", uid=u["id"], **kw)
|
||||||
except BadParam as e:
|
except BadParam as e:
|
||||||
return jsonify({"ok": False, "error": "bad_request", "message": str(e)}), 400
|
return jsonify({"ok": False, "error": "bad_request", "message": str(e)}), 400
|
||||||
except collect.Busy as e:
|
except collect.Busy as e:
|
||||||
return jsonify({"ok": False, "error": "busy", "message": str(e)}), 409
|
return jsonify({"ok": False, "error": "busy", "message": str(e)}), 409
|
||||||
|
except collect.NotReady as e:
|
||||||
|
return jsonify({"ok": False, "error": "no_cookie", "message": str(e)}), 409
|
||||||
|
except db.SecretUnreadable as e:
|
||||||
|
return jsonify({"ok": False, "error": "cookie_broken",
|
||||||
|
"message": "已保存的 Cookie 无法解密(实例密钥被更换过):%s。"
|
||||||
|
"请到「配置管理」重新粘贴。" % e}), 409
|
||||||
except collect.ApiError as e:
|
except collect.ApiError as e:
|
||||||
code = 401 if e.cookie_expired else 502
|
code = 401 if e.cookie_expired else 502
|
||||||
return jsonify({"ok": False, "error": "cookie_expired" if e.cookie_expired else "api",
|
return jsonify({"ok": False, "error": "cookie_expired" if e.cookie_expired else "api",
|
||||||
"message": str(e)}), code
|
"message": str(e)}), code
|
||||||
except Exception as e: # noqa: BLE001
|
except Exception as e: # noqa: BLE001
|
||||||
return jsonify({"ok": False, "error": "internal", "message": str(e)}), 500
|
return jsonify({"ok": False, "error": "internal", "message": str(e)}), 500
|
||||||
db.audit(db.get_db(), "collect", (current_user() or {}).get("username"), r["message"],
|
db.audit(db.get_db(), "collect", u["username"], r["message"], request.remote_addr, u["id"])
|
||||||
request.remote_addr)
|
|
||||||
return jsonify({"ok": True, "result": r})
|
return jsonify({"ok": True, "result": r})
|
||||||
|
|
||||||
|
|
||||||
@bp.get("/audit")
|
@bp.get("/audit")
|
||||||
@login_required
|
@login_required
|
||||||
def api_audit():
|
def api_audit():
|
||||||
"""操作审计分页(日志管理页用;原来只能看最近 40 条)。"""
|
"""操作审计分页。管理员看全部(便于追责),普通用户只看自己触发的。"""
|
||||||
conn = db.get_db()
|
conn = db.get_db()
|
||||||
|
u = current_user()
|
||||||
action = _arg("action")
|
action = _arg("action")
|
||||||
page = _int("page", 1, 1, 100000)
|
page = _int("page", 1, 1, 100000)
|
||||||
size = _int("size", 50, 1, 500)
|
size = _int("size", 50, 1, 500)
|
||||||
w, p = "", []
|
w, p = [], []
|
||||||
|
if not u["is_admin"]:
|
||||||
|
w.append("user_id = ?")
|
||||||
|
p.append(u["id"])
|
||||||
if action:
|
if action:
|
||||||
w, p = "WHERE action = ?", [action]
|
w.append("action = ?")
|
||||||
total = conn.execute("SELECT COUNT(*) FROM audit_log %s" % w, p).fetchone()[0]
|
p.append(action)
|
||||||
rows = conn.execute("SELECT * FROM audit_log %s ORDER BY id DESC LIMIT ? OFFSET ?" % w,
|
ws = ("WHERE " + " AND ".join(w)) if w else ""
|
||||||
|
total = conn.execute("SELECT COUNT(*) FROM audit_log %s" % ws, p).fetchone()[0]
|
||||||
|
rows = conn.execute("SELECT * FROM audit_log %s ORDER BY id DESC LIMIT ? OFFSET ?" % ws,
|
||||||
p + [size, (page - 1) * size])
|
p + [size, (page - 1) * size])
|
||||||
|
# 动作清单不带 action 条件,否则只剩下当前那一个动作可选
|
||||||
|
base_p = [u["id"]] if not u["is_admin"] else []
|
||||||
|
base = "WHERE user_id = ?" if not u["is_admin"] else ""
|
||||||
actions = [r[0] for r in conn.execute(
|
actions = [r[0] for r in conn.execute(
|
||||||
"SELECT DISTINCT action FROM audit_log ORDER BY action")]
|
"SELECT DISTINCT action FROM audit_log %s ORDER BY action" % base, base_p)]
|
||||||
return jsonify({"total": total, "page": page, "size": size,
|
return jsonify({"total": total, "page": page, "size": size,
|
||||||
"pages": max(1, (total + size - 1) // size),
|
"pages": max(1, (total + size - 1) // size),
|
||||||
|
"scope": "all" if u["is_admin"] else "self",
|
||||||
"actions": actions,
|
"actions": actions,
|
||||||
"items": [dict(r) for r in rows]})
|
"items": [dict(r) for r in rows]})
|
||||||
|
|
||||||
@@ -236,13 +279,26 @@ def api_audit():
|
|||||||
@bp.post("/maintenance/<action>")
|
@bp.post("/maintenance/<action>")
|
||||||
@login_required
|
@login_required
|
||||||
def api_maintenance(action):
|
def api_maintenance(action):
|
||||||
"""把 CLI 里的维护动作搬到页面上:补全 prompt / VACUUM / 导出 CSV。"""
|
"""把 CLI 里的维护动作搬到页面上。
|
||||||
|
|
||||||
|
只有 `vacuum` 是**实例级**动作(整个库一起整理),所以它仅管理员可用;
|
||||||
|
其余三个都只作用于当前账号自己的数据。
|
||||||
|
"""
|
||||||
conn = db.get_db()
|
conn = db.get_db()
|
||||||
user = (current_user() or {}).get("username")
|
u = current_user()
|
||||||
|
uid = u["id"]
|
||||||
|
if action == "vacuum" and not u["is_admin"]:
|
||||||
|
return jsonify({"ok": False, "error": "forbidden",
|
||||||
|
"message": "数据库整理是整库操作,仅管理员可执行"}), 403
|
||||||
try:
|
try:
|
||||||
if action == "fill-prompt":
|
if action == "fill-prompt":
|
||||||
try:
|
try:
|
||||||
n = collect.fill_prompt(conn, log=lambda m: None)
|
n = collect.fill_prompt(conn, uid, log=lambda m: None)
|
||||||
|
except collect.NotReady as e:
|
||||||
|
return jsonify({"ok": False, "error": "no_cookie", "message": str(e)}), 409
|
||||||
|
except db.SecretUnreadable as e:
|
||||||
|
return jsonify({"ok": False, "error": "cookie_broken",
|
||||||
|
"message": "已保存的 Cookie 无法解密,请重新粘贴:%s" % e}), 409
|
||||||
except collect.ApiError as e:
|
except collect.ApiError as e:
|
||||||
return jsonify({"ok": False, "error": "api", "message": str(e)}), 502
|
return jsonify({"ok": False, "error": "api", "message": str(e)}), 502
|
||||||
msg = "补全 %d 条 User Prompt" % n
|
msg = "补全 %d 条 User Prompt" % n
|
||||||
@@ -250,19 +306,20 @@ def api_maintenance(action):
|
|||||||
before = os.path.getsize(config.SQLITE_PATH) if os.path.exists(config.SQLITE_PATH) else 0
|
before = os.path.getsize(config.SQLITE_PATH) if os.path.exists(config.SQLITE_PATH) else 0
|
||||||
conn.execute("PRAGMA wal_checkpoint(TRUNCATE)")
|
conn.execute("PRAGMA wal_checkpoint(TRUNCATE)")
|
||||||
conn.execute("VACUUM")
|
conn.execute("VACUUM")
|
||||||
|
conn.execute("PRAGMA optimize")
|
||||||
after = os.path.getsize(config.SQLITE_PATH) if os.path.exists(config.SQLITE_PATH) else 0
|
after = os.path.getsize(config.SQLITE_PATH) if os.path.exists(config.SQLITE_PATH) else 0
|
||||||
msg = "数据库整理完成:%s → %s" % (_human(before), _human(after))
|
msg = "数据库整理完成:%s → %s" % (_human(before), _human(after))
|
||||||
elif action == "export-csv":
|
elif action == "export-csv":
|
||||||
path, n = collect.export_csv(conn)
|
path, n = collect.export_csv(conn, uid, username=u["username"])
|
||||||
msg = "已导出 %d 条到 %s" % (n, os.path.relpath(path, config.BASE_DIR))
|
msg = "已导出 %d 条到 %s" % (n, os.path.relpath(path, config.BASE_DIR))
|
||||||
elif action == "recount":
|
elif action == "recount":
|
||||||
n = collect.record_count(conn)
|
n = collect.record_count(conn, uid)
|
||||||
msg = "存档当前 %d 条记录" % n
|
msg = "存档当前 %d 条记录" % n
|
||||||
else:
|
else:
|
||||||
return jsonify({"ok": False, "error": "unknown", "message": "未知维护动作"}), 404
|
return jsonify({"ok": False, "error": "unknown", "message": "未知维护动作"}), 404
|
||||||
except Exception as e: # noqa: BLE001
|
except Exception as e: # noqa: BLE001
|
||||||
return jsonify({"ok": False, "error": "internal", "message": str(e)}), 500
|
return jsonify({"ok": False, "error": "internal", "message": str(e)}), 500
|
||||||
db.audit(conn, "maintenance:" + action, user, msg, request.remote_addr)
|
db.audit(conn, "maintenance:" + action, u["username"], msg, request.remote_addr, uid)
|
||||||
return jsonify({"ok": True, "message": msg})
|
return jsonify({"ok": True, "message": msg})
|
||||||
|
|
||||||
|
|
||||||
@@ -276,16 +333,18 @@ def _human(n):
|
|||||||
@bp.get("/settings")
|
@bp.get("/settings")
|
||||||
@login_required
|
@login_required
|
||||||
def api_settings_get():
|
def api_settings_get():
|
||||||
|
"""当前账号的**有效配置**(不含任何凭证明文)。"""
|
||||||
conn = db.get_db()
|
conn = db.get_db()
|
||||||
s = db.get_settings(conn)
|
u = current_user()
|
||||||
if (s.get("cookie") or "").strip():
|
s = db.get_settings(conn, uid=u["id"])
|
||||||
s["cookie_hint"] = "%d 字符,…%s" % (len(s["cookie"]), s["cookie"][-12:])
|
st = db.secret_state(conn, "cookie", u["id"])
|
||||||
else:
|
s["cookie_hint"] = ("%d 字符,…%s" % (st["chars"], st["tail"])) if st["set"] else ""
|
||||||
s["cookie_hint"] = ""
|
s["cookie_broken"] = st["broken"]
|
||||||
s.pop("cookie", None) # 不回传明文凭证
|
|
||||||
# 内部簿记键(slot:09:00 这类调度槽位标记)不属于配置项,绝不外泄
|
# 内部簿记键(slot:09:00 这类调度槽位标记)不属于配置项,绝不外泄
|
||||||
for k in [k for k in list(s) if config.is_internal_key(k)]:
|
for k in [k for k in list(s) if config.is_internal_key(k)]:
|
||||||
s.pop(k, None)
|
s.pop(k, None)
|
||||||
|
s["_globalKeys"] = sorted(config.GLOBAL_KEYS)
|
||||||
|
s["_canEditGlobal"] = bool(u["is_admin"])
|
||||||
return jsonify(s)
|
return jsonify(s)
|
||||||
|
|
||||||
|
|
||||||
@@ -293,137 +352,200 @@ def api_settings_get():
|
|||||||
@login_required
|
@login_required
|
||||||
def api_settings_post():
|
def api_settings_post():
|
||||||
conn = db.get_db()
|
conn = db.get_db()
|
||||||
body = request.get_json(silent=True) or {}
|
u = current_user()
|
||||||
if not isinstance(body, dict):
|
uid = u["id"]
|
||||||
return jsonify({"ok": False, "message": "请求体必须是对象"}), 400
|
body = _json_body()
|
||||||
changed, errors, ignored = [], [], []
|
changed, errors, ignored, denied = [], [], [], []
|
||||||
for k, v in body.items():
|
for k, v in body.items():
|
||||||
if config.is_internal_key(k):
|
if config.is_internal_key(k):
|
||||||
ignored.append(k)
|
ignored.append(k)
|
||||||
continue # slot:* 是调度簿记,不允许前台写
|
continue # slot:* 是调度簿记,不允许前台写
|
||||||
|
if k not in config.DEFAULTS:
|
||||||
|
errors.append("未知配置项:%s" % k)
|
||||||
|
continue
|
||||||
|
if config.is_global_key(k) and not u["is_admin"]:
|
||||||
|
# 实例级配置(接口基址、注册开关)只有管理员能改 ——
|
||||||
|
# 否则任意注册用户都能把大家的数据采集指向别的服务器
|
||||||
|
denied.append(k)
|
||||||
|
continue
|
||||||
if k == "cookie":
|
if k == "cookie":
|
||||||
if not str(v).strip():
|
raw = str(v).strip()
|
||||||
|
if not raw:
|
||||||
continue # 空值不动,避免误清
|
continue # 空值不动,避免误清
|
||||||
if str(v).strip().lower() in ("__clear__", "-"):
|
if raw.lower() in ("__clear__", "-"):
|
||||||
db.set_setting(conn, "cookie", "")
|
db.set_secret(conn, "cookie", "", uid)
|
||||||
changed.append(k)
|
changed.append(k)
|
||||||
continue
|
continue
|
||||||
val, err = config.normalize_setting(k, v)
|
val, err = config.normalize_setting(k, v)
|
||||||
if err:
|
if err:
|
||||||
errors.append(err)
|
errors.append(err)
|
||||||
continue
|
continue
|
||||||
db.set_setting(conn, k, val)
|
if k in config.ENCRYPTED_KEYS:
|
||||||
|
db.set_secret(conn, k, val, uid)
|
||||||
|
else:
|
||||||
|
db.set_setting(conn, k, val, uid)
|
||||||
changed.append(k)
|
changed.append(k)
|
||||||
|
if denied:
|
||||||
|
errors.append("以下为实例级配置,仅管理员可修改:%s" % "、".join(sorted(denied)))
|
||||||
if errors:
|
if errors:
|
||||||
db.audit(conn, "settings_rejected", (current_user() or {}).get("username"),
|
db.audit(conn, "settings_rejected", u["username"], ";".join(errors)[:500],
|
||||||
";".join(errors)[:500], request.remote_addr)
|
request.remote_addr, uid)
|
||||||
return jsonify({"ok": False, "error": "invalid", "message": ";".join(errors),
|
return jsonify({"ok": False, "error": "invalid", "message": ";".join(errors),
|
||||||
"errors": errors, "changed": sorted(changed)}), 400
|
"errors": errors, "changed": sorted(changed)}), 400
|
||||||
# 调整调度配置后清掉槽位标记,让新时刻立即生效
|
# 调整调度配置后清掉槽位标记,让新时刻立即生效(只清自己的)
|
||||||
if {"schedule_times", "schedule_enabled"} & set(changed):
|
if {"schedule_times", "schedule_enabled"} & set(changed):
|
||||||
conn.execute("DELETE FROM settings WHERE key LIKE ?", (scheduler.SLOT_PREFIX + "%",))
|
conn.execute("DELETE FROM settings WHERE user_id=? AND key LIKE ?",
|
||||||
db.audit(conn, "settings", (current_user() or {}).get("username"),
|
(uid, scheduler.SLOT_PREFIX + "%"))
|
||||||
"修改:" + (",".join(sorted(changed)) or "(无变化)"), request.remote_addr)
|
db.audit(conn, "settings", u["username"],
|
||||||
|
"修改:" + (",".join(sorted(changed)) or "(无变化)"), request.remote_addr, uid)
|
||||||
return jsonify({"ok": True, "changed": sorted(changed), "ignored": sorted(ignored)})
|
return jsonify({"ok": True, "changed": sorted(changed), "ignored": sorted(ignored)})
|
||||||
|
|
||||||
|
|
||||||
@bp.post("/password")
|
@bp.post("/password")
|
||||||
@login_required
|
@login_required
|
||||||
def api_password():
|
def api_password():
|
||||||
from ..security import hash_password, verify_password
|
|
||||||
conn = db.get_db()
|
conn = db.get_db()
|
||||||
body = request.get_json(silent=True) or {}
|
|
||||||
u = current_user()
|
u = current_user()
|
||||||
|
body = _json_body()
|
||||||
row = conn.execute("SELECT * FROM users WHERE id=?", (u["id"],)).fetchone()
|
row = conn.execute("SELECT * FROM users WHERE id=?", (u["id"],)).fetchone()
|
||||||
if row is None or not verify_password(row["password_hash"], body.get("old") or ""):
|
if row is None or not security.verify_password(row["password_hash"], body.get("old") or ""):
|
||||||
return jsonify({"ok": False, "message": "原密码不正确"}), 400
|
return jsonify({"ok": False, "message": "原密码不正确"}), 400
|
||||||
new = (body.get("new") or "").strip()
|
new = (body.get("new") or "").strip()
|
||||||
err = _check_password(new, body.get("new2"))
|
err = security.password_problem(new, body.get("new2"), u["username"])
|
||||||
if err:
|
if err:
|
||||||
return jsonify({"ok": False, "message": err}), 400
|
return jsonify({"ok": False, "message": err}), 400
|
||||||
conn.execute("UPDATE users SET password_hash=? WHERE id=?", (hash_password(new), u["id"]))
|
conn.execute("UPDATE users SET password_hash=? WHERE id=?",
|
||||||
db.audit(conn, "password", u["username"], "修改登录密码", request.remote_addr)
|
(security.hash_password(new), u["id"]))
|
||||||
|
db.audit(conn, "password", u["username"], "修改登录密码", request.remote_addr, u["id"])
|
||||||
return jsonify({"ok": True, "message": "密码已更新"})
|
return jsonify({"ok": True, "message": "密码已更新"})
|
||||||
|
|
||||||
|
|
||||||
# ---------------- 用户管理(原来只有 CLI passwd) ----------------
|
@bp.post("/profile")
|
||||||
def _check_password(new, new2=None):
|
@login_required
|
||||||
if len(new or "") < 6:
|
def api_profile():
|
||||||
return "密码至少 6 位"
|
"""自助修改个人资料(显示名 / 邮箱)。用户名不可改 —— 它是审计里的主键。"""
|
||||||
if len(new) > 128:
|
conn = db.get_db()
|
||||||
return "密码过长(上限 128 位)"
|
u = current_user()
|
||||||
if new2 is not None and new2 != new:
|
body = _json_body()
|
||||||
return "两次输入的新密码不一致"
|
changed = []
|
||||||
return None
|
if "display_name" in body:
|
||||||
|
name = (body.get("display_name") or "").strip()[:64] or u["username"]
|
||||||
|
conn.execute("UPDATE users SET display_name=? WHERE id=?", (name, u["id"]))
|
||||||
|
changed.append("显示名")
|
||||||
|
if "email" in body:
|
||||||
|
email = (body.get("email") or "").strip()[:config.PROFILE_EMAIL_MAX]
|
||||||
|
if email and ("@" not in email or " " in email):
|
||||||
|
return jsonify({"ok": False, "message": "邮箱格式不正确"}), 400
|
||||||
|
conn.execute("UPDATE users SET email=? WHERE id=?", (email or None, u["id"]))
|
||||||
|
changed.append("邮箱")
|
||||||
|
if not changed:
|
||||||
|
return jsonify({"ok": False, "message": "没有要修改的内容"}), 400
|
||||||
|
db.audit(conn, "profile", u["username"], "修改:" + "、".join(changed),
|
||||||
|
request.remote_addr, u["id"])
|
||||||
|
return jsonify({"ok": True, "message": "已更新:" + "、".join(changed)})
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------- 用户管理(管理员) ----------------
|
||||||
|
def _user_public(r):
|
||||||
|
"""用户行 -> 可下发结构。**绝不包含口令散列,也不包含任何凭证。**"""
|
||||||
|
return {"id": r["id"], "username": r["username"], "display_name": r["display_name"],
|
||||||
|
"email": r["email"], "is_admin": bool(r["is_admin"]),
|
||||||
|
"status": r["status"] or "active", "created_at": r["created_at"],
|
||||||
|
"register_ip": r["register_ip"], "last_login_at": r["last_login_at"],
|
||||||
|
"last_login_ip": r["last_login_ip"], "login_count": r["login_count"]}
|
||||||
|
|
||||||
|
|
||||||
@bp.get("/users")
|
@bp.get("/users")
|
||||||
@admin_required
|
@admin_required
|
||||||
def api_users():
|
def api_users():
|
||||||
rows = db.get_db().execute(
|
rows = db.get_db().execute("SELECT * FROM users ORDER BY id").fetchall()
|
||||||
"SELECT id,username,display_name,is_admin,created_at,last_login_at,login_count"
|
return jsonify({"items": [_user_public(r) for r in rows]})
|
||||||
" FROM users ORDER BY id").fetchall()
|
|
||||||
return jsonify({"items": [dict(r) for r in rows]})
|
|
||||||
|
|
||||||
|
|
||||||
@bp.post("/users")
|
@bp.post("/users")
|
||||||
@admin_required
|
@admin_required
|
||||||
def api_user_create():
|
def api_user_create():
|
||||||
from ..security import hash_password
|
|
||||||
conn = db.get_db()
|
conn = db.get_db()
|
||||||
body = request.get_json(silent=True) or {}
|
me = current_user()
|
||||||
|
body = _json_body()
|
||||||
name = (body.get("username") or "").strip()
|
name = (body.get("username") or "").strip()
|
||||||
pwd = (body.get("password") or "").strip()
|
pwd = (body.get("password") or "").strip()
|
||||||
if not name or len(name) > 32:
|
err = security.username_problem(name) or security.password_problem(
|
||||||
return jsonify({"ok": False, "message": "用户名必填且不超过 32 字符"}), 400
|
pwd, body.get("password2"), name)
|
||||||
err = _check_password(pwd, body.get("password2"))
|
|
||||||
if err:
|
if err:
|
||||||
return jsonify({"ok": False, "message": err}), 400
|
return jsonify({"ok": False, "message": err}), 400
|
||||||
exist = conn.execute("SELECT id FROM users WHERE username=?", (name,)).fetchone()
|
if db.user_by_name(conn, name):
|
||||||
if exist:
|
|
||||||
return jsonify({"ok": False, "message": "用户名已存在"}), 400
|
return jsonify({"ok": False, "message": "用户名已存在"}), 400
|
||||||
conn.execute("INSERT INTO users(username,password_hash,display_name,is_admin,created_at)"
|
# 默认建**普通账号**:多用户系统里「默认给管理员」是最常见的越权起点
|
||||||
" VALUES(?,?,?,?,?)",
|
adm = 1 if str(body.get("is_admin", "0")) in ("1", "true", "on") else 0
|
||||||
(name, hash_password(pwd), (body.get("display_name") or name).strip()[:64],
|
cur = conn.execute(
|
||||||
1 if str(body.get("is_admin", "1")) in ("1", "true", "on") else 0,
|
"INSERT INTO users(username,password_hash,display_name,email,is_admin,status,created_at)"
|
||||||
db.now_str()))
|
" VALUES(?,?,?,?,?, 'active', ?)",
|
||||||
db.audit(conn, "user_create", (current_user() or {}).get("username"), "新建用户 " + name,
|
(name, security.hash_password(pwd),
|
||||||
request.remote_addr)
|
(body.get("display_name") or name).strip()[:64],
|
||||||
return jsonify({"ok": True, "message": "已创建用户 " + name})
|
(body.get("email") or "").strip()[:128] or None, adm, db.now_str()))
|
||||||
|
db.audit(conn, "user_create", me["username"],
|
||||||
|
"新建用户 %s(%s)" % (name, "管理员" if adm else "普通"), request.remote_addr, me["id"])
|
||||||
|
return jsonify({"ok": True, "message": "已创建用户 %s" % name, "id": cur.lastrowid})
|
||||||
|
|
||||||
|
|
||||||
@bp.post("/users/<int:uid>")
|
@bp.post("/users/<int:uid>")
|
||||||
@admin_required
|
@admin_required
|
||||||
def api_user_update(uid):
|
def api_user_update(uid):
|
||||||
from ..security import hash_password
|
|
||||||
conn = db.get_db()
|
conn = db.get_db()
|
||||||
row = conn.execute("SELECT * FROM users WHERE id=?", (uid,)).fetchone()
|
me = current_user()
|
||||||
|
row = db.user_by_id(conn, uid)
|
||||||
if row is None:
|
if row is None:
|
||||||
return jsonify({"ok": False, "message": "用户不存在"}), 404
|
return jsonify({"ok": False, "message": "用户不存在"}), 404
|
||||||
body = request.get_json(silent=True) or {}
|
body = _json_body()
|
||||||
me = current_user()
|
|
||||||
changed = []
|
changed = []
|
||||||
if "display_name" in body:
|
if "display_name" in body:
|
||||||
conn.execute("UPDATE users SET display_name=? WHERE id=?",
|
conn.execute("UPDATE users SET display_name=? WHERE id=?",
|
||||||
((body.get("display_name") or "").strip()[:64], uid))
|
((body.get("display_name") or "").strip()[:64], uid))
|
||||||
changed.append("显示名")
|
changed.append("显示名")
|
||||||
|
if "email" in body:
|
||||||
|
email = (body.get("email") or "").strip()[:config.PROFILE_EMAIL_MAX]
|
||||||
|
if email and ("@" not in email or " " in email):
|
||||||
|
return jsonify({"ok": False, "message": "邮箱格式不正确"}), 400
|
||||||
|
conn.execute("UPDATE users SET email=? WHERE id=?", (email or None, uid))
|
||||||
|
changed.append("邮箱")
|
||||||
if "is_admin" in body:
|
if "is_admin" in body:
|
||||||
v = 1 if str(body.get("is_admin")) in ("1", "true", "on") else 0
|
v = 1 if str(body.get("is_admin")) in ("1", "true", "on") else 0
|
||||||
if uid == me["id"] and not v:
|
if uid == me["id"] and not v:
|
||||||
return jsonify({"ok": False, "message": "不能取消自己的管理员身份"}), 400
|
return jsonify({"ok": False, "message": "不能取消自己的管理员身份"}), 400
|
||||||
|
if not v and row["is_admin"]:
|
||||||
|
left = conn.execute("SELECT COUNT(*) FROM users WHERE is_admin=1 AND status='active'"
|
||||||
|
" AND id<>?", (uid,)).fetchone()[0]
|
||||||
|
if left == 0:
|
||||||
|
return jsonify({"ok": False,
|
||||||
|
"message": "至少要保留一个启用状态的管理员"}), 400
|
||||||
conn.execute("UPDATE users SET is_admin=? WHERE id=?", (v, uid))
|
conn.execute("UPDATE users SET is_admin=? WHERE id=?", (v, uid))
|
||||||
changed.append("管理员")
|
changed.append("管理员")
|
||||||
|
if "status" in body:
|
||||||
|
v = "active" if str(body.get("status")) in ("active", "1", "true", "on") else "disabled"
|
||||||
|
if uid == me["id"] and v != "active":
|
||||||
|
return jsonify({"ok": False, "message": "不能停用自己的账号"}), 400
|
||||||
|
if v != "active":
|
||||||
|
left = conn.execute("SELECT COUNT(*) FROM users WHERE is_admin=1 AND status='active'"
|
||||||
|
" AND id<>?", (uid,)).fetchone()[0]
|
||||||
|
if row["is_admin"] and left == 0:
|
||||||
|
return jsonify({"ok": False,
|
||||||
|
"message": "至少要保留一个启用状态的管理员"}), 400
|
||||||
|
conn.execute("UPDATE users SET status=? WHERE id=?", (v, uid))
|
||||||
|
changed.append("状态→" + ("启用" if v == "active" else "停用"))
|
||||||
pwd = (body.get("password") or "").strip()
|
pwd = (body.get("password") or "").strip()
|
||||||
if pwd:
|
if pwd:
|
||||||
err = _check_password(pwd, body.get("password2"))
|
err = security.password_problem(pwd, body.get("password2"), row["username"])
|
||||||
if err:
|
if err:
|
||||||
return jsonify({"ok": False, "message": err}), 400
|
return jsonify({"ok": False, "message": err}), 400
|
||||||
conn.execute("UPDATE users SET password_hash=? WHERE id=?", (hash_password(pwd), uid))
|
conn.execute("UPDATE users SET password_hash=? WHERE id=?",
|
||||||
|
(security.hash_password(pwd), uid))
|
||||||
changed.append("密码")
|
changed.append("密码")
|
||||||
if not changed:
|
if not changed:
|
||||||
return jsonify({"ok": False, "message": "没有要修改的内容"}), 400
|
return jsonify({"ok": False, "message": "没有要修改的内容"}), 400
|
||||||
db.audit(conn, "user_update", me["username"],
|
db.audit(conn, "user_update", me["username"],
|
||||||
"修改用户 %s:%s" % (row["username"], "、".join(changed)), request.remote_addr)
|
"修改用户 %s:%s" % (row["username"], "、".join(changed)),
|
||||||
|
request.remote_addr, me["id"])
|
||||||
return jsonify({"ok": True, "message": "已更新:" + "、".join(changed)})
|
return jsonify({"ok": True, "message": "已更新:" + "、".join(changed)})
|
||||||
|
|
||||||
|
|
||||||
@@ -432,15 +554,41 @@ def api_user_update(uid):
|
|||||||
def api_user_delete(uid):
|
def api_user_delete(uid):
|
||||||
conn = db.get_db()
|
conn = db.get_db()
|
||||||
me = current_user()
|
me = current_user()
|
||||||
row = conn.execute("SELECT * FROM users WHERE id=?", (uid,)).fetchone()
|
row = db.user_by_id(conn, uid)
|
||||||
if row is None:
|
if row is None:
|
||||||
return jsonify({"ok": False, "message": "用户不存在"}), 404
|
return jsonify({"ok": False, "message": "用户不存在"}), 404
|
||||||
if uid == me["id"]:
|
if uid == me["id"]:
|
||||||
return jsonify({"ok": False, "message": "不能删除当前登录的自己"}), 400
|
return jsonify({"ok": False, "message": "不能删除当前登录的自己"}), 400
|
||||||
n = conn.execute("SELECT COUNT(*) FROM users").fetchone()[0]
|
if conn.execute("SELECT COUNT(*) FROM users").fetchone()[0] <= 1:
|
||||||
if n <= 1:
|
|
||||||
return jsonify({"ok": False, "message": "至少要保留一个账号"}), 400
|
return jsonify({"ok": False, "message": "至少要保留一个账号"}), 400
|
||||||
|
if row["is_admin"]:
|
||||||
|
left = conn.execute("SELECT COUNT(*) FROM users WHERE is_admin=1 AND status='active'"
|
||||||
|
" AND id<>?", (uid,)).fetchone()[0]
|
||||||
|
if left == 0:
|
||||||
|
return jsonify({"ok": False, "message": "至少要保留一个启用状态的管理员"}), 400
|
||||||
|
keep = str(_json_body().get("keep_data", "")).strip() in ("1", "true", "on", "yes")
|
||||||
|
if not keep:
|
||||||
|
# 默认连同数据一起删 —— 留下孤儿数据既占空间,也会在重新注册
|
||||||
|
# 同名用户时被新用户看到(历史遗留 user_id 复用风险)
|
||||||
|
conn.execute("DELETE FROM usage_records WHERE user_id=?", (uid,))
|
||||||
|
conn.execute("DELETE FROM settings WHERE user_id=?", (uid,))
|
||||||
|
conn.execute("DELETE FROM collect_runs WHERE user_id=?", (uid,))
|
||||||
conn.execute("DELETE FROM users WHERE id=?", (uid,))
|
conn.execute("DELETE FROM users WHERE id=?", (uid,))
|
||||||
db.audit(conn, "user_delete", me["username"], "删除用户 " + row["username"],
|
db.audit(conn, "user_delete", me["username"],
|
||||||
request.remote_addr)
|
"删除用户 %s(%s)" % (row["username"], "保留其数据" if keep else "连同数据一并删除"),
|
||||||
|
request.remote_addr, me["id"])
|
||||||
return jsonify({"ok": True, "message": "已删除 " + row["username"]})
|
return jsonify({"ok": True, "message": "已删除 " + row["username"]})
|
||||||
|
|
||||||
|
|
||||||
|
@bp.post("/captcha")
|
||||||
|
@login_required
|
||||||
|
def api_captcha_note():
|
||||||
|
"""给前端一个「验证码怎么工作」的自述,便于排障时自检。"""
|
||||||
|
conn = db.get_db()
|
||||||
|
return jsonify({
|
||||||
|
"policy": db.get_setting(conn, "captcha_policy", "always"),
|
||||||
|
"length": db.get_int(conn, "captcha_length", 4),
|
||||||
|
"ttl_seconds": 300,
|
||||||
|
"image_url": "/captcha.png",
|
||||||
|
"note": "答案只存在服务端 captchas 表;一次性使用,校验后立即删除。",
|
||||||
|
})
|
||||||
|
|||||||
@@ -115,6 +115,9 @@ code {
|
|||||||
.topbar .me { margin-left: auto; display: flex; align-items: center; gap: 10px; flex: 0 0 auto; }
|
.topbar .me { margin-left: auto; display: flex; align-items: center; gap: 10px; flex: 0 0 auto; }
|
||||||
.topbar .who { color: var(--sub); font-size: 12.5px; }
|
.topbar .who { color: var(--sub); font-size: 12.5px; }
|
||||||
.topbar .who b { color: var(--text); font-weight: 600; }
|
.topbar .who b { color: var(--text); font-weight: 600; }
|
||||||
|
/* 用户名同时是「个人中心」入口,所以是 <a>:去掉下划线并给悬浮反馈 */
|
||||||
|
.topbar .who { display: flex; align-items: center; gap: 6px; text-decoration: none; }
|
||||||
|
.topbar .who:hover, .topbar .who:hover b { color: var(--cyan); }
|
||||||
|
|
||||||
/* ---------------- 布局 ---------------- */
|
/* ---------------- 布局 ---------------- */
|
||||||
.wrap { max-width: 1480px; margin: 0 auto; padding: 20px 22px 60px; }
|
.wrap { max-width: 1480px; margin: 0 auto; padding: 20px 22px 60px; }
|
||||||
@@ -379,6 +382,23 @@ select option { background: #111a2e; color: var(--text); }
|
|||||||
margin: 16px 0 0; padding-top: 14px; border-top: 1px solid var(--line);
|
margin: 16px 0 0; padding-top: 14px; border-top: 1px solid var(--line);
|
||||||
color: var(--dim); font-size: 11.5px; line-height: 1.7;
|
color: var(--dim); font-size: 11.5px; line-height: 1.7;
|
||||||
}
|
}
|
||||||
|
/* 注册页字段比登录页多,卡片给宽一点(沿用 .login 的纵向节奏) */
|
||||||
|
.login.wide { max-width: 470px; }
|
||||||
|
/* 字段说明(如「3~32 位,字母或数字开头」)跟在 label 文字后面 */
|
||||||
|
.login label em.unit { font-style: normal; margin-left: 6px; font-size: 11px; color: var(--dim); }
|
||||||
|
|
||||||
|
/* 验证码:输入框与图片并排一行(.caprow 是**新组件独有**前缀,
|
||||||
|
刻意不叫 .bar —— 页面里 .bar 是筛选条,带着 backdrop-filter 与 margin,
|
||||||
|
撞名会导致整块文字被虚化且被顶高) */
|
||||||
|
.caprow { display: flex; align-items: center; gap: 10px; }
|
||||||
|
.caprow input { flex: 1 1 auto; min-width: 0; letter-spacing: 2px; }
|
||||||
|
/* 图片按 PNG 原始高度显示(150x56,scale=5),不缩放才最清晰 */
|
||||||
|
.capimg {
|
||||||
|
flex: 0 0 auto; height: 56px; width: auto; display: block;
|
||||||
|
border: 1px solid var(--line); border-radius: var(--r-ctl);
|
||||||
|
background: var(--panel-dim); cursor: pointer; transition: .15s;
|
||||||
|
}
|
||||||
|
.capimg:hover { border-color: var(--cyan); }
|
||||||
|
|
||||||
/* ---------------- 错误页 ---------------- */
|
/* ---------------- 错误页 ---------------- */
|
||||||
.errpage { text-align: center; padding: 66px 24px; }
|
.errpage { text-align: center; padding: 66px 24px; }
|
||||||
|
|||||||
@@ -4,6 +4,7 @@
|
|||||||
<meta charset="utf-8">
|
<meta charset="utf-8">
|
||||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||||
<meta name="color-scheme" content="dark">
|
<meta name="color-scheme" content="dark">
|
||||||
|
<meta name="robots" content="noindex, nofollow">
|
||||||
<title>{% block title %}{{ project_title }}{% endblock %}</title>
|
<title>{% block title %}{{ project_title }}{% endblock %}</title>
|
||||||
<link rel="icon" href="{{ url_for('static', filename='favicon.svg') }}">
|
<link rel="icon" href="{{ url_for('static', filename='favicon.svg') }}">
|
||||||
<link rel="stylesheet" href="{{ url_for('static', filename='css/app.css') }}">
|
<link rel="stylesheet" href="{{ url_for('static', filename='css/app.css') }}">
|
||||||
@@ -11,7 +12,11 @@
|
|||||||
<body>
|
<body>
|
||||||
{% set nav = active|default('') %}
|
{% set nav = active|default('') %}
|
||||||
{% set on_dash = request.path.startswith('/dashboard') %}
|
{% set on_dash = request.path.startswith('/dashboard') %}
|
||||||
{% if current_user() %}
|
{# 变量名刻意叫 cur 而不是 me:模板里的 {% set %} 会覆盖子模板传入的同名变量,
|
||||||
|
而 current_user() 只含 id/username/display_name/is_admin —— 曾因此让
|
||||||
|
个人中心把 me.created_at 渲染成空(子模板的 me 是完整的用户行)。 #}
|
||||||
|
{% set cur = current_user() %}
|
||||||
|
{% if cur %}
|
||||||
<header class="topbar">
|
<header class="topbar">
|
||||||
<div class="brand">
|
<div class="brand">
|
||||||
<span class="dot"></span>
|
<span class="dot"></span>
|
||||||
@@ -25,12 +30,15 @@
|
|||||||
<a href="{{ url_for('views.tasks') }}" class="{{ 'on' if nav=='tasks' }}">任务管理</a>
|
<a href="{{ url_for('views.tasks') }}" class="{{ 'on' if nav=='tasks' }}">任务管理</a>
|
||||||
<a href="{{ url_for('views.config_page') }}" class="{{ 'on' if nav=='config' }}">配置管理</a>
|
<a href="{{ url_for('views.config_page') }}" class="{{ 'on' if nav=='config' }}">配置管理</a>
|
||||||
<a href="{{ url_for('views.logs') }}" class="{{ 'on' if nav=='logs' }}">日志管理</a>
|
<a href="{{ url_for('views.logs') }}" class="{{ 'on' if nav=='logs' }}">日志管理</a>
|
||||||
{% if current_user().is_admin %}
|
{% if cur.is_admin %}
|
||||||
<a href="{{ url_for('views.users_page') }}" class="{{ 'on' if nav=='users' }}">用户管理</a>
|
<a href="{{ url_for('views.users_page') }}" class="{{ 'on' if nav=='users' }}">用户管理</a>
|
||||||
{% endif %}
|
{% endif %}
|
||||||
</nav>
|
</nav>
|
||||||
<div class="me">
|
<div class="me">
|
||||||
<span class="who"><b>{{ current_user().display_name }}</b>{% if current_user().is_admin %} <span class="tag accent">管理员</span>{% endif %}</span>
|
<a class="who" href="{{ url_for('views.profile_page') }}" title="个人中心">
|
||||||
|
<b>{{ cur.display_name }}</b>
|
||||||
|
{% if cur.is_admin %}<span class="tag accent">管理员</span>{% endif %}
|
||||||
|
</a>
|
||||||
{# 退出用 POST + CSRF:GET 型退出会被 <img src="/logout"> 这类请求静默触发 #}
|
{# 退出用 POST + CSRF:GET 型退出会被 <img src="/logout"> 这类请求静默触发 #}
|
||||||
<form method="post" action="{{ url_for('views.logout_post') }}" style="margin:0">
|
<form method="post" action="{{ url_for('views.logout_post') }}" style="margin:0">
|
||||||
<input type="hidden" name="_csrf" value="{{ csrf_token() }}">
|
<input type="hidden" name="_csrf" value="{{ csrf_token() }}">
|
||||||
@@ -52,8 +60,8 @@
|
|||||||
</main>
|
</main>
|
||||||
|
|
||||||
<footer class="foot">
|
<footer class="foot">
|
||||||
<b>{{ project_title }}</b> · {{ project_name }} · 采集 / 存储 / 呈现三合一 · 数据正本 <code>data/usage.sqlite</code><br>
|
<b>{{ project_title }}</b> · {{ project_name }} · 多用户 · 采集在 Web 进程内按各账号配置的时刻执行<br>
|
||||||
采集在 Web 进程内按配置时刻执行,无需外部计划任务
|
每个账号只使用并只见得到自己的 Cookie 与用量数据;数据正本 <code>data/usage.sqlite</code>
|
||||||
</footer>
|
</footer>
|
||||||
|
|
||||||
<script>
|
<script>
|
||||||
|
|||||||
@@ -5,24 +5,35 @@
|
|||||||
<div class="pagehead">
|
<div class="pagehead">
|
||||||
<div>
|
<div>
|
||||||
<h1>配置管理</h1>
|
<h1>配置管理</h1>
|
||||||
<p class="lead">凭证、采集参数、维护动作都在这里;所有配置存在数据库,改完立即生效</p>
|
<p class="lead">这里改的都是<b>你自己账号</b>的配置:凭证、采集参数、维护动作;改完立即生效</p>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
|
{% if s.cookie_broken %}
|
||||||
|
<div class="flash error" style="margin-bottom:16px">
|
||||||
|
已保存的 Cookie <b>无法解密</b>(通常是 <code>data/instance.json</code> 里的
|
||||||
|
<code>cookie_key</code> 被更换或文件丢失)。请重新粘贴一次;在此之前该账号的采集会失败。
|
||||||
|
</div>
|
||||||
|
{% endif %}
|
||||||
|
|
||||||
<section class="card">
|
<section class="card">
|
||||||
<div class="cardhead">
|
<div class="cardhead">
|
||||||
<h2>云端凭证</h2>
|
<h2>我的云端凭证</h2>
|
||||||
<span class="tag {{ 'ok' if s.cookie_hint else 'bad' }}">{{ '已配置' if s.cookie_hint else '未配置' }}</span>
|
<span class="tag {{ 'ok' if s.cookie_hint else 'bad' }}">{{ '已配置' if s.cookie_hint else '未配置' }}</span>
|
||||||
</div>
|
</div>
|
||||||
<p class="hint">
|
<p class="hint">
|
||||||
{% if s.cookie_hint %}当前 Cookie:{{ s.cookie_hint }}(页面与接口都不回传明文){% endif %}
|
{% if s.cookie_hint %}当前 Cookie:{{ s.cookie_hint }}
|
||||||
|
{% if s.cookie_at %}({{ s.cookie_at }} 更新){% endif %}
|
||||||
|
—— 页面与接口都不回传明文,数据库里也是密文。{% endif %}
|
||||||
<br>获取方式:Chrome 打开 <code>https://www.workbuddy.cn/profile/plans-usage</code> → F12 → Network →
|
<br>获取方式:Chrome 打开 <code>https://www.workbuddy.cn/profile/plans-usage</code> → F12 → Network →
|
||||||
任选一个 <code>billing</code> 请求 → 复制 Request Headers 里的 <code>cookie</code> 与 <code>user-agent</code>
|
任选一个 <code>billing</code> 请求 → 复制 Request Headers 里的 <code>cookie</code> 与 <code>user-agent</code>
|
||||||
(<b>两者必须取自同一次请求</b>),粘贴到下面。
|
(<b>两者必须取自同一次请求</b>),粘贴到下面。
|
||||||
|
<br><b>请粘贴你自己账号的 Cookie</b>:采集只使用本人凭证,各账号的数据互不可见。
|
||||||
</p>
|
</p>
|
||||||
<form id="formCred">
|
<form id="formCred">
|
||||||
<label class="col">Cookie
|
<label class="col">Cookie
|
||||||
<textarea name="cookie" rows="4" placeholder="留空表示不修改;填 - 表示清空已保存的 Cookie" spellcheck="false"></textarea>
|
<textarea name="cookie" rows="4" autocomplete="off" spellcheck="false"
|
||||||
|
placeholder="留空表示不修改;填 - 表示清空已保存的 Cookie"></textarea>
|
||||||
</label>
|
</label>
|
||||||
<label class="col">User-Agent
|
<label class="col">User-Agent
|
||||||
<textarea name="user_agent" rows="2" spellcheck="false">{{ s.user_agent }}</textarea>
|
<textarea name="user_agent" rows="2" spellcheck="false">{{ s.user_agent }}</textarea>
|
||||||
@@ -35,8 +46,15 @@
|
|||||||
<section class="card">
|
<section class="card">
|
||||||
<h2>采集参数</h2>
|
<h2>采集参数</h2>
|
||||||
<form id="formCollect">
|
<form id="formCollect">
|
||||||
<label class="row"><span>接口基址</span><input name="api_base" value="{{ s.api_base }}" spellcheck="false"></label>
|
{# 实例级键:所有账号共用,只有管理员能改;普通账号只读展示 #}
|
||||||
<label class="row"><span>接口路径</span><input name="api_path" value="{{ s.api_path }}" spellcheck="false"></label>
|
<label class="row"><span>接口基址</span>
|
||||||
|
<input name="api_base" value="{{ s.api_base }}" spellcheck="false"
|
||||||
|
{{ '' if is_admin else 'disabled' }}>
|
||||||
|
{% if not is_admin %}<em class="unit">实例级,仅管理员可改</em>{% endif %}</label>
|
||||||
|
<label class="row"><span>接口路径</span>
|
||||||
|
<input name="api_path" value="{{ s.api_path }}" spellcheck="false"
|
||||||
|
{{ '' if is_admin else 'disabled' }}>
|
||||||
|
{% if not is_admin %}<em class="unit">实例级</em>{% endif %}</label>
|
||||||
{% set b = num_settings %}
|
{% set b = num_settings %}
|
||||||
<label class="row"><span>分页大小</span>
|
<label class="row"><span>分页大小</span>
|
||||||
<input name="page_size" type="number" min="{{ b.page_size[0] }}" max="{{ b.page_size[1] }}" value="{{ s.page_size }}">
|
<input name="page_size" type="number" min="{{ b.page_size[0] }}" max="{{ b.page_size[1] }}" value="{{ s.page_size }}">
|
||||||
@@ -75,7 +93,8 @@
|
|||||||
<label class="col">新密码<input name="new" type="password" autocomplete="new-password"></label>
|
<label class="col">新密码<input name="new" type="password" autocomplete="new-password"></label>
|
||||||
<label class="col">确认新密码<input name="new2" type="password" autocomplete="new-password"></label>
|
<label class="col">确认新密码<input name="new2" type="password" autocomplete="new-password"></label>
|
||||||
<button class="btn primary" type="submit">修改密码</button>
|
<button class="btn primary" type="submit">修改密码</button>
|
||||||
<p class="hint">至少 6 位。修改成功后当前会话仍有效,不必重新登录。</p>
|
<p class="hint">至少 {{ pwd_min }} 位,且需包含大写字母、小写字母、数字、符号中的至少两类。
|
||||||
|
修改成功后当前会话仍有效,不必重新登录。</p>
|
||||||
</form>
|
</form>
|
||||||
|
|
||||||
<hr class="sect-divider">
|
<hr class="sect-divider">
|
||||||
@@ -84,18 +103,62 @@
|
|||||||
<button class="btn" type="button" data-maint="fill-prompt"
|
<button class="btn" type="button" data-maint="fill-prompt"
|
||||||
title="把云端仍保留、但本地为空的 User Prompt 补回来">补全缺失 Prompt</button>
|
title="把云端仍保留、但本地为空的 User Prompt 补回来">补全缺失 Prompt</button>
|
||||||
<button class="btn" type="button" data-maint="export-csv"
|
<button class="btn" type="button" data-maint="export-csv"
|
||||||
title="导出与官网 xlsx 同构的全量 CSV 到 data/exports/">导出全量 CSV</button>
|
title="导出与官网 xlsx 同构的 CSV 到 data/exports/">导出我的 CSV</button>
|
||||||
|
{% if is_admin %}
|
||||||
<button class="btn" type="button" data-maint="vacuum"
|
<button class="btn" type="button" data-maint="vacuum"
|
||||||
title="checkpoint + VACUUM,回收删除后的空闲页">整理数据库</button>
|
title="checkpoint + VACUUM,回收删除后的空闲页(整库操作,仅管理员)">整理数据库</button>
|
||||||
|
{% endif %}
|
||||||
</div>
|
</div>
|
||||||
<p class="hint">补全 Prompt 需要联网并逐天重拉云端;导出与整理只动本地数据。</p>
|
<p class="hint">补全 Prompt 需要联网并逐天重拉云端;导出与整理只动本地数据。</p>
|
||||||
<div class="btnrow" style="margin-top:14px">
|
<div class="btnrow" style="margin-top:14px">
|
||||||
<a class="btn ghost" href="{{ url_for('views.records_export') }}">按当前明细页筛选导出</a>
|
<a class="btn ghost" href="{{ url_for('views.records_export') }}">按当前明细页筛选导出</a>
|
||||||
{% if current_user().is_admin %}<a class="btn ghost" href="{{ url_for('views.users_page') }}">用户管理</a>{% endif %}
|
<a class="btn ghost" href="{{ url_for('views.profile_page') }}">个人中心</a>
|
||||||
|
{% if is_admin %}<a class="btn ghost" href="{{ url_for('views.users_page') }}">用户管理</a>{% endif %}
|
||||||
</div>
|
</div>
|
||||||
</section>
|
</section>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
|
{% if is_admin %}
|
||||||
|
<section class="card">
|
||||||
|
<div class="cardhead">
|
||||||
|
<h2>实例级设置</h2>
|
||||||
|
<span class="tag accent">仅管理员可改,对所有账号生效</span>
|
||||||
|
</div>
|
||||||
|
<form id="formGlobal">
|
||||||
|
{% set b = num_settings %}
|
||||||
|
<label class="row"><span>开放自助注册</span>
|
||||||
|
<select name="allow_register">
|
||||||
|
<option value="1" {{ 'selected' if s.allow_register != '0' }}>允许任何人注册</option>
|
||||||
|
<option value="0" {{ 'selected' if s.allow_register == '0' }}>关闭注册(只能由管理员建号)</option>
|
||||||
|
</select>
|
||||||
|
</label>
|
||||||
|
<label class="row"><span>同 IP 每日注册上限</span>
|
||||||
|
<input name="register_max_per_ip" type="number"
|
||||||
|
min="{{ b.register_max_per_ip[0] }}" max="{{ b.register_max_per_ip[1] }}"
|
||||||
|
value="{{ s.register_max_per_ip }}">
|
||||||
|
<em class="unit">{{ b.register_max_per_ip[0] }}~{{ b.register_max_per_ip[1] }} 个/天</em></label>
|
||||||
|
<label class="row"><span>验证码策略</span>
|
||||||
|
<select name="captcha_policy">
|
||||||
|
<option value="always" {{ 'selected' if s.captcha_policy == 'always' }}>始终要求(推荐)</option>
|
||||||
|
<option value="adaptive" {{ 'selected' if s.captcha_policy == 'adaptive' }}>仅连续失败后要求(对日常更友好)</option>
|
||||||
|
<option value="off" {{ 'selected' if s.captcha_policy == 'off' }}>关闭(不推荐)</option>
|
||||||
|
</select>
|
||||||
|
</label>
|
||||||
|
<label class="row"><span>验证码位数</span>
|
||||||
|
<input name="captcha_length" type="number"
|
||||||
|
min="{{ b.captcha_length[0] }}" max="{{ b.captcha_length[1] }}"
|
||||||
|
value="{{ s.captcha_length }}">
|
||||||
|
<em class="unit">4~6 位</em></label>
|
||||||
|
<button class="btn primary" type="submit">保存实例设置</button>
|
||||||
|
<p class="hint">
|
||||||
|
验证码的答案只存在服务端 <code>captchas</code> 表里(下发到浏览器的只是一个随机 id),
|
||||||
|
一次性使用、5 分钟过期 —— 所以答案不会随会话 Cookie 泄漏出去。
|
||||||
|
关闭验证码会显著放大被撞库与批量注册的风险,只有在前面已经有可信网关时才考虑。
|
||||||
|
</p>
|
||||||
|
</form>
|
||||||
|
</section>
|
||||||
|
{% endif %}
|
||||||
|
|
||||||
{% endblock %}
|
{% endblock %}
|
||||||
|
|
||||||
{% block scripts %}
|
{% block scripts %}
|
||||||
@@ -104,6 +167,8 @@
|
|||||||
WBU.bindForm('#formCred', '/api/settings');
|
WBU.bindForm('#formCred', '/api/settings');
|
||||||
WBU.bindForm('#formCollect', '/api/settings');
|
WBU.bindForm('#formCollect', '/api/settings');
|
||||||
WBU.bindForm('#formPwd', '/api/password', {validate: d => d.new === d.new2 ? null : '两次输入的新密码不一致'});
|
WBU.bindForm('#formPwd', '/api/password', {validate: d => d.new === d.new2 ? null : '两次输入的新密码不一致'});
|
||||||
|
var fg = document.querySelector('#formGlobal');
|
||||||
|
if (fg) WBU.bindForm('#formGlobal', '/api/settings');
|
||||||
WBU.bindMaint('[data-maint]');
|
WBU.bindMaint('[data-maint]');
|
||||||
</script>
|
</script>
|
||||||
{% endblock %}
|
{% endblock %}
|
||||||
|
|||||||
@@ -15,9 +15,24 @@
|
|||||||
<label>密码
|
<label>密码
|
||||||
<input name="password" type="password" autocomplete="current-password" required>
|
<input name="password" type="password" autocomplete="current-password" required>
|
||||||
</label>
|
</label>
|
||||||
|
{% if need_captcha %}
|
||||||
|
<label>验证码
|
||||||
|
<span class="caprow">
|
||||||
|
<input name="captcha" maxlength="6" autocomplete="off" spellcheck="false"
|
||||||
|
required placeholder="不区分大小写">
|
||||||
|
{# 点击换一张:URL 带时间戳,避免浏览器复用已被消费的旧图 #}
|
||||||
|
<img class="capimg" alt="图形验证码" title="看不清?点一下换一张"
|
||||||
|
src="{{ url_for('views.captcha_png', purpose='login') }}&t={{ range(1000000)|random }}"
|
||||||
|
onclick="this.src='{{ url_for('views.captcha_png', purpose='login') }}&t=' + Date.now();">
|
||||||
|
</span>
|
||||||
|
</label>
|
||||||
|
{% endif %}
|
||||||
<button class="btn primary" type="submit">登 录</button>
|
<button class="btn primary" type="submit">登 录</button>
|
||||||
|
{% if allow_register %}
|
||||||
|
<p class="foot-note">还没有账号?<a href="{{ url_for('views.register') }}">自助注册</a></p>
|
||||||
|
{% endif %}
|
||||||
<p class="foot-note">
|
<p class="foot-note">
|
||||||
首次部署默认账号 <code>admin</code> / <code>admin123</code>,登录后请立即到「配置管理」修改密码。<br>
|
首次部署默认账号 <code>admin</code> / <code>admin123</code>,登录后请立即修改密码。<br>
|
||||||
连续输错 {{ max_fails }} 次将锁定 {{ lock_minutes }} 分钟;登录状态保持 {{ session_hours }} 小时。
|
连续输错 {{ max_fails }} 次将锁定 {{ lock_minutes }} 分钟;登录状态保持 {{ session_hours }} 小时。
|
||||||
</p>
|
</p>
|
||||||
</form>
|
</form>
|
||||||
|
|||||||
@@ -0,0 +1,103 @@
|
|||||||
|
{% extends "base.html" %}
|
||||||
|
{% block title %}个人中心 · {{ project_title }}{% endblock %}
|
||||||
|
{% block body %}
|
||||||
|
|
||||||
|
<div class="pagehead">
|
||||||
|
<div>
|
||||||
|
<h1>个人中心</h1>
|
||||||
|
<p class="lead">账号 <b>{{ me.username }}</b> · 注册于 {{ (me.created_at or '')[:16] }} ·
|
||||||
|
最近登录 {{ (me.last_login_at or '未登录')[:19] }}</p>
|
||||||
|
</div>
|
||||||
|
<div class="actions">
|
||||||
|
<a class="btn" href="{{ url_for('views.config_page') }}">管理我的凭证</a>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="kpis">
|
||||||
|
<div class="kpi" style="--c:var(--cyan)">
|
||||||
|
<span>我的记录</span><b>{{ '{:,}'.format(my.n or 0) }}</b>
|
||||||
|
<i>{{ my.d0 or '—' }} ~ {{ my.d1 or '—' }}</i>
|
||||||
|
</div>
|
||||||
|
<div class="kpi" style="--c:var(--violet)">
|
||||||
|
<span>我的积分</span><b>{{ '%.2f'|format(my.c or 0) }}</b>
|
||||||
|
<i>仅统计归属本账号的数据</i>
|
||||||
|
</div>
|
||||||
|
<div class="kpi" style="--c:var(--blue)">
|
||||||
|
<span>采集次数</span><b>{{ '{:,}'.format(runs) }}</b>
|
||||||
|
<i>{% if sch.last %}最近 {{ sch.last.started_at[5:16] if sch.last.started_at else '—' }}{% else %}尚无采集{% endif %}</i>
|
||||||
|
</div>
|
||||||
|
<div class="kpi" style="--c:{{ 'var(--green)' if cred.set and not cred.broken else 'var(--red)' }}">
|
||||||
|
<span>我的 Cookie</span>
|
||||||
|
<b>{% if cred.broken %}无法解密{% elif cred.set %}已配置{% else %}未配置{% endif %}</b>
|
||||||
|
<i>{% if cred.set and not cred.broken %}{{ cred.chars }} 字符,结尾 …{{ cred.tail }}
|
||||||
|
{%- elif cred.broken %}实例密钥被更换,请重新粘贴
|
||||||
|
{%- else %}采集需要本人凭证{% endif %}</i>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="grid2">
|
||||||
|
<section class="card">
|
||||||
|
<h2>修改资料</h2>
|
||||||
|
<form id="formProfile">
|
||||||
|
<label class="row"><span>用户名</span>
|
||||||
|
<input value="{{ me.username }}" disabled spellcheck="false">
|
||||||
|
<em class="unit">登录名不可改</em></label>
|
||||||
|
<label class="row"><span>显示名</span>
|
||||||
|
<input name="display_name" maxlength="64" value="{{ me.display_name or '' }}" spellcheck="false"></label>
|
||||||
|
<label class="row"><span>邮箱</span>
|
||||||
|
<input name="email" type="email" maxlength="128" value="{{ me.email or '' }}" spellcheck="false"></label>
|
||||||
|
<button class="btn primary" type="submit">保存资料</button>
|
||||||
|
</form>
|
||||||
|
</section>
|
||||||
|
|
||||||
|
<section class="card">
|
||||||
|
<h2>修改登录密码</h2>
|
||||||
|
<form id="formPwd">
|
||||||
|
<label class="col">原密码<input name="old" type="password" autocomplete="current-password"></label>
|
||||||
|
<label class="col">新密码<input name="new" type="password" autocomplete="new-password"></label>
|
||||||
|
<label class="col">确认新密码<input name="new2" type="password" autocomplete="new-password"></label>
|
||||||
|
<button class="btn primary" type="submit">修改密码</button>
|
||||||
|
<p class="hint">至少 {{ pwd_min }} 位,且需包含大写字母、小写字母、数字、符号中的至少两类。
|
||||||
|
修改成功后当前会话仍有效,不必重新登录。</p>
|
||||||
|
</form>
|
||||||
|
</section>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<section class="card">
|
||||||
|
<div class="cardhead">
|
||||||
|
<h2>我的采集凭证</h2>
|
||||||
|
<span class="tag {{ 'ok' if cred.set and not cred.broken else ('bad' if not cred.set else 'warn') }}">
|
||||||
|
{{ '正常' if cred.set and not cred.broken else ('未配置' if not cred.set else '需要重配') }}</span>
|
||||||
|
</div>
|
||||||
|
<table class="kv">
|
||||||
|
<tr><th>状态</th><td>
|
||||||
|
{% if cred.broken %}<span class="tag bad">已保存但无法解密</span>,请到「配置管理」重新粘贴
|
||||||
|
{% elif cred.set %}<span class="tag ok">已保存(密文入库)</span>
|
||||||
|
{% else %}<span class="tag bad">未配置</span>{% endif %}
|
||||||
|
</td></tr>
|
||||||
|
<tr><th>字符数 / 尾部</th><td class="mono">{{ cred.chars or 0 }} / {{ ('…' + cred.tail) if cred.tail else '—' }}</td></tr>
|
||||||
|
<tr><th>最后更新</th><td class="mono">{{ cred.at or '—' }}</td></tr>
|
||||||
|
<tr><th>调度</th><td>
|
||||||
|
{% if sch.enabled %}{{ sch.times | join(' · ') or '未设置时刻' }}{% else %}<span class="tag bad">已停用</span>{% endif %}
|
||||||
|
{% if sch.next_run %}· 下次 <span class="mono">{{ sch.next_run }}</span>{% endif %}
|
||||||
|
</td></tr>
|
||||||
|
</table>
|
||||||
|
<p class="hint">
|
||||||
|
凭证以密文形式存在数据库里(主密钥在 <code>data/instance.json</code>),
|
||||||
|
页面与接口**任何时候都不回传明文**,只显示长度与尾部 4 位。要更换请到
|
||||||
|
<a href="{{ url_for('views.config_page') }}">配置管理</a>粘贴新的 Cookie 与 User-Agent
|
||||||
|
(两者必须取自同一次浏览器请求)。
|
||||||
|
</p>
|
||||||
|
</section>
|
||||||
|
|
||||||
|
{% endblock %}
|
||||||
|
|
||||||
|
{% block scripts %}
|
||||||
|
<script src="{{ url_for('static', filename='js/app.js') }}"></script>
|
||||||
|
<script>
|
||||||
|
WBU.bindForm('#formProfile', '/api/profile');
|
||||||
|
WBU.bindForm('#formPwd', '/api/password', {
|
||||||
|
validate: function (d) { return d.new === d.new2 ? null : '两次输入的新密码不一致'; }
|
||||||
|
});
|
||||||
|
</script>
|
||||||
|
{% endblock %}
|
||||||
@@ -0,0 +1,46 @@
|
|||||||
|
{% extends "base.html" %}
|
||||||
|
{% block title %}注册 · {{ project_title }}{% endblock %}
|
||||||
|
{% block body %}
|
||||||
|
<div class="loginwrap">
|
||||||
|
<form class="card login wide" method="post" action="{{ url_for('views.register') }}">
|
||||||
|
<div class="logo">W</div>
|
||||||
|
<h1>注册 {{ project_title }}</h1>
|
||||||
|
<p class="hint">注册后请粘贴<strong>你自己账号</strong>的 Cookie —— 采集只使用本人的凭证,各账号数据互相隔离。</p>
|
||||||
|
<input type="hidden" name="_csrf" value="{{ csrf_token() }}">
|
||||||
|
|
||||||
|
<label>用户名 <em class="unit">3~32 位,字母或数字开头</em>
|
||||||
|
<input name="username" value="{{ username or '' }}" autocomplete="username"
|
||||||
|
autofocus required maxlength="32" spellcheck="false">
|
||||||
|
</label>
|
||||||
|
<label>显示名 <em class="unit">留空则与用户名相同</em>
|
||||||
|
<input name="display_name" value="{{ display_name or '' }}" maxlength="64" autocomplete="nickname">
|
||||||
|
</label>
|
||||||
|
<label>邮箱 <em class="unit">选填,便于日后找回</em>
|
||||||
|
<input name="email" type="email" value="{{ email or '' }}" maxlength="128" autocomplete="email">
|
||||||
|
</label>
|
||||||
|
<label>密码 <em class="unit">至少 {{ pwd_min }} 位,需含两类以上字符</em>
|
||||||
|
<input name="password" type="password" autocomplete="new-password" required>
|
||||||
|
</label>
|
||||||
|
<label>确认密码
|
||||||
|
<input name="password2" type="password" autocomplete="new-password" required>
|
||||||
|
</label>
|
||||||
|
{% if need_captcha %}
|
||||||
|
<label>验证码
|
||||||
|
<span class="caprow">
|
||||||
|
<input name="captcha" maxlength="6" autocomplete="off" spellcheck="false"
|
||||||
|
required placeholder="不区分大小写">
|
||||||
|
<img class="capimg" alt="图形验证码" title="看不清?点一下换一张"
|
||||||
|
src="{{ url_for('views.captcha_png', purpose='register') }}&t={{ range(1000000)|random }}"
|
||||||
|
onclick="this.src='{{ url_for('views.captcha_png', purpose='register') }}&t=' + Date.now();">
|
||||||
|
</span>
|
||||||
|
</label>
|
||||||
|
{% endif %}
|
||||||
|
<button class="btn primary" type="submit">注 册</button>
|
||||||
|
<p class="foot-note">已有账号?<a href="{{ url_for('views.login') }}">返回登录</a></p>
|
||||||
|
<p class="foot-note">
|
||||||
|
注册受来源限额与图形验证码双重保护;同一来源每天可注册的账号数由管理员设定。<br>
|
||||||
|
连续输错 {{ max_fails }} 次将锁定 {{ lock_minutes }} 分钟。
|
||||||
|
</p>
|
||||||
|
</form>
|
||||||
|
</div>
|
||||||
|
{% endblock %}
|
||||||
@@ -5,10 +5,12 @@
|
|||||||
<div class="pagehead">
|
<div class="pagehead">
|
||||||
<div>
|
<div>
|
||||||
<h1>用户管理</h1>
|
<h1>用户管理</h1>
|
||||||
<p class="lead">门户在局域网可访问,因此必须靠账号隔离;这里维护账号、管理员身份与密码</p>
|
<p class="lead">维护账号、状态与管理员身份。单价数据<b>按账号隔离</b>——管理员也看不到别人的用量与凭证</p>
|
||||||
</div>
|
</div>
|
||||||
<div class="actions">
|
<div class="actions">
|
||||||
<span class="tag accent">仅管理员可见</span>
|
<span class="tag accent">仅管理员可见</span>
|
||||||
|
<span class="tag {{ 'ok' if allow_register else 'mute' }}">
|
||||||
|
自助注册:{{ '已开放' if allow_register else '已关闭' }}</span>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
@@ -17,21 +19,25 @@
|
|||||||
<h2>新建账号</h2>
|
<h2>新建账号</h2>
|
||||||
<form id="formNewUser">
|
<form id="formNewUser">
|
||||||
<label class="row"><span>用户名</span>
|
<label class="row"><span>用户名</span>
|
||||||
<input name="username" maxlength="32" placeholder="登录名(≤32 字符)" autocomplete="off" spellcheck="false"></label>
|
<input name="username" maxlength="32" placeholder="3~32 位,字母或数字开头"
|
||||||
|
autocomplete="off" spellcheck="false"></label>
|
||||||
<label class="row"><span>显示名</span>
|
<label class="row"><span>显示名</span>
|
||||||
<input name="display_name" maxlength="64" placeholder="留空则与用户名相同"></label>
|
<input name="display_name" maxlength="64" placeholder="留空则与用户名相同"></label>
|
||||||
|
<label class="row"><span>邮箱</span>
|
||||||
|
<input name="email" type="email" maxlength="128" placeholder="选填"></label>
|
||||||
<label class="row"><span>密码</span>
|
<label class="row"><span>密码</span>
|
||||||
<input name="password" type="password" autocomplete="new-password"></label>
|
<input name="password" type="password" autocomplete="new-password"></label>
|
||||||
<label class="row"><span>确认密码</span>
|
<label class="row"><span>确认密码</span>
|
||||||
<input name="password2" type="password" autocomplete="new-password"></label>
|
<input name="password2" type="password" autocomplete="new-password"></label>
|
||||||
<label class="row"><span>权限</span>
|
<label class="row"><span>权限</span>
|
||||||
<select name="is_admin">
|
<select name="is_admin">
|
||||||
<option value="1">管理员(可管理用户)</option>
|
<option value="0">普通账号(只管自己的凭证与数据)</option>
|
||||||
<option value="0">普通账号(只读数据与日志)</option>
|
<option value="1">管理员(可管理用户与实例设置)</option>
|
||||||
</select>
|
</select>
|
||||||
</label>
|
</label>
|
||||||
<button class="btn primary" type="submit">创建账号</button>
|
<button class="btn primary" type="submit">创建账号</button>
|
||||||
<p class="hint">密码至少 6 位、最多 128 位。普通账号不能用本页,也调不动用户管理接口。</p>
|
<p class="hint">密码至少 8 位且需含两类以上字符。新账号默认是<b>普通账号</b>;
|
||||||
|
管理员身份请显式选择。无论哪种身份,都需要各自配置自己的 Cookie 才能采集。</p>
|
||||||
</form>
|
</form>
|
||||||
</section>
|
</section>
|
||||||
|
|
||||||
@@ -42,29 +48,39 @@
|
|||||||
</div>
|
</div>
|
||||||
<div class="tablewrap">
|
<div class="tablewrap">
|
||||||
<table class="tbl" id="userTable">
|
<table class="tbl" id="userTable">
|
||||||
<thead><tr><th class="num">ID</th><th>用户名</th><th>显示名</th><th>权限</th>
|
<thead><tr><th class="num">ID</th><th>用户名</th><th>显示名</th><th>权限 / 状态</th>
|
||||||
<th>最后登录</th><th class="num">次数</th><th>操作</th></tr></thead>
|
<th class="num">我的数据</th><th>最后登录</th><th>操作</th></tr></thead>
|
||||||
<tbody>
|
<tbody>
|
||||||
{% for u in users %}
|
{% for u in users %}
|
||||||
<tr data-uid="{{ u.id }}" data-name="{{ u.username }}">
|
<tr data-uid="{{ u.id }}" data-name="{{ u.username }}"
|
||||||
|
data-status="{{ u.status or 'active' }}">
|
||||||
<td class="num muted">{{ u.id }}</td>
|
<td class="num muted">{{ u.id }}</td>
|
||||||
<td class="nowrap"><b>{{ u.username }}</b>
|
<td class="nowrap"><b>{{ u.username }}</b>
|
||||||
{% if u.id == me.id %}<span class="tag accent">当前</span>{% endif %}</td>
|
{% if u.id == me.id %}<span class="tag accent">当前</span>{% endif %}
|
||||||
|
{% if u.email %}<br><span class="muted sm">{{ u.email }}</span>{% endif %}</td>
|
||||||
<td><input class="inp inp-sm" name="display_name" maxlength="64"
|
<td><input class="inp inp-sm" name="display_name" maxlength="64"
|
||||||
value="{{ u.display_name or '' }}" spellcheck="false"></td>
|
value="{{ u.display_name or '' }}" spellcheck="false"></td>
|
||||||
<td>
|
<td class="nowrap">
|
||||||
{# 不能取消自己的管理员身份,所以本人的下拉直接禁用(服务端也会再拦一次) #}
|
{# 不能取消自己的管理员身份,也不能停用自己(服务端也会再拦一次) #}
|
||||||
<select class="inp inp-sm" name="is_admin" {{ 'disabled' if u.id == me.id }}>
|
<select class="inp inp-sm" name="is_admin" {{ 'disabled' if u.id == me.id }}>
|
||||||
<option value="1" {{ 'selected' if u.is_admin }}>管理员</option>
|
<option value="1" {{ 'selected' if u.is_admin }}>管理员</option>
|
||||||
<option value="0" {{ 'selected' if not u.is_admin }}>普通</option>
|
<option value="0" {{ 'selected' if not u.is_admin }}>普通</option>
|
||||||
</select>
|
</select>
|
||||||
|
<select class="inp inp-sm" name="status" {{ 'disabled' if u.id == me.id }}>
|
||||||
|
<option value="active" {{ 'selected' if (u.status or 'active') == 'active' }}>启用</option>
|
||||||
|
<option value="disabled" {{ 'selected' if u.status == 'disabled' }}>停用</option>
|
||||||
|
</select>
|
||||||
</td>
|
</td>
|
||||||
<td class="mono sm nowrap">{{ u.last_login_at or '—' }}</td>
|
<td class="num nowrap">{{ '{:,}'.format(u.recs or 0) }} 条
|
||||||
<td class="num">{{ u.login_count }}</td>
|
<br><span class="muted sm">{{ '%.2f'|format(u.credits or 0) }} 分</span></td>
|
||||||
|
<td class="mono sm nowrap">{{ (u.last_login_at or '—')[:16] }}
|
||||||
|
{% if u.last_login_ip %}<br><span class="muted">{{ u.last_login_ip }}</span>{% endif %}</td>
|
||||||
<td class="nowrap">
|
<td class="nowrap">
|
||||||
<button class="btn sm" type="button" data-act="save">保存</button>
|
<button class="btn sm" type="button" data-act="save">保存</button>
|
||||||
<button class="btn sm ghost" type="button" data-act="pwd">改密</button>
|
<button class="btn sm ghost" type="button" data-act="pwd">改密</button>
|
||||||
{% if u.id != me.id %}
|
{% if u.id != me.id %}
|
||||||
|
<button class="btn sm ghost" type="button" data-act="toggle">
|
||||||
|
{{ '停用' if (u.status or 'active') == 'active' else '启用' }}</button>
|
||||||
<button class="btn sm danger" type="button" data-act="del">删除</button>
|
<button class="btn sm danger" type="button" data-act="del">删除</button>
|
||||||
{% endif %}
|
{% endif %}
|
||||||
</td>
|
</td>
|
||||||
@@ -75,14 +91,19 @@
|
|||||||
</tbody>
|
</tbody>
|
||||||
</table>
|
</table>
|
||||||
</div>
|
</div>
|
||||||
<p class="hint">「改密」会依次询问新密码与确认;管理员不能取消自己的管理员身份,任何人也不能删除自己。</p>
|
<p class="hint">
|
||||||
|
「停用」会让该账号<b>立刻</b>失效(每个请求都会校验状态,不必等会话过期),
|
||||||
|
其数据与 Cookie 都保留;「删除」是<b>不可逆</b>的,会连同该账号的用量数据与 Cookie
|
||||||
|
一起删除(接口层另有保留数据的开关,供脚本调用时指定)。
|
||||||
|
管理员不能取消自己的管理员身份、不能停用或删除自己,也不能删掉最后一个启用的管理员。
|
||||||
|
</p>
|
||||||
</section>
|
</section>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<section class="card">
|
<section class="card">
|
||||||
<div class="cardhead">
|
<div class="cardhead">
|
||||||
<h2>用户操作审计</h2>
|
<h2>账号操作审计</h2>
|
||||||
<span class="hint">最近 20 条</span>
|
<span class="hint">最近 20 条(含注册与登录失败)</span>
|
||||||
</div>
|
</div>
|
||||||
<div class="tablewrap scroll-y">
|
<div class="tablewrap scroll-y">
|
||||||
<table class="tbl">
|
<table class="tbl">
|
||||||
@@ -139,20 +160,31 @@
|
|||||||
if (act === 'save') {
|
if (act === 'save') {
|
||||||
var fields = {};
|
var fields = {};
|
||||||
fields.display_name = row.querySelector('[name=display_name]').value;
|
fields.display_name = row.querySelector('[name=display_name]').value;
|
||||||
var sel = row.querySelector('[name=is_admin]');
|
var adm = row.querySelector('[name=is_admin]');
|
||||||
if (sel && !sel.disabled) fields.is_admin = sel.value;
|
if (adm && !adm.disabled) fields.is_admin = adm.value;
|
||||||
|
var st = row.querySelector('[name=status]');
|
||||||
|
if (st && !st.disabled) fields.status = st.value;
|
||||||
done(WBU.post('/api/users/' + uid, fields));
|
done(WBU.post('/api/users/' + uid, fields));
|
||||||
} else if (act === 'pwd') {
|
} else if (act === 'pwd') {
|
||||||
var p1 = window.prompt('为用户「' + name + '」设置新密码(至少 6 位)');
|
var p1 = window.prompt('为用户「' + name + '」设置新密码(至少 8 位,需含两类字符)');
|
||||||
if (p1 === null) { btn.disabled = false; btn.textContent = old; return; }
|
if (p1 === null) { btn.disabled = false; btn.textContent = old; return; }
|
||||||
var p2 = window.prompt('再输入一次新密码以确认');
|
var p2 = window.prompt('再输入一次新密码以确认');
|
||||||
if (p2 === null) { btn.disabled = false; btn.textContent = old; return; }
|
if (p2 === null) { btn.disabled = false; btn.textContent = old; return; }
|
||||||
if (p1 !== p2) { WBU.say('两次输入的密码不一致', 'warn'); btn.disabled = false; btn.textContent = old; return; }
|
if (p1 !== p2) { WBU.say('两次输入的密码不一致', 'warn'); btn.disabled = false; btn.textContent = old; return; }
|
||||||
done(WBU.post('/api/users/' + uid, { password: p1, password2: p2 }));
|
done(WBU.post('/api/users/' + uid, { password: p1, password2: p2 }));
|
||||||
} else if (act === 'del') {
|
} else if (act === 'toggle') {
|
||||||
if (!window.confirm('确定删除用户「' + name + '」?该操作不可撤销(其历史审计记录会保留)。')) {
|
var cur = row.dataset.status === 'active';
|
||||||
|
var next = cur ? 'disabled' : 'active';
|
||||||
|
if (!window.confirm(cur ? ('停用「' + name + '」?其数据与 Cookie 会保留,但无法登录。')
|
||||||
|
: ('启用「' + name + '」?'))) {
|
||||||
btn.disabled = false; btn.textContent = old; return;
|
btn.disabled = false; btn.textContent = old; return;
|
||||||
}
|
}
|
||||||
|
done(WBU.post('/api/users/' + uid, { status: next }));
|
||||||
|
} else if (act === 'del') {
|
||||||
|
var keep = window.confirm('确定删除用户「' + name + '」?\n\n'
|
||||||
|
+ '点「确定」= 连同其用量数据与 Cookie 一起删除(推荐)\n'
|
||||||
|
+ '点「取消」= 取消本次删除');
|
||||||
|
if (!keep) { btn.disabled = false; btn.textContent = old; return; }
|
||||||
done(WBU.post('/api/users/' + uid + '/delete', {}));
|
done(WBU.post('/api/users/' + uid + '/delete', {}));
|
||||||
} else {
|
} else {
|
||||||
btn.disabled = false; btn.textContent = old;
|
btn.disabled = false; btn.textContent = old;
|
||||||
|
|||||||
@@ -5,12 +5,20 @@
|
|||||||
"""页面路由(Jinja 模板)。
|
"""页面路由(Jinja 模板)。
|
||||||
|
|
||||||
分工:
|
分工:
|
||||||
/ 概览(KPI + 入口)
|
/ 概览(KPI + 入口)
|
||||||
/dashboard ECharts 交互大屏(独立静态页,登录后可达,数据走 /api/bundle)
|
/dashboard ECharts 交互大屏(独立静态页,登录后可达,数据走 /api/bundle)
|
||||||
/tasks 任务管理:调度开关/时刻、手动触发、运行历史
|
/records 数据明细:分页、筛选、搜索、导出
|
||||||
/config 配置管理:Cookie / UA / 采集参数 / 改密码
|
/tasks 任务管理:调度开关/时刻、手动触发、运行历史
|
||||||
/logs 日志管理:采集逐次明细 + 应用日志尾部
|
/config 配置管理:本人的 Cookie / UA / 采集参数
|
||||||
/records 数据明细:分页、筛选、搜索、导出
|
/logs 日志管理:采集逐次明细 + 应用日志尾部
|
||||||
|
/profile 个人中心:资料、密码、凭证状态
|
||||||
|
/users 用户管理(仅管理员)
|
||||||
|
/register 自助注册(受 allow_register 开关约束)
|
||||||
|
/captcha.png 图形验证码
|
||||||
|
|
||||||
|
**多用户约定**
|
||||||
|
所有数据类页面都只取 `current_user()["id"]` 那份数据;管理员在
|
||||||
|
「用户管理」里能看到账号列表,但**看不到别人的用量与凭证**。
|
||||||
"""
|
"""
|
||||||
import csv
|
import csv
|
||||||
import io
|
import io
|
||||||
@@ -18,12 +26,11 @@ import os
|
|||||||
import sqlite3
|
import sqlite3
|
||||||
|
|
||||||
from flask import (Blueprint, current_app, flash, jsonify, redirect, render_template,
|
from flask import (Blueprint, current_app, flash, jsonify, redirect, render_template,
|
||||||
request, send_from_directory, url_for)
|
request, send_from_directory, session, url_for)
|
||||||
|
|
||||||
from .. import collect, config, db, query, scheduler
|
from .. import collect, config, db, query, scheduler, security
|
||||||
from ..security import (admin_required, clear_fail, current_user, is_locked, lock_left,
|
from ..security import (admin_required, current_user, is_admin, login_required,
|
||||||
login_ok, login_required, login_session, logout_session,
|
safe_next)
|
||||||
note_fail, safe_next)
|
|
||||||
|
|
||||||
bp = Blueprint("views", __name__)
|
bp = Blueprint("views", __name__)
|
||||||
|
|
||||||
@@ -32,42 +39,182 @@ def _ip():
|
|||||||
return request.headers.get("X-Forwarded-For", request.remote_addr or "").split(",")[0].strip()
|
return request.headers.get("X-Forwarded-For", request.remote_addr or "").split(",")[0].strip()
|
||||||
|
|
||||||
|
|
||||||
|
def _uid():
|
||||||
|
"""当前账号 id。调用方必须已过 @login_required。"""
|
||||||
|
u = current_user()
|
||||||
|
return u["id"] if u else 0
|
||||||
|
|
||||||
|
|
||||||
|
def _shift(days):
|
||||||
|
from datetime import datetime, timedelta
|
||||||
|
return (datetime.now() + timedelta(days=days)).strftime("%Y-%m-%d")
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------- 验证码 ----------------
|
||||||
|
@bp.get("/captcha.png")
|
||||||
|
def captcha_png():
|
||||||
|
"""生成一张验证码。答案只写进 captchas 表,会话里只记 id。"""
|
||||||
|
purpose = (request.args.get("purpose") or "login").strip().lower()
|
||||||
|
if purpose not in ("login", "register"):
|
||||||
|
purpose = "login"
|
||||||
|
if not security.captcha_fetch_allowed(_ip()):
|
||||||
|
return "验证码请求过于频繁,请稍后再试", 429
|
||||||
|
try:
|
||||||
|
png = security.issue_captcha(db.get_db(), purpose)
|
||||||
|
except sqlite3.Error:
|
||||||
|
return "验证码服务暂不可用", 503
|
||||||
|
resp = current_app.response_class(png, mimetype="image/png")
|
||||||
|
# 必须禁缓存:否则浏览器复用旧图,而服务端那张已经被消费掉了,
|
||||||
|
# 表现为「图没变但怎么输都错」。
|
||||||
|
resp.headers["Cache-Control"] = "no-store, no-cache, must-revalidate, max-age=0"
|
||||||
|
resp.headers["Pragma"] = "no-cache"
|
||||||
|
return resp
|
||||||
|
|
||||||
|
|
||||||
# ---------------- 登录 ----------------
|
# ---------------- 登录 ----------------
|
||||||
def _login_ctx(**kw):
|
def _login_ctx(conn=None, **kw):
|
||||||
"""登录页共用的上下文:锁定阈值/会话时长都从配置读,避免模板里写死数字。"""
|
"""登录页共用的上下文:锁定阈值/会话时长都从配置读,避免模板里写死数字。
|
||||||
|
|
||||||
|
`need_captcha` 与 `allow_register` 也在这里补齐 —— 登录页与注册页
|
||||||
|
必须对「要不要验证码」保持一致,否则会出现「页面没给输入框、
|
||||||
|
服务端却在校验」的死循环。
|
||||||
|
"""
|
||||||
kw.setdefault("max_fails", config.MAX_LOGIN_FAILS)
|
kw.setdefault("max_fails", config.MAX_LOGIN_FAILS)
|
||||||
kw.setdefault("lock_minutes", config.LOGIN_LOCK_MINUTES)
|
kw.setdefault("lock_minutes", config.LOGIN_LOCK_MINUTES)
|
||||||
kw.setdefault("session_hours", config.SESSION_HOURS)
|
kw.setdefault("session_hours", config.SESSION_HOURS)
|
||||||
|
kw.setdefault("pwd_min", config.PASSWORD_MIN)
|
||||||
|
if conn is not None:
|
||||||
|
kw.setdefault("need_captcha", security.captcha_required(conn, _ip(), kw.get("username") or ""))
|
||||||
|
kw.setdefault("allow_register", security.register_allowed(conn))
|
||||||
return kw
|
return kw
|
||||||
|
|
||||||
|
|
||||||
@bp.route("/login", methods=["GET", "POST"])
|
@bp.route("/login", methods=["GET", "POST"])
|
||||||
def login():
|
def login():
|
||||||
nxt = request.values.get("next") or ""
|
nxt = request.values.get("next") or ""
|
||||||
|
conn = db.get_db()
|
||||||
if request.method == "POST":
|
if request.method == "POST":
|
||||||
ip = _ip()
|
ip = _ip()
|
||||||
if is_locked(ip):
|
|
||||||
n = lock_left(ip)
|
|
||||||
flash("登录失败次数过多,请 %d 秒后再试" % n, "error")
|
|
||||||
return render_template("login.html", **_login_ctx(next_url=nxt)), 429
|
|
||||||
username = (request.form.get("username") or "").strip()
|
username = (request.form.get("username") or "").strip()
|
||||||
pwd = request.form.get("password") or ""
|
pwd = request.form.get("password") or ""
|
||||||
conn = db.get_db()
|
|
||||||
user = login_ok(conn, username, pwd)
|
left = security.auth_locked(ip, username)
|
||||||
|
if left:
|
||||||
|
security.audit_login_fail(conn, username, "已锁定,剩余 %d 秒" % left, ip)
|
||||||
|
flash("登录失败次数过多,请 %d 秒后再试" % left, "error")
|
||||||
|
return render_template("login.html",
|
||||||
|
**_login_ctx(conn, next_url=nxt, username=username,
|
||||||
|
need_captcha=True)), 429
|
||||||
|
|
||||||
|
# 验证码**先于**口令校验:否则攻击者可以拿「密码对不对」当信号,
|
||||||
|
# 在解验证码之前就把字典跑完。
|
||||||
|
need_cap = security.captcha_required(conn, ip, username)
|
||||||
|
if need_cap and not security.consume_captcha(conn, "login", request.form.get("captcha")):
|
||||||
|
n = security.note_auth_fail(ip, username)
|
||||||
|
security.audit_login_fail(conn, username, "验证码错误(第 %d 次)" % n, ip)
|
||||||
|
flash("验证码不正确或已过期,请重新输入", "error")
|
||||||
|
return render_template("login.html",
|
||||||
|
**_login_ctx(conn, username=username, next_url=nxt,
|
||||||
|
need_captcha=True)), 400
|
||||||
|
|
||||||
|
user, err = security.login_ok(conn, username, pwd)
|
||||||
if user is None:
|
if user is None:
|
||||||
n = note_fail(ip)
|
n = security.note_auth_fail(ip, username)
|
||||||
db.audit(conn, "login_failed", username, "第 %d 次失败" % n, ip)
|
security.audit_login_fail(conn, username, err + "(第 %d 次)" % n, ip)
|
||||||
flash("用户名或密码不正确(剩余尝试 %d 次)" % max(0, config.MAX_LOGIN_FAILS - n), "error")
|
flash("%s(剩余尝试 %d 次)" % (err, max(0, config.MAX_LOGIN_FAILS - n)), "error")
|
||||||
# 必须把 next 显式回填:失败后 request.args 为空,
|
# 必须把 next 显式回填:失败后 request.args 为空,
|
||||||
# 若模板从 request.args 取值会导致跳转目标丢失(历史 bug)。
|
# 若模板从 request.args 取值会导致跳转目标丢失(历史 bug)。
|
||||||
return render_template("login.html", **_login_ctx(username=username, next_url=nxt)), 401
|
return render_template("login.html",
|
||||||
clear_fail(ip)
|
**_login_ctx(conn, username=username, next_url=nxt,
|
||||||
login_session(user)
|
need_captcha=True)), 401
|
||||||
db.audit(conn, "login", username, "登录成功", ip)
|
security.clear_auth_fail(ip, username)
|
||||||
return redirect(safe_next(nxt, url_for("views.overview")))
|
security.login_session(user)
|
||||||
|
conn.execute("UPDATE users SET last_login_ip=? WHERE id=?", (ip, user["id"]))
|
||||||
|
db.audit(conn, "login", username, "登录成功", ip, user["id"])
|
||||||
|
target = safe_next(nxt, "")
|
||||||
|
if not target:
|
||||||
|
# 新注册 / 还没配凭证 → 直接带到配置页,少一步摸索
|
||||||
|
cred = db.secret_state(conn, "cookie", user["id"])
|
||||||
|
target = url_for("views.config_page") if not cred["set"] else url_for("views.overview")
|
||||||
|
return redirect(target)
|
||||||
|
|
||||||
if current_user():
|
if current_user():
|
||||||
return redirect(url_for("views.overview"))
|
return redirect(url_for("views.overview"))
|
||||||
return render_template("login.html", **_login_ctx(next_url=nxt))
|
return render_template("login.html", **_login_ctx(conn, next_url=nxt))
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------- 注册 ----------------
|
||||||
|
def _register_ctx(**kw):
|
||||||
|
kw.setdefault("max_fails", config.MAX_LOGIN_FAILS)
|
||||||
|
kw.setdefault("lock_minutes", config.LOGIN_LOCK_MINUTES)
|
||||||
|
kw.setdefault("pwd_min", config.PASSWORD_MIN)
|
||||||
|
return kw
|
||||||
|
|
||||||
|
|
||||||
|
@bp.route("/register", methods=["GET", "POST"])
|
||||||
|
def register():
|
||||||
|
conn = db.get_db()
|
||||||
|
if not security.register_allowed(conn):
|
||||||
|
return render_template("error.html", code=403,
|
||||||
|
message="管理员已关闭自助注册,请联系管理员开通账号"), 403
|
||||||
|
if current_user():
|
||||||
|
return redirect(url_for("views.overview"))
|
||||||
|
|
||||||
|
if request.method == "POST":
|
||||||
|
ip = _ip()
|
||||||
|
username = (request.form.get("username") or "").strip()
|
||||||
|
display = (request.form.get("display_name") or "").strip()[:64]
|
||||||
|
email = (request.form.get("email") or "").strip()[:128]
|
||||||
|
pwd = request.form.get("password") or ""
|
||||||
|
pwd2 = request.form.get("password2") or ""
|
||||||
|
ctx = _register_ctx(username=username, display_name=display, email=email,
|
||||||
|
need_captcha=True)
|
||||||
|
|
||||||
|
left = security.auth_locked(ip, username)
|
||||||
|
if left:
|
||||||
|
flash("操作过于频繁,请 %d 秒后再试" % left, "error")
|
||||||
|
return render_template("register.html", **ctx), 429
|
||||||
|
|
||||||
|
# 注册一律要验证码:这是唯一能让陌生人写库的入口
|
||||||
|
if not security.consume_captcha(conn, "register", request.form.get("captcha")):
|
||||||
|
security.note_auth_fail(ip, username)
|
||||||
|
db.audit(conn, "register_rejected", username or "-", "验证码错误", ip)
|
||||||
|
flash("验证码不正确或已过期,请重新输入", "error")
|
||||||
|
return render_template("register.html", **ctx), 400
|
||||||
|
|
||||||
|
ok, n, limit = security.register_quota(conn, ip)
|
||||||
|
if not ok:
|
||||||
|
db.audit(conn, "register_rejected", username or "-",
|
||||||
|
"同 IP 当日注册数已达上限 %d" % limit, ip)
|
||||||
|
flash("同一来源每天最多注册 %d 个账号,请明天再试或联系管理员" % limit, "error")
|
||||||
|
return render_template("register.html", **ctx), 429
|
||||||
|
|
||||||
|
err = security.username_problem(username) or security.password_problem(pwd, pwd2, username)
|
||||||
|
if err:
|
||||||
|
security.note_auth_fail(ip, username)
|
||||||
|
db.audit(conn, "register_rejected", username or "-", err, ip)
|
||||||
|
flash(err, "error")
|
||||||
|
return render_template("register.html", **ctx), 400
|
||||||
|
if db.user_by_name(conn, username):
|
||||||
|
# 用户名唯一性本来就暴露(注册时要查重),这里如实告知
|
||||||
|
flash("用户名已被占用,请换一个", "error")
|
||||||
|
return render_template("register.html", **ctx), 400
|
||||||
|
|
||||||
|
cur = conn.execute(
|
||||||
|
"INSERT INTO users(username,password_hash,display_name,email,is_admin,status,"
|
||||||
|
" created_at,register_ip) VALUES(?,?,?,?,0,'active',?,?)",
|
||||||
|
(username, security.hash_password(pwd), display or username, email or None,
|
||||||
|
db.now_str(), ip))
|
||||||
|
uid = cur.lastrowid
|
||||||
|
db.audit(conn, "register", username, "自助注册成功(账号 #%d)" % uid, ip, uid)
|
||||||
|
# 注册即登录:少一次输密码,也顺手把会话建立起来
|
||||||
|
row = db.user_by_id(conn, uid)
|
||||||
|
security.login_session(row)
|
||||||
|
security.clear_auth_fail(ip, username)
|
||||||
|
flash("注册成功。请粘贴你自己账号的 Cookie —— 采集只使用本人的凭证。", "ok")
|
||||||
|
return redirect(url_for("views.config_page"))
|
||||||
|
|
||||||
|
return render_template("register.html", **_register_ctx(need_captcha=True))
|
||||||
|
|
||||||
|
|
||||||
@bp.post("/logout")
|
@bp.post("/logout")
|
||||||
@@ -76,8 +223,8 @@ def logout_post():
|
|||||||
"""退出登录改为 POST + CSRF:GET 型退出会被 <img src> 这类请求静默触发。"""
|
"""退出登录改为 POST + CSRF:GET 型退出会被 <img src> 这类请求静默触发。"""
|
||||||
u = current_user()
|
u = current_user()
|
||||||
if u:
|
if u:
|
||||||
db.audit(db.get_db(), "logout", u["username"], "", _ip())
|
db.audit(db.get_db(), "logout", u["username"], "", _ip(), u["id"])
|
||||||
logout_session()
|
security.logout_session()
|
||||||
flash("已退出登录", "ok")
|
flash("已退出登录", "ok")
|
||||||
return redirect(url_for("views.login"))
|
return redirect(url_for("views.login"))
|
||||||
|
|
||||||
@@ -96,29 +243,25 @@ def logout():
|
|||||||
@login_required
|
@login_required
|
||||||
def overview():
|
def overview():
|
||||||
conn = db.get_db()
|
conn = db.get_db()
|
||||||
mf = query.manifest(conn)
|
uid = _uid()
|
||||||
t = query.totals(conn)
|
mf = query.manifest(conn, uid)
|
||||||
|
t = query.totals(conn, uid)
|
||||||
today = db.now_str()[:10]
|
today = db.now_str()[:10]
|
||||||
st = query.summary(conn, today, today)
|
st = query.summary(conn, uid, today, today)
|
||||||
d30 = query.summary(conn, _shift(-29), today)
|
d30 = query.summary(conn, uid, _shift(-29), today)
|
||||||
# 昨日对比:昨日整日 vs 今日(残日),让「今天偏少」有参照
|
# 昨日对比:昨日整日 vs 今日(残日),让「今天偏少」有参照
|
||||||
y = _shift(-1)
|
y = _shift(-1)
|
||||||
yest = query.summary(conn, y, y)
|
yest = query.summary(conn, uid, y, y)
|
||||||
dims = query.dims(conn)
|
dims = query.dims(conn, uid)
|
||||||
# 注意:这里的 SQL 必须把模板用到的列都选出来(模板渲染 r.fetched,
|
# 注意:这里的 SQL 必须把模板用到的列都选出来(模板渲染 r.fetched,
|
||||||
# 少选一列并不会报错,只会静默渲染成空白 —— 历史 bug)。
|
# 少选一列并不会报错,只会静默渲染成空白 —— 历史 bug)。
|
||||||
runs = conn.execute(
|
runs = conn.execute(
|
||||||
"SELECT id,trigger,status,started_at,duration_ms,fetched,added,dup,total,conflicts,message"
|
"SELECT id,trigger,status,started_at,duration_ms,fetched,added,dup,total,conflicts,message"
|
||||||
" FROM collect_runs ORDER BY id DESC LIMIT 8").fetchall()
|
" FROM collect_runs WHERE user_id=? ORDER BY id DESC LIMIT 8", (uid,)).fetchall()
|
||||||
return render_template("overview.html", mf=mf, totals=t, today_stat=st, stat30=d30,
|
return render_template("overview.html", mf=mf, totals=t, today_stat=st, stat30=d30,
|
||||||
yesterday=yest, yday=y,
|
yesterday=yest, yday=y,
|
||||||
models=dims["model"][:8], clients=dims["client"],
|
models=dims["model"][:8], clients=dims["client"],
|
||||||
runs=runs, sch=_sch_info(conn), active="overview")
|
runs=runs, sch=_sch_info(conn, uid), active="overview")
|
||||||
|
|
||||||
|
|
||||||
def _shift(days):
|
|
||||||
from datetime import datetime, timedelta
|
|
||||||
return (datetime.now() + timedelta(days=days)).strftime("%Y-%m-%d")
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------- 大屏(独立 ECharts 页)----------------
|
# ---------------- 大屏(独立 ECharts 页)----------------
|
||||||
@@ -133,16 +276,17 @@ def dashboard():
|
|||||||
|
|
||||||
|
|
||||||
# ---------------- 任务管理 ----------------
|
# ---------------- 任务管理 ----------------
|
||||||
def _sch_info(conn):
|
def _sch_info(conn, uid):
|
||||||
sch = scheduler.get_scheduler()
|
sch = scheduler.get_scheduler()
|
||||||
nxt = scheduler.next_run_at(conn)
|
nxt = scheduler.next_run_at(conn, uid)
|
||||||
last = conn.execute("SELECT * FROM collect_runs ORDER BY id DESC LIMIT 1").fetchone()
|
last = conn.execute("SELECT * FROM collect_runs WHERE user_id=? ORDER BY id DESC LIMIT 1",
|
||||||
|
(uid,)).fetchone()
|
||||||
return {
|
return {
|
||||||
"running": sch.running,
|
"running": sch.running,
|
||||||
"enabled": db.get_bool(conn, "schedule_enabled", True),
|
"enabled": db.get_bool(conn, "schedule_enabled", True, uid),
|
||||||
"times": scheduler.slots(conn),
|
"times": scheduler.slots(conn, uid),
|
||||||
"next_run": nxt.strftime("%Y-%m-%d %H:%M:%S") if nxt else None,
|
"next_run": nxt.strftime("%Y-%m-%d %H:%M:%S") if nxt else None,
|
||||||
"catch_up": db.get_bool(conn, "catch_up", True),
|
"catch_up": db.get_bool(conn, "catch_up", True, uid),
|
||||||
"interval": sch.interval,
|
"interval": sch.interval,
|
||||||
"last": dict(last) if last else None,
|
"last": dict(last) if last else None,
|
||||||
"lock": os.path.exists(collect.LOCK_PATH),
|
"lock": os.path.exists(collect.LOCK_PATH),
|
||||||
@@ -154,14 +298,16 @@ def _sch_info(conn):
|
|||||||
@login_required
|
@login_required
|
||||||
def tasks():
|
def tasks():
|
||||||
conn = db.get_db()
|
conn = db.get_db()
|
||||||
|
uid = _uid()
|
||||||
page = _int_arg("page", 1, 1, 10 ** 6)
|
page = _int_arg("page", 1, 1, 10 ** 6)
|
||||||
size = 20
|
size = 20
|
||||||
total = conn.execute("SELECT COUNT(*) FROM collect_runs").fetchone()[0]
|
total = conn.execute("SELECT COUNT(*) FROM collect_runs WHERE user_id=?",
|
||||||
runs = conn.execute("SELECT * FROM collect_runs ORDER BY id DESC LIMIT ? OFFSET ?",
|
(uid,)).fetchone()[0]
|
||||||
(size, (page - 1) * size)).fetchall()
|
runs = conn.execute("SELECT * FROM collect_runs WHERE user_id=? ORDER BY id DESC"
|
||||||
s = db.get_settings(conn)
|
" LIMIT ? OFFSET ?", (uid, size, (page - 1) * size)).fetchall()
|
||||||
|
s = db.get_settings(conn, uid=uid)
|
||||||
pages = max(1, (total + size - 1) // size)
|
pages = max(1, (total + size - 1) // size)
|
||||||
return render_template("tasks.html", runs=runs, sch=_sch_info(conn),
|
return render_template("tasks.html", runs=runs, sch=_sch_info(conn, uid),
|
||||||
s_times=s.get("schedule_times") or "",
|
s_times=s.get("schedule_times") or "",
|
||||||
s_grace=s.get("catch_up_grace_hours") or "12",
|
s_grace=s.get("catch_up_grace_hours") or "12",
|
||||||
page=page, pages=pages, total=total,
|
page=page, pages=pages, total=total,
|
||||||
@@ -191,29 +337,60 @@ def _page_window(page, pages, span=9):
|
|||||||
@login_required
|
@login_required
|
||||||
def config_page():
|
def config_page():
|
||||||
conn = db.get_db()
|
conn = db.get_db()
|
||||||
s = db.get_settings(conn)
|
uid = _uid()
|
||||||
|
s = db.get_settings(conn, uid=uid)
|
||||||
for k in [k for k in list(s) if config.is_internal_key(k)]:
|
for k in [k for k in list(s) if config.is_internal_key(k)]:
|
||||||
s.pop(k, None)
|
s.pop(k, None)
|
||||||
cookie = (s.pop("cookie", "") or "")
|
# 加密键在 get_settings 里已经被置空,这里补上「状态摘要」给页面显示
|
||||||
s["cookie_hint"] = ("%d 字符,结尾 …%s" % (len(cookie), cookie[-16:])) if cookie else ""
|
st = db.secret_state(conn, "cookie", uid)
|
||||||
return render_template("config.html", s=s, sch=_sch_info(conn),
|
s["cookie_hint"] = ("%d 字符,结尾 …%s" % (st["chars"], st["tail"])) if st["set"] else ""
|
||||||
|
s["cookie_broken"] = st["broken"]
|
||||||
|
s["cookie_at"] = st["at"]
|
||||||
|
return render_template("config.html", s=s, sch=_sch_info(conn, uid),
|
||||||
secret_keys=config.SECRET_KEYS,
|
secret_keys=config.SECRET_KEYS,
|
||||||
num_settings=config.NUM_SETTINGS,
|
num_settings=config.NUM_SETTINGS,
|
||||||
|
pwd_min=config.PASSWORD_MIN,
|
||||||
|
is_admin=is_admin(),
|
||||||
|
global_keys=config.GLOBAL_KEYS,
|
||||||
active="config")
|
active="config")
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------- 个人中心 ----------------
|
||||||
|
@bp.get("/profile")
|
||||||
|
@login_required
|
||||||
|
def profile_page():
|
||||||
|
conn = db.get_db()
|
||||||
|
u = current_user()
|
||||||
|
row = db.user_by_id(conn, u["id"])
|
||||||
|
cred = db.secret_state(conn, "cookie", u["id"])
|
||||||
|
my = conn.execute(
|
||||||
|
"SELECT COUNT(*) n, COALESCE(SUM(credits),0) c, MIN(day) d0, MAX(day) d1"
|
||||||
|
" FROM usage_records WHERE user_id=?", (u["id"],)).fetchone()
|
||||||
|
runs = conn.execute("SELECT COUNT(*) FROM collect_runs WHERE user_id=?",
|
||||||
|
(u["id"],)).fetchone()[0]
|
||||||
|
return render_template("profile.html", me=dict(row), cred=cred, my=dict(my),
|
||||||
|
runs=runs, sch=_sch_info(conn, u["id"]),
|
||||||
|
pwd_min=config.PASSWORD_MIN, active="profile")
|
||||||
|
|
||||||
|
|
||||||
# ---------------- 用户管理 ----------------
|
# ---------------- 用户管理 ----------------
|
||||||
@bp.get("/users")
|
@bp.get("/users")
|
||||||
@admin_required
|
@admin_required
|
||||||
def users_page():
|
def users_page():
|
||||||
conn = db.get_db()
|
conn = db.get_db()
|
||||||
users = conn.execute(
|
users = conn.execute(
|
||||||
"SELECT id,username,display_name,is_admin,created_at,last_login_at,login_count"
|
"SELECT u.id,u.username,u.display_name,u.email,u.is_admin,u.status,u.created_at,"
|
||||||
" FROM users ORDER BY id").fetchall()
|
" u.register_ip,u.last_login_at,u.last_login_ip,u.login_count,"
|
||||||
|
" (SELECT COUNT(*) FROM usage_records r WHERE r.user_id=u.id) recs,"
|
||||||
|
" (SELECT COALESCE(SUM(credits),0) FROM usage_records r WHERE r.user_id=u.id) credits"
|
||||||
|
" FROM users u ORDER BY u.id").fetchall()
|
||||||
audits = conn.execute("SELECT * FROM audit_log WHERE action LIKE 'user%'"
|
audits = conn.execute("SELECT * FROM audit_log WHERE action LIKE 'user%'"
|
||||||
|
" OR action IN ('register','register_rejected','password','login_failed')"
|
||||||
" ORDER BY id DESC LIMIT 20").fetchall()
|
" ORDER BY id DESC LIMIT 20").fetchall()
|
||||||
return render_template("users.html", users=users, audits=audits,
|
return render_template("users.html", users=users, audits=audits,
|
||||||
me=current_user(), active="users")
|
me=current_user(), allow_register=db.get_bool(
|
||||||
|
conn, "allow_register", True),
|
||||||
|
active="users")
|
||||||
|
|
||||||
|
|
||||||
# ---------------- 日志管理 ----------------
|
# ---------------- 日志管理 ----------------
|
||||||
@@ -221,24 +398,50 @@ def users_page():
|
|||||||
@login_required
|
@login_required
|
||||||
def logs():
|
def logs():
|
||||||
conn = db.get_db()
|
conn = db.get_db()
|
||||||
|
u = current_user()
|
||||||
|
uid = u["id"]
|
||||||
|
adm = bool(u["is_admin"])
|
||||||
|
|
||||||
run_id = request.args.get("run")
|
run_id = request.args.get("run")
|
||||||
detail = None
|
detail = None
|
||||||
if run_id and str(run_id).isdigit():
|
if run_id and str(run_id).isdigit():
|
||||||
detail = conn.execute("SELECT * FROM collect_runs WHERE id=?", (int(run_id),)).fetchone()
|
# 明细也必须限本人:否则改一个 ?run= 就能看到别人的采集日志
|
||||||
|
if adm:
|
||||||
|
detail = conn.execute("SELECT * FROM collect_runs WHERE id=?",
|
||||||
|
(int(run_id),)).fetchone()
|
||||||
|
else:
|
||||||
|
detail = conn.execute("SELECT * FROM collect_runs WHERE id=? AND user_id=?",
|
||||||
|
(int(run_id), uid)).fetchone()
|
||||||
|
|
||||||
status = request.args.get("status") or ""
|
status = request.args.get("status") or ""
|
||||||
w, p = ("WHERE status = ?", [status]) if status in ("ok", "warn", "error", "running") else ("", [])
|
w, p = ("WHERE user_id = ? AND status = ?", [uid, status]) \
|
||||||
# 操作审计:按动作筛选 + 分页(原来只能看最近 40 条,等于不可查)
|
if status in ("ok", "warn", "error", "running") else ("WHERE user_id = ?", [uid])
|
||||||
|
|
||||||
|
# 操作审计:管理员看全部(便于追责),普通用户只看自己触发的
|
||||||
|
aw, ap = [], []
|
||||||
|
if not adm:
|
||||||
|
aw.append("user_id = ?")
|
||||||
|
ap.append(uid)
|
||||||
act = request.args.get("act") or ""
|
act = request.args.get("act") or ""
|
||||||
aw, ap = ("WHERE action = ?", [act]) if act else ("", [])
|
if act:
|
||||||
|
aw.append("action = ?")
|
||||||
|
ap.append(act)
|
||||||
|
aw_sql = ("WHERE " + " AND ".join(aw)) if aw else ""
|
||||||
|
# 动作清单的统计基数不能带 action 条件(否则永远只剩一个动作可选)
|
||||||
|
base = ("WHERE user_id = ?" if not adm else "")
|
||||||
|
base_p = [uid] if not adm else []
|
||||||
|
|
||||||
apage = _int_arg("apage", 1, 1, 10 ** 6)
|
apage = _int_arg("apage", 1, 1, 10 ** 6)
|
||||||
asize = 20
|
asize = 20
|
||||||
atotal = conn.execute("SELECT COUNT(*) FROM audit_log %s" % aw, ap).fetchone()[0]
|
atotal = conn.execute("SELECT COUNT(*) FROM audit_log %s" % aw_sql, ap).fetchone()[0]
|
||||||
audits = conn.execute("SELECT * FROM audit_log %s ORDER BY id DESC LIMIT ? OFFSET ?" % aw,
|
audits = conn.execute("SELECT * FROM audit_log %s ORDER BY id DESC LIMIT ? OFFSET ?" % aw_sql,
|
||||||
ap + [asize, (apage - 1) * asize]).fetchall()
|
ap + [asize, (apage - 1) * asize]).fetchall()
|
||||||
# 注意传的是 sqlite3.Row 列表而不是纯字符串列表:模板要用 a[0]=动作、a[1]=次数,
|
# 注意传的是 sqlite3.Row 列表而不是纯字符串列表:模板要用 a[0]=动作、a[1]=次数,
|
||||||
# 若在这里就用推导式取 r[0],模板里的 a[0] 会变成「字符串的第一个字符」。
|
# 若在这里就用推导式取 r[0],模板里的 a[0] 会变成「字符串的第一个字符」。
|
||||||
actions = conn.execute(
|
actions = conn.execute(
|
||||||
"SELECT action, COUNT(*) n FROM audit_log GROUP BY action ORDER BY n DESC, action").fetchall()
|
"SELECT action, COUNT(*) n FROM audit_log %s GROUP BY action ORDER BY n DESC, action"
|
||||||
|
% base, base_p).fetchall()
|
||||||
|
|
||||||
page = _int_arg("page", 1, 1, 10 ** 6)
|
page = _int_arg("page", 1, 1, 10 ** 6)
|
||||||
size = 30
|
size = 30
|
||||||
total = conn.execute("SELECT COUNT(*) FROM collect_runs %s" % w, p).fetchone()[0]
|
total = conn.execute("SELECT COUNT(*) FROM collect_runs %s" % w, p).fetchone()[0]
|
||||||
@@ -251,13 +454,21 @@ def logs():
|
|||||||
apage_window=_page_window(apage, apages, span=7),
|
apage_window=_page_window(apage, apages, span=7),
|
||||||
page=page, pages=max(1, (total + size - 1) // size), total=total,
|
page=page, pages=max(1, (total + size - 1) // size), total=total,
|
||||||
page_window=_page_window(page, max(1, (total + size - 1) // size)),
|
page_window=_page_window(page, max(1, (total + size - 1) // size)),
|
||||||
status=status,
|
status=status, audit_all=adm,
|
||||||
active="logs")
|
active="logs")
|
||||||
|
|
||||||
|
|
||||||
@bp.get("/logs/tail")
|
@bp.get("/logs/tail")
|
||||||
@login_required
|
@login_required
|
||||||
def logs_tail():
|
def logs_tail():
|
||||||
|
"""应用日志尾部(进程级,所有账号看到的是同一份)。
|
||||||
|
|
||||||
|
只对管理员开放:日志里会打印数据库路径、账号名等运行信息,
|
||||||
|
没有理由让任意注册用户读到整个实例的运行轨迹。
|
||||||
|
"""
|
||||||
|
if not is_admin():
|
||||||
|
return jsonify({"ok": False, "error": "forbidden",
|
||||||
|
"message": "应用日志仅管理员可查看"}), 403
|
||||||
n = _int_arg("lines", 200, 10, 2000)
|
n = _int_arg("lines", 200, 10, 2000)
|
||||||
path = config.APP_LOG
|
path = config.APP_LOG
|
||||||
if not os.path.exists(path):
|
if not os.path.exists(path):
|
||||||
@@ -289,6 +500,7 @@ def _day_args():
|
|||||||
@login_required
|
@login_required
|
||||||
def records():
|
def records():
|
||||||
conn = db.get_db()
|
conn = db.get_db()
|
||||||
|
uid = _uid()
|
||||||
frm, to = _day_args()
|
frm, to = _day_args()
|
||||||
model = request.args.get("model") or None
|
model = request.args.get("model") or None
|
||||||
client = request.args.get("client") or None
|
client = request.args.get("client") or None
|
||||||
@@ -296,10 +508,10 @@ def records():
|
|||||||
order = request.args.get("order") or "ts_desc"
|
order = request.args.get("order") or "ts_desc"
|
||||||
page = _int_arg("page", 1, 1, 10 ** 6)
|
page = _int_arg("page", 1, 1, 10 ** 6)
|
||||||
size = _int_arg("size", 50, 10, query.MAX_PAGE_SIZE)
|
size = _int_arg("size", 50, 10, query.MAX_PAGE_SIZE)
|
||||||
data = query.records_page(conn, frm, to, model=model, client=client, q=q,
|
data = query.records_page(conn, uid, frm, to, model=model, client=client, q=q,
|
||||||
page=page, size=size, order=order)
|
page=page, size=size, order=order)
|
||||||
# 只算一次 dims:query.dims() 内部有 3 条 GROUP BY,重复调用纯属浪费
|
# 只算一次 dims:query.dims() 内部有 3 条 GROUP BY,重复调用纯属浪费
|
||||||
d = query.dims(conn)
|
d = query.dims(conn, uid)
|
||||||
models = [r["name"] for r in d["model"]]
|
models = [r["name"] for r in d["model"]]
|
||||||
clients = [r["name"] for r in d["client"]]
|
clients = [r["name"] for r in d["client"]]
|
||||||
# 注意:不要把含 "items" 键的 dict 直接交给模板——Jinja 的属性查找会先命中
|
# 注意:不要把含 "items" 键的 dict 直接交给模板——Jinja 的属性查找会先命中
|
||||||
@@ -321,6 +533,8 @@ def records_export():
|
|||||||
数据用 query.iter_records 流式取,不把整个结果集读进内存。
|
数据用 query.iter_records 流式取,不把整个结果集读进内存。
|
||||||
"""
|
"""
|
||||||
from flask import Response
|
from flask import Response
|
||||||
|
u = current_user()
|
||||||
|
uid = u["id"]
|
||||||
frm, to = _day_args()
|
frm, to = _day_args()
|
||||||
model = request.args.get("model") or None
|
model = request.args.get("model") or None
|
||||||
client = request.args.get("client") or None
|
client = request.args.get("client") or None
|
||||||
@@ -340,7 +554,7 @@ def records_export():
|
|||||||
# 自己开一条连接,并在流结束时关掉。
|
# 自己开一条连接,并在流结束时关掉。
|
||||||
own = db.connect()
|
own = db.connect()
|
||||||
try:
|
try:
|
||||||
for r in query.iter_records(own, frm, to, model=model, client=client,
|
for r in query.iter_records(own, uid, frm, to, model=model, client=client,
|
||||||
q=q, order=order):
|
q=q, order=order):
|
||||||
buf.seek(0)
|
buf.seek(0)
|
||||||
buf.truncate(0)
|
buf.truncate(0)
|
||||||
@@ -350,7 +564,8 @@ def records_export():
|
|||||||
finally:
|
finally:
|
||||||
own.close()
|
own.close()
|
||||||
|
|
||||||
name = "usage_%s_%s.csv" % (frm or "all", to or db.now_str()[:10])
|
# 文件名带账号名:多人导出到同一目录时不会互相覆盖
|
||||||
|
name = "usage_%s_%s_%s.csv" % (u["username"], frm or "all", to or db.now_str()[:10])
|
||||||
resp = Response(gen(), mimetype="text/csv; charset=utf-8",
|
resp = Response(gen(), mimetype="text/csv; charset=utf-8",
|
||||||
headers={"Content-Disposition": 'attachment; filename="%s"' % name})
|
headers={"Content-Disposition": 'attachment; filename="%s"' % name})
|
||||||
# 导出可能很慢,避免 nginx 之类的前置代理先缓冲整个响应体
|
# 导出可能很慢,避免 nginx 之类的前置代理先缓冲整个响应体
|
||||||
|
|||||||