From 73c292b7c2c936a0c6f5e12444919f4011661441 Mon Sep 17 00:00:00 2001 From: Peanut Date: Sun, 17 May 2026 09:54:17 +0800 Subject: [PATCH] docs: add analytics and tts design spec --- .../2026-05-17-analytics-and-tts-design.md | 403 ++++++++++++++++++ 1 file changed, 403 insertions(+) create mode 100644 docs/superpowers/specs/2026-05-17-analytics-and-tts-design.md diff --git a/docs/superpowers/specs/2026-05-17-analytics-and-tts-design.md b/docs/superpowers/specs/2026-05-17-analytics-and-tts-design.md new file mode 100644 index 0000000..de4b742 --- /dev/null +++ b/docs/superpowers/specs/2026-05-17-analytics-and-tts-design.md @@ -0,0 +1,403 @@ +# Analytics and Text-to-Speech Design + +Date: 2026-05-17 + +## Goal + +Add two complete but independently shippable capabilities: + +1. Mini program analytics: collect user behavior events from the user-facing mini program and show behavior, funnel, and preference statistics in `web-admin`. +2. Script text-to-speech: let users listen to generated scripts and creative content through a CPU-friendly open-source TTS service deployed on `101.200.208.45`. + +The implementation should fit the current project shape: + +- `backend-single`: Spring Boot 2.7, MyBatis Plus, MySQL, Redis. +- `mini-program`: uni-app/Vue. +- `web-admin`: Vue, Element Plus, ECharts. +- Deployment target: `101.200.208.45` with passwordless SSH. + +## Scope + +### Change 1: Analytics + +The mini program gets a lightweight analytics SDK. It collects automatic page/session events and explicit business events, then sends batched events to the backend. The backend stores raw events in MySQL and exposes admin-only aggregation APIs. `web-admin` adds a behavior analytics page. + +### Change 2: Text-to-Speech + +The mini program gets a script reading control on the script detail page. The backend owns task creation, permission checks, text cleaning, audio cache lookup, and public audio URLs. A local Python FastAPI service backed by an open-source CPU-friendly TTS engine generates the audio files. + +TTS-related actions are also tracked as analytics events, so admins can see which content is most often requested and played. + +## Recommended Approach + +Use the lightweight in-project approach rather than adding a separate analytics platform or data warehouse. + +Reasons: + +- The requested analysis is business-specific: script style, inspiration prompt, life event tags, path mapping, and reading behavior. +- MySQL aggregation is enough for the first version and matches the existing backend. +- Operational complexity stays low. +- The TTS service can be swapped later without changing mini program behavior. + +For TTS, use MeloTTS or another CPU-friendly open-source Chinese TTS service for the first version. The server has no GPU, so heavier real-time models such as CosyVoice are not the default production path. The backend interface should remain engine-neutral so a better engine can replace it later. + +## Data Model + +### `t_analytics_event` + +Stores raw user behavior events. + +Fields: + +- `id`: primary key. +- `user_id`: nullable current user id from token. +- `anonymous_id`: long-lived local mini program id for anonymous or pre-login behavior. +- `session_id`: id for one app launch/session. +- `event_name`: event key, for example `page_view`, `script_generate_success`. +- `event_type`: broad category, for example `page`, `script`, `life_event`, `tts`. +- `page_path`: current page. +- `referrer_path`: previous page. +- `properties`: JSON object for business attributes such as script style, length, tags, button source, error code. +- `device_info`: JSON object for platform, model, system version, SDK version, WeChat version. +- `duration_ms`: optional page stay time, generation time, or playback time. +- `occurred_at`: client event time. +- `server_time`: backend receive time. +- common fields: `create_time`, `update_time`, `is_deleted`, `remarks`. + +Indexes: + +- `idx_analytics_event_name` +- `idx_analytics_event_type` +- `idx_analytics_event_user_id` +- `idx_analytics_event_anonymous_id` +- `idx_analytics_event_occurred_at` +- `idx_analytics_event_name_time (event_name, occurred_at)` +- `idx_analytics_event_user_time (user_id, occurred_at)` + +### `t_tts_task` + +Stores text-to-speech generation tasks and cache metadata. + +Fields: + +- `id`: primary key. +- `user_id`: owner. +- `source_type`: first version uses `epic_script`. +- `source_id`: related script id. +- `text_hash`: hash of cleaned text plus voice. +- `text_length`: cleaned text length. +- `voice`: selected voice, default Mandarin voice. +- `status`: `pending`, `processing`, `success`, `failed`. +- `audio_url`: URL returned to mini program. +- `audio_path`: server file path. +- `duration_ms`: generated audio duration if available. +- `error_message`: failure reason for retry/debug. +- `request_count`: number of times this cached audio has been requested. +- common fields: `create_time`, `update_time`, `is_deleted`, `remarks`. + +Indexes: + +- `idx_tts_task_user_source (user_id, source_type, source_id)` +- `idx_tts_task_text_hash (text_hash)` +- `idx_tts_task_status (status)` +- `idx_tts_task_create_time (create_time)` + +Successful tasks with the same `text_hash + voice` should reuse audio instead of regenerating. + +## Backend API Design + +### Mini Program Analytics API + +`POST /analytics/events/batch` + +Accepts batched mini program events. It is available before and after login. If a token is present, the backend fills `user_id`; otherwise the event is stored with `anonymous_id`. + +Request shape: + +```json +{ + "anonymousId": "anon_xxx", + "sessionId": "session_xxx", + "deviceInfo": {}, + "events": [ + { + "eventName": "script_detail_view", + "eventType": "script", + "pagePath": "/pages/main/ScriptDetailView", + "referrerPath": "/pages/main/ScriptView", + "properties": {}, + "durationMs": 12000, + "occurredAt": "2026-05-17 09:30:00" + } + ] +} +``` + +Validation rules: + +- Max 50 events per request. +- Event names must match a safe naming pattern such as lowercase letters, digits, and underscores. +- `properties` and `device_info` have strict size limits. +- Sensitive fields are not accepted. +- Invalid individual events can be skipped while valid events are stored, with a response count. + +### Admin Analytics API + +All endpoints are under `/admin/analytics/**` and require admin auth. + +- `GET /admin/analytics/overview`: PV, UV, event count, active users, average stay time, TTS request/play counts. +- `GET /admin/analytics/trend`: daily or hourly event trend for selected event names. +- `GET /admin/analytics/funnel`: key funnel steps, first version: + `app_launch -> profile_complete -> life_event_create -> script_generate_success -> path_select`. +- `GET /admin/analytics/preferences`: script style, script length, inspiration prompt type, life event tags, and TTS source popularity. +- `GET /admin/analytics/top-events`: hot events and error events. +- `GET /admin/analytics/users`: user behavior ranking by event count, recent activity, and selected event filters. + +Default query window is 7 days. Maximum query window is 90 days. + +### Mini Program TTS API + +`POST /tts/tasks` + +Creates or reuses a TTS task. + +Request: + +```json +{ + "sourceType": "epic_script", + "sourceId": "script_id", + "voice": "default_zh_female" +} +``` + +Behavior: + +- Validate the current user owns the source content. +- Load the script content from `t_epic_script`. +- Clean Markdown and unsupported symbols. +- Enforce text length limit. +- Compute `text_hash`. +- If a successful matching task exists, return `success` and `audioUrl`. +- If a pending or processing matching task exists, return its task id. +- Otherwise create a pending task and process it asynchronously. + +`GET /tts/tasks/{id}` + +Returns task status and audio URL if successful. + +`GET /tts/tasks/by-source` + +Checks whether a source already has a TTS task for the current user and selected voice. + +### Internal Python TTS API + +The Python service listens only on localhost or private network. + +`POST http://127.0.0.1:19110/synthesize` + +Request: + +```json +{ + "text": "cleaned text", + "voice": "default_zh_female", + "outputPath": "/data/uploads/emotion-museum/tts/hash.mp3" +} +``` + +Response: + +```json +{ + "success": true, + "audioPath": "/data/uploads/emotion-museum/tts/hash.mp3", + "durationMs": 123456, + "engine": "melotts" +} +``` + +The Python service is not exposed directly to the mini program. + +## Mini Program Design + +### Analytics SDK + +Add `mini-program/src/services/analytics.js`. + +Responsibilities: + +- `initAnalytics()`: create or restore `anonymous_id`, create `session_id`, capture device info. +- `track(eventName, properties)`: queue a business event. +- `trackPageView(pagePath, options)`: record page enter. +- `trackPageLeave(pagePath)`: record page leave and stay duration. +- `flush()`: send queued events to `/analytics/events/batch`. + +Queue behavior: + +- Keep at most 100 local queued events. +- Flush every 10 seconds, when 20 events are queued, when the app goes to background, and before page unload where supported. +- Failed requests keep the queue for retry. +- Analytics failures never block product flows. + +Initial event coverage: + +- App/session: launch, show, hide. +- Page: view and leave for main pages. +- Script: inspiration view/click, generate start/success/failure, detail view, select/path mapping. +- Life event: create, update, AI assist, favorite, share. +- TTS: request, task success, play, pause, complete, error. + +### TTS Player UI + +Add a script-specific audio component, for example `ScriptAudioPlayer.vue`, instead of mixing with the existing background `MusicPlayer.vue`. + +States: + +- No audio: show generate reading action. +- Pending/processing: show generating state. +- Success: show play/pause. +- Failed: show retry. + +Playback uses `uni.createInnerAudioContext()`. The component releases audio resources when leaving the page. + +`ScriptDetailView.vue` adds this component in the hero area or bottom action area. + +## Web Admin Design + +Add route and menu item for `/analytics`. + +Create a first-version behavior analytics page with existing Element Plus and ECharts patterns. + +Sections: + +- Overview cards: PV, UV, active users, event count, average stay, TTS requests, TTS plays. +- Trend chart: selected core events by day. +- Funnel chart: launch to profile, event creation, script generation, path selection. +- Preference panels: script style, script length, inspiration type, life event tags, most-read content. +- Top events table: event name, count, users, failure rate where applicable. + +The page must handle empty data, sparse data, and loading/error states. + +## Privacy and Safety + +Do not collect: + +- Phone numbers. +- Tokens. +- SMS codes. +- Full script text. +- Full life event text. +- Long exception stack traces. +- Any sensitive identity fields. + +Preference analysis stores only enumerations and small metadata, for example `style=career`, `length=medium`, `tag=growth`, `word_count=1200`. + +First version user analysis is behavior aggregation only, not sensitive psychological or identity inference. + +## Performance and Reliability + +Analytics: + +- Batch insert events. +- Keep admin queries bounded by date range. +- Default analytics range is 7 days. +- Maximum analytics range is 90 days. +- Add indexes for event/time and user/time queries. +- Consider daily aggregate tables only after raw event volume becomes a problem. + +TTS: + +- Per-user generation rate limit, for example 2 per minute and 20 per day. +- Text length limit, for example 5000 cleaned characters. +- Python TTS concurrency limit of 1 or 2 workers on CPU. +- Task timeout marks the task as `failed`. +- Audio cache uses `text_hash + voice`. +- Old unused audio files can be cleaned later by scheduled job. +- Python service is private and managed with systemd. + +## Deployment + +Change 1 deployment: + +1. Add SQL migration for analytics table. +2. Deploy backend analytics write API. +3. Deploy admin analytics aggregation API. +4. Deploy web-admin analytics page. +5. Deploy mini program analytics SDK and event coverage. + +Change 2 deployment: + +1. Install Python runtime and TTS service on `101.200.208.45`. +2. Add systemd service for the TTS FastAPI app. +3. Add health check for local TTS service. +4. Add SQL migration for TTS task table. +5. Deploy backend TTS task APIs. +6. Deploy mini program audio player. +7. Connect TTS events to analytics. + +## Testing Plan + +Analytics backend: + +- Batch event insert succeeds. +- Anonymous events are accepted. +- Logged-in events fill `user_id`. +- Oversized batches are rejected or capped. +- Oversized JSON properties are rejected. +- Invalid event names are rejected. +- Admin overview/trend/funnel/preference calculations are correct on seeded data. + +Mini program analytics: + +- Launch and background events flush. +- Page view and leave events include duration. +- Script generation and detail events are captured. +- Network failure keeps queued events and retries later. +- Analytics failures do not show user-facing errors. + +Web admin: + +- Empty data renders cleanly. +- Sparse data renders cleanly. +- Seven-day trends render. +- Filters refresh data. +- Permission guard works through existing admin auth. + +TTS backend: + +- Current user can create task for own script. +- User cannot create task for another user's script. +- Existing successful cache returns immediately. +- Pending task returns task id. +- Failed task exposes retry path. +- Text length limit works. +- Python service failure marks task failed. + +Python TTS: + +- Short Chinese text generates audio. +- Long Chinese text within limit generates audio. +- Output file exists at requested path. +- Health check returns healthy. +- CPU and memory usage are acceptable on the server. + +Mini program TTS: + +- Detail page shows correct state for no task, generating, success, and failed. +- Play, pause, complete, and error events are tracked. +- Leaving page releases audio resources. + +## Open Decisions for Implementation + +- Exact TTS engine package and model version after a quick server resource check. +- Exact voice identifiers exposed in the first version. +- Whether long scripts should be truncated in version one or split into multiple audio segments. +- Whether admin analytics should live as one `/analytics` page or split into sub-tabs. + +The default implementation should choose the simplest working options: + +- MeloTTS/FastAPI for CPU-friendly generation. +- One default Mandarin voice. +- Text length limit with clear retry/failure message, no segment stitching in version one. +- One admin analytics page with internal sections.