peanut
89fc42819d
- 新增 AI 场景路由控制器和管理接口 - 新增 ASR 语音识别服务及前后端集成 - 同步 AI Runtime 客户端到 Web/小程序/Life-Script - 完善 AI 配置测试修复和管理后台路由配置 - 新增数据库迁移脚本 Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
65 lines
3.0 KiB
Markdown
65 lines
3.0 KiB
Markdown
# Mini Program ASR Service Implementation Plan
|
|
|
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
|
|
**Goal:** Deploy a private Chinese ASR service on `101.200.208.45` and wire the mini program voice orb to transcribe speech into the wish input.
|
|
|
|
**Architecture:** A local FastAPI ASR service listens on `127.0.0.1:19120` and uses FunASR/SenseVoiceSmall when available. The Java backend exposes `/api/asr/transcribe` as the authenticated upload proxy, and the mini program records audio with `uni.getRecorderManager()` then uploads it through the existing request service.
|
|
|
|
**Tech Stack:** FunASR/SenseVoiceSmall, Python 3.11, FastAPI, Spring Boot multipart upload, uni-app recorder/upload APIs.
|
|
|
|
---
|
|
|
|
### Task 1: ASR Service
|
|
|
|
**Files:**
|
|
- Create: `backend-single/asr-service/app.py`
|
|
- Create: `backend-single/asr-service/requirements.txt`
|
|
- Create: `backend-single/asr-service/emotion-museum-asr.service`
|
|
- Create: `backend-single/asr-service/README.md`
|
|
|
|
- [x] Add a FastAPI service with `/health` and `/transcribe`.
|
|
- [x] Accept an uploaded audio file, save it under `/tmp/emotion-museum-asr`, run the ASR model, and return JSON with `success`, `text`, `language`, `durationMs`, `engine`, and `errorMessage`.
|
|
- [x] Keep the service bound to `127.0.0.1:19120`.
|
|
|
|
### Task 2: Java Backend Proxy
|
|
|
|
**Files:**
|
|
- Create: `backend-single/src/main/java/com/emotion/dto/response/asr/AsrTranscribeResponse.java`
|
|
- Create: `backend-single/src/main/java/com/emotion/service/AsrService.java`
|
|
- Create: `backend-single/src/main/java/com/emotion/service/impl/AsrServiceImpl.java`
|
|
- Create: `backend-single/src/main/java/com/emotion/controller/AsrController.java`
|
|
- Modify: `backend-single/src/main/resources/application.yml`
|
|
- Modify: `backend-single/src/main/resources/application-prod.yml`
|
|
|
|
- [x] Add `emotion.asr` config for `enabled`, `engine-url`, `max-file-size`, and `allowed-types`.
|
|
- [x] Validate upload presence, size, and suffix.
|
|
- [x] Forward multipart audio to the local ASR service.
|
|
- [x] Return a normal `Result<AsrTranscribeResponse>` to the mini program.
|
|
|
|
### Task 3: Mini Program Recording
|
|
|
|
**Files:**
|
|
- Create: `mini-program/src/services/asr.js`
|
|
- Modify: `mini-program/src/pages/main/ScriptView.vue`
|
|
|
|
- [x] Use `uni.getRecorderManager()` to start recording on press.
|
|
- [x] Stop recording on release/cancel.
|
|
- [x] Upload the recorded temp file to `/asr/transcribe`.
|
|
- [x] Fill `wishText` with recognized text and keep the current visual theme.
|
|
- [x] Track success/failure analytics.
|
|
|
|
### Task 4: Deploy And Verify
|
|
|
|
**Commands:**
|
|
- `npm run build:mp-weixin`
|
|
- `mvn -DskipTests package`
|
|
- Deploy backend jar to `101.200.208.45`.
|
|
- Install/start `emotion-museum-asr`.
|
|
- Check `curl http://127.0.0.1:19120/health`.
|
|
|
|
**Acceptance:**
|
|
- Pressing and releasing the mini program voice orb sends audio to backend ASR.
|
|
- Recognized Chinese text appears in the wish input.
|
|
- Server ASR and TTS services both remain active.
|