diff --git a/README.md b/README.md index e9e0bb4c..4011362e 100644 --- a/README.md +++ b/README.md @@ -198,9 +198,9 @@ Every item below is one more layer sedimented into a default — a capability yo - 🎨 **Style Pack Marketplace** — browse, install, and like community **style packs** from the in-app Marketplace, and publish your own (custom system prompt per pack, switchable by hotkey). Backed by a moderated marketplace backend; uploads are reviewed before they go public. - ⚡ **Streaming insertion** — polished text is written to the cursor character by character to reduce perceived latency, with an automatic one-shot-paste fallback. Toggle in Settings → Recording. - **Toggle and push-to-talk** recording modes, plus a **MediaPlayPause trigger** so wired-earbud inline controls can start and stop recording. `Esc` cancels at any phase, including polish and insertion. -- **Cloud ASR**: Volcengine streaming ASR (bigasr), iFlytek realtime ASR (RTASR), Alibaba Cloud Bailian (classic realtime / Qwen3 realtime / Fun-ASR-Flash file transcription), StepFun StepAudio (batch + realtime), Zhipu GLM-ASR, Xiaomi MiMo ASR, OrcaRouter audio-input Gemini, ElevenLabs Scribe, OpenAI-compatible batch transcription (OpenAI Whisper / Groq / SiliconFlow SenseVoice / OpenRouter / ZenMux), and Apple Speech (macOS). +- **Cloud ASR**: Volcengine streaming ASR (bigasr), Tencent Cloud Hunyuan realtime ASR (Hy-ASR), iFlytek realtime ASR (RTASR), Alibaba Cloud Bailian (classic realtime / Qwen3 realtime / Fun-ASR-Flash file transcription), StepFun StepAudio (batch + realtime), Zhipu GLM-ASR, Xiaomi MiMo ASR, OrcaRouter audio-input Gemini, ElevenLabs Scribe, OpenAI-compatible batch transcription (OpenAI Whisper / Groq / SiliconFlow SenseVoice / OpenRouter / ZenMux), and Apple Speech (macOS). - **Local ASR**: bundled Qwen3-ASR (0.6B / 1.7B) via vendored `Open-Less/qwen-asr` (macOS); Windows Foundry Local Whisper and sherpa-onnx (experimental) variants. -- **Polish providers**: Ark (Volcengine), DeepSeek, OpenAI, Google Gemini, Codex OAuth, SiliconFlow, Atlas Cloud, Xiaomi MiMo, CometAPI, OpenRouter, OrcaRouter, Alibaba Cloud Coding Plan, CodingPlanX, MiniMax, StepFun, and OpenCode Zen — plus any OpenAI-compatible endpoint you bring. +- **Polish providers**: Ark (Volcengine), DeepSeek, OpenAI, Google Gemini, Codex OAuth, SiliconFlow, Atlas Cloud, Xiaomi MiMo, Tencent Cloud TokenHub, CometAPI, OpenRouter, OrcaRouter, Alibaba Cloud Coding Plan, CodingPlanX, MiniMax, StepFun, and OpenCode Zen — plus any OpenAI-compatible endpoint you bring. - **Four output modes**: raw, light polish, structured (**AI-prompt mode**), and formal. Plus a **translation hotkey** that converts speech directly into the configured target language ([#43](../../issues/43)). - **Selection-ask QA panel** — a separate hotkey opens a floating panel that runs voice Q&A against the highlighted text in any app ([#118](../../issues/118)). - **Main window**: Overview / History / Vocab / Style / Marketplace / Settings. Persistent tray icon, plus a mini status capsule that floats on screen and follows the display you are typing on (multi-monitor). diff --git a/README.zh.md b/README.zh.md index 4dde3063..88759401 100644 --- a/README.zh.md +++ b/README.zh.md @@ -203,9 +203,9 @@ OpenLess 只做一件事:**把语音变成可用的书面文字(尤其是 AI 提 - 🎨 **风格包市场**——在应用内的 Marketplace 浏览、安装、点赞社区**风格包**,并发布自己的(每个包一套自定义系统提示词,可用快捷键切换)。由经过审核的市场后端支撑;上传内容公开前会经过审核。 - ⚡ **流式插入**——润色后的文本逐字符写入光标以降低感知延迟,并带有自动的一次性粘贴回退。可在 设置 → 录音 中切换。 - **切换式与按住说话(push-to-talk)** 两种录音模式,外加 **MediaPlayPause 触发**,让有线耳机的线控也能开始 / 停止录音。`Esc` 可在任意阶段取消,包括润色与插入。 -- **云端 ASR**:Volcengine 流式 ASR(bigasr)、讯飞实时语音转写(RTASR)、阿里云百炼(经典实时 / Qwen3 实时 / Fun-ASR-Flash 录音文件)、阶跃星辰 StepAudio(批式 + 实时)、智谱 GLM-ASR、小米 MiMo ASR、OrcaRouter 音频输入 Gemini、ElevenLabs Scribe、OpenAI 兼容批量转写(OpenAI Whisper / Groq / 硅基流动 SenseVoice / OpenRouter / ZenMux),以及 Apple Speech(macOS)。 +- **云端 ASR**:Volcengine 流式 ASR(bigasr)、腾讯云混元实时 ASR(Hy-ASR)、讯飞实时语音转写(RTASR)、阿里云百炼(经典实时 / Qwen3 实时 / Fun-ASR-Flash 录音文件)、阶跃星辰 StepAudio(批式 + 实时)、智谱 GLM-ASR、小米 MiMo ASR、OrcaRouter 音频输入 Gemini、ElevenLabs Scribe、OpenAI 兼容批量转写(OpenAI Whisper / Groq / 硅基流动 SenseVoice / OpenRouter / ZenMux),以及 Apple Speech(macOS)。 - **本地 ASR**:通过 vendored 的 `Open-Less/qwen-asr` 内置 Qwen3-ASR(0.6B / 1.7B)(macOS);Windows 上的 Foundry Local Whisper 与 sherpa-onnx(实验性)变体。 -- **润色提供方**:Ark(火山方舟)、DeepSeek、OpenAI、Google Gemini、Codex OAuth、硅基流动、Atlas Cloud、小米 MiMo、CometAPI、OpenRouter、OrcaRouter、阿里云 Coding Plan、CodingPlanX、MiniMax、StepFun、OpenCode Zen,以及你自带的任意 OpenAI 兼容端点。 +- **润色提供方**:Ark(火山方舟)、DeepSeek、OpenAI、Google Gemini、Codex OAuth、硅基流动、Atlas Cloud、小米 MiMo、腾讯云 TokenHub、CometAPI、OpenRouter、OrcaRouter、阿里云 Coding Plan、CodingPlanX、MiniMax、StepFun、OpenCode Zen,以及你自带的任意 OpenAI 兼容端点。 - **四种输出模式**:原文、轻度润色、结构化(**AI 提示词模式**)、正式。另有一个**翻译快捷键**,将语音直接转换为所配置的目标语言([#43](../../issues/43))。 - **选区问答面板**——一个独立快捷键打开浮动面板,针对任意应用中被高亮选中的文本进行语音问答([#118](../../issues/118))。 - **主窗口**:概览 / 历史 / 词典 / 风格 / 市场 / 设置。常驻托盘图标,以及一个浮于屏幕、并跟随你正在输入的显示器的迷你状态胶囊(多显示器)。 diff --git a/openless-all/app/crates/openless-core/src/asr/mod.rs b/openless-all/app/crates/openless-core/src/asr/mod.rs index b016e1b1..a8177bfe 100644 --- a/openless-all/app/crates/openless-core/src/asr/mod.rs +++ b/openless-all/app/crates/openless-core/src/asr/mod.rs @@ -8,6 +8,7 @@ pub mod mimo; pub mod pcm; pub mod qwen_realtime; pub mod stepfun_realtime; +pub mod tencent_cloud; pub mod volcengine; pub mod wav; pub mod whisper; @@ -20,6 +21,7 @@ pub use elevenlabs::ElevenLabsBatchASR; pub use mimo::MimoBatchASR; pub use qwen_realtime::{Qwen3RealtimeASR, Qwen3RealtimeCredentials}; pub use stepfun_realtime::{StepfunRealtimeASR, StepfunRealtimeCredentials}; +pub use tencent_cloud::{TencentCloudCredentials, TencentCloudStreamingASR}; pub use volcengine::{VolcengineCredentials, VolcengineStreamingASR}; pub use whisper::WhisperBatchASR; pub use xfyun::{XfyunCredentials, XfyunStreamingASR}; diff --git a/openless-all/app/crates/openless-core/src/asr/tencent_cloud.rs b/openless-all/app/crates/openless-core/src/asr/tencent_cloud.rs new file mode 100644 index 00000000..4832843d --- /dev/null +++ b/openless-all/app/crates/openless-core/src/asr/tencent_cloud.rs @@ -0,0 +1,1000 @@ +//! 腾讯云实时语音识别 WebSocket 客户端。 +//! +//! 官方文档：https://cloud.tencent.com/document/api/1093/48982 +//! +//! 协议要点： +//! - 端点：`wss://asr.cloud.tencent.com/asr/v2/`； +//! - 鉴权：查询参数按字典序拼接后，以 SecretKey 做 HMAC-SHA1，再 Base64； +//! - 音频：16 kHz / 16-bit / 单声道 PCM，每 200ms 发送 6400 bytes； +//! - 收尾：发送文本消息 `{"type":"end"}`，等待 `final=1`； +//! - 默认模型：`Hy-ASR-3.0-preview`（腾讯云当前最新混元 ASR Preview）。 + +use std::collections::BTreeMap; +use std::sync::Arc; +use std::time::Duration; + +use base64::Engine; +use futures_util::{SinkExt, StreamExt}; +use hmac::{Hmac, Mac}; +use parking_lot::Mutex as ParkingMutex; +use serde_json::Value; +use sha1::Sha1; +use tokio::net::TcpStream; +use tokio::sync::{mpsc, oneshot, Mutex as AsyncMutex}; +use tokio_tungstenite::tungstenite::client::IntoClientRequest; +use tokio_tungstenite::tungstenite::Message; +use tokio_tungstenite::{connect_async, MaybeTlsStream, WebSocketStream}; +use uuid::Uuid; + +use crate::config::{TaskSpawner, TokioTaskSpawner}; +use crate::ports::{TextStreamChunk, TextStreamSink}; + +use super::{AudioConsumer, RawTranscript}; + +pub const PROVIDER_ID: &str = "tencent-cloud"; +pub const DEFAULT_ENDPOINT: &str = "wss://asr.cloud.tencent.com/asr/v2"; +pub const DEFAULT_MODEL: &str = "Hy-ASR-3.0-preview"; +pub const TARGET_AUDIO_CHUNK_BYTES: usize = 6_400; + +const BYTES_PER_MS: u64 = 32; +const MAX_AUDIO_SEND_RATE: u64 = 2; +const CONNECT_TIMEOUT: Duration = Duration::from_secs(5); +const HANDSHAKE_TIMEOUT: Duration = Duration::from_secs(5); +const FRAME_SEND_TIMEOUT: Duration = Duration::from_secs(5); +const FINAL_RESULT_TIMEOUT: Duration = Duration::from_secs(15); + +type WsStream = WebSocketStream>; +type WsSink = futures_util::stream::SplitSink; +type SharedWriter = Arc>>; + +enum SendItem { + Audio(Vec), + End(oneshot::Sender>), +} + +#[derive(Clone, Debug)] +pub struct TencentCloudCredentials { + pub app_id: String, + pub secret_id: String, + pub secret_key: String, + pub model: String, +} + +impl TencentCloudCredentials { + pub fn auth_ok(&self) -> bool { + !self.app_id.trim().is_empty() + && !self.secret_id.trim().is_empty() + && !self.secret_key.trim().is_empty() + } + + fn resolved_model(&self) -> &str { + let model = self.model.trim(); + if model.is_empty() { + DEFAULT_MODEL + } else { + model + } + } +} + +#[derive(Debug, Clone, thiserror::Error)] +pub enum TencentCloudASRError { + #[error("credentials missing")] + CredentialsMissing, + #[error("connection failed")] + ConnectionFailed, + #[error("credentials rejected or service unavailable ({0})")] + AuthRejected(i64), + #[error("account unavailable ({0})")] + AccountUnavailable(i64), + #[error("rate limited")] + RateLimited, + #[error("service unavailable ({0})")] + ServiceUnavailable(i64), + #[error("request failed ({0})")] + TaskFailed(i64), + #[error("no final result")] + NoFinalResult, + #[error("final result timed out")] + FinalResultTimeout, +} + +#[derive(Default)] +struct SyncState { + pending_audio: Vec, + bytes_sent: u64, + started: bool, + finished: bool, + final_tx: Option>>, + final_segments: BTreeMap, + partial_segments: BTreeMap, + last_result_text: String, +} + +pub struct TencentCloudStreamingASR { + credentials: TencentCloudCredentials, + endpoint: String, + task_spawner: Arc, + state: ParkingMutex, + writer: SharedWriter, + final_rx: ParkingMutex>>>, + handshake_tx: ParkingMutex>>>, + send_tx: ParkingMutex>>, + partial_sink: ParkingMutex>>, +} + +impl TencentCloudStreamingASR { + pub fn new(credentials: TencentCloudCredentials) -> Self { + Self::with_task_spawner(credentials, Arc::new(TokioTaskSpawner)) + } + + pub fn with_task_spawner( + credentials: TencentCloudCredentials, + task_spawner: Arc, + ) -> Self { + Self::with_endpoint(credentials, task_spawner, DEFAULT_ENDPOINT.to_string()) + } + + fn with_endpoint( + credentials: TencentCloudCredentials, + task_spawner: Arc, + endpoint: String, + ) -> Self { + Self { + credentials, + endpoint, + task_spawner, + state: ParkingMutex::new(SyncState::default()), + writer: Arc::new(AsyncMutex::new(None)), + final_rx: ParkingMutex::new(None), + handshake_tx: ParkingMutex::new(None), + send_tx: ParkingMutex::new(None), + partial_sink: ParkingMutex::new(None), + } + } + + pub fn set_partial_sink(&self, sink: Arc) { + *self.partial_sink.lock() = Some(sink); + } + + pub fn connect_url(&self) -> String { + connect_url_at( + &self.endpoint, + &self.credentials, + chrono::Utc::now().timestamp(), + random_nonce(), + Uuid::new_v4().to_string(), + ) + } + + pub async fn open_session(self: &Arc) -> Result<(), TencentCloudASRError> { + if !self.credentials.auth_ok() { + return Err(TencentCloudASRError::CredentialsMissing); + } + + let request = self + .connect_url() + .into_client_request() + .map_err(|_| TencentCloudASRError::ConnectionFailed)?; + let (ws, _) = tokio::time::timeout(CONNECT_TIMEOUT, connect_async(request)) + .await + .map_err(|_| TencentCloudASRError::ConnectionFailed)? + .map_err(|_| TencentCloudASRError::ConnectionFailed)?; + let (write, read) = ws.split(); + *self.writer.lock().await = Some(write); + + let (final_tx, final_rx) = oneshot::channel(); + let (send_tx, mut send_rx) = mpsc::unbounded_channel::(); + let (handshake_tx, handshake_rx) = oneshot::channel(); + { + let mut state = self.state.lock(); + *state = SyncState::default(); + state.final_tx = Some(final_tx); + } + *self.final_rx.lock() = Some(final_rx); + *self.send_tx.lock() = Some(send_tx); + *self.handshake_tx.lock() = Some(handshake_tx); + + let writer = Arc::clone(&self.writer); + let weak_self = Arc::downgrade(self); + let task_spawner = Arc::clone(&self.task_spawner); + task_spawner.spawn(Box::pin(async move { + let mut next_audio_send_at = None; + while let Some(item) = send_rx.recv().await { + let (result, done) = match item { + SendItem::Audio(chunk) => { + if let Some(send_at) = next_audio_send_at { + tokio::time::sleep_until(send_at).await; + } + let interval_ms = + (chunk.len() as u64 / BYTES_PER_MS / MAX_AUDIO_SEND_RATE).max(1); + let result = send_binary(&writer, chunk).await; + next_audio_send_at = + Some(tokio::time::Instant::now() + Duration::from_millis(interval_ms)); + (result, None) + } + SendItem::End(done) => { + (send_text(&writer, r#"{"type":"end"}"#).await, Some(done)) + } + }; + if let Err(error) = &result { + log::error!("[tencent-cloud-asr] websocket send failed"); + if let Some(this) = weak_self.upgrade() { + this.finish_error(error.clone()); + } + } + if let Some(done) = done { + let _ = done.send(result.clone()); + } + if result.is_err() { + break; + } + } + })); + + let weak_self = Arc::downgrade(self); + let task_spawner = Arc::clone(&self.task_spawner); + task_spawner.spawn(Box::pin(async move { + let mut read = read; + while let Some(message) = read.next().await { + let Some(this) = weak_self.upgrade() else { + break; + }; + match message { + Ok(Message::Text(text)) => { + if !this.handle_text_message(&text) { + break; + } + } + Ok(Message::Close(_)) => { + this.finish_on_close(); + break; + } + Ok(_) => {} + Err(_) => { + log::error!("[tencent-cloud-asr] receive loop failed"); + this.finish_with_partial_or_error(TencentCloudASRError::ConnectionFailed); + break; + } + } + } + })); + + match tokio::time::timeout(HANDSHAKE_TIMEOUT, handshake_rx).await { + Ok(Ok(Ok(()))) => Ok(()), + Ok(Ok(Err(error))) => { + self.cancel(); + Err(error) + } + Ok(Err(_)) => { + self.cancel(); + Err(TencentCloudASRError::ConnectionFailed) + } + Err(_) => { + self.cancel(); + Err(TencentCloudASRError::ConnectionFailed) + } + } + } + + pub async fn send_last_frame(&self) -> Result<(), TencentCloudASRError> { + self.flush_pending_audio(); + let (done_tx, done_rx) = oneshot::channel(); + let sender = self + .send_tx + .lock() + .as_ref() + .cloned() + .ok_or(TencentCloudASRError::ConnectionFailed)?; + sender + .send(SendItem::End(done_tx)) + .map_err(|_| TencentCloudASRError::ConnectionFailed)?; + done_rx + .await + .map_err(|_| TencentCloudASRError::ConnectionFailed)? + } + + pub async fn await_final_result(&self) -> Result { + self.await_final_result_with_timeout(FINAL_RESULT_TIMEOUT) + .await + } + + pub async fn await_final_result_with_timeout( + &self, + timeout: Duration, + ) -> Result { + let Some(receiver) = self.final_rx.lock().take() else { + return Err(TencentCloudASRError::NoFinalResult); + }; + match tokio::time::timeout(timeout, receiver).await { + Ok(Ok(result)) => result, + Ok(Err(_)) => Err(TencentCloudASRError::NoFinalResult), + Err(_) => { + log::error!( + "[tencent-cloud-asr] final result timed out after {} ms", + timeout.as_millis() + ); + self.cancel(); + Err(TencentCloudASRError::FinalResultTimeout) + } + } + } + + pub fn cancel(&self) { + self.state.lock().pending_audio.clear(); + *self.handshake_tx.lock() = None; + *self.send_tx.lock() = None; + let writer = Arc::clone(&self.writer); + self.task_spawner.spawn(Box::pin(async move { + let mut guard = writer.lock().await; + if let Some(mut ws) = guard.take() { + let _ = ws.close().await; + } + })); + self.signal_error(TencentCloudASRError::NoFinalResult); + } + + fn flush_pending_audio(&self) { + let leftover = { + let mut state = self.state.lock(); + if state.pending_audio.is_empty() { + return; + } + let leftover = std::mem::take(&mut state.pending_audio); + state.bytes_sent += leftover.len() as u64; + leftover + }; + self.enqueue_audio(leftover); + } + + fn enqueue_audio(&self, chunk: Vec) { + let Some(sender) = self.send_tx.lock().as_ref().cloned() else { + return; + }; + if sender.send(SendItem::Audio(chunk)).is_err() { + self.signal_error(TencentCloudASRError::ConnectionFailed); + } + } + + fn handle_text_message(&self, text: &str) -> bool { + let value: Value = match serde_json::from_str(text) { + Ok(value) => value, + Err(error) => { + log::warn!("[tencent-cloud-asr] invalid json event: {error}"); + return true; + } + }; + let code = value.get("code").and_then(Value::as_i64).unwrap_or(-1); + if code != 0 { + let error = classify_server_error(code); + if let Some(sender) = self.handshake_tx.lock().take() { + let _ = sender.send(Err(error.clone())); + } + self.finish_error(error); + return false; + } + + self.mark_started(); + if let Some(result) = value.get("result") { + self.record_result(result); + } + if value.get("final").and_then(Value::as_i64) == Some(1) { + self.finish_success(); + return false; + } + true + } + + fn mark_started(&self) { + let mut state = self.state.lock(); + if !state.started { + state.started = true; + } + drop(state); + if let Some(sender) = self.handshake_tx.lock().take() { + let _ = sender.send(Ok(())); + } + } + + fn record_result(&self, result: &Value) { + let Some(text) = result.get("voice_text_str").and_then(Value::as_str) else { + return; + }; + let text = text.trim(); + if text.is_empty() { + return; + } + let index = result.get("index").and_then(Value::as_i64).unwrap_or(0); + let slice_type = result + .get("slice_type") + .and_then(Value::as_i64) + .unwrap_or(1); + let mut state = self.state.lock(); + state.last_result_text = text.to_string(); + let mut delta = None; + if slice_type == 2 { + state.final_segments.insert(index, text.to_string()); + state.partial_segments.remove(&index); + } else { + let previous = state + .partial_segments + .get(&index) + .map(String::as_str) + .unwrap_or(""); + delta = text + .strip_prefix(previous) + .filter(|suffix| !suffix.is_empty()) + .map(str::to_string); + state.partial_segments.insert(index, text.to_string()); + } + drop(state); + if let Some(delta) = delta { + if let Some(sink) = self.partial_sink.lock().clone() { + let _ = sink.publish(TextStreamChunk { + text: delta, + offset: 0, + }); + } + } + } + + fn finish_on_close(&self) { + if let Some(sender) = self.handshake_tx.lock().take() { + let _ = sender.send(Err(TencentCloudASRError::ConnectionFailed)); + return; + } + self.finish_with_partial_or_error(TencentCloudASRError::NoFinalResult); + } + + fn finish_with_partial_or_error(&self, error: TencentCloudASRError) { + let has_text = { + let state = self.state.lock(); + !state.last_result_text.trim().is_empty() + || !state.final_segments.is_empty() + || !state.partial_segments.is_empty() + }; + if has_text { + self.finish_success(); + } else { + self.finish_error(error); + } + } + + fn finish_success(&self) { + let (sender, text, duration_ms) = { + let mut state = self.state.lock(); + if state.finished { + return; + } + state.finished = true; + state.pending_audio.clear(); + let mut segments = state.final_segments.clone(); + for (index, text) in &state.partial_segments { + segments.entry(*index).or_insert_with(|| text.clone()); + } + let text = if segments.is_empty() { + state.last_result_text.clone() + } else { + super::mimo::join_transcript_chunks( + &segments.into_values().collect::>(), + ) + }; + (state.final_tx.take(), text, state.bytes_sent / BYTES_PER_MS) + }; + *self.send_tx.lock() = None; + if let Some(sender) = sender { + let _ = sender.send(Ok(RawTranscript { text, duration_ms })); + } + self.close_writer(); + } + + fn signal_error(&self, error: TencentCloudASRError) { + let sender = { + let mut state = self.state.lock(); + if state.finished { + return; + } + state.finished = true; + state.final_tx.take() + }; + *self.send_tx.lock() = None; + if let Some(sender) = sender { + let _ = sender.send(Err(error)); + } + } + + fn finish_error(&self, error: TencentCloudASRError) { + if let Some(sender) = self.handshake_tx.lock().take() { + let _ = sender.send(Err(error.clone())); + } + self.signal_error(error); + self.close_writer(); + } + + fn close_writer(&self) { + let writer = Arc::clone(&self.writer); + self.task_spawner.spawn(Box::pin(async move { + let mut guard = writer.lock().await; + if let Some(mut ws) = guard.take() { + let _ = ws.close().await; + } + })); + } +} + +impl AudioConsumer for TencentCloudStreamingASR { + fn consume_pcm_chunk(&self, pcm: &[u8]) { + let chunks = { + let mut state = self.state.lock(); + if !state.started || state.finished { + return; + } + state.pending_audio.extend_from_slice(pcm); + let mut chunks = Vec::new(); + while state.pending_audio.len() >= TARGET_AUDIO_CHUNK_BYTES { + let chunk = state + .pending_audio + .drain(..TARGET_AUDIO_CHUNK_BYTES) + .collect::>(); + state.bytes_sent += chunk.len() as u64; + chunks.push(chunk); + } + chunks + }; + for chunk in chunks { + self.enqueue_audio(chunk); + } + } +} + +async fn send_binary(writer: &SharedWriter, data: Vec) -> Result<(), TencentCloudASRError> { + let mut guard = writer.lock().await; + let Some(ws) = guard.as_mut() else { + return Err(TencentCloudASRError::ConnectionFailed); + }; + tokio::time::timeout(FRAME_SEND_TIMEOUT, ws.send(Message::Binary(data))) + .await + .map_err(|_| TencentCloudASRError::ConnectionFailed)? + .map_err(|_| TencentCloudASRError::ConnectionFailed) +} + +async fn send_text(writer: &SharedWriter, data: &str) -> Result<(), TencentCloudASRError> { + let mut guard = writer.lock().await; + let Some(ws) = guard.as_mut() else { + return Err(TencentCloudASRError::ConnectionFailed); + }; + tokio::time::timeout(FRAME_SEND_TIMEOUT, ws.send(Message::Text(data.to_string()))) + .await + .map_err(|_| TencentCloudASRError::ConnectionFailed)? + .map_err(|_| TencentCloudASRError::ConnectionFailed) +} + +fn connect_url_at( + base_endpoint: &str, + credentials: &TencentCloudCredentials, + timestamp: i64, + nonce: u32, + voice_id: String, +) -> String { + let endpoint = format!( + "{}/{}", + base_endpoint.trim_end_matches('/'), + credentials.app_id.trim() + ); + let parsed = url::Url::parse(&endpoint).expect("static Tencent Cloud endpoint parses"); + let host_and_path = format!( + "{}{}", + parsed.host_str().expect("Tencent Cloud endpoint has host"), + parsed.path() + ); + let params = BTreeMap::from([ + ( + "engine_model_type".to_string(), + credentials.resolved_model().to_string(), + ), + ("expired".to_string(), (timestamp + 86_400).to_string()), + ("filter_dirty".to_string(), "0".to_string()), + ("filter_empty_result".to_string(), "1".to_string()), + ("filter_modal".to_string(), "1".to_string()), + ("filter_punc".to_string(), "0".to_string()), + ("needvad".to_string(), "1".to_string()), + ("nonce".to_string(), nonce.to_string()), + ( + "secretid".to_string(), + credentials.secret_id.trim().to_string(), + ), + ("timestamp".to_string(), timestamp.to_string()), + ("voice_format".to_string(), "1".to_string()), + ("voice_id".to_string(), voice_id), + ]); + let query = query_string(¶ms); + let signing_text = format!("{host_and_path}?{query}"); + let signature = compute_signature(&signing_text, &credentials.secret_key); + format!("{endpoint}?{query}&signature={}", url_encode(&signature)) +} + +fn query_string(params: &BTreeMap) -> String { + params + .iter() + .map(|(key, value)| format!("{key}={value}")) + .collect::>() + .join("&") +} + +fn url_encode(value: &str) -> String { + url::form_urlencoded::byte_serialize(value.as_bytes()).collect() +} + +pub fn compute_signature(signing_text: &str, secret_key: &str) -> String { + let mut mac = Hmac::::new_from_slice(secret_key.trim().as_bytes()) + .expect("HMAC accepts any key length"); + mac.update(signing_text.as_bytes()); + base64::engine::general_purpose::STANDARD.encode(mac.finalize().into_bytes()) +} + +fn random_nonce() -> u32 { + let bytes = Uuid::new_v4().into_bytes(); + u32::from_le_bytes([bytes[0], bytes[1], bytes[2], bytes[3]]) % 1_000_000_000 + 1 +} + +fn classify_server_error(code: i64) -> TencentCloudASRError { + match code { + 4002 | 4003 => TencentCloudASRError::AuthRejected(code), + 4004 | 4005 => TencentCloudASRError::AccountUnavailable(code), + 4006 => TencentCloudASRError::RateLimited, + 4009 => TencentCloudASRError::ConnectionFailed, + 5000..=5002 => TencentCloudASRError::ServiceUnavailable(code), + _ => TencentCloudASRError::TaskFailed(code), + } +} + +#[cfg(test)] +mod tests { + use super::*; + use std::sync::atomic::{AtomicUsize, Ordering}; + + struct DelayedFirstSpawner { + spawned: AtomicUsize, + release: Arc, + } + + impl TaskSpawner for DelayedFirstSpawner { + fn spawn(&self, task: futures_util::future::BoxFuture<'static, ()>) { + if self.spawned.fetch_add(1, Ordering::SeqCst) == 0 { + let release = Arc::clone(&self.release); + tokio::spawn(async move { + release.acquire_owned().await.unwrap().forget(); + task.await; + }); + } else { + tokio::spawn(task); + } + } + } + + fn credentials() -> TencentCloudCredentials { + TencentCloudCredentials { + app_id: "1259220000".into(), + secret_id: "test-secret-id".into(), + secret_key: "key".into(), + model: DEFAULT_MODEL.into(), + } + } + + #[test] + fn hmac_sha1_signature_matches_known_vector() { + assert_eq!( + compute_signature("The quick brown fox jumps over the lazy dog", "key"), + "3nybhbi3iqa8ino29wqQcBydtNk=" + ); + } + + #[test] + fn connect_url_uses_sorted_signed_query_and_encodes_signature() { + let url = connect_url_at( + DEFAULT_ENDPOINT, + &credentials(), + 1_700_000_000, + 42, + "voice-id".into(), + ); + assert!(url.starts_with( + "wss://asr.cloud.tencent.com/asr/v2/1259220000?engine_model_type=Hy-ASR-3.0-preview&expired=1700086400" + )); + assert!(url.contains("&nonce=42&secretid=test-secret-id×tamp=1700000000")); + assert!(url.contains("&voice_format=1&voice_id=voice-id&signature=")); + assert!(!url.ends_with('=')); + } + + #[test] + fn stable_segments_replace_partials_and_keep_order() { + let asr = TencentCloudStreamingASR::new(credentials()); + asr.record_result(&serde_json::json!({ + "slice_type": 1, + "index": 1, + "voice_text_str": "世界" + })); + asr.record_result(&serde_json::json!({ + "slice_type": 2, + "index": 0, + "voice_text_str": "你好" + })); + let state = asr.state.lock(); + assert_eq!( + state.final_segments.get(&0).map(String::as_str), + Some("你好") + ); + assert_eq!( + state.partial_segments.get(&1).map(String::as_str), + Some("世界") + ); + } + + #[test] + fn server_errors_are_classified_for_actions() { + assert!(matches!( + classify_server_error(4002), + TencentCloudASRError::AuthRejected(_) + )); + assert!(matches!( + classify_server_error(4006), + TencentCloudASRError::RateLimited + )); + assert!(matches!( + classify_server_error(5000), + TencentCloudASRError::ServiceUnavailable(5000) + )); + } + + #[test] + fn auth_ok_requires_all_three_keys() { + assert!(credentials().auth_ok()); + assert!(!TencentCloudCredentials { + app_id: "app".into(), + secret_id: "id".into(), + secret_key: String::new(), + model: DEFAULT_MODEL.into(), + } + .auth_ok()); + } + + #[tokio::test] + async fn websocket_session_sends_all_audio_before_end_and_returns_final_text() { + let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); + let endpoint = format!("ws://{}/asr/v2", listener.local_addr().unwrap()); + let server = tokio::spawn(async move { + let (stream, _) = listener.accept().await.unwrap(); + let mut ws = tokio_tungstenite::accept_async(stream).await.unwrap(); + ws.send(Message::Text( + serde_json::json!({ "code": 0, "message": "success" }).to_string(), + )) + .await + .unwrap(); + + let first = ws.next().await.unwrap().unwrap(); + let second = ws.next().await.unwrap().unwrap(); + let end = ws.next().await.unwrap().unwrap(); + assert!(matches!(first, Message::Binary(bytes) if bytes.len() == 6_400)); + assert!(matches!(second, Message::Binary(bytes) if bytes.len() == 100)); + assert_eq!(end, Message::Text(r#"{"type":"end"}"#.to_string())); + + ws.send(Message::Text( + serde_json::json!({ + "code": 0, + "message": "success", + "result": { "slice_type": 2, "index": 0, "voice_text_str": "你好" } + }) + .to_string(), + )) + .await + .unwrap(); + ws.send(Message::Text( + serde_json::json!({ "code": 0, "message": "success", "final": 1 }).to_string(), + )) + .await + .unwrap(); + }); + let asr = Arc::new(TencentCloudStreamingASR::with_endpoint( + credentials(), + Arc::new(TokioTaskSpawner), + endpoint, + )); + + asr.open_session().await.unwrap(); + asr.consume_pcm_chunk(&vec![1; 6_500]); + asr.send_last_frame().await.unwrap(); + let transcript = asr.await_final_result().await.unwrap(); + + assert_eq!(transcript.text, "你好"); + assert_eq!(transcript.duration_ms, 203); + server.await.unwrap(); + } + + #[tokio::test] + async fn end_waits_for_queued_audio_even_when_writer_start_is_delayed() { + let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); + let endpoint = format!("ws://{}/asr/v2", listener.local_addr().unwrap()); + let (first_tx, mut first_rx) = oneshot::channel(); + let server = tokio::spawn(async move { + let (stream, _) = listener.accept().await.unwrap(); + let mut ws = tokio_tungstenite::accept_async(stream).await.unwrap(); + ws.send(Message::Text( + serde_json::json!({ "code": 0, "message": "success" }).to_string(), + )) + .await + .unwrap(); + first_tx.send(ws.next().await.unwrap().unwrap()).unwrap(); + assert!(matches!( + ws.next().await.unwrap().unwrap(), + Message::Binary(bytes) if bytes.len() == 100 + )); + assert_eq!( + ws.next().await.unwrap().unwrap(), + Message::Text(r#"{"type":"end"}"#.to_string()) + ); + ws.send(Message::Text( + serde_json::json!({ "code": 0, "message": "success", "final": 1 }).to_string(), + )) + .await + .unwrap(); + }); + let release = Arc::new(tokio::sync::Semaphore::new(0)); + let asr = Arc::new(TencentCloudStreamingASR::with_endpoint( + credentials(), + Arc::new(DelayedFirstSpawner { + spawned: AtomicUsize::new(0), + release: Arc::clone(&release), + }), + endpoint, + )); + asr.open_session().await.unwrap(); + asr.consume_pcm_chunk(&vec![1; 6_500]); + let finishing = { + let asr = Arc::clone(&asr); + tokio::spawn(async move { asr.send_last_frame().await }) + }; + + assert!( + tokio::time::timeout(Duration::from_millis(1_800), &mut first_rx) + .await + .is_err(), + "end must not bypass audio queued on a delayed writer" + ); + release.add_permits(1); + assert!(matches!( + tokio::time::timeout(Duration::from_secs(2), &mut first_rx) + .await + .unwrap() + .unwrap(), + Message::Binary(bytes) if bytes.len() == 6_400 + )); + finishing.await.unwrap().unwrap(); + assert!(asr.await_final_result().await.unwrap().text.is_empty()); + server.await.unwrap(); + } + + #[tokio::test] + async fn buffered_audio_replay_stays_within_tencent_rate_limit() { + let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); + let endpoint = format!("ws://{}/asr/v2", listener.local_addr().unwrap()); + let (elapsed_tx, elapsed_rx) = oneshot::channel(); + let server = tokio::spawn(async move { + let (stream, _) = listener.accept().await.unwrap(); + let mut ws = tokio_tungstenite::accept_async(stream).await.unwrap(); + ws.send(Message::Text( + serde_json::json!({ "code": 0, "message": "success" }).to_string(), + )) + .await + .unwrap(); + let mut received_at = Vec::new(); + for _ in 0..3 { + assert!(matches!( + ws.next().await.unwrap().unwrap(), + Message::Binary(bytes) if bytes.len() == TARGET_AUDIO_CHUNK_BYTES + )); + received_at.push(tokio::time::Instant::now()); + } + assert_eq!( + ws.next().await.unwrap().unwrap(), + Message::Text(r#"{"type":"end"}"#.to_string()) + ); + elapsed_tx + .send(received_at[2].duration_since(received_at[0])) + .unwrap(); + ws.send(Message::Text( + serde_json::json!({ "code": 0, "message": "success", "final": 1 }).to_string(), + )) + .await + .unwrap(); + }); + let asr = Arc::new(TencentCloudStreamingASR::with_endpoint( + credentials(), + Arc::new(TokioTaskSpawner), + endpoint, + )); + + asr.open_session().await.unwrap(); + asr.consume_pcm_chunk(&vec![1; TARGET_AUDIO_CHUNK_BYTES * 3]); + asr.send_last_frame().await.unwrap(); + asr.await_final_result().await.unwrap(); + + assert!( + elapsed_rx.await.unwrap() >= Duration::from_millis(180), + "buffered audio must be paced instead of sent as one burst" + ); + server.await.unwrap(); + } + + #[tokio::test] + async fn websocket_rejection_does_not_expose_server_message() { + let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); + let endpoint = format!("ws://{}/asr/v2", listener.local_addr().unwrap()); + let server = tokio::spawn(async move { + let (stream, _) = listener.accept().await.unwrap(); + let mut ws = tokio_tungstenite::accept_async(stream).await.unwrap(); + ws.send(Message::Text( + serde_json::json!({ + "code": 4002, + "message": "SENSITIVE_SERVER_RESPONSE" + }) + .to_string(), + )) + .await + .unwrap(); + }); + let asr = Arc::new(TencentCloudStreamingASR::with_endpoint( + credentials(), + Arc::new(TokioTaskSpawner), + endpoint, + )); + + let error = asr.open_session().await.unwrap_err(); + + assert!(matches!(error, TencentCloudASRError::AuthRejected(4002))); + assert!(!error.to_string().contains("SENSITIVE_SERVER_RESPONSE")); + server.await.unwrap(); + } + + #[tokio::test] + async fn cancel_closes_the_socket_and_unblocks_waiters() { + let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); + let endpoint = format!("ws://{}/asr/v2", listener.local_addr().unwrap()); + let (closed_tx, closed_rx) = oneshot::channel(); + let server = tokio::spawn(async move { + let (stream, _) = listener.accept().await.unwrap(); + let mut ws = tokio_tungstenite::accept_async(stream).await.unwrap(); + ws.send(Message::Text( + serde_json::json!({ "code": 0, "message": "success" }).to_string(), + )) + .await + .unwrap(); + let closed = matches!( + ws.next().await, + Some(Ok(Message::Close(_))) | None | Some(Err(_)) + ); + closed_tx.send(closed).unwrap(); + }); + let asr = Arc::new(TencentCloudStreamingASR::with_endpoint( + credentials(), + Arc::new(TokioTaskSpawner), + endpoint, + )); + asr.open_session().await.unwrap(); + + asr.cancel(); + + assert!(tokio::time::timeout(Duration::from_secs(2), closed_rx) + .await + .unwrap() + .unwrap()); + assert!(matches!( + asr.send_last_frame().await, + Err(TencentCloudASRError::ConnectionFailed) + )); + assert!(matches!( + asr.await_final_result().await, + Err(TencentCloudASRError::NoFinalResult) + )); + server.await.unwrap(); + } +} diff --git a/openless-all/app/crates/openless-core/src/cloud_providers.rs b/openless-all/app/crates/openless-core/src/cloud_providers.rs index 0793bbe4..159c69b8 100644 --- a/openless-all/app/crates/openless-core/src/cloud_providers.rs +++ b/openless-all/app/crates/openless-core/src/cloud_providers.rs @@ -12,11 +12,13 @@ use std::time::{Duration, Instant}; use futures_util::future::BoxFuture; use parking_lot::{Mutex, RwLock}; +use crate::asr::tencent_cloud::TencentCloudASRError; use crate::asr::{ BailianCredentials, BailianRealtimeASR, DashScopeMultimodalASR, DictionaryHotword, ElevenLabsBatchASR, MimoBatchASR, Qwen3RealtimeASR, Qwen3RealtimeCredentials, - StepfunRealtimeASR, StepfunRealtimeCredentials, VolcengineCredentials, VolcengineStreamingASR, - WhisperBatchASR, XfyunCredentials, XfyunStreamingASR, + StepfunRealtimeASR, StepfunRealtimeCredentials, TencentCloudCredentials, + TencentCloudStreamingASR, VolcengineCredentials, VolcengineStreamingASR, WhisperBatchASR, + XfyunCredentials, XfyunStreamingASR, }; use crate::config::{TaskSpawner, TokioTaskSpawner}; use crate::credentials::{ @@ -24,7 +26,8 @@ use crate::credentials::{ ASR_API_KEY_ACCOUNT, ASR_ENDPOINT_ACCOUNT, ASR_MODEL_ACCOUNT, ASR_VOCABULARY_ID_ACCOUNT, LLM_API_KEY_ACCOUNT, LLM_ENDPOINT_ACCOUNT, LLM_EXTRA_HEADERS_ACCOUNT, LLM_TEMPERATURE_ACCOUNT, OMNI_API_KEY_ACCOUNT, OMNI_ENDPOINT_ACCOUNT, OMNI_EXTRA_HEADERS_ACCOUNT, OMNI_MODEL_ACCOUNT, - OMNI_TEMPERATURE_ACCOUNT, VOLCENGINE_ACCESS_KEY_ACCOUNT, VOLCENGINE_API_KEY_ACCOUNT, + OMNI_TEMPERATURE_ACCOUNT, TENCENT_CLOUD_APP_ID_ACCOUNT, TENCENT_CLOUD_SECRET_ID_ACCOUNT, + TENCENT_CLOUD_SECRET_KEY_ACCOUNT, VOLCENGINE_ACCESS_KEY_ACCOUNT, VOLCENGINE_API_KEY_ACCOUNT, VOLCENGINE_APP_KEY_ACCOUNT, VOLCENGINE_AUTH_MODE_ACCOUNT, VOLCENGINE_RESOURCE_ID_ACCOUNT, XFYUN_API_KEY_ACCOUNT, XFYUN_APP_ID_ACCOUNT, }; @@ -60,6 +63,7 @@ pub const SHARED_CLOUD_ASR_PROVIDER_TYPES: &[&str] = &[ "openai-compatible", "xiaomi-mimo-asr", "iflytek", + "tencent-cloud", ]; pub const SHARED_CLOUD_LLM_PROVIDER_TYPES: &[&str] = &[ @@ -79,6 +83,7 @@ pub const SHARED_CLOUD_LLM_PROVIDER_TYPES: &[&str] = &[ "minimax", "stepfun", "opencode", + "tencentTokenHub", "custom", "custom_responses", "custom_messages", @@ -119,6 +124,7 @@ enum CloudTranscriptionSessionKind { QwenRealtime(Arc), StepfunRealtime(Arc), Xfyun(Arc), + TencentCloud(Arc), } struct CloudTranscriptionSession { @@ -169,6 +175,9 @@ impl AudioConsumer for CloudTranscriptionSession { provider.consume_pcm_chunk(pcm) } CloudTranscriptionSessionKind::Xfyun(provider) => provider.consume_pcm_chunk(pcm), + CloudTranscriptionSessionKind::TencentCloud(provider) => { + provider.consume_pcm_chunk(pcm) + } } } } @@ -235,6 +244,16 @@ impl TranscriptionSession for CloudTranscriptionSession { timeout_transcription(Duration::from_secs(120), provider.await_final_result()) .await? } + CloudTranscriptionSessionKind::TencentCloud(provider) => { + provider + .send_last_frame() + .await + .map_err(map_tencent_asr_error)?; + provider + .await_final_result() + .await + .map_err(map_tencent_asr_error)? + } }; Ok(TranscriptOutput { text: transcript.text, @@ -254,6 +273,7 @@ impl TranscriptionSession for CloudTranscriptionSession { CloudTranscriptionSessionKind::QwenRealtime(provider) => provider.cancel(), CloudTranscriptionSessionKind::StepfunRealtime(provider) => provider.cancel(), CloudTranscriptionSessionKind::Xfyun(provider) => provider.cancel(), + CloudTranscriptionSessionKind::TencentCloud(provider) => provider.cancel(), } Box::pin(async { Ok(()) }) } @@ -632,6 +652,62 @@ async fn build_cloud_transcription_session( provider.open_session().await.map_err(map_asr_error)?; (CloudTranscriptionSessionKind::Xfyun(provider), None) } + ActiveAsrProviderKind::TencentCloud => { + let app_id = read_channel_credential( + credentials, + CredentialNamespace::Asr, + channel_id, + TENCENT_CLOUD_APP_ID_ACCOUNT, + ) + .await? + .unwrap_or_default(); + let secret_id = read_channel_credential( + credentials, + CredentialNamespace::Asr, + channel_id, + TENCENT_CLOUD_SECRET_ID_ACCOUNT, + ) + .await? + .unwrap_or_default(); + let secret_key = read_channel_credential( + credentials, + CredentialNamespace::Asr, + channel_id, + TENCENT_CLOUD_SECRET_KEY_ACCOUNT, + ) + .await? + .unwrap_or_default(); + require_configured(&app_id, "Tencent Cloud AppID")?; + require_configured(&secret_id, "Tencent Cloud SecretID")?; + require_configured(&secret_key, "Tencent Cloud SecretKey")?; + let model = read_channel_credential( + credentials, + CredentialNamespace::Asr, + channel_id, + ASR_MODEL_ACCOUNT, + ) + .await? + .filter(|value| !value.trim().is_empty()) + .unwrap_or_else(|| crate::asr::tencent_cloud::DEFAULT_MODEL.to_string()); + let provider = Arc::new(TencentCloudStreamingASR::with_task_spawner( + TencentCloudCredentials { + app_id, + secret_id, + secret_key, + model: model.clone(), + }, + Arc::clone(&task_spawner), + )); + provider.set_partial_sink(Arc::clone(&partials)); + provider + .open_session() + .await + .map_err(map_tencent_asr_error)?; + ( + CloudTranscriptionSessionKind::TencentCloud(provider), + Some(model), + ) + } }; Ok((kind, crate::AsrCallLabel::new(effective, label_model))) } @@ -676,6 +752,22 @@ fn map_asr_error(error: impl std::fmt::Display) -> BackendError { ) } +fn map_tencent_asr_error(error: TencentCloudASRError) -> BackendError { + let retryable = matches!( + &error, + TencentCloudASRError::ConnectionFailed + | TencentCloudASRError::RateLimited + | TencentCloudASRError::ServiceUnavailable(_) + | TencentCloudASRError::NoFinalResult + | TencentCloudASRError::FinalResultTimeout + ); + BackendError::new( + BackendErrorCode::Provider, + format!("Tencent Cloud ASR failed: {error}"), + ) + .retryable(retryable) +} + pub struct SharedCloudTextPolisher { credentials: Arc, active: Arc>>>, @@ -1868,6 +1960,35 @@ mod tests { use super::*; use crate::{InMemoryCredentialStore, ProviderInvocation, SecretValue}; + #[test] + fn tencent_cloud_errors_keep_retryability_and_stable_public_messages() { + use crate::asr::tencent_cloud::TencentCloudASRError; + + for error in [ + TencentCloudASRError::ConnectionFailed, + TencentCloudASRError::RateLimited, + TencentCloudASRError::NoFinalResult, + TencentCloudASRError::FinalResultTimeout, + TencentCloudASRError::ServiceUnavailable(5000), + ] { + let error = map_tencent_asr_error(error); + assert_eq!(error.code, BackendErrorCode::Provider); + assert!(error.retryable, "{error:?}"); + assert!(error.details.is_none()); + } + for error in [ + TencentCloudASRError::CredentialsMissing, + TencentCloudASRError::AuthRejected(4002), + TencentCloudASRError::AccountUnavailable(4004), + TencentCloudASRError::TaskFailed(4001), + ] { + let error = map_tencent_asr_error(error); + assert_eq!(error.code, BackendErrorCode::Provider); + assert!(!error.retryable, "{error:?}"); + assert!(error.details.is_none()); + } + } + struct IgnoreTextStreamSink; impl TextStreamSink for IgnoreTextStreamSink { diff --git a/openless-all/app/crates/openless-core/src/credentials.rs b/openless-all/app/crates/openless-core/src/credentials.rs index db651678..dc627063 100644 --- a/openless-all/app/crates/openless-core/src/credentials.rs +++ b/openless-all/app/crates/openless-core/src/credentials.rs @@ -68,6 +68,9 @@ pub const VOLCENGINE_AUTH_MODE_ACCOUNT: &str = "volcengine.auth_mode"; pub const VOLCENGINE_API_KEY_ACCOUNT: &str = "volcengine.api_key"; pub const XFYUN_APP_ID_ACCOUNT: &str = "xfyun.app_id"; pub const XFYUN_API_KEY_ACCOUNT: &str = "xfyun.api_key"; +pub const TENCENT_CLOUD_APP_ID_ACCOUNT: &str = "tencent_cloud.app_id"; +pub const TENCENT_CLOUD_SECRET_ID_ACCOUNT: &str = "tencent_cloud.secret_id"; +pub const TENCENT_CLOUD_SECRET_KEY_ACCOUNT: &str = "tencent_cloud.secret_key"; pub const LLM_API_KEY_ACCOUNT: &str = "ark.api_key"; pub const LLM_MODEL_ACCOUNT: &str = "ark.model_id"; pub const LLM_ENDPOINT_ACCOUNT: &str = "ark.endpoint"; diff --git a/openless-all/app/crates/openless-core/src/credentials_legacy.rs b/openless-all/app/crates/openless-core/src/credentials_legacy.rs index 6b5bc878..74b3f190 100644 --- a/openless-all/app/crates/openless-core/src/credentials_legacy.rs +++ b/openless-all/app/crates/openless-core/src/credentials_legacy.rs @@ -99,6 +99,9 @@ struct LegacyEntry { advanced_config: Option, xfyun_app_id: Option, xfyun_api_key: Option, + tencent_cloud_app_id: Option, + tencent_cloud_secret_id: Option, + tencent_cloud_secret_key: Option, temperature: Option, extra_headers: Option>, request_format: Option, @@ -127,6 +130,9 @@ impl Default for LegacyEntry { advanced_config: None, xfyun_app_id: None, xfyun_api_key: None, + tencent_cloud_app_id: None, + tencent_cloud_secret_id: None, + tencent_cloud_secret_key: None, temperature: None, extra_headers: None, request_format: None, @@ -153,6 +159,9 @@ impl LegacyEntry { &self.advanced_config, &self.xfyun_app_id, &self.xfyun_api_key, + &self.tencent_cloud_app_id, + &self.tencent_cloud_secret_id, + &self.tencent_cloud_secret_key, &self.request_format, &self.messages_thinking, &self.max_tokens, @@ -379,6 +388,15 @@ fn decode_entry( (ASR_ADVANCED_CONFIG_ACCOUNT, entry.advanced_config), (XFYUN_APP_ID_ACCOUNT, entry.xfyun_app_id), (XFYUN_API_KEY_ACCOUNT, entry.xfyun_api_key), + (TENCENT_CLOUD_APP_ID_ACCOUNT, entry.tencent_cloud_app_id), + ( + TENCENT_CLOUD_SECRET_ID_ACCOUNT, + entry.tencent_cloud_secret_id, + ), + ( + TENCENT_CLOUD_SECRET_KEY_ACCOUNT, + entry.tencent_cloud_secret_key, + ), ]); } if let Some(account) = extra_headers { @@ -432,6 +450,9 @@ pub fn read_legacy_accounts( ASR_ADVANCED_CONFIG_ACCOUNT, XFYUN_APP_ID_ACCOUNT, XFYUN_API_KEY_ACCOUNT, + TENCENT_CLOUD_APP_ID_ACCOUNT, + TENCENT_CLOUD_SECRET_ID_ACCOUNT, + TENCENT_CLOUD_SECRET_KEY_ACCOUNT, ][..], ), ( diff --git a/openless-all/app/crates/openless-core/src/llm_protocol.rs b/openless-all/app/crates/openless-core/src/llm_protocol.rs index ee0e01e7..0c4314ba 100644 --- a/openless-all/app/crates/openless-core/src/llm_protocol.rs +++ b/openless-all/app/crates/openless-core/src/llm_protocol.rs @@ -39,7 +39,7 @@ impl LlmRequestFormat { } pub fn selectable(provider: &str) -> bool { - !matches!(provider, "gemini" | "codex_oauth") + !matches!(provider, "gemini" | "codex_oauth" | "tencentTokenHub") } pub fn parse(value: &str) -> Result { @@ -531,6 +531,32 @@ mod tests { .is_err()); } + #[tokio::test] + async fn tokenhub_is_fixed_to_chat_completions() { + let store = InMemoryCredentialStore::default(); + store + .write( + CredentialKey::new( + CredentialNamespace::Llm, + Some("tokenhub".into()), + REQUEST_FORMAT_ACCOUNT, + ) + .unwrap(), + SecretValue::new("messages"), + ) + .await + .unwrap(); + + assert!(!LlmRequestFormat::selectable("tencentTokenHub")); + assert_eq!( + LlmProtocolConfig::load(&store, "tokenhub", "tencentTokenHub") + .await + .unwrap() + .format, + LlmRequestFormat::ChatCompletions + ); + } + #[test] fn urls_and_auth_follow_format_without_losing_gateway_paths() { for descriptor in crate::provider_rules::provider_descriptors(crate::ProviderKind::Llm) { diff --git a/openless-all/app/crates/openless-core/src/polish.rs b/openless-all/app/crates/openless-core/src/polish.rs index 9bd15dc1..13d4e5d1 100644 --- a/openless-all/app/crates/openless-core/src/polish.rs +++ b/openless-all/app/crates/openless-core/src/polish.rs @@ -189,6 +189,7 @@ fn is_builtin_llm_provider(provider_id: &str) -> bool { | "minimax" | "stepfun" | "opencode" + | "tencentTokenHub" ) } @@ -1649,14 +1650,20 @@ pub(crate) fn apply_openai_compatible_thinking_control( model: &str, thinking_enabled: bool, ) { + if provider_id.trim() == "tencentTokenHub" { + apply_tokenhub_chat_thinking_control(body, model, thinking_enabled); + return; + } // 优先按 provider_id 预设分派；custom / 未声明 provider 时回退到 base_url 兜底, // 让用户用"自定义"preset 接入 MiniMax 也能正确下发 thinking 控制参数。 // Zen 是多模型网关，仅 DeepSeek 模型使用 DeepSeek 的思考参数。 let is_opencode = provider_id.trim() == "opencode" - || (matches!(provider_id.trim(), "custom" | "custom_responses" | "custom_messages") - && url::Url::parse(base_url.trim()) - .ok() - .is_some_and(|url| url.host_str() == Some("opencode.ai"))); + || (matches!( + provider_id.trim(), + "custom" | "custom_responses" | "custom_messages" + ) && url::Url::parse(base_url.trim()) + .ok() + .is_some_and(|url| url.host_str() == Some("opencode.ai"))); let control = if is_opencode { model .trim() @@ -1709,6 +1716,31 @@ pub(crate) fn apply_openai_compatible_thinking_control( } } +fn apply_tokenhub_chat_thinking_control(body: &mut Value, model: &str, enabled: bool) { + use crate::provider_rules::TokenHubChatModelPolicy::*; + + match crate::provider_rules::tokenhub_chat_model_policy(model) { + Some(Hy3) => { + body["thinking"] = json!({ "type": if enabled { "enabled" } else { "disabled" } }); + if enabled { + body["reasoning_effort"] = json!("medium"); + } + } + Some(ToggleThinking) => { + body["thinking"] = json!({ "type": if enabled { "enabled" } else { "disabled" } }); + } + Some(QwenThinking) => body["enable_thinking"] = json!(enabled), + Some(AdaptiveThinking) => { + body["thinking"] = json!({ "type": if enabled { "adaptive" } else { "disabled" } }); + } + Some(AlwaysThinking) if enabled => { + body["thinking"] = json!({ "type": "enabled" }); + } + Some(KimiK3) if enabled => body["reasoning_effort"] = json!("max"), + Some(AlwaysThinking | KimiK3 | Plain) | None => {} + } +} + #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub(crate) enum ThinkingControl { ReasoningEffort, @@ -2143,8 +2175,12 @@ mod tests { #[tokio::test] async fn all_text_entrypoints_use_the_selected_protocol_over_http() { for (format, (preset, prefix)) in LlmRequestFormat::ALL.into_iter().flat_map(|format| { - [("custom", "/gateway/v1"), ("opencode", "/zen/v1"), ("opencode", "/zen/go/v1")] - .map(|entry| (format, entry)) + [ + ("custom", "/gateway/v1"), + ("opencode", "/zen/v1"), + ("opencode", "/zen/go/v1"), + ] + .map(|entry| (format, entry)) }) { let listener = TcpListener::bind("127.0.0.1:0").unwrap(); let address = listener.local_addr().unwrap(); @@ -2311,7 +2347,11 @@ mod tests { ("custom", "https://opencode.ai/zen/v1", true), ("custom_responses", "https://opencode.ai/zen/v1", true), ("custom_messages", "https://opencode.ai/zen/v1", true), - ("custom", "https://OPENCODE.AI:443/zen/go/v1/chat/completions", true), + ( + "custom", + "https://OPENCODE.AI:443/zen/go/v1/chat/completions", + true, + ), ("custom", "https://opencode.ai.example/zen/v1", false), ("custom", "https://fakeopencode.ai/zen/v1", false), ("custom", "https://opencode.ai@example.com/zen/v1", false), @@ -2323,22 +2363,37 @@ mod tests { let provider = OpenAICompatibleLLMProvider::new( OpenAICompatibleConfig::new(preset, "test", endpoint, "key", model) .with_thinking_enabled(enabled) - .with_protocol(LlmProtocolConfig { format, ..Default::default() }), + .with_protocol(LlmProtocolConfig { + format, + ..Default::default() + }), ); - let body = provider.chat_body(false, vec![json!({"role":"user","content":"hi"})]); + let body = + provider.chat_body(false, vec![json!({"role":"user","content":"hi"})]); match format { - LlmRequestFormat::ChatCompletions if zen && model.starts_with("deepseek-") => { - assert_eq!(body["thinking"]["type"], if enabled { "enabled" } else { "disabled" }); + LlmRequestFormat::ChatCompletions + if zen && model.starts_with("deepseek-") => + { + assert_eq!( + body["thinking"]["type"], + if enabled { "enabled" } else { "disabled" } + ); } LlmRequestFormat::Messages if enabled => { assert_eq!(body["thinking"]["type"], "adaptive"); } - _ => assert!(body.get("thinking").is_none(), "{preset} {endpoint} {model} {format:?}"), + _ => assert!( + body.get("thinking").is_none(), + "{preset} {endpoint} {model} {format:?}" + ), } assert!(body.get("reasoning_effort").is_none()); assert!(body.get("enable_thinking").is_none()); if format == LlmRequestFormat::Responses { - assert_eq!(body["reasoning"]["effort"], if enabled { "medium" } else { "low" }); + assert_eq!( + body["reasoning"]["effort"], + if enabled { "medium" } else { "low" } + ); assert!(body.get("messages").is_none()); } else { assert!(body.get("reasoning").is_none()); @@ -3224,6 +3279,88 @@ mod tests { assert_eq!(body["thinking"]["type"], "adaptive"); } + #[test] + fn tokenhub_chat_thinking_matches_model_capabilities() { + let body = |model: &str, enabled: bool| { + OpenAICompatibleLLMProvider::new( + OpenAICompatibleConfig::new( + "tencentTokenHub", + "Tencent TokenHub", + "https://tokenhub.tencentmaas.com/v1", + "k", + model, + ) + .with_thinking_enabled(enabled), + ) + .chat_body(false, vec![json!({ "role": "user", "content": "hi" })]) + }; + + for model in [ + "hy3", + "hy4-preview", + "deepseek-v4-pro", + "deepseek/deepseek-v4-flash", + "glm-5.2", + "glm-5v-turbo", + "kimi-k2.6", + "kimi-k2.5", + ] { + assert_eq!(body(model, true)["thinking"]["type"], "enabled", "{model}"); + assert_eq!( + body(model, false)["thinking"]["type"], + "disabled", + "{model}" + ); + } + assert_eq!(body("hy3", true)["reasoning_effort"], "medium"); + + assert_eq!(body("qwen3.5-plus", true)["enable_thinking"], true); + assert_eq!(body("qwen3.5-plus", false)["enable_thinking"], false); + + assert_eq!(body("minimax-m3", true)["thinking"]["type"], "adaptive"); + assert_eq!(body("minimax-m3", false)["thinking"]["type"], "disabled"); + + for model in ["glm-5.3", "kimi-k2.7-code", "minimax-m2.7"] { + assert!(body(model, false).get("thinking").is_none(), "{model}"); + assert_eq!(body(model, true)["thinking"]["type"], "enabled"); + } + let kimi_k3 = body("kimi-k3", true); + assert_eq!(kimi_k3["reasoning_effort"], "max"); + assert!(kimi_k3.get("thinking").is_none()); + assert!(body("kimi-k3", false).get("reasoning_effort").is_none()); + + for model in [ + "hy-mt2-pro", + "hy-role", + "hunyuan-role-latest", + "mimo-v2.5-pro", + "future-model", + ] { + let body = body(model, true); + assert!(body.get("thinking").is_none(), "{model}"); + assert!(body.get("enable_thinking").is_none(), "{model}"); + assert!(body.get("reasoning_effort").is_none(), "{model}"); + } + } + + #[test] + fn custom_tokenhub_endpoint_does_not_invent_model_policy() { + for base_url in [ + "https://tokenhub.tencentmaas.com/v1/", + "https://api.lkeap.cloud.tencent.com/plan/v3", + ] { + let provider = OpenAICompatibleLLMProvider::new( + OpenAICompatibleConfig::new("custom", "Custom", base_url, "k", "hy3") + .with_thinking_enabled(false), + ); + + let body = provider.chat_body(false, vec![json!({ "role": "user", "content": "hi" })]); + + assert!(body.get("thinking").is_none()); + assert!(body.get("reasoning_effort").is_none()); + } + } + #[test] fn openai_chat_body_falls_back_to_base_url_for_custom_minimax_endpoint() { // 用 "custom" preset + 自定义 MiniMax base_url 接入时,base_url 兜底 diff --git a/openless-all/app/crates/openless-core/src/provider_rules.rs b/openless-all/app/crates/openless-core/src/provider_rules.rs index a2efeb6a..d48ca28d 100644 --- a/openless-all/app/crates/openless-core/src/provider_rules.rs +++ b/openless-all/app/crates/openless-core/src/provider_rules.rs @@ -21,6 +21,7 @@ const MIMO_PROVIDER_ID: &str = "xiaomi-mimo-asr"; const DASHSCOPE_MULTIMODAL_PROVIDER_ID: &str = "bailian-fun-asr-flash"; const ELEVENLABS_PROVIDER_ID: &str = "elevenlabs"; const XFYUN_PROVIDER_ID: &str = "iflytek"; +const TENCENT_CLOUD_PROVIDER_ID: &str = "tencent-cloud"; const ASR_PROVIDER_TYPES: &[(&str, &str)] = &[ ("volcengine", "asrVolcengine"), @@ -39,6 +40,7 @@ const ASR_PROVIDER_TYPES: &[(&str, &str)] = &[ (OPENAI_COMPATIBLE_ASR_PROVIDER_ID, "asrOpenAiCompatible"), ("xiaomi-mimo-asr", "asrXiaomiMimo"), (XFYUN_PROVIDER_ID, "asrIflytek"), + (TENCENT_CLOUD_PROVIDER_ID, "asrTencentCloud"), ("foundry-local-whisper", "asrFoundryLocalWhisper"), ("local-whisper", "asrLocalWhisper"), ("sherpa-onnx-local", "asrSherpaOnnxLocal"), @@ -65,6 +67,7 @@ const LLM_PROVIDER_TYPES: &[(&str, &str)] = &[ ("minimax", "minimax"), ("stepfun", "stepfun"), ("opencode", "opencode"), + ("tencentTokenHub", "tencentTokenHub"), ("custom", "customChatCompletions"), ("custom_responses", "customResponses"), ("custom_messages", "customMessages"), @@ -112,6 +115,7 @@ pub enum AuthRequirement { ApiKeyUnlessCustomEndpoint, Volcengine, Xfyun, + TencentCloud, OAuth, } @@ -189,11 +193,14 @@ fn provider_descriptor_with_label( match id.as_str() { "volcengine" => AuthRequirement::Volcengine, XFYUN_PROVIDER_ID => AuthRequirement::Xfyun, + TENCENT_CLOUD_PROVIDER_ID => AuthRequirement::TencentCloud, OPENAI_COMPATIBLE_ASR_PROVIDER_ID => AuthRequirement::EndpointModelOptionalApiKey, _ => AuthRequirement::ApiKey, }, match id.as_str() { - "volcengine" | XFYUN_PROVIDER_ID => ValidationProbe::AsrSilenceAllowsNoFinal, + "volcengine" | XFYUN_PROVIDER_ID | TENCENT_CLOUD_PROVIDER_ID => { + ValidationProbe::AsrSilenceAllowsNoFinal + } "stepfun" => ValidationProbe::StepfunNoSpeech, DASHSCOPE_MULTIMODAL_PROVIDER_ID => ValidationProbe::AsrNonSilent, _ => ValidationProbe::AsrSilence, @@ -305,6 +312,7 @@ pub enum ActiveAsrProviderKind { WhisperCompatible, Volcengine, Xfyun, + TencentCloud, } /// Non-secret facts read by a platform credential adapter. Core evaluates @@ -321,6 +329,9 @@ pub struct CredentialConfiguration { pub volcengine_resource_id: bool, pub xfyun_app_id: bool, pub xfyun_api_key: bool, + pub tencent_cloud_app_id: bool, + pub tencent_cloud_secret_id: bool, + pub tencent_cloud_secret_key: bool, pub llm_api_key: bool, pub llm_endpoint: bool, pub llm_endpoint_matches_default: bool, @@ -403,6 +414,11 @@ pub fn auth_requirement_satisfied( } AuthRequirement::Volcengine => volcengine_configured(configuration), AuthRequirement::Xfyun => configuration.xfyun_app_id && configuration.xfyun_api_key, + AuthRequirement::TencentCloud => { + configuration.tencent_cloud_app_id + && configuration.tencent_cloud_secret_id + && configuration.tencent_cloud_secret_key + } AuthRequirement::OAuth => configuration.codex_oauth && model, } } @@ -418,7 +434,8 @@ pub fn api_key_required( match descriptor.auth_requirement { AuthRequirement::None | AuthRequirement::EndpointModelOptionalApiKey - | AuthRequirement::OAuth => false, + | AuthRequirement::OAuth + | AuthRequirement::TencentCloud => false, AuthRequirement::ApiKeyUnlessCustomEndpoint => { let Some(endpoint) = configured_endpoint.filter(|value| !value.trim().is_empty()) else { @@ -480,6 +497,7 @@ pub fn default_asr_model(provider_type: &str) -> Option<&'static str> { "orcarouter" => Some(crate::asr::mimo::ORCAROUTER_DEFAULT_MODEL), "zenmux" => Some(crate::asr::whisper::ZENMUX_DEFAULT_MODEL), "xiaomi-mimo-asr" => Some(crate::asr::mimo::DEFAULT_MODEL), + TENCENT_CLOUD_PROVIDER_ID => Some(crate::asr::tencent_cloud::DEFAULT_MODEL), _ => None, } } @@ -501,6 +519,7 @@ pub fn default_llm_endpoint(provider_type: &str) -> Option<&'static str> { "codingPlanX" => Some("https://api.codingplanx.ai/v1"), "minimax" => Some("https://api.minimaxi.com/v1"), "stepfun" => Some("https://api.stepfun.com/v1"), + "tencentTokenHub" => Some("https://tokenhub.tencentmaas.com/v1"), _ => None, } } @@ -521,10 +540,77 @@ pub fn default_llm_model(provider_type: &str) -> Option<&'static str> { "codingPlanX" => Some("gpt-5-mini"), "minimax" => Some("MiniMax-M3"), "stepfun" => Some("step-1o-turbo-vision"), + "tencentTokenHub" => Some("hy3"), _ => None, } } +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum TokenHubChatModelPolicy { + Hy3, + ToggleThinking, + QwenThinking, + AdaptiveThinking, + AlwaysThinking, + KimiK3, + Plain, +} + +pub(crate) fn tokenhub_chat_model_policy(model: &str) -> Option { + use TokenHubChatModelPolicy::*; + + // ponytail: /models lacks capability metadata; keep this one family classifier until it does. + let model = model.trim().to_ascii_lowercase(); + let policy = if matches!(model.as_str(), "hy3" | "hy3-preview") { + Hy3 + } else if model.starts_with("qwen3.5-") { + QwenThinking + } else if model == "minimax-m3" { + AdaptiveThinking + } else if model == "kimi-k3" { + KimiK3 + } else if matches!( + model.as_str(), + "glm-5.3" | "glm-5.3-flash" | "minimax-m2.7" | "minimax-m2.5" + ) || model.starts_with("kimi-k2.7-code") + { + AlwaysThinking + } else if model.starts_with("hy4-") + || model.starts_with("deepseek-") + || model.starts_with("deepseek/") + || matches!( + model.as_str(), + "glm-5.2" + | "glm-5.1" + | "glm-5" + | "glm-5-turbo" + | "glm-5v-turbo" + | "kimi-k2.6" + | "kimi-k2.5" + ) + { + ToggleThinking + } else { + let minimax_language = model + .strip_prefix("minimax-m") + .and_then(|suffix| suffix.chars().next()) + .is_some_and(|character| character.is_ascii_digit()); + if model.starts_with("hy-mt2-") + || model.starts_with("hy-role") + || model.starts_with("hunyuan-role") + || model.starts_with("glm-") + || model.starts_with("kimi-") + || minimax_language + || model.starts_with("mimo-v") + { + Plain + } else { + return None; + } + }; + Some(policy) +} + pub fn default_omni_endpoint(provider_type: &str) -> Option<&'static str> { match provider_type { "openai" => Some("https://api.openai.com/v1"), @@ -576,6 +662,7 @@ pub fn active_asr_provider_kind(id: &str) -> ActiveAsrProviderKind { DASHSCOPE_MULTIMODAL_PROVIDER_ID => ActiveAsrProviderKind::DashScopeMultimodal, ELEVENLABS_PROVIDER_ID => ActiveAsrProviderKind::ElevenLabs, XFYUN_PROVIDER_ID => ActiveAsrProviderKind::Xfyun, + TENCENT_CLOUD_PROVIDER_ID => ActiveAsrProviderKind::TencentCloud, value if is_whisper_compatible_provider(value) => ActiveAsrProviderKind::WhisperCompatible, _ => ActiveAsrProviderKind::Volcengine, } @@ -609,6 +696,10 @@ pub fn is_xfyun_provider(id: &str) -> bool { id == XFYUN_PROVIDER_ID } +pub fn is_tencent_cloud_provider(id: &str) -> bool { + id == TENCENT_CLOUD_PROVIDER_ID +} + pub fn is_whisper_compatible_provider(id: &str) -> bool { matches!( id, @@ -1003,12 +1094,25 @@ mod tests { assert!(crate::cloud_providers::SHARED_CLOUD_LLM_PROVIDER_TYPES.contains(&"opencode")); let descriptor = provider_descriptor(ProviderKind::Llm, "opencode").unwrap(); assert_eq!(descriptor.label_key, "opencode"); - assert_eq!(descriptor.default_endpoint.as_deref(), Some("https://opencode.ai/zen/v1")); - assert_eq!(descriptor.default_model.as_deref(), Some("deepseek-v4-flash")); - assert_eq!(descriptor.default_request_format, Some(LlmRequestFormat::ChatCompletions)); + assert_eq!( + descriptor.default_endpoint.as_deref(), + Some("https://opencode.ai/zen/v1") + ); + assert_eq!( + descriptor.default_model.as_deref(), + Some("deepseek-v4-flash") + ); + assert_eq!( + descriptor.default_request_format, + Some(LlmRequestFormat::ChatCompletions) + ); assert_eq!(descriptor.supported_request_formats, LlmRequestFormat::ALL); assert_eq!(descriptor.validation_probe, ValidationProbe::LlmText); - assert!(api_key_required(ProviderKind::Llm, "opencode", Some("https://opencode.ai/zen/v1/chat/completions/"))); + assert!(api_key_required( + ProviderKind::Llm, + "opencode", + Some("https://opencode.ai/zen/v1/chat/completions/") + )); let mut configuration = CredentialConfiguration { llm_endpoint: true, llm_endpoint_matches_default: true, @@ -1020,6 +1124,49 @@ mod tests { assert!(llm_configured("opencode", &configuration)); } + #[test] + fn tencent_cloud_asr_and_tokenhub_supply_defaults_and_credentials() { + assert!(crate::cloud_providers::SHARED_CLOUD_ASR_PROVIDER_TYPES.contains(&"tencent-cloud")); + assert!( + crate::cloud_providers::SHARED_CLOUD_LLM_PROVIDER_TYPES.contains(&"tencentTokenHub") + ); + let asr = provider_descriptor(ProviderKind::Asr, "tencent-cloud").unwrap(); + assert_eq!(asr.label_key, "asrTencentCloud"); + assert_eq!( + asr.default_model.as_deref(), + Some(crate::asr::tencent_cloud::DEFAULT_MODEL) + ); + assert_eq!(asr.auth_requirement, AuthRequirement::TencentCloud); + assert_eq!( + asr.validation_probe, + ValidationProbe::AsrSilenceAllowsNoFinal + ); + let mut configuration = CredentialConfiguration::default(); + assert!(!auth_requirement_satisfied(&asr, &configuration)); + configuration.tencent_cloud_app_id = true; + configuration.tencent_cloud_secret_id = true; + configuration.tencent_cloud_secret_key = true; + assert!(auth_requirement_satisfied(&asr, &configuration)); + assert_eq!( + active_asr_provider_kind("tencent-cloud"), + ActiveAsrProviderKind::TencentCloud + ); + + let llm = provider_descriptor(ProviderKind::Llm, "tencentTokenHub").unwrap(); + assert_eq!(llm.label_key, "tencentTokenHub"); + assert_eq!( + llm.default_endpoint.as_deref(), + Some("https://tokenhub.tencentmaas.com/v1") + ); + assert_eq!(llm.default_model.as_deref(), Some("hy3")); + assert_eq!( + llm.auth_requirement, + AuthRequirement::ApiKeyUnlessCustomEndpoint + ); + assert_eq!(llm.default_request_format, None); + assert!(llm.supported_request_formats.is_empty()); + } + #[test] fn descriptors_are_the_single_source_for_defaults_auth_and_probes() { let compatible = provider_descriptor(ProviderKind::Asr, "openai-compatible").unwrap(); diff --git a/openless-all/app/crates/openless-core/src/provider_service.rs b/openless-all/app/crates/openless-core/src/provider_service.rs index a3bbd298..d4a89c3f 100644 --- a/openless-all/app/crates/openless-core/src/provider_service.rs +++ b/openless-all/app/crates/openless-core/src/provider_service.rs @@ -418,7 +418,7 @@ fn validate_configuration(resolved: &ResolvedProvider) -> Result<(), BackendErro ) && api_key.trim().is_empty() && !matches!( descriptor.auth_requirement, - AuthRequirement::Volcengine | AuthRequirement::Xfyun + AuthRequirement::Volcengine | AuthRequirement::Xfyun | AuthRequirement::TencentCloud ) { let label = match resolved.kind { @@ -436,7 +436,10 @@ fn validate_configuration(resolved: &ResolvedProvider) -> Result<(), BackendErro if model.is_none() && !matches!( descriptor.auth_requirement, - AuthRequirement::None | AuthRequirement::Volcengine | AuthRequirement::Xfyun + AuthRequirement::None + | AuthRequirement::Volcengine + | AuthRequirement::Xfyun + | AuthRequirement::TencentCloud ) { return Err(invalid_request("provider model is not configured")); @@ -450,7 +453,10 @@ fn validate_configuration(resolved: &ResolvedProvider) -> Result<(), BackendErro if endpoint.is_none() && !matches!( descriptor.auth_requirement, - AuthRequirement::None | AuthRequirement::Volcengine | AuthRequirement::Xfyun + AuthRequirement::None + | AuthRequirement::Volcengine + | AuthRequirement::Xfyun + | AuthRequirement::TencentCloud ) { return Err(provider_error("provider endpoint is not configured")); @@ -650,6 +656,7 @@ async fn fetch_models( .ok_or_else(|| provider_error("provider endpoint is not configured"))?; let url = models_url(endpoint)?; let is_gemini = resolved.provider_type == "gemini"; + let tokenhub_chat_only = resolved.provider_type == "tencentTokenHub"; let orcarouter_filter = (resolved.provider_type == "orcarouter").then(|| { let endpoint_type = match (resolved.kind, resolved.protocol.format) { (ProviderKind::Llm, LlmRequestFormat::Responses) => "openai-response", @@ -709,7 +716,12 @@ async fn fetch_models( if response.body.len() > MODEL_LIST_MAX_BYTES { return Err(provider_error("provider model response is too large")); } - parse_model_list(&response.body, is_gemini, orcarouter_filter) + parse_model_list( + &response.body, + is_gemini, + orcarouter_filter, + tokenhub_chat_only, + ) } #[derive(Clone, Copy)] @@ -722,6 +734,7 @@ fn parse_model_list( body: &[u8], is_gemini: bool, orcarouter_filter: Option, + tokenhub_chat_only: bool, ) -> Result, BackendError> { let value: serde_json::Value = serde_json::from_slice(body) .map_err(|_| provider_error("provider model response is invalid JSON"))?; @@ -756,6 +769,10 @@ fn parse_model_list( .and_then(serde_json::Value::as_array) .ok_or_else(|| provider_error("provider model response is missing data"))? .iter() + .filter(|item| { + !tokenhub_chat_only + || item.get("status").and_then(serde_json::Value::as_str) == Some("online") + }) .filter(|item| { orcarouter_filter.is_none_or(|filter| { let supports_endpoint = item @@ -781,6 +798,10 @@ fn parse_model_list( .filter_map(|item| item.get("id").and_then(serde_json::Value::as_str)) .map(str::trim) .filter(|name| !name.is_empty()) + .filter(|name| { + !tokenhub_chat_only + || crate::provider_rules::tokenhub_chat_model_policy(name).is_some() + }) .filter(|name| { if !orcarouter_filter.is_some_and(|filter| filter.kind == ProviderKind::Asr) { return true; @@ -1338,6 +1359,7 @@ mod tests { br#"{"data":[{"id":"gpt-z"},{"id":""},{"id":"gpt-a"},{"id":"gpt-z"}]}"#, false, None, + false, ) .unwrap(); assert_eq!(models, vec!["gpt-a", "gpt-z"]); @@ -1349,6 +1371,7 @@ mod tests { br#"{"models":[{"name":"models/gemini-z","supportedGenerationMethods":["generateContent"]},{"name":"models/embedding","supportedGenerationMethods":["embedContent"]},{"name":"gemini-a"}]}"#, true, None, + false, ) .unwrap(); assert_eq!(models, vec!["gemini-a", "gemini-z"]); @@ -1356,11 +1379,87 @@ mod tests { #[test] fn invalid_model_response_is_a_provider_error_without_body() { - let error = parse_model_list(br#"{"error":"secret-key"}"#, false, None).unwrap_err(); + let error = parse_model_list(br#"{"error":"secret-key"}"#, false, None, false).unwrap_err(); assert_eq!(error.code, BackendErrorCode::Provider); assert!(!format!("{error:?}").contains("secret-key")); } + #[tokio::test] + async fn tokenhub_catalog_lists_only_online_language_models() { + let credentials = Arc::new(InMemoryCredentialStore::default()); + let channel = create_channel_with_values( + &credentials, + ChannelKind::Llm, + "tencentTokenHub", + &[(LLM_API_KEY_ACCOUNT, "fixture-key")], + ) + .await; + let transport = Arc::new(FakeProviderTransport::default()); + transport.push_response( + 200, + br#"{"object":"list","data":[ + {"id":"hy3","status":"online"}, + {"id":"hy4-preview","status":"online"}, + {"id":"hy-mt2-pro","status":"online"}, + {"id":"hy-role","status":"online"}, + {"id":"hunyuan-role-latest","status":"online"}, + {"id":"deepseek/deepseek-v4-flash","status":"online"}, + {"id":"glm-5.3","status":"online"}, + {"id":"kimi-k3","status":"online"}, + {"id":"minimax-m3","status":"online"}, + {"id":"qwen3.5-flash","status":"online"}, + {"id":"mimo-v2.5-pro","status":"online"}, + {"id":"deepseek-v4-pro","status":"pre-offline"}, + {"id":"glm-future"}, + {"id":"hy-image-v3","status":"online"}, + {"id":"hy-video-v1.5","status":"online"}, + {"id":"HY-3D-3.1","status":"online"}, + {"id":"hy-asr-3.0-preview","status":"online"}, + {"id":"minimax-speech-2.8-hd","status":"online"}, + {"id":"kinfra-text-embedding-4b","status":"online"} + ]}"# + .to_vec(), + ); + let service = ProviderService::new_with_transport( + credentials, + Arc::new(crate::TokioTaskSpawner), + transport.clone(), + ); + + let result = service + .list_models(ProviderRequest { + kind: ProviderKind::Llm, + channel_id: Some(channel), + thinking_enabled: false, + }) + .await + .unwrap(); + + assert_eq!( + result.models, + vec![ + "deepseek/deepseek-v4-flash", + "glm-5.3", + "hunyuan-role-latest", + "hy-mt2-pro", + "hy-role", + "hy3", + "hy4-preview", + "kimi-k3", + "mimo-v2.5-pro", + "minimax-m3", + "qwen3.5-flash", + ] + ); + assert_eq!( + transport.requests()[0].headers, + vec![( + "Authorization".to_string(), + "Bearer fixture-key".to_string() + )] + ); + } + #[tokio::test] async fn orcarouter_catalog_uses_core_channel_defaults_and_filters_capabilities() { let catalog = r#"{"data":[ diff --git a/openless-all/app/linux-egui/src/credentials.rs b/openless-all/app/linux-egui/src/credentials.rs index 123e285b..0f506728 100644 --- a/openless-all/app/linux-egui/src/credentials.rs +++ b/openless-all/app/linux-egui/src/credentials.rs @@ -5,7 +5,8 @@ use futures_util::future::BoxFuture; use openless_core::credentials::{ ASR_API_KEY_ACCOUNT, ASR_ENDPOINT_ACCOUNT, ASR_MODEL_ACCOUNT, LLM_API_KEY_ACCOUNT, LLM_ENDPOINT_ACCOUNT, LLM_MODEL_ACCOUNT, OMNI_API_KEY_ACCOUNT, OMNI_ENDPOINT_ACCOUNT, - OMNI_MODEL_ACCOUNT, VOLCENGINE_ACCESS_KEY_ACCOUNT, VOLCENGINE_API_KEY_ACCOUNT, + OMNI_MODEL_ACCOUNT, TENCENT_CLOUD_APP_ID_ACCOUNT, TENCENT_CLOUD_SECRET_ID_ACCOUNT, + TENCENT_CLOUD_SECRET_KEY_ACCOUNT, VOLCENGINE_ACCESS_KEY_ACCOUNT, VOLCENGINE_API_KEY_ACCOUNT, VOLCENGINE_APP_KEY_ACCOUNT, VOLCENGINE_AUTH_MODE_ACCOUNT, VOLCENGINE_RESOURCE_ID_ACCOUNT, XFYUN_API_KEY_ACCOUNT, XFYUN_APP_ID_ACCOUNT, }; @@ -395,6 +396,21 @@ impl CredentialStore for LinuxCredentialStore { &active_asr_provider, XFYUN_API_KEY_ACCOUNT, ), + tencent_cloud_app_id: has( + CredentialNamespace::Asr, + &active_asr_provider, + TENCENT_CLOUD_APP_ID_ACCOUNT, + ), + tencent_cloud_secret_id: has( + CredentialNamespace::Asr, + &active_asr_provider, + TENCENT_CLOUD_SECRET_ID_ACCOUNT, + ), + tencent_cloud_secret_key: has( + CredentialNamespace::Asr, + &active_asr_provider, + TENCENT_CLOUD_SECRET_KEY_ACCOUNT, + ), llm_api_key: has( CredentialNamespace::Llm, &active_llm_provider, diff --git a/openless-all/app/linux-egui/src/main.rs b/openless-all/app/linux-egui/src/main.rs index bb2e9717..19942ce7 100644 --- a/openless-all/app/linux-egui/src/main.rs +++ b/openless-all/app/linux-egui/src/main.rs @@ -74,6 +74,7 @@ mod linux_app { model: String, auth_mode: String, resource_id: String, + app_id: String, // Secret inputs are intentionally write-only. Loading an editor never // exposes an existing key into egui state, logs or screenshots. primary_secret: String, @@ -1632,6 +1633,7 @@ mod linux_app { } openless_core::AuthRequirement::Volcengine => "火山引擎凭据", openless_core::AuthRequirement::Xfyun => "讯飞 AppID + API Key", + openless_core::AuthRequirement::TencentCloud => "腾讯云 AppID + SecretID + SecretKey", openless_core::AuthRequirement::OAuth => "OAuth", } } @@ -1711,6 +1713,19 @@ mod linux_app { } else { (String::new(), String::new()) }; + let app_id = if descriptor.auth_requirement == openless_core::AuthRequirement::TencentCloud + { + read_provider_value( + &backend, + kind, + &channel.id, + openless_core::credentials::TENCENT_CLOUD_APP_ID_ACCOUNT, + ) + .await? + .unwrap_or_default() + } else { + String::new() + }; Ok(ProviderEditor { kind, name: channel.name.clone(), @@ -1720,6 +1735,7 @@ mod linux_app { model, auth_mode, resource_id, + app_id, primary_secret: String::new(), secondary_secret: String::new(), }) @@ -1777,6 +1793,18 @@ mod linux_app { secret_edit(ui, "AppID", &mut editor.primary_secret); secret_edit(ui, "API Key", &mut editor.secondary_secret); } + openless_core::AuthRequirement::TencentCloud => { + ui.horizontal(|ui| { + ui.label("腾讯云 AppID"); + ui.text_edit_singleline(&mut editor.app_id); + }); + secret_edit(ui, "SecretID", &mut editor.primary_secret); + secret_edit(ui, "SecretKey", &mut editor.secondary_secret); + ui.horizontal(|ui| { + ui.label("Model"); + ui.text_edit_singleline(&mut editor.model); + }); + } _ => { secret_edit(ui, "API Key（留空表示不修改）", &mut editor.primary_secret); ui.horizontal(|ui| { @@ -1913,6 +1941,40 @@ mod linux_app { ) .await?; } + openless_core::AuthRequirement::TencentCloud => { + write_or_remove_provider_value( + &backend, + editor.kind, + channel_id, + openless_core::credentials::TENCENT_CLOUD_APP_ID_ACCOUNT, + &editor.app_id, + ) + .await?; + write_secret_if_entered( + &backend, + editor.kind, + channel_id, + openless_core::credentials::TENCENT_CLOUD_SECRET_ID_ACCOUNT, + &editor.primary_secret, + ) + .await?; + write_secret_if_entered( + &backend, + editor.kind, + channel_id, + openless_core::credentials::TENCENT_CLOUD_SECRET_KEY_ACCOUNT, + &editor.secondary_secret, + ) + .await?; + write_or_remove_provider_value( + &backend, + editor.kind, + channel_id, + model_account(editor.kind), + &editor.model, + ) + .await?; + } _ => { write_or_remove_provider_value( &backend, @@ -1958,6 +2020,11 @@ mod linux_app { openless_core::credentials::XFYUN_APP_ID_ACCOUNT, openless_core::credentials::XFYUN_API_KEY_ACCOUNT, ], + openless_core::AuthRequirement::TencentCloud => &[ + openless_core::credentials::TENCENT_CLOUD_APP_ID_ACCOUNT, + openless_core::credentials::TENCENT_CLOUD_SECRET_ID_ACCOUNT, + openless_core::credentials::TENCENT_CLOUD_SECRET_KEY_ACCOUNT, + ], _ => &[api_key_account(editor.kind)], }; for account in accounts { diff --git a/openless-all/app/src-tauri/src/commands/credentials.rs b/openless-all/app/src-tauri/src/commands/credentials.rs index f9fac199..b75d68fb 100644 --- a/openless-all/app/src-tauri/src/commands/credentials.rs +++ b/openless-all/app/src-tauri/src/commands/credentials.rs @@ -535,6 +535,9 @@ fn credential_configuration( volcengine_resource_id: configured(&snap.volcengine_resource_id), xfyun_app_id: configured(&snap.xfyun_app_id), xfyun_api_key: configured(&snap.xfyun_api_key), + tencent_cloud_app_id: configured(&snap.tencent_cloud_app_id), + tencent_cloud_secret_id: configured(&snap.tencent_cloud_secret_id), + tencent_cloud_secret_key: configured(&snap.tencent_cloud_secret_key), llm_api_key: configured(&snap.ark_api_key), llm_endpoint: llm_endpoint.is_some(), llm_endpoint_matches_default: llm_endpoint.is_some_and(|endpoint| { @@ -799,7 +802,10 @@ fn account_provider_kind(account: CredentialAccount) -> CredentialProviderKind { | CredentialAccount::AsrVocabularyId | CredentialAccount::AsrAdvancedConfig | CredentialAccount::XfyunAppId - | CredentialAccount::XfyunApiKey => CredentialProviderKind::Asr, + | CredentialAccount::XfyunApiKey + | CredentialAccount::TencentCloudAppId + | CredentialAccount::TencentCloudSecretId + | CredentialAccount::TencentCloudSecretKey => CredentialProviderKind::Asr, CredentialAccount::OmniApiKey | CredentialAccount::OmniEndpoint | CredentialAccount::OmniModel => CredentialProviderKind::Omni, @@ -831,6 +837,9 @@ fn parse_account(s: &str) -> Result { "asr.advanced_config" => Ok(CredentialAccount::AsrAdvancedConfig), "xfyun.app_id" => Ok(CredentialAccount::XfyunAppId), "xfyun.api_key" => Ok(CredentialAccount::XfyunApiKey), + "tencent_cloud.app_id" => Ok(CredentialAccount::TencentCloudAppId), + "tencent_cloud.secret_id" => Ok(CredentialAccount::TencentCloudSecretId), + "tencent_cloud.secret_key" => Ok(CredentialAccount::TencentCloudSecretKey), "omni.api_key" => Ok(CredentialAccount::OmniApiKey), "omni.endpoint" => Ok(CredentialAccount::OmniEndpoint), "omni.model" => Ok(CredentialAccount::OmniModel), diff --git a/openless-all/app/src-tauri/src/persistence/credentials.rs b/openless-all/app/src-tauri/src/persistence/credentials.rs index eaa643a4..03d25386 100644 --- a/openless-all/app/src-tauri/src/persistence/credentials.rs +++ b/openless-all/app/src-tauri/src/persistence/credentials.rs @@ -349,6 +349,15 @@ struct CredsAsrEntry { /// 讯飞实时语音转写 APIKey（接口密钥）。 #[serde(skip_serializing_if = "Option::is_none")] xfyunApiKey: Option, + /// 腾讯云账号 AppID（实时语音识别 WebSocket 路径参数）。 + #[serde(skip_serializing_if = "Option::is_none")] + tencentCloudAppId: Option, + /// 腾讯云 API 密钥 SecretID。 + #[serde(skip_serializing_if = "Option::is_none")] + tencentCloudSecretId: Option, + /// 腾讯云 API 密钥 SecretKey。 + #[serde(skip_serializing_if = "Option::is_none")] + tencentCloudSecretKey: Option, } impl CredsAsrEntry { @@ -378,6 +387,17 @@ impl CredsAsrEntry { && self.advancedConfig.as_deref().unwrap_or("").is_empty() && self.xfyunAppId.as_deref().unwrap_or("").is_empty() && self.xfyunApiKey.as_deref().unwrap_or("").is_empty() + && self.tencentCloudAppId.as_deref().unwrap_or("").is_empty() + && self + .tencentCloudSecretId + .as_deref() + .unwrap_or("") + .is_empty() + && self + .tencentCloudSecretKey + .as_deref() + .unwrap_or("") + .is_empty() } } @@ -1611,6 +1631,11 @@ fn lookup_account(root: &CredsRoot, account: CredentialAccount) -> Option asr.and_then(|e| pick(&e.advancedConfig)), CredentialAccount::XfyunAppId => asr.and_then(|e| pick(&e.xfyunAppId)), CredentialAccount::XfyunApiKey => asr.and_then(|e| pick(&e.xfyunApiKey)), + CredentialAccount::TencentCloudAppId => asr.and_then(|e| pick(&e.tencentCloudAppId)), + CredentialAccount::TencentCloudSecretId => asr.and_then(|e| pick(&e.tencentCloudSecretId)), + CredentialAccount::TencentCloudSecretKey => { + asr.and_then(|e| pick(&e.tencentCloudSecretKey)) + } CredentialAccount::OmniApiKey => omni.and_then(|e| pick(&e.apiKey)), CredentialAccount::OmniEndpoint => omni.and_then(|e| pick(&e.baseURL)), CredentialAccount::OmniModel => omni.and_then(|e| pick(&e.model)), @@ -1720,6 +1745,18 @@ fn write_account(root: &mut CredsRoot, account: CredentialAccount, value: Option let entry = root.providers.asr.entry(asr_id).or_default(); entry.xfyunApiKey = normalized; } + CredentialAccount::TencentCloudAppId => { + let entry = root.providers.asr.entry(asr_id).or_default(); + entry.tencentCloudAppId = normalized; + } + CredentialAccount::TencentCloudSecretId => { + let entry = root.providers.asr.entry(asr_id).or_default(); + entry.tencentCloudSecretId = normalized; + } + CredentialAccount::TencentCloudSecretKey => { + let entry = root.providers.asr.entry(asr_id).or_default(); + entry.tencentCloudSecretKey = normalized; + } CredentialAccount::OmniApiKey => { let entry = root.omni.providers.entry(omni_id).or_default(); entry.apiKey = normalized; @@ -1760,6 +1797,12 @@ pub enum CredentialAccount { XfyunAppId, /// 讯飞实时语音转写 APIKey。 XfyunApiKey, + /// 腾讯云账号 AppID。 + TencentCloudAppId, + /// 腾讯云 API 密钥 SecretID。 + TencentCloudSecretId, + /// 腾讯云 API 密钥 SecretKey。 + TencentCloudSecretKey, /// 多模态（Omni）模型的 API Key。仅多模态管线读取。 OmniApiKey, /// 多模态（Omni）模型的 Base URL。 @@ -1789,6 +1832,9 @@ impl CredentialAccount { CredentialAccount::AsrAdvancedConfig => "asr.advanced_config", CredentialAccount::XfyunAppId => "xfyun.app_id", CredentialAccount::XfyunApiKey => "xfyun.api_key", + CredentialAccount::TencentCloudAppId => "tencent_cloud.app_id", + CredentialAccount::TencentCloudSecretId => "tencent_cloud.secret_id", + CredentialAccount::TencentCloudSecretKey => "tencent_cloud.secret_key", CredentialAccount::OmniApiKey => "omni.api_key", CredentialAccount::OmniEndpoint => "omni.endpoint", CredentialAccount::OmniModel => "omni.model", @@ -1812,6 +1858,9 @@ impl CredentialAccount { CredentialAccount::AsrAdvancedConfig, CredentialAccount::XfyunAppId, CredentialAccount::XfyunApiKey, + CredentialAccount::TencentCloudAppId, + CredentialAccount::TencentCloudSecretId, + CredentialAccount::TencentCloudSecretKey, CredentialAccount::OmniApiKey, CredentialAccount::OmniEndpoint, CredentialAccount::OmniModel, @@ -1832,6 +1881,9 @@ pub struct CredentialsSnapshot { pub asr_model: Option, pub xfyun_app_id: Option, pub xfyun_api_key: Option, + pub tencent_cloud_app_id: Option, + pub tencent_cloud_secret_id: Option, + pub tencent_cloud_secret_key: Option, pub ark_api_key: Option, pub ark_model_id: Option, pub ark_endpoint: Option, @@ -1853,6 +1905,9 @@ fn credentials_snapshot(root: &CredsRoot, include_omni: bool) -> CredentialsSnap asr_model: lookup_account(root, CredentialAccount::AsrModel), xfyun_app_id: lookup_account(root, CredentialAccount::XfyunAppId), xfyun_api_key: lookup_account(root, CredentialAccount::XfyunApiKey), + tencent_cloud_app_id: lookup_account(root, CredentialAccount::TencentCloudAppId), + tencent_cloud_secret_id: lookup_account(root, CredentialAccount::TencentCloudSecretId), + tencent_cloud_secret_key: lookup_account(root, CredentialAccount::TencentCloudSecretKey), ark_api_key: lookup_account(root, CredentialAccount::ArkApiKey), ark_model_id: lookup_account(root, CredentialAccount::ArkModelId), ark_endpoint: lookup_account(root, CredentialAccount::ArkEndpoint), diff --git a/openless-all/app/src/i18n/en.ts b/openless-all/app/src/i18n/en.ts index 43e30e48..6d25f7c8 100644 --- a/openless-all/app/src/i18n/en.ts +++ b/openless-all/app/src/i18n/en.ts @@ -1012,6 +1012,7 @@ export const en: typeof zhCN = { minimax: 'MiniMax (M3)', stepfun: 'StepFun', opencode: 'OpenCode Zen', + tencentTokenHub: 'Tencent Cloud TokenHub', customChatCompletions: "Custom · Chat Completions", customResponses: "Custom · Responses", customMessages: "Custom · Messages", @@ -1030,6 +1031,7 @@ export const en: typeof zhCN = { asrOpenAiCompatible: 'Custom OpenAI-compatible', asrXiaomiMimo: 'Xiaomi MiMo ASR', asrIflytek: 'iFlytek Realtime ASR', + asrTencentCloud: 'Tencent Cloud Hunyuan Realtime ASR', asrElevenLabs: 'ElevenLabs Scribe', asrSherpaOnnxLocal: 'Local sherpa-onnx (experimental)', asrFoundryLocalWhisper: 'Local Whisper (Foundry Local)', @@ -1063,6 +1065,11 @@ export const en: typeof zhCN = { xfyunAppIdLabel: 'AppID', xfyunApiKeyLabel: 'API Key', xfyunNote: 'Get AppID and API Key from the iFlytek Open Platform "Realtime ASR" service page. Audio is 16 kHz / 16-bit / mono PCM; the standard API has no hotword parameter (configure personalized hotwords in the iFlytek console), and the language defaults to Mandarin Chinese.', + tencentCloudAppIdLabel: 'Tencent Cloud AppID', + tencentCloudSecretIdLabel: 'SecretID', + tencentCloudSecretKeyLabel: 'SecretKey', + tencentCloudNote: 'Uses Tencent Cloud Speech Recognition API credentials. The default Hy-ASR-3.0-preview supports Chinese, English, and 20 dialects; Preview accepts only mono 16 kHz PCM up to 60 seconds and does not yet support context or hotword boosting.', + tencentTokenHubNote: 'Only online language models are listed. Some models always use reasoning; turning reasoning off keeps that model\'s fixed behavior.', localAsrActiveNotice: 'Local ASR ({{name}}) is currently active. Switch or disable it from the Advanced tab.', localAsrTakeoverHint: 'Once "{{name}}" is enabled, the ASR provider will be taken over.', asrProviderTakenOver: 'A local engine is active. Pick another provider in the dropdown above to switch (the local engine stops automatically); manage local models under Services → Local models.', diff --git a/openless-all/app/src/i18n/ja.ts b/openless-all/app/src/i18n/ja.ts index 68dd9431..36a57b13 100644 --- a/openless-all/app/src/i18n/ja.ts +++ b/openless-all/app/src/i18n/ja.ts @@ -1014,6 +1014,7 @@ export const ja: typeof zhCN = { minimax: 'MiniMax（M3）', stepfun: 'StepFun（階躍星辰）', opencode: 'OpenCode Zen', + tencentTokenHub: 'Tencent Cloud TokenHub', customChatCompletions: "カスタム · Chat Completions", customResponses: "カスタム · Responses", customMessages: "カスタム · Messages", @@ -1032,6 +1033,7 @@ export const ja: typeof zhCN = { asrOpenAiCompatible: 'カスタム OpenAI 互換', asrXiaomiMimo: 'Xiaomi MiMo ASR', asrIflytek: 'iFlytek リアルタイム音声認識', + asrTencentCloud: 'Tencent Cloud Hunyuan リアルタイム ASR', asrElevenLabs: 'ElevenLabs Scribe', asrSherpaOnnxLocal: 'ローカル sherpa-onnx（実験的）', asrFoundryLocalWhisper: 'ローカル Whisper（Foundry Local）', @@ -1065,6 +1067,11 @@ export const ja: typeof zhCN = { xfyunAppIdLabel: 'AppID', xfyunApiKeyLabel: 'API Key', xfyunNote: 'iFlytek オープンプラットフォームの「リアルタイム音声認識」サービスページで AppID と API Key を取得します。音声は 16kHz / 16bit / モノラル PCM。標準版 API にホットワード引数はありません（iFlytek コンソールで個別ホットワードを設定）。言語はデフォルトで中国語（普通話）です。', + tencentCloudAppIdLabel: 'Tencent Cloud AppID', + tencentCloudSecretIdLabel: 'SecretID', + tencentCloudSecretKeyLabel: 'SecretKey', + tencentCloudNote: 'Tencent Cloud 音声認識 API の認証情報を使用します。既定の Hy-ASR-3.0-preview は中国語・英語・20 方言に対応します。Preview は 60 秒以内の 16kHz モノラル PCM のみ対応し、コンテキストとホットワード強化は未対応です。', + tencentTokenHubNote: '現在オンラインの言語モデルのみを表示します。一部のモデルは常に思考を使用し、思考をオフにしてもモデル固有の動作を維持します。', localAsrActiveNotice: '現在「{{name}}」を使用中。「詳細設定」タブから切り替えまたは無効化できます。', localAsrTakeoverHint: '「{{name}}」を有効化すると ASR プロバイダーが引き継がれます。', asrProviderTakenOver: 'ローカルエンジンを使用中です。上のドロップダウンで別のプロバイダーを選ぶと切り替えられます（ローカルエンジンは自動的に停止します）。ローカルモデルは「サービス → ローカルモデル」で管理します。', diff --git a/openless-all/app/src/i18n/ko.ts b/openless-all/app/src/i18n/ko.ts index 8ffe7000..09fd69a9 100644 --- a/openless-all/app/src/i18n/ko.ts +++ b/openless-all/app/src/i18n/ko.ts @@ -1014,6 +1014,7 @@ export const ko: typeof zhCN = { minimax: 'MiniMax (M3)', stepfun: 'StepFun', opencode: 'OpenCode Zen', + tencentTokenHub: 'Tencent Cloud TokenHub', customChatCompletions: "사용자 지정 · Chat Completions", customResponses: "사용자 지정 · Responses", customMessages: "사용자 지정 · Messages", @@ -1032,6 +1033,7 @@ export const ko: typeof zhCN = { asrOpenAiCompatible: '커스텀 OpenAI 호환', asrXiaomiMimo: 'Xiaomi MiMo ASR', asrIflytek: 'iFlytek 실시간 음성 인식', + asrTencentCloud: 'Tencent Cloud Hunyuan 실시간 ASR', asrElevenLabs: 'ElevenLabs Scribe', asrSherpaOnnxLocal: '로컬 sherpa-onnx(실험적)', asrFoundryLocalWhisper: '로컬 Whisper(Foundry Local)', @@ -1065,6 +1067,11 @@ export const ko: typeof zhCN = { xfyunAppIdLabel: 'AppID', xfyunApiKeyLabel: 'API Key', xfyunNote: 'iFlytek 오픈 플랫폼 "실시간 음성 인식" 서비스 페이지에서 AppID와 API Key를 가져옵니다. 오디오는 16kHz/16bit/모노 PCM입니다. 표준 API에는 핫워드 매개변수가 없으며(iFlytek 콘솔에서 개별 핫워드 설정), 언어는 기본적으로 중국어(보통화)입니다.', + tencentCloudAppIdLabel: 'Tencent Cloud AppID', + tencentCloudSecretIdLabel: 'SecretID', + tencentCloudSecretKeyLabel: 'SecretKey', + tencentCloudNote: 'Tencent Cloud 음성 인식 API 자격 증명을 사용합니다. 기본 Hy-ASR-3.0-preview는 중국어·영어·20개 방언을 지원합니다. Preview는 60초 이내의 16kHz 모노 PCM만 지원하며, 컨텍스트와 핫워드 강화는 아직 지원하지 않습니다.', + tencentTokenHubNote: '현재 온라인인 언어 모델만 표시합니다. 일부 모델은 항상 추론을 사용하며, 추론을 꺼도 해당 모델의 고정 동작을 유지합니다.', localAsrActiveNotice: '현재 "{{name}}" 사용 중. "고급" 탭에서 전환 또는 비활성화할 수 있습니다.', localAsrTakeoverHint: '"{{name}}" 활성화 시 ASR 프로바이더가 인수됩니다.', asrProviderTakenOver: '현재 로컬 엔진을 사용 중입니다. 위의 드롭다운에서 다른 공급자를 선택하면 전환됩니다(로컬 엔진은 자동으로 중지됨). 로컬 모델은 「서비스 → 로컬 모델」에서 관리합니다.', diff --git a/openless-all/app/src/i18n/zh-CN.ts b/openless-all/app/src/i18n/zh-CN.ts index 22496dcc..b49e8e2b 100644 --- a/openless-all/app/src/i18n/zh-CN.ts +++ b/openless-all/app/src/i18n/zh-CN.ts @@ -1010,6 +1010,7 @@ export const zhCN = { minimax: 'MiniMax（M3）', stepfun: 'StepFun（阶跃星辰）', opencode: 'OpenCode Zen', + tencentTokenHub: '腾讯云 TokenHub', customChatCompletions: "自定义 · Chat Completions", customResponses: "自定义 · Responses", customMessages: "自定义 · Messages", @@ -1028,6 +1029,7 @@ export const zhCN = { asrOpenAiCompatible: '自定义 OpenAI 兼容', asrXiaomiMimo: '小米 MiMo ASR', asrIflytek: '讯飞实时语音转写', + asrTencentCloud: '腾讯云混元实时 ASR', asrElevenLabs: 'ElevenLabs Scribe', asrSherpaOnnxLocal: '本地 sherpa-onnx（实验性）', asrFoundryLocalWhisper: '本地 Whisper（Foundry Local）', @@ -1061,6 +1063,11 @@ export const zhCN = { xfyunAppIdLabel: 'AppID', xfyunApiKeyLabel: 'API Key', xfyunNote: '在讯飞开放平台「实时语音转写」服务页获取 AppID 与 API Key。音频为 16kHz/16bit/单声道 PCM；标准版接口暂不支持热词参数（可在讯飞控制台配置个性化热词），语种默认中文普通话。', + tencentCloudAppIdLabel: '腾讯云 AppID', + tencentCloudSecretIdLabel: 'SecretID', + tencentCloudSecretKeyLabel: 'SecretKey', + tencentCloudNote: '使用腾讯云「语音识别」服务的 API 密钥。默认 Hy-ASR-3.0-preview 支持中英与 20 种方言；Preview 仅支持 60 秒以内的 16kHz 单声道 PCM，暂不支持上下文或热词增强。', + tencentTokenHubNote: '仅显示当前在线的语言模型。部分模型始终启用思考；关闭思考开关时将沿用该模型的固定行为。', localAsrActiveNotice: '当前已启用「{{name}}」，可在「高级」中切换或禁用。', localAsrTakeoverHint: '启动「{{name}}」后，ASR 提供商将被接管。', asrProviderTakenOver: '当前用的是本地引擎，在上方下拉直接选其它供应商即可切换（本地引擎会自动停用）；本地模型在「服务 → 本地模型」里管理。', diff --git a/openless-all/app/src/i18n/zh-TW.ts b/openless-all/app/src/i18n/zh-TW.ts index 71b4cacd..3d60d311 100644 --- a/openless-all/app/src/i18n/zh-TW.ts +++ b/openless-all/app/src/i18n/zh-TW.ts @@ -1012,6 +1012,7 @@ export const zhTW: typeof zhCN = { minimax: 'MiniMax（M3）', stepfun: 'StepFun（階躍星辰）', opencode: 'OpenCode Zen', + tencentTokenHub: '騰訊雲 TokenHub', customChatCompletions: "自訂 · Chat Completions", customResponses: "自訂 · Responses", customMessages: "自訂 · Messages", @@ -1030,6 +1031,7 @@ export const zhTW: typeof zhCN = { asrOpenAiCompatible: '自訂 OpenAI 相容', asrXiaomiMimo: '小米 MiMo ASR', asrIflytek: '訊飛即時語音轉寫', + asrTencentCloud: '騰訊雲混元即時 ASR', asrElevenLabs: 'ElevenLabs Scribe', asrSherpaOnnxLocal: '本地 sherpa-onnx（實驗性）', asrFoundryLocalWhisper: '本地 Whisper（Foundry Local）', @@ -1063,6 +1065,11 @@ export const zhTW: typeof zhCN = { xfyunAppIdLabel: 'AppID', xfyunApiKeyLabel: 'API Key', xfyunNote: '在訊飛開放平台「即時語音轉寫」服務頁取得 AppID 與 API Key。音訊為 16kHz/16bit/單聲道 PCM；標準版介面暫不支援熱詞參數（可在訊飛控制台設定個人化熱詞），語種預設中文普通話。', + tencentCloudAppIdLabel: '騰訊雲 AppID', + tencentCloudSecretIdLabel: 'SecretID', + tencentCloudSecretKeyLabel: 'SecretKey', + tencentCloudNote: '使用騰訊雲「語音辨識」服務的 API 金鑰。預設 Hy-ASR-3.0-preview 支援中英與 20 種方言；Preview 僅支援 60 秒內的 16kHz 單聲道 PCM，暫不支援上下文或熱詞增強。', + tencentTokenHubNote: '僅顯示目前在線的語言模型。部分模型始終啟用思考；關閉思考開關時將沿用該模型的固定行為。', localAsrActiveNotice: '當前已啓用「{{name}}」，可在「高級」中切換或停用。', localAsrTakeoverHint: '啓動「{{name}}」後，ASR 提供商將被接管。', asrProviderTakenOver: '目前使用的是本地引擎，在上方下拉直接選其他供應商即可切換（本地引擎會自動停用）；本地模型在「服務 → 本地模型」裡管理。', diff --git a/openless-all/app/src/lib/ipc/mock-provider-descriptors.json b/openless-all/app/src/lib/ipc/mock-provider-descriptors.json index f991e8fb..e5c15e31 100644 --- a/openless-all/app/src/lib/ipc/mock-provider-descriptors.json +++ b/openless-all/app/src/lib/ipc/mock-provider-descriptors.json @@ -15,6 +15,7 @@ {"kind":"llm","providerType":"minimax","labelKey":"minimax","defaultEndpoint":"https://api.minimaxi.com/v1","defaultModel":"MiniMax-M3","authRequirement":"api_key_unless_custom_endpoint","validationProbe":"llm_text","staticModels":[],"defaultRequestFormat":"chat_completions","supportedRequestFormats":["chat_completions","responses","messages"]}, {"kind":"llm","providerType":"stepfun","labelKey":"stepfun","defaultEndpoint":"https://api.stepfun.com/v1","defaultModel":"step-1o-turbo-vision","authRequirement":"api_key_unless_custom_endpoint","validationProbe":"llm_text","staticModels":[],"defaultRequestFormat":"chat_completions","supportedRequestFormats":["chat_completions","responses","messages"]}, {"kind":"llm","providerType":"opencode","labelKey":"opencode","defaultEndpoint":"https://opencode.ai/zen/v1","defaultModel":"deepseek-v4-flash","authRequirement":"api_key_unless_custom_endpoint","validationProbe":"llm_text","staticModels":[],"defaultRequestFormat":"chat_completions","supportedRequestFormats":["chat_completions","responses","messages"]}, + {"kind":"llm","providerType":"tencentTokenHub","labelKey":"tencentTokenHub","defaultEndpoint":"https://tokenhub.tencentmaas.com/v1","defaultModel":"hy3","authRequirement":"api_key_unless_custom_endpoint","validationProbe":"llm_text","staticModels":[],"defaultRequestFormat":null,"supportedRequestFormats":[]}, {"kind":"llm","providerType":"custom","labelKey":"customChatCompletions","defaultEndpoint":null,"defaultModel":null,"authRequirement":"api_key_unless_custom_endpoint","validationProbe":"llm_text","staticModels":[],"defaultRequestFormat":"chat_completions","supportedRequestFormats":["chat_completions","responses","messages"]}, {"kind":"llm","providerType":"custom_responses","labelKey":"customResponses","defaultEndpoint":null,"defaultModel":null,"authRequirement":"api_key_unless_custom_endpoint","validationProbe":"llm_text","staticModels":[],"defaultRequestFormat":"responses","supportedRequestFormats":["chat_completions","responses","messages"]}, {"kind":"llm","providerType":"custom_messages","labelKey":"customMessages","defaultEndpoint":null,"defaultModel":null,"authRequirement":"api_key_unless_custom_endpoint","validationProbe":"llm_text","staticModels":[],"defaultRequestFormat":"messages","supportedRequestFormats":["chat_completions","responses","messages"]}, @@ -34,6 +35,7 @@ {"kind":"asr","providerType":"openai-compatible","labelKey":"asrOpenAiCompatible","defaultEndpoint":null,"defaultModel":null,"authRequirement":"endpoint_model_optional_api_key","validationProbe":"asr_silence","staticModels":[],"defaultRequestFormat":null,"supportedRequestFormats":[]}, {"kind":"asr","providerType":"xiaomi-mimo-asr","labelKey":"asrXiaomiMimo","defaultEndpoint":"https://api.xiaomimimo.com/v1","defaultModel":"mimo-v2.5-asr","authRequirement":"api_key","validationProbe":"asr_silence","staticModels":["mimo-v2.5-asr"],"defaultRequestFormat":null,"supportedRequestFormats":[]}, {"kind":"asr","providerType":"iflytek","labelKey":"asrIflytek","defaultEndpoint":null,"defaultModel":null,"authRequirement":"xfyun","validationProbe":"asr_silence_allows_no_final","staticModels":[],"defaultRequestFormat":null,"supportedRequestFormats":[]}, + {"kind":"asr","providerType":"tencent-cloud","labelKey":"asrTencentCloud","defaultEndpoint":null,"defaultModel":"Hy-ASR-3.0-preview","authRequirement":"tencent_cloud","validationProbe":"asr_silence_allows_no_final","staticModels":[],"defaultRequestFormat":null,"supportedRequestFormats":[]}, {"kind":"asr","providerType":"foundry-local-whisper","labelKey":"asrFoundryLocalWhisper","defaultEndpoint":null,"defaultModel":null,"authRequirement":"none","validationProbe":"unsupported","staticModels":[],"defaultRequestFormat":null,"supportedRequestFormats":[]}, {"kind":"asr","providerType":"local-whisper","labelKey":"asrLocalWhisper","defaultEndpoint":null,"defaultModel":null,"authRequirement":"none","validationProbe":"unsupported","staticModels":[],"defaultRequestFormat":null,"supportedRequestFormats":[]}, {"kind":"asr","providerType":"sherpa-onnx-local","labelKey":"asrSherpaOnnxLocal","defaultEndpoint":null,"defaultModel":null,"authRequirement":"none","validationProbe":"unsupported","staticModels":[],"defaultRequestFormat":null,"supportedRequestFormats":[]}, diff --git a/openless-all/app/src/lib/ipc/providers.ts b/openless-all/app/src/lib/ipc/providers.ts index f536d8ce..2011e04e 100644 --- a/openless-all/app/src/lib/ipc/providers.ts +++ b/openless-all/app/src/lib/ipc/providers.ts @@ -11,6 +11,7 @@ export type AuthRequirement = | 'api_key_unless_custom_endpoint' | 'volcengine' | 'xfyun' + | 'tencent_cloud' | 'o_auth' export interface ProviderDescriptor { diff --git a/openless-all/app/src/pages/Overview.tsx b/openless-all/app/src/pages/Overview.tsx index 4c499e52..c0370edd 100644 --- a/openless-all/app/src/pages/Overview.tsx +++ b/openless-all/app/src/pages/Overview.tsx @@ -54,6 +54,7 @@ const LLM_NAME_KEY_BY_ID: Record = { alibabaCoding: 'alibabaCoding', codingPlanX: 'codingPlanX', opencode: 'opencode', + tencentTokenHub: 'tencentTokenHub', custom: 'custom', }; diff --git a/openless-all/app/src/pages/settings/LlmProtocolFields.test.ts b/openless-all/app/src/pages/settings/LlmProtocolFields.test.ts index f36e915c..b2aad90a 100644 --- a/openless-all/app/src/pages/settings/LlmProtocolFields.test.ts +++ b/openless-all/app/src/pages/settings/LlmProtocolFields.test.ts @@ -23,6 +23,10 @@ const opencode = presets.find(p => p.id === 'opencode'); assert(opencode?.defaultRequestFormat === 'chat_completions' && opencode.defaultEndpoint === 'https://opencode.ai/zen/v1' && opencode.defaultModel === 'deepseek-v4-flash', 'OpenCode browser preset must retain Core defaults'); +const tokenhub = presets.find(p => p.id === 'tencentTokenHub'); +assert(tokenhub?.defaultRequestFormat === null + && tokenhub.supportedRequestFormats?.length === 0, + 'TokenHub browser preset must stay fixed to Chat Completions'); assert(presets.find(p => p.id === 'custom_messages')?.defaultRequestFormat === 'messages', 'Picker must retain Core protocol defaults'); for (const preset of presets) { assert(preset.supportedRequestFormats?.length === (preset.defaultRequestFormat ? 3 : 0), 'Picker must retain Core protocol capabilities'); diff --git a/openless-all/app/src/pages/settings/ProvidersSection.test.ts b/openless-all/app/src/pages/settings/ProvidersSection.test.ts index a89e8a41..a2bd1c47 100644 --- a/openless-all/app/src/pages/settings/ProvidersSection.test.ts +++ b/openless-all/app/src/pages/settings/ProvidersSection.test.ts @@ -7,6 +7,12 @@ const atlascloudPreset = LLM_LABELS.find(p => p.id === 'atlascloud'); if (LLM_LABELS.find(p => p.id === 'opencode')?.nameKey !== 'opencode') { throw new Error('OpenCode LLM label is missing'); } +if (LLM_LABELS.find(p => p.id === 'tencentTokenHub')?.nameKey !== 'tencentTokenHub') { + throw new Error('Tencent Cloud TokenHub LLM label is missing'); +} +if (ASR_LABELS.find(p => p.id === 'tencent-cloud')?.nameKey !== 'asrTencentCloud') { + throw new Error('Tencent Cloud ASR label is missing'); +} if (!atlascloudPreset) { throw new Error('Atlas Cloud LLM preset is missing'); diff --git a/openless-all/app/src/pages/settings/ProvidersSection.tsx b/openless-all/app/src/pages/settings/ProvidersSection.tsx index 514081a5..598013ac 100644 --- a/openless-all/app/src/pages/settings/ProvidersSection.tsx +++ b/openless-all/app/src/pages/settings/ProvidersSection.tsx @@ -94,7 +94,7 @@ export const LLM_LABELS = [ ['codex_oauth', 'codexOAuth'], ['mimo', 'mimo'], ['cometapi', 'cometapi'], ['openrouterFree', 'openrouterFree'], ['orcarouter', 'orcarouter'], ['alibabaCoding', 'alibabaCoding'], ['codingPlanX', 'codingPlanX'], ['minimax', 'minimax'], ['stepfun', 'stepfun'], - ['opencode', 'opencode'], + ['opencode', 'opencode'], ['tencentTokenHub', 'tencentTokenHub'], ['custom', 'customChatCompletions'], ['custom_responses', 'customResponses'], ['custom_messages', 'customMessages'], ].map(([id, nameKey]) => ({ id, nameKey })) as readonly { id: string; nameKey: string }[]; @@ -280,6 +280,11 @@ export function ChannelCredentialFields({ + {providerType === 'tencentTokenHub' && ( +
+ {t('settings.providers.tencentTokenHubNote')} +
+ )} ); } @@ -365,6 +370,29 @@ export function ChannelCredentialFields({ ); } + if (descriptor?.authRequirement === 'tencent_cloud') { + return ( + <> + + + + +
+ {t('settings.providers.tencentCloudNote')} +
+ + + ); + } + // 本地引擎（qwen3 / sherpa / foundry / Apple 语音）没有 key 与地址；模型的下载与 // 切换仍由「高级 → 本地模型」里的 负责，这里只说明一句。 if (descriptor?.authRequirement === 'none') { diff --git a/openless-all/app/src/pages/settings/shared.tsx b/openless-all/app/src/pages/settings/shared.tsx index 7a191f8a..55b79094 100644 --- a/openless-all/app/src/pages/settings/shared.tsx +++ b/openless-all/app/src/pages/settings/shared.tsx @@ -228,6 +228,7 @@ export const ASR_LABELS = [ { id: 'openai-compatible', nameKey: 'asrOpenAiCompatible' }, { id: 'xiaomi-mimo-asr', nameKey: 'asrXiaomiMimo' }, { id: 'iflytek', nameKey: 'asrIflytek' }, + { id: 'tencent-cloud', nameKey: 'asrTencentCloud' }, { id: 'foundry-local-whisper', nameKey: 'asrFoundryLocalWhisper' }, { id: 'local-whisper', nameKey: 'asrLocalWhisper' }, { id: 'sherpa-onnx-local', nameKey: 'asrSherpaOnnxLocal' },