3 篇文章带有标签 “multilingual”

本文档介绍 FunAudioLLM，这是一个旨在增强人类与大型语言模型（LLM）之间自然语音交互的框架。其核心是两个创新模型：用于高精度多语种语音识别、情感识别和音频事件检测的 SenseVoice；以及用于多语种、音色和情感控制的自然语音生成的 CosyVoice。SenseVoice 具有极低的延迟并支持超过 50 种语言，而 CosyVoice 在多语种语音生成、零样本语音生成、跨语言语音克隆以及指令遵循能力方面表现出色。与 SenseVoice 和 CosyVoice 相关的模型已在 Modelscope 和 Huggingface 上开源，同时相应的训练、推理和微调代码也已在 GitHub 上发布。通过将这些模型与 LLM 集成，FunAudioLLM 能够实现语音翻译、情感语音聊天、交互式播客和富有表现力的有声读物叙述等应用，从而推动语音交互技术的边界。

SenseVoice

• GitHub - SenseVoice
• Modelscope - SenseVoice多语言语音理解模型Small

CosyVoice 2

• GitHub - CosyVoice
• Modelscope - CosyVoice语音生成大模型2.0-0.5B

参考资料 FunAudioLLM: Voice Understanding and Generation Foundation Models for Nat

Abstract(摘要)

This report introduces the Qwen2 series, the latest addition to our large language models and large multimodal models. We release a comprehensive suite of foundational and instruction-tuned language models, encompassing a parameter range from 0.5 to 72 billion, featuring dense models and a Mixture-of-Experts model. Qwen2 surpasses most prior open-weight models, including its predecessor Qwen1.

Seamless Communication

• ASR: Automatic speech recognition for 96 languages.
• S2ST: Speech-to-Speech translation from 100 source speech languages into 35 target speech languages.
• S2TT: Speech-to-text translation from 100 source speech languages into 95 target text languages.
• T2ST: Text-to-Speech translation from 95 source text languages into 35 target speech languages.
• T2TT: Text-to-text translation (MT) from 95 source text languages into 95 target text languages.

SeamlessM4T 概述

安装 [Seamless Communication][seamless_communication]

克隆仓库 git clone https://github.