RNNTBundle¶

class torchaudio.pipelines.RNNTBundle[source]¶

用于执行具有 RNN-T 模型的自动语音识别 (ASR，语音到文本) 推理的组件包。该数据类。

更具体地说，该类提供的方法可以生成特征提取管道、包装指定 RNN-T 模型的解码器以及输出令牌后处理器，这些共同构成了一个完整的端到端 ASR 推理管道，该管道在给定原始波形的情况下生成文本序列。

它可以支持非流式（全上下文）推理以及流式推理。

用户不应直接实例化此类对象；相反，用户应使用模块内存在的实例（代表预训练模型），例如 torchaudio.pipelines.EMFORMER_RNNT_BASE_LIBRISPEECH。

示例

>>> import torchaudio
>>> from torchaudio.pipelines import EMFORMER_RNNT_BASE_LIBRISPEECH
>>> import torch
>>>
>>> # Non-streaming inference.
>>> # Build feature extractor, decoder with RNN-T model, and token processor.
>>> feature_extractor = EMFORMER_RNNT_BASE_LIBRISPEECH.get_feature_extractor()
100%|███████████████████████████████| 3.81k/3.81k [00:00<00:00, 4.22MB/s]
>>> decoder = EMFORMER_RNNT_BASE_LIBRISPEECH.get_decoder()
Downloading: "https://download.pytorch.org/torchaudio/models/emformer_rnnt_base_librispeech.pt"
100%|███████████████████████████████| 293M/293M [00:07<00:00, 42.1MB/s]
>>> token_processor = EMFORMER_RNNT_BASE_LIBRISPEECH.get_token_processor()
100%|███████████████████████████████| 295k/295k [00:00<00:00, 25.4MB/s]
>>>
>>> # Instantiate LibriSpeech dataset; retrieve waveform for first sample.
>>> dataset = torchaudio.datasets.LIBRISPEECH("/home/librispeech", url="test-clean")
>>> waveform = next(iter(dataset))[0].squeeze()
>>>
>>> with torch.no_grad():
>>>     # Produce mel-scale spectrogram features.
>>>     features, length = feature_extractor(waveform)
>>>
>>>     # Generate top-10 hypotheses.
>>>     hypotheses = decoder(features, length, 10)
>>>
>>> # For top hypothesis, convert predicted tokens to text.
>>> text = token_processor(hypotheses[0][0])
>>> print(text)
he hoped there would be stew for dinner turnips and carrots and bruised potatoes and fat mutton pieces to [...]
>>>
>>>
>>> # Streaming inference.
>>> hop_length = EMFORMER_RNNT_BASE_LIBRISPEECH.hop_length
>>> num_samples_segment = EMFORMER_RNNT_BASE_LIBRISPEECH.segment_length * hop_length
>>> num_samples_segment_right_context = (
>>>     num_samples_segment + EMFORMER_RNNT_BASE_LIBRISPEECH.right_context_length * hop_length
>>> )
>>>
>>> # Build streaming inference feature extractor.
>>> streaming_feature_extractor = EMFORMER_RNNT_BASE_LIBRISPEECH.get_streaming_feature_extractor()
>>>
>>> # Process same waveform as before, this time sequentially across overlapping segments
>>> # to simulate streaming inference. Note the usage of ``streaming_feature_extractor`` and ``decoder.infer``.
>>> state, hypothesis = None, None
>>> for idx in range(0, len(waveform), num_samples_segment):
>>>     segment = waveform[idx: idx + num_samples_segment_right_context]
>>>     segment = torch.nn.functional.pad(segment, (0, num_samples_segment_right_context - len(segment)))
>>>     with torch.no_grad():
>>>         features, length = streaming_feature_extractor(segment)
>>>         hypotheses, state = decoder.infer(features, length, 10, state=state, hypothesis=hypothesis)
>>>     hypothesis = hypotheses[0]
>>>     transcript = token_processor(hypothesis[0])
>>>     if transcript:
>>>         print(transcript, end=" ", flush=True)
he hoped there would be stew for dinner turn ips and car rots and bru 'd oes and fat mut ton pieces to [...]

使用 RNNTBundle 的教程

使用 Emformer RNN-T 进行在线 ASR

使用 Emformer RNN-T 进行设备端 ASR

属性¶

hop_length¶

property RNNTBundle.hop_length: int¶

模型输入的帧之间的样本数。

类型: int

n_fft¶

property RNNTBundle.n_fft: int¶

要使用的 FFT 窗口的大小。

类型: int

n_mels¶

property RNNTBundle.n_mels: int¶

要从输入波形中提取的梅尔频谱图特征的数量。

类型: int

right_context_length¶

property RNNTBundle.right_context_length: int¶

模型输入的右上下文块中的帧数。

类型: int

sample_rate¶

property RNNTBundle.sample_rate: int¶

输入波形的采样率（每秒周期数）。

类型: int

segment_length¶

property RNNTBundle.segment_length: int¶

模型输入的片段中的帧数。

类型: int

方法¶

get_decoder¶

RNNTBundle.get_decoder() → RNNTBeamSearch[source]¶

构建 RNN-T 解码器。

返回: RNNTBeamSearch

get_feature_extractor¶

RNNTBundle.get_feature_extractor() → FeatureExtractor[source]¶

为非流式（全上下文）ASR 构建特征提取器。

返回: FeatureExtractor

get_streaming_feature_extractor¶

RNNTBundle.get_streaming_feature_extractor() → FeatureExtractor[source]¶

为流式（同步）ASR 构建特征提取器。

返回: FeatureExtractor

get_token_processor¶

RNNTBundle.get_token_processor() → TokenProcessor[source]¶

构建令牌处理器。

返回: TokenProcessor