注意

点击此处下载完整的示例代码

使用混合 Demucs 进行音乐源分离¶

本教程展示了如何使用混合 Demucs 模型来进行音乐分离。

1. 概述¶

音乐分离的过程包括以下步骤：

构建混合 Demucs 管道。
将波形格式化为预期大小的块，并（带重叠）循环遍历这些块，然后将其输入管道。
收集输出块，并根据重叠方式进行组合。

混合 Demucs [Défossez, 2021] 模型是 Demucs 模型的改进版本，Demucs 是一个基于波形的模型，可以将音乐分离成各自的音源，如人声、贝斯和鼓。混合 Demucs 有效地利用了频谱图来学习频率域，并且还使用了时间卷积。

2. 准备¶

首先，我们安装必要的依赖项。第一个要求是 torchaudio 和 torch。

import torch
import torchaudio

print(torch.__version__)
print(torchaudio.__version__)

import matplotlib.pyplot as plt

2.10.0.dev20251013+cu126
2.8.0a0+1d65bbe

from IPython.display import Audio
from torchaudio.pipelines import HDEMUCS_HIGH_MUSDB_PLUS
from torchaudio.utils import _download_asset

3. 构建管道¶

预训练模型权重和相关的管道组件被打包为 torchaudio.pipelines.HDEMUCS_HIGH_MUSDB_PLUS()。这是一个在 MUSDB18-HQ 和额外的内部额外训练数据上训练的 torchaudio.models.HDemucs 模型。这个特定模型适用于更高的采样率，大约 44.1 kHz，nfft 值为 4096，模型实现深度为 6。

bundle = HDEMUCS_HIGH_MUSDB_PLUS

model = bundle.get_model()

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")

model.to(device)

sample_rate = bundle.sample_rate

print(f"Sample rate: {sample_rate}")

0%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
6%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
1%
1%
1%
2%
2%
3%
3%
3%
4%
4%
5%
5%
5%
6%
6%
6%
7%
7%
8%
8%
8%
9%
9%
0%
0%
0%
1%
1%
2%
2%
2%
3%
3%
4%
4%
4%
5%
5%
5%
6%
6%
7%
7%
7%
8%
8%
9%
9%
9%
0%
0%
Sample rate: 44100

4. 配置应用程序函数¶

由于 HDemucs 是一个庞大且内存消耗的模型，因此很难获得足够的内存一次性将模型应用于整个歌曲。为了绕过这个限制，可以通过将歌曲分成更小的片段，并逐个片段运行模型，然后重新组合，来获得完整歌曲的分离音轨。

在这样做时，确保每个片段之间有一些重叠非常重要，以适应边缘的伪影。由于模型的特性，有时边缘会包含不准确或不需要的声音。

我们提供了下面的分块和组合的示例实现。此实现将在两侧各使用 1 秒的重叠，然后对每一侧执行线性淡入和淡出。使用淡入的重叠，我将这些片段加在一起，以确保整个过程中音量恒定。这通过减少模型输出的边缘部分的使用来适应伪影。

https://download.pytorch.org/torchaudio/tutorial-assets/HDemucs_Drawing.jpg

from torchaudio.transforms import Fade


def separate_sources(
    model,
    mix,
    segment=10.0,
    overlap=0.1,
    device=None,
):
    """
    Apply model to a given mixture. Use fade, and add segments together in order to add model segment by segment.

    Args:
        segment (int): segment length in seconds
        device (torch.device, str, or None): if provided, device on which to
            execute the computation, otherwise `mix.device` is assumed.
            When `device` is different from `mix.device`, only local computations will
            be on `device`, while the entire tracks will be stored on `mix.device`.
    """
    if device is None:
        device = mix.device
    else:
        device = torch.device(device)

    batch, channels, length = mix.shape

    chunk_len = int(sample_rate * segment * (1 + overlap))
    start = 0
    end = chunk_len
    overlap_frames = overlap * sample_rate
    fade = Fade(fade_in_len=0, fade_out_len=int(overlap_frames), fade_shape="linear")

    final = torch.zeros(batch, len(model.sources), channels, length, device=device)

    while start < length - overlap_frames:
        chunk = mix[:, :, start:end]
        with torch.no_grad():
            out = model.forward(chunk)
        out = fade(out)
        final[:, :, :, start:end] += out
        if start == 0:
            fade.fade_in_len = int(overlap_frames)
            start += int(chunk_len - overlap_frames)
        else:
            start += chunk_len
        end += chunk_len
        if end >= length:
            fade.fade_out_len = 0
    return final


def plot_spectrogram(stft, title="Spectrogram"):
    magnitude = stft.abs()
    spectrogram = 20 * torch.log10(magnitude + 1e-8).numpy()
    _, axis = plt.subplots(1, 1)
    axis.imshow(spectrogram, cmap="viridis", vmin=-60, vmax=0, origin="lower", aspect="auto")
    axis.set_title(title)
    plt.tight_layout()

5. 运行模型¶

最后，我们运行模型并将分离的音轨文件存储在目录中。

作为测试歌曲，我们将使用 NightOwl 的 "A Classic Education"，来自 MedleyDB (Creative Commons BY-NC-SA 4.0)。这首歌曲也位于 MUSDB18-HQ 数据集中的 train 音源里。

为了用不同的歌曲进行测试，可以更改下面的变量名和 URL，以及参数，以不同的方式测试歌曲分离器。

# We download the audio file from our storage. Feel free to download another file and use audio from a specific path
SAMPLE_SONG = _download_asset("tutorial-assets/hdemucs_mix.wav")
waveform, sample_rate = torchaudio.load(SAMPLE_SONG)  # replace SAMPLE_SONG with desired path for different song
waveform = waveform.to(device)
mixture = waveform

# parameters
segment: int = 10
overlap = 0.1

print("Separating track")

ref = waveform.mean(0)
waveform = (waveform - ref.mean()) / ref.std()  # normalization

sources = separate_sources(
    model,
    waveform[None],
    device=device,
    segment=segment,
    overlap=overlap,
)[0]
sources = sources * ref.std() + ref.mean()

sources_list = model.sources
sources = list(sources)

audios = dict(zip(sources_list, sources))

4%
9%
3%
7%
2%
6%
0%
5%
9%
3%
8%
2%
6%
1%
5%
9%
4%
8%
2%
7%
1%
5%
0%
4%
8%
3%
7%
1%
6%
0%
4%
9%
3%
7%
2%
6%
0%
5%
9%
3%
8%
2%
6%
1%
5%
9%
4%
8%
2%
7%
1%
5%
0%
4%
8%
3%
7%
1%
6%
0%
4%
9%
3%
7%
2%
6%
0%
5%
9%
3%
8%
2%
6%
1%
5%
9%
4%
8%
2%
7%
1%
5%
0%
4%
8%
3%
7%
1%
6%
0%
4%
9%
3%
7%
2%
6%
0%
5%
9%
3%
8%
2%
6%
1%
5%
9%
4%
8%
2%
7%
1%
5%
0%
4%
8%
3%
7%
1%
6%
0%
4%
9%
3%
7%
2%
6%
0%
5%
9%
3%
8%
2%
6%
1%
5%
9%
4%
8%
2%
7%
1%
5%
0%
4%
8%
3%
7%
1%
6%
0%
4%
9%
3%
7%
2%
6%
0%
5%
9%
3%
8%
2%
6%
1%
5%
9%
4%
8%
2%
7%
1%
5%
0%
4%
8%
3%
7%
1%
6%
0%
4%
9%
3%
7%
2%
6%
0%
5%
9%
3%
8%
2%
6%
1%
5%
9%
4%
8%
2%
7%
1%
5%
0%
4%
8%
3%
7%
1%
6%
0%
4%
9%
3%
7%
2%
6%
0%
5%
9%
4%
8%
2%
7%
1%
5%
0%
4%
8%
3%
7%
0%
Separating track

5.1 分离音轨¶

默认加载的预训练权重集合包含 4 个分离的音源：鼓、贝斯、其他和人声，按此顺序。它们已存储在字典 "audios" 中，因此可以通过那里访问。对于这四个音源，每个都有一个单独的单元格，它将创建音频、频谱图以及计算 SDR 分数。SDR 是信号失真比，本质上是音频“质量”的表示。

N_FFT = 4096
N_HOP = 4
stft = torchaudio.transforms.Spectrogram(
    n_fft=N_FFT,
    hop_length=N_HOP,
    power=None,
)

5.2 音频分段与处理¶

以下是分段 5 秒音轨以输入频谱图并计算 respective SDR 分数的处理步骤。

def output_results(original_source: torch.Tensor, predicted_source: torch.Tensor, source: str):
    # If you have installed the mir_eval package, you can calculate the SDR score with
    # `mir_eval.separation.bss_eval_sources(original_source.detach().numpy(), predicted_source.detach().numpy())[0].mean()`
    plot_spectrogram(stft(predicted_source)[0], f"Spectrogram - {source}")
    return Audio(predicted_source, rate=sample_rate)


segment_start = 150
segment_end = 155

frame_start = segment_start * sample_rate
frame_end = segment_end * sample_rate

drums_original = _download_asset("tutorial-assets/hdemucs_drums_segment.wav")
bass_original = _download_asset("tutorial-assets/hdemucs_bass_segment.wav")
vocals_original = _download_asset("tutorial-assets/hdemucs_vocals_segment.wav")
other_original = _download_asset("tutorial-assets/hdemucs_other_segment.wav")

drums_spec = audios["drums"][:, frame_start:frame_end].cpu()
drums, sample_rate = torchaudio.load(drums_original)

bass_spec = audios["bass"][:, frame_start:frame_end].cpu()
bass, sample_rate = torchaudio.load(bass_original)

vocals_spec = audios["vocals"][:, frame_start:frame_end].cpu()
vocals, sample_rate = torchaudio.load(vocals_original)

other_spec = audios["other"][:, frame_start:frame_end].cpu()
other, sample_rate = torchaudio.load(other_original)

mix_spec = mixture[:, frame_start:frame_end].cpu()

4%
9%
3%
7%
2%
6%
0%
4%
9%
3%
7%
2%
6%
0%

4%
9%
3%
7%
2%
6%
0%
4%
9%
3%
7%
2%
6%
0%

4%
9%
3%
7%
2%
6%
0%
4%
9%
3%
7%
2%
6%
0%

4%
9%
3%
7%
2%
6%
0%
4%
9%
3%
7%
2%
6%
0%

5.3 频谱图与音频¶

在接下来的 5 个单元格中，您可以看到相应的音频及其频谱图。使用频谱图可以清楚地可视化音频。

混合片段来自原始音轨，其余音轨是模型的输出。

# Mixture Clip
plot_spectrogram(stft(mix_spec)[0], "Spectrogram - Mixture")
Audio(mix_spec, rate=sample_rate)

鼓的 SDR、频谱图和音频

# Drums Clip
output_results(drums, drums_spec, "drums")

贝斯的 SDR、频谱图和音频

# Bass Clip
output_results(bass, bass_spec, "bass")

人声的 SDR、频谱图和音频

# Vocals Audio
output_results(vocals, vocals_spec, "vocals")

其他的 SDR、频谱图和音频

# Other Clip
output_results(other, other_spec, "other")

# Optionally, the full audios can be heard in from running the next 5
# cells. They will take a bit longer to load, so to run simply uncomment
# out the ``Audio`` cells for the respective track to produce the audio
# for the full song.
#

# Full Audio
# Audio(mixture, rate=sample_rate)

# Drums Audio
# Audio(audios["drums"], rate=sample_rate)

# Bass Audio
# Audio(audios["bass"], rate=sample_rate)

# Vocals Audio
# Audio(audios["vocals"], rate=sample_rate)

# Other Audio
# Audio(audios["other"], rate=sample_rate)

脚本总运行时间: ( 0 分 48.655 秒)

由 Sphinx-Gallery 生成的画廊