关键点变换

本示例说明了如何定义和使用关键点。在本教程中，我们使用这张来自前哥伦布时期陶瓷雕像的图片。该图像被指定为“公共领域”(https://www.metmuseum.org/art/collection/search/502727)。

注意

关键点支持已在 TorchVision 0.23 中发布，目前为 BETA 功能。我们不期望 API 会发生更改，但可能存在一些罕见的边缘情况。如果您发现任何问题，请在我们的 bug 跟踪器上报告：https://github.com/pytorch/vision/issues?q=is:open+is:issue

首先，一些设置代码

from PIL import Image
from pathlib import Path
import matplotlib.pyplot as plt


import torch
from torchvision.tv_tensors import KeyPoints
from torchvision.transforms import v2
from helpers import plot

plt.rcParams["figure.figsize"] = [10, 5]
plt.rcParams["savefig.bbox"] = "tight"

# if you change the seed, make sure that the transformed output
# still make sense
torch.manual_seed(0)

# If you're trying to run that on Colab, you can download the assets and the
# helpers from https://github.com/pytorch/vision/tree/main/gallery/
orig_img = Image.open(Path('../assets') / 'pottery.jpg')

创建关键点

关键点通过实例化 KeyPoints 类来创建。

orig_pts = KeyPoints(
    [
        [
            [445, 700],  # nose
            [320, 660],
            [370, 660],
            [420, 660],  # left eye
            [300, 620],
            [420, 620],  # left eyebrow
            [475, 665],
            [515, 665],
            [555, 655],  # right eye
            [460, 625],
            [560, 600],  # right eyebrow
            [370, 780],
            [450, 760],
            [540, 780],
            [450, 820],  # mouth
        ],
    ],
    canvas_size=(orig_img.size[1], orig_img.size[0]),
)

plot([(orig_img, orig_pts)])

变换图示

使用 RandomRotation

rotater = v2.RandomRotation(degrees=(0, 180), expand=True)
rotated_imgs = [rotater((orig_img, orig_pts)) for _ in range(4)]
plot([(orig_img, orig_pts)] + rotated_imgs)

使用 Pad

padded_imgs_and_points = [
    v2.Pad(padding=padding)(orig_img, orig_pts)
    for padding in (30, 50, 100, 200)
]
plot([(orig_img, orig_pts)] + padded_imgs_and_points)

使用 Resize

resized_imgs = [
    v2.Resize(size=size)(orig_img, orig_pts)
    for size in (300, 500, 1000, orig_img.size)
]
plot([(orig_img, orig_pts)] + resized_imgs)

使用 RandomPerspective

perspective_transformer = v2.RandomPerspective(distortion_scale=0.6, p=1.0)
perspective_imgs = [perspective_transformer(orig_img, orig_pts) for _ in range(4)]
plot([(orig_img, orig_pts)] + perspective_imgs)

使用 CenterCrop

center_crops_and_points = [
    v2.CenterCrop(size=size)(orig_img, orig_pts)
    for size in (300, 500, 1000, orig_img.size)
]
plot([(orig_img, orig_pts)] + center_crops_and_points)

使用 RandomRotation

rotater = v2.RandomRotation(degrees=(0, 180))
rotated_imgs = [rotater((orig_img, orig_pts)) for _ in range(4)]
plot([(orig_img, orig_pts)] + rotated_imgs)