로봇이 실외에서 자율주행을 하려면?

그림 1. 우아한형제들 로보틱스LAB의 실외 자율주행 배달로봇 ‘딜리’

• 충격, 진동, 온도, 습도, 물, 먼지 등을 견디는 내구성
• 긴 배터리 수명과 낮은 발열을 위한 전성비
• 좁은 로봇 내부 공간에 맞는 작은 크기의 부품들
• 다양한 센서를 연결하기 위한 많은 포트 및 높은 호환성

NVIDIA GPU와 Jetson 플랫폼의 특징

양자화

TensorRT를 이용한 최적화 방법

표 1. 예시 코드의 테스트 환경

PyTorch 모델을 TensorRT로 추론하는 방법 비교

1. ONNX 모델 변환 후 TensorRT 엔진 변환

2. Torch-TensorRT를 이용한 추론

ONNX 모델 변환

import torch
import torchvision

# Load ResNet-18 ImageNet-pretrained model using torchvision.
model = torchvision.models.resnet18(pretrained=True)
model.eval()

# Dummy input with the model input size.
dummy_input = torch.randn(1, 3, 224, 224)

# Save ONNX model to ONNX_PATH.
torch.onnx.export(model, 
                  dummy_input, 
                  ONNX_PATH, 
                  opset_version=13,
                  input_names=["input"], 
                  output_names=["output"])

Polygraphy를 이용한 훈련 후 양자화 및 TensorRT 엔진 변환

1. Calibrator 객체 생성

import os

from PIL import Image
from polygraphy.backend import trt as poly_trt
from torchvision import transforms

# Preprocess of ResNet-18 training.
transform = transforms.Compose([transforms.Resize((256, 256)), 
                                transforms.CenterCrop((224, 224)),
                                transforms.ToTensor(), 
                                transforms.Normalize([0.485, 0.456, 0.406], 
                                                     [0.229, 0.224, 0.225])])

# Polygraphy needs a generator-type data loader.
val_list = os.listdir(IMAGE_DIR)
def data_generator():
    for image in val_list:
        # Preprocess.
        image_path = os.path.join(IMAGE_DIR, image)
        image = transform(Image.open(image_path).convert("RGB"))
        # Add batch dimension.
        image = image.unsqueeze(0)
        # Polygraphy uses numpy input.
        image = image.numpy()
        # Dict key must be the same as ONNX input name.
        yield {"input": image}

calibrator = poly_trt.Calibrator(data_loader=data_generator())

2. ONNX 파일 로드 및 IBuilderConfig 객체 생성

builder, network, parser = poly_trt.network_from_onnx_path(path=ONNX_PATH)

# Each type flag must be set to true.
builder_config = poly_trt.create_config(builder=builder,
                                        network=network,
                                        int8=True,
                                        fp16=True,
                                        calibrator=calibrator)

3. TensorRT 엔진 빌드 및 저장

engine = poly_trt.engine_from_network(network=(builder, network, parser),
                                      config=builder_config)

# TensorRT engine will be saved to ENGINE_PATH.
poly_trt.save_engine(engine, ENGINE_PATH)

4. 추론 방법

# Load serialized engine using 'open'.
engine = poly_trt.engine_from_bytes(open(ENGINE_PATH, "rb").read())

with poly_trt.TrtRunner(engine) as runner:
    # Preprocess.
    image = transform(Image.open(IMAGE_PATH).convert("RGB"))
    image = image.unsqueeze(0).numpy()
    # Input dict keys are the same as 'input_names' arg in 'torch.onnx.export'.
    output_dict = runner.infer({"input": image})
    # Output dict keys are the same as 'output_names' arg in 'torch.onnx.export'.
    output = output_dict["output"]

양자화 단계별 성능 비교

비교군 설정

1. FP32 원본 모델

builder_config = poly_trt.create_config(builder=builder,
                                        network=network)

2. 랜덤 데이터를 Calibrator에 통과시킨 INT8 모델

# Generate random data instead of real data.
def data_generator():
    for _ in range(1):
        image = torch.randn(1, 3, 224, 224).numpy()
        yield {"input": image}

# Set calibrator with a new data generator.
calibrator = poly_trt.Calibrator(data_loader=data_generator())

실험 방식

import os
import time

from PIL import Image
from polygraphy.backend import trt as poly_trt
import tqdm

# Load TensorRT engine from ENGINE_PATH.
engine = poly_trt.engine_from_bytes(open(ENGINE_PATH, "rb").read())

# Refer to Hugging Face for IMAGENET2012_CLASSES.
class_list = list(IMAGENET2012_CLASSES.keys())
val_list = os.listdir(IMAGE_DIR)

total_infer_time = 0.0
total_correct = 0
with poly_trt.TrtRunner(engine) as runner:
    for image in tqdm.tqdm(val_list):
        # Get label from file name.
        label = image.split("_")[-1].split(".")[0]
        # Preprocess.
        image_path = os.path.join(IMAGE_DIR, image)
        image = transform(Image.open(image_path).convert("RGB"))
        image = image.unsqueeze(0).numpy()
        # Get inference time using 'time' module.
        before_infer = time.time()
        output_dict = runner.infer({"input": image})
        total_infer_time += time.time() - before_infer
        output = output_dict["output"]
        predicted = output.argmax()
        # Calculate total number of correct outputs.
        total_correct += class_list[predicted.item()] == label

# Calculate and print average values.
accuracy = 100 * total_correct / len(val_list)
avg_infer_time_ms = 1000 * total_infer_time / len(val_list)
print(f"Accuracy: {accuracy:.2f} %")
print(f"Average inference time: {avg_infer_time_ms:.2f} ms")

실험 결과 및 결론

마치며

박준수

우아한형제들 로보틱스LAB에서 컴퓨터비전과 머신러닝 연구 개발을 담당하고 있습니다.