Memory-Aware GPU Resource Scheduling Prediction for DLRM Inference Services with BiGRU-Transformer Models

Abstract

Deep learning recommendation models (DLRMs) combine large sparse embeddings with latency-sensitive inference, making host-memory availability as important as accelerator capacity. This study examines per-instance memory-reservation prediction using the Alibaba cluster-trace-gpu-v2025 dataset. The trace contains 23,871 instances from 156 services, including 16,485 CPU-node (CN) instances and 7,386 heterogeneous GPU-node (HN) instances. A BiGRU-Transformer encodes the eight most recent instances of the same service and fuses that history with current CPU, GPU, RDMA, disk, role, and density specifications. It is compared with train-median, role-app median, ridge regression, random forest, gradient boosting, static MLP, GRU, and Transformer baselines under a chronological 70/15/15 split. On the 3,581-instance test period, the BiGRU-Transformer achieves 16.82 GiB MAE and 0.9811 R2, the strongest result among the neural sequence variants. RandomForest achieves the lowest overall MAE at 1.64 GiB and an R2 of 0.9979, indicating that tree partitions capture the trace's discrete reservation templates particularly well. A capacity-normalized scheduling replay converts prediction error into placement risk: RandomForest records 473.86 memory-overflow node-hours, compared with 4,656.53 for the role-app median baseline and 3,332.46 for the BiGRU-Transformer. The results show that same-service histories add predictive value to neural models, while tree ensembles provide more accurate reservations on this trace. A practical scheduler can therefore combine template-sensitive tabular prediction with sequence-based drift signals and role-aware safety margins.