DVFO: Learning-Based DVFS for Energy-Efficient Edge-Cloud Collaborative Inference

Citation Author(s):
Ziyang
Zhang
Harbin Institute of Technology
Submitted by:
Zhang ziyang Zhang
Last updated:
Thu, 01/11/2024 - 06:17
DOI:
10.21227/2gxq-nz15
Data Format:
License:
0
0 ratings - Please login to submit your rating.

Abstract 

Our released weight dataset for fusion results in edge-cloud collaborative inference contains the corresponding weighted summation weights under 50,000 edge-cloud collaborative DNN inference tasks, listing the five heterogeneous NVIDIA edge devices they use (NVIDIA Jetson Nano, TX2, NX, Orin NX, and AGX Orin), computing power (1.9~275TOPS), DNN model type (EfficientNet-B0, ViT-B16), and network bandwidth (0.5~8Mbps). We use a workstation with an NVIDIA GTX 3080 GPU as the cloud server and leverage Linux trickle to adjust the network bandwidth between the edge device and the cloud server. The computing power of edge devices is set by the DVFS governor. Two DNN models are implemented based on PyTorch.

Instructions: 

We collected 50,000 pieces of data and stored them as five csv files for each of the five edge devices, Jetson Nano, Jetson TX2, Xavier NX, Orin NX, and AGX Orin, respectively.

The file structure of each column in each csv is depicted as follows:

  1. TOPS:the computing power of edge devices (in tera operations per second (TOPS)). Specifically,  Jetson Nano's TOPS is between 0~1.9, Jetson TX2's TOPS is between 0~5.34, Xavier NX's TOPS is between 0~21, Orin NX's TOPS is between 0~100, AGX Orin's TOPS is between 0~275.
  2. DNN Type:DNN model type, 1 is EfficientNet-B0, 2 is ViT-B16.
  3. Bandwidth:network bandwidth between the edge device and the cloud server. (in Mbps)
  4. Lambda:weighted summation weights for fusing edge DNN and cloud DNN inference results.