System level cache prefetching algorithms for complex GPU workloads

Sammanfattning

Prefetching is a well known concept for CPUs but for GPUs it is fairly unexplored. The memory management of a GPU plays a crucial role in its performance, and cache prefetching has the potential to lower the overall latency. This thesis compares different types of prefetching methods for GPUs and remaking some CPU prefetchers to fit the GPU architecture. All these prefetchers were then put inside the system level cache (SLC), between the GPU and external memory. Five different methods were tested on framework based on ARM’s GPU model. The thesis was mainly based upon prefetching tech- niques discussed in the following papers: Adaptive Stream Detec- tion[11], Best-offset [18], Many-thread aware [17], APOGEE [26], and Last-level collective cache prefetcher [19]. The prefetchers pro- duced in this thesis were either heavily inspired by or implemented as closely as possible to the designs in the papers. The thesis concludes that for graphics workloads the best prefetch- ers implemented can achieve 0.51-1.29% decrease in GPU cycles on average, depending on the chosen GPU configurations, while also lowering the estimated energy usage.