Reinforcement Learning based Dynamic Link Configuration for Energy-Efficient NoC

Multiprocessor system-on-chips (MPSoCs) and chip multiprocessors (CMPs) are moving towards the integration of hundreds of heterogeneous cores on a single chip. Network-on-Chip (NoC) has been the superior interconnect fabric for MPSoCs and CMPs. Due to the rise of dark silicon with the end of Dennard Scaling, it becomes essential to design energy-efficient and high performance reconfigurable NoC between heterogeneous components based on the applications workloads (e.g., task communication demands). Because of the large and complex NoC design space in heterogeneous architectures (link bandwidth, big/little cores and routers, number of components, etc.), it becomes difficult to explore within a reasonable time for optimal trade-offs of energy-performance. Furthermore, reactive resource configuration of NoC is not effective in reducing power consumption. Therefore, we propose reinforcement learning (RL) technique to proactively configure the NoC link-bandwidths in heterogeneous architectures for energy-efficiency. NoC link-bandwidths are configured dynamically based on the learning of the system (e.g., link utilization, buffer utilization). RL agent automatically learns from past and current experiences and take the configuration decision intelligently on heterogeneous architectures to achieve its objective. Simulations under real-world benchmarks show that RL based technique reduces the energy-delay product by 30% and improves the throughput by 50% compared to non-machine learning based solutions.