TurboSched

Ready to give LWN a try?

With a subscription to LWN, you can stay current with what is happening in the Linux and free-software community and take advantage of subscriber-only site features. We are pleased to offer you a free trial subscription, no credit card required, so that you can see for yourself. Please, join us!

Parth Shah discussed the problem of sustaining "turbo" frequencies on SMP systems. Modern multicore systems have support for turbo frequencies, which are frequencies above the range of the rated frequencies that can be sustained by a small number of CPUs in the chip under certain power and thermal constraints. However, due to these very power and thermal constraints, it is harder to sustain these turbo frequencies for longer durations. Shah said that IBM POWER9 systems have a margin of around 18% for turbo range and sustaining these frequencies can provide better single-threaded performance.

In real-world scenarios such as high-performance computing, tasks are classified into two categories:

1. CPU-intensive: Tasks that benefit from a higher frequency and typically run for longer durations.
2. Jitter: These are short-lived, low-utilization tasks, typically performing some housekeeping operations.

Experimentation showed that when we run a mix of these two types of tasks, the task wakeup logic would wake up an idle core even for a jitter task. This would result in an increase in the power consumption, which would, in most cases, throttle the frequency on the other busy cores that were running in a turbo frequency range.

So Shah suggested that if there were a mechanism to classify the tasks as jitter tasks, then the CFS wakeup logic could be tweaked to pick an already running core with spare capacity. This task-packing policy will ensure that idle cores aren't woken up to run jitter tasks, thereby allowing the busy cores to sustain turbo frequency for a longer duration. He further showed that this approach proved to be better than isolating all the jitter tasks into a small group of cores. Thus, there was scope for doing task packing from the kernel scheduler in a dynamic manner.

Shah also discussed the challenge of determining the spare capacity left in the core before deciding whether the jitter task can be packed into such a core. In general, determining the capacity of the core from the capacities of the constituent threads is not straightforward. On POWER processors, Shah currently uses a formula that would compute the core-capacity in terms of the online threads of the core, such that SMT2 (two threads per core) would have 1.25x the capacity of a single thread, SMT4 would have 1.5x capacity of a single thread and SMT8 would have 2x the capacity of a single thread. However, there is scope for improvement. With this, Shah determines whether a jitter task can be packed into one of the already running cores, as long as it has spare capacity to accommodate the jitter task.

To test his solution, Shah used a synthetic workload generator that can spawn the aforementioned types of tasks. On this workload, he was able to demonstrate that with his approach we could sustain the turbo frequency for 16% longer compared to the existing CFS task-wakeup logic. This translated to 12% benefit in operations per second when compared to the CFS algorithm.

For classifying the tasks as jitter, he is currently using the UCLAMP framework in which tasks that have their max-utilization set to the lowest value are considered to be jitter tasks. This has an added advantage that even when the jitter tasks are running on potentially idle cores, the schedutil governor will be running them at the lowest frequency.

He further clarified that the existing Energy-Aware Scheduling that got merged earlier this year was not applicable in its current form for SMT systems. It wasn't trivial to extend the model to SMT systems.

He concluded his session saying that he is willing to explore the option of extending the EAS model if it can allow for sustaining turbo frequencies for a longer duration. He also solicited feedback on his patch set posted on the kernel mailing list.

[LWN looked at TurboSched in early July.]