NVMe APST high latency power states being skipped

[Kai-Heng Feng]

On Thu, Jun 1, 2017 at 7:32 PM, Christoph Hellwig <hch at infradead.org> wrote:
> On Thu, Jun 01, 2017 at 04:19:26PM +0800, Kai-Heng Feng wrote:
>> >> I did some benchmark on the high latency SanDisk A400:
>> >> Kernel compilation, no PS3/PS4:
>> >> real    23m36.466s
>> >> user    115m49.944s
>> >> sys     10m58.352s
>> >>
>> >> Kernel compilation, allow PS3/PS4:
>> >> real    24m40.308s
>> >> user    116m12.600s
>> >> sys     11m47.484s
>> >
>> > That's quite a bit of a slow down.  Can we play a bit with the
>> > entry latency (maybe just for PS4) so that we still get into the
>> > modes, but not as quickly?
>>
>> I changed the "Idle Time Prior to Transition" from 55000ms to 110000ms,
>> The time to compile kernel source is roughly the same.
>>
>> My guess is that since the filesystem is constantly doing I/O while
>> compiling kernel, nvme never (or rarely) hits PS4.
>
> That doesn't explain why we need more than a minute more for the
> compile.  How reproducible are these numbers, btw?

After some more test, actually I am getting similar values despite
APST on or off. This is a super hot machine so throttled CPU might be
a more impactful factor here.

Can't directly get power state from NVMe make the latency between
power state transition hard to measure.

At least after a long idle, I typed `ls` and everything showed
instantly, I can't feel the latency at all. Probably the "real"
latency is not as bad as NVMe claimed.

Maybe add an extra knob which can directly control deepest allow power
state? Userspace tools can control deepest power state through this
knob.