1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
|
.. |struct dev_pm_ops| replace:: :c:type:`struct dev_pm_ops <dev_pm_ops>`
.. |struct dev_pm_domain| replace:: :c:type:`struct dev_pm_domain <dev_pm_domain>`
.. |struct bus_type| replace:: :c:type:`struct bus_type <bus_type>`
.. |struct device_type| replace:: :c:type:`struct device_type <device_type>`
.. |struct class| replace:: :c:type:`struct class <class>`
.. |struct wakeup_source| replace:: :c:type:`struct wakeup_source <wakeup_source>`
.. |struct device| replace:: :c:type:`struct device <device>`
==============================
Device Power Management Basics
==============================
::
Copyright (c) 2010-2011 Rafael J. Wysocki <rjw@sisk.pl>, Novell Inc.
Copyright (c) 2010 Alan Stern <stern@rowland.harvard.edu>
Copyright (c) 2016 Intel Corp., Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Most of the code in Linux is device drivers, so most of the Linux power
management (PM) code is also driver-specific. Most drivers will do very
little; others, especially for platforms with small batteries (like cell
phones), will do a lot.
This writeup gives an overview of how drivers interact with system-wide
power management goals, emphasizing the models and interfaces that are
shared by everything that hooks up to the driver model core. Read it as
background for the domain-specific work you'd do with any specific driver.
Two Models for Device Power Management
======================================
Drivers will use one or both of these models to put devices into low-power
states:
System Sleep model:
Drivers can enter low-power states as part of entering system-wide
low-power states like "suspend" (also known as "suspend-to-RAM"), or
(mostly for systems with disks) "hibernation" (also known as
"suspend-to-disk").
This is something that device, bus, and class drivers collaborate on
by implementing various role-specific suspend and resume methods to
cleanly power down hardware and software subsystems, then reactivate
them without loss of data.
Some drivers can manage hardware wakeup events, which make the system
leave the low-power state. This feature may be enabled or disabled
using the relevant :file:`/sys/devices/.../power/wakeup` file (for
Ethernet drivers the ioctl interface used by ethtool may also be used
for this purpose); enabling it may cost some power usage, but let the
whole system enter low-power states more often.
Runtime Power Management model:
Devices may also be put into low-power states while the system is
running, independently of other power management activity in principle.
However, devices are not generally independent of each other (for
example, a parent device cannot be suspended unless all of its child
devices have been suspended). Moreover, depending on the bus type the
device is on, it may be necessary to carry out some bus-specific
operations on the device for this purpose. Devices put into low power
states at run time may require special handling during system-wide power
transitions (suspend or hibernation).
For these reasons not only the device driver itself, but also the
appropriate subsystem (bus type, device type or device class) driver and
the PM core are involved in runtime power management. As in the system
sleep power management case, they need to collaborate by implementing
various role-specific suspend and resume methods, so that the hardware
is cleanly powered down and reactivated without data or service loss.
There's not a lot to be said about those low-power states except that they are
very system-specific, and often device-specific. Also, that if enough devices
have been put into low-power states (at runtime), the effect may be very similar
to entering some system-wide low-power state (system sleep) ... and that
synergies exist, so that several drivers using runtime PM might put the system
into a state where even deeper power saving options are available.
Most suspended devices will have quiesced all I/O: no more DMA or IRQs (except
for wakeup events), no more data read or written, and requests from upstream
drivers are no longer accepted. A given bus or platform may have different
requirements though.
Examples of hardware wakeup events include an alarm from a real time clock,
network wake-on-LAN packets, keyboard or mouse activity, and media insertion
or removal (for PCMCIA, MMC/SD, USB, and so on).
Interfaces for Entering System Sleep States
===========================================
There are programming interfaces provided for subsystems (bus type, device type,
device class) and device drivers to allow them to participate in the power
management of devices they are concerned with. These interfaces cover both
system sleep and runtime power management.
Device Power Management Operations
----------------------------------
Device power management operations, at the subsystem level as well as at the
device driver level, are implemented by defining and populating objects of type
|struct dev_pm_ops| defined in :file:`include/linux/pm.h`. The roles of the
methods included in it will be explained in what follows. For now, it should be
sufficient to remember that the last three methods are specific to runtime power
management while the remaining ones are used during system-wide power
transitions.
There also is a deprecated "old" or "legacy" interface for power management
operations available at least for some subsystems. This approach does not use
|struct dev_pm_ops| objects and it is suitable only for implementing system
sleep power management methods in a limited way. Therefore it is not described
in this document, so please refer directly to the source code for more
information about it.
Subsystem-Level Methods
-----------------------
The core methods to suspend and resume devices reside in
|struct dev_pm_ops| pointed to by the :c:member:`ops` member of
|struct dev_pm_domain|, or by the :c:member:`pm` member of |struct bus_type|,
|struct device_type| and |struct class|. They are mostly of interest to the
people writing infrastructure for platforms and buses, like PCI or USB, or
device type and device class drivers. They also are relevant to the writers of
device drivers whose subsystems (PM domains, device types, device classes and
bus types) don't provide all power management methods.
Bus drivers implement these methods as appropriate for the hardware and the
drivers using it; PCI works differently from USB, and so on. Not many people
write subsystem-level drivers; most driver code is a "device driver" that builds
on top of bus-specific framework code.
For more information on these driver calls, see the description later;
they are called in phases for every device, respecting the parent-child
sequencing in the driver model tree.
:file:`/sys/devices/.../power/wakeup` files
-------------------------------------------
All device objects in the driver model contain fields that control the handling
of system wakeup events (hardware signals that can force the system out of a
sleep state). These fields are initialized by bus or device driver code using
:c:func:`device_set_wakeup_capable()` and :c:func:`device_set_wakeup_enable()`,
defined in :file:`include/linux/pm_wakeup.h`.
The :c:member:`power.can_wakeup` flag just records whether the device (and its
driver) can physically support wakeup events. The
:c:func:`device_set_wakeup_capable()` routine affects this flag. The
:c:member:`power.wakeup` field is a pointer to an object of type
|struct wakeup_source| used for controlling whether or not the device should use
its system wakeup mechanism and for notifying the PM core of system wakeup
events signaled by the device. This object is only present for wakeup-capable
devices (i.e. devices whose :c:member:`can_wakeup` flags are set) and is created
(or removed) by :c:func:`device_set_wakeup_capable()`.
Whether or not a device is capable of issuing wakeup events is a hardware
matter, and the kernel is responsible for keeping track of it. By contrast,
whether or not a wakeup-capable device should issue wakeup events is a policy
decision, and it is managed by user space through a sysfs attribute: the
:file:`power/wakeup` file. User space can write the "enabled" or "disabled"
strings to it to indicate whether or not, respectively, the device is supposed
to signal system wakeup. This file is only present if the
:c:member:`power.wakeup` object exists for the given device and is created (or
removed) along with that object, by :c:func:`device_set_wakeup_capable()`.
Reads from the file will return the corresponding string.
The initial value in the :file:`power/wakeup` file is "disabled" for the
majority of devices; the major exceptions are power buttons, keyboards, and
Ethernet adapters whose WoL (wake-on-LAN) feature has been set up with ethtool.
It should also default to "enabled" for devices that don't generate wakeup
requests on their own but merely forward wakeup requests from one bus to another
(like PCI Express ports).
The :c:func:`device_may_wakeup()` routine returns true only if the
:c:member:`power.wakeup` object exists and the corresponding :file:`power/wakeup`
file contains the "enabled" string. This information is used by subsystems,
like the PCI bus type code, to see whether or not to enable the devices' wakeup
mechanisms. If device wakeup mechanisms are enabled or disabled directly by
drivers, they also should use :c:func:`device_may_wakeup()` to decide what to do
during a system sleep transition. Device drivers, however, are not expected to
call :c:func:`device_set_wakeup_enable()` directly in any case.
It ought to be noted that system wakeup is conceptually different from "remote
wakeup" used by runtime power management, although it may be supported by the
same physical mechanism. Remote wakeup is a feature allowing devices in
low-power states to trigger specific interrupts to signal conditions in which
they should be put into the full-power state. Those interrupts may or may not
be used to signal system wakeup events, depending on the hardware design. On
some systems it is impossible to trigger them from system sleep states. In any
case, remote wakeup should always be enabled for runtime power management for
all devices and drivers that support it.
:file:`/sys/devices/.../power/control` files
--------------------------------------------
Each device in the driver model has a flag to control whether it is subject to
runtime power management. This flag, :c:member:`runtime_auto`, is initialized
by the bus type (or generally subsystem) code using :c:func:`pm_runtime_allow()`
or :c:func:`pm_runtime_forbid()`; the default is to allow runtime power
management.
The setting can be adjusted by user space by writing either "on" or "auto" to
the device's :file:`power/control` sysfs file. Writing "auto" calls
:c:func:`pm_runtime_allow()`, setting the flag and allowing the device to be
runtime power-managed by its driver. Writing "on" calls
:c:func:`pm_runtime_forbid()`, clearing the flag, returning the device to full
power if it was in a low-power state, and preventing the
device from being runtime power-managed. User space can check the current value
of the :c:member:`runtime_auto` flag by reading that file.
The device's :c:member:`runtime_auto` flag has no effect on the handling of
system-wide power transitions. In particular, the device can (and in the
majority of cases should and will) be put into a low-power state during a
system-wide transition to a sleep state even though its :c:member:`runtime_auto`
flag is clear.
For more information about the runtime power management framework, refer to
:file:`Documentation/power/runtime_pm.txt`.
Calling Drivers to Enter and Leave System Sleep States
======================================================
When the system goes into a sleep state, each device's driver is asked to
suspend the device by putting it into a state compatible with the target
system state. That's usually some version of "off", but the details are
system-specific. Also, wakeup-enabled devices will usually stay partly
functional in order to wake the system.
When the system leaves that low-power state, the device's driver is asked to
resume it by returning it to full power. The suspend and resume operations
always go together, and both are multi-phase operations.
For simple drivers, suspend might quiesce the device using class code
and then turn its hardware as "off" as possible during suspend_noirq. The
matching resume calls would then completely reinitialize the hardware
before reactivating its class I/O queues.
More power-aware drivers might prepare the devices for triggering system wakeup
events.
Call Sequence Guarantees
------------------------
To ensure that bridges and similar links needing to talk to a device are
available when the device is suspended or resumed, the device hierarchy is
walked in a bottom-up order to suspend devices. A top-down order is
used to resume those devices.
The ordering of the device hierarchy is defined by the order in which devices
get registered: a child can never be registered, probed or resumed before
its parent; and can't be removed or suspended after that parent.
The policy is that the device hierarchy should match hardware bus topology.
[Or at least the control bus, for devices which use multiple busses.]
In particular, this means that a device registration may fail if the parent of
the device is suspending (i.e. has been chosen by the PM core as the next
device to suspend) or has already suspended, as well as after all of the other
devices have been suspended. Device drivers must be prepared to cope with such
situations.
System Power Management Phases
------------------------------
Suspending or resuming the system is done in several phases. Different phases
are used for suspend-to-idle, shallow (standby), and deep ("suspend-to-RAM")
sleep states and the hibernation state ("suspend-to-disk"). Each phase involves
executing callbacks for every device before the next phase begins. Not all
buses or classes support all these callbacks and not all drivers use all the
callbacks. The various phases always run after tasks have been frozen and
before they are unfrozen. Furthermore, the ``*_noirq`` phases run at a time
when IRQ handlers have been disabled (except for those marked with the
IRQF_NO_SUSPEND flag).
All phases use PM domain, bus, type, class or driver callbacks (that is, methods
defined in ``dev->pm_domain->ops``, ``dev->bus->pm``, ``dev->type->pm``,
``dev->class->pm`` or ``dev->driver->pm``). These callbacks are regarded by the
PM core as mutually exclusive. Moreover, PM domain callbacks always take
precedence over all of the other callbacks and, for example, type callbacks take
precedence over bus, class and driver callbacks. To be precise, the following
rules are used to determine which callback to execute in the given phase:
1. If ``dev->pm_domain`` is present, the PM core will choose the callback
provided by ``dev->pm_domain->ops`` for execution.
2. Otherwise, if both ``dev->type`` and ``dev->type->pm`` are present, the
callback provided by ``dev->type->pm`` will be chosen for execution.
3. Otherwise, if both ``dev->class`` and ``dev->class->pm`` are present,
the callback provided by ``dev->class->pm`` will be chosen for
execution.
4. Otherwise, if both ``dev->bus`` and ``dev->bus->pm`` are present, the
callback provided by ``dev->bus->pm`` will be chosen for execution.
This allows PM domains and device types to override callbacks provided by bus
types or device classes if necessary.
The PM domain, type, class and bus callbacks may in turn invoke device- or
driver-specific methods stored in ``dev->driver->pm``, but they don't have to do
that.
If the subsystem callback chosen for execution is not present, the PM core will
execute the corresponding method from the ``dev->driver->pm`` set instead if
there is one.
Entering System Suspend
-----------------------
When the system goes into the freeze, standby or memory sleep state,
the phases are: ``prepare``, ``suspend``, ``suspend_late``, ``suspend_noirq``.
1. The ``prepare`` phase is meant to prevent races by preventing new
devices from being registered; the PM core would never know that all the
children of a device had been suspended if new children could be
registered at will. [By contrast, from the PM core's perspective,
devices may be unregistered at any time.] Unlike the other
suspend-related phases, during the ``prepare`` phase the device
hierarchy is traversed top-down.
After the ``->prepare`` callback method returns, no new children may be
registered below the device. The method may also prepare the device or
driver in some way for the upcoming system power transition, but it
should not put the device into a low-power state. Moreover, if the
device supports runtime power management, the ``->prepare`` callback
method must not update its state in case it is necessary to resume it
from runtime suspend later on.
For devices supporting runtime power management, the return value of the
prepare callback can be used to indicate to the PM core that it may
safely leave the device in runtime suspend (if runtime-suspended
already), provided that all of the device's descendants are also left in
runtime suspend. Namely, if the prepare callback returns a positive
number and that happens for all of the descendants of the device too,
and all of them (including the device itself) are runtime-suspended, the
PM core will skip the ``suspend``, ``suspend_late`` and
``suspend_noirq`` phases as well as all of the corresponding phases of
the subsequent device resume for all of these devices. In that case,
the ``->complete`` callback will be invoked directly after the
``->prepare`` callback and is entirely responsible for putting the
device into a consistent state as appropriate.
Note that this direct-complete procedure applies even if the device is
disabled for runtime PM; only the runtime-PM status matters. It follows
that if a device has system-sleep callbacks but does not support runtime
PM, then its prepare callback must never return a positive value. This
is because all such devices are initially set to runtime-suspended with
runtime PM disabled.
This feature also can be controlled by device drivers by using the
``DPM_FLAG_NEVER_SKIP`` and ``DPM_FLAG_SMART_PREPARE`` driver power
management flags. [Typically, they are set at the time the driver is
probed against the device in question by passing them to the
:c:func:`dev_pm_set_driver_flags` helper function.] If the first of
these flags is set, the PM core will not apply the direct-complete
procedure described above to the given device and, consequenty, to any
of its ancestors. The second flag, when set, informs the middle layer
code (bus types, device types, PM domains, classes) that it should take
the return value of the ``->prepare`` callback provided by the driver
into account and it may only return a positive value from its own
``->prepare`` callback if the driver's one also has returned a positive
value.
2. The ``->suspend`` methods should quiesce the device to stop it from
performing I/O. They also may save the device registers and put it into
the appropriate low-power state, depending on the bus type the device is
on, and they may enable wakeup events.
However, for devices supporting runtime power management, the
``->suspend`` methods provided by subsystems (bus types and PM domains
in particular) must follow an additional rule regarding what can be done
to the devices before their drivers' ``->suspend`` methods are called.
Namely, they can only resume the devices from runtime suspend by
calling :c:func:`pm_runtime_resume` for them, if that is necessary, and
they must not update the state of the devices in any other way at that
time (in case the drivers need to resume the devices from runtime
suspend in their ``->suspend`` methods).
3. For a number of devices it is convenient to split suspend into the
"quiesce device" and "save device state" phases, in which cases
``suspend_late`` is meant to do the latter. It is always executed after
runtime power management has been disabled for the device in question.
4. The ``suspend_noirq`` phase occurs after IRQ handlers have been disabled,
which means that the driver's interrupt handler will not be called while
the callback method is running. The ``->suspend_noirq`` methods should
save the values of the device's registers that weren't saved previously
and finally put the device into the appropriate low-power state.
The majority of subsystems and device drivers need not implement this
callback. However, bus types allowing devices to share interrupt
vectors, like PCI, generally need it; otherwise a driver might encounter
an error during the suspend phase by fielding a shared interrupt
generated by some other device after its own device had been set to low
power.
At the end of these phases, drivers should have stopped all I/O transactions
(DMA, IRQs), saved enough state that they can re-initialize or restore previous
state (as needed by the hardware), and placed the device into a low-power state.
On many platforms they will gate off one or more clock sources; sometimes they
will also switch off power supplies or reduce voltages. [Drivers supporting
runtime PM may already have performed some or all of these steps.]
If :c:func:`device_may_wakeup(dev)` returns ``true``, the device should be
prepared for generating hardware wakeup signals to trigger a system wakeup event
when the system is in the sleep state. For example, :c:func:`enable_irq_wake()`
might identify GPIO signals hooked up to a switch or other external hardware,
and :c:func:`pci_enable_wake()` does something similar for the PCI PME signal.
If any of these callbacks returns an error, the system won't enter the desired
low-power state. Instead, the PM core will unwind its actions by resuming all
the devices that were suspended.
Leaving System Suspend
----------------------
When resuming from freeze, standby or memory sleep, the phases are:
``resume_noirq``, ``resume_early``, ``resume``, ``complete``.
1. The ``->resume_noirq`` callback methods should perform any actions
needed before the driver's interrupt handlers are invoked. This
generally means undoing the actions of the ``suspend_noirq`` phase. If
the bus type permits devices to share interrupt vectors, like PCI, the
method should bring the device and its driver into a state in which the
driver can recognize if the device is the source of incoming interrupts,
if any, and handle them correctly.
For example, the PCI bus type's ``->pm.resume_noirq()`` puts the device
into the full-power state (D0 in the PCI terminology) and restores the
standard configuration registers of the device. Then it calls the
device driver's ``->pm.resume_noirq()`` method to perform device-specific
actions.
2. The ``->resume_early`` methods should prepare devices for the execution
of the resume methods. This generally involves undoing the actions of
the preceding ``suspend_late`` phase.
3. The ``->resume`` methods should bring the device back to its operating
state, so that it can perform normal I/O. This generally involves
undoing the actions of the ``suspend`` phase.
4. The ``complete`` phase should undo the actions of the ``prepare`` phase.
For this reason, unlike the other resume-related phases, during the
``complete`` phase the device hierarchy is traversed bottom-up.
Note, however, that new children may be registered below the device as
soon as the ``->resume`` callbacks occur; it's not necessary to wait
until the ``complete`` phase with that.
Moreover, if the preceding ``->prepare`` callback returned a positive
number, the device may have been left in runtime suspend throughout the
whole system suspend and resume (the ``suspend``, ``suspend_late``,
``suspend_noirq`` phases of system suspend and the ``resume_noirq``,
``resume_early``, ``resume`` phases of system resume may have been
skipped for it). In that case, the ``->complete`` ca
|