summaryrefslogtreecommitdiffstats
path: root/tools/power/pm-graph/README
diff options
context:
space:
mode:
authorDaniel Baumann <daniel.baumann@progress-linux.org>2024-04-11 08:27:49 +0000
committerDaniel Baumann <daniel.baumann@progress-linux.org>2024-04-11 08:27:49 +0000
commitace9429bb58fd418f0c81d4c2835699bddf6bde6 (patch)
treeb2d64bc10158fdd5497876388cd68142ca374ed3 /tools/power/pm-graph/README
parentInitial commit. (diff)
downloadlinux-ace9429bb58fd418f0c81d4c2835699bddf6bde6.tar.xz
linux-ace9429bb58fd418f0c81d4c2835699bddf6bde6.zip
Adding upstream version 6.6.15.upstream/6.6.15
Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>
Diffstat (limited to 'tools/power/pm-graph/README')
-rw-r--r--tools/power/pm-graph/README660
1 files changed, 660 insertions, 0 deletions
diff --git a/tools/power/pm-graph/README b/tools/power/pm-graph/README
new file mode 100644
index 0000000000..047ce1d764
--- /dev/null
+++ b/tools/power/pm-graph/README
@@ -0,0 +1,660 @@
+ _
+ _ __ _ __ ___ __ _ _ __ __ _ _ __ | |__
+ | '_ \| '_ ` _ \ _____ / _` | '__/ _` | '_ \| '_ \
+ | |_) | | | | | |_____| (_| | | | (_| | |_) | | | |
+ | .__/|_| |_| |_| \__, |_| \__,_| .__/|_| |_|
+ |_| |___/ |_|
+
+ pm-graph: suspend/resume/boot timing analysis tools
+ Version: 5.11
+ Author: Todd Brandt <todd.e.brandt@intel.com>
+ Home Page: https://www.intel.com/content/www/us/en/developer/topic-technology/open/pm-graph/overview.html
+
+ Report bugs/issues at bugzilla.kernel.org Tools/pm-graph
+ - https://bugzilla.kernel.org/buglist.cgi?component=pm-graph&product=Tools
+
+ Full documentation available online & in man pages
+ - Getting Started:
+ https://www.intel.com/content/www/us/en/developer/articles/technical/usage.html
+
+ - Feature Summary:
+ https://www.intel.com/content/www/us/en/developer/topic-technology/open/pm-graph/features.html
+
+ - upstream version in git:
+ git clone https://github.com/intel/pm-graph/
+
+ Table of Contents
+ - Overview
+ - Setup
+ - Usage
+ - Basic Usage
+ - Dev Mode Usage
+ - Proc Mode Usage
+ - Endurance Testing
+ - Usage Examples
+ - Configuration Files
+ - Usage Examples
+ - Config File Options
+ - Custom Timeline Entries
+ - Adding/Editing Timeline Functions
+ - Adding/Editing Dev Timeline Source Functions
+ - Verifying your Custom Functions
+ - Testing on consumer linux Operating Systems
+ - Android
+
+------------------------------------------------------------------
+| OVERVIEW |
+------------------------------------------------------------------
+
+ This tool suite is designed to assist kernel and OS developers in optimizing
+ their linux stack's suspend/resume & boot time. Using a kernel image built
+ with a few extra options enabled, the tools will execute a suspend or boot,
+ and will capture dmesg and ftrace data. This data is transformed into a set of
+ timelines and a callgraph to give a quick and detailed view of which devices
+ and kernel processes are taking the most time in suspend/resume & boot.
+
+------------------------------------------------------------------
+| SETUP |
+------------------------------------------------------------------
+
+ Package Requirements
+ - runs with python2 or python3, choice is made by /usr/bin/python link
+ - python
+ - python-configparser (for python2 sleepgraph)
+ - python-requests (for stresstester.py)
+ - linux-tools-common (for turbostat usage in sleepgraph)
+
+ Ubuntu:
+ sudo apt-get install python python-configparser python-requests linux-tools-common
+
+ Fedora:
+ sudo dnf install python python-configparser python-requests linux-tools-common
+
+ The tools can most easily be installed via git clone and make install
+
+ $> git clone http://github.com/intel/pm-graph.git
+ $> cd pm-graph
+ $> sudo make install
+ $> man sleepgraph ; man bootgraph
+
+ Setup involves some minor kernel configuration
+
+ The following kernel build options are required for all kernels:
+ CONFIG_DEVMEM=y
+ CONFIG_PM_DEBUG=y
+ CONFIG_PM_SLEEP_DEBUG=y
+ CONFIG_FTRACE=y
+ CONFIG_FUNCTION_TRACER=y
+ CONFIG_FUNCTION_GRAPH_TRACER=y
+ CONFIG_KPROBES=y
+ CONFIG_KPROBES_ON_FTRACE=y
+
+ In kernel 3.15.0, two patches were upstreamed which enable the
+ v3.0 behavior. These patches allow the tool to read all the
+ data from trace events instead of from dmesg. You can enable
+ this behavior on earlier kernels with these patches:
+
+ (kernel/pre-3.15/enable_trace_events_suspend_resume.patch)
+ (kernel/pre-3.15/enable_trace_events_device_pm_callback.patch)
+
+ If you're using bootgraph, or sleepgraph with a kernel older than 3.15.0,
+ the following additional kernel parameters are required:
+ (e.g. in file /etc/default/grub)
+ GRUB_CMDLINE_LINUX_DEFAULT="... initcall_debug log_buf_len=32M ..."
+
+ If you're using a kernel older than 3.11-rc2, the following simple
+ patch must be applied to enable ftrace data:
+ in file: kernel/power/suspend.c
+ in function: int suspend_devices_and_enter(suspend_state_t state)
+ remove call to "ftrace_stop();"
+ remove call to "ftrace_start();"
+
+ There is a patch which does this for kernel v3.8.0:
+ (kernel/pre-3.11-rc2/enable_ftrace_in_suspendresume.patch)
+
+
+
+------------------------------------------------------------------
+| USAGE |
+------------------------------------------------------------------
+
+Basic Usage
+___________
+
+ 1) First configure a kernel using the instructions from the previous sections.
+ Then build, install, and boot with it.
+ 2) Open up a terminal window and execute the mode list command:
+
+ %> sudo ./sleepgraph.py -modes
+ ['freeze', 'mem', 'disk']
+
+ Execute a test using one of the available power modes, e.g. mem (S3):
+
+ %> sudo ./sleepgraph.py -m mem -rtcwake 15
+
+ or with a config file
+
+ %> sudo ./sleepgraph.py -config config/suspend.cfg
+
+ When the system comes back you'll see the script finishing up and
+ creating the output files in the test subdir. It generates output
+ files in subdirectory: suspend-mmddyy-HHMMSS. The ftrace file can
+ be used to regenerate the html timeline with different options
+
+ HTML output: <hostname>_<mode>.html
+ raw dmesg output: <hostname>_<mode>_dmesg.txt
+ raw ftrace output: <hostname>_<mode>_ftrace.txt
+
+ View the html in firefox or chrome.
+
+
+Dev Mode Usage
+______________
+
+ Developer mode adds information on low level source calls to the timeline.
+ The tool sets kprobes on all delay and mutex calls to see which devices
+ are waiting for something and when. It also sets a suite of kprobes on
+ subsystem dependent calls to better fill out the timeline.
+
+ The tool will also expose kernel threads that don't normally show up in the
+ timeline. This is useful in discovering dependent threads to get a better
+ idea of what each device is waiting for. For instance, the scsi_eh thread,
+ a.k.a. scsi resume error handler, is what each SATA disk device waits for
+ before it can continue resume.
+
+ The timeline will be much larger if run with dev mode, so it can be useful
+ to set the -mindev option to clip out any device blocks that are too small
+ to see easily. The following command will give a nice dev mode run:
+
+ %> sudo ./sleepgraph.py -m mem -rtcwake 15 -mindev 1 -dev
+
+ or with a config file
+
+ %> sudo ./sleepgraph.py -config config/suspend-dev.cfg
+
+
+Proc Mode Usage
+_______________
+
+ Proc mode adds user process info to the timeline. This is done in a manner
+ similar to the bootchart utility, which graphs init processes and their
+ execution as the system boots. This tool option does the same thing but for
+ the period before and after suspend/resume.
+
+ In order to see any process info, there needs to be some delay before or
+ after resume since processes are frozen in suspend_prepare and thawed in
+ resume_complete. The predelay and postdelay args allow you to do this. It
+ can also be useful to run in x2 mode with an x2 delay, this way you can
+ see process activity before and after resume, and in between two
+ successive suspend/resumes.
+
+ The command can be run like this:
+
+ %> sudo ./sleepgraph.py -m mem -rtcwake 15 -x2 -x2delay 1000 -predelay 1000 -postdelay 1000 -proc
+
+ or with a config file
+
+ %> sudo ./sleepgraph.py -config config/suspend-proc.cfg
+
+------------------------------------------------------------------
+| ENDURANCE TESTING |
+------------------------------------------------------------------
+
+ The best way to gauge the health of a system is to run a series of
+ suspend/resumes over an extended period and analyze the behavior. This can be
+ accomplished with sleepgraph's -multi argument. You specify two numbers: the
+ number of tests to run OR the duration in days, hours, or minutes, and the
+ delay in seconds between them. For instance, -multi 20 5: execute 20 tests with
+ a 5 second delay between each, or -multi 24h 0: execute tests over a 24 hour
+ period with no delay between tests. You can include any other options you like
+ to generate the data you want. It's most useful to collect dev mode timelines
+ as the kprobes don't alter the performance much and you get more insight.
+
+ On completion, the output folder contains a series of folders for the
+ individual test data and a set of summary pages in the root. The summary.html
+ file is a tabular list of the tests with relevant info and links. The
+ summary-issue.html and summary-devices.html files include data taken from
+ all tests on kernel issues and device performance. The folder looks like this:
+
+ suspend-xN-{date}-{time}:
+ summary.html
+ summary-issues.html
+ summary-devices.html
+ suspend-{date}-{time} (1)
+ suspend-{date}-{time} (2)
+ ...
+
+ These are the relevant arguments to use for testing:
+
+ -m mode
+ Mode to initiate for suspend e.g. mem, freeze, standby (default: mem).
+
+ -rtcwake t
+ Use rtcwake to autoresume after t seconds (default: 15).
+
+ -gzip (optional)
+ Gzip the trace and dmesg logs to save space. The tool can also read in
+ gzipped logs for processing. This reduces the multitest folder size.
+
+ -dev (optional)
+ Add kernel source calls and threads to the timeline (default: disabled).
+
+ -multi n d
+ Execute n consecutive tests at d seconds intervals. The outputs will be
+ created in a new subdirectory: suspend-xN-{date}-{time}. When the multitest
+ run is done, the -summary command is called automatically to create summary
+ html files for all the data (unless you use -skiphtml). -skiphtml will
+ speed up the testing by not creating timelines or summary html files. You
+ can then run the tool again at a later time with -summary and -genhtml to
+ create the timelines.
+
+ -skiphtml (optional)
+ Run the test and capture the trace logs, but skip the timeline and summary
+ html generation. This can greatly speed up overall testing. You can then
+ copy the data to a faster host machine and run -summary -genhtml to
+ generate the timelines and summary.
+
+ These are the relevant commands to use after testing is complete:
+
+ -summary indir
+ Generate or regenerate the summary for a -multi test run. Creates three
+ files: summary.html, summary-issues.html, and summary-devices.html in the
+ current folder. summary.html is a table of tests with relevant info sorted
+ by kernel/host/mode, and links to the test html files. summary-issues.html
+ is a list of kernel issues found in dmesg from all the tests.
+ summary-devices.html is a list of devices and times from all the tests.
+
+ -genhtml
+ Used with -summary to regenerate any missing html timelines from their
+ dmesg and ftrace logs. This will require a significant amount of time if
+ there are thousands of tests.
+
+Usage Examples
+_______________
+
+ A multitest is initiated like this:
+
+ %> sudo ./sleepgraph.py -m mem -rtcwake 10 -dev -gzip -multi 2000 0
+
+ or you can skip timeline generation in order to speed things up
+
+ %> sudo ./sleepgraph.py -m mem -rtcwake 10 -dev -gzip -multi 2000 0 -skiphtml
+
+ The tool will produce an output folder with all the test subfolders inside.
+ Each test subfolder contains the dmesg/ftrace logs and/or the html timeline
+ depending on whether you used the -skiphtml option. The root folder contains
+ the summary.html files.
+
+ The summary for an existing multitest is generated like this:
+
+ %> cd suspend-x2000-{date}-{time}
+ %> sleepgraph.py -summary .
+
+ or if you need to generate the html timelines you can use -genhtml
+
+ %> cd suspend-xN-{date}-{time}
+ %> sleepgraph.py -summary . -genhtml
+
+------------------------------------------------------------------
+| CONFIGURATION FILES |
+------------------------------------------------------------------
+
+ Since 4.0 we've moved to using config files in lieu of command line options.
+ The config folder contains a collection of typical use cases.
+ There are corresponding configs for other power modes:
+
+ Simple suspend/resume with basic timeline (mem/freeze/standby)
+ config/suspend.cfg
+ config/freeze.cfg
+ config/standby.cfg
+
+ Dev mode suspend/resume with dev timeline (mem/freeze/standby)
+ config/suspend-dev.cfg
+ config/freeze-dev.cfg
+ config/standby-dev.cfg
+
+ Simple suspend/resume with timeline and callgraph (mem/freeze/standby)
+ config/suspend-callgraph.cfg
+ config/freeze-callgraph.cfg
+ config/standby-callgraph.cfg
+
+ Sample proc mode x2 run using mem suspend
+ config/suspend-x2-proc.cfg
+
+ Sample for editing timeline funcs (moves internal functions into config)
+ config/custom-timeline-functions.cfg
+
+ Sample debug config for serio subsystem
+ config/debug-serio-suspend.cfg
+
+
+Usage Examples
+______________
+
+ Run a simple mem suspend:
+ %> sudo ./sleepgraph.py -config config/suspend.cfg
+
+ Run a mem suspend with callgraph data:
+ %> sudo ./sleepgraph.py -config config/suspend-callgraph.cfg
+
+ Run a mem suspend with dev mode detail:
+ %> sudo ./sleepgraph.py -config config/suspend-dev.cfg
+
+
+Config File Options
+___________________
+
+ [Settings]
+
+ # Verbosity: print verbose messages (def: false)
+ verbose: false
+
+ # Suspend Mode: e.g. standby, mem, freeze, disk (def: mem)
+ mode: mem
+
+ # Output Directory Format: {hostname}, {date}, {time} give current values
+ output-dir: suspend-{hostname}-{date}-{time}
+
+ # Automatic Wakeup: use rtcwake to wakeup after X seconds (def: infinity)
+ rtcwake: 15
+
+ # Add Logs: add the dmesg and ftrace log to the html output (def: false)
+ addlogs: false
+
+ # Sus/Res Gap: insert a gap between sus & res in the timeline (def: false)
+ srgap: false
+
+ # Custom Command: Command to execute in lieu of suspend (def: "")
+ command: echo mem > /sys/power/state
+
+ # Proc mode: graph user processes and cpu usage in the timeline (def: false)
+ proc: false
+
+ # Dev mode: graph source functions in the timeline (def: false)
+ dev: false
+
+ # Suspend/Resume x2: run 2 suspend/resumes back to back (def: false)
+ x2: false
+
+ # x2 Suspend Delay: time delay between the two test runs in ms (def: 0 ms)
+ x2delay: 0
+
+ # Pre Suspend Delay: nclude an N ms delay before (1st) suspend (def: 0 ms)
+ predelay: 0
+
+ # Post Resume Delay: include an N ms delay after (last) resume (def: 0 ms)
+ postdelay: 0
+
+ # Min Device Length: graph only dev callbacks longer than min (def: 0.001 ms)
+ mindev: 0.001
+
+ # Callgraph: gather ftrace callgraph data on all timeline events (def: false)
+ callgraph: false
+
+ # Expand Callgraph: pre-expand the callgraph treeviews in html (def: false)
+ expandcg: false
+
+ # Min Callgraph Length: show callgraphs only if longer than min (def: 1 ms)
+ mincg: 1
+
+ # Timestamp Precision: number of sig digits in timestamps (0:S, [3:ms], 6:us)
+ timeprec: 3
+
+ # Device Filter: show only devs whose name/driver includes one of these strings
+ devicefilter: _cpu_up,_cpu_down,i915,usb
+
+ # Override default timeline entries:
+ # Do not use the internal default functions for timeline entries (def: false)
+ # Set this to true if you intend to only use the ones defined in the config
+ override-timeline-functions: true
+
+ # Override default dev timeline entries:
+ # Do not use the internal default functions for dev timeline entries (def: false)
+ # Set this to true if you intend to only use the ones defined in the config
+ override-dev-timeline-functions: true
+
+ # Call Loop Max Gap (dev mode only)
+ # merge loops of the same call if each is less than maxgap apart (def: 100us)
+ callloop-maxgap: 0.0001
+
+ # Call Loop Max Length (dev mode only)
+ # merge loops of the same call if each is less than maxlen in length (def: 5ms)
+ callloop-maxlen: 0.005
+
+------------------------------------------------------------------
+| CUSTOM TIMELINE ENTRIES |
+------------------------------------------------------------------
+
+Adding or Editing Timeline Functions
+____________________________________
+
+ The tool uses an array of function names to fill out empty spaces in the
+ timeline where device callbacks don't appear. For instance, in suspend_prepare
+ the tool adds the sys_sync and freeze_processes calls as virtual device blocks
+ in the timeline to show you where the time is going. These calls should fill
+ the timeline with contiguous data so that most kernel execution is covered.
+
+ It is possible to add new function calls to the timeline by adding them to
+ the config. It's also possible to copy the internal timeline functions into
+ the config so that you can override and edit them. Place them in the
+ timeline_functions_ARCH section with the name of your architecture appended.
+ i.e. for x86_64: [timeline_functions_x86_64]
+
+ Use the override-timeline-functions option if you only want to use your
+ custom calls, or leave it false to append them to the internal ones.
+
+ This section includes a list of functions (set using kprobes) which use both
+ symbol data and function arg data. The args are pulled directly from the
+ stack using this architecture's registers and stack formatting. Each entry
+ can include up to four pieces of info: The function name, a format string,
+ an argument list, and a color. But only a function name is required.
+
+ For a full example config, see config/custom-timeline-functions.cfg. It pulls
+ all the internal timeline functions into the config and allows you to edit
+ them.
+
+ Entry format:
+
+ function: format{fn_arg1}_{fn_arg2} fn_arg1 fn_arg2 ... [color=purple]
+
+ Required Arguments:
+
+ function: The symbol name for the function you want probed, this is the
+ minimum required for an entry, it will show up as the function
+ name with no arguments.
+
+ example: _cpu_up:
+
+ Optional Arguments:
+
+ format: The format to display the data on the timeline in. Use braces to
+ enclose the arg names.
+
+ example: CPU_ON[{cpu}]
+
+ color: The color of the entry block in the timeline. The default color is
+ transparent, so the entry shares the phase color. The color is an
+ html color string, either a word, or an RGB.
+
+ example: [color=#CC00CC]
+
+ arglist: A list of arguments from registers/stack addresses. See URL:
+ https://www.kernel.org/doc/Documentation/trace/kprobetrace.txt
+
+ example: cpu=%di:s32
+
+ Here is a full example entry. It displays cpu resume calls in the timeline
+ in orange. They will appear as CPU_ON[0], CPU_ON[1], etc.
+
+ [timeline_functions_x86_64]
+ _cpu_up: CPU_ON[{cpu}] cpu=%di:s32 [color=orange]
+
+
+Adding or Editing Dev Mode Timeline Source Functions
+____________________________________________________
+
+ In dev mode, the tool uses an array of function names to monitor source
+ execution within the timeline entries.
+
+ The function calls are displayed inside the main device/call blocks in the
+ timeline. However, if a function call is not within a main timeline event,
+ it will spawn an entirely new event named after the caller's kernel thread.
+ These asynchronous kernel threads will populate in a separate section
+ beneath the main device/call section.
+
+ The tool has a set of hard coded calls which focus on the most common use
+ cases: msleep, udelay, schedule_timeout, mutex_lock_slowpath, etc. These are
+ the functions that add a hardcoded time delay to the suspend/resume path.
+ The tool also includes some common functions native to important
+ subsystems: ata, i915, and ACPI, etc.
+
+ It is possible to add new function calls to the dev timeline by adding them
+ to the config. It's also possible to copy the internal dev timeline
+ functions into the config so that you can override and edit them. Place them
+ in the dev_timeline_functions_ARCH section with the name of your architecture
+ appended. i.e. for x86_64: [dev_timeline_functions_x86_64]
+
+ Use the override-dev-timeline-functions option if you only want to use your
+ custom calls, or leave it false to append them to the internal ones.
+
+ The format is the same as the timeline_functions_x86_64 section. It's a
+ list of functions (set using kprobes) which use both symbol data and function
+ arg data. The args are pulled directly from the stack using this
+ architecture's registers and stack formatting. Each entry can include up
+ to four pieces of info: The function name, a format string, an argument list,
+ and a color. But only the function name is required.
+
+ For a full example config, see config/custom-timeline-functions.cfg. It pulls
+ all the internal dev timeline functions into the config and allows you to edit
+ them.
+
+ Here is a full example entry. It displays the ATA port reset calls as
+ ataN_port_reset in the timeline. This is where most of the SATA disk resume
+ time goes, so it can be helpful to see the low level call.
+
+ [dev_timeline_functions_x86_64]
+ ata_eh_recover: ata{port}_port_reset port=+36(%di):s32 [color=#CC00CC]
+
+
+Verifying your custom functions
+_______________________________
+
+ Once you have a set of functions (kprobes) defined, it can be useful to
+ perform a quick check to see if you formatted them correctly and if the system
+ actually supports them. To do this, run the tool with your config file
+ and the -status option. The tool will go through all the kprobes (both
+ custom and internal if you haven't overridden them) and actually attempts
+ to set them in ftrace. It will then print out success or fail for you.
+
+ Note that kprobes which don't actually exist in the kernel won't stop the
+ tool, they just wont show up.
+
+ For example:
+
+ sudo ./sleepgraph.py -config config/custom-timeline-functions.cfg -status
+ Checking this system (myhostname)...
+ have root access: YES
+ is sysfs mounted: YES
+ is "mem" a valid power mode: YES
+ is ftrace supported: YES
+ are kprobes supported: YES
+ timeline data source: FTRACE (all trace events found)
+ is rtcwake supported: YES
+ verifying timeline kprobes work:
+ _cpu_down: YES
+ _cpu_up: YES
+ acpi_pm_finish: YES
+ acpi_pm_prepare: YES
+ freeze_kernel_threads: YES
+ freeze_processes: YES
+ sys_sync: YES
+ thaw_processes: YES
+ verifying dev kprobes work:
+ __const_udelay: YES
+ __mutex_lock_slowpath: YES
+ acpi_os_stall: YES
+ acpi_ps_parse_aml: YES
+ intel_opregion_init: NO
+ intel_opregion_register: NO
+ intel_opregion_setup: NO
+ msleep: YES
+ schedule_timeout: YES
+ schedule_timeout_uninterruptible: YES
+ usleep_range: YES
+
+
+------------------------------------------------------------------
+| TESTING ON CONSUMER LINUX OPERATING SYSTEMS |
+------------------------------------------------------------------
+
+Android
+_______
+
+ The easiest way to execute on an android device is to run the android.sh
+ script on the device, then pull the ftrace log back to the host and run
+ sleepgraph.py on it.
+
+ Here are the steps:
+
+ [download and install the tool on the device]
+
+ host%> wget https://raw.githubusercontent.com/intel/pm-graph/master/tools/android.sh
+ host%> adb connect 192.168.1.6
+ host%> adb root
+ # push the script to a writeable location
+ host%> adb push android.sh /sdcard/
+
+ [check whether the tool will run on your device]
+
+ host%> adb shell
+ dev%> cd /sdcard
+ dev%> sh android.sh status
+ host : asus_t100
+ kernel : 3.14.0-i386-dirty
+ modes : freeze mem
+ rtcwake : supported
+ ftrace : supported
+ trace events {
+ suspend_resume: found
+ device_pm_callback_end: found
+ device_pm_callback_start: found
+ }
+ # the above is what you see on a system that's properly patched
+
+ [execute the suspend]
+
+ # NOTE: The suspend will only work if the screen isn't timed out,
+ # so you have to press some keys first to wake it up b4 suspend)
+ dev%> sh android.sh suspend mem
+ ------------------------------------
+ Suspend/Resume timing test initiated
+ ------------------------------------
+ hostname : asus_t100
+ kernel : 3.14.0-i386-dirty
+ mode : mem
+ ftrace out : /mnt/shell/emulated/0/ftrace.txt
+ dmesg out : /mnt/shell/emulated/0/dmesg.txt
+ log file : /mnt/shell/emulated/0/log.txt
+ ------------------------------------
+ INITIALIZING FTRACE........DONE
+ STARTING FTRACE
+ SUSPEND START @ 21:24:02 (rtcwake in 10 seconds)
+ <adb connection will now terminate>
+
+ [retrieve the data from the device]
+
+ # I find that you have to actually kill the adb process and
+ # reconnect sometimes in order for the connection to work post-suspend
+ host%> adb connect 192.168.1.6
+ # (required) get the ftrace data, this is the most important piece
+ host%> adb pull /sdcard/ftrace.txt
+ # (optional) get the dmesg data, this is for debugging
+ host%> adb pull /sdcard/dmesg.txt
+ # (optional) get the log, which just lists some test times for comparison
+ host%> adb pull /sdcard/log.txt
+
+ [create an output html file using sleepgraph.py]
+
+ host%> sleepgraph.py -ftrace ftrace.txt
+
+ You should now have an output.html with the android data, enjoy!