Warning, /sdk/heaptrack/README.md is written in an unsupported language. File is not indexed.

0001 # heaptrack - a heap memory profiler for Linux
0002 
0003 ![heaptrack_gui summary page](screenshots/gui_summary.png?raw=true "heaptrack_gui summary page")
0004 
0005 Heaptrack traces all memory allocations and annotates these events with stack traces.
0006 Dedicated analysis tools then allow you to interpret the heap memory profile to:
0007 
0008 - find hotspots that need to be optimized to reduce the **memory footprint** of your application
0009 - find **memory leaks**, i.e. locations that allocate memory which is never deallocated
0010 - find **allocation hotspots**, i.e. code locations that trigger a lot of memory allocation calls
0011 - find **temporary allocations**, which are allocations that are directly followed by their deallocation
0012 
0013 ## Using heaptrack
0014 
0015 The recommended way is to launch your application and start tracing from the beginning:
0016 
0017     heaptrack <your application and its parameters>
0018 
0019     heaptrack output will be written to "/tmp/heaptrack.APP.PID.gz"
0020     starting application, this might take some time...
0021 
0022     ...
0023 
0024     heaptrack stats:
0025         allocations:            65
0026         leaked allocations:     60
0027         temporary allocations:  1
0028 
0029     Heaptrack finished! Now run the following to investigate the data:
0030 
0031         heaptrack_gui "/tmp/heaptrack.APP.PID.gz"
0032 
0033 Alternatively, you can attach to an already running process:
0034 
0035     heaptrack --pid $(pidof <your application>)
0036 
0037     heaptrack output will be written to "/tmp/heaptrack.APP.PID.gz"
0038     injecting heaptrack into application via GDB, this might take some time...
0039     injection finished
0040 
0041     ...
0042 
0043     Heaptrack finished! Now run the following to investigate the data:
0044 
0045         heaptrack_gui "/tmp/heaptrack.APP.PID.gz"
0046 
0047 ## Building heaptrack
0048 
0049 Heaptrack is split into two parts: The data collector, i.e. `heaptrack` itself, and the
0050 analyzer GUI called `heaptrack_gui`. The following summarizes the dependencies for these
0051 two parts as they can be build independently. You will find corresponding development
0052 packages on all major distributions for these dependencies.
0053 
0054 On an embedded device or older Linux distribution, you will only want to build `heaptrack`.
0055 The data can then be analyzed on a different machine with a more modern Linux distribution
0056 that has access to the required GUI dependencies.
0057 
0058 If you need help with building, deploying or using heaptrack, you can contact KDAB for
0059 commercial support: https://www.kdab.com/software-services/workshops/profiling-workshops/
0060 
0061 ### Shared dependencies
0062 
0063 Both parts require the following tools and libraries:
0064 
0065 - cmake 2.8.9 or higher
0066 - a C\+\+11 enabled compiler like g\+\+ or clang\+\+
0067 - zlib
0068 - optionally: zstd for faster (de)compression
0069 - elfutils
0070 - libdl
0071 - pthread
0072 - libc
0073 
0074 ### `heaptrack` dependencies
0075 
0076 The heaptrack data collector and the simplistic `heaptrack_print` analyzer depend on the
0077 following libraries:
0078 
0079 - boost 1.41 or higher: iostreams, program_options
0080 - libunwind
0081 
0082 For runtime-attaching, you will need `gdb` installed.
0083 
0084 ### `heaptrack_gui` dependencies
0085 
0086 The graphical user interface to interpret and analyze the data collected by heaptrack
0087 depends on Qt 5 and some KDE libraries:
0088 
0089 - extra-cmake-modules
0090 - Qt 5.2 or higher: Core, Widgets
0091 - KDE Frameworks 5: CoreAddons, I18n, ItemModels, ThreadWeaver, ConfigWidgets, KIO, IconThemes
0092 
0093 When any of these dependencies is missing, `heaptrack_gui` will not be build.
0094 Optionally, install the following dependencies to get additional features in
0095 the GUI:
0096 
0097 - KDiagram: KChart (for chart visualizations)
0098 
0099 ### Compiling
0100 
0101 Run the following commands to compile heaptrack. Do pay attention to the output
0102 of the CMake command, as it will tell you about missing dependencies!
0103 
0104     cd heaptrack # i.e. the source folder
0105     mkdir build
0106     cd build
0107     cmake -DCMAKE_BUILD_TYPE=Release .. # look for messages about missing dependencies!
0108     make -j$(nproc)
0109 
0110 #### Compile `heaptrack_gui` on macOS using homebrew
0111 
0112 `heaptrack_print` and `heaptrack_gui` can be built on platforms other than Linux, using the dependencies mentioned above.
0113 On macOS the dependencies can be installed easily using [homebrew](http://brew.sh) and the [KDE homebrew tap](https://github.com/KDE-mac/homebrew-kde).
0114 
0115     brew install qt@5
0116 
0117     # prepare tap
0118     brew tap kde-mac/kde https://invent.kde.org/packaging/homebrew-kde.git
0119     "$(brew --repo kde-mac/kde)/tools/do-caveats.sh"
0120     
0121     # install dependencies
0122     brew install kde-mac/kde/kf5-kcoreaddons kde-mac/kde/kf5-kitemmodels kde-mac/kde/kf5-kconfigwidgets \
0123                  kde-mac/kde/kf5-kio kde-mac/kde/kdiagram \
0124                  extra-cmake-modules ki18n threadweaver \
0125                  boost zstd gettext
0126     
0127     # run manual steps as printed by brew
0128     ln -sfv "$(brew --prefix)/share/kf5" "$HOME/Library/Application Support"
0129     ln -sfv "$(brew --prefix)/share/knotifications5" "$HOME/Library/Application Support"
0130     ln -sfv "$(brew --prefix)/share/kservices5" "$HOME/Library/Application Support"
0131     ln -sfv "$(brew --prefix)/share/kservicetypes5" "$HOME/Library/Application Support"
0132 
0133 To compile make sure to use Qt from homebrew and to have gettext in the path:
0134 
0135     cd heaptrack # i.e. the source folder
0136     mkdir build
0137     cd build
0138     CMAKE_PREFIX_PATH=/opt/homebrew/opt/qt@5 PATH=$PATH:/opt/homebrew/opt/gettext/bin cmake ..
0139     cmake -DCMAKE_BUILD_TYPE=Release .. # look for messages about missing dependencies!
0140     make heaptrack_gui heaptrack_print
0141 
0142 ## Interpreting the heap profile
0143 
0144 Heaptrack generates data files that are impossible to analyze for a human. Instead, you need
0145 to use either `heaptrack_print` or `heaptrack_gui` to interpret the results.
0146 
0147 ### heaptrack_gui
0148 
0149 ![heaptrack_gui flamegraph page](screenshots/gui_flamegraph.png?raw=true "heaptrack_gui flamegraph page")
0150 
0151 ![heaptrack_gui allocations chart page](screenshots/gui_allocations_chart.png?raw=true "heaptrack_gui allocations chart page")
0152 
0153 The highly recommended way to analyze a heap profile is by using the `heaptrack_gui` tool.
0154 It depends on Qt 5 and KF 5 to graphically visualize the recorded data. It features:
0155 
0156 - a summary page of the data
0157 - bottom-up and top-down tree views of the code locations that allocated memory with
0158   their aggregated cost and stack traces
0159 - flame graph visualization
0160 - graphs of allocation costs over time
0161 
0162 ### heaptrack_print
0163 
0164 The `heaptrack_print` tool is a command line application with minimal dependencies. It takes
0165 the heap profile, analyzes it, and prints the results in ASCII format to the command line.
0166 
0167 In its most simple form, you can use it like this:
0168 
0169     heaptrack_print heaptrack.APP.PID.gz | less
0170 
0171 By default, the report will contain three sections:
0172 
0173     MOST CALLS TO ALLOCATION FUNCTIONS
0174     PEAK MEMORY CONSUMERS
0175     MOST TEMPORARY ALLOCATIONS
0176 
0177 Each section then lists the top ten hotspots, i.e. code locations that triggered e.g.
0178 the most memory allocations.
0179 
0180 Have a look at `heaptrack_print --help` for changing the output format and other options.
0181 
0182 Note that you can use this tool to convert a heaptrack data file to the Massif data format.
0183 You can generate a collapsed stack report for consumption by `flamegraph.pl`.
0184 
0185 ## Comparison to Valgrind's massif
0186 
0187 The idea to build heaptrack was born out of the pain in working with Valgrind's massif.
0188 Valgrind comes with a huge overhead in both memory and time, which sometimes prevent you
0189 from running it on larger real-world applications. Most of what Valgrind does is not
0190 needed for a simple heap profiler.
0191 
0192 ### Advantages of heaptrack over massif
0193 
0194 - *speed and memory overhead*
0195 
0196   Multi-threaded applications are not serialized when you trace them with heaptrack and
0197   even for single-threaded applications the overhead in both time and memory is significantly
0198   lower. Most notably, you only pay a price when you allocate memory -- time-intensive CPU
0199   calculations are not slowed down at all, contrary to what happens in Valgrind.
0200 
0201 - *more data*
0202 
0203   Valgrind's massif aggregates data before writing the report. This step loses a lot of
0204   useful information. Most notably, you are not longer able to find out how often memory
0205   was allocated, or where temporary allocations are triggered. Heaptrack does not aggregate the
0206   data until you interpret it, which allows for more useful insights into your allocation patterns.
0207 
0208 ### Advantages of massif over heaptrack
0209 
0210 - *ability to profile page allocations as heap*
0211 
0212   This allows you to heap-profile applications that use pool allocators that circumvent
0213   malloc & friends. Heaptrack can in principle also profile such applications, but it
0214   requires code changes to annotate the memory pool implementation.
0215 
0216 - *ability to profile stack allocations*
0217 
0218   This is inherently impossible to implement efficiently in heaptrack as far as I know.
0219 
0220 ## Contributing to heaptrack
0221 
0222 As a FOSS project, we welcome contributions of any form. You can help improve the project by:
0223 
0224 - submitting bug reports at https://bugs.kde.org/enter_bug.cgi?product=Heaptrack
0225 - contributing patches via https://invent.kde.org/sdk/heaptrack
0226 - translating the GUI with the help of https://l10n.kde.org/
0227 - writing documentation on https://userbase.kde.org/Heaptrack
0228 
0229 When submitting bug reports, you can anonymize your data with the `tools/anonymize` script:
0230 
0231     tools/anonymize heaptrack.APP.PID.gz heaptrack.bug_report_data.gz
0232 
0233 ## Known bugs and limitations
0234 
0235 ### Issues with old gold linker
0236 
0237 Libunwind may produce bogus backtraces when unwinding from code linked with old versions of the gold linker.
0238 In such cases, recording with heaptrack seems to work and produces data files. But parsing these data files
0239 with heaptrack_gui will often lead to out-of-memory crashes. Looking at the data with heaptrack_print, one
0240 will see garbage backtraces that are completely broken.
0241 
0242 If you encounter such issues, try to relink your application and also libunwind with `ld.bfd` instead of `ld.gold`.
0243 You can see if you are affected by running the libunwind unit tests via `make check`. But do note that you
0244 need to relink your application too, not only libunwind.
0245 
0246 ### Executables built with ASAN (Address Sanitizer)
0247 
0248 If you run heaptrack on an application built with ASAN, you'll likely get this fatal error on startup:
0249 
0250     ASan runtime does not come first in initial library list [...]
0251 
0252 The solution is to pass the `--asan` flag to heaptrack.
0253 Note: this only work for binaries built with `gcc` (i.e. those which link to `libasan.so`).
0254 Binaries built with `clang`'s ASAN enabled are not supported at the moment.
0255