Warning, /sdk/heaptrack/README.md is written in an unsupported language. File is not indexed.

0001 # heaptrack - a heap memory profiler for Linux
0002 
0003 ![heaptrack_gui summary page](screenshots/gui_summary.png?raw=true "heaptrack_gui summary page")
0004 
0005 Heaptrack traces all memory allocations and annotates these events with stack traces.
0006 Dedicated analysis tools then allow you to interpret the heap memory profile to:
0007 
0008 - find hotspots that need to be optimized to reduce the **memory footprint** of your application
0009 - find **memory leaks**, i.e. locations that allocate memory which is never deallocated
0010 - find **allocation hotspots**, i.e. code locations that trigger a lot of memory allocation calls
0011 - find **temporary allocations**, which are allocations that are directly followed by their deallocation
0012 
0013 ## Using heaptrack
0014 
0015 The recommended way is to launch your application and start tracing from the beginning:
0016 
0017     heaptrack <your application and its parameters>
0018 
0019     heaptrack output will be written to "/tmp/heaptrack.APP.PID.gz"
0020     starting application, this might take some time...
0021 
0022     ...
0023 
0024     heaptrack stats:
0025         allocations:            65
0026         leaked allocations:     60
0027         temporary allocations:  1
0028 
0029     Heaptrack finished! Now run the following to investigate the data:
0030 
0031         heaptrack_gui "/tmp/heaptrack.APP.PID.gz"
0032 
0033 Alternatively, you can attach to an already running process:
0034 
0035     heaptrack --pid $(pidof <your application>)
0036 
0037     heaptrack output will be written to "/tmp/heaptrack.APP.PID.gz"
0038     injecting heaptrack into application via GDB, this might take some time...
0039     injection finished
0040 
0041     ...
0042 
0043     Heaptrack finished! Now run the following to investigate the data:
0044 
0045         heaptrack_gui "/tmp/heaptrack.APP.PID.gz"
0046 
0047 ## Building heaptrack
0048 
0049 Heaptrack is split into two parts: The data collector, i.e. `heaptrack` itself, and the
0050 analyzer GUI called `heaptrack_gui`. The following summarizes the dependencies for these
0051 two parts as they can be build independently. You will find corresponding development
0052 packages on all major distributions for these dependencies.
0053 
0054 On an embedded device or older Linux distribution, you will only want to build `heaptrack`.
0055 The data can then be analyzed on a different machine with a more modern Linux distribution
0056 that has access to the required GUI dependencies.
0057 
0058 If you need help with building, deploying or using heaptrack, you can contact KDAB for
0059 commercial support: https://www.kdab.com/software-services/workshops/profiling-workshops/
0060 
0061 ### Shared dependencies
0062 
0063 Both parts require the following tools and libraries:
0064 
0065 - cmake 2.8.9 or higher
0066 - a C\+\+11 enabled compiler like g\+\+ or clang\+\+
0067 - zlib
0068 - optionally: zstd for faster (de)compression
0069 - libdl
0070 - pthread
0071 - libc
0072 
0073 ### `heaptrack` dependencies
0074 
0075 The heaptrack data collector and the simplistic `heaptrack_print` analyzer depend on the
0076 following libraries:
0077 
0078 - boost 1.41 or higher: iostreams, program_options
0079 - libunwind
0080 
0081 For runtime-attaching, you will need `gdb` installed.
0082 
0083 ### `heaptrack_gui` dependencies
0084 
0085 The graphical user interface to interpret and analyze the data collected by heaptrack
0086 depends on Qt 5 and some KDE libraries:
0087 
0088 - extra-cmake-modules
0089 - Qt 5.2 or higher: Core, Widgets
0090 - KDE Frameworks 5: CoreAddons, I18n, ItemModels, ThreadWeaver, ConfigWidgets, KIO, IconThemes
0091 
0092 When any of these dependencies is missing, `heaptrack_gui` will not be build.
0093 Optionally, install the following dependencies to get additional features in
0094 the GUI:
0095 
0096 - KDiagram: KChart (for chart visualizations)
0097 
0098 ### Compiling
0099 
0100 Run the following commands to compile heaptrack. Do pay attention to the output
0101 of the CMake command, as it will tell you about missing dependencies!
0102 
0103     cd heaptrack # i.e. the source folder
0104     mkdir build
0105     cd build
0106     cmake -DCMAKE_BUILD_TYPE=Release .. # look for messages about missing dependencies!
0107     make -j$(nproc)
0108 
0109 #### Compile `heaptrack_gui` on macOS using homebrew
0110 
0111 `heaptrack_print` and `heaptrack_gui` can be built on platforms other than Linux, using the dependencies mentioned above.
0112 On macOS the dependencies can be installed easily using [homebrew](http://brew.sh) and the [KDE homebrew tap](https://github.com/KDE-mac/homebrew-kde).
0113 
0114     # prepare tap
0115     brew tap kde-mac/kde https://invent.kde.org/packaging/homebrew-kde.git
0116     "$(brew --repo kde-mac/kde)/tools/do-caveats.sh"
0117     
0118     # install dependencies
0119     brew install kde-mac/kde/kf5-kcoreaddons kde-mac/kde/kf5-kitemmodels kde-mac/kde/kf5-kconfigwidgets \
0120                  kde-mac/kde/kf5-kio kde-mac/kde/kdiagram \
0121                  kde-extra-cmake-modules kde-ki18n kde-threadweaver \
0122                  boost zstd gettext
0123     
0124     # run manual steps as printed by brew
0125     ln -sfv "$(brew --prefix)/share/kf5" "$HOME/Library/Application Support"
0126     ln -sfv "$(brew --prefix)/share/knotifications5" "$HOME/Library/Application Support"
0127     ln -sfv "$(brew --prefix)/share/kservices5" "$HOME/Library/Application Support"
0128     ln -sfv "$(brew --prefix)/share/kservicetypes5" "$HOME/Library/Application Support"
0129 
0130 To compile make sure to use Qt from homebrew and to have gettext in the path:
0131 
0132     cd heaptrack # i.e. the source folder
0133     mkdir build
0134     cd build
0135     CMAKE_PREFIX_PATH=/usr/local/opt/qt PATH=$PATH:/usr/local/opt/gettext/bin cmake ..
0136     cmake -DCMAKE_BUILD_TYPE=Release .. # look for messages about missing dependencies!
0137     make heaptrack_gui heaptrack_print
0138 
0139 ## Interpreting the heap profile
0140 
0141 Heaptrack generates data files that are impossible to analyze for a human. Instead, you need
0142 to use either `heaptrack_print` or `heaptrack_gui` to interpret the results.
0143 
0144 ### heaptrack_gui
0145 
0146 ![heaptrack_gui flamegraph page](screenshots/gui_flamegraph.png?raw=true "heaptrack_gui flamegraph page")
0147 
0148 ![heaptrack_gui allocations chart page](screenshots/gui_allocations_chart.png?raw=true "heaptrack_gui allocations chart page")
0149 
0150 The highly recommended way to analyze a heap profile is by using the `heaptrack_gui` tool.
0151 It depends on Qt 5 and KF 5 to graphically visualize the recorded data. It features:
0152 
0153 - a summary page of the data
0154 - bottom-up and top-down tree views of the code locations that allocated memory with
0155   their aggregated cost and stack traces
0156 - flame graph visualization
0157 - graphs of allocation costs over time
0158 
0159 ### heaptrack_print
0160 
0161 The `heaptrack_print` tool is a command line application with minimal dependencies. It takes
0162 the heap profile, analyzes it, and prints the results in ASCII format to the command line.
0163 
0164 In its most simple form, you can use it like this:
0165 
0166     heaptrack_print heaptrack.APP.PID.gz | less
0167 
0168 By default, the report will contain three sections:
0169 
0170     MOST CALLS TO ALLOCATION FUNCTIONS
0171     PEAK MEMORY CONSUMERS
0172     MOST TEMPORARY ALLOCATIONS
0173 
0174 Each section then lists the top ten hotspots, i.e. code locations that triggered e.g.
0175 the most memory allocations.
0176 
0177 Have a look at `heaptrack_print --help` for changing the output format and other options.
0178 
0179 Note that you can use this tool to convert a heaptrack data file to the Massif data format.
0180 You can generate a collapsed stack report for consumption by `flamegraph.pl`.
0181 
0182 ## Comparison to Valgrind's massif
0183 
0184 The idea to build heaptrack was born out of the pain in working with Valgrind's massif.
0185 Valgrind comes with a huge overhead in both memory and time, which sometimes prevent you
0186 from running it on larger real-world applications. Most of what Valgrind does is not
0187 needed for a simple heap profiler.
0188 
0189 ### Advantages of heaptrack over massif
0190 
0191 - *speed and memory overhead*
0192 
0193   Multi-threaded applications are not serialized when you trace them with heaptrack and
0194   even for single-threaded applications the overhead in both time and memory is significantly
0195   lower. Most notably, you only pay a price when you allocate memory -- time-intensive CPU
0196   calculations are not slowed down at all, contrary to what happens in Valgrind.
0197 
0198 - *more data*
0199 
0200   Valgrind's massif aggregates data before writing the report. This step loses a lot of
0201   useful information. Most notably, you are not longer able to find out how often memory
0202   was allocated, or where temporary allocations are triggered. Heaptrack does not aggregate the
0203   data until you interpret it, which allows for more useful insights into your allocation patterns.
0204 
0205 ### Advantages of massif over heaptrack
0206 
0207 - *ability to profile page allocations as heap*
0208 
0209   This allows you to heap-profile applications that use pool allocators that circumvent
0210   malloc & friends. Heaptrack can in principle also profile such applications, but it
0211   requires code changes to annotate the memory pool implementation.
0212 
0213 - *ability to profile stack allocations*
0214 
0215   This is inherently impossible to implement efficiently in heaptrack as far as I know.
0216 
0217 ## Contributing to heaptrack
0218 
0219 As a FOSS project, we welcome contributions of any form. You can help improve the project by:
0220 
0221 - submitting bug reports at https://bugs.kde.org/enter_bug.cgi?product=Heaptrack
0222 - contributing patches via https://invent.kde.org/sdk/heaptrack
0223 - translating the GUI with the help of https://l10n.kde.org/
0224 - writing documentation on https://userbase.kde.org/Heaptrack
0225 
0226 When submitting bug reports, you can anonymize your data with the `tools/anonymize` script:
0227 
0228     tools/anonymize heaptrack.APP.PID.gz heaptrack.bug_report_data.gz
0229 
0230 ## Known bugs and limitations
0231 
0232 ### Issues with old gold linker
0233 
0234 Libunwind may produce bogus backtraces when unwinding from code linked with old versions of the gold linker.
0235 In such cases, recording with heaptrack seems to work and produces data files. But parsing these data files
0236 with heaptrack_gui will often lead to out-of-memory crashes. Looking at the data with heaptrack_print, one
0237 will see garbage backtraces that are completely broken.
0238 
0239 If you encounter such issues, try to relink your application and also libunwind with `ld.bfd` instead of `ld.gold`.
0240 You can see if you are affected by running the libunwind unit tests via `make check`. But do note that you
0241 need to relink your application too, not only libunwind.