Warning, /sdk/heaptrack/README.md is written in an unsupported language. File is not indexed.

0001 # heaptrack - a heap memory profiler for Linux
0002 
0003 ![heaptrack_gui summary page](screenshots/gui_summary.png?raw=true "heaptrack_gui summary page")
0004 
0005 Heaptrack traces all memory allocations and annotates these events with stack traces.
0006 Dedicated analysis tools then allow you to interpret the heap memory profile to:
0007 
0008 - find hotspots that need to be optimized to reduce the **memory footprint** of your application
0009 - find **memory leaks**, i.e. locations that allocate memory which is never deallocated
0010 - find **allocation hotspots**, i.e. code locations that trigger a lot of memory allocation calls
0011 - find **temporary allocations**, which are allocations that are directly followed by their deallocation
0012 
0013 ## Using heaptrack
0014 
0015 The recommended way is to launch your application and start tracing from the beginning:
0016 
0017     heaptrack <your application and its parameters>
0018 
0019     heaptrack output will be written to "/tmp/heaptrack.APP.PID.gz"
0020     starting application, this might take some time...
0021 
0022     ...
0023 
0024     heaptrack stats:
0025         allocations:            65
0026         leaked allocations:     60
0027         temporary allocations:  1
0028 
0029     Heaptrack finished! Now run the following to investigate the data:
0030 
0031         heaptrack_gui "/tmp/heaptrack.APP.PID.gz"
0032 
0033 Alternatively, you can attach to an already running process:
0034 
0035     heaptrack --pid $(pidof <your application>)
0036 
0037     heaptrack output will be written to "/tmp/heaptrack.APP.PID.gz"
0038     injecting heaptrack into application via GDB, this might take some time...
0039     injection finished
0040 
0041     ...
0042 
0043     Heaptrack finished! Now run the following to investigate the data:
0044 
0045         heaptrack_gui "/tmp/heaptrack.APP.PID.gz"
0046 
0047 ## Building heaptrack
0048 
0049 Heaptrack is split into two parts: The data collector, i.e. `heaptrack` itself, and the
0050 analyzer GUI called `heaptrack_gui`. The following summarizes the dependencies for these
0051 two parts as they can be build independently. You will find corresponding development
0052 packages on all major distributions for these dependencies.
0053 
0054 On an embedded device or older Linux distribution, you will only want to build `heaptrack`.
0055 The data can then be analyzed on a different machine with a more modern Linux distribution
0056 that has access to the required GUI dependencies.
0057 
0058 If you need help with building, deploying or using heaptrack, you can contact KDAB for
0059 commercial support: https://www.kdab.com/software-services/workshops/profiling-workshops/
0060 
0061 ### Shared dependencies
0062 
0063 Both parts require the following tools and libraries:
0064 
0065 - cmake 2.8.9 or higher
0066 - a C\+\+11 enabled compiler like g\+\+ or clang\+\+
0067 - zlib
0068 - optionally: zstd for faster (de)compression
0069 - libdl
0070 - pthread
0071 - libc
0072 
0073 ### `heaptrack` dependencies
0074 
0075 The heaptrack data collector and the simplistic `heaptrack_print` analyzer depend on the
0076 following libraries:
0077 
0078 - boost 1.41 or higher: iostreams, program_options
0079 - libunwind
0080 
0081 For runtime-attaching, you will need `gdb` installed.
0082 
0083 ### `heaptrack_gui` dependencies
0084 
0085 The graphical user interface to interpret and analyze the data collected by heaptrack
0086 depends on Qt 5 and some KDE libraries:
0087 
0088 - extra-cmake-modules
0089 - Qt 5.2 or higher: Core, Widgets
0090 - KDE Frameworks 5: CoreAddons, I18n, ItemModels, ThreadWeaver, ConfigWidgets, KIO, IconThemes
0091 
0092 When any of these dependencies is missing, `heaptrack_gui` will not be build.
0093 Optionally, install the following dependencies to get additional features in
0094 the GUI:
0095 
0096 - KDiagram: KChart (for chart visualizations)
0097 
0098 ### Compiling
0099 
0100 Run the following commands to compile heaptrack. Do pay attention to the output
0101 of the CMake command, as it will tell you about missing dependencies!
0102 
0103     cd heaptrack # i.e. the source folder
0104     mkdir build
0105     cd build
0106     cmake -DCMAKE_BUILD_TYPE=Release .. # look for messages about missing dependencies!
0107     make -j$(nproc)
0108 
0109 #### Compile `heaptrack_gui` on macOS using homebrew
0110 
0111 `heaptrack_print` and `heaptrack_gui` can be built on platforms other than Linux, using the dependencies mentioned above.
0112 On macOS the dependencies can be installed easily using [homebrew](http://brew.sh) and the [KDE homebrew tap](https://github.com/KDE-mac/homebrew-kde).
0113 
0114     brew install qt@5
0115 
0116     # prepare tap
0117     brew tap kde-mac/kde https://invent.kde.org/packaging/homebrew-kde.git
0118     "$(brew --repo kde-mac/kde)/tools/do-caveats.sh"
0119     
0120     # install dependencies
0121     brew install kde-mac/kde/kf5-kcoreaddons kde-mac/kde/kf5-kitemmodels kde-mac/kde/kf5-kconfigwidgets \
0122                  kde-mac/kde/kf5-kio kde-mac/kde/kdiagram \
0123                  kde-extra-cmake-modules kde-ki18n kde-threadweaver \
0124                  boost zstd gettext
0125     
0126     # run manual steps as printed by brew
0127     ln -sfv "$(brew --prefix)/share/kf5" "$HOME/Library/Application Support"
0128     ln -sfv "$(brew --prefix)/share/knotifications5" "$HOME/Library/Application Support"
0129     ln -sfv "$(brew --prefix)/share/kservices5" "$HOME/Library/Application Support"
0130     ln -sfv "$(brew --prefix)/share/kservicetypes5" "$HOME/Library/Application Support"
0131 
0132 To compile make sure to use Qt from homebrew and to have gettext in the path:
0133 
0134     cd heaptrack # i.e. the source folder
0135     mkdir build
0136     cd build
0137     CMAKE_PREFIX_PATH=/opt/homebrew/opt/qt@5 PATH=$PATH:/opt/homebrew/opt/gettext/bin cmake ..
0138     cmake -DCMAKE_BUILD_TYPE=Release .. # look for messages about missing dependencies!
0139     make heaptrack_gui heaptrack_print
0140 
0141 ## Interpreting the heap profile
0142 
0143 Heaptrack generates data files that are impossible to analyze for a human. Instead, you need
0144 to use either `heaptrack_print` or `heaptrack_gui` to interpret the results.
0145 
0146 ### heaptrack_gui
0147 
0148 ![heaptrack_gui flamegraph page](screenshots/gui_flamegraph.png?raw=true "heaptrack_gui flamegraph page")
0149 
0150 ![heaptrack_gui allocations chart page](screenshots/gui_allocations_chart.png?raw=true "heaptrack_gui allocations chart page")
0151 
0152 The highly recommended way to analyze a heap profile is by using the `heaptrack_gui` tool.
0153 It depends on Qt 5 and KF 5 to graphically visualize the recorded data. It features:
0154 
0155 - a summary page of the data
0156 - bottom-up and top-down tree views of the code locations that allocated memory with
0157   their aggregated cost and stack traces
0158 - flame graph visualization
0159 - graphs of allocation costs over time
0160 
0161 ### heaptrack_print
0162 
0163 The `heaptrack_print` tool is a command line application with minimal dependencies. It takes
0164 the heap profile, analyzes it, and prints the results in ASCII format to the command line.
0165 
0166 In its most simple form, you can use it like this:
0167 
0168     heaptrack_print heaptrack.APP.PID.gz | less
0169 
0170 By default, the report will contain three sections:
0171 
0172     MOST CALLS TO ALLOCATION FUNCTIONS
0173     PEAK MEMORY CONSUMERS
0174     MOST TEMPORARY ALLOCATIONS
0175 
0176 Each section then lists the top ten hotspots, i.e. code locations that triggered e.g.
0177 the most memory allocations.
0178 
0179 Have a look at `heaptrack_print --help` for changing the output format and other options.
0180 
0181 Note that you can use this tool to convert a heaptrack data file to the Massif data format.
0182 You can generate a collapsed stack report for consumption by `flamegraph.pl`.
0183 
0184 ## Comparison to Valgrind's massif
0185 
0186 The idea to build heaptrack was born out of the pain in working with Valgrind's massif.
0187 Valgrind comes with a huge overhead in both memory and time, which sometimes prevent you
0188 from running it on larger real-world applications. Most of what Valgrind does is not
0189 needed for a simple heap profiler.
0190 
0191 ### Advantages of heaptrack over massif
0192 
0193 - *speed and memory overhead*
0194 
0195   Multi-threaded applications are not serialized when you trace them with heaptrack and
0196   even for single-threaded applications the overhead in both time and memory is significantly
0197   lower. Most notably, you only pay a price when you allocate memory -- time-intensive CPU
0198   calculations are not slowed down at all, contrary to what happens in Valgrind.
0199 
0200 - *more data*
0201 
0202   Valgrind's massif aggregates data before writing the report. This step loses a lot of
0203   useful information. Most notably, you are not longer able to find out how often memory
0204   was allocated, or where temporary allocations are triggered. Heaptrack does not aggregate the
0205   data until you interpret it, which allows for more useful insights into your allocation patterns.
0206 
0207 ### Advantages of massif over heaptrack
0208 
0209 - *ability to profile page allocations as heap*
0210 
0211   This allows you to heap-profile applications that use pool allocators that circumvent
0212   malloc & friends. Heaptrack can in principle also profile such applications, but it
0213   requires code changes to annotate the memory pool implementation.
0214 
0215 - *ability to profile stack allocations*
0216 
0217   This is inherently impossible to implement efficiently in heaptrack as far as I know.
0218 
0219 ## Contributing to heaptrack
0220 
0221 As a FOSS project, we welcome contributions of any form. You can help improve the project by:
0222 
0223 - submitting bug reports at https://bugs.kde.org/enter_bug.cgi?product=Heaptrack
0224 - contributing patches via https://invent.kde.org/sdk/heaptrack
0225 - translating the GUI with the help of https://l10n.kde.org/
0226 - writing documentation on https://userbase.kde.org/Heaptrack
0227 
0228 When submitting bug reports, you can anonymize your data with the `tools/anonymize` script:
0229 
0230     tools/anonymize heaptrack.APP.PID.gz heaptrack.bug_report_data.gz
0231 
0232 ## Known bugs and limitations
0233 
0234 ### Issues with old gold linker
0235 
0236 Libunwind may produce bogus backtraces when unwinding from code linked with old versions of the gold linker.
0237 In such cases, recording with heaptrack seems to work and produces data files. But parsing these data files
0238 with heaptrack_gui will often lead to out-of-memory crashes. Looking at the data with heaptrack_print, one
0239 will see garbage backtraces that are completely broken.
0240 
0241 If you encounter such issues, try to relink your application and also libunwind with `ld.bfd` instead of `ld.gold`.
0242 You can see if you are affected by running the libunwind unit tests via `make check`. But do note that you
0243 need to relink your application too, not only libunwind.