Warning, /multimedia/kdenlive/dev-docs/mlt-intro.md is written in an unsupported language. File is not indexed.

0001 # MLT Concepts
0002 
0003 This article gives you some basic understanding about at least some MLT concepts. MLT is the media engine Kdenlive relies on for composing and rendering audio and video.
0004 
0005 Please understand that we can’t and don’t strive to give an extensive and exhaustive introduction to MLT. Instead, we see this as a kind of layman’s intro to foster some rough first understanding of the workings of MLT in order to understand how Kdenlive makes use of MLT.
0006 
0007 If you want to learn more about MLT please read the official [MLT Framework Design][mlt-docs] documentation.
0008 
0009 ## The Basic Four Services of MLT
0010 
0011 At MLT’s core are its four basic services (or types of services):
0012 
0013 * **producers** are sources of individual audio and video frames. As we’ll see later, there are basic (simple) producers, as well as more complex ones.
0014 * **consumers** are the sinks for audio and video frames. Examples are a screen display such as Kdenlive’s monitors, or an audio/video encoder such as AAC/H.264 that writes video container files.
0015 * **filters** (in MLT parlance) sit somewhere in between the producers and consomers. These filters modify frames as they are passing through them. In Kdenlive’s user interface, you usually see filters as effects.
0016 * **transitions** (in MLT parlance) combine two input frames into a single new output frame. See below for more details.
0017 
0018 ### Producers
0019 
0020 So producers produce frames. In the MLT framework, they produce frames only when asked to do so, because MLT adheres to pulling frames; that’s why there are tractors and consumers.
0021 
0022 Alas, the producer services can be further divided into:
0023 
0024 * **basic producers** produce frames directly from audio/video sources; these sources may be files, devices, title clips, color clips, and other sources. See MLT basic producers for more details.
0025 * **playlists** are sequential containers for producers and optional intermediate blank spaces.
0026 * **tractors** are …
0027 * **multitracks** are …
0028 
0029 #### Basic Producers
0030 
0031 MLT comes with a lot of producers, which we cannot all tackle here. Instead, we’re just shining some light one what we think will be the most notable producers in the context of Kdenlive. These are (in order of appearance):
0032 
0033 * **avformat** and **avformat-novalidate** for reading the myriad of different audio/video media format using FFmpeg
0034 * **timewarp** for speeing up or slowing down audio/video media
0035 * **pixbuf** for reading and understanding the many different image formats
0036 * **kdenlivetitle** for rendering Kdenlive titles
0037 * **color/colour** producer for producing uni-color frames
0038 
0039 ##### Avformat Producer
0040 
0041 The `avformat` producer is probably the most-used staple producer that produces frames from audio/video media files using the famous FFmpeg multimedia handling library. Whatever you can throw successfully at FFmpeg to decode or encode it, can thus be used in Kdenlive. For more details, please refer to MLT’s avformat producer documentation.
0042 
0043 ##### Avformat-novalidate Producer
0044 
0045 This variant of the `avformat` producer named `avformat-novalidate` is used to speed up loading playlists with known good media files. (See also this MLT commit message about avformat-novalidate.)
0046 
0047 ##### Timewarp Producer
0048 
0049 The `timewarp` producer is relatively new that allows video and audio to be sped up (max. 20x) or slowed down (down to 0.01x). It can even reverse audio and video within the same range between 0.01x and 20x.
0050 
0051 Timewarp actually is an encapsulated producer. See also the MLT timewarp producer documentation. Kdenlive uses this producer internally to emulate the speed effect (which isn’t, technically spoken, an MLT filter, but an MLT producer).
0052 
0053 ##### Pixbuf Producer
0054 
0055 The `pixmap` producer produces frames from a wide range of bitmap and vector graphics formats. Under the hood, it uses the GDK pixbuf, hence its name (see also the
0056 MLT pixbuf producer documentation). Some notable features of this producer are:
0057 
0058 * it supports image sequences. For this, the resource property (that is, path and filename) needs to contain the wildcard “/.all.” (note the trailing dot). The producer then loads all image files matching the following file extensions from the path.
0059 * some of the supported image file formats are:
0060  * PNG (.png)
0061  * JPEG (.jpg, jpeg)
0062  * SVG (.svg)
0063  * TIFF
0064  * XPM
0065  * …as well as many more.
0066 
0067 ##### Kdenlivetitle Producer
0068 
0069 Feed the `kdenlivetitle` producer some XML and it will produce frames containing beautiful titles. The XML describing a title has to be directly included with this service in its `xmldata` property.
0070 Color/Colour Producers
0071 
0072 This producer has two names: `color` and `colour`. It simply produces frames of only a single color. The color and alpha information is encoded in the `resource` property of the producer. This resource property can take on these values:
0073 
0074 * a **32bit hex value** in textual form of RRGGBBAA. Here, RR is the 8bit red channel value ranging from `00` to `ff`. Similar, GG and BB encode the 8bit green and blue channel values, respectively. AA is an 8 bit alpha value, with `ff` denoting full opacity, and `00` meaning full transparency.
0075 * few **well-known colo(u)r names**, in particular, `black` and `white`.
0076 
0077 **Note:** Kdenlive uses the `colour`(!) producer for its timeline-internal lowest black track. In contrast, Kdenlive uses `color` producers for all the (project) bin color clips. Now that’s politically correct!
0078 
0079 ### Tractors
0080 
0081 MLT has a somewhat unique design in that it doesn’t make use of a rendering tree. Instead, MLT uses what it terms “networks” (in quotes, see also the [MLT Framework Design][mlt-docs] document). These networks connect producers to consumers, with filters (for the effects) and transitions (for mixing) in between.
0082 
0083 An important element for pulling frames from producers on multiple (timeline) tracks and coordinating them correctly in time in order to arrive at a final frame for output is MLT’s tractor. Yes, it’s really called a “tractor”
0084 
0085 **Note:** later on we will learn about playlists, which are a (sub)type of producers. The important difference between tractors and playlists is, that playlists can use multiple producers only one after another (that is, sequentially). In contrast, tractors can use producers in parallel at the same time.
0086 
0087 ### Consumers
0088 
0089 TODO
0090 
0091 ### Filters
0092 
0093 TODO
0094 
0095 ### Transitions
0096 
0097 The MLT term transition will probably confuse a lot of readers; more so, the longer they have worked with other non-linear video editors. Simply spoken, MLT’s transitions are mixers that combine exactly two input frames into a new single output frame.
0098 
0099 **Note:** this is slightly simplified on purpose; more technically spoken, MLT’s transitions still outputs two frames, where the so-called “B frame” comes out unmodified, and only the “A frame” is mixed from the input A and B frames.
0100 
0101 [mlt-docs]: https://www.mltframework.org/docs/framework/
0102