HLS/MPEG-DASH/RTMP with nginx

With a few open source tools, you can stream a playlist of videos in real-time over RTMP, MPEG-DASH, and HLS in a fully automated manner.

In this case, we’ll be using several software packages:

  1. GStreamer/Libav for decoding and encoding
  2. Liquidsoap for playlist management
  3. nginx for the server
  4. nginx-rtmp-module for providing HLS/MPEG-DASH/RTMP streaming

Using Ubuntu 14.XX (or possibly anything Debian-based) will be your shortest path of resistance, as compiling these packages by hand is a considerable chore and other distributions will likely lack many of these packages. If you want to use a different OS, you can virtualize Ubuntu, possibly with a lightweight container. You will not get very far with older versions of Ubuntu (such as 12.04) as they will also lack many of these packages.

GStreamer and Liquidsoap

GStreamer is a multimedia framework similar to DirectShow, in that it allows the linking together of components such as decoders, encoders, and so on to handle media. You’ll want to install the entirety of GStreamer if you want to play a wide variety of video formats. In addition, the “gstreamer1.0-libav” package enables you to make use of the Libav library to decode media.

Note: Libav is a hostile fork of the FFmpeg project and the relevant package maintainer for Debian/Ubuntu has chosen to distribute Libav rather than FFmpeg (which is still alive and kicking).

sudo apt-get -y install libgstreamer1.0-0 gstreamer1.0-plugins-base \
   gstreamer1.0-plugins-ugly gstreamer1.0-plugins-good \
   gstreamer1.0-plugins-bad gstreamer1.0-tools gstreamer1.0-libav

Then you’ll want to install Liquidsoap. Liquidsoap provides a powerful language to tie together multimedia components, and we’ll need it for playlist support. (It does a lot more though.)

sudo apt-get -y install liquidsoap liquidsoap-plugin-all

Be aware that you’ll also be installing all of LiquidSoap’s plugins (and dependencies). In reality, you do not need the whole package to simply stream a playlist, but you’ll have more freedom to play with LiquidSoap’s various features if you choose to install everything.

nginx and nginx-rtmp

Next, you’ll want to install Nginx and the nginx-rtmp-module.

Nginx is most well known to be a high-performance web server and reverse proxy, but the nginx-rtmp-module turns nginx into a streaming media server capable of speaking RTMP, MPEG-DASH, and Apple HTTP Live Streaming (HLS) protocols.

Flash-based video players will use RTMP, but HTML5/mobile players will use MPEG-DASH or HLS. The latter two protocols provide adaptive streaming, where videos are spliced into small chunks in real-time, allowing the client to switch to a lower quality “stream” in the middle of a video (see: YouTube).

Compiling from source

While nginx is readily available in the repositories on Debian/Ubuntu, it does not come with the nginx-rtmp-module, so we’ll have to compile nginx ourselves.

sudo apt-get update
sudo apt-get install dpkg-dev git-core
mkdir ~/nginx-compile
cd ~/nginx-compile
apt-get source nginx
sudo apt-get build-dep nginx
git clone https://github.com/arut/nginx-rtmp-module.git
cd nginx-VERSION

You’ll then want to edit the debian/rules file and add the following to the “config.status.full” entry on a new line, after all the other --add-module lines.

--add-module=/path/to/nginx-compile/nginx-rtmp-module \

Proceed to compile the package.

dpkg-buildpackage -b

Afterwards, you can install both the newly created “common” and “full” packages. Make sure to apt-get remove nginx first if you have already installed nginx via the package manager.

cd ../
sudo dpkg --install nginx-common_1.4.6-1ubuntu3_all.deb
sudo dpkg --install nginx-full_1.4.6-1ubuntu3_amd64.deb

Configuring nginx

You’ll want to edit /etc/nginx/nginx.conf in your favorite editor and add the following section to the end of the file. Note that my snippet below will only configure nginx to provide a RTMP stream, but check out nginx-rtmp-module’s wiki to learn how to setup MPEG-DASH or HLS support.

rtmp {
  server {
    listen 1935;
    publish_time_fix off; # compatibility with GStreamer

    application example {
      live on;
      allow publish 127.0.0.1;
      deny publish all;
      allow play all;
    }
  }
}

Start nginx.

service start nginx

Streaming a test video

You can test whether your setup is sound using the gst-launch-1.0 program provided with GStreamer. (Be sure to change the path to the test video file.)

gst-launch-1.0 \
  filesrc location="/path/to/file.mp4" ! decodebin name=t \
  t. ! videoconvert ! x264enc bitrate=2000 ! queue ! flvmux name=mux \
  t. ! audioconvert ! voaacenc bitrate=128000 ! queue ! mux. \
  mux. ! rtmpsink location="rtmp://127.0.0.1:1935/example/live live=1"

To actually watch the stream, you can use the rtmpdump program and pipe the result to something like VLC Media Player (VLC does not yet offer native support for viewing RTMP streams).

You can also configure one of the freely available Flash streaming video players to play your stream.

Configuring Liquidsoap

With your favorite text editor again, create a stream.liq file and in it, place the following code.

set("frame.video.width", 1280)
set("frame.video.height", 720)
set("frame.video.samplerate", 25)
set("gstreamer.add_borders", true)

s = single("/path/to/video.mp4")

s = fallback([s, blank()])

output.gstreamer.audio_video(
  video_pipeline=
    "videoconvert ! x264enc bitrate=2000 ! video/x-h264,profile=baseline ! queue ! mux.",
  audio_pipeline=
    "audioconvert ! voaacenc bitrate=128000 ! queue ! mux.",
  pipeline=
    "flvmux name=mux ! rtmpsink location=\"rtmp://127.0.0.1:1935/example/live live=1\"",
  s)

Run your script file by executing liquidsoap stream.liq in your terminal. If all goes well, you should be able to watch your video by connecting to nginx as you had done before with the gst-launch tool.

Setting up a playlist

At this point, you can change

s = single("/path/to/video.mp4")

into

s = playlist("/path/to/playlist.ext")

(See Liquidsoap’s supported playlist formats.)

In conclusion

While the method I have outlined does work, it is not the only way to go about doing this. For example, you could concatenate a list of videos together with FFmpeg and send it off to your media server using the -realtime flag, skipping half of the steps in this article. You could even use one of those “screen streaming” programs with playlist support.

Remember also that video encoding (and to a smaller degree with most formats, decoding) is a CPU-intensive task. If you do not provide enough resources, the stream will stutter.

Easy JVM CPU profiling with WarmRoast

WarmRoast is a web-based CPU sampler for the Java Virtual Machine.

  • Adjustable sampling frequency.
  • Supports loading MCP mappings for deobfuscating class and method names.
  • Web-based — perform the profiling on a remote server and view the results in your browser.
  • Collapse and expand nodes to see details.
  • Easily view CPU usage per method at a glance.
  • Hover to highlight all child methods as a group.
  • See the percentage of CPU time for each method relative to its parent methods.
  • Maintains style and function with use of “File -> Save As” (in tested browsers).

Continue reading →

WorldEdit selection API on Bukkit

WorldEdit provides an easy-to-use API for Bukkit plugins to access the player’s current region selection.

A large part of WorldEdit uses an abstract internal API for easy portability  For your convenience, we provide a Bukkit-native selection API that makes it easier to work with that part of WorldEdit. It is possible to “reach through” to access to the internal features of WorldEdit, but if you are interested in only getting selections, that shouldn’t be needed.

Getting a selection

To get a selection, you need to:

  1. Get a reference to WorldEditPlugin from Bukkit’s plugin manager.
  2. Cast the reference from JavaPlugin to a more specific WorldEditPlugin.
  3. Get the selection by calling getSelection().

See the snippet below.

Continue reading →

Understanding the “game clock”

Understanding how your server ticks underneath can be helpful when diagnosing problems regarding slow performance. We’ll cover several topics.

  1. An overview of the game logic loop
  2. How this all applies to modded servers
  3. An explanation of transient problems
  4. A guide to profiling your server
  5. A macro-view of improving performance

The game logic loop

The most important part of Minecraft is the game loop. In there is a clock keeping everything synchronized, going tick-tock at an ideally constant rate: twenty times per second, with each tick or tock lasting hopefully at most 50 milliseconds. During each cycle, game elements have to do their work — skeletons have to figure out where to walk, minecarts have to travel forward, grass has to spread, and light problems have to be checked. That’s a lengthy to-do list, considering how many blocks and entities are on a loaded world at at time (thousands). This tick-tock interval is called the tick rate or “TPS,” and you can get some mod or plugin to give you your server’s rate.

vanilla

Quite a few things are on the to-do list, but as far as the “main game elements” go, those items are blocks, entities, and tile entities. Blocks include simple things such as sandstone, cobble, sugar cane, and things of that nature. Entities are your standard animals and monsters, in addition to paintings, item drops, and “objects that don’t fit perfectly on a grid.” Tile entities are a special kind of block, as they can hold any type of data (think storing an inventory, something sugar cane doesn’t have to do), and sometimes they must think separately. As previously mentioned, each loop has a budget of 50ms and each portion consumes some part of this pie. Any leftover time is considered idle.

Continue reading →

Session theft with a MITM in MC 1.2

Update: The man-in-the-middle attack described in this post is now ineffective as it was fixed in Minecraft 1.3, but the solution still works if you desire extra security.

A “session stealer” exploit in Minecraft is now being used increasingly widely to gain operator privileges on a target server. Anyone with the ability (or the freely available tools) can impersonate another user, which comes in handy to gain elevated privileges on a given server. The underlying exploit is actually extremely old, and while a small number of people have long been aware of it, news broke to the greater community recently and now it is a real relevant concern. On the flip side, the bug will finally be fixed (after three years) with the release of Minecraft 1.3. Hooray public disclosure.

exploit_logic

But I don’t want to wait for a fix

If you run a server, it’s understandable that you would want a fix now. You can ensure that you don’t expose your own account, but you also have to ensure that your own moderators do not either. That can be harder than it sounds, especially depending on who you are working with, and so some — even minor — protection to use would be nice. Unfortunately, it is not possible to fix the problem from just the server’s side, and so people have attempted to work around the problem by using a mix of some rather inconvenient solutions (mostly involving a /login command). Instead, I decided on a different novel way to protect the accounts of my moderators, described below.

Continue reading →

Security vulnerability allowing account spoofing

There’s this interesting exploit in Minecraft that lets you login under someone’s name without ever needing to know the person’s password. All the attacker needs to do is get you to join his/her server once. This client-side fix patches your game so that it won’t let your server tell you to authenticate against a “blank” server ID. Lymia and I reported it to Mojang a while ago, and while Jeb just put a fix in 1.8, there’s a mistake with the fix. You can download a ZIP to install it like any other mod (put the files into minecraft.jar), or Windows users can use the setup program to automatically install the fix:

A server-side fix has recently made it into a Bukkit, but your account can still be abused to join unpatched servers. This client-side patch prevents any server from exploiting your account. To understand how the exploit works, here’s a review of how Minecraft would authenticate for “Frank”:

 

  1. Client->Server: Your game tells the server that it wants to join as “Frank”
  2. Server->Client: The server responds with the ID “afe93b31c” (randomized)
  3. Your game tells Minecraft.net that “Frank” is joining “afe93b31c”
  4. Client->Server: The client tells the server that it’s ready
  5. The server asks Minecraft.net to see if “Frank” has joined “afe93b31c” (if not, then the real Frank never joined the server)

Here’s how the exploit works: You get your victim to join your custom server that sends a blank ID. Frank’s game tells Minecraft.net that Frank is joining “” (a blank ID). Frank joins your server, plays around, and has no clue about what’s going on. You then join Frank’s server where Frank is an administrator, but you immediately skip to step #4: You tell the server that you already did step #3 and you get in. Why does this work? On the server, the server ID is blank if you never complete step 1, so in step #5, Frank’s server asks Minecraft.net if “Frank” has joined “”, which Frank did earlier but on your server.

From a more technical perspective, the hacked client never sends the initial handshake packet. The server ID starts out as a blank string, and it’s only initialized if you send the handshake packet. If you skip the handshake packet and just jump to the login packet, then the server ID stays as a blank string. Before 1.8, your client would freely accept a blank server ID. 1.8 prevents a server from giving you a blank server ID, but Lymia noticed that the game doesn’t URL-escape the server ID, so a server ID of “&” is essentially blank. The patch makes the routine URL-escapes the ID.

Addendum: Someone mentioned a “man-in-the-middle” attack that allows for name spoofing, but that’s a different issue that can only be fixed with significant changes to how Minecraft handles authentication. (A MITM is also more difficult to pull off well.)

Introduction to PHP streams

Streams is a feature of PHP that was introduced in PHP 4.3 to unify the methods for working on files, sockets, and other similar resources. PHP 4.3 came out a long time ago, but many PHP developers never learned how to properly use streams in PHP, much to my dismay. Many use cURL for accessing HTTP resources, but I’m not a huge fan of cURL, because it has an awful interface in PHP and it presents yet another dependency. While the HTTP stream handler in PHP isn’t perfect, it works very well for most situations.

To begin, let’s consider what examples of streams could be:

  • A file on a hard drive
  • A HTTP connection to a website
  • A UDP connection to a server
  • A ZIP file
  • A SSH tunnel
  • A Vorbis/OGG file

What are some common operations on all of those? Primarily, they share the ability to be read from and written to. The power of PHP’s streams is that you can access all of them using the same set of functions, and if there’s something you wish to “stream-ify,” you can write your own “stream wrapper.” In addition to reading and writing, the streams framework in PHP also allows for other operations, such as renaming and deleting.

Continue reading →

Resizing with Avisynth and keeping aspect ratio

I had some videos I needed to work in batch, but I needed to resize them and keep their aspect ratio. AviSynth is a great video post-production utility, but its resize functions won’t add black bars. I’ve written the following AviSynth script to fix that. The following functions will also try to use an even width/height if the format requires it.

It is released to the public domain.

function BilinearResizeBars(clip vid, int target_width, int target_height) {
    orig_width = vid.width
    orig_height = vid.height
    return (target_width / target_height < orig_width / orig_height) ? \
        vid.BilinearResizeBarsWidth(target_width, target_height) : \
        vid.BilinearResizeBarsHeight(target_width, target_height)
}

function BilinearResizeBarsWidth(clip vid, int target_width, int target_height) {
    orig_width = vid.width
    orig_height = vid.height
    try {
        new_height = int(target_width * float(orig_height) / orig_width)
        vid = vid.BilinearResize(target_width, new_height) \
            .AddBorders(0, ceil((target_height - new_height) / 2), 0, \
                floor((target_height - new_height) / 2))
    } catch (error) { # Assuming that height must be even
        new_height = int(target_width * float(orig_height) / orig_width)
        vid = vid.BilinearResize(target_width, new_height - 1) \
            .AddBorders(0, ceil((target_height - new_height) / 2), 0, \
                floor((target_height - new_height) / 2) + 1)
    }
    return vid
}

function BilinearResizeBarsHeight(clip vid, int target_width, int target_height) {
    orig_width = vid.width
    orig_height = vid.height
    try {
        new_width = int(target_height * float(orig_width) / orig_height)
        vid = vid.BilinearResize(new_width, target_height) \
            .AddBorders(ceil((target_width - new_width) / 2), 0, \
                floor((target_width - new_width) / 2), 0)
    } catch (error) { # Assuming that width must be even
        new_width = new_width
        vid = vid.BilinearResize(new_width - 1, target_height) \
            .AddBorders(ceil((target_width - new_width) / 2) + 1, 0, \
                floor((target_width - new_width) / 2), 0)
    }
    return vid
}