HLS video streaming from scratch with FFmpeg and Java
Turn one MP4 into an adaptive, encrypted, CDN-friendly stream. Bitrate ladders, segment sizes, ABR, AES-128 and a Java pipeline that runs it all, with charts at every step.
· 8 min read · Tanmay Singh
Put an .mp4 behind a <video> tag and it works... until someone watches on a train. The player
picks one file at one quality. If the network drops, it stalls. If the viewer leaves after 30 seconds,
you've still paid to send them several minutes of buffered video.
HTTP Live Streaming (HLS) fixes this with a surprisingly simple idea: cut the video into small files, encode them at several qualities, and let the player choose, segment by segment. Every request is plain HTTP, so any CDN can cache it.
This post builds the whole thing: the FFmpeg commands, the reasoning behind each number, and a Java pipeline that drives it all. It's the design behind my TranscodeX SDK.
1. How HLS works
An HLS stream is three kinds of files:
The master playlist advertises the available qualities (renditions):
#EXTM3U
#EXT-X-VERSION:7
#EXT-X-STREAM-INF:BANDWIDTH=5478000,RESOLUTION=1920x1080,CODECS="avc1.640028,mp4a.40.2"
1080p/playlist.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=3124000,RESOLUTION=1280x720,CODECS="avc1.64001f,mp4a.40.2"
720p/playlist.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=1594000,RESOLUTION=854x480,CODECS="avc1.64001e,mp4a.40.2"
480p/playlist.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=952000,RESOLUTION=640x360,CODECS="avc1.64001e,mp4a.40.2"
360p/playlist.m3u8
Each media playlist lists the segments for one quality:
#EXTM3U
#EXT-X-VERSION:7
#EXT-X-TARGETDURATION:6
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-MAP:URI="init.mp4"
#EXTINF:6.000,
seg_000.m4s
#EXTINF:6.000,
seg_001.m4s
#EXTINF:4.320,
seg_002.m4s
#EXT-X-ENDLIST
The player then repeats one simple loop:
All the intelligence lives in the player. The server just serves static files, which is why HLS scales so cheaply.
2. Designing the bitrate ladder
The ladder is the set of renditions you encode. Here's a solid default for 16:9, 30 fps H.264:
| Rendition | Resolution | Video bitrate | Max rate (×1.07) | Buffer (×1.5) | Audio |
|---|---|---|---|---|---|
| 1080p | 1920×1080 | 5000 kbps | 5350 kbps | 7500 kbps | 128 kbps AAC |
| 720p | 1280×720 | 2800 kbps | 2996 kbps | 4200 kbps | 128 kbps AAC |
| 480p | 854×480 | 1400 kbps | 1498 kbps | 2100 kbps | 96 kbps AAC |
| 360p | 640×360 | 800 kbps | 856 kbps | 1200 kbps | 96 kbps AAC |
Each step roughly doubles the bitrate. That spacing gives the player useful choices without switching so often that the viewer notices.
Why not just "more bitrate = better"?
Quality rises fast at first, then flattens out. Each resolution also has a range where it's the best choice: at low bitrates, a smaller picture encoded cleanly looks better than a large one full of compression blocks.
Read the chart from left to right. Below ~1000 kbps the 360p curve is on top, then 720p takes over, and 1080p only wins above roughly 3000 kbps. The ladder should follow whichever curve is on top (that top edge is called the convex hull).
3. The FFmpeg command, line by line
One FFmpeg process can decode once and encode all four renditions:
ffmpeg -i input.mp4 \
-filter_complex "[0:v]split=4[a][b][c][d]; \
[a]scale=-2:1080[v1080]; [b]scale=-2:720[v720]; \
[c]scale=-2:480[v480]; [d]scale=-2:360[v360]" \
-map "[v1080]" -c:v:0 libx264 -b:v:0 5000k -maxrate:v:0 5350k -bufsize:v:0 7500k \
-map "[v720]" -c:v:1 libx264 -b:v:1 2800k -maxrate:v:1 2996k -bufsize:v:1 4200k \
-map "[v480]" -c:v:2 libx264 -b:v:2 1400k -maxrate:v:2 1498k -bufsize:v:2 2100k \
-map "[v360]" -c:v:3 libx264 -b:v:3 800k -maxrate:v:3 856k -bufsize:v:3 1200k \
-map a:0 -map a:0 -map a:0 -map a:0 -c:a aac -ac 2 \
-b:a:0 128k -b:a:1 128k -b:a:2 96k -b:a:3 96k \
-preset veryfast -profile:v high \
-g 60 -keyint_min 60 -sc_threshold 0 \
-f hls -hls_time 6 -hls_playlist_type vod \
-hls_segment_type fmp4 -hls_flags independent_segments \
-hls_segment_filename "out/%v/seg_%03d.m4s" \
-master_pl_name master.m3u8 \
-var_stream_map "v:0,a:0,name:1080p v:1,a:1,name:720p v:2,a:2,name:480p v:3,a:3,name:360p" \
"out/%v/playlist.m3u8"
The flags that matter most:
| Flag | What it does | Why |
|---|---|---|
split=4 + scale=-2:H |
Decode once, scale four ways | Decoding is expensive; -2 keeps the aspect ratio with an even width |
-maxrate / -bufsize |
Caps short bitrate spikes | Without a cap, an explosion scene can be 3× the average and stall a viewer on a weak connection |
-g 60 -keyint_min 60 |
A keyframe every 60 frames = 2 s at 30 fps | Segments must start with a keyframe, so 6 s segments get clean cuts |
-sc_threshold 0 |
No extra keyframes on scene changes | Keeps keyframes aligned across all renditions, so switching quality is seamless |
-hls_segment_type fmp4 |
Fragmented MP4 instead of MPEG-TS | Less overhead, and the same files work for DASH and modern codecs |
-hls_time 6 |
Target segment length | The big trade-off, covered next |
4. How long should a segment be?
Shorter segments mean faster starts and quicker quality switches, but many more HTTP requests and playlist entries.
The request count is exact arithmetic, for one viewer watching one hour:
| Segment length | Segment requests / hour | Requests for 100k concurrent viewers |
|---|---|---|
| 2 s | 1,800 | 50,000 req/s |
| 4 s | 900 | 25,000 req/s |
| 6 s | 600 | ~16,700 req/s |
| 10 s | 360 | 10,000 req/s |
6 seconds is a good default for on-demand video and is Apple's own recommendation. Go down to 2 s, or use Low-Latency HLS with partial segments, only for live streams where a few seconds of delay matter.
5. What it costs to store
Storage is just bitrate × duration. For one hour of video with the ladder above:
1080p takes about half the space, but on mobile it's watched far less than half the time. That's a strong reason to try per-title encoding, or a better codec (HEVC and AV1 reach similar quality at roughly 30–50% less bitrate, at the cost of slower encodes and patchier device support).
6. Speed: presets and parallelism
x264 presets trade encoding time for compression efficiency:
veryfast is a good choice for a self-hosted pipeline. Going from veryfast to slow usually saves
around 10–15% in file size, for 4–5× the CPU time. That's only worth it when the same file is
watched millions of times.
Run the renditions in parallel
One FFmpeg process with split is simple, but it's limited by the slowest rendition. On a machine with many
cores you can run one process per rendition at the same time:
Each rendition runs a little slower in parallel because they share the CPU, but the total time drops from the sum of all renditions to the longest one: about 190 s down to about 110 s here. To go further, chunked encoding splits the source at keyframes into, say, 30 s pieces, encodes them on many machines and joins the segment lists. This is how big platforms turn a 2-hour film around in minutes.
7. Encrypting segments with AES-128
HLS has encryption built in. Every segment is encrypted, and the playlist tells the player where to fetch the key:
openssl rand 16 > enc.key # 128-bit content key
openssl rand -hex 16 # optional IV
cat > keyinfo.txt <<EOF
https://api.example.com/keys/video-42
/secure/keys/video-42/enc.key
EOF
ffmpeg ... -hls_key_info_file keyinfo.txt ...
FFmpeg writes this into every media playlist:
#EXT-X-KEY:METHOD=AES-128,URI="https://api.example.com/keys/video-42"
The segments can sit on a public CDN. Only the key endpoint needs protecting:
@RestController
@RequestMapping("/keys")
class KeyController {
private final KeyStore keys;
private final Entitlements entitlements;
@GetMapping(value = "/{videoId}", produces = MediaType.APPLICATION_OCTET_STREAM_VALUE)
ResponseEntity<byte[]> key(@PathVariable String videoId, @AuthenticationPrincipal Jwt user) {
if (!entitlements.canWatch(user.getSubject(), videoId)) {
return ResponseEntity.status(HttpStatus.FORBIDDEN).build();
}
return ResponseEntity.ok()
.cacheControl(CacheControl.noStore()) // never cache keys at the CDN
.body(keys.contentKey(videoId));
}
}
8. The Java pipeline
The orchestration is a small state machine, one job per uploaded video:
Step 1: probe the source
Never trust an upload's file extension. Ask ffprobe what's really inside:
record SourceInfo(double durationSec, int width, int height, double fps, boolean hasAudio) {}
SourceInfo probe(Path input) throws IOException, InterruptedException {
Process p = new ProcessBuilder(
"ffprobe", "-v", "error", "-print_format", "json",
"-show_format", "-show_streams", input.toString())
.redirectErrorStream(true)
.start();
JsonNode root = mapper.readTree(p.getInputStream());
if (p.waitFor() != 0) throw new InvalidMediaException(input);
JsonNode video = stream(root, "video").orElseThrow(() -> new InvalidMediaException(input));
return new SourceInfo(
root.at("/format/duration").asDouble(),
video.get("width").asInt(),
video.get("height").asInt(),
parseRate(video.get("avg_frame_rate").asText()), // "30000/1001" → 29.97
stream(root, "audio").isPresent());
}
Use the result to trim the ladder. Never upscale: a 720p upload shouldn't get a fake 1080p rendition.
List<Rendition> ladderFor(SourceInfo src) {
return DEFAULT_LADDER.stream()
.filter(r -> r.height() <= src.height())
.toList();
}
Step 2: encode with live progress
Pass -progress pipe:1 and FFmpeg writes machine-readable key=value lines to stdout. That's
enough to drive a progress bar:
void encode(Job job, Rendition r, SourceInfo src, Consumer<Double> onProgress) throws Exception {
List<String> cmd = FfmpegCommand.hls(job.input(), job.outDir(), r, src)
.withArgs("-progress", "pipe:1", "-nostats")
.build();
Process p = new ProcessBuilder(cmd)
.redirectError(job.logFile(r).toFile()) // keep stderr for debugging
.start();
try (var lines = p.inputReader().lines()) {
lines.filter(l -> l.startsWith("out_time_us="))
.mapToLong(l -> Long.parseLong(l.substring(12)))
.forEach(us -> onProgress.accept(Math.min(1.0, us / 1e6 / src.durationSec())));
}
if (p.waitFor() != 0) {
throw new EncodeFailedException(job.id(), r.name(), job.logFile(r));
}
}
Step 3: run renditions in parallel with virtual threads
Each FFmpeg process does its heavy work in native code. The Java thread just waits on the process, so virtual threads are a perfect fit. A semaphore stops you starting more encodes than the CPU can handle:
private final Semaphore encodeSlots = new Semaphore(Runtime.getRuntime().availableProcessors() / 2);
void encodeAll(Job job, SourceInfo src) throws Exception {
List<Rendition> ladder = ladderFor(src);
Map<String, Double> progress = new ConcurrentHashMap<>();
try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
List<Future<?>> tasks = ladder.stream()
.map(r -> executor.submit(() -> {
encodeSlots.acquire();
try {
encode(job, r, src, pct -> {
progress.put(r.name(), pct);
job.reportProgress(average(progress, ladder.size()));
});
} finally {
encodeSlots.release();
}
return null;
}))
.toList();
for (Future<?> task : tasks) task.get(); // rethrows the first failure
}
writeMasterPlaylist(job, ladder);
}
With one process per rendition, the master playlist is written by the Java code instead of
FFmpeg. It's just a text file built from the ladder, and building it in Java lets you set accurate
BANDWIDTH values from the actual peak segment bitrates.
Step 4: publish with the right headers
| File | Content-Type | Cache-Control | Why |
|---|---|---|---|
*.m3u8 (VOD) |
application/vnd.apple.mpegurl |
public, max-age=300 |
Small; lets you re-publish a fix quickly |
*.m3u8 (live) |
application/vnd.apple.mpegurl |
max-age=1 |
Changes every segment |
init.mp4, *.m4s |
video/mp4 |
public, max-age=31536000, immutable |
Content never changes, so it can be cached forever |
| Key endpoint | application/octet-stream |
no-store |
Must be checked on every request |
9. Adaptive bitrate in action
Here's what a good player does during one minute on a commuter train. The chosen rendition stays a step below the measured bandwidth, so the buffer has room to absorb a sudden drop:
What's going on:
- Start low (0–6 s): with no measurements yet, start at 480p so the first frame appears quickly.
- Climb in steps (6–18 s): only switch up after the higher rendition has been sustainable for a while.
- Drop quickly (24–30 s): in the tunnel, fall straight to 360p. Stalling is far worse than a softer picture.
- Recover carefully (36–60 s): step back up one rung at a time to avoid bouncing between qualities.
On the web, hls.js does all of this for you. Safari plays HLS natively:
const video = document.querySelector('video');
const src = 'https://cdn.example.com/videos/42/master.m3u8';
if (Hls.isSupported()) {
const hls = new Hls({
capLevelToPlayerSize: true, // match quality to player size
xhrSetup: xhr => (xhr.withCredentials = true) // cookie for key API
});
hls.loadSource(src);
hls.attachMedia(video);
} else if (video.canPlayType('application/vnd.apple.mpegurl')) {
video.src = src; // Safari / iOS
}
10. The full picture
Checklist
- Probe every upload; never upscale
- Ladder spaced about 2× apart, with
maxrate/bufsizecaps - Fixed GOP aligned across renditions (
-g,-sc_threshold 0) - 6 s fMP4 segments for on-demand video
- Renditions in parallel, limited by a semaphore
- Segments cached
immutable; playlists cached briefly; keysno-store - AES-128 for access control; real DRM only if you actually need it
- Next: per-title ladders, HEVC/AV1 renditions and Low-Latency HLS
The complete pipeline, with Spring auto-configuration, thumbnails and the key endpoint, is in TranscodeX SDK. If you build something with it, or find a flag I've got wrong, tell me.