Some audio delivery formats look simple on the surface but have enough quirks in their container structure to cause real headaches. This is a deep dive into the m4a box format, segmented delivery, and the stitching logic we built to handle it correctly.
Audio file formats have always been more complicated than they look. The file extension tells you very little about the actual internal structure. A .m4a file is an MPEG-4 container, which is an extension of the QuickTime container format, which is a hierarchical box-based structure that can contain almost anything. The audio data itself is just one component inside that container, and understanding the container structure matters a lot when you want to do anything programmatic with the files.
We ran into this while building an internal media processing tool at Execross. Part of what the tool does is work with audio content that is delivered in a segmented format rather than as a single file. Segmented delivery is common for streaming applications because it allows playback to start before the entire file is downloaded, but it creates specific challenges when you need to work with the content as a single continuous piece.
This post is a technical walkthrough of the m4a container format, how segmented audio delivery works, and the stitching logic we built in Go to reassemble segments into coherent files. It is aimed at engineers building media processing tools who want to understand what is actually happening inside these files rather than treating them as black boxes.
What is actually inside an m4a file
An m4a file is structured as a hierarchy of boxes, sometimes called atoms in older documentation. Each box has a four-byte big-endian size field, followed by a four-byte ASCII type identifier, followed by the box contents. The size field includes the 8-byte header itself. Boxes can be nested inside other boxes, which is how the hierarchical structure is represented.
The boxes you care about most when working with m4a programmatically are these four:
ftyp is the file type box. It always appears at the start of the file and identifies it as a particular kind of MPEG-4 container. Checking for a valid ftyp is the minimum validation you should do on any file you receive.
moov is the movie box. It contains all the metadata about the audio content: codec parameters, duration, timing information, sample tables that describe the structure of the audio data. A decoder needs the moov box to make sense of any raw audio data.
mdat is the media data box. It contains the actual encoded audio samples. This is the bulk of the file by size in virtually all cases.
moof is the movie fragment box, used in fragmented m4a files. Segmented delivery typically uses fragmented m4a, where each segment has its own moof describing the samples in that segment.
The critical structural point for our purposes is that moov and mdat are separate boxes. The moov box is metadata, the mdat box is raw data. A decoder needs both, in the right relationship, to work correctly.
How segmented audio delivery works
In a segmented delivery model, the audio is split into short chunks, typically a few seconds each. Each chunk is a self-contained file that can be fetched independently over HTTP. A manifest file describes the segments: how many there are, where to fetch each one, and timing information.
This is similar in concept to HLS (HTTP Live Streaming) for video. The key difference for audio-only fragmented m4a is in the initialization segment. The first segment, sometimes called the initialization segment or just the header, contains the moov box with codec and timing information for the entire track. All subsequent segments contain moof and mdat boxes with the actual audio data for their time window.
This structure makes sense for streaming: the client fetches the initialization segment to configure the decoder, then streams subsequent segments as playback progresses. The decoder has everything it needs to decode each segment because the initialization segment provided the codec context.
The challenge for processing is that you cannot just concatenate these segments into a single file. The resulting file would have a valid moov at the start, but each subsequent segment's moof and mdat would not be properly indexed in the moov's sample table. Most decoders would only play the first segment correctly.
Parsing the box structure in Go
Before we can do anything useful with the segments, we need to be able to parse the box structure. Go is well suited for binary parsing work because the standard library has good support for reading binary data, and the type system makes it easy to represent the parsed structure clearly.
`go
package audio
import (
"encoding/binary"
"fmt"
"io"
)
// Box represents a parsed MPEG-4 box (atom)
type Box struct {
Size uint32
Type string
Offset int
}
// DataOffset returns the byte offset where this box's
// data begins (after the 8-byte header)
func (b Box) DataOffset() int {
return b.Offset + 8
}
// DataSize returns the number of bytes of data in this box
func (b Box) DataSize() int {
return int(b.Size) - 8
}
// ParseBoxes reads the top-level box structure from a byte slice
func ParseBoxes(data []byte) ([]Box, error) {
var boxes []Box
offset := 0
for offset < len(data) {
if offset+8 > len(data) {
break
}
size := binary.BigEndian.Uint32(data[offset : offset+4])
if size < 8 {
return nil, fmt.Errorf("invalid box size %d at offset %d", size, offset)
}
if offset+int(size) > len(data) {
return nil, fmt.Errorf("box at offset %d extends beyond data", offset)
}
boxType := string(data[offset+4 : offset+8])
boxes = append(boxes, Box{
Size: size,
Type: boxType,
Offset: offset,
})
offset += int(size)
}
return boxes, nil
}
func FindBox(boxes []Box, boxType string) (Box, bool) {
for _, b := range boxes {
if b.Type == boxType {
return b, true
}
}
return Box{}, false
}
func HasBox(boxes []Box, boxType string) bool {
_, ok := FindBox(boxes, boxType)
return ok
}
`
The parsing code is straightforward. We read 8 bytes at a time to get the size and type of each box, validate the size, and step forward through the data. We do not recurse into nested boxes here because for our stitching use case we only need the top-level structure.
Stitching segments together
Now for the core problem. We have an initialization segment with a moov box, and we have one or more subsequent segments each with moof and mdat boxes. We want to combine them into a single valid m4a file.
The approach that works correctly is to output the initialization segment as-is (preserving its moov box), then for each subsequent segment, extract the mdat content and append it. We rebuild a minimal file that has the codec context from the initialization segment and the audio data from all segments in order.
`go
package audio
import (
"fmt"
)
// StitchResult holds the output and some stats about the stitch operation
type StitchResult struct {
Data []byte
SegmentCount int
TotalBytes int
}
// StitchSegments combines an initialization segment and one or more
// media segments into a single m4a file.
//
// The initialization segment must contain a moov box.
// Media segments should contain mdat boxes with audio data.
func StitchSegments(initSegment []byte, mediaSegments [][]byte) (*StitchResult, error) {
// Validate the initialization segment
initBoxes, err := ParseBoxes(initSegment)
if err != nil {
return nil, fmt.Errorf("parsing init segment: %w", err)
}
if !HasBox(initBoxes, "moov") {
return nil, fmt.Errorf("init segment missing moov box")
}
result := make([]byte, len(initSegment))
copy(result, initSegment)
for i, segment := range mediaSegments {
boxes, err := ParseBoxes(segment)
if err != nil {
return nil, fmt.Errorf("segment %d: parsing boxes: %w", i, err)
}
mdatBox, ok := FindBox(boxes, "mdat")
if !ok {
return nil, fmt.Errorf("segment %d: no mdat box found", i)
}
// Extract just the audio data from the mdat box (skip the 8-byte header)
start := mdatBox.DataOffset()
end := start + mdatBox.DataSize()
if end > len(segment) {
return nil, fmt.Errorf("segment %d: mdat extends beyond data", i)
}
result = append(result, segment[start:end]...)
}
return &StitchResult{
Data: result,
SegmentCount: len(mediaSegments) + 1,
TotalBytes: len(result),
}, nil
}
// ValidateM4A checks that a byte slice looks like a valid m4a file.
// This is a shallow check: it verifies box structure and required
// top-level boxes, but does not validate codec parameters.
func ValidateM4A(data []byte) error {
boxes, err := ParseBoxes(data)
if err != nil {
return fmt.Errorf("invalid box structure: %w", err)
}
required := []string{"ftyp", "moov", "mdat"}
for _, boxType := range required {
if !HasBox(boxes, boxType) {
return fmt.Errorf("missing required box: %s", boxType)
}
}
return nil
}
`
The key operation is the mdat extraction. We find the mdat box using our parser, calculate the offset where the actual audio data starts (skipping the 8-byte box header), and append just that data to our output buffer.
Fetching segments concurrently
For reasonable throughput, you want to fetch segments in parallel rather than sequentially. A three-minute audio track might have 36 segments at five seconds each. Sequential HTTP requests for all of them would be unnecessarily slow.
Go's concurrency model is well suited for this. We use a bounded goroutine pool to control the degree of parallelism, which keeps us from making too many simultaneous requests.
`go
package audio
import (
"fmt"
"io"
"net/http"
"sync"
"time"
)
// FetchResult holds the result of fetching one segment
type FetchResult struct {
Index int
Data []byte
Err error
}
// FetchSegments downloads a list of URLs concurrently using a bounded
// worker pool. Results are returned in the same order as the input URLs.
func FetchSegments(urls []string, maxWorkers int) ([][]byte, error) {
results := make([]FetchResult, len(urls))
jobs := make(chan int, len(urls))
var wg sync.WaitGroup
client := &http.Client{Timeout: 30 * time.Second}
for w := 0; w < maxWorkers; w++ {
wg.Add(1)
go func() {
defer wg.Done()
for idx := range jobs {
data, err := fetchWithBackoff(client, urls[idx], 3)
results[idx] = FetchResult{Index: idx, Data: data, Err: err}
}
}()
}
for i := range urls {
jobs <- i
}
close(jobs)
wg.Wait()
segments := make([][]byte, len(urls))
for _, r := range results {
if r.Err != nil {
return nil, fmt.Errorf("segment %d: %w", r.Index, r.Err)
}
segments[r.Index] = r.Data
}
return segments, nil
}
func fetchWithBackoff(client *http.Client, url string, maxRetries int) ([]byte, error) {
var lastErr error
backoff := 500 * time.Millisecond
for attempt := 0; attempt < maxRetries; attempt++ {
if attempt > 0 {
time.Sleep(backoff)
backoff *= 2
}
resp, err := client.Get(url)
if err != nil {
lastErr = err
continue
}
body, err := io.ReadAll(resp.Body)
resp.Body.Close()
if err != nil {
lastErr = err
continue
}
if resp.StatusCode == 429 || resp.StatusCode >= 500 {
lastErr = fmt.Errorf("HTTP %d", resp.StatusCode)
continue
}
if resp.StatusCode != 200 {
return nil, fmt.Errorf("HTTP %d (non-retryable)", resp.StatusCode)
}
return body, nil
}
return nil, fmt.Errorf("all retries exhausted: %w", lastErr)
}
`
The worker count is something you tune based on your situation. For audio segment fetching we typically use 3 to 5 workers. More workers means faster downloads but higher request rates, which increases the chance of triggering rate limits on the CDN.
Putting the pieces together
Here is how the full pipeline looks when you connect the fetcher and the stitcher:
`go
package main
import (
"fmt"
"os"
"your-module/audio"
)
type Manifest struct {
InitURL string
SegmentURLs []string
}
func processAudioTrack(manifest Manifest, outputPath string) error {
// Fetch the initialization segment first
initData, err := audio.FetchSegments([]string{manifest.InitURL}, 1)
if err != nil {
return fmt.Errorf("fetching init segment: %w", err)
}
// Fetch all media segments concurrently
segmentData, err := audio.FetchSegments(manifest.SegmentURLs, 4)
if err != nil {
return fmt.Errorf("fetching segments: %w", err)
}
// Stitch everything together
result, err := audio.StitchSegments(initData[0], segmentData)
if err != nil {
return fmt.Errorf("stitching: %w", err)
}
// Validate before writing
if err := audio.ValidateM4A(result.Data); err != nil {
return fmt.Errorf("validation failed: %w", err)
}
fmt.Printf("Stitched %d segments, %d bytes total
",
result.SegmentCount, result.TotalBytes)
return os.WriteFile(outputPath, result.Data, 0644)
}
`
Edge cases we ran into
The main thing we underestimated was variation in how different audio sources structure their segment manifests. Our initial implementation assumed fairly regular segment sizes and a simple init-then-media structure. In practice we encountered segments with unexpected box layouts, manifests where the initialization data was embedded inline rather than in a separate segment, and tracks with non-standard durations per segment.
If you are building something similar, build flexibility into the manifest parsing from the start. Do not assume a fixed segment count or uniform segment duration. Handle the case where initialization data might be in the first media segment rather than a dedicated init segment.
The ValidateM4A function has been one of the more useful things we added. Running it before writing output files caught several edge cases where the stitching produced structurally invalid output, which would have been much harder to diagnose downstream.
Performance in practice
For a typical three-minute audio track with 36 five-second segments, the full pipeline runs in about 2 to 4 seconds on a VPS with decent network connectivity. Most of that time is network fetching. The parsing and stitching operations are essentially instantaneous for files of this size.
Memory usage is linear in the total audio data size, which is expected. A three-minute m4a file at reasonable audio quality is typically 3 to 8 megabytes, well within comfortable limits for in-memory processing.
The code has been running in production for several months without any significant issues. The main ongoing maintenance has been adjusting the retry logic parameters as CDN behavior has varied.