InBroadcast - April 2022 - 70

70
www.inbroadcast.com | Vol: 12 - Issue 4 | April 2022
InFile
Ancillary Data and Compression
- Details that Matter!
By Ciro A. Noronha, Ph.D., Cobalt Digital Inc.
this
Introduction
Video compression is cool. You see
those great-looking 4K/HDR images
with a lot of detail, and are amazed
to find out that you can get them at
very reasonable bit rates. The audio
people also do a great job - you have
surround, immersive audio that
satisfies even the most demanding
listener. All of that in return for very
reasonable bit rates, much less than
the video. What is not to love? We
are done, right?
Not so fast - there is yet another
data type: data itself! Or, more
specifically, " ancillary data " , which
is a catch-all name for everything
else that
is
video, but still needs to be there.
The most
type is Closed Captioning, which
is a legal
places.
other important
common ancillary data
requirement in many
However, there are many
ancillary
data
types around, and a well-rounded
compression system needs to do as
good a job with them as it does with
audio and video.
For video and audio, a few welldefined
standards specify how you
handle the media.
1. How is
this
ancillary
ancillary
data
represented in the baseband signal?
2. How is
data
represented in the compressed
signal?
A Little Bit Of History
Let us start some history, going
back to the analog video days - the
good old composite signal.
Since
analog video was, well, analog, data
had to be mapped into this analog
signal. One of the first
types of
neither audio nor
data was Closed-Captioning (CC),
and that went into line 21 in the
Vertical Blanking Interval (VBI). The
data was converted to an analog
waveform, and transmitted with the
video (like a modem). Other types of
data received the same treatment -
examples are Wide Screen Signaling
(WSS) and Teletext in Europe.
Mapping this
compressed
data into the
stream had a rocky
start. There were no standards, and
people started solving the problem
in different ways. For example, for
line 21 CC, folks figured out that the
waveform survived
compression,
and some systems simply encoded
that line as part of the video.
The
You compress
the content, put it in PES packets,
map the PES into a transport stream,
and off you go. For ancillary data,
however, there is a maze of more or
less disjointed standards here and
there, and a bit of detective work
is needed to bring it all
together.
However, for any type of ancillary
data, everything boils down to the
following two questions:
decoder in the other side just
decoded and put it back, and CC
actually worked! This was, of course,
very wasteful of bits, because CC
was only carrying 4 bytes per frame
(about 960 bits/second). It would
be a lot more efficient to stick these
4 bytes somewhere in the bitstream
(since it was carrying bytes already).
The question then became " where
in the bitstream does it go " , keeping
in mind that ideally CC should go
together with the frame it belonged.
Well, it is not like that anymore.
Organizations such as ETSI, ANSI,
ATSC, SMPTE and others stepped
in and defined appropriate formats
in the compressed bitstream.
The
MPEG folks created the concept of
registered
and
unregistered
user
data in H.264 and later, so one could
identify the format of the field.
What about the baseband video?
It went digital too, with SDI. It was
no longer necessary
to represent
data in waveforms - you are sending
bytes, just put the data in the bytes!
That is SMPTE ST 291M. Ancillary
data lives in neat little packets in the
video; these packets are identified
by two 8-bit codes, a DID (Data ID)
and an SDID (Secondary Data ID).
This allows for 65,536 different data
types, and SMPTE keeps a registry
for that.
This answers our first
question, namely " how is ancillary
data represented in the baseband
signal? " The answer is ST 291M
packets. What is inside the packets
varies with the ancillary data type,
but at the baseband signal level, they
all look the same!
Ancillary Data In The
Compressed Signal
The second part of our discussion is
how the ancillary data is represented
in the compressed signal.
It comes
down to two options:
1. The ancillary data is included
Cobalt Digital's
9992-ENC/DEC
inside the user data of the video
elementary
stream, using the
https://www.cobaltdigital.com
existing registration mechanisms.
One advantage of this method is
that the synchronization of the data
with the video is implicit, because
the
ancillary data
goes
Both methods are used in practice,
for different types of data.
Closed-Captioning, Active Format
Descriptor (AFD), and High Dynamic
Range (HDR) metadata (ST 2108) are
ancillary data types that go into the
video elementary stream. The latter
two types
require frame-accurate
synchronization with the video.
SCTE-35 ad insertion markers and
OP-47 Subtitles travel as a separate
PID in the transport stream. They use
different
SCTE-35 markers
synchronization methods.
are carried
as
private table sections, which, by
themselves have no synchronization
with the video.
markers
However, the ad
refer to a frame in the
future by its PTS, taking care of the
synchronization aspect. OP-47
Subtitles use the PES format, which
includes a PTS, and that
The MPEG-2 folks, very helpfully,
had put in the space for " user data "
at every frame. Just an unformatted
block of bytes
arbitrary size.
of more or less
This was the ideal
place to put in the ancillary data,
like CC. However, since MPEG-2 did
not define the format of this user
area, everybody went their own way,
with a whole bunch of proprietary
implementations, which did not
necessarily even interoperate.
with the associated frame of video.
Another advantage is low overhead.
The disadvantage is that the data is
buried in the video and some parsing
is need to get to it.
Or
2. If you are generating a Transport
Stream, the ancillary data can go on
its own PID. This makes it a lot easier
to access and demultiplex, but now
need to synchronize it with the video.
You also need to announce the data
in the PMT.
allows
synchronization with the video - this
is the same mechanism used for A/V
synchronization.
our " get out of jail
Finally, we get to SMPTE ST 2038,
free card " .
transmit the
If
there is no defined standard for
some ancillary data packet you want
to transmit, you just
packet as is; that is ST 2038. It does
not care what is in the packet or
what its DID/SDID is; it just delivers
the packet to the other side. Like
OP-47, ST 2038 uses PES packets for
synchronization.
The Cobalt 9992-ENC/DEC series of
encoders/decoders support
all the
together
ancillary data types described in this
article.
http://www.inbroadcast.com https://www.cobaltdigital.com https://www.cobaltdigital.com https://www.cobaltdigital.com https://www.cobaltdigital.com https://www.cobaltdigital.com https://www.cobaltdigital.com

InBroadcast - April 2022

Table of Contents for the Digital Edition of InBroadcast - April 2022

Contents
InBroadcast - April 2022 - Intro
InBroadcast - April 2022 - Cover1
InBroadcast - April 2022 - Cover2
InBroadcast - April 2022 - Contents
InBroadcast - April 2022 - 4
InBroadcast - April 2022 - 5
InBroadcast - April 2022 - 6
InBroadcast - April 2022 - 7
InBroadcast - April 2022 - 8
InBroadcast - April 2022 - 9
InBroadcast - April 2022 - 10
InBroadcast - April 2022 - 11
InBroadcast - April 2022 - 12
InBroadcast - April 2022 - 13
InBroadcast - April 2022 - 14
InBroadcast - April 2022 - 15
InBroadcast - April 2022 - 16
InBroadcast - April 2022 - 17
InBroadcast - April 2022 - 18
InBroadcast - April 2022 - 19
InBroadcast - April 2022 - 20
InBroadcast - April 2022 - 21
InBroadcast - April 2022 - 22
InBroadcast - April 2022 - 23
InBroadcast - April 2022 - 24
InBroadcast - April 2022 - 25
InBroadcast - April 2022 - 26
InBroadcast - April 2022 - 27
InBroadcast - April 2022 - 28
InBroadcast - April 2022 - 29
InBroadcast - April 2022 - 30
InBroadcast - April 2022 - 31
InBroadcast - April 2022 - 32
InBroadcast - April 2022 - 33
InBroadcast - April 2022 - 34
InBroadcast - April 2022 - 35
InBroadcast - April 2022 - 36
InBroadcast - April 2022 - 37
InBroadcast - April 2022 - 38
InBroadcast - April 2022 - 39
InBroadcast - April 2022 - 40
InBroadcast - April 2022 - 41
InBroadcast - April 2022 - 42
InBroadcast - April 2022 - 43
InBroadcast - April 2022 - 44
InBroadcast - April 2022 - 45
InBroadcast - April 2022 - 46
InBroadcast - April 2022 - 47
InBroadcast - April 2022 - 48
InBroadcast - April 2022 - 49
InBroadcast - April 2022 - 50
InBroadcast - April 2022 - 51
InBroadcast - April 2022 - 52
InBroadcast - April 2022 - 53
InBroadcast - April 2022 - 54
InBroadcast - April 2022 - 55
InBroadcast - April 2022 - 56
InBroadcast - April 2022 - 57
InBroadcast - April 2022 - 58
InBroadcast - April 2022 - 59
InBroadcast - April 2022 - 60
InBroadcast - April 2022 - 61
InBroadcast - April 2022 - 62
InBroadcast - April 2022 - 63
InBroadcast - April 2022 - 64
InBroadcast - April 2022 - 65
InBroadcast - April 2022 - 66
InBroadcast - April 2022 - 67
InBroadcast - April 2022 - 68
InBroadcast - April 2022 - 69
InBroadcast - April 2022 - 70
InBroadcast - April 2022 - 71
InBroadcast - April 2022 - 72
InBroadcast - April 2022 - 73
InBroadcast - April 2022 - 74
InBroadcast - April 2022 - 75
InBroadcast - April 2022 - 76
InBroadcast - April 2022 - 77
InBroadcast - April 2022 - 78
InBroadcast - April 2022 - 79
InBroadcast - April 2022 - 80
InBroadcast - April 2022 - 81
InBroadcast - April 2022 - 82
InBroadcast - April 2022 - 83
InBroadcast - April 2022 - 84
InBroadcast - April 2022 - 85
InBroadcast - April 2022 - 86
InBroadcast - April 2022 - 87
InBroadcast - April 2022 - 88
InBroadcast - April 2022 - 89
InBroadcast - April 2022 - 90
InBroadcast - April 2022 - 91
InBroadcast - April 2022 - 92
InBroadcast - April 2022 - 93
InBroadcast - April 2022 - 94
InBroadcast - April 2022 - 95
InBroadcast - April 2022 - 96
InBroadcast - April 2022 - 97
InBroadcast - April 2022 - 98
InBroadcast - April 2022 - 99
InBroadcast - April 2022 - 100
InBroadcast - April 2022 - 101
InBroadcast - April 2022 - Cover3
InBroadcast - April 2022 - Cover4
https://europe.nxtbook.com/nxteu/lesommet/inbroadcast_202205
https://europe.nxtbook.com/nxteu/lesommet/inbroadcast_202204
https://europe.nxtbook.com/nxteu/lesommet/inbroadcast_202203
https://europe.nxtbook.com/nxteu/lesommet/inbroadcast_202202
https://europe.nxtbook.com/nxteu/lesommet/inbroadcast_202201
https://europe.nxtbook.com/nxteu/lesommet/inbroadcast_202112
https://europe.nxtbook.com/nxteu/lesommet/inbroadcast_202111
https://europe.nxtbook.com/nxteu/lesommet/inbroadcast_202110
https://europe.nxtbook.com/nxteu/lesommet/inbroadcast_202109
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_202108
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_202107
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_202106
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_202105
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_PTZ
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_202104
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_202103
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_202102
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_202101
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_202012
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_202011
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_202010
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_202009
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_202008
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_202007
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_202006
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_202005
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_202004
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_202003
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_202002
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_202001
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_INREVIEW2019
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201912
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201911
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_NewYork2019
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201910
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201909
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_inshowguide2019
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201908
http://europe.nxtbook.com/nxteu/lesommet/CTA-DAY3
http://europe.nxtbook.com/nxteu/lesommet/CTA-DAY2
http://europe.nxtbook.com/nxteu/lesommet/CTA-DAY1
http://europe.nxtbook.com/nxteu/lesommet/CTA-VISITOR
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201906
http://europe.nxtbook.com/nxteu/lesommet/CTA-PREVIEW
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201905
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201904
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_NABSHOW2019
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201903
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_cabsat2019
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201902
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201901
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_INREVIEW2018
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201812
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201811
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_NewYork2018
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201810
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_inshowguide2018
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201809
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201808
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201806
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201805
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201804
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_NABSHOW2018
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201803
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201802
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_cabsat2018
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201801
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201712
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201711
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201710
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_NABSHOWNYexpo2017
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201709
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_inshowguide2017
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201708
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201707
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201705
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_NABSHOW2017
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201704
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_cabsat2017
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201703
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201702
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201701
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_INREVIEW2016
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201612
https://www.nxtbook.com/nxteu/lesommet/inbroadcast_201611
https://www.nxtbook.com/nxteu/lesommet/inbroadcast_NABSHOWNYexpo
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_20110708_DEMO
https://www.nxtbook.com/nxteu/lesommet/inbroadcast_201610
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_inshowguide2016
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201609
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201608
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201607
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201606
http://europe.nxtbook.com/nxtbooks/lesommet/inbroadcast_APAC03
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201605
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_NABSHOW2016
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201604
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201603
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_cabsat2016
https://www.nxtbook.com/nxteu/lesommet/inbroadcast_201602
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201601
https://www.nxtbook.com/nxtbooks/lesommet/inbroadcast_APAC02
https://www.nxtbook.com/nxteu/lesommet/inbroadcast_201512
https://www.nxtbook.com/nxteu/lesommet/inbroadcast_201511
https://www.nxtbook.com/nxteu/lesommet/inbroadcast_CCW15expo
https://www.nxtbook.com/nxteu/lesommet/inbroadcast_201510
https://www.nxtbook.com/nxtbooks/lesommet/inbroadcast_APAC01
https://www.nxtbook.com/nxteu/lesommet/inbroadcast_201509
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_IBC2015
https://www.nxtbook.com/nxteu/lesommet/inbroadcast_201508
https://www.nxtbook.com/nxteu/lesommet/inbroadcast_201507
https://www.nxtbook.com/nxteu/lesommet/inbroadcast_201506
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_BROADCASTASIA2015
https://www.nxtbook.com/nxteu/lesommet/inbroadcast_201505
https://www.nxtbook.com/nxteu/lesommet/inbroadcast_201504
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_NABSHOW2015
https://www.nxtbook.com/nxteu/lesommet/inbroadcast_201503
https://www.nxtbook.com/nxteu/lesommet/inbroadcast_cabsat2015
https://www.nxtbook.com/nxteu/lesommet/inbroadcast_201502
https://www.nxtbook.com/nxteu/lesommet/inbroadcast_201501
https://www.nxtbook.com/nxteu/lesommet/inbroadcast_201412
https://www.nxtbook.com/nxteu/lesommet/inbroadcast_201411
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201410
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201409
http://europe.nxtbook.com/nxteu/lesommet/inbroadcastIBCPRINTEDGUIDE_201409
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201408
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201407
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201406
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201405
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201404
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201403
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201402
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201401
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201312
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201311
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201310
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201309
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201308
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201307
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201306
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201305
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201304
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201303
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201302
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201301
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201211
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201210
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201209
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201208
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201207
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201206
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201205
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201204
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201203
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_201202
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_20121201
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_20111011
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_20110708
http://europe.nxtbook.com/nxteu/lesommet/inbroadcast_20110506
https://www.nxtbookmedia.com