RP-NDI-SRCINFO_2020_08

proposal for carriage of technical media source information in NDI

This page contains a technical proposal which is not currently a published or formal standard. If you have comments or feedback on the content of this proposal PLEASE contact us with your contribution.

 

This proposal defines a standardised wrapper to carry source media information or technical genealogy over NDI by way of a recommended practice technical note.

 

Basic Premise:

When using NDI to carry a stream of content it may pass through stages of analysis, where important technical information is derived, or it may be generated in a form which benefits from including technical data about the contents.

For example an NDI Stream converted from a stream which originated in a long gop form, for example in an MPEGTS, a mechanism is required to preserve some source media characteristics when converting to NDI. For example it may be useful to know where the original media keyframes fell, as well as information about the interlaced / progressive nature of the original source. It may also be useful to understand the original media presentation timestamps - since they may related to other temporarlly referenced metdata like SCTE-35. This information can be used by downstream systems processing the NDI Streams.

 

Video-Frame Attached Example

In this proposal, the metadata is sent using the NDI per-frame metadata messages attached to discrete video frames which is already defined in the NDI Protocol.   A number of optional message tags can be housed in the outer <SRC_TECHNICAL_INFO> wrapper.

 

 

Example :

ndi_video_frame.p_metadata =

<SRC_TECHNICAL_INFO>

     <src_keyframe>1</src_keyframe>

     <src_interlaced>1</src_interlaced>

     <src_topfield>1</src_topfield>

     <src_pts>13453466568678</src_pts>

</SRC_TECHNICAL_INFO>

 

<src_keyframe> indicates whether this frame was a keyframe in the original long gop source

<src_interlaced> indicates whether the original source frame/field was part of an interlaced pair

<src_topfield> assuming interlaced, this indicates whether this NDI 'frame' is a top field of a pair. Note that NDI can indicate this automatically when sending field based NDI but in some cases systems convert fields from interlaced sources to either progressive NDI frames. An example would be behaviour of some h.265 encoders which convert 1080i50 to 540p50 - and this information may be useful when undoing that process and restoring a properly defined interlaced NDI stream.

<src_pts> indicates the original presentation timestamp for this video frame. This may be essential when interpreting things like SCTE-35 splice data.

 

 

Real Time Metadata Examples

The examples below are not specific to individual video frames and as such are passed in real time metadata of the NDI Stream.

 

 

An extension to this proposal would allow an NDI Stream to carry content source identifiers to establish genealogy of an NDI Stream.

p_metadata =

<SRC_TECHNICAL_INFO>

     <genealogy>

          <source>

               <source_identifier type="ISO">1</source_identifier>

          </source>

          <source>

               <source_identifier type="NDISource">NDISOURCEMULTI01 (CAM1)</source_identifier>

          </source>

         <source>

               <source_identifier type="UID">345345345345</source_identifier>

          </source>

         <source>

               <source_identifier type="FILE">/home/ubuntu/media/Tom_and_Jerry_season5_ep1.mp4</source_identifier>

          </source>

          <source>

               <source_identifier type="TRANSITION">DISSOLVE</source_identifier>

          </source>

        <source>

               <source_identifier type="TAG">Some Identifier</source_identifier>

          </source>

     </genealogy>

</SRC_TECHNICAL_INFO>

 

This extensible mechanism allows for each NDI video frame to state its genealogy, for tracking back to sources for rights management, or for regeneration of EDL events from vision mixers.

For example a playout server could easily insert the source filename into the stream for traffic as-run systems, or other rights management, or even downstream metadata systems.

 

Note that the NDI:iXML mechanism may provide more scope for very detailed source content communication.

 

A vision mixer could publish the sources as it cuts or dissolves such that a downstream system could capture the stream and create an NLE representation of the program tracking back to ISO feeds, so the stream could be re-cut. Where 2 sources overlap, a 3rd "Transition" source could provide a simple indication of how the sources share the stream, eg DISSOLVE, WIPE, BOX

 

Another extension to this proposal would allow an NDI Stream to carry content showing the health of the NDI Stream. It is anticipated that an upstream analyser could inject this data into a stream for processing later in the chain. For example a frozen image detector upstream could inject technical info to alarm this condition, and this would be acted upon by downstream systems such as a multiviewer (to alert the user) or a failover switch.

 

p_metadata =

<SRC_TECHNICAL_INFO>

     <health>

         <video>FROZEN</video>  [ OK, FROZEN, MISSING ]

         <audio>MUTE</audio>  [  OK, MUTE, OVERLOAD, MISSING, PHASE, NOISY  ]

   </health>

</SRC_TECHNICAL_INFO>

 

 

The example above means that the health information applies to the NDI Video stram which carries this metadata. However, a variation on the same message can be used on a pure metadata NDI Connection to carry information about  multiple external NDI Streams.

 

The example below may be emitted by a Multiviewer, to communicate the health of all its input sources.  A follow on module such as a 'penalty box' can use this information to decide whether to 'flag up' problems in the penalty box.  Also this can be fed to an alarm module to drive graphical alarms or other alerts.

 

p_metadata =

<SRC_TECHNICAL_INFO>

     <source>

          <source_identifier type="NDISource" name="CAMERA 1">NDISOURCEMULTI01 (CAM1)</source_identifier>

          <health>

               <video>FROZEN</video>  [ OK, FROZEN, MISSING ]

               <audio>MUTE</audio>  [  OK, MUTE, OVERLOAD, MISSING, PHASE, NOISY  ]

          </health>

     </source>

     <source>

          <source_identifier type="NDISource" name="CAMERA 2">NDISOURCEMULTI02 (CAM2)</source_identifier>

          <health>

               <video>FROZEN</video>  [ OK, FROZEN, MISSING ]

               <audio>MUTE</audio>  [  OK, MUTE, OVERLOAD, MISSING, PHASE, NOISY  ]

          </health>

      </source>

</SRC_TECHNICAL_INFO>

 

 

Another extension to the health data adds an <R128_Info> section to detail upstream audio loudness analysis and which can drive the 'OVERLOAD' parameter in audio health, this can also drive downstream metering. Data values are LUFS

 

p_metadata =

<SRC_TECHNICAL_INFO>

<health>

         <video>OK</video>

          <audio>OVERLOAD</audio>

         <R128_Info>

            <target>-24.0</target>  [ entered in analyser ]

            <momentary>-23.0</momentary>

            <short>-22.0</short>

            <integrated>-21.0</integrated>

            <range>2.0</range>

            <low>-23</low>

            <high>-21</high>

        </R128_Info>

</health>

</SRC_TECHNICAL_INFO>

 

If you have any questions, or you would like to engage Sienna for NDI Consultancy or Custom Development, please contact info @ sienna.tv