If the increasing interest concerning digital watermarking during the last decade is most likely due to the increase in concern over copyright protection of digital content, it is also emphasised by its incredible commercial potential. The following section is consequently completely dedicated to the presentation of various applications in which digital watermarking can bring a valuable support in the context of video. Digital video watermarking may indeed be used in many various applications and some of them are far from the original copyright enforcement context. The applications presented in this section have been gathered in Table 42.2. This is not an exhaustive list and many applications are still to be imagined.
Purpose of the embedded watermark
Prevent unauthorised copying.
Identify the video item being broadcasted.
Trace back a malicious user.
Insure that the original content has not been altered.
Enhance video coding
Bring additional information e.g. for error correction.
The Digital Versatile Disk (DVD) and DVD players appeared on the consumer market in late 1996. This new technology was enthusiastically welcomed since DVD players provide a very high-quality video signal. However, the advantages of digital video are counterbalanced by an increased risk of illegal copying. In contrast to traditional VHS tape copying, each copy of digital video data is a perfect reproduction. This raised the concern of copyright owners and Hollywood studios requested that several levels of copy protection should be investigated before any device with digital video recording capabilities could be introduced.
The Copy Protection Technical Working Group (CPTWG) has consequently been created in order to work on copy protection issues in DVD. A standard has not been defined yet. However a system, which could become the future specification for DVD copy protection, has been defined . The three first components are already built in consumer devices and the other three are still under development.
The Content Scrambling System (CSS). This method developed by Matsushita scrambles MPEG-2 video. A pair of keys is required for descrambling: one is unique to the disk and the other is specific to the MPEG file being descrambled. Scrambled content is not viewable.
The Analog Protection System (APS). This system developed by Macrovision modifies NTSC/PAL. The resulting video signal can be displayed on televisions but cannot be recorded on VCR's. However, the data on a disk are not NTSC/PAL encoded and APS has to be applied after encoding in the DVD player. Some bits are consequently stored in the MPEG stream header and give the information of whether and how APS should be applied.
The Copy Generation Management System (CGMS). This is a pair of bits stored in the header of an MPEG stream encoding one of three possible rules for copying: copy-always, copy-never and copy-once. The copy-once case is included so that time-shifting is allowed i.e. a copy of broadcast media is made for later viewing.
5C. A coalition of five companies designs this mechanism. It allows several compliant devices, connected to the same digital video bus e.g. IEEE1394 (firewire), to exchange keys in an authenticated manner so that encrypted data can be sent over the bus. Noncompliant devices do not have access to the keys and cannot decrypt the data.
Watermarking. The main purpose of watermarking is to provide a more secure solution than storing bits in the MPEG stream header. In DVD, digital watermarking is primarily intended for the CGMS bits and secondary for the APS bits.
Physical identifiers. The idea is to design secure physical media identifiers in order to be able to distinguish between original media and copies.
Figure 42.4 shows how those mechanisms have been put together in the DVD so that copy protection is enforced. The additional performance brought by watermarking is emphasized by the dark walls.
Figure 42.4: DVD copy-protection system
Everything starts when Hollywood studios release a new copyrighted DVD with CGMS bits encoding the message copy-never. Both CSS keys are stored on the lead-in area of the DVD. This area is only read by compliant players. This prevents factory-pressed legal disks from being displayed by noncompliant players. Moreover bit-for-bit illegal copies will contain CSS scrambled content, but not the keys. As a result, such illegal copies cannot be displayed by any player, compliant or not. If the output signal given by compliant players is digital, CGMS bits prevent copying in the compliant world while 5C will avoid any communication with any noncompliant devices. However, to date, analog monitors are still widespread and even compliant players output an analog signal for compatibility. Since CGMS bits do not survive digital to analog conversion, watermarking is introduced in order to avoid copying in the compliant world. Unfortunately, in the noncompliant world, APS only disables copying of analog NTSC/PAL signals on VHS tapes. Disks without CSS or CGMS can then be easily generated e.g. thanks to a simple PC with a video capture card.
Now illegal disks containing unscrambled content without CSS or CGMS are available. They may have been generated as seen previously. But they can also be generated directly from an original legal disk since CSS was cracked in 1999 . The remaining CGMS bits can then be trivially stripped from the MPEG stream. Such illegal copies can of course be displayed by noncompliant players but watermarking has to be introduced in order to prevent those copies to enter the compliant world. Compliant players will detect the copy-never watermark embedded in unscrambled DVD-RAM and will refuse playback. The video signal given by a noncompliant player can be recorded by noncompliant recording devices. However watermarking prevents copying with compliant devices. The whole protection system results in two hermetically separated worlds. A consumer should have both types of players in order to display legal and illegal disks. The expense of such a strategy will help to "keep honest people honest".
It is important for DVD recorders to support the copy-once case in order to allow time shifting. When the recorder detects the copy-once message, it should modify the stream so that the hidden message becomes copy-never. This can be easily done in the case of stored bits in the MPEG header but it is less straightforward when using watermarking. Two proposals are investigated. The first one consists in superimposing a second watermark when a copy-once watermark is detected. The two watermarks together will then encode the message copy-never. The second proposal avoids remarking and exploits the ticket concept . The idea is to use two hidden signals: an embedded watermark W and a physical ticket T. There exists a relationship between the two signals which can be written Fn(T)=W, where F(.) is a one way hash function and n is the number of allowed passages through compliant devices. The ticket is decremented each time the data go through a compliant player or recorder. In other terms, the ticket is modified according to the relation T'=F(T). During playback, the ticket in transit can be embedded in MPEG user_data bits or in the blanking intervals of the NTSC/PAL standard. During recording, the ticket can be physically marked in the wobble  in the lead-in of optical disks.
Many valuable products are distributed over the television network. News items, such as those sold by companies like Reuters or Associated Press, can be worth over 100,000 USD. In France, during the final of the 2002 FIFA World Cup Korea Japan™, advertisers had to pay 100,000 Euros in order to broadcast a thirty-second commercial break shot on television. The same commercial would even have been billed 220,000 Euros if the French national team had played during the final. Owners of copyrighted videos want to get their royalties each time their property is broadcasted. The whole television market is worth many billions of dollars and Intellectual Property Rights violations are likely to occur. As a result, a broadcast surveillance system has to be built in order to check all broadcasted channels. This will help verifying that content owners get paid correctly and that advertisers get what they have paid for. Such a mechanism will prevent confidence tricks such as the one discovered in Japan in 1997 when two TV stations were convicted of overbooking air time .
The most naive approach of broadcast monitoring consists of a pool of human observers watching the broadcasts and recording whatever they see. However, this low-cost method is far from being optimal. Human employees are expensive and are not foolproof. As a result, research has been conducted in order to find a way of automating broadcast monitoring. The first approach, referred to as passive monitoring, basically makes a computer simulate a human observer: it monitors the broadcasts and compares the received signals with a database of known videos. This approach is nonintrusive and does not require cooperation from advertisers or broadcasters. However such a system has two major drawbacks. First, it relies on the comparison between received signals against a large database, which is nontrivial in practice. Pertinent signatures, clearly identifying each video, have to be defined and an efficient search for nearest neighbours in a large database has to be designed. This results in a system that is not fully reliable. This may be accurate for acquiring competitive market research data i.e. when a company wants to know how much its competitors spend in advertising. On the contrary, a small error rate (5%) is dramatic for verification services because of the large amount of money at stake. The second con is that the reference database is likely to be large and the storage and management costs might become rapidly prohibitive.
In order to reach the accuracy required for verification services, a new kind of systems, referred as active monitoring, has been designed. The underlying idea is to transmit computer-recognizable identification information along with the data. Such identification information is straightforward to decode reliably and to interpret correctly. This approach is known to be simpler to implement than passive monitoring. First implementations of active monitoring placed the identification information in a separate area of the broadcast signal e.g. the Vertical Blanking Interval (VBI) of an analog NTSC/PAL video signal. However dissimulating identification data into other data is exactly the purpose of digital watermarking. Even if watermark embedding is more complicated than storing information in some unused part of a video stream, digital watermarking can be considered as a robust way to implement active monitoring. The European project VIVA (Visual Identity Verification Auditor) proved the feasibility of such a system . The participants used a real-time watermarking scheme which provides active monitoring services over a satellite link. The complexity of the detection algorithm is moderate enough to allow simultaneous monitoring of many channels.
The explosion of the Internet has created a new way of acquiring copyrighted content. When a user wants to obtain a new video clip or a new movie, the simplest strategy is to log on the Internet and to use one of the popular peer-to-peer systems e.g. Napster, Gnutella, KaZaA, Morpheus. Multimedia digital contents, stored throughout the world on thousands of computers logged on at the same moment, will instantly get accessible. As a result, European engineering students often download and watch the most recent Hollywood films a long time before they are released in their own country. The situation is even worse in audio with the exchange of MP3  files. As a result, copyright owners lose a large amount of royalties . Legal action has been taken to ban such distributed systems but, when Napster was sentenced guilty, two other systems appeared. The basic problem does not come from peer-to-peer systems. It would be a great tool if only legal data was transiting on such distributed networks. The problem is that a traitor has made available copyrighted material without any kind of permission. The basic idea would consequently be to be able to identify the traitor when an illegal copy is found in order to sue him in court. This can be done by embedding an indelible and invisible watermark identifying the customer.
In the near future, the way people are looking at TV will be significantly modified. Video streaming is indeed likely to become more and more widespread. It is consequently necessary to find a way of protecting digital video content and digital watermarking seems to be a potential candidate . Two major applications are liable to take the homes by storm: Pay-Per-View (PPV) and Video-On-Demand (VOD). In both applications, digital watermarking can be used in order to enforce a fingerprinting policy. The customer ID is embedded into the delivered video data in order to trace back any user breaking his/her licence agreement. The main difference resides in the watermarking strategy as depicted in Figure 42.5. Embedding the watermark on the customer side has been suggested  but it should be avoided if possible in order to prevent reverse engineering. In a PPV environment, a video server multicasts some videos and customers have only to connect to the server in order to obtain the video. The video server is passive. At a given moment, it delivers the same video stream to multiple users. In order to enforce fingerprinting, a proposed method  is to have each network element (router, node or whatever) embed a piece of watermark as the video stream is relayed. The resulting watermark will contain a trace of the route followed by the video stream. Such a strategy requires support from network providers, who might not be forthcoming about it. In a VOD framework, the video server is active. It receives a request from a customer and sends the requested video. It is a multi-unicast strategy. This time, the video server can insert a watermark identifying the customer since each connection is dedicated to only one customer.
Figure 42.5: Alternative watermarking strategies for video streaming.
Another fingerprinting application has been considered with the apparition of a new kind of piracy. Nowadays illegal copying of brand new movies projected onto cinema screen by means of a handheld video camera has become a common practice. The most memorable example is surely when, one week after its US release, the very anticipated "Starwars Episode I: The Phantom Menace" was available on the Internet in a low quality version, with visible head shadows of audience members. Although the quality of such copies is usually very low, their economical impact can be enormous. Moreover, the upcoming digital cinema format to be introduced in theatres raises some concern. With higher visual quality, the threat becomes larger and Hollywood studios want to oblige cinema owners to prevent the presence of video cameras in their premises. Once again, digital watermarking could provide a solution . A watermark can be embedded during show time identifying the cinema, the presentation date and time. If an illegal copy created with a video camera is found, the watermark is extracted and the cinema to blame is identified. After many blames, the cinema is sanctioned with a ban on the availability of content.
Large amounts of video data are distributed throughout the Internet every day. More and more video cameras are installed in public facilities for surveillance purposes. However, popular video editing softwares permit today to easily tamper with video content, as shown in Figure 42.6, and video content is no more reliable. For example, in some countries, a video shot from a surveillance camera cannot be used as a piece of evidence in a courtroom because it is not considered trustworthy enough. When someone is emailed a somewhat unusual video, it is quite impossible to determine if it is an original or a hoax. Authentication techniques are consequently needed in order to ensure authenticity of video content. Methods have to be designed for verifying the originality of video content and preventing forgery. When a customer purchases video content via electronic commerce, he wants to be sure that it comes from the alleged producer and that no one has tampered with the content. The very first research efforts for data authentication used cryptography. The major drawback of such an approach is that it provides a complete verification. In other terms, the data is considered as untouchable and the data for authentication has to be exactly the same one as the original one. But this strong constraint might be too restricting. One might prefer to allow some distortions on the digital data if the original content has not been significantly modified. This is typically the case in wireless environment where some noise is added to the data. This approach is referred to as content verification.
Figure 42.6: Original and tampered video scenes
Researchers have investigated the use of digital watermarking in order to verify the integrity of digital video content. A basic approach consists in regularly embedding an incremental timestamp in the frames of the video . As a result, frame cuts, foreign frame insertion, frame swapping and frame rate alteration can be easily detected. This approach is very efficient for detecting temporal alteration of the video stream. However, it might fail in detecting alterations of the content itself e.g. a character is completely removed from a movie.
Investigations have consequently been conducted in order to prevent modifications of the content of the video itself. One proposal  embeds the edge map of each frame in the video stream. During the verification process, if the video content has been modified, there will be a mismatch between the extracted edge map from the verified video and the watermarked edge map. The detector will consequently report content tampering. Another proposal exploits the idea that a movie is made up of one audio and one video stream and that both need to be protected against unauthorised tampering. The fundamental idea is then to combine video and audio watermarking  in order to obtain an efficient authenticating system. Features of both streams are embedded one into another. Modification from either the sound track, or the video track, is immediately spotted by the detector, since the extracted and watermarked features will differ.
Copyright protection is historically the very first targeted applications for digital watermarking. The underlying strategy consists in embedding a watermark, identifying the copyright owner, in digital multimedia data. If an illegal copy is found, the copyright owner can prove his/her paternity thanks to the embedded watermark and can sue the indelicate user in court. This perfect scenario is however likely to be disturbed by malicious users in the real world . If an attacker adds a second watermark into a video clip, both the original owner and the attacker can claim ownership and therefore defeat the purpose of using watermarking. Using the original video clip during the verification procedure happens to prevent the multiple ownership problems in some cases. However, this problem still holds if the watermarking algorithm is invertible because it allows the attacker to produce his/her own counterfeited original video clip. In this case, both the original owner and the attacker have an original video clip which contains the watermark of the other one. As a result, nobody can claim ownership! This situation is referred to as the deadlock problem in the watermarking community. Watermark algorithms are consequently required to be noninvertible in order to provide copyright protection services and they are often backed up by an elaborated protocol with a trusted third party. Copyright protection has been investigated for video watermarking  even if this is not the most targeted application.
Instead of protecting the whole video stream, copyright owners might rather want to protect only a part of the video content. The commercial value in a video is indeed often concentrated in a small number of video objects e.g. the face of an actor. Moreover, future video formats will distinguish the different objects in a video. This will be the case with the upcoming MPEG-4 format. Recent research has consequently investigated digital watermarking of video objects . Watermarking video objects prevents unauthorised reuse in other video clips. However video objects are likely to be submitted to various video editing such as scaling, rotation, shifting and flipping. As a result, special care must be taken regarding the resilience of the watermark against such processings. This can be quite easily obtained thanks to a geometrical normalisation , according to the moments and axes of the video object, prior to embedding and extraction.
The attentive reader may have noticed that video watermarking and video coding are two conflicting technologies. A perfect video codec should remove any extra redundant information. In other terms, two visually similar videos should have the same compressed representation. If one day, such an optimal video codec is designed, then video watermarking will disappear since unwatermarked and watermarked data would have the same compressed representation. Digital watermarking can be consequently seen as the exploitation of the features of the compression algorithms in order to hide information. However recent research has shown that digital watermarking can benefit the coding community. The video coding process can be sequenced in two steps. During source coding, any redundant information is removed in order to obtain the most possible compressed representation of the data while keeping its original visual quality. This compressed representation is then submitted to channel coding, where extra redundant information is added for error correction. Channel coding is mandatory since errors are likely to occur during the transmission, e.g. in a wireless environment. Digital watermarking can be introduced as an alternative solution for introducing error correcting information after source coding, without inducing any overhead . Experiments have demonstrated the feasibility of such an approach and results are even reported showing that digital watermarking can have better performances than traditional error correction mechanisms .
Embedding useful data directly into the video stream can spare much storage space. A typical video stream is made up of two different parallel streams: the audio and video streams. Those two streams need to be synchronised during playback for pleasant viewing, which is difficult to maintain during cropping operations. Hiding the audio stream into the video one  will implicitly provide efficient and robust synchronisation, while significantly reducing the required storage need or available bandwidth. In the same fashion, the actual Picture-in-Picture system can be improved by hiding a video stream into another one . This technology, present in many television sets, uses separate data streams in order to superimpose a small video window over the full-size video displayed on the television set. Digital watermarking allows embedding the secondary video stream into the carrier one. During playback, the watermark is extracted and the embedded video is displayed in a window within the host video. With such an approach, only one stream needs to be transmitted. This approach can be extended so that a user can switch to the PG version of an R rated movie, with alternative dialogs and scenes replacing inappropriate content.
The wobble is a radial deviation of the position of pits and lands relative to the ideal spiral. Noncompliant recorders will not insert a ticket and the illegal disk will not enter the compliant world.
The MPEG-1 audio layer 3 (MP3) is a popular audio format for transmitting audio files across the Internet.