Designing and Coding the SimpleDelay DMO

Designing and Coding the SimpleDelay DMO

The design of the SimpleDelay DMO is straightforward. Input samples enter the DMO, are held for a specific length of time (as defined in the header file), are mixed with later input samples, and are passed to the output. When the DMO begins operation, silence is sent as output samples until sufficient time has passed for the input samples to appear as output samples. This process iterates until no more input samples are presented; once all remaining output samples have been sent to the DMO s caller, operation of the DMO ceases.

The definition and internal structure of a DMO is analogous to that of a DirectShow filter. There are methods to negotiate media type, negotiate buffer size, and perform data transformations. However, whereas DirectShow filters are defined as descendants of CBaseFilter or perhaps CTransformFilter, DMOs are declared as descendants from a class template, IMediaObjectImpl. That template provides the framework for functionality such as sanity checks on input parameters and setting and storing the media type but the public methods call equivalent internal methods where the real work gets done, and these need to be overridden by the programmer. Here s the class definition for the Simple Delay class CDelay:

class ATL_NO_VTABLE CDelay : public IMediaObjectImpl<CDelay, 1, 1>, // DMO Template 1 input, 1 output public CComObjectRootEx<CComMultiThreadModel>, public CComCoClass<CDelay, &CLSID_Delay>, public IMediaObjectInPlace { public: CDelay() : m_nWet(DEFAULT_WET_DRY_MIX), m_dwDelay(DEFAULT_DELAY) { m_pUnkMarshaler = NULL; } DECLARE_REGISTRY_RESOURCEID(IDR_DELAY) DECLARE_GET_CONTROLLING_UNKNOWN() DECLARE_PROTECT_FINAL_CONSTRUCT() BEGIN_COM_MAP(CDelay) COM_INTERFACE_ENTRY(IMediaObject) COM_INTERFACE_ENTRY(IMediaObjectInPlace) COM_INTERFACE_ENTRY_AGGREGATE(IID_IMarshal, m_pUnkMarshaler.p) END_COM_MAP() HRESULT FinalConstruct() { return CoCreateFreeThreadedMarshaler( GetControllingUnknown(), &m_pUnkMarshaler.p); } void FinalRelease() { FreeStreamingResources(); // In case client does not call this m_pUnkMarshaler.Release(); } CComPtr<IUnknown> m_pUnkMarshaler; public: // Declare internal methods required by ImediaObjectImpl. HRESULT InternalGetInputStreamInfo(DWORD dwInputStreamIndex, DWORD *pdwFlags); HRESULT InternalGetOutputStreamInfo(DWORD dwOutputStreamIndex, DWORD *pdwFlags); HRESULT InternalCheckInputType(DWORD dwInputStreamIndex, const DMO_MEDIA_TYPE *pmt); HRESULT InternalCheckOutputType(DWORD dwOutputStreamIndex, const DMO_MEDIA_TYPE *pmt); HRESULT InternalGetInputType(DWORD dwInputStreamIndex, DWORD dwTypeIndex, DMO_MEDIA_TYPE *pmt); HRESULT InternalGetOutputType(DWORD dwOutputStreamIndex, DWORD dwTypeIndex, DMO_MEDIA_TYPE *pmt); HRESULT InternalGetInputSizeInfo(DWORD dwInputStreamIndex, DWORD *pcbSize, DWORD *pcbMaxLookahead, DWORD *pcbAlignment); HRESULT InternalGetOutputSizeInfo(DWORD dwOutputStreamIndex, DWORD *pcbSize, DWORD *pcbAlignment); HRESULT InternalGetInputMaxLatency(DWORD dwInputStreamIndex, REFERENCE_TIME *prtMaxLatency); HRESULT InternalSetInputMaxLatency(DWORD dwInputStreamIndex, REFERENCE_TIME rtMaxLatency); HRESULT InternalFlush(); HRESULT InternalDiscontinuity(DWORD dwInputStreamIndex); HRESULT InternalAllocateStreamingResources(); HRESULT InternalFreeStreamingResources(); HRESULT InternalProcessInput(DWORD dwInputStreamIndex, IMediaBuffer *pBuffer, DWORD dwFlags, REFERENCE_TIME rtTimestamp, REFERENCE_TIME rtTimelength); HRESULT InternalProcessOutput(DWORD dwFlags, DWORD cOutputBufferCount, DMO_OUTPUT_DATA_BUFFER *pOutputBuffers, DWORD *pdwStatus); HRESULT InternalAcceptingInput(DWORD dwInputStreamIndex); // IMediaObjectInPlace methods. STDMETHOD(Process)(ULONG ulSize, BYTE *pData, REFERENCE_TIME refTimeStart, DWORD dwFlags); STDMETHOD(Clone)(IMediaObjectInPlace **ppMediaObject); STDMETHOD(GetLatency)(REFERENCE_TIME *pLatencyTime); private: // TypesMatch: Return true if all the required fields match. bool TypesMatch(const DMO_MEDIA_TYPE *pmt1, const DMO_MEDIA_TYPE *pmt2); // CheckPcmFormat: Return S_OK if pmt is a valid PCM audio type. HRESULT CheckPcmFormat(const DMO_MEDIA_TYPE *pmt); // DoProcessOutput: Process data. HRESULT DoProcessOutput( BYTE *pbData, // Pointer to the output buffer const BYTE *pbInputData, // Pointer to the input buffer DWORD dwQuantaToProcess // Number of quanta to process ); void FillBufferWithSilence(void); // GetPcmType: Get our only preferred PCM type HRESULT GetPcmType(DMO_MEDIA_TYPE *pmt); // Members CComPtr<IMediaBuffer> m_pBuffer; // Pointer to current input buffer BYTE *m_pbInputData; // Pointer to data in input buffer DWORD m_cbInputLength; // Length of the data REFERENCE_TIME m_rtTimestamp; // Most recent timestamp bool m_bValidTime; // Is timestamp valid? WAVEFORMATEX *m_pWave; // Pointer to WAVEFORMATEX struct BYTE *m_pbDelayBuffer; // circular buffer for delay samples DWORD m_cbDelayBuffer; // size of the delay buffer BYTE *m_pbDelayPtr; // ptr to next delay sample long m_nWet; // Wet portion of wet/dry mix DWORD m_dwDelay; // Delay in ms bool Is8Bit() { return (m_pWave->wBitsPerSample == 8); } // Moves the delay pointer around the circular buffer. void IncrementDelayPtr(size_t size) { m_pbDelayPtr += size; if (m_pbDelayPtr + size > m_pbDelayBuffer + m_cbDelayBuffer) { m_pbDelayPtr = m_pbDelayBuffer; } } };

The class definition of CDelay begins with a declaration of the four ancestor classes. IMediaObjectImpl, as previously explained, provides the basic framework for the DMO. Its initialization parameters define the CDelay DMO as possessing one input and one output. CComObjectRootEx defines the interfaces required by COM to make the object visible across the operating system. The COM map defined in the first part of the class definition is managed by CCom ObjectRootEx. CComCoClass is used by the class factory, which creates instances of the DMO based on its GUID. Both CoComObjectRootEx and CComCoClass are Active Template Library (ATL) constructs; they re by-products of an ATL wizard that was used to implement SimpleDelay. IMediaObjectImpl doesn t require ATL, but ATL does handle a lot of the COM details for you and is more convenient.

IMediaObjectInPlace implements a DMO that performs in-place transformations on buffer data. The in-place transform means that this object needs to implement a Process method, which transforms the buffer data without moving it to a separate output buffer. All DMOs, whether they do in-place transformations or not, need to implement separate ProcessInput and ProcessOutput methods, which are part of the IMediaObject interface, and copy data from the input to output buffer during the transform process, much like a DirectShow transform filter does.

Implementing the DMO Media and Buffer Negotiation Methods

Two methods declared in CDelay, CDelay::InternalGetInputStreamInfo and CDelay::InternalGetOutputStreamInfo, are methods required by the IMedia ObjectImpl class template. They correspond to the GetInputStreamInfo and GetOutputStreamInfo methods on the IMediaObject interface. Calls to these methods use bit fields to signal information about the input and output streams. For the input stream, here are the possible values given by DMO_INPUT_ STREAM_INFO_FLAGS:

enum _DMO_INPUT_STREAM_INFO_FLAGS { DMO_INPUT_STREAMF_WHOLE_SAMPLES = 0x00000001, DMO_INPUT_STREAMF_SINGLE_SAMPLE_PER_BUFFER = 0x00000002, DMO_INPUT_STREAMF_FIXED_SAMPLE_SIZE = 0x00000004, DMO_INPUT_STREAMF_HOLDS_BUFFERS = 0x00000008 };

The output stream values are defined by DMO_OUTPUT_STREAM_ INFO_FLAGS.

enum _DMO_OUTPUT_STREAM_INFO_FLAGS { DMO_OUTPUT_STREAMF_WHOLE_SAMPLES = 0x00000001, DMO_OUTPUT_STREAMF_SINGLE_SAMPLE_PER_BUFFER = 0x00000002, DMO_OUTPUT_STREAMF_FIXED_SAMPLE_SIZE = 0x00000004, DMO_OUTPUT_STREAMF_DISCARDABLE = 0x00000008, DMO_OUTPUT_STREAMF_OPTIONAL = 0x00000010 };

In the implementation of CDelay::InternalGetInputStreamInfo, CDelay informs callers that it s expecting an input stream that consists of whole samples and that these samples must have a fixed size. The same flags are set for CDelay::InternalGetOutputStreamInfo, indicating that it will send streams consisting of whole samples of a fixed size. The functions of other bits in the DMO_INPUT_STREAM_INFO_FLAGS and DMO_OUTPUT_STREAM_INFO_FLAGS enumerations are given in the DirectX SDK documentation.

Four CDelay methods directly handle media type negotiation: CDelay::InternalCheckInputType, CDelay::InternalCheckOutputType, CDelay::InternalGetInputType, and CDelay::InternalGetOutputType. The Get methods query the DMO for its preferred media types, and the Check methods determine whether the DMO will accept a media type. As is true for DirectShow filters, DMOs can negotiate the media types they can accept by sending a list of possible types back to the caller. In the case of the SimpleDelay DMO, that list consists only of PCM audio, so these methods ensure that only a valid media stream validated by the CDelay::CheckPcmFormat method is received by the DMO for processing. Any other media types are rejected by the DMO. Here s the implementation of CDelay::CheckPcmFormat:

HRESULT CDelay::CheckPcmFormat(const DMO_MEDIA_TYPE *pmt) { if (pmt->majortype == MEDIATYPE_Audio && pmt->subtype == MEDIASUBTYPE_PCM && pmt->formattype == FORMAT_WaveFormatEx && pmt->cbFormat == sizeof(WAVEFORMATEX) && pmt->pbFormat != NULL) { // Check the format block WAVEFORMATEX *pWave = (WAVEFORMATEX*)pmt->pbFormat; if ((pWave->wFormatTag == WAVE_FORMAT_PCM) && (pWave->wBitsPerSample == 8 pWave->wBitsPerSample == 16) && (pWave->nBlockAlign == pWave->nChannels * pWave->wBitsPerSample / 8) && (pWave->nAvgBytesPerSec == pWave->nSamplesPerSec * pWave->nBlockAlign)) { return S_OK; } } return DMO_E_INVALIDTYPE; }

The SimpleDelay DMO isn t very picky about the kind of audio stream it wants to receive, but the input and output types must match. (This is a requirement for any DMO that supports in-place processing.) The stream must be PCM audio, and it must be either 8 or 16 bits per sample. The fields of the passed DMO_MEDIA_TYPE structure are examined within CDelay::CheckPcmFormat. Here s the composition of DMO_MEDIA_TYPE:

typedef struct _DMOMediaType { GUID majortype; GUID subtype; BOOL bFixedSizeSamples; BOOL bTemporalCompression; ULONG lSampleSize; GUID formattype; IUnknown *pUnk; ULONG cbFormat; [size_is(cbFormat)] BYTE * pbFormat; } DMO_MEDIA_TYPE;

The fields within a DMO_MEDIA_TYPE structure have the same names and the same purposes as the equivalent fields within the AM_MEDIA_TYPE structure, which was covered in Chapter 10. As with AM_MEDIA_TYPE, it s important to remember that cbFormat must hold the size of the data structure pointed to by pbFormat, if any.

HRESULT CDelay::InternalGetInputSizeInfo(DWORD dwInputStreamIndex, DWORD *pcbSize, DWORD *pcbMaxLookahead, DWORD *pcbAlignment) { // IMediaObjectImpl validates this for us... _ASSERTE(InputTypeSet(dwInputStreamIndex)); // And we expect only PCM audio types. _ASSERTE(InputType(dwInputStreamIndex)->formattype == FORMAT_WaveFormatEx); WAVEFORMATEX *pWave = (WAVEFORMATEX*)InputType(dwInputStreamIndex)->pbFormat; *pcbSize = pWave->nBlockAlign; *pcbMaxLookahead = 0; *pcbAlignment = 1; return S_OK; } HRESULT CDelay::InternalGetOutputSizeInfo(DWORD dwOutputStreamIndex, DWORD *pcbSize, DWORD *pcbAlignment) { // IMediaObjectImpl validates this for us... _ASSERTE(OutputTypeSet(dwOutputStreamIndex)); // And we expect only PCM audio types. _ASSERTE(OutputType(dwOutputStreamIndex)->formattype == FORMAT_WaveFormatEx); WAVEFORMATEX *pWave = (WAVEFORMATEX*)OutputType(dwOutputStreamIndex)->pbFormat; *pcbSize = pWave->nBlockAlign; *pcbAlignment = 1; return S_OK; }

The CDelay DMO implements two methods crucial to the buffer allocation process: CDelay::InternalGetInputSizeInfo and CDelay::InternalGetOutputSizeInfo. Both methods use the DMO_MEDIA_TYPE data structure to calculate the total buffer size needed the size of one full sample of audio data. Because this is an in-place transformation of buffer data, both methods should return the same values.

Two other methods, CDelay::InternalGetInputMaxLatency and CDelay::InternalSetInputMaxLatency, establish latency values on the input stream. Latency is the difference between the input timestamp on a stream as it enters a DMO and the output timestamp on the transformed stream. DMOs allow their latency values to be read and set so that other processing elements can properly synchronize their streams with the DMO. In the case of CDelay, both methods return the error message E_NOTIMPL, which means that latency is not implemented in this DMO and zero latency can be assumed.

Implementing DMO Internal Methods

Four methods handle states and conditions internal to the DMO. Two of these methods, CDelay::InternalAllocateStreamingResources and CDelay::InternalFreeStreamingResources, simply allocate and free a sample buffer used to create the sample delay. If your own DMO has resource allocation requirements to handle a media stream, these routines manage the allocation and release of those resources. The client might not call the public versions of these methods, but the IMediaObjectImpl class template will invoke them if the client doesn t.

Two other methods, CDelay::InternalFlush and CDelay::InternalDiscontinuity, handle states generated by the client in its normal operation. When CDelay::InternalFlush is executed, all data held by the DMO is released and any sound samples are wiped out with a call to CDelay::FillBufferWithSilence, which resets the delay buffer.

void CDelay::FillBufferWithSilence() { if (Is8Bit()) FillMemory(m_pbDelayBuffer, m_cbDelayBuffer, 0x80); else ZeroMemory(m_pbDelayBuffer, m_cbDelayBuffer); }

There are two ways to clear the sample buffer, depending on the number of bits per sample. If the PCM waveform is composed of 8-bit samples, the entire buffer of byte values must be set to 128 silence for 8-bit PCM. If the waveform is composed of 16-bit samples, the entire buffer is zeroed.

When the media stream ceases flowing to a DMO or the media type of the stream changes, the condition is signaled with a discontinuity. A discontinuity indicates that there s a forthcoming break in the stream. Some DMOs will need to continue to provide output data until their internal buffers empty, but they can use the discontinuity signal to reject any input buffers until such time as their output buffers have emptied. At this point, the DMO is ready, waiting for stream input to begin again. The CDelay::InternalDiscontinuity method doesn t do anything in the SimpleDelay DMO beyond returning an S_OK response, but in the case of a more complex DMO, this method could be used to trigger a number of state-driven events, such as tracking whether the DMO is ready to receive more input.

Another method, CDelay::InternalAcceptingInput, is queried when the client wants to know whether the DMO s input stream can accept more input. If a buffer pool is allocated to a DMO, the implementation of the InternalAcceptingInput method can determine whether the DMO can accept another input buffer. In this case, we return S_FALSE if the DMO can t accept an input buffer (we re full, thank you very much) or S_OK if the DMO is able to receive another input buffer.

Implementing DMO Data Processing Methods

An in-place DMO must have publicly accessible ProcessInput and Process Output methods. The CDelay DMO has two similar internal methods, C Delay::InternalProcessInput and CDelay::InternalProcessOutput. Here s the implementation of both methods:

HRESULT CDelay::InternalProcessInput(DWORD dwInputStreamIndex, IMediaBuffer *pBuffer, DWORD dwFlags, REFERENCE_TIME rtTimestamp, REFERENCE_TIME rtTimelength) { _ASSERTE(m_pBuffer == NULL); HRESULT hr = pBuffer->GetBufferAndLength(&m_pbInputData, &m_cbInputLength); if (FAILED(hr)) { return hr; } if (m_cbInputLength <= 0) return E_FAIL; m_pBuffer = pBuffer; if (dwFlags & DMO_INPUT_DATA_BUFFERF_TIME) { m_bValidTime = true; m_rtTimestamp = rtTimestamp; } else { m_bValidTime = false; } return S_OK; } HRESULT CDelay::InternalProcessOutput(DWORD dwFlags, DWORD cOutputBufferCount, DMO_OUTPUT_DATA_BUFFER *pOutputBuffers, DWORD *pdwStatus) { BYTE *pbData; DWORD cbData; DWORD cbOutputLength; DWORD cbBytesProcessed; CComPtr<IMediaBuffer> pOutputBuffer = pOutputBuffers[0].pBuffer; if (!m_pBuffer !pOutputBuffer) { return S_FALSE; // Did not produce output } // Get the size of our output buffer. HRESULT hr = pOutputBuffer->GetBufferAndLength(&pbData, &cbData); hr = pOutputBuffer->GetMaxLength(&cbOutputLength); if (FAILED(hr)) { return hr; } // Skip past any valid data in the output buffer. pbData += cbData; cbOutputLength -= cbData; if (cbOutputLength < m_pWave->nBlockAlign) { return E_FAIL; } // Calculate how many quanta we can process. bool bComplete = false; if (m_cbInputLength > cbOutputLength) { cbBytesProcessed = cbOutputLength; } else { cbBytesProcessed = m_cbInputLength; bComplete = true; } DWORD dwQuanta = cbBytesProcessed / m_pWave->nBlockAlign; // The actual data we write may be // less than the available buffer length // due to the block alignment. cbBytesProcessed = dwQuanta * m_pWave->nBlockAlign; hr = DoProcessOutput(pbData, m_pbInputData, dwQuanta); if (FAILED(hr)) { return hr; } hr = pOutputBuffer->SetLength(cbBytesProcessed + cbData); if (m_bValidTime) { pOutputBuffers[0].dwStatus = DMO_OUTPUT_DATA_BUFFERF_TIME; pOutputBuffers[0].rtTimestamp = m_rtTimestamp; // Estimate how far along we are... pOutputBuffers[0].dwStatus = DMO_OUTPUT_DATA_BUFFERF_TIMELENGTH; pOutputBuffers[0].rtTimelength = (cbBytesProcessed / m_pWave->nAvgBytesPerSec) * UNITS; } if (bComplete) { m_pBuffer = NULL; // Release input buffer } else { pOutputBuffers[0].dwStatus = DMO_OUTPUT_DATA_BUFFERF_INCOMPLETE; m_cbInputLength -= cbBytesProcessed; m_pbInputData += cbBytesProcessed; m_rtTimestamp += pOutputBuffers[0].rtTimelength; } return S_OK; }

The CDelay::InternalProcessInput method receives a pointer to the IMedia Buffer interface that manages the sample data buffer. The method verifies that the length of the data in the buffer is greater than zero. If the timestamp is expected to be valid on the sample because the appropriate DMO_INPUT_DATA_BUFFER_FLAG bit is set a class variable is set and the sample s timestamp is stored.

Although the DMO receives one call to CDelay::InternalProcessInput for every buffer presented to it, the DMO can process one buffer per stream on each call to its corresponding output method, CDelay::InternalProcessOutput. If you have more than one output stream from a DMO, each stream s buffer will be processed with each call to CDelay::InternalProcessOutput. This method is passed the number of buffers to be processed and a list of pointers to DMO_OUTPUT_DATA_BUFFER structures that hold pointers to the IMedia Buffer interfaces for each sample and timestamp information. (In our case, the CDelay DMO has only one output stream.)

The method sets up the information it needs to make a method call to CDelay::DoProcessOutput, which performs the in-place transform of the buffer data. Here s the implementation of that private method:

HRESULT CDelay::DoProcessOutput(BYTE *pbData, const BYTE *pbInputData, DWORD dwQuanta) { DWORD sample, channel, num_channels; num_channels = m_pWave->nChannels; if (Is8Bit()) { for (sample = 0; sample < dwQuanta; ++sample) { for (channel = 0; channel < num_channels; ++channel) { // 8-bit sound is 0..255 with 128 == silence. // Get the input sample and normalize to -128..127. int i = pbInputData[sample * num_channels + channel] - 128; // Get the delay sample and normalize to -128..127. int delay = m_pbDelayPtr[0] - 128; m_pbDelayPtr[0] = i + 128; IncrementDelayPtr(sizeof(unsigned char)); i = (i * (100 - m_nWet)) / 100 + (delay * m_nWet) / 100; // Truncate. if (i > 127) i = 127; if (i < -128) i = -128; pbData[sample * num_channels + channel] = (unsigned char)(i+128); } } } else // 16-bit { for (sample = 0; sample < dwQuanta; ++sample) { for (channel = 0; channel < num_channels; ++channel) { int i = ((short*)pbInputData)[sample * num_channels + channel]; int delay = ((short*)m_pbDelayPtr)[0]; ((short*)m_pbDelayPtr)[0] = i; IncrementDelayPtr(sizeof(short)); i = (i * (100 - m_nWet)) / 100 + (delay * m_nWet) / 100; // Truncate. if (i > 32767) i = 32767; if (i < -32768) i = -32768; ((short*)pbData)[sample * num_channels + channel] = (short)i; } } } return S_OK; }

This method manipulates the sound data in the buffer passed to it, mixing it together with sound data in another buffer (this is how you get a 2-second delay) and then producing an output from the mixture of the sound data. There are different pathways for 8-bit sound and 16-bit sound because the arithmetic involved in each is slightly different, but the basic algorithm is the same for both: input sample + delay sample = output sample.

Implementing the IMediaObjectInPlace Methods

Three methods must be overridden as part of the implementation of any class descendent from IMediaObjectInPlace: Process, Clone, and GetLatency. Here is the method that, in normal circumstances, performs the data transform for the SimpleDelay DMO, CDelay::Process:

STDMETHODIMP CDelay::Process(ULONG ulSize, BYTE *pData, REFERENCE_TIME refTimeStart, DWORD dwFlags) { if (dwFlags &= ~DMO_INPLACE_ZERO) return E_INVALIDARG; if (!pData) { return E_POINTER; } LockIt lock(this); if (!InputTypeSet(0) !OutputTypeSet(0)) { return DMO_E_TYPE_NOT_SET; } // Make sure all streams have media types set // and resources are allocated. HRESULT hr = AllocateStreamingResources(); if (SUCCEEDED(hr)) hr = DoProcessOutput(pData, pData, ulSize / m_pWave->nBlockAlign); return hr; }

The method determines that the pointers are valid, that in-place data transformations are allowed, and that the input and output types have been set (which implies that the DMO is connected to something outside itself). This parameter validation is not performed for IMediaObjectInPlace methods (although it is for IMediaObject methods), so we have to do it ourselves. The method invokes CDelay::AllocateStreamingResources to ensure that memory is available for the transform operation and then calls CDelay::DoProcessOutput to handle the actual data transform. The CDelay::Clone method is an implementation of the base class method that creates new copies of objects. In this implementation, the media type information for both the input and output streams is copied to the new object. Finally CDelay::GetLatency should return the actual time it takes the DMO to process buffer data. In this case, CDelay::GetLatency returns a value of zero, or no latency, which is consistent with the value returned by CDelay::InternalGetLatency.