Streams

A stream is a bidirectional pipe connecting a source and a destination. Sequences of data can be read from and written to a stream, enabling two end points to communicate with each other simply by placing data onto the pipe. In many cases, as is the case with file I/O, one end of the pipe is controlled by the operating system (OS). In others, such as with network sockets, both sides can be software exchanging data. It's even possible for a single application to own both sides of the pipe, for example when passing messages in-memory between two isolated software components.

The System.IO.Stream base class provides a common abstract interface that can be used to access such sequential data sources without concern over the particular details or special mechanics used to work them. File handles, for instance, are complex resources whose lifetime must be carefully coordinated with the OS. This is almost entirely transparent to the developer who uses the FileStream class, however, because the interface is almost entirely derived from the standard Stream type. We'll see later how another set of rich types layer on top of streams, increasing the level of abstraction even higher. It is exactly this high level of abstraction that makes the .NET Framework a very powerful tool for doing more with less code; the I/O types are poster children for this cause.

Working with the Base Class

There isn't very much you can actually do with the Stream class itself. It is abstract, and hence cannot be constructed. You should spend some time learning its interface in isolation, however, because it will make working with concrete types of data a simple matter of mapping the idea of streams on top of it. Stream has on the order of 10 direct subclasses in the Framework, ranging from data sources, such as files, to the network. We discuss these concrete implementations immediately following this section.

Before diving into the specifics of real implementations of Stream, let's first review the abstract class's interface. The examples in this section all operate under the assumption that a concrete implementation has been obtained, although we won't cover how to do that just that. We will take a look at reading and writing to streams, in that order, in the following paragraphs.

Stream is a fairly low-level type. All read or write operations deal with raw, individual 8-bit bytes, even when the data source is character based. For encoded streams (Unicode or UTF-8, for instance), this has the consequence that one character might span multiple bytes, making manual translation a tricky task. The higher-level reader and writer classes mentioned above handle this and other related encoding issues. Until we look at them later, let's just pretend that you want to work only with raw 8-bit bytes. This is entirely reasonable, especially when parsing custom binary file formats or single byte character encoded (e.g., ASCII) text, for example.

Reading Data

Each stream has a single data source, representing a backing store of bytes that can be read from or written to. This source is logically finite, meaning that when reading data you will often have to worry about end of stream (EOS) conditions when available data has been exhausted. Sometimes a stream will block execution when a read is issued and there are no bytes available to read; this is the case with some network data sources, for example. Note that not every stream is capable of being read from. A stream's bool CanRead property will return true if it is capable of being read from; if it returns false, however, reading from it will typically throw a NotSupportedException.

You can read single bytes or entire blocks of bytes at a time. Two specific instance methods can be used to synchronously read from a Stream: int ReadByte() and int Read(byte[] buffer, int offset, int count). ReadByte reads only 1 byte at a time and returns -1 to indicate an EOS condition. Read, on the other hand, reads a block of bytes, and returns 0 to indicate EOS.

Note

You might wonder why ReadByte returns a 4-byte Int32 yet only advances through streams at a rate of 1 byte at a time. Why doesn't the API simply use a Byte, then? One reason is that Byte is not signed, yet the method uses a signed integer -1 to indicate EOS. Technically, an Int16 could have been used but because the C# int keyword — the most widely used integer type on the platform — maps to an Int32, using Int32 eliminates unnecessary runtime coercions.

As an example of using ReadByte, consider this snippet of code, which advances through a Stream one byte at a time, printing out each one to the console along the way:

 using (Stream s = /*...*/) {     int read;     while ((read = s.ReadByte()) != -1)     {         Console.Write("{0} ", read);     } } 

While this pattern may look strange if you've never seen it before, such while-based loops for advancing through a stream are very common. We perform the read and EOS check on the same line (read = s.ReadByte()) != 1, making it more compact (albeit perhaps with a minor sacrifice in readability). If ReadByte returns a number other than -1, it will be in the range of 0–255, inclusive. This number represents the single byte that was read from the stream. We then print it to the console using Console.Write (detailed later).

You'll also notice that, even when using a plain-text file, this code prints out the raw byte value instead of the character it represents. For an ASCII file where each character is represented by a single byte, casting the read variable to char while printing will actually print the real character the byte represents:

 Console.WriteLine("{0} ", (char)read); 

However, for files containing UTF-8 or Unicode contents, this line of code will not work correctly. We must worry about special encoding details for byte-to-char mapping. In Unicode, a single character is represented by 2 bytes, while in UTF-8 a single character can be represented by a varying number of bytes. You'll see how to deal with this situation when we take a look at the TextReader class later in this section.

The int Read(byte[] buffer, int offset, int count) method takes a byte[] buffer into which it will write the data it reads from the stream. The method also takes two integers that specify an offset into the buffer array at which to start writing, and the maximum count of bytes to read from the stream. The sum of offset and count must not exceed the size of the target buffer's length; otherwise, an ArgumentOutOfRangeException will be thrown. This code demonstrates using Read to advance through a stream 4096 bytes at a time:

 using (Stream s = /*...*/) {     int readCount;     byte[] buffer = new byte[4096];     while ((readCount = s.Read(buffer, 0, buffer.Length)) != 0)     {         for (int i = 0; i < readCount; i++)         {             Console.Write("{0} ", buffer[i]);         }     } } 

The code structure is similar to the one-byte-at-a-time approach shown above, with the primary difference that we obtain an entire chunk of bytes from the stream at once. Read returns an integer indicating the number of bytes successfully read from the stream. You should never assume that the size of the data read is equal to the size of the array. In the above example, for instance, this would only always be true if the size of the stream's data were a multiple of 4096 bytes. The last read before an EOS occurs, however, will often be less than the buffer's size. This means that your buffer will only be partially populated on the last read, leaving behind a bit of data from the previous read.

Writing Data

A stream's data source is often mutable, enabling you to write data to be committed to the other end of the stream. In the case of file streams, for example, writing to it will cause the OS to receive the data and commit it to the appropriate file on disk (e.g., using WriteFile). For network streams, data will be physically transmitted over the network to a listener on the other end of the connection. Similar to reading, if you attempt to write to certain streams that are not ready to receive data, for example network streams, the thread could block execution until it becomes ready. Not all streams support writing — check the CanWrite predicate to see if a given stream can be written to.

Not surprisingly, the idioms for writing are nearly identical to those used in reading. There are two instance methods on Stream to perform synchronous writing: void WriteByte(byte value) and void Write(byte[] buffer, int offset, int count). As with reading, WriteByte enables you to write a single byte to a stream at once, while Write allows entire buffers of data to be written. Neither returns a value because there isn't a need to detect EOS conditions when writing.

The void Write(byte[] buffer, int offset, int count) method will write a set of bytes contained in buffer, beginning at the offset index and stopping after writing out count bytes. Most stream backing stores will grow to accommodate writes that exceed its current size, although some fixed-size streams can legitimately throw an exception should this occur. As a brief example, consider this snippet of code, which demonstrates both reading from and writing to a stream:

 using (Stream from = /*...*/) using (Stream to = /*...*/) {     int readCount;     byte[] buffer = new byte[1024];     while ((readCount = from.Read(buffer, 0, buffer.Length)) != 0)     {         to.Write(buffer, 0, readCount);     } } 

This code actually copies the entire contents of a one source stream to another destination stream. The example works on heterogeneous backing stores, meaning that this simple code could actually copy directly from a file stream to a network or memory stream, for example, among other possibilities. Hopefully, you are beginning to see why the stream abstraction is a very powerful tool to have around.

Seeking to a Position

As you have witnessed, a stream is logically positioned within the data source to which it is attached. When you read from or write to it, you are implicitly advancing this position by the number of bytes read or written. Many streams are finite in size. In other words, they have a maximum position. In fact, most such streams expose a concrete size through their Length property and their current absolute position via their Position property. Seeking simply means to explicitly change position in the stream without needing to read from or write to it, sometimes permitting forward-only seeking, but other times permitting forward and backward movement. It's as if you are rewinding and/or fast-forwarding through the stream's contents.

Not all streams support seeking, a policy decision that is entirely dependent on the type of backing store used. For example, a file has a fixed size that can be calculated and will typically support seeking (using Win32's SetFilePointer function); conversely, a network socket is open-ended in size and traditionally does not support seeking. Some network protocols communicate the expected length of the data to be sent, but raw sockets know nothing of this type of information. A common label for seekable streams is random access streams because they can be viewed as huge arrays of data with which we may arbitrarily access elements. To determine whether a stream supports seeking, ask the CanSeek property.

Assuming that you have a random access stream, you use the long Seek(long offset, SeekOrigin origin) method to physically perform the seek operation. The stream logically positions itself at the position indicated by origin and then adds the number of bytes specified by offset. The SeekOrigin enumeration offers three values: Begin, Current, and End.

• Begin positions just before the first byte of the stream. Assuming an offset of 0, the next ReadByte will return the first byte of the stream.

• Current uses the current position in the stream (Position when supported).

• End positions past the last byte of the stream (i.e., a ReadByte will return an EOS). An offset of -1 positions at the last byte of the stream (i.e., ReadByte will return the last byte of the stream).

Positive and negative offsets are supported. Seeking past the end of a stream is illegal, meaning that positive offsets are not compatible with an End origin. Similarly, seeking before the beginning of a stream is illegal, and thus negative offsets are not compatible with a Begin origin. The Seek method returns the new position as an offset from the beginning of the stream.

 using (Stream s = /*...*/) {     s.Seek(8, SeekOrigin.Current);     int b1 = s.ReadByte();     s.Seek(0, SeekOrigin.Begin);     int b2 = s.ReadByte());     s.Seek(-1, SeekOrigin.End);     int b3 = s.ReadByte(); } 

This snippet of code reads 3 bytes. First, it seeks to just before the eighth byte from the current position of the stream (assuming the stream is positioned just before the first byte), and reads it with ReadByte; then it jumps back to the very beginning and reads the first byte of the stream; lastly, it seeks to the byte at the very end of the stream and reads that. Clearly jumping around arbitrarily like this isn't entirely useful. But some file formats specify offsets and sizes (e.g., PE files), using which you will Seek to a specific offset to read a chunk of data.

Closing and Disposing

Since nearly every stream in the Framework encapsulates access to critical system resources, it is imperative that you indicate when you are done with an instance. A FileStream, for instance, holds on to a HANDLE, which might keep a file exclusively locked; if you fail to Close it, the HANDLE will remain open until either the process shuts down or the finalizer thread kicks in. Clearly this situation is not very attractive, considering that a process might remain open for quite some time and that garbage collection is nondeterministic. You should explicitly release the stream's resources by calling either the Close or Dispose method (but not both!).

Note

Interestingly, you actually can't make direct a call to the Dispose method because it is privately implemented on the Stream class. To access it, you must do so through an ID isposable-typed reference, or via the C# using keyword.

The recommended approach to stream cleanup is to wrap any use of a Stream in a C# using block. Of course, for non-C# languages, you should use your language's construct (e.g., stack allocation in C++/CLI); in the worst case, ensure that both success and failure paths manually invoke one of the cleanup methods. This code snippet illustrates the C# technique:

 using (Stream s = /*...*/) {     // Once you leave the block, s will be closed for you.     // There is no need to manually call s.Close()... } 

This construct ensures that Dispose gets called on the Stream at the end of the block. As discussed along with the general notion of disposability in Chapter 5, this ends up inside a finally block to ensure execution in the face of unhandled exceptions. It's not always possible to follow such a simple using-based pattern — for example, when a stream is a member on another type, when its life extends beyond a single block, or when spanning asynchronous invocations. In such cases, you must still take great care to manually call the Close method when you're done using it.

Buffering Reads and Writes

Many Stream implementations provide intrinsic buffering for high performance. The consequence is twofold: (1) buffered streams often read more data than requested, storing the excess in memory and operating on that and only rereading once you have exhausted the buffer; and, (2) writes will not be immediately committed to the backing store but instead held in memory until the buffer size has been exceeded. We will see later how to manually flush writes to the backing store (via the Flush method), sometimes needed to force a write-through immediately.

Buffering can significantly improve performance when working with backing stores that are expensive to access. Files are a great example of this performance win: the cost of accessing the disk is amortized over a larger ratio of in-memory writes to physical to-disk writes. When the cost of writing to a hard disk is several orders of magnitude higher than accessing memory (even worse when considering accesses to processor cache), this is a very important architectural feature. Thankfully, file streams in the .NET Framework employ this technique.

You can wrap any ordinary stream inside an instance of a BufferedStream to guarantee buffers take place. To do so, just create an instance using the BufferedStream constructor, passing in the target Stream that is to be wrapped. Subsequently using the new BufferedStream to perform read and write operations will buffer data, intercepting calls to the underlying stream and adding its own buffering behavior. The BufferedStream(Stream stream) uses a default buffer size of 4K, although a specific buffer size may be supplied through the BufferedStream(Stream stream, int bufferSize) constructor instead.

Flushing Buffered Data

Any streams that buffer writes must flush these buffers to commit writes to the underlying data store. This usually happens once the buffer becomes full or the stream is closed, whichever happens first. In the worst case, if a Stream is left unclosed its Finalize method will ensure that writes get flushed. Under typical circumstances this behavior is sufficient. In some cases, however, you might need to force the accumulated updates out of the buffer and into the underlying store. To do so, make a call to Stream's Flush method.

As noted before, not all streams buffer data. If Flush is called on such a stream, there will be no observable effect. This is not like trying to write to a nonwritable stream, for example, where an exception will be thrown. Flushing a closed stream, on the other hand, is an error.

Asynchronous I/O

The above descriptions of reading and writing applied to synchronous reads and writes only. The Stream class also enables asynchronous reads and writes by using the standard Asynchronous Programming Model (APM) pattern, the general form of which is described further in Chapter 10. The concepts behind this pattern are described in more depth there. The pattern enables highly scalable I/O — for example using I/O Completion Ports on Windows — and permits worker threads to make forward progress in parallel with the stream operation. This can be useful, for example, if you are responding to an event on the UI thread; in such cases, you want to avoid blocking the UI (leading to hangs, "…(Not Responding)" title bars, etc.). We cover it here in just enough detail to understand how it applies in the context of stream-based I/O.

The BeginRead and BeginWrite methods take similar arguments to the Read and Write methods covered earlier. Each has a corresponding EndRead and EndWrite function:

 public virtual IAsyncResult BeginRead(byte[] buffer, int offset, int count,     AsyncCallback callback, object state); public virtual int EndRead(IAsyncResult asyncResult); public virtual IAsyncResult BeginWrite(byte[] buffer, int offset, int count,     AsyncCallback callback, object state); public virtual void EndWrite(IAsyncResult asyncResult); 

An operation on the stream called in this fashion occurs on another thread (typically the ThreadPool, but sometimes doesn't consume a thread at all). Notice that each method takes an additional AsyncCallback and object parameter. Furthermore, each returns an IAsyncResult. These are all used to rendezvous with the completion in one of three ways:

• An AsyncCallback delegate can be supplied that will be called when the I/O request has been completed. This delegate is given access to the results of the operation for processing.

• The IsCompleted property on the IAsyncResult can be polled to determine whether the I/O request has been completed. This returns true or false to indicate this condition. It should not be used to "spin" on completion (e.g., while (!result.IsComplete); is very bad) — please refer to one of the other mechanisms for that.

• You can block the worker thread and wait for completion using the AsyncWaitHandle on the IAsyncResult. This can be explicitly done using the WaitOne method on the WaitHandle. But simply passing the IAsyncResult to the EndXxx method will implicitly block on this handle if IsCompleted is false. Please refer to Chapter 10 for details on wait handles.

If you don't intend to use the callback, pass null for the callback parameter. Similarly, if you don't need to share state across method calls, pass null for state.

The code in Listing 7-1 shows the first of these strategies. The BeginRead is passed a callback pointing at our custom ReadCallback function which, once the asynchronous operation finishes, gets invoked and processes the data. It completes the read operation and obtains the return value by making calling EndRead. Lastly, it writes the processed data and spins up another asynchronous read to get the next block of bytes. This too invokes the callback, and so forth. This continues until the entire stream has been exhausted.

Listing 7-1: Reading I/O asynchronously using a callback

 void BusinessOperation() {     // Perhaps we got the Stream from a FileOpenDialog on the UI.     // We open it and then initiate the read...     Stream s = /*...*/;     // Create a new holder for cross-method information, and kick off the async:     StreamReadState state = new StreamReadState(s, 4096);     s.BeginRead(state.Buffer, 0, state.Buffer.Length, ReadCallback, state);     // Now we can continue working...     // If we're doing some UI operation, the UI won't be blocked while     // the asynchronous I/O occurs. } struct StreamReadState {     public Stream Stream;     public byte[] Buffer;     public StreamReadState(Stream stream, int count)     {         this.Stream = stream;         this.Buffer = new byte[count];     } } void ReadCallback(IAsyncResult ar) {     // This gets called once the BeginRead operation is done.     // Calling the EndRead function gets us the bytes read:     StreamReadState state = (StreamReadState)ar.AsyncState;     int bytesRead = state.Stream.EndRead(ar);     // Now we are free to process the data. For example, we might fill in     // a text-box on the UI, log the data, etc.     if (bytesRead == state.Buffer.Length)     {         // Kick off another asynchronous read, to get the next chunk of data:         state.Stream.BeginRead(state.Buffer, 0, state.Buffer.Length,             ReadCallback, state);     }     else     {         // EOS has occurred. Close the Stream and perform any completion logic.         state.Stream.Close();         // Any logic after the entire I/O operation has occurred goes here.         // You might tear down a progress bar, notify the user, etc.     } } 

You'll also notice in this example that we had to manually create a new StreamReadState type to pass information from the method beginning the asynchronous operation to the callback method completing it. This isn't always necessary. Other types of shared state might have been used, but the single object parameter for BeginRead wasn't sufficient: we needed to access two data structures in this case, the byte[] buffer being filled (to read the data) and the Stream used to initiate the read (so we can call EndRead on it). Wrapping these things up into a dedicated data structure is often the clearest and most straightforward approach.

An alternative approach is to simply wait to be signaled by the stream itself that I/O has completed. Many programmers find the style where your "begin" and "end" logic must be split into two functions unnatural. And it certainly makes figuring out when precisely to Close the Stream less than straightforward. You might write similar code as is shown in Listing 7-2.

Listing 7-2: Reading I/O asynchronously using a blocking wait

 void BusinessOperation() {     using (Stream s = /*...*/)     {         byte[] buffer = new byte[4096];         int bytesRead;         do         {             // Kick off the asynchronous read:             IAsyncResult ar = s.BeginRead(buffer, 0, buffer.Length, null, null);             // Now we can continue working...             // If we're doing some UI operation, the UI won't be blocked while             // the asynchronous I/O occurs.             // Wait for the read to complete:             bytesRead = s.EndRead(ar);             // Now we are free to process the data. For example, we might fill in             // a text-box on the UI, log the data, etc.         }         while (bytesRead == buffer.Length);     } } 

This is admittedly a much simpler approach because state does not have to be shared across asynchronous method calls. The logic is easier to follow. The "Now we can continue working…" section can perform some application update work, such as processing messages, incrementing some piece of UI (a progress indicator perhaps), while the asynchronous work is happening in the background. But beware, there is one (potentially large) problem in that example.

This technique might block upon calling EndRead. If the work consumes less time than the I/O operation does, when we reach EndRead the function will ordinarily wait on the underlying IAsyncResult's WaitHandle. This will put the OS thread into a wait state. Under some circumstances, this can cause problems; most streams will perform a blocking wait in managed code, which does not lead to unresponsive UI problems, but other stream implementations are free to do what they wish. If an EndRead results in blocking without pumping UI messages, the result can be a hung application.

Similarly, assume that our goal is to read the entire stream as quickly as possible. Well, if the work between BeginRead and EndRead is longer than the time it takes to perform the I/O, we've wasted some delta. Specifically, the difference between the time it took to do that work and the time it took for the I/O to complete is time wasted that could have been used on the next read operation. These two facts should make it evident that there is a tradeoff between efficiency and complexity that you will have to make when deciding between the patterns at your disposal.

Readers and Writers

As you've seen above, you can read or write to a stream using raw bytes. The capabilities of the Stream class for these types of operations, however, are admittedly very primitive. Stream's purpose in life is to be a cleanly factored abstraction on top of which more powerful abstractions can be built. Thus, in the vast majority of cases, you will actually want to use the higher-level reader and writer classes. Although readers and writers can operate on non-Stream backing stores (for example, we'll see shortly a set of classes that enable you to treat a string as a data source), they are highly related to streams.

There are two general families of readers and writers in the Framework: text and binary. The text-based classes perform automatic encoding and decoding of text, solving the tricky problem briefly touched on earlier in this chapter: how to accurately convert raw bytes back into their encoded format upon reading (and doing the reverse while writing). Similarly, the binary-based classes enable you to read and write values in the underlying stream as any arbitrary primitive data-type, taking care of tedious conversions and varying byte-sized reads and writes for you.

Reading and Writing Text

The text family of readers and writers derive from two common abstract base classes: System.IO .TextReader and TextWriter. Their sole purpose is to supply a unified and common interface for generic read operations against a textual data source. Two sets of implementations are available that derive from these common types, allowing for stream (StreamReader and StreamWriter) and string (StringReader and StringWriter) operations. The former consumes any arbitrary Stream as its backing store, while the latter makes use of a String or StringBuilder. We'll look at each of these in detail.

Using Streams As a Backing Store

We saw how to read raw data off of a Stream itself earlier in this chapter. To appreciate the utility of StreamReader, consider what code we'd have to write and maintain just to read standard 16-bit Unicode characters from a backing store such as a file. "OK," so you say, Unicode is easy. All you have to do is read bytes in pairs, combine them in the correct order (remember: endian ordering matters), and you'd be able to construct the right chars. What about UTF-8? You would have to know how to recognize single- or many-byte sequences and convert them into the respective double-byte representation, among other things. And all this work is just to support two of the many possible encodings. We discuss encodings in more detail in Chapter 8.

Reading encoded text from a Stream is a very common activity. The StreamReader type performs the necessary decoding logic so that you can work with raw chars and/or strings. An instance is constructed by passing in either a Stream instance to its constructor or alternatively a string-based file path, in which case a Stream is silently constructed for you. There are several more complete overloads available that take such things as a specific Encoding, a buffer size, and whether to detect byte-order-marks (BOMs, described in Chapter 8).

Once you created a reader, it is very much like working with a raw Stream that returns 16-bit characters instead of 8-bit bytes. You can either read a single char or a buffer of chars with a single method call. The Read method will obtain the next character from the underlying Stream, returning -1 if the EOS has been reached (using int to provide the signed capacity). Read(char[] buffer, int index, int count) reads count characters from the stream into buffer starting at the specified array index. It returns an integer to indicate how many characters were read, where 0 indicates EOS. Lastly, the Peek method will read ahead by a single character but doesn't advance the position of the stream.

This example reads and prints out a UTF-8 encoded Stream's contents one character at a time:

 Stream s = /*...*/; using (StreamReader sr = new StreamReader(s, Encoding.UTF8)) {     int readCount;     char[] buffer = new char[4096];     while ((readCount = sr.Read(buffer, 0, 4096)) != 0)     {         for (int i = 0; i < readCount; i++)         {             Console.Write(buffer[i]);         }     } } 

This piece of code looks much like the Stream-based example shown above. You can, of course, construct a new string instance using the char[] data, too, in order to perform more complex string-based manipulation. For example, new string(buffer) will generate a new string based on the character array; you must be careful to account for the case where the buffer's entire contents were not filled by the method (e.g., on the last read).

There are also a couple convenient methods to make reading and working with strings more straightforward. ReadLine will read and return characters up to the next newline character sequence, omitting the newline from the String returned. This method returns null to indicate EOS. The following code snippet is functionally equivalent to the one above:

 Stream s = /*...*/; using (StreamReader sr = new StreamReader(s, Encoding.UTF8)) {     string line;     while ((line = sr.ReadLine()) != null)     {         Console.WriteLine(line);     } } 

Another method, ReadToEnd, similarly returns a string instance. But this time, the reader reads from the current position to the very end of the underlying stream. It too returns null to indicate EOS. The above two samples can be consolidated into the following:

 Stream s = /*...*/; using (StreamReader sr = new StreamReader(s, Encoding.UTF8)) {     Console.WriteLine(sr.ReadToEnd()); } 

Note that the File.ReadAllText function — described further below — can do the same in just one line of code! Because ReadToEnd actually pulls the entire file contents into memory as a string, this can be an extremely inefficient mechanism to process large files. In such cases, using a single fixed-size buffer ensures that the memory usage of your application does not increase linearly with respect to the size of the file your program operates against. With that said, for small-to-midsized files, reading the entire text in one operation will result in much less overhead due to less disk seek time and fewer API calls down through the kernel.

Just as the StreamReader class performs decoding of input as it is read, StreamWriter handles encoding CLR strings and chars into the correct byte sequences during write operations. It too wraps an existing Stream instance and takes an Encoding value to indicate how to perform encoding of characters, both passed into its constructor. As with its sibling reader, this type performs buffering of writes. The size of its internal buffer can be customized by passing an integer bufferSize to the appropriate constructor overload. Also note that setting the AutoFlush property to true will force a Flush operation after every write.

Working with a StreamWriter is much like writing with a Stream, although there are many more overloads for the writing operations. There are two primary write operations, Write and WriteLine, the latter of which is equivalent to a Write followed by a call to Write(Environment.NewLine). Each has a large number of overloads, one for each primitive value type (bool, int, long, and so on), as well as string- and object-based ones. For the primitive and string overloads, the written result is the value passed in, while the object overload writes the result of calling ToString on the argument (or nothing if the argument is null):

 Stream s = /*...*/; using (StreamWriter sw = new StreamWriter(s, Encoding.UTF8)) {     sw.Write("Balance: "); // string overload     sw.Write(30232.30m); // decimal overload     sw.Write(", HasSavings? "); // string overload (again)     sw.Write(true); // bool overload     sw.WriteLine(‘.'); // char overload     // Etc... } 

Write also has two char[] overloads, which mirror those available with the reader class. Write(char[] buffer) writes the entire contents of buffer to the stream, and Write(char[] buffer, int index, int count) writes count characters from buffer starting with the element at position index.

Lastly, there is a set of overloads to Write and WriteLine, which provides a convenient way to access String.Format-like functionality. The above example of using StreamWriter to output a set of variable values mixed with strings could have been written more compactly as:

 sw.WriteLine("Balance: {0}, HasSavings? {1}.", 30232.30m, true); 

We discussed formatting in general in Chapter 5 and the various internationalization issues that can arise in Chapter 8. Please refer to coverage there for more information about using formatting functions in the Framework.

When you are finished with a reader or writer, you should either call Close or Dispose, just as you would a Stream. In response, the underlying Stream's Close method will be invoked. If you intend to have multiple readers or writers on the same shared stream, you will have to orchestrate the closing of the underlying stream outside of the reader's scope of responsibility.

Using Strings As a Backing Store

The StringReader type enables reading from a string using the same familiar TextReader interface. All of the methods on StringReader are identical to the operations already discussed in the context of text readers; thus, this section presents only a few simple examples of its usage:

 String contents = "..."; using (StringReader reader = new StringReader(contents)) {     int c;     while ((c = reader.Read()) != -1)     {         Console.Write("{0} ", (char)c);     } } 

A StringReader is instantiated by passing an instance of the string from which it is to read. In this example, we simply read through one character at a time, printing out each to the console. Since all strings in the Framework are Unicode natively, clearly there is no need to supply an Encoding as you would with a StreamReader.

Just as with other types of readers, you can read from multi-line strings a line at a time, as follows:

 string contents = "..."; using (StringReader reader = new StringReader(contents)) {     int lineNo = 0;     string line;     while ((line = reader.ReadLine()) != null)     {         Console.WriteLine("Line#{0}: {1}", ++lineNo, line);     } } 

Just as with StringReader, StringWriter mimics the corresponding TextWriter type's interface. This writer uses a StringBuilder as its backing store, enabling you to modify it through a writer interface and subsequently retrieve the contents as a StringBuilder instance. Again, the functionality offered by this type does not diverge from the base TextWriter interface, and thus I will only show a few short examples of its use.

 StringWriter writer = new StringWriter(); writer.Write("Name: {0}, Age: {1}", "Henry", 32); Console.WriteLine(writer.ToString()); 

In this snippet, we create and write some simple formatted data to the writer. The constructors for StringWriter range from the simple no-argument version, which creates an underlying StringBuilder instance for you, to the complex, where you can supply a prebuilt StringBuilder and/or a custom IFormatProvider. Once data has been written to a writer, there are two ways to access the string: the ToString method converts the contents into a string and returns it, while GetStringBuilder returns the underlying builder which you can work with further.

Reading and Writing Binary Data

Reading and writing data with the System.IO.BinaryReader and BinaryWriter types provides finer-grained control over the data in the backing store — much like Stream — while still offering a higher level of abstraction. You are able to work with bytes in either raw form or as natively encoded primitive data types. The capability to work with binary data is useful when consuming or producing files conforming to precise file formats or custom serialization of binary data.

Unlike the text-based classes that derive from a common set of abstract base classes and that have multiple implementations, there are just the two simple concrete types BinaryReader and BinaryWriter. Many of the idioms you will see are comparable to the text-based means of reading and writing, so coverage of the straightforward operations will be a bit lighter than above. Reading and writing operations are symmetric, meaning that anything written as a specific data type can be read back as that data type.

Note

Note that .NET Framework supports a variety of object serialization constructs. The binary serialization infrastructure — found in System.Runtime.Serialization.Formatters.Binary — relies on the BinaryReader and BinaryWriter types. Custom binary serialization offers power and flexibility to interoperate with predefined data formats, however, so it is has a plethora of useful scenarios. We don't explicitly discuss serialization in detail in this book.

Both the reader and writer types operate on streams and hence are very similar to the StreamReader and StreamWriter classes outlined above. BinaryReader offers a set of T ReadT(...) operations, where T is some primitive data type; for example, bool ReadBoolean() reads a single byte from the underlying stream and constructs a bool from it (that is, 0x0000 turns into false, and everything else true). There are identical overloads for each primitive — such as char, int, double, and so forth — which similarly interpret and convert the data for you. The ReadByte and ReadBytes methods enable you to work with raw, uninterpreted data.

Most of the read operations are straightforward, consuming n bytes from the stream at a time, where n is the size of the primitive's CLR storage size. If there are less than n bytes available in the underlying stream when calling one of these methods, an EndOfStreamException will be generated and should be treated as an ordinary EOS situation (the use of an exception here is unfortunate). A few methods do not follow this same pattern, however: the buffer-based reads, such as Read(byte[] buffer, int index, int count), for example, act just like the Stream-based read methods, returning a value less than count to indicate that the end of the stream has been reached. Similarly, the array-based reads, such as byte[] ReadBytes(int count), will simply return an array with a size less than count to indicate EOS.

The character- and string-based overloads will consume a varying number of bytes based on the encoding supplied to the BinaryReader constructor. If using UTF-8 as the encoding (the default), for example, a single ReadChar could result in a varying number of bytes being read depending on the code-point of the underlying character. The ReadString method can consume an open-ended number of characters. It employs a convention of interpreting the first byte as a 7-bit encoded integer indicating the length of the string that follows and uses that to determine how many bytes to read. This is referred to as length prefixing and is symmetric with the way the WriteString method of the BinaryWriter type works. This makes consumption of BinaryWriter-generated string data straightforward but for strings serialized any other way requires that you determine the length and read the individual characters using some alternative convention.

As an example of these APIs, imagine that we have to work with a custom binary file format containing a set of serialized employee records. This might be for legacy integration or simply as an efficient on-disk serialization format for large sets of data. Say that we have a type Employee defined as follows:

 struct Employee {     public string FirstName;     public string LastName;     public int Extension;     public string SocialSecurityNumber;     public bool Salaried; } 

If the convention were to (1) write a length to indicate the total number of records and (2) serialize each record by sequentially writing each field using the native data sizes and without any sort of delimiters, deserializing a collection of employees instance from a Stream would be as simple as follows:

 List<Employee> DeserializeEmployees(Stream stream) {     BinaryReader reader = new BinaryReader(stream);     int count = reader.ReadInt32();     List<Employee> employees = new List<Employee>(count);     for (int i = 0; i < count; i++)     {         Employee e = new Employee();         e.FirstName = reader.ReadString();         e.LastName = reader.ReadString();         e.Extension = reader.ReadInt32();         e.SocialSecurityNumber = reader.ReadString();         e.Salaried = reader.ReadBoolean();         employees.Add(e);     }     return employees; } 

This code reads a single Int32 to indicate the number of records we expect; the function doesn't handle EOS explicitly because it indicates corrupt data (unless it occurs on the first read, which is a problem the caller should deal with). We then read primitives, copy them to the appropriate field, and add each populated Employee record to the list. We discuss the serialization process and corresponding SerializeEmployees method in the BinaryWriter section that follows.

Like BinaryReader, the BinaryWriter enables you to write to streams using raw binary-encoded data. It offers a large number of Write overloads in order to write out a variety of primitive data types, including byte[] and char[]write overloads. In most cases, a Write simply outputs data in the native memory format of the specified data type. However, for character and string data, the writer will use the encoding provided at construction time to determine how data is laid out in the backing store. Moreover, strings are prefixed with a 7-bit encoded integer representing its length, as noted above. Multi-byte data structures are written in little endian order.

To complete the example from above, let's take a look at the corresponding SerializeEmployees method, which creates the data that DeserializeEmployees consumes:

 void SerializeEmployees(Stream s, ICollection<Employee> employees) {     BinaryWriter writer = new BinaryWriter(s);     writer.Write(employees.Count);     foreach (Employee e in employees)     {         writer.Write(e.FirstName);         writer.Write(e.LastName);         writer.Write(e.Extension);         writer.Write(e.SocialSecurityNumber);         writer.Write(e.Salaried);     } } 

Binary readers and writers also support flushing and closing of the underlying stream through the Flush, Close, and Dispose methods.

Files and Directories

Working with files and directories stored on disk are among the most common I/O tasks System.IO can be used for. Reading from and writing to files are just as simple as instantiating the right type of Stream and reading through it using the tools and techniques you've already seen detailed above. Additional APIs beyond this enable accessing and writing to directory and file attributes stored in the file system, including timestamps and security Access Control Lists (ACLs). This section will demonstrate how to access and manipulate such information.

Opening a File

There are countless ways to open a file in the .NET Framework. In fact, you could say that there are too many (and you'd be absolutely correct): File, FileInfo, FileStream, StreamReader, and StreamWriter all provide mechanisms to do so. To reduce the onset of massive confusion, we'll focus on using the static File class. This type offers a broad set of methods with which to open files. The other types offer constructors that take string filenames and a set of other open options common to the File APIs. FileInfo offers several methods (named Open*) much like the File methods we are about to discuss. So the discussion below about arguments to the File operations pertains to the other means of opening files equally as much.

A "file" itself is not an object that you will instantiate and work with in the Framework (although the FileInfo class does get us close). Instead, open file HANDLEs are encapsulated inside a FileStream instance. File is a static class that exposes a set of factory and utility methods, each of which enables you to open and perform common file-based activities. If you wish to obtain data like the file's creation time, for example, File provides it through a collection of methods, all of which take the string filename as an argument.

When opening a file, the File class will obtain a file HANDLE from the OS (via the OpenFile Win32 API) with certain rights to perform read and/or write operations. Permissions are negotiated based on the current process's identity, or thread impersonation token, and the relevant NTFS security attached to the target file. (You can manipulate these ACLs programmatically. We'll see how to access them in this chapter, but detailed discussion of file ACLs can be found in Chapter 9.) This file HANDLE gets wrapped up in and assigned to a FileStream instance. When the stream is closed, either through manual (Close, Dispose) or automated (Finalize) cleanup, the HANDLE is closed (via the CloseHandle Win32 function).

As hinted at above, using the Open-family of methods on the File class is the easiest way to open and work with files on disk. They all take a string path to the file, which can be specified as either an absolute or relative to the current working directory (i.e., Environment.CurrentDirectory). They return a new FileStream. The simplest of them, Open, is the most flexible in terms of which arguments it accepts. It offers overloads that accept combinations of mode, access, and share enumeration values.

The mode parameter represents the OS file mode in which the file is opened. This is the only required parameter. It is represented by a value of type FileMode, the possible values of which are shown in the table below.

Open table as spreadsheet

FileMode Value	Description
OpenOrCreate	It specifies that if a file exists at the target path, it will be opened; otherwise, a new file is created and opened.
Create	Requests that a new file is created regardless of whether one exists already.If a file already exists at the target path, it will be overwritten.
CreateNew	Creates a new file and opens it. If a file already exists at the specified location, an IOException will be thrown.
Open	Opens an existing file, and throws a FileNotFoundException if no file exists currently at the specified location.
Append	Behaves just like OpenOrCreate (in that it opens if it exists, creates it if it doesn't) but immediately seeks to the end of the file after opening. This enables you to begin writing at the very end of the file easily.
Truncate	Behaves like Open, but clears the file's contents upon opening.

The access parameter is of type FileAccess. Specifying this explicitly ensures the HANDLE is opened with the correct permissions for the operations you intend to issue against it. ReadWrite is the default unless specified manually, allowing both read and write operations. There are also Read and Write values if you intend to do one or the other.

Lastly, the share parameter, an enumeration value of type FileShare, allows you to control the behavior should another program (or a concurrent part of your own program) try to access the same file simultaneously. None is the default when opening a file for Write or ReadWrite access, meaning that concurrent access is disallowed during writes. The other values ReadWrite, Read, and Write specify that other code may perform that particular operation while you have the HANDLE open.

For example, this code opens some files using various combinations of these parameters:

 using (FileStream fs1 =     File.Open("...", FileMode.Open) {     // fs1 is opened for     //   FileAccess.ReadWrite (default) and     //   FileShare.None (default)     /*...*/ } using (FileStream fs2 =     File.Open("...", FileMode.Append, FileAccess.Write)) {     // fs2 is opened for     //   FileAccess.Write     //   FileShare.None (default)     // and is positioned at the end (because of FileMode.Append)     /*...*/ } using (FileStream fs3 =     File.Open("...", FileMode.Truncate, FileAccess.ReadWrite, FileShare.Read)) {     // fs3 is opened for     //   FileAccess.ReadWrite     //   FileShare.Read     // and has had its entire contents truncated (due to FileMode.Truncate } 

Two convenient related methods are available: OpenRead and OpenWrite just take a single string filename and return FileStream with FileAccess.Read and FileAccess.Write, respectively. They are opened with FileMode.OpenOrCreate and FileShare.None. In many cases, you'll want to wrap the stream in either a reader or writer to simplify logic; OpenText opens a stream for read-only access and automatically wraps and returns it in a StreamReader for you:

 using (FileStream fs1 = File.OpenRead("...")) { /*... */ } using (FileStream fs2 = File.OpenRead("...")) { /*...*/ } using (FileStream fs3 = File.OpenWrite("...")) { /*...*/ } using (StreamReader sr = File.OpenText("...")) { /*...*/ } 

Aside from opening a new file using Open, you can create an entirely new empty file with the Create method. It takes a path argument specifying the absolute or relative location and will return a FileStream referencing the newly created file. This is just a convenient wrapper over a call to Open(FileMode.Create, FileAccess.ReadWrite). Similarly, CreateText creates a new file and returns a StreamWriter.

Additional File Operations

Regardless of how you have opened the FileStream, reading from and writing to it occurs precisely as we've already seen in the above text. Whether this is raw Read, Write, BeginRead, and similar calls to the Stream itself, or more complex interactions through the StreamReader or StreamWriter types, there is nothing specific to working with files that you need to be concerned about. FileStream does have a few interesting operations, however, beyond the general Stream functionality already discussed.

GetAccessControl and SetAccessControl on both the FileStream and FileInfo types surface the underlying Windows ACLs that protect the file opened. We discuss ACLs in more detail in Chapter 8.

The void Lock(long position, long length) method ensures exclusive write access to the identified segment of the file (position through position + length), which can be useful if you've chosen to allow concurrent writes to a file (e.g., by supplying a specific FileShare value to the Open method). Reads are still allowed. The corresponding void Unlock(long position, long length) method undoes this lock. This enables finer granularity over the locking of files rather than locking the entire file, as with FileShare.None.

A Word on I/O Completion Ports

Windows provides a feature called I/O Completion Ports to enable highly scalable asynchronous I/O with maximum throughput. It's particularly useful on servers to accommodate large workloads. When a file is opened for overlapped I/O and associated with a completion port, a thread needn't even be used to wait for a given operation. Instead, an entire pool of threads is associated with each completion port, and once any I/O completes, one of the threads is awoken and handed the buffer for processing. The .NET Framework ThreadPool dedicates a configurable number of I/O Completion threads to a sole completion port which it manages.

When you use asynchronous file I/O on a FileStream (i.e., asynchronous BeginRead and BeginWrite via the FileStream class), it can implicitly use I/O Completion Ports without significant work on your behalf. In order for this to occur, you need to indicate that the FileStream should open the file for overlapped I/O. This can be done with FileStream's constructor overloads in two different ways: pass in true for the bool useAsync parameter or pass in FileOptions.Asynchronous for the options parameter. You can't do either using the File type that we used for examples above. Doing this causes any callback and/or IAsyncResult handle-signaling to occur on one of the ThreadPool I/O Completion threads.

You can manually interact with I/O Completion Ports in a number of ways. A detailed description is (unfortunately) outside of the scope of this book. Any HANDLE opened for overlapped I/O (e.g., by specifying FILE_FLAG_OVERLAPPED when calling the Win32 CreateFile function) can be wrapped in a System.Threading.Overlapped instance and can be bound to the ThreadPool's Completion Port with the ThreadPool's BindHandle API. Taking a leap further, you can even create additional ports, associate threads for completion manually, and bind handles yourself. This requires significant interoperability with Win32 functions through the P/Invoke layer. Future versions of the Framework will likely ship with improved managed APIs to more flexibly interact with completion ports.

File-System Management

A set of methods spread across the FileInfo and DirectoryInfo types (and some on FileStream) permit you to interact with the file system. This includes both inspecting and modifying attributes on NTFS files and directories. Each of the info types is instantiated with a string path argument representing the file or directory path for which the type will be used. Unlike, say, the File class, you must actually construct instances to perform operations against them.

For example, DirectoryInfo permits you to create a new directory (with Create) or subdirectory (with CreateSubdirectory). It also enables you to enumerate subdirectories or files inside of a directory:

 DirectoryInfo root = new DirectoryInfo(@" C:\Program Files\"); // Note: AllDirectories traverses all sub-directories under the root: DirectoryInfo[] dirs = root.GetDirectories("*", SearchOption.AllDirectories); foreach (DirectoryInfo subDir in dirs) {     // ... } FileInfo[] files = root.GetFiles(); foreach (FileInfo file in files) {     // ... } 

In addition to those types, the Path class surfaces a set of static methods that make working with NTFS file-system paths simpler. It simply takes strings and performs comparisons and modifications on them so that you needn't do it by hand. It also has a set of fields which expose file-system constants:

 public static class Path {     // Methods     public static string ChangeExtension(string path, string extension);     public static string Combine(string path1, string path2);     public static string GetDirectoryName(string path);     public static string GetExtension(string path);     public static string GetFileName(string path);     public static string GetFileNameWithoutExtension(string path);     public static string GetFullPath(string path);     public static char[] GetInvalidFileNameChars();     public static char[] GetInvalidPathChars();     public static string GetPathRoot(string path);     public static string GetRandomFileName();     public static string GetTempFileName();     public static string GetTempPath();     public static bool HasExtension(string path);     public static bool IsPathRooted(string path);     // Fields     public static readonly char AltDirectorySeparatorChar;     public static readonly char DirectorySeparatorChar;     public static readonly char PathSeparator;     public static readonly char VolumeSeparatorChar; } 

We'll now take a look at some more common file-system operations requiring detailed coverage.

Copying and/or Moving Files and Directories

You could easily write code using techniques shown above to manually copy bytes between FileStreams, effectively copying or moving a file. However, copying a file on Windows actually entails copying any ACLs and extended attributes, so it's actually more involved than it appears at first glance. Luckily, there are methods on File that perform both of these activities for you: Copy and Move. The FileInfo class has similar methods CopyTo and MoveTo on it which use the file itself as the source.

Open and Move both accept two string arguments: a sourceFileName and destinationFileName. Copy also has an override, using which you may indicate whether it should overwrite an existing file — if it exists — at destinationFileName. The default is false. You should also note that Move is typically just a rename (when the source and destination are on the same volume) and is very efficiently implemented. In such cases, it does not actually copy bytes at all; it simply updates a directory table entry. Note that moving files across volumes does not carry security descriptors along for the ride.

You can also move an entire directory using the DirectoryInfo type's MoveTo method. Like FileInfo.MoveTo, it takes a string destination path and uses a very efficient mechanism to move the directory, often just modifying a file-system table entry.

Deleting Files and Directories

File's Delete method will delete a file from the disk permanently. You may also accomplish the same thing by using FileInfo's Delete method on an instance. These functions do not send the to the Recycle Bin. If you wish to send something to the Recycle Bin instead of permanently deleting it, you'll have to P/Invoke to the Win32 SHFileOperation function. This can help your applications to play nicely in with the Windows environment — permitting your users to restore data at a later point if necessary — but comes at a cost due to the lack of built-in Framework functionality.

The DirectoryInfo type also offers a Delete method. It has two variants, one with no parameters and the other that accepts a bool to indicate whether to perform a "recursive delete." If the former is called, or the latter with false, the target directory is removed only if it contains no files. If a recursive delete is specified, any files and subdirectories inside the target directory will be removed recursively.

Temporary Files

Software tasks often require temporary files. But in most cases, you ordinarily don't particularly care what the file is named and would prefer not to have to worry about the directory in which it is created. The static GetTempFileName method on the System.IO.Path type requests that the OS allocates a new 0-byte temporary file, and returns a string representing its path. This is guaranteed to be unique and avoids clashing with other programs that are also using temporary files.

The directory in which the file is created is based on the Path.GetTempPath method, which you can access directly. It is based on one of the following values (whichever is found first): the TMP environment variable, the TEMP environment variable, the USERPROFILE environment variable, the Windows root directory.

Once you obtain the path to the temporary file, you can use it to open a FileStream. The OS will auto delete the file eventually, but you are encouraged to manually do so once you are done using it. This can help to avoid cluttering the user's disk with temporary files that must then be cleaned up by the system.

Change Notifications

The FileSystemWatcher class offers a mechanism to monitor for and be notified of any changes to a particular location on the file system. This type uses the same mechanisms in the OS that antivirus and other similar pieces of software use in order to perform an operation each time an activity takes place on the file system. The watcher offers a set of events to which you may subscribe: Created, Changed, Renamed, and Deleted. When constructing an instance, you pass in a directory path to watch and an optional file filter. For example, new FileSystemWatcher(@" c:\", "*.exe") will monitor the C:\ directory for any events pertaining to *.EXE-named files. The FileSystemWatcher class can be configured through properties to indicate whether to use subdirectories (IncludeSubdirectories), the type of changes that trigger an event (NotifyFilter), and the number of events to buffer (InternalBufferSize).

The code snippet in Listing 7-3 demonstrates asynchronous monitoring of a variety of file-system events that might be interesting. The Created, Changed, and Deleted events use the same void FileSystem EventHandler(object, FileSystemEventArgs) signature, and the event arguments indicate which type of event was raised. This enables you to handle all of these events inside the same handler. Renamed uses a different signature, as it provides additional information about the old filename before the rename occurred (e.g., OldFullPath and OldName, representing the path and filename before the rename occurred, respectively).

Listing 7-3: Watching for filesystem events

 FileSystemWatcher watcher; void SetupWatcherEvents() {     watcher = new FileSystemWatcher(@" c:\", "*.exe");     watcher.Created += OnCreatedOrChanged;     watcher.Changed += OnCreatedOrChanged;     watcher.Deleted += OnDeleted;     watcher.Renamed += OnRenamed; } void OnCreatedOrChanged(object sender, FileSystemEventArgs e) {     switch (e.ChangeType)     {         case WatcherChangeTypes.Created:             // Logic for creation...             Console.WriteLine("‘{0}' created", e.FullPath);             break;         case WatcherChangeTypes.Changed:             // Logic for changes...             Console.WriteLine("‘{0}' changed", e.FullPath);             break;     } } void OnDeleted(object sender, FileSystemEventArgs e) {     // Note: we used a different handler; OnCreatedOrChanged could have been used.     // Logic for deletion...     Console.WriteLine("‘{0}' deleted", e.FullPath); } void OnRenamed(object sender, RenamedEventArgs e) {     // Logic for rename...     Console.WriteLine("‘{0}' renamed to ‘{1}'", e.OldFullPath, e.FullPath); } 

Instead of registering events, you can wait synchronously for an event to occur through the WaitFor Changed method. This method blocks until a new file-system event occurs. You may also specify a flags-style enumeration value of type WatcherChangeTypes to constrain the events that will wake up the blocking wait. The method returns a WaitForChangedResult containing information very similar to the FileSystemEventArgs type shown above:

 FileSystemWatcher watcher = new FileSystemWatcher(@" c:\", "*.exe"); while (true) {     // We're only interested in change or rename events:     WaitForChangedResult result =         watcher.WaitForChanged(         WatcherChangeTypes.Changed | WatcherChangeTypes.Renamed);     // Once we get here, a change has occurred:     switch (result.ChangeType)     {         case WatcherChangeTypes.Changed:             Console.WriteLine("{0}: File '{1}' was changed",                 DateTime.Now, result.Name);             break;         case WatcherChangeTypes.Renamed:             Console.WriteLine("{0}: File ‘{1}' was renamed to ‘{2}'",                 DateTime.Now, result.OldName, result.Name);             break;     } } 

Note that one slight drawback of this approach is that additional events occurring while you are processing an event might slip past your handler code. WaitForChangedResult synchronously receives an event; any events that occur while this method is not blocked (e.g., in the switch block above) will be lost if you don't have asynchronous event handlers set up to watch for them.

Other Stream Implementations

There are several concrete implementations of the abstract Stream base class aside from just FileStream. Many of these extend Stream's behavior in minor ways, the major purpose being to hook up to and orchestrate reads from and writes to specific types of backing stores. This section provides an overview of the most common of these types, discussing only where their operations differ from the base Stream type. Coverage of the NetworkStream type can be found below.

Other stream types are omitted when they would require a lengthy discussion of technologies outside of the scope of this book. For example, System.Security.Cryptography.CryptoStream is not discussed; while very useful, enabling you to read files from NTFS's Encrypted File System (EFS) capabilities, it is highly dependent on the cryptography APIs. Likewise, the System.Net.AuthenticatedStreams, NegotiateStream and SslStream, would require discussion of network authentication protocols. Nonetheless, they are very useful for some scenarios. Please refer to SDK documentation for details on these types.

Buffered Streams

A buffered stream is simply a wrapper on top of an existing stream, placing simple read and write buffering in between. This can be useful for streams that do not intrinsically use buffering. The operations provided by the BufferedStream class are identical to the base Stream type, and its sole constructor takes a Stream instance to wrap. Because this is nothing but a simple wrapper over an underlying stream, its level of support for reading, writing, and seeking is inherited from its child stream.

Compressed Streams

The streams in the System.IO.Compression namespace (System.dll assembly) provide support for compression and decompression of data using the popular Deflate and GZIP algorithms, via the types DeflateStream and GZipStream, respectively These are well-known, lossless algorithms for arbitrary binary data. They can be used for more efficient storage at the cost of overhead of CPU time during compression while writing and decompression while reading. Deflate is described by RFC 1951, and GZIP is described by RFC 1952; GZIP by default uses the same algorithm as Deflate, but is extensible to support alternative compression algorithms. Both implementations are identical in the Framework for 2.0, and therefore we discuss only GZipStream below.

The GZipStream type wraps an underlying stream used as the backing store, and performs compression and decompression functions transparently while you work with it. When constructed, the Stream to wrap and a CompressionMode must be specified. The mode tells the stream which operation to perform on a read or write, the two possible values of which are Compress or Decompress. If Compress is specified, all data written to the stream is compressed and reading from the stream is prohibited. Conversely, if Decompress is specified, all data read from the stream is decompressed and writing to the stream is prohibited.

All operations are otherwise performed as you would with a normal Stream. This makes reading a decompressing file, for example, as simple as wrapping a FileStream with a GZipStream and plugging it into existing Stream-based logic:

 using (GZipStream s =     new GZipStream(File.OpenRead("..."), CompressionMode.Decompress)) using (StreamReader sr = new StreamReader(s)) {     string line;     while ((line = sr.ReadLine()) != null)     {         Console.WriteLine(line);     } } 

This small snippet of code actually decompresses a file as it reads it, and writes the result to the console. It is just as simple to compress files as they are written.

Memory Streams

A MemoryStream uses a managed block memory as its backing store, represented as a simple array of bytes, that is, a byte[]. Memory streams support reading, writing (assuming the writable constructor argument isn't false), and random access (seeking). A number of constructor overloads are available. MemoryStream(int capacity) creates an array with a number of elements equal to capacity. Several overloads take a byte[] as an argument, enabling you to use the provided array as the backing store; a copy is not made upon instantiation, so the internal buffer that gets used by the stream is the same instance passed to the constructor. Overloads exist that take a publiclyVisible argument (true by default), which is used to control whether references to the internal buffer will be handed out by GetBuffer; if false, any call to this method will throw an UnauthorizedAccessException.

With MemoryStream, you can perform some very powerful memory mapping operations. For example, you can take a network- or file-based source and pull its entire contents into a block of managed memory:

 using (MemoryStream ms = new MemoryStream()) {     using (FileStream fs = /*...*/)     {         byte[] block = new byte[4096];         int read;         while ((read = fs.Read(block, 0, block.Length)) != 0)         {             ms.Write(block, 0, read);         }     }     // Do something interesting with ‘ms'... } 

Very much like the MemoryStream class, UnmanagedMemoryStream uses a chunk of memory as its backing store. The store, however, is a section of unmanaged memory represented as an unmanaged byte pointer, that is byte*, instead of memory that is managed by the CLR's GC. This is useful for interoperating with unmanaged code in addition to raw pointer-based arithmetic.

To hook up an UnmanagedMemoryStream to its backing store, you simply create an instance, passing in an unmanaged byte* to the beginning of the memory segment and the length of the segment (in number of bytes) as arguments. Reading and writing is then constrained to this chunk of memory. Because the target memory is of a fixed well-known size and location, this type of stream supports random access (seeking). Moreover, the PositionPointer property retrieves a byte* referring to the stream's current position; it can be used to seek to any arbitrary pointer that is within the stream's region of memory.