12.10 Improving Performance with a MemoryStream

12.10 Improving Performance with a MemoryStream

You want to eliminate the performance bottleneck that occurs when continuously writing large amounts of data to a file by working with a memory-based stream.

Technique

The memory stream is a specialized stream class that allows you to work with an in-memory representation of a file. The MemoryStream class contains the usual methods to read and write different data types but also contains several methods that allow you to set the current position of the stream anywhere within the defined memory block. You can create a MemoryStream object by passing in a prebuilt array of bytes, an integer value, or no parameters. When you pass in an array of bytes, the MemoryStream is limited to working with that array only. Any attempts to write past the bounds of the array result in an exception being thrown. If you pass an integer or no parameters into the MemoryStream constructor instead, the internal memory buffer expands as necessary to accommodate extra data. You can pass in an integer value specifying the initial size of the internal buffer.

The MemoryStream class also contains an internal position representing the current location where reading and writing takes place. By using the Seek method, you can move that position relative to the beginning, end, or current position. For instance, to move the current stream location 1KB from the beginning of the memory stream, call Seek passing the integer 1024 and the SeekOrigin value of SeekOrigin.Begin . The other SeekOrigin values are SeekOrigin.End and SeekOrigin.Current . The code in Listing 12.1 shows how to use a MemoryStream class to quickly manipulate the contents of a file. The application itself reads in a file specified on the command line, places the file's data into a MemoryStream object, and then proceeds to swap the beginning bytes with the ending bytes, which has the effect of reversing the bytes in the file. When the reverse is finished, the MemoryStream is written to a new file using a BinaryWriter .

Listing 12.1 Reversing a File with a MemoryStream

 using System; using System.IO; namespace _10_MemoryStream {     class Class1     {         [STAThread]         static void Main(string[] args)         {             if( args.Length <= 0  args.Length > 2 )             {                 DisplayUsage();                 return;             }             if( File.Exists( args[0] ) == false )             {                 DisplayUsage();                 return;             }             ReverseFile( args[0], args[1] );             return;         }         static void ReverseFile( string src, string dest )         {             // open src             FileStream fsSrc = File.OpenRead( src );             BinaryReader rdr = new BinaryReader( fsSrc );             // create dest             FileStream fsDest = File.Open( dest, FileMode.Create );             // create memory stream             MemoryStream memStream = new MemoryStream();             int curChar = 0;             // read source into memory stream             while( (curChar = rdr.Read()) != -1 )             {                 memStream.WriteByte( Convert.ToByte(curChar) );             }             for( int i = 0; i < memStream.Length/2; ++i )             {                 byte tempTop, tempBottom;                 // get top byte                 memStream.Seek( i, SeekOrigin.Begin );                 tempTop = (byte) memStream.ReadByte();                 // get bottom byte                 memStream.Seek( -i-1, SeekOrigin.End );                 tempBottom = (byte) memStream.ReadByte();                 // set new bottom byte                 memStream.Seek( -1, SeekOrigin.Current );                 memStream.WriteByte( tempTop );                 // set new top byte                 memStream.Seek( i, SeekOrigin.Begin );                 memStream.WriteByte( tempBottom );             }             // write memory stream to disk             memStream.Seek( 0, SeekOrigin.Begin );             byte[] memBytes = memStream.ToArray();             BinaryWriter bw = new BinaryWriter( fsDest );             bw.Write( memBytes );             // clean up (note: closing BinaryReader and             // BinaryWriter closes streams also)             bw.Close();             rdr.Close();         }         static void DisplayUsage()         {             Console.WriteLine( "Usage: reverse <source filename> " +                 "<destination filename>\n" );         }     } } 

Comments

Memory files, as they are most commonly referred to, are very powerful constructs that eliminate one of the biggest bottlenecks of computers today, file I/O. Once data has to move along the computer bus from the registers and RAM to a fixed disk, the performance of an application can come to a grinding halt. Naturally, if your application only periodically writes files to disk, then regular writing using a FileStream or TextWriter is perfectly fine.

We should mention a few caveats about using a MemoryStream that are readily apparent. The first concerns tradeoffs. Using a MemoryStream means you have to trade off system memory. If your MemoryStream object becomes quite large, then you might get to a point where the virtual-memory system of the operating system begins thrashing. In other words, once your memory becomes full and the operating system has to temporarily free some of that memory by writing it to disk, the file I/O bottleneck really comes into play, and it slows down not only your application but the entire system as well. However, with memory sizes in today's computers, it would take a very large MemoryStream to get the virtual-memory system to start thrashing.

One other caveat concerns volatility. The MemoryStream object that you are using exists purely in memory. We shouldn't have to tell what would happen if the power to your computer all of a sudden shut off. If the MemoryStream were somehow linked to the document a user was working on, then all his information would be lost. Sure, it's a regular occurrence anyway, but you still should think about utilizing an auto-save system whereby the MemoryStream object is flushed after a certain period of time to a temporary location on the hard disk.