Stream wrapper to make Stream seekable?

Question

I have a readonly System.IO.Stream implementation that is not seekable (and its Position always returns 0). I need to send it to a consumer that do

Answer 1

Seeking forwards is easy enough (just read), but you can't seek backwards without buffering. Maybe just:

using(var ms = new MemoryStream()) {
    otherStream.CopyTo(ms);
    ms.Position = 0;
    // now work with ms
}

This, however, is only suitable for small-to-moderate streams (not GB), that are known to end (which streams are not requires to do). If you need a larger stream, a FileStream to a temp-file would work, but is significantly more IO-intensive.

Answer 2

Another solution might be to create your own stream class which wraps the other stream. Implement Seek as a NOP.

class MyStream : Stream
{
    public MyStream(Stream baseStream) { this.baseStream = baseStream; }
    private Stream baseStream;

    // Delegate all operations except Seek/CanSeek to baseStream

    public override bool CanSeek { get { return true; } }
    public override long Seek(long offset, SeekOrigin origin) { return baseStream.Position; }
}

If the player is seeking for no good reason, this might just work.

Answer 3

If using System.Net.WebClient, instead of using OpenRead() which returns a Stream use webClient.DownloadData("https://your.url") to get a byte array which you can then turn into a MemoryStream. Here is an example:

byte[] buffer = client.DownloadData(testBlobFile);
using (var stream = new MemoryStream(buffer))
{
    ... your code using the stream ...
}

Obviously this downloads everything before the Stream is created, so it may defeat the purpose of using a Stream.

Answer 4

I just use the MakeStreamSeekable method from the Amazon SDK:

---

MakeStreamSeekable Method (input)

Converts a non-seekable stream into a System.IO.MemoryStream. A MemoryStream's position can be moved arbitrarily.

Declaration Syntax

C#

public static Stream MakeStreamSeekable(
    Stream input
)

Parameters

*input* ([Stream][2]) The stream to be converted

Return Value

A seekable MemoryStream

Remarks

MemoryStreams use byte arrays as their backing store. Please use this judicially as it is likely that a very large stream will cause system resources to be used up.

Answer 5

Here's a wrapper to make any Stream seekable for read operations.

It works by caching reads from the underlying stream, up to the number of bytes specified in the constructor. This will come in handy when memory constraints prohibit Marc Gravell's solution.

Supported seek operations:

• seeking forward using SeekOrigin.Current and SeekOrigin.Begin works for arbitrary offsets
• seeking backwards using SeekOrigin.Current and SeekOrigin.Begin works for down to -seekBackBufferSize bytes from the current position in the underlying stream (which may differ from readSeekableStream.Position after a previous backwards seek)
• seeking using SeekOrigin.End works for offset >= -seekBackBufferSize && offset <= 0

General remarks

• the Seek method and the Position property are completely handled internally and do not involve the underlying stream (which would only throw anyway)
• seeking affects the read part of the stream only, hence the name of the class
• all write operations are simply delegated through to the underlying stream
• wrapping already seekable streams with this would be a waste of resources
• some problems addressed by ReadSeekableStream below can also be solved by my PeekableStream class

This implementation is fresh and not yet battle-hardened. I have however unit tested it for quite a few seek/read cases and corner cases, and cross-compared it against a (freely seekable) MemoryStream.

public class ReadSeekableStream : Stream
{
    private long _underlyingPosition;
    private readonly byte[] _seekBackBuffer;
    private int _seekBackBufferCount;
    private int _seekBackBufferIndex;
    private readonly Stream _underlyingStream;

    public ReadSeekableStream(Stream underlyingStream, int seekBackBufferSize)
    {
        if (!underlyingStream.CanRead)
            throw new Exception("Provided stream " + underlyingStream + " is not readable");
        _underlyingStream = underlyingStream;
        _seekBackBuffer = new byte[seekBackBufferSize];
    }

    public override bool CanRead { get { return true; } }
    public override bool CanSeek { get { return true; } }

    public override int Read(byte[] buffer, int offset, int count)
    {
        int copiedFromBackBufferCount = 0;
        if (_seekBackBufferIndex < _seekBackBufferCount)
        {
            copiedFromBackBufferCount = Math.Min(count, _seekBackBufferCount - _seekBackBufferIndex);
            Buffer.BlockCopy(_seekBackBuffer, _seekBackBufferIndex, buffer, offset, copiedFromBackBufferCount);
            offset += copiedFromBackBufferCount;
            count -= copiedFromBackBufferCount;
            _seekBackBufferIndex += copiedFromBackBufferCount;
        }
        int bytesReadFromUnderlying = 0;
        if (count > 0)
        {
            bytesReadFromUnderlying = _underlyingStream.Read(buffer, offset, count);
            if (bytesReadFromUnderlying > 0)
            {
                _underlyingPosition += bytesReadFromUnderlying;

                var copyToBufferCount = Math.Min(bytesReadFromUnderlying, _seekBackBuffer.Length);
                var copyToBufferOffset = Math.Min(_seekBackBufferCount, _seekBackBuffer.Length - copyToBufferCount);
                var bufferBytesToMove = Math.Min(_seekBackBufferCount - 1, copyToBufferOffset);

                if (bufferBytesToMove > 0)
                    Buffer.BlockCopy(_seekBackBuffer, _seekBackBufferCount - bufferBytesToMove, _seekBackBuffer, 0, bufferBytesToMove);
                Buffer.BlockCopy(buffer, offset, _seekBackBuffer, copyToBufferOffset, copyToBufferCount);
                _seekBackBufferCount = Math.Min(_seekBackBuffer.Length, _seekBackBufferCount + copyToBufferCount);
                _seekBackBufferIndex = _seekBackBufferCount;
            }
        }
        return copiedFromBackBufferCount + bytesReadFromUnderlying;
    }

    public override long Seek(long offset, SeekOrigin origin)
    {
        if (origin == SeekOrigin.End) 
            return SeekFromEnd((int) Math.Max(0, -offset));

        var relativeOffset = origin == SeekOrigin.Current
            ? offset
            : offset - Position;

        if (relativeOffset == 0)
            return Position;
        else if (relativeOffset > 0)
            return SeekForward(relativeOffset);
        else
            return SeekBackwards(-relativeOffset);
    }

    private long SeekForward(long origOffset)
    {
        long offset = origOffset;
        var seekBackBufferLength = _seekBackBuffer.Length;

        int backwardSoughtBytes = _seekBackBufferCount - _seekBackBufferIndex;
        int seekForwardInBackBuffer = (int) Math.Min(offset, backwardSoughtBytes);
        offset -= seekForwardInBackBuffer;
        _seekBackBufferIndex += seekForwardInBackBuffer;

        if (offset > 0)
        {
            // first completely fill seekBackBuffer to remove special cases from while loop below
            if (_seekBackBufferCount < seekBackBufferLength)
            {
                var maxRead = seekBackBufferLength - _seekBackBufferCount;
                if (offset < maxRead)
                    maxRead = (int) offset;
                var bytesRead = _underlyingStream.Read(_seekBackBuffer, _seekBackBufferCount, maxRead);
                _underlyingPosition += bytesRead;
                _seekBackBufferCount += bytesRead;
                _seekBackBufferIndex = _seekBackBufferCount;
                if (bytesRead < maxRead)
                {
                    if (_seekBackBufferCount < offset)
                        throw new NotSupportedException("Reached end of stream seeking forward " + origOffset + " bytes");
                    return Position;
                }
                offset -= bytesRead;
            }

            // now alternate between filling tempBuffer and seekBackBuffer
            bool fillTempBuffer = true;
            var tempBuffer = new byte[seekBackBufferLength];
            while (offset > 0)
            {
                var maxRead = offset < seekBackBufferLength ? (int) offset : seekBackBufferLength;
                var bytesRead = _underlyingStream.Read(fillTempBuffer ? tempBuffer : _seekBackBuffer, 0, maxRead);
                _underlyingPosition += bytesRead;
                var bytesReadDiff = maxRead - bytesRead;
                offset -= bytesRead;
                if (bytesReadDiff > 0 /* reached end-of-stream */ || offset == 0) 
                {
                    if (fillTempBuffer)
                    {
                        if (bytesRead > 0)
                        {
                            Buffer.BlockCopy(_seekBackBuffer, bytesRead, _seekBackBuffer, 0, bytesReadDiff);
                            Buffer.BlockCopy(tempBuffer, 0, _seekBackBuffer, bytesReadDiff, bytesRead);
                        }
                    }
                    else
                    {
                        if (bytesRead > 0)
                            Buffer.BlockCopy(_seekBackBuffer, 0, _seekBackBuffer, bytesReadDiff, bytesRead);
                        Buffer.BlockCopy(tempBuffer, bytesRead, _seekBackBuffer, 0, bytesReadDiff);
                    }
                    if (offset > 0)
                        throw new NotSupportedException("Reached end of stream seeking forward " + origOffset + " bytes");
                }
                fillTempBuffer = !fillTempBuffer;
            }
        }
        return Position;
    }

    private long SeekBackwards(long offset)
    {
        var intOffset = (int)offset;
        if (offset > int.MaxValue || intOffset > _seekBackBufferIndex)
            throw new NotSupportedException("Cannot currently seek backwards more than " + _seekBackBufferIndex + " bytes");
        _seekBackBufferIndex -= intOffset;
        return Position;
    }

    private long SeekFromEnd(long offset)
    {
        var intOffset = (int) offset;
        var seekBackBufferLength = _seekBackBuffer.Length;
        if (offset > int.MaxValue || intOffset > seekBackBufferLength)
            throw new NotSupportedException("Cannot seek backwards from end more than " + seekBackBufferLength + " bytes");

        // first completely fill seekBackBuffer to remove special cases from while loop below
        if (_seekBackBufferCount < seekBackBufferLength)
        {
            var maxRead = seekBackBufferLength - _seekBackBufferCount;
            var bytesRead = _underlyingStream.Read(_seekBackBuffer, _seekBackBufferCount, maxRead);
            _underlyingPosition += bytesRead;
            _seekBackBufferCount += bytesRead;
            _seekBackBufferIndex = Math.Max(0, _seekBackBufferCount - intOffset);
            if (bytesRead < maxRead)
            {
                if (_seekBackBufferCount < intOffset)
                    throw new NotSupportedException("Could not seek backwards from end " + intOffset + " bytes");
                return Position;
            }
        }
        else
        {
            _seekBackBufferIndex = _seekBackBufferCount;
        }

        // now alternate between filling tempBuffer and seekBackBuffer
        bool fillTempBuffer = true;
        var tempBuffer = new byte[seekBackBufferLength];
        while (true)
        {
            var bytesRead = _underlyingStream.Read(fillTempBuffer ? tempBuffer : _seekBackBuffer, 0, seekBackBufferLength);
            _underlyingPosition += bytesRead;
            var bytesReadDiff = seekBackBufferLength - bytesRead;
            if (bytesReadDiff > 0) // reached end-of-stream
            {
                if (fillTempBuffer)
                {
                    if (bytesRead > 0)
                    {
                        Buffer.BlockCopy(_seekBackBuffer, bytesRead, _seekBackBuffer, 0, bytesReadDiff);
                        Buffer.BlockCopy(tempBuffer, 0, _seekBackBuffer, bytesReadDiff, bytesRead);
                    }
                }
                else
                {
                    if (bytesRead > 0)
                        Buffer.BlockCopy(_seekBackBuffer, 0, _seekBackBuffer, bytesReadDiff, bytesRead);
                    Buffer.BlockCopy(tempBuffer, bytesRead, _seekBackBuffer, 0, bytesReadDiff);
                }
                _seekBackBufferIndex -= intOffset;
                return Position;
            }
            fillTempBuffer = !fillTempBuffer;
        }
    }

    public override long Position
    {
        get { return _underlyingPosition - (_seekBackBufferCount - _seekBackBufferIndex); }
        set { Seek(value, SeekOrigin.Begin); }
    }

    protected override void Dispose(bool disposing)
    {
        if (disposing)
            _underlyingStream.Close();
        base.Dispose(disposing);
    }

    public override bool CanTimeout { get { return _underlyingStream.CanTimeout; } }
    public override bool CanWrite { get { return _underlyingStream.CanWrite; } }
    public override long Length { get { return _underlyingStream.Length; } }
    public override void SetLength(long value) { _underlyingStream.SetLength(value); }
    public override void Write(byte[] buffer, int offset, int count) { _underlyingStream.Write(buffer, offset, count); }
    public override void Flush() { _underlyingStream.Flush(); }
}