Is TIdHTTPServer Compatible with Microsoft BITS

Question

We are trying to write an update server for our software using the TIdHTTPServer component. Currently we are serving an XML file that lists the available updates and their file versions etc.., when the client program finds a updated version it should start to download it using BITS.

Now this is where we have a problem, our programs are requesting the XML file and seeing there is an update available. It then creates a BITS job to download it, however BITS keeps reporting that the download failed. We can download the file using the same URL and IE/Firefox/Chrome.

so my question:

Is TIdHTTPServer compatible with BITS?

I ask this as I have discovered that there are these download requirements for bits to work.
HTTP Requirements for BITS Downloads

BITS supports HTTP and HTTPS downloads and uploads and requires that the server supports the HTTP/1.1 protocol. For downloads, the HTTP server's Head method must return the file size and its Get method must support the Content-Range and Content-Length headers. As a result, BITS only transfers static file content and generates an error if you try to transfer dynamic content, unless the ASP, ISAPI, or CGI script supports the Content-Range and Content-Length headers.

BITS can use an HTTP/1.0 server as long as it meets the Head and Get method requirements.

To support downloading ranges of a file, the server must support the following requirements:

Allow MIME headers to include the standard Content-Range and Content-Type headers, plus a maximum of 180 bytes of other headers. Allow a maximum of two CR/LFs between the HTTP headers and the first boundary string.

Answer 1

Just found a bug in indy that prevents transfer of files over 2.1GB when using range requests.

here it is

IdHTTPHeaderInfo.pas aprox line 770

procedure TIdEntityRange.SetText(const AValue: String);
var
  LValue, S: String;
begin
  LValue := Trim(AValue);
  if LValue <> '' then
  begin
    S := Fetch(LValue, '-'); {do not localize}
    if S <> '' then begin
      FStartPos := StrToIntDef(S, -1);
      FEndPos := StrToIntDef(Fetch(LValue), -1);
      FSuffixLength := -1;
    end else begin
      FStartPos := -1;
      FEndPos := -1;
      FSuffixLength := StrToIntDef(Fetch(LValue), -1);
    end;
  end else begin
    FStartPos := -1;
    FEndPos := -1;
    FSuffixLength := -1;
  end;
end;

This should be

procedure TIdEntityRange.SetText(const AValue: String);
var
  LValue, S: String;
begin
  LValue := Trim(AValue);
  if LValue <> '' then
  begin
    S := Fetch(LValue, '-'); {do not localize}
    if S <> '' then begin
      FStartPos := StrToInt64Def(S, -1);
      FEndPos := StrToInt64Def(Fetch(LValue), -1);
      FSuffixLength := -1;
    end else begin
      FStartPos := -1;
      FEndPos := -1;
      FSuffixLength := StrToInt64Def(Fetch(LValue), -1);
    end;
  end else begin
    FStartPos := -1;
    FEndPos := -1;
    FSuffixLength := -1;
  end;
end;

One for Remy to fix

Answer 2

When you handle the OnCommandGet event, you are given a TIdRequestHeaderInfo, which descends from TIdEntityHeaderInfo; that contains all the headers the request contained, and it even parses out some header values to read as properties, including ContentRangeStart, ContentRangeEnd, and ContentLength.

You can use those properties to populate the stream that you assign to the TIdHTTPResponseInfo.ContentStream property. The entire stream will get sent.

It's your job to differentiate between GET and HEAD requests; OnCommandGet will get triggered either way. Check the IdHTTPRequestInfo.CommandType property.

So, although Indy may not support BITS, it provides all the tools you need to write a program that does support BITS.

Answer 3

So the answer to this question is:

Yes TIdHTTPServer is Bits Compatible.

But only if you are prepared to do the work yourself.

As suggested by @Rob Kennedy and Myself it is possible to read the headers and send the data back using the requested ranges, one chunk at a time.

Here is an example of what I am doing in the OnCommandGet event

procedure TForm3.IdHTTPServer1CommandGet(AContext: TIdContext;
  ARequestInfo: TIdHTTPRequestInfo; AResponseInfo: TIdHTTPResponseInfo);
var
  Ranges : TIdEntityRanges;
  DataChunk: TMemoryStream;
  ReqFile: TFileStream;
  ChunkLength: Int64;
  Directory, FileName: string;
begin
  Directory := 'H:';

  case ARequestInfo.Ranges.Count of
  0:
    begin
      //serve file normally
    end;
  1:
    begin
      //serve range of bytes specified for file

      filename := Directory + ARequestInfo.Document;

      if FileExists(FileName) then
      begin
        ReqFile := TFileStream.Create(FileName, fmOpenRead);
        try
          ChunkLength := Succ(ARequestInfo.Ranges.Ranges[0].EndPos - ARequestInfo.Ranges.Ranges[0].StartPos);

          if ChunkLength > ReqFile.Size then
            ChunkLength := ReqFile.Size;

          DataChunk := TMemoryStream.Create;
          DataChunk.Posistion := ARequestInfo.Ranges.Ranges[0].StartPos;  
          DataChunk.CopyFrom(ReqFile, ChunkLength);

          AResponseInfo.ContentStream := DataChunk;
          AResponseInfo.ContentType := IdHTTPServer1.MIMETable.GetFileMIMEType(FileName);
          AResponseInfo.ContentRangeUnits := ARequestInfo.Ranges.Units;
          AResponseInfo.ContentRangeStart := ARequestInfo.Ranges.Ranges[0].StartPos;
          AResponseInfo.ContentRangeEnd := ARequestInfo.Ranges.Ranges[0].StartPos + Pred(ChunkLength);
          AResponseInfo.ContentRangeInstanceLength := ReqFile.Size;
          AResponseInfo.ResponseNo := 206;
        finally
          ReqFile.Free;
        end;
      end
      else
        AResponseInfo.ResponseNo := 404;

    end
  else
    begin
      //serve the file as multipart/byteranges
    end;
  end;

end;

This is by no means finished but it shows the basics of responding to the range requests from BITS. Most importantly it works.

Any comments on the code would be appreciated, constructive criticism always welcome.