前言
前情回顧:上一篇我們遺留了兩個問題,一個是未完全實現斷點續傳,另外則是在響應時是返回StreamContent還是PushStreamContent呢?這一節我們重點來解決這兩個問題,同時就在此過程中需要注意的地方一併指出,若有錯誤之處,請指出。
StreamContent compare to PushStreamContent
我們來看看StreamContent代碼,如下:
public class StreamContent : HttpContent
{
// Fields
private int bufferSize;
private Stream content;
private bool contentConsumed;
private const int defaultBufferSize = 0x1000;
private long start;
// Methods
public StreamContent(Stream content);
]
public StreamContent(Stream content, int bufferSize);
protected override Task<Stream> CreateContentReadStreamAsync();
protected override void Dispose(bool disposing);
private void PrepareContent();
protected override Task SerializeToStreamAsync(Stream stream, TransportContext context);
protected internal override bool TryComputeLength(out long length);
// Nested Types
private class ReadOnlyStream : DelegatingStream
{......}
}
似乎沒有什麼可看的,但是有一句話我們需要注意,如下:
private const int defaultBufferSize = 0x1000;
在StreamContent的第二個構造函數爲
public StreamContent(Stream content, int bufferSize);
上述給定的默認一次性輸入到緩衝區大小爲4k,這對我們有何意義呢?當我們寫入到響應中時,一般我們直接利用的是第一個構造函數,如下:
var response = new HttpResponseMessage();
response.Content = new StreamContent(fileStream);
到這裏我們明白了這麼做是有問題的,當下載時默認讀取的是4k,如果文件比較大下載的時間則有延長,所以我們在返回時一定要給定緩衝大小,那麼給定多少呢?爲達到更好的性能最多是80k,如下:
private const int BufferSize = 80 * 1024;
response.Content = new StreamContent(fileStream, BufferSize);
此時下載的速度則有很大的改善,有人就說了爲何是80k呢?這個問題我也不知道,老外驗證過的,這是鏈接【.NET Asynchronous stream read/write】。
好了說完StreamContent,接下來我們來看看PushStreamContent,從字面意思來爲推送流內容,難道是充分利用了緩衝區嗎,猜測可以有,就怕沒有任何想法,我們用源碼來證明看看。
我們只需看看WebHost模式下對於緩衝策略是怎麼選擇的,我們看看此類 WebHostBufferPolicySelector 實現,代碼如下:
/// <summary>
/// Provides an implementation of <see cref="IHostBufferPolicySelector"/> suited for use
/// in an ASP.NET environment which provides direct support for input and output buffering.
/// </summary>
public class WebHostBufferPolicySelector : IHostBufferPolicySelector
{
....../// <summary>
/// Determines whether the host should buffer the <see cref="HttpResponseMessage"/> entity body.
/// </summary>
/// <param name="response">The <see cref="HttpResponseMessage"/>response for which to determine
/// whether host output buffering should be used for the response entity body.</param>
/// <returns><c>true</c> if buffering should be used; otherwise a streamed response should be used.</returns>
public virtual bool UseBufferedOutputStream(HttpResponseMessage response)
{
if (response == null)
{
throw Error.ArgumentNull("response");
}
// Any HttpContent that knows its length is presumably already buffered internally.
HttpContent content = response.Content;
if (content != null)
{
long? contentLength = content.Headers.ContentLength;
if (contentLength.HasValue && contentLength.Value >= 0)
{
return false;
}
// Content length is null or -1 (meaning not known).
// Buffer any HttpContent except StreamContent and PushStreamContent
return !(content is StreamContent || content is PushStreamContent);
}
return false;
}
}
從上述如下一句可以很明顯的知道:
return !(content is StreamContent || content is PushStreamContent);
除了StreamContent和PushStreamContent的HttpContent之外,其餘都進行緩衝,所以二者的區別不在於緩衝,那到底是什麼呢?好了我們還未查看PushStreamContent的源碼,我們繼續往下走,查看其源代碼如下,我們僅僅只看關於這個類的描述以及第一個構造函數即可,如下:
/// <summary>
/// Provides an <see cref="HttpContent"/> implementation that exposes an output <see cref="Stream"/>
/// which can be written to directly. The ability to push data to the output stream differs from the
/// <see cref="StreamContent"/> where data is pulled and not pushed.
/// </summary>
public class PushStreamContent : HttpContent
{
private readonly Func<Stream, HttpContent, TransportContext, Task> _onStreamAvailable;
/// <summary>
/// Initializes a new instance of the <see cref="PushStreamContent"/> class. The
/// <paramref name="onStreamAvailable"/> action is called when an output stream
/// has become available allowing the action to write to it directly. When the
/// stream is closed, it will signal to the content that is has completed and the
/// HTTP request or response will be completed.
/// </summary>
/// <param name="onStreamAvailable">The action to call when an output stream is available.</param>
public PushStreamContent(Action<Stream, HttpContent, TransportContext> onStreamAvailable)
: this(Taskify(onStreamAvailable), (MediaTypeHeaderValue)null)
{
}
......
}
對於此類的描述大意是:PushStreamContent與StreamContent的不同在於,PushStreamContent在於將數據push【推送】到輸出流中,而StreamContent則是將數據從流中【拉取】。
貌似有點晦澀,我們來舉個例子,在webapi中我們常常這樣做,讀取文件流並返回到響應流中,若是StreamContent,我們會如下這樣做:
response.Content = new StreamContent(File.OpenRead(filePath));
上面的釋義我用大括號着重括起,StreamContent着重於【拉取】,當響應時此時將從文件流寫到輸出流,通俗一點說則是我們需要從文件流中去獲取數據並寫入到輸出流中。我們再來看看PushStreamContent的用法,如下:
XDocument xDoc = XDocument.Load("cnblogs_backup.xml", LoadOptions.None);
PushStreamContent xDocContent = new PushStreamContent(
(stream, content, context) =>
{
xDoc.Save(stream);
stream.Close();
},
"application/xml");
PushStreamContent着重於【推送】,當我們加載xml文件時,當我們一旦進行保存時此時則會將數據推送到輸出流中。
二者區別在於:StreamContent從流中【拉取】數據,而PushStreamContent則是將數據【推送】到流中。
那麼此二者應用的場景是什麼呢?
(1)對於下載文件我們則可以通過StreamContent來實現直接從流中拉取,若下載視頻流此時則應該利用PushStreamContent來實現,因爲未知服務器視頻資源的長度,此視頻資源來源於別的地方。
(2)數據量巨大,發送請求到webapi時利用PushStreamContent。
當發送請求時,常常序列化數據並請求webapi,我們可能這樣做:
var client = new HttpClient();
string json = JsonConvert.SerializeObject(data);
var response = await client.PostAsync(uri, new StringContent(json));
當數據量比較小時沒問題,若數據比較大時進行序列化此時則將序列化的字符串加載到內存中,鑑於此這麼做不可行,此時我們應該利用PushStreamContent來實現。
var client = new HttpClient();
var content = new PushStreamContent((stream, httpContent, transportContext) =>
{
var serializer = new JsonSerializer();
using (var writer = new StreamWriter(stream))
{
serializer.Serialize(writer, data);
}
});
var response = await client.PostAsync(uri, content);
爲什麼要這樣做呢?我們再來看看源碼,裏面存在這樣一個方法。
protected override Task SerializeToStreamAsync(Stream stream, TransportContext context);
其內部實現利用異步狀態機實現,所以當數據量巨大時利用PushStreamContent來返回將會有很大的改善,至此,關於二者的區別以及常見的應用場景已經敘述完畢,接下來我們繼續斷點續傳問題。
斷點續傳改進
上一篇我們講過獲取Range屬性中的集合通過如下:
request.Headers.Range
我們只取該集合中的第一個範圍元素,通過如下
RangeItemHeaderValue range = rangeHeader.Ranges.First();
此時我們忽略了返回的該範圍對象中有當前下載的進度
range.From.HasValue
range.To.HasValue
我們獲取二者的值然後進行重寫Stream實時讀取剩餘部分,下面我們一步一步來看。
定義文件操作接口
public interface IFileProvider
{
bool Exists(string name);
FileStream Open(string name);
long GetLength(string name);
}
實現該操作文件接口
public class FileProvider : IFileProvider
{
private readonly string _filesDirectory;
private const string AppSettingsKey = "DownloadDir";
public FileProvider()
{
var fileLocation = ConfigurationManager.AppSettings[AppSettingsKey];
if (!String.IsNullOrWhiteSpace(fileLocation))
{
_filesDirectory = fileLocation;
}
}
/// <summary>
/// 判斷文件是否存在
/// </summary>
/// <param name="name"></param>
/// <returns></returns>
public bool Exists(string name)
{
string file = Directory.GetFiles(_filesDirectory, name, SearchOption.TopDirectoryOnly)
.FirstOrDefault();
return true;
}
/// <summary>
/// 打開文件
/// </summary>
/// <param name="name"></param>
/// <returns></returns>
public FileStream Open(string name)
{
var fullFilePath = Path.Combine(_filesDirectory, name);
return File.Open(fullFilePath,
FileMode.Open, FileAccess.Read, FileShare.Read);
}
/// <summary>
/// 獲取文件長度
/// </summary>
/// <param name="name"></param>
/// <returns></returns>
public long GetLength(string name)
{
var fullFilePath = Path.Combine(_filesDirectory, name);
return new FileInfo(fullFilePath).Length;
}
}
獲取範圍對象中的值進行賦值給封裝的對象
public class FileInfo
{
public long From;
public long To;
public bool IsPartial;
public long Length;
}
下載控制器,對文件操作進行初始化
public class FileDownloadController : ApiController
{
private const int BufferSize = 80 * 1024;
private const string MimeType = "application/octet-stream";
public IFileProvider FileProvider { get; set; }
public FileDownloadController()
{
FileProvider = new FileProvider();
}
......
}
接下來則是文件下載的邏輯,首先判斷請求文件是否存在,然後獲取文件的長度
if (!FileProvider.Exists(fileName))
{
throw new HttpResponseException(HttpStatusCode.NotFound);
}
long fileLength = FileProvider.GetLength(fileName);
將請求中的範圍對象From和To的值並判斷當前已經下載進度以及剩餘進度
private FileInfo GetFileInfoFromRequest(HttpRequestMessage request, long entityLength)
{
var fileInfo = new FileInfo
{
From = 0,
To = entityLength - 1,
IsPartial = false,
Length = entityLength
};
var rangeHeader = request.Headers.Range;
if (rangeHeader != null && rangeHeader.Ranges.Count != 0)
{
if (rangeHeader.Ranges.Count > 1)
{
throw new HttpResponseException(HttpStatusCode.RequestedRangeNotSatisfiable);
}
RangeItemHeaderValue range = rangeHeader.Ranges.First();
if (range.From.HasValue && range.From < 0 || range.To.HasValue && range.To > entityLength - 1)
{
throw new HttpResponseException(HttpStatusCode.RequestedRangeNotSatisfiable);
}
fileInfo.From = range.From ?? 0;
fileInfo.To = range.To ?? entityLength - 1;
fileInfo.IsPartial = true;
fileInfo.Length = entityLength;
if (range.From.HasValue && range.To.HasValue)
{
fileInfo.Length = range.To.Value - range.From.Value + 1;
}
else if (range.From.HasValue)
{
fileInfo.Length = entityLength - range.From.Value + 1;
}
else if (range.To.HasValue)
{
fileInfo.Length = range.To.Value + 1;
}
}
return fileInfo;
}
在響應頭信息中的對象ContentRangeHeaderValue設置當前下載進度以及其他響應信息
private void SetResponseHeaders(HttpResponseMessage response, FileInfo fileInfo,
long fileLength, string fileName)
{
response.Headers.AcceptRanges.Add("bytes");
response.StatusCode = fileInfo.IsPartial ? HttpStatusCode.PartialContent
: HttpStatusCode.OK;
response.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment");
response.Content.Headers.ContentDisposition.FileName = fileName;
response.Content.Headers.ContentType = new MediaTypeHeaderValue(MimeType);
response.Content.Headers.ContentLength = fileInfo.Length;
if (fileInfo.IsPartial)
{
response.Content.Headers.ContentRange
= new ContentRangeHeaderValue(fileInfo.From, fileInfo.To, fileLength);
}
}
最重要的一步則是將FileInfo對象的值傳遞給我們自定義實現的流監控當前下載進度。
public class PartialContentFileStream : Stream
{
private readonly long _start;
private readonly long _end;
private long _position;
private FileStream _fileStream;
public PartialContentFileStream(FileStream fileStream, long start, long end)
{
_start = start;
_position = start;
_end = end;
_fileStream = fileStream;
if (start > 0)
{
_fileStream.Seek(start, SeekOrigin.Begin);
}
}
/// <summary>
/// 將緩衝區數據寫到文件
/// </summary>
public override void Flush()
{
_fileStream.Flush();
}
/// <summary>
/// 設置當前下載位置
/// </summary>
/// <param name="offset"></param>
/// <param name="origin"></param>
/// <returns></returns>
public override long Seek(long offset, SeekOrigin origin)
{
if (origin == SeekOrigin.Begin)
{
_position = _start + offset;
return _fileStream.Seek(_start + offset, origin);
}
else if (origin == SeekOrigin.Current)
{
_position += offset;
return _fileStream.Seek(_position + offset, origin);
}
else
{
throw new NotImplementedException("SeekOrigin.End未實現");
}
}
/// <summary>
/// 依據偏離位置讀取
/// </summary>
/// <param name="buffer"></param>
/// <param name="offset"></param>
/// <param name="count"></param>
/// <returns></returns>
public override int Read(byte[] buffer, int offset, int count)
{
int byteCountToRead = count;
if (_position + count > _end)
{
byteCountToRead = (int)(_end - _position) + 1;
}
var result = _fileStream.Read(buffer, offset, byteCountToRead);
_position += byteCountToRead;
return result;
}
public override IAsyncResult BeginRead(byte[] buffer, int offset, int count, AsyncCallback callback, object state)
{
int byteCountToRead = count;
if (_position + count > _end)
{
byteCountToRead = (int)(_end - _position);
}
var result = _fileStream.BeginRead(buffer, offset,
count, (s) =>
{
_position += byteCountToRead;
callback(s);
}, state);
return result;
}
......
}
更新上述下載的完整邏輯
public HttpResponseMessage GetFile(string fileName)
{
fileName = "HBuilder.windows.5.2.6.zip";
if (!FileProvider.Exists(fileName))
{
throw new HttpResponseException(HttpStatusCode.NotFound);
}
long fileLength = FileProvider.GetLength(fileName);
var fileInfo = GetFileInfoFromRequest(this.Request, fileLength);
var stream = new PartialContentFileStream(FileProvider.Open(fileName),
fileInfo.From, fileInfo.To);
var response = new HttpResponseMessage();
response.Content = new StreamContent(stream, BufferSize);
SetResponseHeaders(response, fileInfo, fileLength, fileName);
return response;
}
下面我們來看看演示結果:
好了,到了這裏我們也得到了我們想要的結果。
總結
本節我們將上節遺留的問題一一進行比較詳細的敘述並最終解決,是不是就這麼完全結束了呢?那本節定義爲中篇豈不是不對頭了,本節是在web端進行下載,下節我們利用webclient來進行斷點續傳。想了想無論是mvc上傳下載,還是利用webapi來上傳下載又或者是將mvc和webapi結合來上傳下載基本都已經囊括,這都算是在項目中比較常用的吧,所以也就花了很多時間去研究。對於webapi的斷點續傳關鍵它本身就提供了比較多的api來給我們調用,所以還是很不錯,webapi一個很輕量的服務框架,你值得擁有see u