HTTP206 status code and C # implementation of multi-threaded download Demo

Basic knowledge

HTTP206 status code

The client can add a range in the HTTP header to obtain some content of the request object. If the server supports some content requests, it will return 206 status codes. You can also judge whether the server supports partial content requests by studying and judging the HTTP header of the server response and the Accept Ranges value.

This is the client request header. You can see the Range:

This is the server reply. You can see the status code, partial content type and content length:

Multithreading technology

The process is the father and mother, in charge of many thread sons. A single process can be viewed as the current application program, under which multiple threads can execute different logic, and threads share the resources of the entire process.

Threads are the basic unit of CPU scheduling. There is only one thread executing at any time on the CPU. The thread scheduling algorithm of the system can make threads achieve the effect similar to concurrent execution on a macro level. Therefore, in general, a single thread will not block threads responsible for other functions in the application program. Take eating as an example. Single process multithreading is equivalent to multiple people eating at a table. They share the resources of the entire table while "simultaneously" doing their own work.

Implementation of Multithread Download

thinking

Build an HTTP request, and judge whether it supports partial content requests according to the server reply.

• If it is supported, initialize the thread pool, build the number of threads set by the user, calculate the starting and ending points of the file stream that each thread needs to be responsible for, initialize the thread to start downloading, and merge the files downloaded by each thread according to the download log after all downloads are completed.
• If not, initialize a thread and download the corresponding file.

In order to save costs, single thread initialization must be reusable, that is, each thread can be used multiple times to avoid unnecessary overhead caused by creating new classes and threads every time you download.

Programming

The core content of the program is the multi-threaded download module. Its brief UML diagram is as follows:

Explanation:

• SingleThreadDownloader: single thread download class, including a thread reference, initialization method, etc.
• MTDController: Multi thread download control class, including a single thread linked list, initialization method, allocation method, etc.

MTDController will parse the incoming parameters, generate one or more single thread classes, allocate the calculated work (for example, thread 0 is responsible for 0-49 bytes, thread 1 is responsible for 50-100 bytes), and let threads start.
Please note that this is just a principle Demo, and many functions have not been added.

Important code

Thread running parameter class:

 class MTDThreadInfo { public HttpWebRequest MTDRequest { get;  set; } public string MTDFileName { get;  set; } public long MTDFileSize { get;  set; } public string MTDFilePath { get;  internal set; } }

Single thread download class:

 class SingleThreadDownoad{ float progress = 0; private Thread mThread; private HttpWebRequest mRequest; private MTDThreadInfo mMtdThreadInfo; public void StartDownLoad() { //Pass download parameters and start asynchronous execution thread mThread.Start(mMtdThreadInfo); } public void Start(object mtdInfo) { string sb; MTDThreadInfo mMTDInfo = mtdInfo as MTDThreadInfo; if (mMTDInfo == null) { //Incoming parameter error return; } HttpWebRequest tRequest = mMTDInfo.MTDRequest; //Set Save Path sb = mMTDInfo.MTDFilePath; //Get the length of accepted content long fileSize = tRequest.GetResponse(). ContentLength; if (fileSize > 0) { //TODO writes the segmentation information of the log module file //Create download file FileStream fs = File.Open(sb.ToString(), FileMode.Create); byte[] mByte = new byte[512]; //Get return stream using (Stream bs = tRequest. GetResponse(). GetResponseStream()) { int nreadsize = bs.Read(mByte, 0, 512); while (nreadsize > 0) { fs.Write(mByte, 0, nreadsize); fs.Flush(); nreadsize = bs.Read(mByte, 0, 512); } fs.Close(); bs.Close(); //TODO notification completion tRequest.Abort(); } } } public void Init(string url,  string fileName, long start = 0, long end = 0) { //Create HTTP request mRequest = WebRequest.Create(url) as HttpWebRequest; mRequest.Referer = url; //Add Header if (end !=  0) { mRequest.AddRange(start, end); } //Set thread information class parameters mMtdThreadInfo = new MTDThreadInfo(); mMtdThreadInfo.MTDRequest = mRequest; mMtdThreadInfo.MTDFileSize = end - start; mMtdThreadInfo.MTDFileName = fileName; mMtdThreadInfo.MTDFilePath = MTDCommonData.IOPath.savePath + fileName; //Thread creates parameter to execute Start method ParameterizedThreadStart parameterStart = new ParameterizedThreadStart(Start); //Create Thread mThread = new Thread(parameterStart); } public float GetProgress() { //TODO View Layer Refresh return this.progress; } public void Destroy() { //Thread stops running //TODO subsequent processing mThread.Abort(); } }

Multi thread download control class key code

 /// <summary> ///Initialize threads and allocate segmented tasks /// </summary> /// <param name="threadNum"></param> public void InitThread(int threadNum) { //Size Index long fileSizeIndex = 0; //Last remaining size long lastSize = mFileSize % threadNum; //Average size long avgSize = mFileSize / threadNum; //TODO judges that the server does not support segmented downloading for (int i = 0;  i < threadNum; i++) { //Initialize each thread if (i == (threadNum - 1) ) { //Last thread mThreads[i]. Init(mFileUrl, mFileName + "_" + i,  fileSizeIndex, mFileSize - 1); //The TODO log module records the following information Sw.WriteLine ("[{3}] Thread {0}: {1} bytes - {2} bytes, start execution", i, fileSizeIndex, mFileSize - 1, DateTime.Now); sw.Flush(); break; } mThreads[i]. Init(mFileUrl, mFileName + "_" + i, fileSizeIndex, (fileSizeIndex + avgSize - 1)); //The TODO log module records the following information Sw.WriteLine ("[{3}] Thread {0}: {1} bytes - {2} bytes, start execution", i,fileSizeIndex,fileSizeIndex + avgSize - 1, DateTime.Now); sw.Flush(); fileSizeIndex += avgSize; } sw.Close(); fs.Close(); for (int i = 0;  i < threadNum; i++) { //Start execution thread mThreads[i]. StartDownLoad(); } //TODO updates the View according to the corresponding value //Merge files after TODO } /// <summary> ///Add a download thread /// </summary> /// <param name="url"></param> /// <param name="fileName"></param> /// <param name="filePath"></param> /// <param name="start"></param> /// <param name="end"></param> public void InitThread(string url, string fileName,long start,long end) { SingleThreadDownoad st = new SingleThreadDownoad(); st.Init(url,  fileName ,start, end); mThreads.Add(st); }

Initialize the thread linked list. Note that if the linked list already exists, determine whether the number of single thread classes stored meets the requirements.

Other key points

• C # The default maximum number of threads for HttpWebRequest is 2. Use the following code to modify the maximum number of connections

   ServicePointManager.DefaultConnectionLimit = 10;

• Note the TODO work list in the article
• Pay attention to completing MTDcontroller

Operation results

The test code is as follows:

 MTDController mtdCtrl = new MTDController(); mtdCtrl.Init("//[url]/desktop/bg/acg/58102922_p0.jpg", 4); mtdCtrl.StartDownload();

After multi thread downloading, four files are obtained at the predetermined location (the files are merged in the TODO list, to be improved)

View the download log for testing

Use HEX to check whether the segmented content is consistent with the source file

The file merge function is still in the TODO list. Use HEX to merge four files first

Preview Results

References

1. Let's make speech personification plan!
2. Random photo station of Nian Xiaozhu