Learning without motivation, eating jujube is? In the process of learning nodejs, I will give you a little refreshing.

Here I use the cnodejs community's simplest crawler tutorial to do the basic modification of eel.

 / / the magic module 
require ('superagent') used in site environment 
node v6.10.2
npm v3.10.10
//; 
require ('cheerio'); 
require ('express'); 
require ('ejs');'express'(E); (()); (());

Initialization project
NPM init
Installation module use
NPM install module name --save
Start app.js
Node app.js
You can play happily (the download link is at the end of the article).

If you don't know how to use nodejs
Linux players can see My article Windows players can download.Msi directly to the official website, and install next step. Through train


Basic crawling ideas

  1. The paging button of the page catches all the little sister's page links appearing on this page and stores them.
  2. Using eventproxy to visit every little sister's page concurrently
  3. During the visit, we use cheerio (similar to jQuery module) to match the picture address we need, and record all the pictures found.
  4. HTTP requests every picture address to save binary data stream.
  5. FS outputs binary data streams into picture files, and the name of the files is extracted from the original address.
  6. Save to folder

Here, I am afraid that because the request is too large, we can use express to build static pages. query and download To reduce the possibility of crawling sites being crawled to death.

Download address: Xjj.zip

Label: Nodejs , welfare , Cute girl

There are 5 comments.

  1.  Sanko Sanko

    The same goes for the same thing.

  2. Node.js is still too deep.

  3. Change the eel...

  4. Test reviews

Add new comments