Learning without motivation, eating jujube is? In the process of learning nodejs, I will give you a little refreshing.

Here I use the cnodejs community's simplest crawler tutorial to do the basic modification of eel.

 / / initialization project 
npm init
// startup App.js
 Node App.js
 / / you can be happy. A kind of Play fast (see the end of the code)

If you don't know how to use nodejs
Linux players can see My article Windows players can download.Msi directly to the official website, and install next step. Through train


Basic crawling ideas

  1. The paging button of the page catches all the little sister's page links appearing on this page and stores them.
  2. Using eventproxy to visit every little sister's page concurrently
  3. During the visit, we use cheerio (similar to jQuery module) to match the picture address we need, and record all the pictures found.
  4. HTTP requests every picture address to save binary data stream.
  5. FS outputs binary data streams into picture files, and the name of the files is extracted from the original address.
  6. Save to folder

Here, I am afraid that because the request is too large, we can use express to build static pages. query and download To reduce the possibility of crawling sites being crawled to death.

Download address: Xjj.zip

Foundation: installation

server information :
1.System

CentOS release 6.9 (Final)
Kernel r on an M

2.Configuration

Intel (R) Xeon (R) CPU E5-2650 V2 @ 2.60GHz
1G, 1 core (ALI lackey cloud)

Step 1

Here I use my big Taobao source.

 When you enter the storage directory, I put it under /home/nodejs, where do I see you like spicy 
cd /home/nodejs
# download nodejs binary package 
wget Https://npm.taobao.org/mirrors/node/v6.10.2/node-v6.10.2-linux-x86.tar.gz 
tar -xzvf node-v6.10.2-linux-x86. Tar.gz
 To get a folder node-v6.10.2-linux-x86, enter the bin directory 
cd node-v6.10.2-linux-x86/bin
# in this directory to confirm that the download version of nodejs is correct, 
./node -v.

Step 2

Now you can only operate node in this directory, so we have to add it to the user environment.

 Look at the directory where 
pwd
# is currently added, add a soft connection to /usr/local/bin
ln -s /home/nodejs/node-v6.10.2-linux-x86/bin/node /usr/local/bin/node
ln -s /home/nodejs/node-v6.10.2-linux-x86/bin/npm /usr/local/bin/npm



- reading the remainder part -

Less (Fei) female (Zhai) powder (SAO) has a nice color.

Details of tariff

22 card fee 22 yuan / month; small TV card fee 66 yuan / month.
1G nationwide traffic (looks like the first month to send 1G); 4G nationwide traffic (looks like the first month to 2G)
It is free in China, and it is free in China (sending 200 minutes domestic calls).
The traffic volume in the province is 1 yuan 500M, and the flow volume outside the province is 2 yuan 500M (same).
SMS and voice are the same.
Then the gift and the live broadcast privileges are a little bit spicy.
These are trivial things. The key is to avoid the flow of BiliBili. (Note: a month's upper limit seems to be 40G Oh ~)

 Card packaging 1

 Card packaging 2







- reading the remainder part -

 Git

Just now I used the desktop GitHub hand to remend a Revert, and the file was deleted. I made a Revert file on this submission again.

But I watched these two times without any oval submission. 80 Itchy, X for a while.

Find the way to follow this person's own way, and withdraw this way in your own way.

 Git reset --hard <commit_ Id>

Commit_ ID is the form 44c7aa824b0f6383ad1eaf037185ea25cb77878f in log.
Use the desktop version of GitHub to omit the Sha code of 6 digits, and it can also be withdrawn.


In addition to commit_ There are two ways besides ID.

 --HEAD / / last submitted to 
--HEAD^ / / last submission.

 Did not find a good map, P with a survival Road 2

Server configuration: Alibaba is not used in Ali.

  1. System

CentOS 6.5 x86_ Sixty-four

  1. Configuration

1h1g
Disk 40G
Peak 100Mbps (on demand bandwidth), I haven't seen the run up yet, and there are still a few hundred volumes.

0x1. preoperation

Before installing SteamCMD, you need to install the following. Yum update Upgrade the system

CentOS x86_ 64 ( 64bit )

 Yum install mailx postfix curl WGet bzip2 gzip unzip Python TMUX Python

CentOS x64 ( 32bit )

 Yum install mailx postfix curl WGet bzip2 gzip unzip Python TMUX Python

Steam tutorial

Create a user account named steam to run SteamCMD safely, isolating, isolating, "Dan", ",", ",", "and"

It is best not to use root users to run steamCMD.



- reading the remainder part -

Entry gate

I don't want to go to the entry gate.


First pass

 Step1

The difficulty of the first hurdle is that the page is blank. Boring classmates Dragging the mouse in the blank position will find a text which appears to be a handle. Color is set to white. Well, this is simple. F12 console open

 input Document.getElementsByClassName ('text-panel') [0].innerHTML is the text. Document.getElementsByClassName ('text-panel') [0]. InnerHTML.trim () is there no newline and space?

- reading the remainder part -

Python version: 3.5.2

Suppose the following is the string we want to connect to.

 String to connect to

1. + number connection

Written JS Should know

 Print (text[0] + text[1] + text[2])

output

 Heyheyfuckworld

Without considering the memory space, there is nothing to use, huge amount of data, frequent memory operation, and very cumbersome rush.

Two. Space connector

 Print (text[0] text[1] text[2])

output

 Heyheyfuckworld

I don't know why. Spaces and spaces can be connected. Is it Python exclusive to laugh and cry?

Three. Placeholder connection

Don't tell the C language. Printf ("%s", "STR");

 Print ("%s%s%s"% (text[0], text[1], text[2])

output

 Heyheyfuckworld

Four. Join function connection

In fact, we recommend the best use. Join () Efficient

 T is to connect the characters between strings to 
t ='-'
print (t.join (text)).

output

 Heyhey-fuck-world

Five. * connection

This position is used by oneself.

 STR ='abc'
str * 2

output

 Abcabc

Anyway, I don't think I can use eggs, but I can't use them now.

2016, there are many unforgettable things. I also learn a lot and become lazy. Indolence No, it's really lazy. (ha ha ha)

The biggest harvest this year is Wife She met her in the luckiest hour. As most of us agreed, she was also a little girl. She was also very sticky and very understanding. Study Small emotions are mostly caused by learning and making trouble with me.

Forget it or talk about learning. This year, the blog didn't manage much. It didn't go to the statistics. At the end of the year, it was ready to reorganize the blog. The database GG was also destroyed.

Now that the studio teacher has a project, he is learning. EmpireCMS (that is) Empire CMS Spicy), I've played before. DedeCMS It still feels a little similar. It's easy to get started.

AngularJS , NodeJS Looking at it, I hope to make a decent achievement.

My Filming pornography It's also down. A very simple player is writing. GitHub The big guy has API of every major music site, and has changed the plan temporarily. My interface is a bit crude.

Developing ing...


A series of things I do now tend to be more inclined. Full Stack Developer (full stack), I can do everything in my own way, but I am not doing so well. It really bothers me. I hope to have Dalao Give directions.

I didn't remember seeing a sentence in that place.

He has only a key, an index to find the answer, not all. He is not a computer. He can't write down everything.


2016 of the plans are small projects are placed in the LAB, and there are not seen, like small insects, what, and interested can find their own.


The 2017 plan is probably the following.

  1. Keep your blog up.
  2. Perfect worm
  3. MC server maintenance
  4. Online chat room (waiting)
  5. Make a big news.
  6. Follow up