DDoS tells us that there are many sorrows in the world, just because there is no money!

0x00 /  After years of operation and maintenance, the enemy is defeated

Since the operation and maintenance started in 2005 13 years Yes, I have been involved in the operation and maintenance of forums, e-commerce, games, finance, and live broadcasting. It is very complicated, such as WebServer, Database, Netfilter, Docker, Xen, KVM, OpenVZ, Ceph, iSCSI, DNS, load balancing, etc.

After many years of operation and maintenance, the most troublesome and desperate thing is DDoS attacks Yes, every night after going to bed, I am afraid of receiving a call saying that the server has been DDoS.

As for why it is so horrible, I think people who have done operation and maintenance or enterprises that have been attacked should be very clear about what kind of experience it is.

 

0x01 / When I woke up from a nightmare, I was terrified all day long

Dealing with DDoS attacks for the first time It was the winter of 2006. My first job was server operation and maintenance for an online advertising alliance company.

Basically, the daily task is to see The server's hard disk I/O load and database load , and whether there is an error log, which is very common.

Until one day in the winter of 2006, the company hosted the Dell PowerEdge 1850 server in the computer room of Zhejiang Shaoxing Telecom Suddenly inaccessible ， WEB and SSH are inaccessible 。

The boss is in a hurry, and the customer service is in a hurry. It can be said that the ants on the hot pot are restless.

You know For an advertising alliance, the server crashed and the webmaster's income suddenly disappeared , the flow of advertisers suddenly disappeared, which means Webmasters and advertisers will lose 。

So I contacted the network maintenance of Shaoxing Telecom and learned that the server suffered High traffic DDoS attacks , the scale of attack is 2Gbps about, In order to protect the normal operation of other client servers in the cabinet, Seal the Internet IP address of our server.

In order not to affect the business, Finally, find the DDoS defense service paid by Shaoxing Telecom , although the service was finally restored The network latency of servers has also increased, because DDoS attacks continue.

Since that incident, Everyone in the company turns pale when talking about DDoS.

 

0x02 / Spend money like dirt just for life

In the past 15 years, live video broadcast has developed explosively and become popular among the whole people. Half a year after joining this company, The company raised more than 20 million yuan, It may not be many for the live video industry, but it is the first step towards success for the whole company.

With the rapid development of the company, the flow and daily life of the live broadcast platform are growing every day, and there are more and more beautiful little sisters.

When everyone was full of morale and even dreamed of going public, it didn't take long, The company suffered a severe setback 。

On September 15, 2016, it was the Mid Autumn Festival. In the evening, the whole family should eat moon cakes and enjoy the moon, but A long planned DDoS attack , so that we spent a long night in the company.

At 6:00 that night, after dinner, I was ready to go to the park to enjoy the moon with my family, I received a phone call from the company and asked to rush to the company to deal with emergencies.

When I arrived at the company, my O&M colleague said that the platform Login system server and anchor reward system server are attacked by large-scale DDoS , due to the large scale of the attack, CDN service provider Directly make the domain name Back to source processing , a large amount of attack traffic Inrush to source server , IDC machine room direct The attacked IP address Blocking treatment 。

Asked IDC access service providers about the scale of this DDoS attack, The incoming DDoS attack traffic is as high as 200Gbps+

The IDC access service provider said that the total access bandwidth of the computer room was 200Gbps. This attack directly filled the outlet of the computer room. In order not to affect other users, the attacked IP address could only be blocked.

Due to the scale of this DDoS attack exceed With the access bandwidth of the IDC service provider, the IDC service provider has no ability to defend, so Ask for help from cloud service providers 。

Finally, the cloud service provider The price of advanced anti DDoS IP is very high , calculated by day, The daily cost of 300G defense is 25000 yuan, The monthly fee is 370000 yuan If the attack exceeds 300G, additional defense costs should be paid.

But the company's business is paralyzed. In order to resume business as soon as possible, the company opened DDoS defense service by day , stored in advance 100 thousand The DDoS defense service was opened at 8:00 that night after the defense cost of.

After the defense was launched, the attack traffic was intercepted through the DDoS cleaning service of the cloud service provider, and the platform temporarily returned to normal. After a night of observation and communication, The DDoS attack was finally prevented.

Then we suffered Larger DDoS attacks Hackers launch attacks for hours every day, which makes it uneconomical for us to pay DDoS defense fees every day, Finally, the company purchased 370000 monthly DDoS defense services.

This case tells us that, Being rich really means you can do whatever you want 。

 

0x03 / There are many sorrows in the world, just because there is no money

I once saw a post on the Cat's Eye Community, which impressed me deeply. The title of the post was called《 My wife has no money to cure her, she died 》

When people are at the bottom of the real world, they are not even qualified to choose to continue to live. Talk about what opportunities to create miracles.

Isn't that true for startups? A large number of start-ups are sick when they encounter DDoS attacks. They come so suddenly and unprepared. But how many start-ups can bear the burden so easily Hundreds of thousands per month DDoS defense costs ？

How many people have entrepreneurial dreams, create miracles, and want to change the industry, but when encountering DDoS attacks, they are not even qualified to choose to continue to survive. Isn't this a sad thing?

Because of the high cost of DDoS defense and the lack of understanding of DDoS attacks, people often resort to medical treatment in case of emergency.

This is very terrible. It is like being seriously ill, and the cost of the top three hospitals is extremely high. However, choosing private and Putian hospitals that claim to be able to cure but are not reliable often leads to serious illness due to wrong treatment and time delay, and finally people and money are exhausted.

 

0x04 / DDoS attacks

Now let's get down to business and talk about various types of DDoS attacks and some Mitigation means , and Prevent Li Gui, cheats, and garbage prevention service providers And teach you how to distinguish the true and false of advanced defense services from the moisture.

①

SYN Flood attack and defense mode

This is an old DDoS attack type, Forged IP source, which is large in size and difficult to trace , which is a classic attack type.

After a large number of SYN attack packets from forged sources enter the server, the system will generate a large number of SYN_RECV states, and finally run out of the SYN Backlog of the system, resulting in the server being unable to process subsequent TCP requests, resulting in server paralysis.

Just like the above picture, the server resource is exhausted, and the user cannot establish a connection with the server. The attacker's purpose is achieved.

How to defend against SYN Flood attack (In fact, it is a mitigation to improve the processing capacity of the system, but only limited to small attacks)?

Mode 1: Software firewall and system parameter optimization (applicable to SYN Flood attack traffic is less than the server access bandwidth, and the server performance is sufficient)

[Windows system: the registry can be modified to improve the processing capacity of SYN packets]

Enter the [HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesTcpipParameters] item of the registry

1. Enable syn attack protection mode (can significantly improve Windows SYN processing capability)

SynAttackProtect=2 [dword]

2. Increase the number of queues with half open TCP connections

TcpMaxHalfOpen=10000 [dword]

3. Enable dynamic Backlog queue length

EnableDynamicBacklog=1 [dword]

Some small-scale and relatively simple SYN Flood attacks can be prevented by modifying the three registry information

 

[Linux system: Modify sysctl kernel parameters to improve the processing capacity of SYN packets]

1. Enable SYN cookies. When the SYN waiting queue overflows, enable cookies to process

net.ipv4.tcp_syncookies = 1

2. Increase the SYN Backlog queue length

net.ipv4.tcp_max_syn_backlog = 65535

3. iptables limits the SYN frequency. Only two SYN packets are allowed to be sent from each source IP per second. If they are exceeded, they are discarded

iptables -N syn-flood

iptables -A INPUT -p tcp --syn -j syn-flood

iptables -A syn-flood  -p tcp -m limit --limit 2/s --limit-burst 50 -j RETURN

iptables -A syn-flood -j DROP

Method 2: Purchase professional DDoS cloud cleaning and cloud defense services (applicable to scenarios with high SYN Flood attack traffic and intensity)

Before purchasing a professional DDoS cloud cleaning service, you can consult the service provider SYN Flood defense algorithm and mode This is very important. SYN Flood defense algorithms and modes have different impacts on different services.

Although the wrong SYN Flood defense algorithm and mode can defend against SYN Flood attacks, it will also cause the business to be unable to access normally.

Common SYN Flood defense algorithms include:

• SYN Cookies

• SYN Proxy

• SYN Reset

• SYN SafeGuard

If the advanced defense service provider you consult cannot answer or is not professional, it is basically agents and some cheats.

All of the above are my experiences on SYN Flood defense summarized after I contacted various DDoS protection service providers during operation and maintenance. The above algorithms have shortcomings, Therefore, it is necessary to select the appropriate SYN Flood defense algorithm according to the business.

The DDoS defense service is different from other network security defense services. Because the size of DDoS attacks is up to the service provider. Users are unable to verify the actual size of DDoS attacks, resulting in this industry dragons and fishes jumbled together ， Over 95% of them are shoddy 。 Later, we will focus on teaching you how to identify the true and false advanced defense!

②

(ACK RST PSH FIN) Flood attack and defense mode

Attacks such as ACK Flood/RST Flood/PSH Flood/FIN Flood are not as harmful as SYN Flood in essence, but they are easy enough to cause server paralysis 。

As shown in the figure above, Although this type of attack will not cause a large number of SYN_RECVs in the server system, it will cause the server to send a large number of RST packets to the forged source IP.

for instance:

If your server access bandwidth is 1Gbps, assume that the PPS processing capacity of the server OS reaches 1.4Mpps, and the OS design is very awesome, without causing a lot of interruption and locking overhead.

When you are attacked by a 500Mbps ACK Flood, your server will also have an uplink bandwidth of about 500Mbps.

This is very uneconomical, and under normal circumstances, the server OS simply cannot handle a large number of ACK Flood attacks.

So for this attack, I suggest you go directly to the DDoS cloud cleaning and cloud defense services. There is no need to adjust the system, because it is meaningless.

③

UDP Flood attack and defense mode

UDP Flood attacks are more and more common at present, thanks to various software design flaws and the connectionless nature of UDP protocol, which makes UDP Flood attacks are very easy to launch, and can be magnified tens of times and thousands of times.

I made a simple picture for you to see Principle of UDP amplification attack

For website business, UDP protocol is not used, so To defend against this attack, you only need to have enough access bandwidth (As long as the access bandwidth is larger than DDoS attacks), You only need an ACL policy to discard the UDP protocol to defend against this attack.

But for Game business and Live video In terms of business, that is nightmare Yes, because a large part of games and live video services are developed based on UDP protocol.

Because the transmission speed and efficiency of UDP protocol are higher and the delay is lower than TCP protocol, this is the advantage of UDP, but it is also the key reason that UDP attacks are extremely difficult to defend.

It happened that my previous company was engaged in live video broadcasting, and I had a lot of contact with DDoS protection service providers in this regard. I can be very responsible to say that, At present, there are no more than 5 DDoS protection service providers that can provide UDP based services with the ability to defend against such UDP attacks.

Why is it so difficult?

Because when UDP data arrives on the firewall, The firewall does not know whether the UDP packet is good or bad , and there is no way to use some defense algorithms similar to TCP attacks to do trusted authentication of the source.

But it's not impossible to solve the problem The live video company adopts the end cloud linkage mode provided by the cloud service provider to defend against UDP Flood attacks , the effect is very ideal.

However, there are few service providers that can do this kind of end-to-end cloud linked defense algorithm, because Most DDoS cloud cleaning and cloud defense service providers buy hardware firewalls There is no substantive R&D capability and technical strength to drive this end cloud linkage defense algorithm. and Only a service provider that has the ability to completely develop its own DDoS defense algorithm can do this 。

So when encountering UDP attacks, you happen to use UDP protocol to carry services. Don't think too much, Prepare money (at least 100000 yuan a month) Then find a very professional DDoS cloud cleaning service provider to protect you.

 

④

DNS Query attacks and defense methods

DNS Query attacks have been used by me for more than 10 years, The most threatening attack mode , ubiquitous in Chess and card games, private servers, spinach, AV and other profiteering industries, where competition is either yours or mine.

Although I have never met, I have never eaten pork, and I have seen pigs run.

The schematic diagram of attack principle is as follows:

 

This attack The biggest threat That is, by randomly constructing and querying the secondary domain name of the attacked domain name, bypassing the resolution record cache of the recursive DNS server, the recursive DNS server in various regions and cities sends a large number of DNS query requests to the authoritative DNS server. If the performance and bandwidth of the authoritative DNS server where the attacked domain name is located cannot support the bandwidth required for query, it will be directly paralyzed, It also affects other domain names on this authoritative DNS server.

Therefore, it is very difficult and costly to defend against such DNS Query attacks, and it is not necessarily 100% defense.

Especially when the recursive DNS server is under too much pressure, Operators can directly ban the attacked domain names Even if the authoritative DNS server can support it, your domain name cannot be resolved at this time, which means that the service is paralyzed.

How to defend?

This can only be found Cooperation between professional DNS service providers and operators Otherwise, it will be invalid, and the cost should be sky high.

 

⑤

HTTP (s) Flood attack (CC attack) and defense mode

The HTTP (s) Flood attack is as intractable as the SYN Flood attack, but it is also very classic. The attack effect is very significant The defense difficulty is several orders of magnitude higher than SYN Flood attack!

 

It seems very simple to launch an attack, but in fact there are hidden secrets!

HTTP (s) Flood attack back in 2008, the defense was relatively simple, because the browser was single (most of them were IE browsers), and the hardware firewall usually used JS Redirect algorithm to do CC defense, with very significant effect, However, 99% of DDoS hardware firewalls do not support HTTPS scenario defense.

When the mobile Internet is highly developed in 12 years, The traditional hardware firewall has not been able to defend against CC attacks, Various browsers are in full bloom, including 360 browser on PC, Chrome, Firefox, IE, UC browser on mobile phone, QQ browser, Chrome, Firefox browser, etc.

At the same time, the attack software is also changing with each passing day, A large part of attack software can even completely simulate user behavior It is difficult to distinguish the true from the false when using the headless browser to attack the website.

The defense against CC attacks is also based on attack scale.

If the attack scale is not large, consider The attacked page is static, avoiding database queries and dynamic languages.

If the attack scale is huge, and the QPS per second is up to tens of thousands of CC attacks, there are two ways.

Method 1: Purchase a large number of servers and bandwidth, as well as professional hardware load balancing equipment to do load balancing, and make the WEB server and database server into a cluster and high availability architecture, which can greatly improve the defense capability against CC attacks. But the cost may be high.

Method 2: Purchase the services of professional DDoS cloud cleaning and cloud defense service providers, and leave professional things to professional people.

Let me kindly remind you that CC attack defense is very difficult, It is recommended that the defense service provider provide a free defense trial for 1-3 days. If you are not satisfied with the defense effect during the three days, you can change to another one without being cheated.

 

⑥

Slow request attack and defense mode

Slow request attack is a new attack mode in recent years. A large number of requests are sent through a large number of broilers, and each broiler only requests once a second. A large number of broilers will cause the server to suffer a large number of attack requests, but each source IP does not appear to have abnormal behavior.

Slow request CC attacks are more harmful, but the difficulty and cost of launching them are also higher In order to make use of a limited number of broilers to launch a larger attack, attackers usually set the query speed per second of a single broiler to a larger value, such as 5 to 10 times per second. This attack mode can often be intercepted by limiting the source IP frequency, while slow request CC attacks go the other way, and attackers often have enough broiler resources.

For example, if the attacker has 100000 broilers online, each broiler can only make one request per second, and 100000 broilers can also make 100000 requests per second. This is a huge pressure for Web servers, especially for small and medium-sized enterprises, which do not have so much budget to do Web clusters and database clusters, as well as dynamically scalable Web and MySQL, Once faced with such slow connection and slow request CC attacks, the database will be overloaded and the Web server will be paralyzed.

Schematic diagram of slow request attack:

Defense mode:

Solution 1: It is mainly to expand the scale of back-end business servers to shoulder this attack. The cost is very high, but it can be solved.

A. Deploy database cluster to support horizontal expansion and deal with the pressure of database query caused by super CC attacks.

B. Deploy the WEB server cluster to support horizontal expansion, which corresponds to the bottleneck pressure of CPU, memory and kernel connections caused by the super CC attack.

C. The business circuit breaker mechanism and algorithm need to develop its own business circuit breaker protection algorithm, which can provide circuit breaker and downgrade protection for businesses in case of super attacks, to prevent all businesses from collapsing across the line.

Solution 2: Find a professional cloud security service provider to solve this attack.

 

⑦

Impulsive attack and defense mode

 

Another kind of attack is called pulse attack. What is called pulse attack is that the attack flow does not last. It is launched several times per second and can be stopped and launched in time.

This attack is very harmful. Basically, all defense service providers are unwilling to defend against this attack , the reason will be explained in detail below.

First, let's put a PPS diagram of impulsive DDoS attacks we saw earlier:

This kind of attack can launch multiple DDoS attacks in a short time, and quickly stop and strike, This is a nightmare for many cloud security defense service providers.

Why do you say that? Let's first sort out the deployment modes of DDoS hardware firewalls of cloud security service providers and IDC service providers.

Mode A: In line

In line mode deployment DDoS defense system can detect and defend attack traffic Very timely , usually in 1 second DDoS attacks are detected on the left and right, and the defense is enabled. The fastest attack can reach the millisecond level.

As long as the bandwidth is sufficient, it is relatively easy to deal with this impulsive DDoS attack , but it is a threat to black hole traction detection, because The fast rise and fall of impulse attack will reduce the sampling accuracy It is very easy to fail to block such DDoS attacks in time. If the instantaneous attack traffic exceeds the IDC outlet, but the black hole traction system Without such efficient detection, the service will be intermittent, affecting the service under the IDC outlet.

Mode B: Bypass mode (Out of path)

Bypass deployment The mode needs to be composed of DDoS cleaning equipment and DDoS detection equipment, Usually 90% of cloud security service providers and IDC rooms adopt bypass deployment mode This deployment requires the DDoS detection device to detect a DDoS attack before the route of the attacked IP address can be pulled to the DDoS cleaning device.

Usually, DDoS detection devices mostly use sampling instead of full detection. The efficiency of sampling detection is low, the response time is high, and the attack usually needs to last for a period of time Impulsive DDoS attacks may last for several seconds each time In this case, The defense service deployed by the bypass will basically fail The defense service provider needs to manually pull the DDoS cleaning device to clean the traffic.

Pulse attack can also be realized Bypass Mitigation Attack mode, sufficient broilers and fast enough pulse attack frequency, Only 100G-200G attack traffic is needed to disable T-level defense , and the defense is extremely difficult, The pressure and reliability requirements for DDoS cleaning equipment are huge.

It is true that there is no way to defend against impulsive attacks by ourselves, and we can only rely on professional cloud security service providers, and we have strong research and development capabilities and technical support capabilities.

⑧

Multi Vector attack and defense mode

In fact, it doesn't have to be called hybrid vector attack in such a tall place. The name of grounding gas is Mixed DDoS attacks , such DDoS attacks usually only exist in Huge profits and competition , and has a blood feud An opponent's attack.

This attack usually uses all available attack methods to attack the target. The initial purpose is to make the DDoS hardware firewall unable to handle it, but the current DDoS hardware firewall is not afraid of it at all (unless there is a problem with the code and business logic of this DDoS firewall), The only thing to worry about is whether your defense algorithm can fine filter out these malicious traffic Otherwise, multiple attack methods are mixed. If some attack traffic is leaked into the back-end server, it is catastrophic.

There is no good picture for this attack, so I will take one at random

 

Because of the limited space, I will write here first. This article focuses on Popular science on DDoS attack modes and harmfulness The next article will describe in detail the DDoS defense providers Abacus and Defense mode , because the author has had a lot of contact with operators this year, and also learned about the practices and defense methods of some DDoS defense providers, which will be explained in detail in the next article. Coming soon!