Script Programming | Zhang Ge Blog

Recently, I was rebuilding an HTTP Restful API service I was responsible for to integrate support for the gRPC protocol (called tRPC in Goose Factory, which is a local derivative of gRPC). Since I learned this protocol from 0, I also stepped into many holes. Here is an article to record it. I hope it can help friends who also learned gRPC from 0. In the old Rest interface of list parameter assignment, a lot of list parameters are used. After looking at the pb protocol, this parameter is defined as follows: message Foo{ rep...

August 3, 2022

SQLAlchemy failed to connect to MySQL because the password contains the @ symbol

No label

two thousand four hundred and sixty-one twelve zero

Recently, the new young intern is reconstructing an old system that I developed when I joined the company. This system was previously developed using the Python twisted framework. This reconstruction hopes to change to FastAPI, which is the Flyer development framework I shared in my last article. The young man was very helpful. He spent several days learning FastAPI and Flyer by himself and started to develop them. As a result, when using the Flyer framework, the following error was reported when connecting to MySQL: sqlalchemy.exc.OperationalError: (

March 1 · 2022

Resolve the timeout problem of paramiko using the invoke_shell interactive command

python

four thousand one hundred and twenty-four five zero

Recently, a business on hand was developed based on Apache Karaf. It runs in K8S and runs a bunch of bundles. Occasionally, one or two bundle containers are not perceived, but the business may have local exceptions. If you want to promote business optimization, the cycle will be very long. Therefore, you should first set up a regular dial test to monitor whether the bundles are running normally. After looking at the bundles in Karaf, you need ssh to execute the bundle list command, so based on previous experience, you can use Python p

February 8, 2020

Website pre caching tool to improve the overall loading speed of the website

CDN Website optimization Static cache

five thousand six hundred and seventy-one seventy-two six

Due to the harmonious 443 port of telecom broadband, recently I spent a lot of time tossing about the blog I didn't see in the second half of the year. After a lot of trouble (see above), the blog can finally run on the NAS at home. The overall deployment plan is as follows: Figure 1 Figure 1 of Zhang Ge's blog deployment architecture. Pre caching can be seen from the above figure. Since the Tencent Cloud CDN and Alibaba Cloud CDN used for proxy forwarding in the middle do not have caching, if CloudFlare's cache expires, it will need to bypass multiple links to get web content back to the source, which will be much slower than that. Therefore, it is necessary to

June 16, 2017

Problems encountered by Python remote control module paramiko and records of solutions

python shell Shell script

four thousand nine hundred and ninety-five thirty-two seventeen

Recently, we have been developing an automatic operation and maintenance publishing platform. The underlying command line and file channel are mainly based on the paramiko module. We encounter various problems in the use process. This article is mainly used to collect problems and solve records for future use. I Error reading SSH protocol banner, the key word of wrong connection, has been searched by Baidu and Google for a large number of questions, and a few of them have given solutions, but they can't be solved in the end. After constant attempts and interpretation of paramiko source code, I finally solved the problem, here

May 7, 2017

Share a self written Python remote command and file (folder) transfer class

python

eight thousand and forty-five forty thirty-seven

Recently, I was working on the construction of an automatic publishing platform, among which the remote control channel of the Linux system was independently developed by me, which involves the remote command and file transfer operation of the Linux system. Because when writing the Linux password management system, we used Paramiko's SSHClient. Therefore, I still use Paramiko to do the implementation this time. Although the code is short, speaking of the pit in it, I also fill in the bitter tears. First, complete code: # - * - coding: utf-8

January 2, 2017

Use Tencent Cloud COS to store scheduled remote backup websites for cloud objects

shell Seven day circular backup Site Backup Remote Backup

eight thousand six hundred and thirty ninety-four eighty-one

Guide: Two months ago, Zhang Ge's blog shared a tutorial on Python+Shell scheduled backup website to AliCloud OSS, which has been used by many webmaster friends. The feedback is good, and many friends have put forward many optimization suggestions, such as whether to delete local compressed packets after uploading. This article will continue to share another cloud backup scheme. 1、 Advantages analysis of intranet transmission: Like Alibaba Cloud OSS, Tencent Cloud COS also supports intranet and internet file transmission. For Tencent ECS, using intranet transmission is definitely the fastest and most stable backup

October 2, 2016

Python+Shell script combined with AliCloud OSS object storage scheduled remote backup website

Linux local backup Seven day circular backup Compact Backup Site Backup Remote Backup

eleven thousand three hundred and four ninety-eight fifty-one

Guide: There is no doubt that data backup is a crucial work in the sustainable operation of websites. If there is no website with any backup mechanism, it is recommended to improve it as soon as possible. This article will share a safe, stable, fast, reliable and inexpensive backup solution. 1、 Advantages Analysis Zhang Ge's blog has shared an article about website backup two years ago: "Linux/vps Local 7-Day Recycle Backup and 7-Bull Remote Backup Script". Today, it will combine this script again to back up website data to Alibaba Cloud OSS through Alibaba Cloud intranet. For Alibaba Cloud

June 25, 2016

Skillfully Using Echo Command to Solve the Problem of Samba Batch Adding Users

linux shell

five thousand eight hundred and thirty-six thirty-five nineteen

I'm really busy recently, and I don't have time to study and toss about, so I have nothing to share with my blog. Sure enough, it's not easy for a personal blog to insist on originality. Zhang Ge's blog has been online for more than two years, from more than one day to more than one day, to more than one day and more than one week, until now there may be a more rhythmic two weeks... Well, stop talking nonsense. After looking through the impression notes at work, I found that there are still some stocks that can be shared. A long time ago, the group applied for a new batch of development test machines, which needed to deploy the environment. In addition to some common software to install, there is also a Samba that I rarely used before

November 8, 2015

Share an entry-level controllable multithreaded shell script scheme

shell Shell multithreaded script Shell script

five thousand nine hundred and forty-seven thirty-one twelve

When it comes to shell controllable multithreading, most of the online sharing is pipeline control. Zhang Ge's blog has also tried and shared this solution once: Shell+Curl Website Health Check Script, Catching the Lost Link Site of China Blog Alliance, which can be seen by interested friends. Next, Zhang Ge's blog will share another more understandable entry-level controllable multithreaded shell script scheme: task cutting and individual breakdown. Let's begin with a scene description: one day, I received this task in the goose factory, and I need to check the number of

September 28, 2015

CCKiller: Linux lightweight CC attack defense tool, second level check, automatic blackout and release

cckiller Linux security settings WEB firewall firewall

nineteen thousand one hundred and forty-eight two hundred and twenty-eight two hundred and fifty-nine

Zhang Ge's blog shared a defense script for CC attack long ago, which was not very well written, but was accidentally reprinted by 51CTO. Since then, blogs have been used by people to practice their hands. Of course, there are still many friends who use it in the production environment and will leave messages to ask relevant questions. According to the needs of these problems, I spent some time rewriting a relatively satisfactory lightweight CC attack defense script. I gave it a more vivid name: CCKiller, translated as CC Terminator. 1、 Before sharing the function declaration, I must declare

July 1, 2015

Notes on the use of telnet in batch detection of remote ports by bat batch script under Windows

bat windows Batch processing Batch Maintenance Tools

eleven thousand six hundred and sixty-nine twenty-three twenty-one

I haven't written batch processing for many years, but the first case in the new company needs to write a bat script to batch update the configuration file of the collection agent, which involves the port detection of remote IP. I thought it would be as simple as Linux: echo q | telnet - e 'q '$ip $port&amp& Amp; echo "$ip: port connection" | | echo "$ip: port connection" The result is that telnet exit is not executed under Windows