Classified directory archiving: OS

2024 Excellent Open Source Image Station

3 replies
Enterprise station
Tencent Cloud //mirrors.tencent.com/
Alibaba Cloud //mirrors.aliyun.com/
Netease //mirrors.163.com/
Huawei //mirrors.huaweicloud.com/
Capital Online //mirrors.yun-idc.com/
Education station
Tsinghua University //mirrors.tuna.tsinghua.edu.cn/
University of Science and Technology of China //mirrors.ustc.edu.cn/
Beijing Jiaotong University //mirror.bjtu.edu.cn/cn/
Zhejiang University //mirrors.zju.edu.cn/
Official station
CentOS //mirror-status.centos.org/#cn
Ubuntu //launchpad.net/ubuntu/+cdmirrors
Archlinux //www.archlinux.org/mirrors/status/
Debian //www.debian.org/mirror/list
Fedora //admin.fedoraproject.org/ mirrormanager /mirrors

Let's Encrypt SSL certificate renewal failure ascii codec cannot encode

Comment

Today, I reviewed the SSL certificate of the server and found that the Let's Encrypt certificate is about to expire. Check the scheduled task plan log of the crontab and it is also executed normally. For example:

 $ cat /var/log/cron
 ... CROND[31471]: (root) CMD ( /usr/bin/certbot renew --quiet && /bin/systemctl restart nginx ) CROND[31470]: (root) MAIL (mailed 375 bytes of output but got status 0x004b#012) CROND[31482]: (root) CMD (run-parts /etc/cron.hourly) ...

Strangely, the certificate was not renewed normally. Why? Later, the certificate was updated manually:

 $ /usr/bin/certbot renew --quiet
 Attempting to renew cert from /etc/letsencrypt/renewal/renwole.com.conf produced an unexpected error: 'ascii' codec can't encode characters in position 247-248: ordinal not in range(128).  Skipping. All renewal attempts failed.  The following certs could not be renewed: /etc/letsencrypt/live/renwole.com.conf/fullchain.pem (failure) 1 renew failure(s), 0 parse failure(s)

Update failed, prompt“ ascii ”The codec cannot encode characters.

After analysis and research, it is found that the developer has modified the root directory of the website, resulting in LetsEncrypt not finding the relevant configuration file.
PS: Alas, if something goes wrong, it's all about operation and maintenance.

Solution

Modify the site root directory in the following configuration file:

 $ vim /etc/letsencrypt/renewal/renwole.com.conf
 ... # Options used in the renewal process [renewalparams] authenticator = webroot installer = None account = a07a7160ea489g586aeaada1368ce0d6 [[webroot_map]] renwole.com = /apps/data/www/renwolecom ...

Modify the root directory specified by Nginx in blue, and save it by default.

The certificate was updated again successfully.

Use the following command to view the renewal status:

 $ certbot certificates

Centos 7 Add/Remove Swap Partition

Comment

Swap Introduction:

Linux divides physical memory into memory segments, called pages. Swapping refers to the process of copying memory pages to the preset hard disk space (called swap space), in order to free memory for pages. The total size of physical memory and swap space is the total amount of virtual memory available.

Swap is to swap partitions, which is similar to the virtual memory of Windows, but when the physical memory is insufficient, part of the hard disk space is used as virtual memory, thus solving the problem of insufficient physical memory capacity.

Advantages: cost saving.
Disadvantages: insufficient performance.

This method is not limited to Centos 7, and can be used on Linux systems.

Operating user: root.

1. Add swap partition space

Use dd command to create swap partition file /dev/mapper/centos-swap , size 2G:

 $ dd if=/dev/zero of=/dev/mapper/centos-swap bs=1024 count=2048000

Format the swap partition:

 $ mkswap /dev/mapper/centos-swap

Set the swap partition:

 $ mkswap -f /dev/mapper/centos-swap

Activate the swap partition:

 $ swapon /dev/mapper/centos-swap

Set to start automatically:

 $ vim /etc/fstab

Add the following at the bottom of the file:

 /dev/mapper/centos-swap swap swap default 0 0

2. Delete the swap partition

Stop the swap partition in use:

 $ swapoff /dev/mapper/centos-swap

Delete the swap partition file:

 $ rm /dev/mapper/centos-swap

Delete or comment in /etc/fstab The following contents of the file are automatically attached after startup:

 /dev/mapper/centos-swap swap swap default 0 0

be accomplished!

How to use Lsyncd to replicate and synchronize to remote servers in real time

Comment

What is Lsyncd?

Lsyncd official website//axkibe.github.io/

Lsyncd monitors the local tree event monitor interface (inotify or fsevents). It aggregates and groups events together for a few seconds, and then generates a process (or processes) to synchronize these changes. By default, rsync implements synchronization. Therefore, Lsyncd is a lightweight real-time image solution that is relatively easy to install, does not require a new file system or block device, and does not hinder the performance of the local file system.

Rsync+ssh It is an advanced operation configuration. It uses ssh to move files and directories directly on the target, rather than retransmitting the moving target on the line. Fine grained customization can be achieved through configuration files. Custom Action configs You can even write from scratch, from shell Script to Lua Language. This method is simple, powerful and flexible.

Lsyncd 2.2.2 Required on all source and target machines rsync >= 3.1 。

System environment:

RenwoleServer: 10.28.204.65 server
RenwoleClient: 10.28.204.66 client
OS：CentOS Linux release 7.4.1708 (Core) x64

1. Installation of rsync

See:《 CentOS 7 Configure Rsync Data File Synchronization Server 》。

2. Install the extension dependency package

 $ yum install -y gcc gcc-c++ lua lua-devel cmake libxml2 libxml2-devel

3. Compile and install the source code lsyncd

 $ wget //github.com/axkibe/lsyncd/archive/release-2.2.2.tar.gz $ tar xvf release-2.2.2.tar.gz $ cd lsyncd-release-2.2.2 $ cmake -DCMAKE_INSTALL_PREFIX=/usr/local/lsyncd $ make && make install $ ln -s /usr/local/lsyncd/bin/lsyncd /usr/bin/lsyncd

An error may be reported during installation:

-- Configuring incomplete, errors occurred!

install lua-devel OK.

4. Set SSH login without password

Because here rsyncssh To synchronize, you also need to configure the root account to ssh login without password. Please refer to:

《 How to set SSH login without password in Linux 》

5. Configure lsyncd

The following are three common configuration cases

1. Remote synchronization rsyncssh Mode configuration scheme:

 $ vim /etc/lsyncd.conf
 settings { Logfile="/var/log/lsyncd. log", -- log path StatusFile="/var/log/lsyncd. status", -- status file Pidfile="/var/run/lsyncd. pid", -- pid file path StatusInterval=1, -- the shortest time to write the status file Nodaemon=false, -- daemon running MaxProcesses=1, -- Maximum Processes MaxDelays=1, -- maximum delay } sync { Default.rsyncssh, -- default rsync+ssh, rsync version needs to be upgraded to more than 3 versions Source="/apps/www/renwoleblog/", -- source directory Delete=true, -- keep full synchronization host = " root@10.28.204.66 ",          Targetdir="/apps/www/renwoleblog/bak/", -- target directory exclude={                  ". txt" -- files to be excluded }, rsync = { Binary="/usr/bin/rsync", -- install rsync first Archive=true, -- archive Compress=false, -- compression Owner=true, -- owner Perms=true, -- permission whole_file = false }, ssh = { port = 22 } }

2. Local directory synchronization configuration scheme:

 sync { default.rsync, source = "/apps/www/renwoleblog/", target = "/apps/www/renwoleblog/bak/", }

3. Remote synchronization rsync daemon mode configuration scheme

 sync { default.rsync, source    = "/apps/www/renwoleblog/", target    = " renwole@10.28.204.65 ::renwolecom", delete="true", exclude = { ".bak*" }, delay = 30, init = false, rsync = { binary = "/usr/bin/rsync", archive = true, compress = true, verbose   = true, perms = true, password_file = "/etc/rsync.password", _extra    = {"--bwlimit=200"} } }

Key parameter description:

 --# Comment Settings # is global configuration Sync # Define synchronization parameters Rsync # Define synchronization file parameters Ssh # Define the server remote port

lsyncd Multiple profiles allowed sync They do not affect each other.

Note: If it is one to many, please refer to Local Synchronization and modify the target directory.

6. Create systemctl system unit file

To implement systemctl management, please create a configuration file and a script startup file. The command is as follows:

 $ vim /etc/sysconfig/lsyncd

Add the following:

 LSYNCD_OPTIONS="/etc/lsyncd.conf"

To create a startup file:

 $ vim /usr/lib/systemd/system/lsyncd.service

Add the following:

 [Unit] Description=Live Syncing (Mirror) Daemon After=network.target [Service] Type=simple EnvironmentFile=-/etc/sysconfig/lsyncd ExecStart=/usr/local/lsyncd/bin/lsyncd -nodaemon $LSYNCD_OPTIONS [Install] WantedBy=multi-user.target

7. Start lsyncd and add boot auto start

 $ systemctl start lsyncd $ systemctl enable lsyncd

Next, you can go to the source server /apps/www/renwoleblog/ Upload any file inside, and it will be synchronized to the client immediately after completion 10.28.204.66 /apps/www/renwoleblog/bak/ In the directory, you can also check whether the lsyncd log file analysis on the server is synchronized successfully. For example:

 [ root@RenwoleServer  ~] $ cat /var/log/lsyncd.log
 ... Fri Dec 22 01:19:22 2017 Normal: Calling rsync with filter-list of new/modified files/dirs / PCHunter_renwole.com.tar.gz / Fri Dec 22 01:19:24 2017 Normal: Finished (list): 0 Fri Dec 22 01:19:32 2017 Normal: Calling rsync with filter-list of new/modified files/dirs / PCHunter_renwole.com.tar.gz / Fri Dec 22 01:19:34 2017 Normal: Finished (list): 0 Fri Dec 22 01:19:34 2017 Normal: Calling rsync with filter-list of new/modified files/dirs / PCHunter_renwole.com.tar.gz / Fri Dec 22 01:19:36 2017 Normal: Finished (list): 0

Log content display PCHunter_renwole.com.rar The file was successfully synchronized.

in addition lsyncd Is based on inotify + rsync Compared with other synchronization software, the open-source synchronization software is more secure and reliable, and takes less resources, but the configuration is slightly cumbersome.
lsyncd It also supports what kind of command to execute when a specified event is monitored. Because synchronization is triggered by time delay and cumulative event hit times, it is better than inotify In addition, its synchronization speed depends entirely on the quality of your network.

CentOS 7 Configure Rsync Data File Synchronization Server

Comment

What is Rsync?

Rsync is a very flexible command line network synchronization tool. Because of its popularity on Linux and unix like systems, it is included in most Linux distributions by default. While synchronizing files or folders, it can maintain the permissions, time, soft and hard links and other additional information of the original files. It can quickly and safely transmit data, and support incremental updates. During the process of data transmission, compression and decompression operations can be implemented, so less bandwidth can be used. It is also open source software.

Environmental Science:

RenwoleServer: 10.28.204.65 server
RenwoleClient: 10.28.204.66 client
OS：CentOS Linux release 7.4.1708 (Core) x64

1. Install rsync on the server and client respectively

Because part of Linux The distribution is installed by default rsync , but the version used rsync 3.0.9-18.el7 Some are old, so you need to manually install the latest version. Please see the following specific operations:

2. Install rsync

If it is installed by default, please uninstall the old version:

 $ yum remove rsync -y

There are two ways to install rsync:

The advantages of RPM are fast, convenient and time-saving. The specific installation is as follows:

 $ yum -y install epel-release $ wget //mirror.ghettoforge.org/distributions/gf/gf-release-latest.gf.el7.noarch.rpm $ rpm -Uvh gf-release*rpm $ yum --enablerepo=gf-plus install rsync -y

Rsync file:

 /etc/rsyncd.conf /etc/sysconfig/rsyncd /etc/xinetd.d/rsync /usr/bin/rsync /usr/share/doc/rsync-3.1.2/COPYING ......

Source code installation (Recommended) ：

The advantages of this installation method are not only easy to customize, but also the latest installation package can be used.

The installation depends on and downloads the source code package. Compile and install:

 $ cd /tmp $ yum install gcc c++ -y $ wget //download.samba.org/pub/rsync/rsync-3.1.2.tar.gz $ tar zxvf rsync-3.1.2.tar.gz $ cd rsync-3.1.2 $ ./ configure --prefix=/usr/local/rsync $ make -j8 && make install $ ln -s /usr/local/rsync/bin/rsync /usr/bin/rsync

Set rsync auto start script

The source code installation needs to be manually rsync The script provided by default is copied to the system directory so that it can be used systemctl Administration:

 $ cp /tmp/rsync-3.1.2/packaging/systemd/* /usr/lib/systemd/system

Note: Compile and install source code without /etc/rsyncd.conf The main configuration file needs to be created manually, and the RPM installation mode will be generated automatically, but it needs to be reconfigured as required.

The two installation methods can be installed successfully regardless of which method is used.

The following section will introduce the rsync configuration (take the source code compilation, installation and configuration as an example).

3. About the rsync authentication method

There are two common authentication methods for rsync. One is rsync-daemon The other is SSH. In a production environment, you typically use rsync-daemon Authentication mode.

Certification mode description:

one rsync-daemon Authentication: listen to TCP port 873 by default. The premise is that both parties need to install rsync. The client can not start the rsync service, but it needs simple configuration. The server needs to be started and rsync needs to be configured on the server.

two SSH Authentication: system user authentication, that is, transmission through SSH tunnel on rsync. The premise is that the server and client need to establish a password free login. See here《 How to set SSH login without password in Linux 》。 There is no need to configure rsync between the server and the client, nor start the rsync service, just install rsync on both sides.

4. Set the rsync server password 10.28.204.65

This tutorial uses rsync-daemon Authentication mode. Create an access password in the format of user name: password, one per line, clear text.

 $ echo "renwole:renwolecom"  >>/etc/rsync.password $ chmod 600 /etc/rsync.password

5. Configure the rsync server 10.28.204.65

The contents after configuration are as follows:

 $ cat /etc/rsyncd.conf Uid=root # The user running the RSYNC daemon Gid=root # The group running the RSYNC daemon Port=873 # default port #Address=10.28.204.65 # Server IP address #Pid file=/var/run/rsyncd.pid # After the process is started, the process number is stored in the path Lock file=/var/run/rsync. lock # Set lock file name Log file=/var/log/rsyncd.log # Specify the log file of rsync Use chroot=no # Do not use chroot Read only=yes # Read only, do not let the client upload files to the server Transfer logging=yes # Log the transfer operation to the transfer log file Hosts allow=10.28.204.66 # Which hosts are allowed to access (multiple hosts are separated by spaces) Hosts deny=* # Which hosts are denied access Max connections=3 # Maximum connections #Motd file=/etc/rsyncd. motd # Login welcome information (not recommended for production environment) Log format=% t% a% m% f% b # Specify the format of the log record Syslog facility=local3 # message level Timeout=600 # Session timeout. [renwolecom] # Name of the module, which can be customized Path=/apps/www # Directory to be synchronized List=yes # Whether users are allowed to list files, which is true by default Ignore errors # Ignore error messages #Exclude=myrenwole/# Out of sync directories (multiple directories separated by spaces) Comment=RenwoleCombak # Comment content, any Auth users=renwole # Only those users are allowed to connect to the module. Multiple users are separated by Secrets file=/etc/rsyncs.pass # Password file required for authentication

For more configurations, see: //www.gsp.com/cgi-bin/man.cgi? topic=rsyncd.conf

Two notes ：

be careful: If you copy the above configuration items, please remove the comments, otherwise unknown problems may occur.
be careful: The options in the global configuration are valid for all modules; What is defined under the module is only valid for the current module; In addition, the option values defined in the module take precedence over the global configuration.

6. Set firewall

 $ firewall-cmd --add-port=873/tcp --permanent $ firewall-cmd --add-port=873/udp --permanent $ firewall-cmd --reload

7. Configure the rsync client 10.28.204.66

The client does not need to configure the module or start the service. The configuration file only needs simple configuration, such as:

 $ vim /etc/rsyncd.conf uid = nobody gid = nobody use chroot = no max connections = 10 pid file = /var/run/rsyncd.pid lock file = /var/run/rsyncd.lock log file = /var/log/rsyncd.log port = 873 secrets file = /etc/client.pass

8. Set the password 10.28.204.66 on the client

Add password and set permissions:

 $ echo "renwolecom"  >>/etc/client.pass $ chmod 600 /etc/client.pass

explain: Just add the password of the server, and no user is required.

9. Start and add the automatic startup

 $ systemctl start rsync $ systemctl enable rsync $ systemctl list-unit-files

10. Test rsync file synchronization

Rsync client 10.28.204.66 Connection server test

 $ /usr/bin/rsync -avzrtopg --progress --delete --password-file=/etc/client.pass  renwole@10.28.204.65 ::renwolecom /apps/www

Description of client connection parameters:

-avzrtopg Split explanation:

 A # Archive mode, which means that files are transferred recursively and all file attributes are maintained, equal to - rlptgoD; V # Detailed mode output; Z # Compress the backed up files during transmission; R # Recursive processing of subdirectories; Topg # Maintain the original file attributes such as primary and time parameters. --Progress # Display detailed synchronization progress. --Delete # If the server side deletes this file, the client side will delete it accordingly to keep the file consistent.

For more parameters, see rsync help:

 $ /usr/bin/rsync -h

Finally, the problem summary

Possible error messages:

@ERROR: auth failed on module renwole

There are two reasons for this error:

1. Either the user password configured on the server is incorrect.
2. Either the password files of the server and client are not authorized by 600.

 rsync: failed to connect to 10.28.204.65 (10.28.204.65): No route to host (113) rsync error: error in socket IO (code 10) at clientserver.c(125) [Receiver=3.1.2]

This failure to connect to the rsync server only causes an error:

1. The firewall is not allowed eight hundred and seventy-three The port or service is not started. Solution: just close the firewall or pass the port.

If an error is reported during startup, check the log to see how to solve it.

If you think the configuration is troublesome, it is recommended that you use lsyncd , this tool is more useful, please refer to:

《 How to use Lsyncd to replicate and synchronize to remote servers in real time 》

Hyper-V virtual host Linux Centos 7 data disk expansion

Comment

All the time Hyper-V Linux Disk expansion of virtual host is troublesome. Today we will introduce how to use fdisk and resize2fs Data disk capacity expansion.

Description:

Host system: Windows Server 2012 R2 Hyper-V
Virtual host: CentOS Linux release 7.4.1708 (Core)

Usual contact Hyper-V More, so disk expansion is common. Windwos Disk expansion is the simplest. This article will not cover it.
Today I will Centos The virtual host performs the demonstration. The small machine is divided into two disks (both independent disks), and one is the system disk /dev/sda , the other is the database disk /dev/sdb The current requirement is that SDB should change from the original 30GB Expand to 100GB 。

1. Check the disk partition and mounting status before capacity expansion

 [ root@renwole-com  ~]# fdisk -l Disk /dev/sdb: 32.2 GB, 32212254720 bytes, 62914560 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disk label type: dos Disk identifier: 0x5f149419 Device Boot      Start         End      Blocks   Id  System /dev/sdb1            2048    62914559    31456256   83  Linux Disk /dev/sda: 10.7 GB, 10737418240 bytes, 20971520 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disk label type: dos Disk identifier: 0x00043041 Device Boot      Start         End      Blocks   Id  System /dev/sda1   *        2048     2099199     1048576   83  Linux /dev/sda2         2099200    20971519     9436160   8e  Linux LVM Disk /dev/mapper/cl-root: 8585 MB, 8585740288 bytes, 16769024 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disk /dev/mapper/cl-swap: 1073 MB, 1073741824 bytes, 2097152 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes
 [ root@renwole-com  ~]# df -hT Filesystem          Type      Size  Used Avail Use% Mounted on /dev/mapper/cl-root xfs       8.0G  950M  7.1G  12% / devtmpfs            devtmpfs  1.9G     0  1.9G   0% /dev tmpfs               tmpfs     1.9G     0  1.9G   0% /dev/shm tmpfs               tmpfs     1.9G  8.3M  1.9G   1% /run tmpfs               tmpfs     1.9G     0  1.9G   0% /sys/fs/cgroup /dev/sda1           xfs      1014M  138M  877M  14% /boot tmpfs               tmpfs     379M     0  379M   0% /run/user/0 /dev/sdb1           ext4       30G   60M   28G   1% /apps

Note: orange part /dev/sdb1 Partition size is 30GB , which is also attached 30GB 。

2. Unload and mount

delete /etc/fstab The following boot disk auto mount information in the file:

/dev/sdb1 /apps ext4 defaults 0 0

3. Start virtual host disk expansion

Hyper-V virtual machine operations:

open Hyper-V Manager, find the small machine that needs to be expanded and shut down, then right-click Settings, click the disk that needs to be expanded, (usually the data disk) Edit - Next by default - Select Next by extension, this step needs attention, assuming that your original disk is 30G , then you want to expand to 100G , please fill in one hundred Click Next to finish. Then start the virtual host.

View the disk partitions after resizing:

 [ root@renwole-com  ~]# fdisk -l Disk /dev/sda: 10.7 GB, 10737418240 bytes, 20971520 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disk label type: dos Disk identifier: 0x00043041 Device Boot      Start         End      Blocks   Id  System /dev/sda1   *        2048     2099199     1048576   83  Linux /dev/sda2         2099200    20971519     9436160   8e  Linux LVM Disk /dev/sdb: 107.4 GB, 107374182400 bytes, 209715200 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disk label type: dos Disk identifier: 0x5f149419 Device Boot      Start         End      Blocks   Id  System /dev/sdb1            2048    62914559    31456256   83  Linux Disk /dev/mapper/cl-root: 8585 MB, 8585740288 bytes, 16769024 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disk /dev/mapper/cl-swap: 1073 MB, 1073741824 bytes, 2097152 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes

sdb The partition size is changed from the original 30GB Expand to 100GB , but partition sdb1 No capacity expansion is used 70GB What about space?

4. Delete partition

Delete different partitions, delete data, delete partitions and rebuild them:

 [ root@renwole-com  ~]# fdisk /dev/sdb The device presents a logical sector size that is smaller than the physical sector size.  Aligning to a physical sector (or optimal I/O) size boundary is recommended, or performance may be impacted. Welcome to fdisk (util-linux 2.23.2). Changes will remain in memory only, until you decide to write them. Be careful before using the write command. Command (m for help): d #Delete sdb1 partition Selected partition 1 Partition 1 is deleted Command (m for help): n #New Section Partition type: p   primary (0 primary, 0 extended, 4 free) e   extended Select (default p): p Partition number (1-4, default 1): one #Specify partition code First sector (2048-209715199, default 2048): Using default value 2048 Last sector, +sectors or +size{K,M,G} (2048-209715199, default 209715199): Using default value 209715199 Partition 1 of type Linux and of size 100 GiB is set Command (m for help): w #Enter w to save The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks.

Check the disk partition again

 [ root@renwole-com  ~]# fdisk -l ... Disk /dev/sdb: 107.4 GB, 107374182400 bytes, 209715200 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disk label type: dos Disk identifier: 0x5f149419 Device Boot      Start         End      Blocks   Id  System /dev/sdb1            2048   209715199   104856576   83  Linux ...

Has changed from the original 30G Expand to 100G However, it can't be used yet. The following operations are required to ensure successful capacity expansion:

 [ root@renwole-com  ~]# resize2fs -f /dev/sdb1 resize2fs 1.42.9 (28-Dec-2013) Resizing the filesystem on /dev/sdb1 to 26214144 (4k) blocks. The filesystem on /dev/sdb1 is now 26214144 blocks long.

Capacity expansion succeeded.

5. Attach the disk

 [ root@renwole-com  ~]# mount /dev/sdb1 /apps/ [ root@renwole-com  ~]# df -hT Filesystem          Type      Size  Used Avail Use% Mounted on /dev/mapper/cl-root xfs       8.0G  950M  7.1G  12% / devtmpfs            devtmpfs  1.9G     0  1.9G   0% /dev tmpfs               tmpfs     1.9G     0  1.9G   0% /dev/shm tmpfs               tmpfs     1.9G  8.3M  1.9G   1% /run tmpfs               tmpfs     1.9G     0  1.9G   0% /sys/fs/cgroup /dev/sda1           xfs      1014M  138M  877M  14% /boot tmpfs               tmpfs     379M     0  379M   0% /run/user/0 /dev/sdb1           ext4       99G   60M   94G   1% /apps [ root@renwole-com  ~]# ls /apps/ web  mysql

As shown above, the data disk of the Linux virtual host has been successfully expanded, and the data is not lost.

In addition, write your disk information to /etc/fstab File, otherwise the disk needs to be manually mounted after reboot.

Note: In the same sentence, before operating any data, you must first back it up to form a good habit to prevent data loss.

CentOS 7 configuration uses SFTP server

Comment

What is SFTP?

SFTP, namely SSH File Transfer Protocol, or Secure File Transfer Protocol. SFTP is an independent SSH encapsulation protocol package, which works in a similar way through a secure connection. Its advantage is that it can use a secure connection to transfer files and browse file systems on local and remote systems.

In many cases, SFTP is preferable to FTP because it has the most basic security features and the ability to use SSH connections. FTP is an insecure protocol that can only be used in limited circumstances or on networks you trust.

precondition:

The server OpenSSH-Server The minimum version is 4.8p1, because the configuration permission requires new configuration items added by the version ChrootDirectory To complete.

To view the OpenSSH version, the command is as follows:

 $ ssh -V OpenSSH_6.6.1p1, OpenSSL 1.0.1e-fips 11 Feb 2013

1. Create user information

Add user group:

 $ groupadd sftp

Add user:

 $ useradd -g sftp -s /sbin/nologin -M userrenwolecom

Parameter annotation:

 -G # Join user group -S # Specify the shell used by the user after logging in /Sbin/nologin # User is not allowed to log in -M # Do not automatically create user login directory

Set user password:

 $ passwd userrenwolecom

2. Create user directory and set permissions

Create the sftp home directory:

 $ mkdir /data/sftp

Set sftp home directory permissions:

 $ chown root:sftp /data/sftp

The folder owner must be root, and the user group may not be root.

 $ chmod 744 /data/sftp

The permission cannot exceed 755 but does not include 755, otherwise it will cause login error.

3. Create upload directory and set permissions

stay /data/sftp/ Create in Home Directory uploads Folder, and set the owner as: userrenwolecom , the user group belongs to: sftp In this way, the newly added account can have the permission to upload and edit.

 $ mkdir -p /data/sftp/uploads $ chown userrenwolecom:sftp /data/sftp/uploads

4. Modify the sshd_config configuration file

 $ vim /etc/ssh/sshd_config

Comment out this line, for example:

#Subsystem sftp /usr/libexec/openssh/sftp-server

Add the following below this line:

 Subsystem sftp internal sftp # Specify to use the internal sftp service provided by the system Match Group sftp # Users matching sftp group. To match multiple groups, separate them with commas ChrootDirectory/data/sftp/# Restrict user's root directory ForceCommand internal sftp # can only be used for sftp login AllowTcpForwarding no # Prevent users from using port forwarding X11Forward no # Forbid users to use port forwarding

5. Restart the SSH service

 $ systemctl restart sshd

6. Test whether login, upload, download and other operations can be performed

stay 10.28.204.61 The server executes the following command to log in:

 $ sftp -P 12012  userrenwolecom@10.28.204.62 userrenwolecom@10.28.204.62 's password: packet_write_wait: Connection to 10.28.204.62 port 12012: Broken pipe Couldn't read packet: Connection reset by peer

An error is reported. This problem is due to /data/sftp The owner of this directory is changed to root Yes, the user group can root / sftp 。

Test the connection again:

 $ sftp -P 12012  userrenwolecom@10.28.204.62 The authenticity of host '[10.28.204.62]:12012 ([10.28.204.62]:12012)' can't be established. ECDSA key fingerprint is SHA256:/YI/L4RT1QH7lkfxMCAkKnvniQslyUl15mOUKUo8K3k. ECDSA key fingerprint is MD5:6d:b6:f3:93:8e:48:53:24:9d:5d:c2:2a:5f:28:f4:d2. Are you sure you want to continue connecting (yes/no)? YES Warning: Permanently added '[10.28.204.62]:12012' (ECDSA) to the list of known hosts. userrenwolecom@10.28.204.62 's password: [Enter userrenwolecom user password] Connected to 10.28.204.62. sftp>

The connection is successful. Now test to upload a file to 10.28.204.62 The server

upload

 sftp> put /tmp/nginx_log_stat /uploads Uploading /tmp/nginx_log_stat to /uploads/nginx_log_stat /tmp/nginx_log_stat

download

 sftp> get uploads/nginx_log_stat /mnt Fetching /uploads/nginx_log_stat to /mnt/nginx_log_stat /uploads/nginx_log_stat   100%    7   1.1KB/s   00:00 sftp>

delete

 sftp> rm /uploads/nginx_log_stat Removing /uploads/nginx_log_stat

For more commands, see:

 sftp> help

Summary:

Encountered some strange problems in the configuration process, I will sshd_config In the configuration file Match Group sftp Modify to Match User userrenwolecom After the sshd is restarted, the current terminal will not be disconnected. After the correct password is entered in the new terminal, the SSH login flash back (automatically closing the terminal) problem will occur PUTTY still Xshell All the same. Later, I changed the user into a group and it became normal. The specific reason is unknown, but the effect of changing the user into a group is the same. If you know why, please give me some advice.

How to set SSH login without password in Linux

Comment

As an operation and maintenance personnel, there is still room for managing 1-5 machines, but if there are 10, 100 or more servers, it is very tedious to enter the password every time you log in, and it takes time and effort to improve the work efficiency.
Today we use ssh-kengen The command generates a private key&public key pair for password free SSH login. There are two algorithms: RSA and DSA 。

RSA It is an asymmetric encryption algorithm and can be used for encryption and signature.
DSA (Digital Signature Algorithm) can only be used for digital signature algorithms.

The following operations apply to the OS: Centos 7 、 Ubuntu 17 Other systems have not been tested and can be used in theory.

The server:

 10.10.204.63 10.10.204.64

1. How to generate ssh public key

Sign in 10.10.204.63 The server generates a public private key pair:

 [ root@10-10-204-63  ~]# ssh-keygen -b 4096 -t rsa Generating public/private rsa key pair. Enter file in which to save the key (/root/.ssh/id_rsa): Created directory '/root/.ssh'. Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /root/.ssh/id_rsa. Your public key has been saved in /root/.ssh/id_rsa.pub. The key fingerprint is: SHA256:qLcoj2nSzq6G9ZpFQZ/OFqFT+oBDf3ousHkt82F1/xM  root@10-10-204-63.10.10.204.63 The key's randomart image is: +---[RSA 4096]----+ |  . .  o          | | . + =  o         | |  o B =          | |   .  X o         | |  .  o B S .      | |  .= * . . .   E  | |.oo. B *     .  . | |oo+*. O o     ..  | |o*O+o o       .. | +----[SHA256]-----+

Generate after three carriage returns ssh key 。

Notes:

-b Specify the key length. For RSA keys, the minimum requirement is 768 bits, the default is 2048 bits, and the maximum length is 4096 bytes.
-t Specify the type of key to create. You can use: "rsa1" (SSH-1), "rsa" (SSH-2), "dsa" (SSH-2).

2. View the generated file

 [ root@10-10-204-63  ~]# ll .ssh/ total 8 -rw------- 1 root root 3243 Nov 25 15:58 id_rsa -rw-r--r-- 1 root root  758 Nov 25 15:58 id_rsa.pub

explain:

id_rsa Private key
id_rsa.pub Public key

3. Upload the public key to 10.10.204.64

 [ root@10-10-204-63  ~]# ssh-copy-id -i ~/.ssh/id_rsa.pub  root@10.10.204.64 /usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub" The authenticity of host '10.10.204.64 (10.10.204.64)' can't be established. ECDSA key fingerprint is SHA256:/YI/L4RT1QH7lkfxMCAkKnvniQslyUl15mOUKUo8K3k. ECDSA key fingerprint is MD5:6d:b6:f3:93:8e:48:53:24:9d:5d:c2:2a:5f:28:f4:d2. Are you sure you want to continue connecting (yes/no)?  Yes [Enter yes] /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys root@10.10.204.64 's password: [Enter the server password] Number of key(s) added: 1 Now try logging into the machine, with:   "ssh ' root@10.10.204.64 '" and check to make sure that only the key(s) you wanted were added.

Upload succeeded.

4. Modify SSH configuration file

Sign in 10.28.204.64 Modify as follows:

 $ vim /etc/ssh/sshd_config

Remove the following notes:

 RSAAuthentication yes PubkeyAuthentication yes

5. Restart the SSH service

 $ systemctl restart sshd

6. Test password free login 10.10.204.64

 [ root@10-10-204-63  ~]# ssh ' root@10.10.204.64 ' Last failed login: Sat Nov 25 16:09:48 CST 2017 from 83.234.149.66 on ssh:notty There was 1 failed login attempt since the last successful login. Last login: Sat Nov 25 15:57:33 2017 from 36.7.69.84 [ root@10-10-204-64  ~]#

Log in successfully without entering a password.

After successful login, it is recommended to 10.10.204.64 The ssh public key is also generated on the server and uploaded to 10.10.204.63 Server, so that we can mutually login via SSH without password. This is also true for multiple servers.

7. View public key

 [ root@10-10-204-64  ~]# ll /root/.ssh/ total 8 -rw-------  1 root root 758 Nov 25 16:08 authorized_keys -rw-r--r--.  1 root root 175 Aug  9 09:19 known_hosts

authorized_keys Is the name of the public key just uploaded

8. If the public key is lost, you can use the private key to generate the public key again. The command is as follows:

 [ root@10-10-204-63  ~]# ssh-keygen -y -f ~/.ssh/id_rsa > ~/.ssh/id_rsa.pub

end.

MySQL is automatically backed up and submitted to the CodeCloud Git warehouse

Comment

The reason for choosing Code Cloud is that its private warehouse is free, and GitHub Private warehouses are charged, about $7+a month, which seems to be a big expense for individual developers.

Automatically upload the backup to the code cloud when creating Git Before the warehouse, we suggest you read the following articles:

《 How to push local projects to the code cloud or GitHub through Git 》

Not very familiar with Git This article is very useful to our friends. Otherwise, various errors will be reported during the following steps.

Implementation scheme:

    • Log in to BitCloud
    • Create private warehouse
    • The server generates the ssh public key
    • Create SSH public key

The above steps are described in detail above.

1. Create shell script file:

 $ cd /mnt/renwole $ vim mysqlbak.sh

2. Add the following:

 #!/ bin/bash createAt=`date +%Y-%m-%d-%H:%M:%S` mysql_back_path=/mnt/MySQL-Bak /usr/local/mysql/bin/mysqldump -u Database user name -p password  Database name > $mysql_back_path/renwoleblog-$createAt.sql #Automatically delete backups older than 7 days #- type file type f is a file find $mysql_back_path -name "*.sql" -type f -mtime +7 -exec rm -rf {} \; cd $mysql_back_path git add -A git commit -m "${createAt}" git push origin master

be careful: The above path must be an absolute path, otherwise execute crontab after mysqldump Succeeded, but the export result is empty. Manual execution is normal!

3. Set execution permission:

 $ chmod +x mysqlbak.sh

4. Add task plan:

adopt crontab Scheduled execution of backup scripts

 $ crontab -e

Insert the following:

 */50 2 * * * /mnt/renwole/mysqlbak.sh
 $ systemctl restart crond

After the setting is completed, automatic backup will be performed at 2:50 every morning mysql Database and push to the code cloud git Warehouse, which can be used later Git View the historical submitted version. Don't worry about data loss anymore.

How to submit local projects to CodeCloud or GitHub through Git

Comment

What is Code Cloud?

Code Cloud is a stable, efficient and secure cloud software development collaboration platform for developers.
Whether individuals, teams, or enterprises can use code cloud to realize code hosting, project management, and collaborative development.

PS：

To put it bluntly, it is the same as Github. But there are not as many developers and projects as Github. Anyway, they are global.
However, the code cloud is fast (after all, it is in China), the private warehouse is free, and there is no limit on the number of private and public databases. It can be used as a backup warehouse.

OS environment: CentOS Linux release 7.4.1708 (Core)

1. Install and configure Git

 $ yum install git -y $ git config --global user.name "renwole" $ git config --global user.email  renwole@renwole.com

2. Generate ssh public key on the terminal

Execute the following command to generate the ssh key ：

 $ ssh-keygen -t rsa -C " renwole@renwole.com "

3. View the generated public key

 $ cat ~/.ssh/id_rsa.pub ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDR9k1FgZRJN3P7V9tLfWZQ......

Add the above printed screen content to the code cloud.

explain: SSH key Add address: //gitee.com/profile/sshkeys 。 The public key title is arbitrary.

4. Test whether the connection can be trusted

After adding, enter the following command in the terminal:

 $ ssh -T  git@gitee.com The authenticity of host 'gitee.com (120.55.226.24)' can't be established. ECDSA key fingerprint is SHA256:FQGC9Kn/eye1W8icdBgrQp+KkGYoFgbVr17bmjey0Wc. ECDSA key fingerprint is MD5:27:e5:d3:79:2a:9e:eb:6c:93:cd:1f:c1:47:a3:54:b1. Are you sure you want to continue connecting (yes/no)?  yes [Enter yes] Warning: Permanently added 'gitee.com,120.55.226.24' (ECDSA) to the list of known hosts. Authentication failed.

Execute again:

 $ ssh -T  git@gitee.com Welcome to Gitee.com, yourname!

The returned message indicates that it was successful.

5. Create a private warehouse in the code cloud

Open the home page of Code Cloud – login – click the circle+in the upper right corner – fill in your project information – creation is complete.

Open your project address, for example:

//gitee.com/renwole/renwolecom

Click Clone/Download – to obtain your SSH address, which will be used below.

6. Clone the remote warehouse project to the local

 $ cd /mnt $ git clone  git@gitee.com :renwole/renwolecom.git Cloning into 'renwolecom'... remote: Counting objects: 11, done. remote: Compressing objects: 100% (10/10), done. remote: Total 11 (delta 2), reused 0 (delta 0) Receiving objects: 100% (11/11), 19.51 MiB | 5.46 MiB/s, done. Resolving deltas: 100% (2/2), done.
 $ cd renwolecom $ ll total 16 -rw-r--r--.  1 root root 10254 Nov 22 09:26 LICENSE -rw-r--r--.  1 root root    13 Nov 22 09:26 README.md

Successfully cloned to local.

7. Submit project

You can renwolecom The directory is regarded as your project, and you can submit it directly after completing the code.

No matter how many people participate in this project, push Before pull , see if there is merge Code, if present merge Just solve it merge , if none, please execute the following command:

 $ git pull origin master $ git add -A $ git commit -m 'init' $ git push origin master

8. Delete all submission history records of GIT warehouse

 8.1.Checkout $ git checkout --orphan latest_branch 8.2.  Add all the files $ git add -A 8.3.  Commit the changes $ git commit -am "commit message" 8.4.  Delete the branch $ git branch -D master 8.5.Rename the current branch to master $ git branch -m master 8.6.Finally, force update your repository $ git push -f origin master

Delete remote files and keep them locally:

 $git rm -- cached file name $ git commit -m "remove file from remote repository" $ git push

Delete remote folder and keep it locally:

 $git rm -- cached - r folder name $ git commit -m "remove directory from remote repository" $ git push

Note: Please use the - r parameter to delete a folder.

Note: Optional parameters for the last upload project git push origin master -f Indicates that the file to be uploaded is forced to be pushed.

After execution, you can refresh the project address and view push The contents of the warehouse.