Automatically install ESXi 7 using PXE boot and kickstart (UEFI) based on DHCP server

Automatically install ESXi 7 using PXE boot and kickstart (UEFI) based on DHCP server
Installing ESXi 7 with PXE based on UEFI
Details of PXE network boot automated installation of ESXi 7
PXE+Kickstart Batch Install ESXi 7
Using PXE and TFTP to Boot ESXi Installing ESXi 7 Programs

1. Automatically install ESXi using PXE boot and kickstart (UEFI) based on DHCP server 7. Record video

2 Inspur NF5280M5 server uses PXE boot and kickstart (UEFI) to automatically install ESXi 7 based on DHCP server to record video

 #CentOS 7 system installs dhcp server, tftp server server and web server
 yum install -y dhcp tftp-server httpd

1. Configure DHCP Server

 Daemon:/etc/sbin/dhcpd
 Script:/etc/init.d/dhcpd
 Port: 67 (bootps) server, 68 (bootpc) client
 Configuration file:/etc/dhcp/dhcpd.conf
 Lease information:/var/lib/dhcpd/dhcpd.leases
 Configuration file: cp/usr/share/doc/dhcp-4.2.5/dhcpd.conf.example/etc/dhcp/dhcpd.conf
 Modify DHCP listening specific port vi /etc/sysconfig/dhcpd
 
 #Edit DHCP Server Profile
 vi /etc/dhcp/dhcpd.conf
 
 default-lease-time six hundred ;
 max-lease-time seven thousand and two hundred ;
 log-facility local7;
 
 subnet ten .53.220.0 netmask two hundred and fifty-five .255.255.0 {
 range ten .53.220.41 ten .53.220.49;
 option domain-name-servers ten .1.0.1, one hundred and fourteen .114.114.114;    #DNS server IP address
 option routers ten .53.220.1;                              #Gateway address
 next-server ten .53.220.224;                                #Specify TFTP server address
 filename "/mboot.efi" ;                                    #Specify the network boot image file
 }
 #Binding IP address
 host zhangfangzhou {                                     #You can specify it at will, but it must be unique
 hardware ethernet 00 :50:56:99:06:b7;
 fixed-address ten .53.220.48;
 }
 

#DHCP service settings Start up and start DHCP service
systemctl enable dhcpd && systemctl start dhcpd

########Firewall settings
firewall-cmd --zone=public --add-port=67/udp --permanent
firewall-cmd --zone=public --add-port=68/udp --permanent
#Reload configuration
firewall-cmd --reload
Firewall cmd -- list all # View firewall rules and only display firewall policies in/etc/firewalld/zones/public.xml

2. Configure tftp server

 Enable tftp service
 vi /etc/xinetd.d/tftp
 
 # default: off
 # description: The tftp server serves files using the trivial file transfer \
 #       protocol.   The tftp protocol is often used to boot diskless \
 #       workstations,  download configuration files to network-aware printers, \
 #       and to start the installation process for some operating systems.
 service tftp
 {
        socket_type             = dgram
        protocol                 = udp
        wait                     =  yes
        user                     = root
        server                   = /usr/sbin/in.tftpd
        server_args             =  -s /var/lib/tftpboot #TFTP Server Top Level Directory
        disable                 = no     #Change from yes to no
        per_source               =  eleven
        cps                     =  one hundred  two
        flags                   = IPv4
 }
 

#Service settings of tftp Start up and start the tftp service
systemctl enable tftp && systemctl start tftp

########Firewall settings
firewall-cmd --zone=public --add-port=69/tcp --permanent
firewall-cmd --zone=public --add-port=69/udp --permanent
#Reload configuration
firewall-cmd --reload
Firewall cmd -- list all # View firewall rules and only display firewall policies in/etc/firewalld/zones/public.xml

3. Mount ESXi 7 image

 #Create folders, mainly to download ESXi 7 and mount ESXi 7
 mkdir  -p /var/lib/tftpboot/{iso,ESXi70u2}
 wget  -P /var/lib/tftpboot/iso   http://10.53.123.144/ISO/ESXi/VMware-VMvisor-Installer-7.0U2a-17867351.x86_64.iso
 cd /var/lib/tftpboot
 Mount/var/lib/tftpboot/iso/VMware-VMVisor-Installer-7.0U2a-17867351.x86_64.iso/var/lib/tftpboot/ESXi70u2 # You need to remount after restarting
 
 #1. UEFI starts ESXi 7
 Copy the boot image file specified by the above DHCP server to the specified directory. The above setting example is named "mboot. efi", so copy it to that name.
 cd /var/lib/tftpboot/
 cp  -p /var/lib/tftpboot/ESXi70u2/efi/boot/bootx64.efi /var/lib/tftpboot/mboot.efi
 
 Copy the file named boot.cfg directly under tftpboot, which describes the boot settings.
 cp  -p /var/lib/tftpboot/ESXi70u2/efi/boot/boot.cfg /var/lib/tftpboot/boot.cfg
  #2. Traditional BIOS boot ESXi 7 If you want to use the old BIOS, you need the syslinux package version 3.86. From https://www.kernel.org/pub/linux/utils/boot/syslinux/ Download Package cd /tmp wget  https://mirrors.edge.kernel.org/pub/linux/utils/boot/syslinux/3.xx/syslinux-3.86.tar.gz tar xvzf syslinux-3.86.tar.gz cp -p syslinux-3.86/core/mboot.efi /var/lib/tftpboot/ #3. Modify boot.cfg Delete the "/" of the file path in boot.cfg sed -i 's|/||g' boot.cfg

4. Check boot.cfg

 Change the Title and Prefix according to your environment. It is a good idea to use a descriptive name for "title", which is the title character displayed during automatic installation. Prefix specifies the directory where ESXi installers are stored. In this case, it will be ESXi70u2.
 
 Modify title and prefix names Prefix=ESXi70u2 (ESXi70u2 is the folder of ESXi ISO file mount above) 
 # vi /var/lib/tftpboot/boot.cfg
 bootstate = zero
 title = Loading ESXi installer www.zhangfangzhou.cn
 timeout = five prefix=ESXi70u2 kernel=b.b00 kernelopt=runweasel cdromBoot modules=jumpstrt.gz --- useropts.gz --- features.gz --- k.b00 --- uc_intel.b00 --- uc_amd.b00 --- uc_hygon.b00 --- procfs.b00 --- vmx.v00 --- vim.v00 --- tpm.v00 --- sb.v00 --- s.v00 --- atlantic.v00 --- bnxtnet.v00 --- bnxtroce.v00 --- brcmfcoe.v00 --- brcmnvme.v00 --- elxiscsi.v00 --- elxnet.v00 --- i40enu.v00 --- iavmd.v00 --- icen.v00 --- igbn .v00 --- irdman.v00 --- iser.v00 --- ixgben.v00 --- lpfc.v00 --- lpnic.v00 --- lsi_mr3.v00 --- lsi_msgp.v00 --- lsi_msgp.v01 --- lsi_msgp.v02 --- mtip32xx.v00 --- ne1000.v00 --- nenic.v00 --- nfnic.v00 --- nhpsa.v00 --- nmlx4_co.v00 --- nmlx4_en.v00 --- nmlx4_rd.v00 --- nmlx5_co.v00 --- nmlx5_rd.v00 --- ntg3.v00 --- nvme_pci.v00 --- nvmerdma.v00 - -- nvmxnet3.v00 --- nvmxnet3.v01 --- pvscsi.v00 --- qcnic.v00 --- qedentv.v00 --- qedrntv.v00 --- qfle3.v00 --- qfle3f.v00 --- qfle3i.v00 --- qflge.v00 --- rste.v00 --- sfvmk.v00 --- smartpqi.v00 --- vmkata.v00 --- vmkfcoe.v00 --- vmkusb.v00 --- vmw_ahci.v00 --- clusters.v00 --- crx.v00 --- elx_esx_.v00 --- btldr.v00 --- esx_dvfi.v00 --- esx_ui.v0 0 --- esxupdt.v00 --- tpmesxup.v00 --- weaselin.v00 --- loadesx.v00 --- lsuv2_hp.v00 --- lsuv2_in.v00 --- lsuv2_ls.v00 --- lsuv2_nv.v00 --- lsuv2_oe.v00 --- lsuv2_oe.v01 --- lsuv2_oe.v02 --- lsuv2_sm.v00 --- native_m.v00 --- qlnative.v00 --- vdfs.v00 --- vmware_e.v00 --- vsan.v00 --- vsanheal.v00 --- vsanmgmt.v00 --- tools.t00 --- xorg.v00 --- gc. v00 --- imgdb.tgz --- basemisc.tgz --- resvibs.tgz --- imgpayld.tgz build = seven .0.2-0.0.17867351
 updated = zero

At this point, ESXi can be installed through PXE network

5. Configure web server

 #Default website directory/var/www/html
 
 systemctl enable httpd && systemctl start httpd ########Firewall settings firewall-cmd --zone=public --add-port=80/tcp --permanent firewall-cmd --zone=public --add-port=443/tcp --permanent #Reload configuration firewall-cmd --reload Firewall cmd -- list all # View firewall rules and only display firewall policies in/etc/firewalld/zones/public.xml

6. Configure the kickstart of ESXi 7

 The example kickstart file is stored in/etc/vmware/weasel/ks.cfg of ESXi. Copy this to/var/www/html on the http server
 
 #Example 1: Kickstart file to configure static IP address for ESXi 7
 vi /var/www/html/ks.cfg
 #####################
 # Accept the VMware End User License Agreement
 vmaccepteula
 
 # Set the root password for the DCUI and Tech Support Mode
 rootpw P@ssw0rd
 
 # The install media is in the CD-ROM drive
 install --firstdisk  --overwritevmfs
 
 # Set the network on the first network adapter
 network --bootproto = static --device = vmnic0 --ip = ten .53.220.199 --netmask = two hundred and fifty-five .255.255.0 --vlanid = zero  --gateway = ten .53.220.1 --hostname = ten .53.220.199 --nameserver = ten .1.0.1
 
 #vmserialnum
 vmserialnum --esx = 5U4TK-DML1M-M8550-XK1QP-1A052
 # reboot after install
 reboot
 
 # run the following command only on the firstboot
 %firstboot --interpreter = busybox
 
 sleep  ten
 # enable & start remote ESXi Shell (SSH)
 vim-cmd hostsvc/enable_ssh
 vim-cmd hostsvc/start_ssh
 sleep  one
 # enable & start ESXi Shell (TSM)
 vim-cmd hostsvc/enable_esx_shell
 vim-cmd hostsvc/start_esx_shell
 sleep  one
 # enable High Performance
 esxcli system settings advanced set  --option = /Power/CpuPolicy --string-value = "High Performance"
 sleep  one
 #Disable ipv6
 esxcli network ip set  --ipv6-enabled = zero
 sleep  one
 #Don't join the experience
 esxcli system settings advanced set  -o /UserVars/HostClientCEIPOptIn -i  two
 sleep  one
 #enable ntp
 /bin/esxcli system ntp set  --server = ntp.aliyun.com
 sleep  one
 /bin/chkconfig ntpd on
 sleep  one
 #Disable ShellWarning
 esxcli system settings advanced set  -o /UserVars/SuppressShellWarning -i  one
 sleep  one
 cat << EOF > /etc/ssh/keys-root/authorized_keys
 ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAgEA6PP8/RDdYIqUq6DE3zj9Qs8AF3uzfYH5lYrB + mMxxfI8kq2NIzQlsW1KDaH/fWYTkkK120lPUu97lWic9Li3Z3iFR6Nh4q6PVTfBNt4xOx754Ipqtpefk + 9sLZAYEPK8pnRP0QZv7CtDFv842tfUYIrVNnecRQTFfNtnGDcXnO1u2RE1kq6Tr5N3595PbPLDKczjOFnS + jy0MeKKHPJcZfYz7TUTSzTwTHYbPRRaQ8/0eihUwzpAmXRo9NYNle26qp6 + SlRsjGSBcUr0rh0wSe6r/C2btnpOUd//aFvcl8plhyb ++ nivlhB71v + 6I0UcPoSXOIVs/1QuHEMbv7Ircjb2emqHtZDpk8KSYhgV0ZdbAq9XOcux76eok//xtjbleKPAcDMY/KotIEh7QX4NLQxSJOm5gCLkn5kbrHfVx6nWlGzVVds1sDlcnSAWul5lFiI5ZkArXFKcnm + aCnStPpx5SSCpZpMUdZvt8ZA7vLx3xjMDFwv5HTuTFwB9mlZrdfqp5USC2mWC3eAAPE7GxDSfJv9epteIYywIP9NVT3Z4ng9z6jrcFfA4GFlfLrk8J71cnxZ/AWZXXUwp3ooE/Cp3jc473VpK7FZwjQ7Xz9PD8WQgMOO4xnGQhPWlxhRuoTYyQVa0xOBO9gh9Cuc6zq5FQgYQEcSB + /FBJ /YNDIc = www.zhangfangzhou.cn
 EOF
 sleep  one
 date > /finished.stamp
 
 # Restart a last time
 reboot
 
 
 #Example 2 Kickstart file, ESXi 7 obtains IP address through DHCP server
 ​vi /var/www/html/ks.cfg
 # Accept the VMware End User License Agreement
 vmaccepteula
 
 # Set the root password for the DCUI and Tech Support Mode
 rootpw P@ssw0rd
 
 # The install media is in the CD-ROM drive
 install --firstdisk  --overwritevmfs
 
 # Set the network on the first network adapter
 network --bootproto = dhcp
 
 #vmserialnum
 vmserialnum --esx = 5U4TK-DML1M-M8550-XK1QP-1A052
 # reboot after install
 reboot
 
 # run the following command only on the firstboot
 %firstboot --interpreter = busybox
 
 sleep  ten
 # enable & start remote ESXi Shell (SSH)
 vim-cmd hostsvc/enable_ssh
 vim-cmd hostsvc/start_ssh
 sleep  one
 # enable & start ESXi Shell (TSM)
 vim-cmd hostsvc/enable_esx_shell
 vim-cmd hostsvc/start_esx_shell
 sleep  one
 # enable High Performance
 esxcli system settings advanced set  --option = /Power/CpuPolicy --string-value = "High Performance"
 sleep  one
 #Disable ipv6
 esxcli network ip set  --ipv6-enabled = zero
 sleep  one
 #Don't join the experience
 esxcli system settings advanced set  -o /UserVars/HostClientCEIPOptIn -i  two
 sleep  one
 #enable ntp
 /bin/esxcli system ntp set  --server = ntp.aliyun.com
 sleep  one
 /bin/chkconfig ntpd on
 sleep  one
 #Disable ShellWarning
 esxcli system settings advanced set  -o /UserVars/SuppressShellWarning -i  one
 sleep  one
 cat << EOF > /etc/ssh/keys-root/authorized_keys
 ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAgEA6PP8/RDdYIqUq6DE3zj9Qs8AF3uzfYH5lYrB + mMxxfI8kq2NIzQlsW1KDaH/fWYTkkK120lPUu97lWic9Li3Z3iFR6Nh4q6PVTfBNt4xOx754Ipqtpefk + 9sLZAYEPK8pnRP0QZv7CtDFv842tfUYIrVNnecRQTFfNtnGDcXnO1u2RE1kq6Tr5N3595PbPLDKczjOFnS + jy0MeKKHPJcZfYz7TUTSzTwTHYbPRRaQ8/0eihUwzpAmXRo9NYNle26qp6 + SlRsjGSBcUr0rh0wSe6r/C2btnpOUd//aFvcl8plhyb ++ nivlhB71v + 6I0UcPoSXOIVs/1QuHEMbv7Ircjb2emqHtZDpk8KSYhgV0ZdbAq9XOcux76eok//xtjbleKPAcDMY/KotIEh7QX4NLQxSJOm5gCLkn5kbrHfVx6nWlGzVVds1sDlcnSAWul5lFiI5ZkArXFKcnm + aCnStPpx5SSCpZpMUdZvt8ZA7vLx3xjMDFwv5HTuTFwB9mlZrdfqp5USC2mWC3eAAPE7GxDSfJv9epteIYywIP9NVT3Z4ng9z6jrcFfA4GFlfLrk8J71cnxZ/AWZXXUwp3ooE/Cp3jc473VpK7FZwjQ7Xz9PD8WQgMOO4xnGQhPWlxhRuoTYyQVa0xOBO9gh9Cuc6zq5FQgYQEcSB + /FBJ /YNDIc = www.zhangfangzhou.cn
 EOF
 sleep  one
 date > /finished.stamp
 
 # Restart a last time
 reboot

7. Final boot.cfg configuration file

 # vi /var/lib/tftpboot/boot.cfg
 bootstate = zero
 title = Loading ESXi installer www.zhangfangzhou.cn
 timeout = five
 prefix = ESXi70u2
 kernel = b.b00
 kernelopt = ks = http://10.53.220.224/ks.cfg
 modules=jumpstrt.gz --- useropts.gz --- features.gz --- k.b00 --- uc_intel.b00 --- uc_amd.b00 --- uc_hygon.b00 --- procfs.b00 --- vmx.v00 --- vim.v00 --- tpm.v00 --- sb.v00 --- s.v00 --- atlantic.v00 --- bnxtnet.v00 --- bnxtroce.v00 --- brcmfcoe.v00 --- brcmnvme.v00 --- elxiscsi.v00 --- elxnet.v00 --- i40enu.v00 --- iavmd.v00 --- icen.v00 --- igbn .v00 --- irdman.v00 --- iser.v00 --- ixgben.v00 --- lpfc.v00 --- lpnic.v00 --- lsi_mr3.v00 --- lsi_msgp.v00 --- lsi_msgp.v01 --- lsi_msgp.v02 --- mtip32xx.v00 --- ne1000.v00 --- nenic.v00 --- nfnic.v00 --- nhpsa.v00 --- nmlx4_co.v00 --- nmlx4_en.v00 --- nmlx4_rd.v00 --- nmlx5_co.v00 --- nmlx5_rd.v00 --- ntg3.v00 --- nvme_pci.v00 --- nvmerdma.v00 - -- nvmxnet3.v00 --- nvmxnet3.v01 --- pvscsi.v00 --- qcnic.v00 --- qedentv.v00 --- qedrntv.v00 --- qfle3.v00 --- qfle3f.v00 --- qfle3i.v00 --- qflge.v00 --- rste.v00 --- sfvmk.v00 --- smartpqi.v00 --- vmkata.v00 --- vmkfcoe.v00 --- vmkusb.v00 --- vmw_ahci.v00 --- clusters.v00 --- crx.v00 --- elx_esx_.v00 --- btldr.v00 --- esx_dvfi.v00 --- esx_ui.v0 0 --- esxupdt.v00 --- tpmesxup.v00 --- weaselin.v00 --- loadesx.v00 --- lsuv2_hp.v00 --- lsuv2_in.v00 --- lsuv2_ls.v00 --- lsuv2_nv.v00 --- lsuv2_oe.v00 --- lsuv2_oe.v01 --- lsuv2_oe.v02 --- lsuv2_sm.v00 --- native_m.v00 --- qlnative.v00 --- vdfs.v00 --- vmware_e.v00 --- vsan.v00 --- vsanheal.v00 --- vsanmgmt.v00 --- tools.t00 --- xorg.v00 --- gc. v00 --- imgdb.tgz --- basemisc.tgz --- resvibs.tgz --- imgpayld.tgz build = seven .0.2-0.0.17867351
 updated = zero
 Here, you can automatically install ESXi 7 through PXE based on DHCP, automatically set system password, automatically accept license, automatically add product key, automatically enable SSH service, automatically enable shell service, automatically set power mode to high performance, disable IPV6, not participate in (CEIP) customer experience improvement plan, automatically configure NTP, and automatically add private key.

https://www.zhangfangzhou.cn/pxe-kickstart-install-uefi-esxi-7.html

Read and write test of Samsung PCIe 4.0 NVMe SSD 980 PRO hard disk on VMware ESXi through DirectPath and virtualization

Read and write test of Samsung PCIe 4.0 NVMe SSD 980 PRO hard disk on VMware ESXi through DirectPath and virtualization
After storage virtualization, the read performance loss is about 21.5%, and the write performance loss is about 18.4%

Read and Write Tests of Samsung PCIe 4.0 NVMe SSD 980 PRO Hard Drive with DirectPath and Virtualization on VMware ESXi

Physical server NF5280M5
Hard disk PCIEx4 to NMVe
Virtualization software ESXi 7.03
Virtual machine configuration 8 core 16G memory 120G system disk
operating system CentOS7.9
Data disk Samsung PCIe 4.0 NVMe SSD 980 PRO
testing software fio

1. Software installation

 yum -y install epel-release yum -y install fio

2. Mount data directory

 sudo vgcreate lvmgroup /dev/vdb sudo lvcreate -l 100%FREE -n data lvmgroup sudo mkfs.xfs /dev/lvmgroup/data sudo mkdir /data sudo echo "/dev/lvmgroup/data /data xfs defaults 0 0" >>/etc/fstab sudo mount -av

3. Write fio test configuration file: create a configuration file for fio testing. You can use a text editor to create a new file, such as fio_test_config.fio

 [global] ioengine=libaio direct=1 thread=1 rw=randrw bs=4k numjobs=4 size=32G runtime=300 group_reporting [randwrite] [global]: This section specifies the global configuration, which is applicable to all jobs. Ioengine=libaio: specifies that the I/O engine is libaio, which means asynchronous I/O. Direct=1: Enable direct I/O to bypass the file system cache of the operating system. Thread=1: Each job uses one thread. Rw=randrw: the read/write mode is random. Bs=4k: The block size is 4KB. Numjobs=4: the number of jobs is 4, indicating the number of concurrent jobs. Size=32G: The test file size of each job is 32GB. Runtime=300: the test run time is 300 seconds. Group_reporting: Specifies that the statistics of all threads should be reported together.

Pass through DirectPath

 [ root@10-53-220-58  ~]# cd /data/ [ root@10-53-220-58  data]# ls [ root@10-53-220-58  data]# sudo dd if=/dev/zero of=/data/fio_test_file bs=1G count=32 32+0 records in 32+0 records out 34359738368 bytes (34 GB) copied, 36.9548 s, 930 MB/s [ root@10-53-220-58  data]# vi fio_test_config.fio [ root@10-53-220-58  data]# sudo fio fio_test_config.fio randwrite: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B,  ioengine=libaio, iodepth=1 ... fio-3.7 Starting 4 threads randwrite: Laying out IO file (1 file / 32768MiB) randwrite: Laying out IO file (1 file / 32768MiB) randwrite: Laying out IO file (1 file / 32768MiB) randwrite: Laying out IO file (1 file / 32768MiB) Jobs: 4 (f=4): [m(4)][100.0%][r=102MiB/s,w=103MiB/s][r=26.2k,w=26.3k IOPS][eta 00m:00s]  randwrite: (groupid=0,  jobs=4): err= 0: pid=9048: Fri Apr 12 13:55:32 2024 read: IOPS=26.0k, BW=102MiB/s (107MB/s)(29.8GiB/300001msec) slat (nsec): min=3381,  max=69136, avg=4802.53, stdev=1677.41 clat (usec): min=21,  max=19215, avg=113.61, stdev=218.66 lat (usec): min=34,  max=19220, avg=118.47, stdev=218.66 clat percentiles (usec): |  1.00th=[   48],  5.00th=[   51], 10.00th=[   53], 20.00th=[   55], | 30.00th=[   57], 40.00th=[   59], 50.00th=[   65], 60.00th=[   74], | 70.00th=[   77], 80.00th=[   85], 90.00th=[   93], 95.00th=[  343], | 99.00th=[ 1270], 99.50th=[ 1369], 99.90th=[ 1516], 99.95th=[ 1565], | 99.99th=[ 1958] bw (   KiB/s): min=20680, max=36128, per=24.99%, avg=26004.68, stdev=2194.12, samples=2397 iops        : min= 5170, max= 9032, avg=6501.10, stdev=548.54, samples=2397 write: IOPS=25.0k, BW=102MiB/s (106MB/s)(29.7GiB/300001msec) slat (usec): min=4,  max=135, avg= 6.16, stdev= 1.90 clat (nsec): min=1292,  max=1175.4k, avg=27396.71, stdev=3962.20 lat (usec): min=19,  max=1181, avg=33.62, stdev= 4.09 clat percentiles (nsec): |  1.00th=[18304],  5.00th=[21888], 10.00th=[23680], 20.00th=[24960], | 30.00th=[26496], 40.00th=[26752], 50.00th=[27264], 60.00th=[27520], | 70.00th=[28288], 80.00th=[29312], 90.00th=[31616], 95.00th=[34048], | 99.00th=[39168], 99.50th=[41216], 99.90th=[43776], 99.95th=[45312], | 99.99th=[50944] bw (   KiB/s): min=20280, max=36008, per=24.99%, avg=25980.38, stdev=2247.11, samples=2397 iops        : min= 5070, max= 9002, avg=6495.02, stdev=561.78, samples=2397 lat (usec)   : 2=0.04%, 4=0.01%, 10=0.01%, 20=0.90%, 50=50.88% lat (usec)   : 100=44.43%, 250=1.06%, 500=0.46%, 750=0.46%, 1000=0.53% lat (msec)   : 2=1.22%, 4=0.01%, 10=0.01%, 20=0.01% cpu          : usr=4.49%, sys=11.07%, ctx=15587579, majf=0, minf=10 IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, &amp;amp;gt;=64=0.0% submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, &amp;amp;gt;=64=0.0% complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, &amp;amp;gt;=64=0.0% issued rwts: total=7804560,7797520,0,0 short=0,0,0,0 dropped=0,0,0,0 latency   : target=0, window=0, percentile=100.00%, depth=1 Run status group 0 (all jobs): READ: bw=102MiB/s (107MB/s), 102MiB/s-102MiB/s (107MB/s-107MB/s), io=29.8GiB (31.0GB),  run=300001-300001msec WRITE: bw=102MiB/s (106MB/s), 102MiB/s-102MiB/s (106MB/s-106MB/s), io=29.7GiB (31.9GB),  run=300001-300001msec Disk stats (read/write): dm-2: ios=7801557/7794611, merge=0/0, ticks=862107/186550, in_queue=1048584, util=100.00%, aggrios=7804560/7797562, aggrmerge=0/1, aggrticks=857542/181607, aggrin_queue=1039149, aggrutil=100.00% nvme0n1: ios=7804560/7797562, merge=0/1, ticks=857542/181607, in_queue=1039149, util=100.00% www.zhangfangzhou.cn

Virtualization

 [ root@10-53-220-44  data]# sudo dd if=/dev/zero of=/data/fio_test_file bs=1G count=32 32+0 records in 32+0 records out 34359738368 bytes (34 GB) copied, 71.8294 s, 478 MB/s [ root@10-53-220-44  data]# vi fio_test_config.fio [ root@10-53-220-44  data]# sudo fio fio_test_config.fio randwrite: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B,  ioengine=libaio, iodepth=1 ... fio-3.7 Starting 4 threads randwrite: Laying out IO file (1 file / 32768MiB) randwrite: Laying out IO file (1 file / 32768MiB) randwrite: Laying out IO file (1 file / 32768MiB) randwrite: Laying out IO file (1 file / 32768MiB) Jobs: 4 (f=4): [m(4)][100.0%][r=78.2MiB/s,w=77.8MiB/s][r=20.0k,w=19.9k IOPS][eta 00m:00s] randwrite: (groupid=0,  jobs=4): err= 0: pid=9016: Fri Apr 12 14:43:21 2024 read: IOPS=20.4k, BW=79.7MiB/s (83.6MB/s)(23.4GiB/300001msec) slat (usec): min=4,  max=114, avg= 6.78, stdev= 1.98 clat (usec): min=42,  max=38534, avg=134.39, stdev=190.16 lat (usec): min=53,  max=38545, avg=141.24, stdev=190.14 clat percentiles (usec): |  1.00th=[   79],  5.00th=[   87], 10.00th=[   89], 20.00th=[   92], | 30.00th=[   93], 40.00th=[   96], 50.00th=[   98], 60.00th=[  101], | 70.00th=[  104], 80.00th=[  106], 90.00th=[  113], 95.00th=[  161], | 99.00th=[ 1205], 99.50th=[ 1319], 99.90th=[ 1467], 99.95th=[ 1516], | 99.99th=[ 2057] bw (   KiB/s): min=   73, max=24856, per=16.35%, avg=13347.32, stdev=9723.44, samples=2397 iops        : min=   18, max= 6214, avg=3336.67, stdev=2431.02, samples=2397 write: IOPS=20.4k, BW=79.6MiB/s (83.5MB/s)(23.3GiB/300001msec) slat (usec): min=5,  max=115, avg= 8.06, stdev= 1.98 clat (nsec): min=1403,  max=38544k, avg=44914.21, stdev=37185.93 lat (usec): min=32,  max=38560, avg=53.04, stdev=37.23 clat percentiles (usec): |  1.00th=[   36],  5.00th=[   39], 10.00th=[   40], 20.00th=[   42], | 30.00th=[   43], 40.00th=[   44], 50.00th=[   45], 60.00th=[   46], | 70.00th=[   47], 80.00th=[   48], 90.00th=[   50], 95.00th=[   52], | 99.00th=[   57], 99.50th=[   59], 99.90th=[   69], 99.95th=[   87], | 99.99th=[  135] bw (   KiB/s): min=   71, max=25352, per=16.36%, avg=13341.18, stdev=9726.48, samples=2397 iops        : min=   17, max= 6338, avg=3335.13, stdev=2431.78, samples=2397 lat (usec)   : 2=0.01%, 10=0.01%, 20=0.01%, 50=45.37%, 100=33.22% lat (usec)   : 250=19.16%, 500=0.43%, 750=0.43%, 1000=0.46% lat (msec)   : 2=0.91%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01% cpu          : usr=3.53%, sys=11.43%, ctx=12237590, majf=0, minf=11 IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, &amp;amp;gt;=64=0.0% submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, &amp;amp;gt;=64=0.0% complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, &amp;amp;gt;=64=0.0% issued rwts: total=6121601,6116013,0,0 short=0,0,0,0 dropped=0,0,0,0 latency   : target=0, window=0, percentile=100.00%, depth=1 Run status group 0 (all jobs): READ: bw=79.7MiB/s (83.6MB/s), 79.7MiB/s-79.7MiB/s (83.6MB/s-83.6MB/s), io=23.4GiB (25.1GB),  run=300001-300001msec WRITE: bw=79.6MiB/s (83.5MB/s), 79.6MiB/s-79.6MiB/s (83.5MB/s-83.5MB/s), io=23.3GiB (25.1GB),  run=300001-300001msec Disk stats (read/write): dm-2: ios=6120646/6115090, merge=0/0, ticks=806337/255779, in_queue=1062184, util=100.00%, aggrios=6121601/6116035, aggrmerge=0/1, aggrticks=801719/251319, aggrin_queue=1051124, aggrutil=100.00% sdb: ios=6121601/6116035, merge=0/1, ticks=801719/251319, in_queue=1051124, util=100.00% www.zhangfangzhou.cn

4. Results
Direct Path
read: IOPS=26.0k, BW=102MiB/s (107MB/s)(29.8GiB/300001msec)
write: IOPS=25.0k, BW=102MiB/s (106MB/s)(29.7GiB/300001msec)

Virtualization
read: IOPS=20.4k, BW=79.7MiB/s (83.6MB/s)(23.4GiB/300001msec)
write: IOPS=20.4k, BW=79.6MiB/s (83.5MB/s)(23.3GiB/300001msec)

 

Read Write Bandwidth
Direct Path 26.0k 25.0k 102MiB/s (107MB/s)
Virtualization 20.4k 20.4k 79.7MiB/s (83.6MB/s)

After storage virtualization, the read performance loss is about 21.5%, and the write performance loss is about 18.4%

CentOS Linux release 7.9.2009 Compile and install the latest version of HAProxy 2.8 LTS long-term support version, and create shell listening script

Compile and install the HAProxy 2.8 LTS version. The official source package download address is: http://www.haproxy.org/download/

Since the lua version that comes with CentOS7 before is relatively low and does not meet the requirements of the minimum version of lua required by HAProxy (5.3), you need to compile and install the lua environment of the newer version before you can compile and install HAProxy.

1. View the existing lua version. If the version is too old, you need to update the lua first

 lua -v  Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio

2. Installing basic commands and compiling dependent environments

 yum install gcc readline-devel

3. Download and compile lua-5.4.6

 cd /usr/local/src wget --no-check-certificate  https://www.lua.org/ftp/lua-5.4.6.tar.gz tar xvf lua-5.4.6.tar.gz

Start Compiling

 cd  lua-5.4.6 make linux test View the build installed version src/lua -v Lua 5.4.6  Copyright (C) 1994-2023 Lua.org, PUC-Rio Remove the old version and replace it with a new one mv /usr/bin/lua /usr/bin/lua.old cp /usr/local/src/lua-5.4.6/src/lua /usr/bin/lua Confirm the current version lua -v  Lua 5.4.6  Copyright (C) 1994-2023 Lua.org, PUC-Rio

4. Install Haproxy

 Create User useradd -r -s /sbin/nologin haproxy

5. Installation dependency

 yum -y install gcc openssl-devel pcre-devel systemd-devel

6. Download and compile

 wget --no-check-certificate   https://www.haproxy.org/download/2.8/src/haproxy-2.8.9.tar.gz tar xvf haproxy-2.8.9.tar.gz cd haproxy-2.8.9/ make TARGET=linux-glibc USE_OPENSSL=1 USE_PCRE=1 USE_SYSTEMD=1 USE_LUA=1 LUA_INC=/usr/local/src/lua-5.4.6/src LUA_LIB=/usr/local/src/lua-5.4.6/src make install PREFIX=/etc/haproxy Create a soft connection ln -s /etc/haproxy/sbin/haproxy /usr/bin/haproxy www.zhangfangzhou.cn View version haproxy -v HAProxy version 2.8.9-1842fd0 2024/04/05 -  https://haproxy.org/ Status: long-term supported branch - will stop receiving fixes around Q2 2028. Known bugs:  http://www.haproxy.org/bugs/bugs-2.8.9.html Running on: Linux 3.10.0-1160.105.1.el7.x86_64 #1 SMP Thu Dec 7 15:39:45 UTC 2023 x86_64

7. Create Profile

 vi /etc/haproxy/haproxy.cfg global maxconn 100000 chroot /etc/haproxy stats socket /var/tmp/haproxy.sock mode 600 level admin user haproxy group haproxy daemon pidfile /var/tmp/haproxy.pid defaults option http-keep-alive option forwardfor maxconn 100000 mode http timeout connect 300000ms timeout client 300000ms timeout server 300000ms listen stats mode http bind 0.0.0.0:3306 stats enable log global stats uri /haproxy-status stats auth haadmin: Admin@2018.. listen db_port bind :3306 mode tcp log global balance roundrobin option tcplog server db1 10.53.123.104:3306 check inter 3000 fall 2 rise 3 server db2 10.53.123.105:3306 check inter 3000 fall 2 rise 3 server db3 10.53.123.106:3306 check inter 3000 fall 2 rise 3


8. Create service service

 vi /usr/lib/systemd/system/haproxy.service [Unit] Description=HAProxy Load Balancer After=syslog.target network.target [Service] ExecStartPre=/usr/bin/haproxy -f /etc/haproxy/haproxy.cfg -c -q ExecStart=/usr/bin/haproxy -Ws -f /etc/haproxy/haproxy.cfg ExecReload=/bin/kill -USR2 $MAINPID LimitNOFILE=100000 [Install] WantedBy=multi-user.target

systemctl enable haproxy
systemctl start haproxy
9. Set listening script

 vi ha_watchdog.sh #!/ bin/bash #www.zhangfangzhou.cn #Define log file path LOG_FILE="/var/log/haproxy_watchdog.log" #Define the name of the HAProxy process HAPROXY_PROCESS="haproxy" #Check whether the HAProxy process is running check_haproxy_process() { #Use the pgrep command to check whether a process named haproxy exists pgrep -x $HAPROXY_PROCESS > /dev/null } #Start the HAProxy process start_haproxy() { echo "$(date +'%Y-%m-%d %H:%M:%S') - Starting HAProxy... " >> $LOG_FILE systemctl start haproxy } #Main cycle while true; do #Check whether the HAProxy process is running if ! check_haproxy_process; then #If the process does not exist, restart HAProxy and log start_haproxy fi #Check again after sleeping for a period of time sleep 60 done #Set startup chmod +x ha_watchdog.sh vi /etc/rc.local nohup /root/ha_watchdog.sh &

Windows10/11 Enterprise Multi session Edition – RDP for unlimited users

Windows10/11 Enterprise Multi session Edition - RDP for Unlimited Users
Windows 10/11 Enterprise Edition Multi session, formerly known as Windows 10 Enterprise Edition (EVD) for virtual desktops, is a new type of remote desktop session host that allows multiple concurrent interactive sessions. Previously, only Windows Server supported this feature. This function provides users with a familiar Windows 10/11 experience. At the same time, IT can benefit from the cost advantage of multiple sessions and use the existing per user Windows license instead of RDS client access license (CAL).

The biggest difference in this version is that it allows unlimited number of remote desktop users to connect and log in at the same time to realize the virtual desktop function (VDI).

1. System requirements
Install the latest version of Windows 10 or 11, Enterprise or Professional, and then convert it by key.

transformation

 #Run CMD window with administrator privileges slmgr /ipk VMKVQ-3MN6B-BVM9F-YWV97-R9FCX #Install KMS key slmgr /skms kms-shanghai01.cangshui.net #Specify KMS activation server slmgr /ato #Activate #Then restart the system to complete the conversion.

2. Open RDP
Open Remote Desktop

3. After conversion

4. Recommended for use as a virtualized desktop (through the Domain Environment Group Policy)
1. Configure automatic logoff (for example, if the user is idle for 30 minutes, the user will be automatically logged off)
2. Limit the number of connections (prevent virtual machine resource exhaustion)
3. Configure automatic reconnection.
4. Set rules for Remote Desktop Services user session remote control (full control without user authorization)
5. Restrict Remote Desktop Services users to separate Remote Desktop Services sessions (connect users in Task Manager)
PS: Personal network disk. Find a file server and create a share. The share name is userdisk $. Placing $is used to hide the share name. The sharing permission is everyone read/write.

(Then through group policy, configure login scripts, create folders, and configure permissions. Map network drives)
reference resources
https://songxwn.com/windows10-11-Enterprise-Virtual-Desktops/

Windows Server 2022 RDS configuration authorization activation service (multi-user login activation at the same time)

Windows Server 2022 RDS configuration authorization activation service (multi-user login activation at the same time)
The RDP multi user simultaneous login authorization has only a 120 day trial period by default.
1. Add Role

2. Start Activation
In Windows Administrative Tools>Remote Desktop Services>

3. Make sure that the server can connect to the Internet and configure DNS correctly. The default connection is automatic. Click Next

4. Fill in the last name, company, and select the region. You can fill in any. Click Next

5. It is unnecessary to fill in information such as email. Just click Next

6. Select License Program as Enterprise Agreement

The agreement number is 6565792. Click Next.
(The agreement number can be 6565792495443868793215296992)

7. The product version is Windows Server 2022, the license type is RDS Per User CAL, and the quantity is 9999

8. The activation is now complete.

Veeam Backup Replication of Backup Software

Veeam Backup Replication of Backup Software
1、 Why backup
1. Data is a valuable asset for both individual users and enterprises. These software provide a reliable backup mechanism to protect data from hardware failures, accidental deletion, malware, and other catastrophic events. By backing up data regularly, users can ensure the integrity and availability of their data.
2. In case of data loss, system crash or other catastrophic events, it can help users quickly recover data and systems. This is critical to ensure business continuity, especially for enterprises that rely on data and systems. The backup software provides flexible recovery options, which can restore the entire system, single files or folders on demand to meet different recovery requirements.

2、 What are the common backup software
1. Ankenos
2、Veeam
3. Yunqi
4、 NetBackup & Veritas Backup Exec
5、Zerto

3、 Veeam Backup Replication Download Installation
1. Veeam Backup Replication official website ISO file Click to download Veeam Backup Replication 12.1.0.2131_20231206
Installation requirements (recommended)
Recommended operating system Windows Server 2016-2022
The CPU requires at least four cores, and the recommended memory is more than 4GB.
The storage space should be more than 100GB, and the backup space should be calculated according to the demand.
The database can be used with PostgreSQL 15.1 or SQL Server 2012-2022.
Currently supports VMware vSphere 6.0-8.0U2

4、 Veeam Backup Replication Installation

5、 Activate Veeam Backup Replication (VBR) software as Professional
Replace file download
1、 Replacement file download 1
2、 Replacement file download 2

 VeeamLicense.dll #Replace File Veeam_ASv11_1500.lic Veeam_BackupRep_v12_5000_1.lic #Authorization file

6、 Stop service and replace files
After installing the software, run PowerShell with an administrator to execute the following command:

 get-service -displayname Veeam* | stop-service get-service -displayname Veeam*

7、 Start Service

 get-service -displayname Veeam* | start-service

8、 Import Authorization

 Open Veeam Backup&Replication Console software, click the menu in the upper left corner, and select the License option, Click install and select Veeam_BackupRep_v12_5000_1.lic file to import and install.

Reference source
https://www.digiboy.ir/12153/veeam-backup-replication-12-1-0-2131/

Local deployment of AI drawing and drawing tool stable differentiation webui (including multiple models and domestic acceleration)

Local deployment of AI drawing and drawing tool stable differentiation webui (including multiple models and domestic acceleration)
The Stable Diffusion WebUI is a browser interface based on the Gradio library, which is used to visually display and interactively explore the results of the Stable Diffusion model.
GPU video memory size: GPU video memory size limits the amount of data that can be processed by the model. Larger video memory can accommodate more image information and model parameters, so it can support the generation of larger images. When generating large size pictures, more video memory is needed to store intermediate results and generated images. Smaller video memory may limit the ability to generate high-resolution or complex scenes.
GPU main frequency: GPU main frequency refers to the working frequency of GPU, that is, the number of instructions processed per second. Higher dominant frequency can provide faster calculation speed, thus speeding up the process of AI generating pictures. When it is necessary to generate pictures quickly, the GPU with high dominant frequency can effectively shorten the generation time and improve efficiency.
Video memory bit width: The video memory bit width indicates the width of each transmission of data in video memory, usually in bits. The larger bit width means that the GPU can read and write video memory data faster. In the process of image generation, large bit width can improve the speed of data transmission, thus accelerating the speed of model training and reasoning.

Deployment environment
CPU:Intel(R) Xeon(R) Gold 6246 CPU @ 3.30GHz 3.30 GHz
GPU:NVIDIA GRID V100S-32Q(vGPU)
Memory: 64GB
Disk: 1TB
OS: Microsoft Windows 10 Enterprise Edition

Deployment Steps
1. # Installation Base Environment
The basic environment consists of Python 3.10.6 (install Python 3.10.6 and pip, and must install Python version 3.10.6), Git, and CUDA

1.1. # Installing Python

Visit Python 3.10.6 download page https://www.python.org/downloads/release/python-3106/

Find the [Windows installer (64 bit)] and click Download. When installing, check Add Python to PATH.

After installation
Enter Python -- version in the command line. If Python 3.10.6 is returned, the installation is successful.
Enter python - m pip install -- upgrade pip in the command line to install and upgrade pip to the latest version.

1.2. # Installing Git https://git-scm.com/download
Visit the Git download page
Click Download for Windows and 64 bit Git for Windows Setup to download
Command line running git -- version

1.3 # Install CUDA (NVIDIA graphics card user)
Run nvidia smi from the command line to see the CUDA version supported by your video card

 C:\Users\Administrator>cd C:\Program Files\NVIDIA Corporation\NVSMI C:\Program Files\NVIDIA Corporation\NVSMI>nvidia-smi.exe Thu Aug 17 15:06:21 2023 +-----------------------------------------------------------------------------+ | NVIDIA-SMI 474.14        Driver Version: 474.14       CUDA Version: 11.4     | |-------------------------------+----------------------+----------------------+ | GPU   Name            TCC/WDDM | Bus-Id        Disp.A | Volatile Uncorr. ECC | | Fan   Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. | |                               |                      |               MIG M. | |===============================+======================+======================| |   0   GRID V100S-32Q     WDDM  | 00000000:02:01.0  On |                    0 | | N/A    0C     P0    N/A /  N/A |   6305MiB / 32768MiB |      0%      Default | |                               |                      |                  N/A | +-------------------------------+----------------------+----------------------+ +-----------------------------------------------------------------------------+ | Processes:                                                                  | |   GPU   GI   CI        PID   Type   Process name                  GPU Memory | |         ID   ID                                                   Usage      | |=============================================================================| |    0    N/A  N/A      1520    C+G   Insufficient Permissions        N/A      | |    0    N/A  N/A      1540    C+G   Insufficient Permissions        N/A      | |    0    N/A  N/A      1548    C+G   Insufficient Permissions        N/A      | |    0    N/A  N/A      2468      C   ...thon\Python310\python.exe    N/A      |


Next, go to Nvidia's official website and download the corresponding version https://developer.nvidia.com/cuda-toolkit-archive
Select your own operating system version. Pay attention to the offline installation package [exe [local]]. Online installation is still slow.

1.4 # Why do you need to install CUDA driver after NVIDIA graphics card is installed with driver
NVIDIA graphics card driver and CUDA are two different software components, which play different roles in the functions and applications of GPU.

NVIDIA graphics card driver: NVIDIA graphics card driver is the bridge between the operating system and the graphics card. It is responsible for managing and controlling the hardware functions of the graphics card to ensure that the graphics card can work properly and communicate with the operating system.
Installing a video card driver is a basic requirement for using NVIDIA video cards. It provides support and compatibility for video cards.

CUDA (Compute Unified Device Architecture): CUDA is a parallel computing platform and programming model developed by NVIDIA, which allows developers to use the parallel computing capabilities of graphics cards for high-performance computing and acceleration.
CUDA provides a set of APIs and tools to enable developers to write parallel computing programs on graphics cards, and use GPU's large-scale parallel computing capabilities to accelerate various computing tasks, including machine learning, deep learning, scientific computing, etc.

2. # Download project source code

git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git

 D:\>git clone  https://github.com/AUTOMATIC1111/stable-diffusion-webui.git Cloning into 'stable-diffusion-webui'... remote: Enumerating objects: 25982, done. remote: Counting objects: 100% (369/369), done. remote: Compressing objects: 100% (160/160), done. remote: Total 25982 (delta 237), reused 323 (delta 205), pack-reused 25613 Receiving objects: 100% (25982/25982), 31.77 MiB | 12.31 MiB/s, done. Resolving deltas: 100% (18197/18197), done. #Switch to the project root directory cd stable-diffusion-webui

3. # Downloading Model Files
The stable differentiation webui is just a tool, which needs a back-end training model for AI reference modeling.

Current mainstream models include
Stable Diffusion (SD) model : Stable Diffusion is an image generation model based on diffusion model, which is mainly used to generate images biased towards real people. It uses the method of stable diffusion process to generate images step by step through multiple iterations, and has good generation stability and detail retention ability. SD model is usually used to generate realistic face images or character images.
Waifu Difusion (WD) model : Waifu Difusion is a model that focuses on generating images of quadratic characters (often called "waifu"). Based on the idea of diffusion model, it gradually generates the artistic style of the quadratic character image through the iterative generation process. WD model is usually used to create two-dimensional character images, illustrations or animation style images.
New AI Leaks (Naifu) model : Novel AI Leaks is an image generation model that is more inclined to quadratic characters. When generating images, it pays more attention to creating imaginative and creative binary character images. Naifu model is usually used in the fields of art creation, illustration production and quadratic role design.

4. # Running
Double click to run table diffusion webui webui-user.bat
The script will check the dependency itself, download about several gigabytes of things, decompress and install them into a folder (it may fail in the middle, and you need to run webui user. bat again after failure).


venv "D:\stable-diffusion-webui\venv\Scripts\Python.exe"
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug 1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Version: v1.5.1
Commit hash: 68f336bd994bed5442ad95bad6b6ad5564a5409a
Installing gfpgan

When the card is installing gfpgan, you need to switch to the source of Ali

C:\Users\Administrator>pip config set global.index-url https://mirrors.aliyun.com/pypi/simple
Writing to C:\Users\Administrator\AppData\Roaming\pip\pip.ini

5. # Copy to browser for access
(The default is http://127.0.0.1:7860 )(Be careful not to close this window. Close it and exit)

6. # Generate the first AI drawing

1. Prompt: Fill in the feature points you want in Prompt, which can be used as a guide for generating pictures. By providing specific descriptions or instructions, you can guide AI to generate images that meet your expectations.

2. Negative prompt: fill in unwanted feature points in Negative prompt, which can tell AI to avoid generating these features. Negative hints can help you control the content of generated images more accurately.

3. Sampling Steps: specify the steps of AI deduction to control the details and precision of the generated picture. A higher number of steps can produce more detail, but will increase the generation time. It is generally appropriate to choose between 20 and 30.

4. Sampling method: Select the algorithm deduced by AI. Common algorithms include Euler a, Euler, and DDIM. Different sampling methods may affect the effect and style of the generated picture.

5. Image resolution: specify the resolution of the generated image. Higher resolution can obtain clearer and more detailed images. However, attention should be paid to the limitations of video memory to ensure that the resolution does not exceed the range that video memory can handle.

Expansion options:
Restore faces: Check to generate more realistic facial features. Additional runtime libraries need to be downloaded, and they will occupy a certain amount of storage space when used for the first time.

 D:\stable-diffusion-webui\models\Codeformer\codeformer-v0.1.0.pth D:\stable-diffusion-webui\repositories\CodeFormer\weights\facelib\detection_Resnet50_Final.pth D:\stable-diffusion-webui\repositories\CodeFormer\weights\facelib\ parsing_parsenet.pth

Tiling: enables the generated image to be laid seamlessly, similar to the effect of tiles, so that the patterns can be automatically connected.
Highres. fix: Fill the content with higher resolution, but the final image is still the set resolution.
Batch count: specifies the number of times to generate pictures in a run.
Batch size: Specifies the number of pictures generated at the same time.
CFG Scale: controls the extent to which AI refers to your Prompt and Negative Prompt when generating pictures. Higher values will be generated more strictly according to your settings, while lower values will be more free and creative.
Seed (random number seed): used to initialize the initial noise of AI generated pictures. Different seeds will produce different results.
Generate: click this button to start the process of AI generating pictures.
Stable Diffusion checkpoint: Select the model to use, and choose it according to your needs and experience.

Script X/Y/Z plot
1. Use multiple painting methods
Select Sampler for Y type, and then select Sampling methods for all Y values. After testing, these methods work well

Euler a
DPM++2S a
DPM++2M
DPM2 a Karras
DPM2++ 2S a Karras
DPM2++ 2M a Karras

2. Test the influence of lora weight on painting
Fill in txt2img , select Prompt S/R in Y type, and fill in STRONGTH, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1 in Y values

7. Model download
MajicMIX realistic https://www.liblibai.com/modelinfo/bced6d7ec1460ac7b923fc5bc95c4540
Sampler recommendation: Euler
If you want to repair your face, please use after detailer

Pure Beauty https://www.liblibai.com/modelinfo/13bcec1633cd440c9ae986787b733b10
CFG Scale:7
Iteration steps: 15-120
VAE: 840000
Recommendation for sampler: DPM++2M SDE Karras
Please enable ADetailer plug-in when drawing character images. Recommended models: face_yolov8n.pt and face_yolov8s.pt
When generating professional photographic images, it is recommended to add the following words to the forward prompt words in addition to the trigger words such as Bean_Lab, Pure Beauty, 1 girl, best quality: Studio lighting, Canon EOS-1D X Mark II, 70mm ,f2.8,ISO 200, (In addition to the studio light, the azimuth light can also be used according to needs, such as along light, front side light, side light, top light, side backlight, etc. There are many choices for camera models. In addition to the above mentioned, Canon EOS R5 Canon 1Dx3, SONY A7R III, Nikon D6, Fujifilm GFX 100S, Fujifilm 50S II and other HD camera models can be selected. In addition, it is recommended to use large aperture F1.4, F2.0, F2.8, portrait focal length 24-70mm, and sensitivity 100-200). Of course, the above is only for reference, and these data can be debugged according to the final desired effect
Negative Promts/Negative prompt words: ng_deepnegative_v1_75t, EasyNegative

8. Prompt word download
a、 https://github.com/thisjam/sd-webui-oldsix-prompt
Click the code ->DownloadZip on the page, unzip it, put it in the extensions folder under your sd folder, apply it, and restart it

11. Display VAE settings on the page
The default page of stable differentiation webui does not display the VAE setting part, so you need to set it first. First click Settings, and then click the tab User interface in the left menu,
Pull to the bottom and there is an option called Quicksettings list. Add it in the input box, sd_vae,CLIP_stop_at_last_layers, Finally, click "Apply settings" above and click "Reload UI" to refresh the page.
The Clip skip option, which defaults to 1, needs to be changed to the commonly used 2. It can be simply understood that the higher the value, the more classification details will be omitted from AI.

Download VAE model (unsatisfactory effect)
https://huggingface.co/stabilityai/sd-vae-ft-mse-original/tree/main

12. ADetailer plug-in
1、 https://github.com/Bing-su/adetailer
2. Unzip and upload to folder D: table diffusion webui extensions
3. Restart Web UI
4. Fixed Seed
5. Enable ADetailer
Enable ADetailer # Enable

1st # The first ADetailer model
mediapipe_face_mesh

2nd # The second ADetailer model

Model description
Model Target
face_yolov8n.pt 2D / realistic face
face_yolov8s.pt 2D / realistic face
hand_yolov8n.pt 2D / realistic hand
person_yolov8n-seg.pt 2D / realistic person
person_yolov8s-seg.pt 2D / realistic person
mediapipe_face_full realistic face
mediapipe_face_short realistic face
mediapipe_face_mesh realistic face

Self built free NVIDIA vGPU15 License Server license server

Self built free NVIDIA vGPU15 License Server license server. The virtual machine running vGPU will regularly send license requests to the License server.
1. CentOS 7 Installing Docker Service
2. Create Configuration and Build Container
3. Linux client activation
4. Windows Client Activation

1. CentOS 7 Installing Docker Service
#Installation dependency

 sudo yum install -y yum-utils device-mapper-persistent-data lvm2

#Add Docker Source

 sudo yum-config-manager --add-repo  https://download.docker.com/linux/centos/docker-ce.repo

#Install Docker

 sudo yum install -y docker-ce docker-ce-cli containerd.io

#It is recommended to modify the configuration of Docker image accelerator in China

 cat > /etc/docker/daemon.json << EOF { "dns": ["114.114.114.114", "180.76.76.76"], "registry-mirrors": [" https://docker.mirrors.ustc.edu.cn "], "bip": "192.168.199.1/24", "debug": true, "log-level": "info", "live-restore": true } EOF

#Start Docker

 sudo systemctl start docker

#Set startup

 sudo systemctl enable docker

#Pull the latest image

 docker pull collinwebdesigns/fastapi-dls:latest

#Save the image for a rainy day

 docker save -o collinwebdesigns_fastapi-dls.tar collinwebdesigns/fastapi-dls:latest

2. Create Configuration and Build Container
#Create Directory

 mkdir -p /opt/docker/fastapi-dls/cert cd /opt/docker/fastapi-dls/cert

#Generate Public and Private Keys

 openssl genrsa -out /opt/docker/fastapi-dls/cert/instance.private.pem 4096 openssl rsa -in /opt/docker/fastapi-dls/cert/instance.private.pem -outform PEM -pubout -out /opt/docker/fastapi-dls/cert/instance.public.pem

#Generate SSL certificate for web server

 openssl req -x509 -nodes -days 3650 -newkey rsa:4096 -keyout /opt/docker/fastapi-dls/cert/webserver.key -out /opt/docker/fastapi-dls/cert/webserver.crt

Enter subordinate information

 Country Name (2 letter code) [XX]:CN State or Province Name (full name) []:HN Locality Name (eg, city) [Default City]:ZZ Organization Name (eg, company) [Default Company Ltd]:zhangfangzhou Organizational Unit Name (eg, section) []:zhangfangzhou Common Name (eg, your name or your server's hostname) []:zhangfangzhou Email Address []: admin@zhangfangzhou.cn

Original article https://www.zhangfangzhou.cn/free-nvidia-vgpu15-license-server.html

#Configure the creation container 'hostname - i' and replace it with your own IP address

 docker volume create zhangfangzhou-dls-db
 docker run -itd --restart=always -e LEASE_EXPIRE_DAYS=1825 -e DLS_URL=`hostname -i` -e DLS_PORT=8443 -p 8443:443 -v /opt/docker/fastapi-dls/cert:/app/cert -v zhangfangzhou-dls-db:/app/database collinwebdesigns/fastapi-dls:latest

#For example

 docker run -itd --restart=always -e LEASE_EXPIRE_DAYS=1825 -e DLS_URL=10.0.0.7 -e DLS_PORT=8443 -p 8443:443 -v /opt/docker/fastapi-dls/cert:/app/cert -v zhangfangzhou-dls-db:/app/database collinwebdesigns/fastapi-dls:latest

3. Linux client activation

 wget --no-check-certificate -O /etc/nvidia/ClientConfigToken/client_configuration_token_$(date '+%d-%m-%Y-%H-%M-%S').tok  https://10.0.0.7:8443/ -/client-token

#Then restart the nvidia gridd service:

 service nvidia-gridd restart

#You can also use the command line to view authorization details:

 nvidia-smi -q |findstr Lincese

4. Windows Client Activation
Run Powershell with administrator privileges,

 C:\Program Files\NVIDIA Corporation\vGPU Licensing\ClientConfigToken curl.exe --insecure -L -X GET  https://10.0.0.7:8443/ -/client-token -o "C:\Program Files\NVIDIA Corporation\vGPU Licensing\ClientConfigToken\client_configuration_token_$($(Get-Date).tostring('dd-MM-yy-hh-mm-ss')).tok" Restart-Service NVDisplay.ContainerLocalSystem

Container interface
The following table only lists the main interfaces. For more detailed interfaces, refer to the PATH interface documents of/-/docs and/-/redoc.

Explanation of request method PATH
GTE/Redirect to/-/readme
GTE/-/health is used for health checks and also displays the current version and commit hash.
GTE/-/readme HTML page display README.md
GTE/-/docs GET/-/openapi.json shows the interface document of the Swagger UI page
GTE/-/redoc GET/-/openapi.json shows the interface document of the ReDoc page
GTE/-/manage displays very basic UI for deleting source or lease
GTE/-/origins? Leases=false List the registered origin. The origin parameter refers to the reference source including each lease
DELETE/-/origins Delete all sources and their leases
GTE/-/leases? Origin=false Lists the current lease. The origin parameter refers to the reference source including each lease
DELETE/-/lease/{lease_ref} Delete lease
GTE/client token generates client token

Regular Expression Manual

What exactly is a regular expression?

When writing programs or web pages that deal with strings, it is often necessary to find strings that meet some complex rules. Regular expressions are tools for describing these rules. In other words, regular expressions are code that records text rules.

Common metacharacters

code explain
. Matches any character except line feed
w Match letters or numbers or underscores
s Match any whitespace
d Matching number
b Match the beginning or end of a word
^ Start of matching string
$ End of matching string

Common qualifiers

Code/Syntax explain
* Repeat zero or more times
+ Repeat once or more
? Repeat zero or one time
{n} Repeat n times
{n,} Repeat n or more times
{n,m} Repeat n to m times

Common antonyms

Code/Syntax explain
W Match any character that is not a letter, number, underline or Chinese character
S Match any character that is not a whitespace
D Match any non numeric character
B Match a position that is not the beginning or end of a word
Match any character except x
Matches any character except aeiou

Complete set of expressions

character describe
\ Mark the next character as a special character, or a literal character, or a backward reference, or an octal escape character. For example“ n ”Matching character“ n ”。 \n ”Matches a line break. Serial“ \\ ”Match“ \ ”And“ \( ”Then match“ ( ”。
^ Matches the start of the input string. If the Multiline property of RegExp object is set, ^ also matches“ \n ”Or“ \r ”Position after.
$ Matches the end position of the input string. If the Multiline property of the RegExp object is set, $also matches“ \n ”Or“ \r ”Previous position.
* Matches the previous subexpression zero or more times. For example, zo * can match“ z ”And“ zoo ”。* Equivalent to {0,}.
+ Matches the previous subexpression one or more times. For example“ zo+ ”Can match“ zo ”And“ zoo ”, but cannot match“ z ”。+ Equivalent to {1,}.
? Matches the previous subexpression zero or once. For example“ do(es)? ”Can match“ does ”Or“ does ”In“ do ”。? Equivalent to {0,1}.
{ n } n Is a non negative integer. Match Determined n Times. For example“ o{2} ”Cannot match“ Bob ”In“ o ”, but can match“ food ”Two o's in.
{ n ,} n Is a non negative integer. At least match n Times. For example“ o{2,} ”Cannot match“ Bob ”In“ o ”, but can match“ foooood ”All o in. o{1,} ”Equivalent to“ o+ ”。 o{0,} ”Is equivalent to“ o* ”。
{ n , m } m and n Are all non negative integers, where n <= m Least match n Times and at most matches m Times. For example“ o{1,3} ”Will match“ fooooood ”The first three o's in. o{0,1} ”Equivalent to“ o? ”。 Note that there must be no spaces between the comma and the two numbers.
? When this character is immediately followed by any other restrictor (*,+{ n },{ n ,},{ n , m }), the matching pattern is non greedy. The non greedy pattern matches as few strings as possible, while the default greedy pattern matches as many strings as possible. For example, for a string“ oooo ”,“ o+? ”Will match a single“ o ”, and“ o+ ”Will match all“ o ”。
. Matching division“ \ n ”Any single character other than. To match include“ \ n ”Please use any characters like“ (.|\n) ”Mode.
(pattern) Match pattern and get this match. The obtained matches can be obtained from the generated Matches collection. The SubMatches collection is used in VBScript, and the $0... $9 attribute is used in JScript. To match parenthesis characters, use“ \( ”Or“ \) ”。
(?:pattern) The pattern is matched but the matching result is not obtained. That is to say, this is a non acquisition match and will not be stored for future use. This is using the or character“ (|) ”It is useful to combine the parts of a pattern. For example“ industr(?: y|ies) ”Is a comparison“ industry|industries ”A simpler expression.
(?=pattern) Positive pre search, matching the search string at the beginning of any string matching pattern. This is a non acquisition match, that is, the match does not need to be acquired for future use. For example“ Windows(?= 95|98|NT|2000) ”Can match“ Windows2000 ”In“ Windows ”, but cannot match“ Windows3.1 ”In“ Windows ”。 The pre query does not consume characters, that is, after a match occurs, the next matching search starts immediately after the last match, rather than after the character containing the pre query.
(?!pattern) The positive negative pre query matches the search string at the beginning of any string that does not match the pattern. This is a non acquisition match, that is, the match does not need to be acquired for future use. For example“ Windows(?! 95|98|NT|2000) ”Can match“ Windows3.1 ”In“ Windows ”, but cannot match“ Windows2000 ”In“ Windows ”。 The pre query does not consume characters, that is, after a match occurs, the next matching search starts immediately after the last match, rather than after the characters containing the pre query
(?<=pattern) The reverse positive pre check is similar to the positive positive pre check, but in the opposite direction. For example“ (?<=95|98|NT|2000)Windows ”Can match“ 2000Windows ”In“ Windows ”, but cannot match“ 3.1Windows ”In“ Windows ”。
(?<!pattern) The reverse negative pre-test is similar to the positive negative pre-test, but in the opposite direction. For example“ (?<!95|98|NT|2000)Windows ”Can match“ 3.1Windows ”In“ Windows ”, but cannot match“ 2000Windows ”In“ Windows ”。
x|y Matches x or y. For example“ z|food ”Can match“ z ”Or“ food ”。 (z|f)ood ”Then match“ zood ”Or“ food ”。
[xyz] Character set. Match any contained character. For example“ [abc] ”Can match“ plain ”In“ a ”。
[^xyz] A collection of negative characters. Matches any character that is not contained. For example“ [^abc] ”Can match“ plain ”In“ p ”。
[a-z] Character range. Matches any character in the specified range. For example“ [a-z] ”Can match“ a ”To“ z ”Any lowercase alphabetic character in the range.
[^a-z] Negative value character range. Matches any character that is not within the specified range. For example“ [^a-z] ”Can match any not in“ a ”To“ z ”Any character in the range.
\b Match a word boundary, that is, the position between the word and the space. For example“ er\b ”Can match“ never ”In“ er ”, but cannot match“ verb ”In“ er ”。
\B Matches non word boundaries. er\B ”Can match“ verb ”In“ er ”, but cannot match“ never ”In“ er ”。
\cx Matches the control character indicated by x. For example, cM matches a Control-M or carriage return character. The value of x must be one of A-Z or a-z. Otherwise, regard c as a literal“ c ”Character.
\d Matches a numeric character. Equivalent to [0-9].
\D Matches a non numeric character. Equivalent to [^ 0-9].
\f Matches a new page character. Equivalent to x0c and cL.
\n Matches a line break. Equivalent to x0a and cJ.
\r Matches a carriage return. Equivalent to x0d and cM.
\s Matches any white space characters, including spaces, tabs, page feeds, and so on. Equivalent to [ f n r t v].
\S Matches any non whitespace characters. Equivalent to [^ f n r t v].
\t Matches a tab. Equivalent to x09 and cI.
\v Matches a vertical tab. Equivalent to x0b and cK.
\w Matches any word character including underscores. Equivalent to“ [A-Za-z0-9_] ”。
\W Matches any non word character. Equivalent to“ [^A-Za-z0-9_] ”。
\x n matching n , where n Is a hexadecimal escape value. The hexadecimal escape value must be two digits long. For example“ \x41 ”Match“ A ”。 \x041 ”Is equivalent to“ \x04&1 ”。 ASCII encoding can be used in regular expressions
\ num matching num , where num Is a positive integer. Reference to the matched obtained. For example“ (.)\1 ”Matches two consecutive identical characters.
\ n Identifies an octal escape value or a backward reference. If\ n At least before n Subexpressions, then n Is a backward reference. Otherwise, if n Is an octal digit (0-7), then n Escape value for an octal.
\ nm Identifies an octal escape value or a backward reference. If\ nm At least before nm Subexpressions, then nm Is a backward reference. If\ nm At least before n Get, then n As a follow text m The backward reference of. If the previous conditions are not met, if n and m Are octal digits (0-7), then\ nm Will match octal escape value nm
\ nml If n Is an octal digit (0-3), and M and l If both are octal digits (0-7), match octal escape value nm l。
\u n matching n , where n Is a Unicode character represented by four hexadecimal digits. For example, u00A9 matches the copyright symbol( ©)。