preface
In the past UI automation test, we can complete the UI automation test by acquiring page elements, encapsulating and combining them into a series of simulated human operations. However, in the map business test, this method cannot be completed. Maps cannot obtain elements through common element positioning methods, For example, it is a good idea to complete the comparison of the accuracy of the path planning between the new and old versions, and the maturity of the route compared with the competing products, but it is also a good idea to use image recognition. Today, we will introduce how to use image recognition to do some applications in map testing. Let's introduce today's protagonist - OpenCV
OpenCV（Open Source Computer Vision Library is an open source cross platform computer vision library developed using C/C++, which provides many functions that efficiently implement computer vision algorithms, and also provides Python, J AVA, MATLAB and other language interfaces. OpenCV has a wide range of applications, including image stitching, image noise reduction, product quality inspection, human-computer interaction, face recognition, action recognition, action tracking, unmanned driving, etc. OpenCV also provides a machine learning module. You can use normal Bayes, K-nearest neighbor, support vector machine, decision tree, random forest, artificial neural network and other machine learning algorithms.
1、 Installation and engineering construction will not be introduced, but the following modules and tools shall be installed
GTK+2. x or higher;
GCC compiler;
Cmake construction tool;
Libtbb (Intel Threading Building Module)
Python3
2、 Image writing and reading
Image processing depends on getting an image and video, and getting the expected results by applying the "play" of signal processing technology. We write two route planning pictures.
The main image formats supported by OpenCV are:
Windowsbitmaps（*.bmp、*dib）；
Portableimage formats（*.pbm、*.pgm、*.ppm）；
Sunrasters（*.sr、*.ras）；
The formats that require auxiliary libraries are:
JPEG（*.jpeg、*.jpg、*.jpe）；
JPEG 2000（*.jp2）；
Portable Network Graphics（*.png）；
TIFF（*.tiff、*.tif）；
WebP（*.webp）；
3、 Similarity comparison method
There are several methods for OpenCV similarity comparison:
1. Histogram comparison method
2. Image template matching
3. PSNR peak signal to noise ratio
4. SSIM (structural similarity) structure similarity
5. Perceptual hash algorithm
4、 Implementation steps:
1) Plan the starting and ending points of the two versions of the map, save the screenshots, and read two images locally
bool compareLineByHist(Mat img,Mat orgImg){ Mat tmpImg; resize(img, tmpImg, Size(orgImg.cols, orgImg.rows)); imshow("Img1", img); imshow("tmpImg", tmpImg); imshow("orgImg", orgImg);
2) Convert the two images into H hue, S saturation and V brightness formats
cvtColor(tmpImg, tmpImg, COLOR_BGR2HSV); cvtColor(orgImg, orgImg, COLOR_BGR2HSV);
3) Build the histogram model of the image and normalize the histogram
int hBins = 256, sBins = 256; int histSize[] = { hBins,sBins };
float hRanges[] = { 0,180 };
4) Compare the histogram models of two pictures and calculate the histogram similarity of pictures
float hRanges[] = { 0,180 };
float sRanges[] = { 0,255 };
const float* ranges[] = { hRanges,sRanges };
int channels[] = { 0,1 };// two-dimensional histogram
MatND hist1, hist2;
calcHist(&tmpImg, 1, channels, Mat(), hist1,2,histSize, ranges, true, false); normalize(hist1, hist1, 0, 1, NORM_MINMAX, -1, Mat()); calcHist(&orgImg, 1, channels, Mat(), hist2, 2, histSize, ranges, true, false); normalize(hist2, hist2, 0, 1, NORM_MINMAX, -1, Mat()); Double similarity Value=compareHist (hist1, hist2, CV_COMP_CORREL); cout<<"similarity:"<<similarity Value<<endl; if (similarity Value>=0.85) 0.85 is the threshold
Sogou test WeChat signal: Qa_xiaoming
