Notes on Text Processing

2020/03/20 08:00
Reading number 109
[Preface]
In the normal testing process, we often encounter various text processing problems, so we summarized and sorted out the commonly used text processing commands and methods.


[Common Text Processing Commands]
awk
 one . awk script structure awk ' BEGIN{statements } statements2 END{ statements } ' two . Working mode one )Execution begin Middle statement block; two )Read a line from the file or stdin, and then execute statements2. Repeat the process until all files are read; three )Execution end Statement block; Special variable: NR NF $0 $1 $2 NR: Indicates the number of records, corresponding to the current line number during execution; NF: Indicates the number of fields. It corresponds to the number of fields in the row during execution; $0 : This variable contains the text content of the current line during execution; $1 : The text content of the first field; $2 : The text content of the second field; Example: Print the second and third fields of each line awk '{print $2,$3}' file Count the number of lines in the file awk  ' END {printNR}'  file
grep
 It is mainly used for text search. It can use regular expressions to search for text and print matching lines. Some commonly used parameters are as follows: -I: Ignore case when searching -v: Reverse matching, select the content that is not matched. -w: Match the whole word, accurate to the word. Both sides of the word must be non character symbols (that is, cannot be alphanumeric or underscore) - x: only the matching items that exactly match the whole line are selected. Exactly match the contents of the whole line (including the contents of spaces that cannot be seen at the beginning and end of the line) -A num: matches the searched line and the num line below the line -B num: matches the searched line and the num line above the line -C num: matches the searched lines and the upper and lower num lines
  
    
  
  
  
Redirect I/O
 command >File Redirects the output to file. command <file Redirects input to file. command >>File Redirects the output to file by appending. command >/dev/null/dev/null is a special file. Anything written to it will be discarded


Practical Application
1. Processing text
In the test, the following text processing situations were encountered:
In multi line structured text like this, you need to extract the text, and then calculate the total time of all the text, so you think of using the previous text processing process.
To extract all the text first, you need to use the previous line of the part, remove the part line, and redirect to the txt file to get the current text:
 grep  -B  one   'part'  example.txt |grep -v  'part'  > result.txt
Then look up the row of time and add up the data in the second column to get the total time.
 grep   'time' example.txt | awk '{sum+= $2 } END {print sum}'
2. Use in shell script
 #!/ bin/bash date>>log.txt while  true do adb shell top -b -n 1|grep demo1 |awk '{print $9}' >> log1.txt adb shell dumpsys meminfo demo2|grep "TOTAL" |awk '{print $2}' |sed -n '2p' >> log2.txt sleep 1 done date>>log.txt
In the process of testing, it can also be introduced into the script to extract the required information according to the log information at different times to assist in recording and analysis.

The above is a brief introduction to some text processing commands. When you encounter text processing problems in your daily work, it will be easier and faster to solve them.


This article is shared from the WeChat official account Sogou QA.
In case of infringement, please contact support@oschina.cn Delete.
Participation in this article“ OSC Source Innovation Plan ”, welcome you to join us and share with us.

Expand to read the full text
Loading
Click to lead the topic 📣 Post and join the discussion 🔥
Reward
zero comment
zero Collection
zero fabulous
 Back to top
Top