In the recent test, Xiao Bian encountered some tests on the grpc interface, stepped on some holes, and summarized some experience. I want to share with you. In this issue, we mainly talk about the processing methods of some special data types in protobuf in python. Since protobuf3 has become the mainstream at present, this article will discuss it directly with proto3.
Past reviews:
Testing gRPC Interface with Python
Testing gRPC Interface with Python (2)
1、 Scalar Value Type
Scalar value type is similar to the basic data type concept we use in programming languages, and the data used to carry is also roughly the same. In python, these scalar value types can find their corresponding python data types, which is simple and convenient to process.
Example:
message Student { string name = 1; int32 age = 2; // true: male, false:female bool sex = 3; }
Python implementation code:
name= "Xiao Wang" age=15 sex=True
student=Student(name=name,age=age,sex=sex)
student=Student() student.name=name student.age=age student.sex=sex
2、 Some special types
In addition to the scalar value types mentioned above, proto3 also defines some other special data types, which are convenient for us to construct and transfer various complex data structures. The official provided a JSON mapping table for these types, which can visually see the basic structure of the data contained in various types.
These types allow us to easily construct various data forms. There are several commonly used types, which are often encountered in the tests carried out by small editors. Let's introduce them in combination with practical examples.
1、message
Message. According to the mapping table, it is similar to the object of the class we use in the programming language. In a class, we can add various other types of data, including the class itself. Through analogy, message has a similar concept. We can add various proto type data, including message. In fact, just like the name of message, message is the core type in protobuf. In the grpc interface, we complete data interaction by sending and receiving messages to achieve the corresponding functions.
Simple message:
message Person { int32 id = one ; string name = two ; string email = three ; }
Messages with other messages:
message Point { int32 latitude = one ; int32 longitude = two ; }
message Feature { string name = one ; Point location = two ; }
Use in Python:
location =Point(latitude= five ,longitude= ten ) Feature =Feature(name= "I'm a name" ,location=location)
2、 Timestamp、Duration
Both types are about time. Timestamp is the time stamp, and Duration is the time length.
In the test of the account service on the AI platform, the message of an account type is defined as follows:
message Account { string account_id = 1; google.protobuf.Timestamp update_at = 2; google.protobuf.Duration time_limit = 3; }
Use in Python:
update_at=Timestamp() update_at.FromJsonString("1970-01-01T00:00:00Z") update_at.GetCurrentTime()
time_limit=Duration() time_limit.FromNanoseconds(1999999999) time_limit.FromSeconds(100)
account=Account(account_id= "account1" , update_at=update_at,time_limit=time_limit)
3、Any
The Any type is special. It can contain different messages. In combination with pack and unpack, you only need to declare one Any to pass messages of various types without declaring multiple fields.
In the conference simultaneous interpretation project, the message of a request needs to transmit two kinds of information - pictures and audio, so the same field can be reused through the Any type:
message ImageData { string index = 1; bytes image = 2; }
message Data { string appid = 1; bytes payload = 2; string extra = 3; }
message Request { google.protobuf.Any body = 1; }
Use in Python:
imageData=msg_pb2.ImageData(index= "001" ,image=open( "1.jpg" , "rb" ).read()) req1=msg_pb2.Request() req1.body.Pack(imageData)
data=msg_pb2.Data(name= "no.1" ,payload=open( "1.wav" , "rb" ).read(),extra= "no use" ) req=msg_pb2.Request() req.body.Pack(data)
4、enum
The enum type has the same concept as the enum type in most other programming languages. It mainly defines the content that can be transferred by setting some fixed values in advance.
In the test of AI platform real name authentication service, a field of authenticator type is required. Because the authenticator type converges, the enum type is used to define:
enum PersonType { PERSONTYPE_UNSPECIFIED = zero ; INDIVIDUAL = one ; LEGAL = two ; AUTHORIZE = three ; }
message Person { string real_name = one ; PersonType person_type = two ; }
Application in Python:
person_type=PersonType.Value( "INDIVIDUAL" ) Person(real_name= "Xiao Wang" , person_type=person_type)
5、map
A map is equivalent to a key value pair in json. It is similar to a dictionary (dict) in Python. We can use Python dict type data to set a map. When a map is declared in proto, it usually has angle brackets to specify the specific types of key and value. For example, map<string, string>means that the key and value of the key value pair are both string types.
In the AI platform authentication related tests, it is necessary to bind several different special attributes to the applications created by users. Each special attribute corresponds to an attribute value. Here, the map type is used:
message App { string appid = one ; map < string , string > extra_informations = two ; }
Application in Python:
extra_informations ={ "name" : "app1" , "expired" : "no" } app =App(appid= "1234567" , extra_informations=extra_informations)
6、repeated
Repeated is equivalent to a list in json. It is similar to a list in Python. We can use Python's list type data to set repeated.
In the test of AI platform account service, different capabilities need to be added to the account. Each capability has multiple attributes, and the type of each capability attribute is consistent with the data type. Here the repeated Type:
message Audience { string name = one ; string tier = two ; } message Account { string account_id = one ; repeated Audience audience = two ; }
Application in Python:
audience =[{ "name" : "ASR" , "tier" : "stand" },{ "name" : "TTS" , "tier" : "free" },{ "name" : "MT" , "tier" : "stand" }] account =Account(account_id= "account1" ,audience=audience)
3、 Problems and skills in practical application
1. Repeated type assignment problem
If the Python code in the repeated type example mentioned above is changed to the following form, an error will be reported at runtime:
audience=[{ "name" : "ASR" , "tier" : "stand" },{ "name" : "TTS" , "tier" : "free" },{ "name" : "MT" , "tier" : "stand" }] account=Account(account_id= "account1" ) account.audience=audience
Error message:
AttributeError: Assignment not allowed to repeated field "name" in protocol message object .
This seems to be different from the two assignment methods of message mentioned above, but the fact is that the repeated type in protobuf does not exactly correspond to the list in python as we think, so there will be problems here. Therefore, in practical application, we should avoid this writing method and try to use the method in the above example. In addition, we can use another way to achieve the same effect:
audience=[{ "name" : "ASR" , "tier" : "stand" },{ "name" : "TTS" , "tier" : "free" },{ "name" : "MT" , "tier" : "stand" }] for audience1 in audience: a=account.audience. add () a.name=audience1[ 'name' ] a.tier=audience1[ 'tier' ]
2. Data construction of complex messages
In the actual test interface, sometimes the structure of a message may be very complex, such as some interfaces of speech recognition services, and the protocol contains many different types of messages and repeated types. This will have some impact on our writing test client code, constructing cases, and parsing cases. Previously, we introduced the method of using the command line to transfer parameters, which is obviously difficult to meet the needs of this situation. It is also very inconvenient to manually spell the message. After some research, we found that in this case, we can use the protobuf library The Parse and MessageToJson methods in json_format can effectively solve the problem. These two methods can realize the mutual conversion of protobuf message and json. Because there are many and flexible ways to process json, we can use the json method when constructing the case, and directly convert json into message through Parse method. After receiving the returned results, you can use the MessageToJson method to convert the message to json, so that for us testers, the data sent and received looks like json, and it will be much easier to prepare test data and verify the results.
Example:
from google.protobuf import json_format json_obj='{ "a1" :1, "a2" :2}' request = json_format.Parse(json_obj,MessageName()) json_result = json_format.MessageToJson(request) print (json_result)
Where MessageName is the name of the message, and json_result is the returned result after being converted to json.
This article introduces some relations between protobuf data type and Python data type, as well as the construction method. Combining the four gRPC interface test request methods introduced in the previous two articles, we can construct various types of data and test various gRPC interfaces. Okay, that's all for this issue. See you next time~
If you want to say anything, you can send the content to the public account of "Sogou Test". Let's talk together~
Welcome to add our Sogou test micro signal , talk about testing with us.
