Thrift Agent Hbase

original
2016/09/22 17:08
Reading number 3.2K

Two ways to use HBase:

1. Directly use the HBase client API, which limits the use of only the java language

2. Use some proxies that can convert requests into APIs. These proxies package the original Java API into other protocols, so that clients can use any external language provided by the API to write programs. The external API implements a special java based service, which can use the API provided by the HTable client internally.

HBase itself supports a wide range of proxy modes, such as REST, Thrift, Avro, etc

For the proxy mode, see an online architecture diagram:

Here we mainly introduce Thrift's external services as an agent of HBase, mainly its advantages in performance and support for various mainstream languages

1. Install HBase and Hadoop

Hadoop version: 2.5.1
Hbase version: 1.2.2

Refer to previous articles:
Hadoop version selection and stand-alone mode: http://codingo.xyz/index.php/2016/08/16/hadoop-stand-alone/
The pseudo distributed mode of Hadoop: http://codingo.xyz/index.php/2016/08/16/hadoop_false_distribute/
Hbase version selection and getting started in stand-alone mode: http://codingo.xyz/index.php/2016/08/17/hbase_standalone/
Pseudo distributed mode of Hbase: http://codingo.xyz/index.php/2016/08/16/hadoop_false_distribute/

2. Install Thrift
Thrift version: 0.9.3
Download address: https://thrift.apache.org/download
Windows platform: directly use thrift-0.9.3.exe
Installation of other platforms: https://thrift.apache.org/docs/install/

3. Compile mode file
Hbase provides the schema file required by Thrift, which is stored in the source code of Hbase. What you need to download is: hbase-1.2.2-src.tar.gz
route:
$HBASE_HOOME/src/main/resources/org/apache/hadoop/hbase/hrift/Hbase.thrift and
$HBASE_HOOME/src/main/resources/org/apache/hadoop/hbase/thrift2/hbase.thrift
Two sets of Thrift files are provided, which are incompatible; According to the official documents, thrift1 is likely to be abandoned. Thrift2 is used in all the following examples

Windows platform coding mode file:

 thrift-0.9.3.exe --gen java hbase.thrift

In fact, java can be selected according to different languages, such as c++, perl, php, python, ruby, etc
The following structure codes are generated

Copy it to the development environment of eclipse. The official hbase-thrift-1.2.2.jar The jar file is actually our class file above.

4. Obtain the language support library provided by Thrift
The above automatically generated class files depend on the support libraries provided by Thrift. The Thrift-0.9.3/lib directory provides support libraries for various languages
Select the java support library and copy other codes to the development environment of eclipse. The official ones are not used here: libthrift-0.9.3.jar

5. Start the service
The above provides the installation and startup of the pseudo distributed mode of Hbase and Hadoop
Hbase provides support for ThriftServer startup

Start and stop Thrift service in non daemon mode:

 hbase thrift2 start hbase thrift2 stop

The background process starts and stops the Thrift service:

 hbase-daemon.sh start thrift2 hbase-daemon.sh stop thrift2

Startup log:

 [ root@bogon  ~]# hbase thrift2 start SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/root/hbase-1.2.2/lib/slf4j-log4j12-1.7.5.jar!/ org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/root/hadoop-2.5.1/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/ org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See  http://www.slf4j.org/codes.html#multiple_bindings  for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl. Log4jLoggerFactory] 2016-09-20 08:57:05,211 INFO  [main] impl. MetricsConfig: loaded properties from hadoop-metrics2-hbase.properties 2016-09-20 08:57:05,367 INFO  [main] impl. MetricsSystemImpl: Scheduled snapshot period at 10 second(s). 2016-09-20 08:57:05,367 INFO  [main] impl. MetricsSystemImpl: HBase metrics system started 2016-09-20 08:57:06,123 INFO  [main] mortbay.log: Logging to org.slf4j.impl. Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log. Slf4jLog 2016-09-20 08:57:06,129 INFO  [main] http. HttpRequestLog: Http request log for http.requests.thrift is not defined 2016-09-20 08:57:06,148 INFO  [main] http. HttpServer: Added global filter 'safety' (class=org.apache.hadoop.hbase.http. HttpServer$QuotingInputFilter) 2016-09-20 08:57:06,148 INFO  [main] http. HttpServer: Added global filter 'clickjackingprevention' (class=org.apache.hadoop.hbase.http. ClickjackingPreventionFilter) 2016-09-20 08:57:06,151 INFO  [main] http. HttpServer: Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib. StaticUserWebFilter$StaticUserFilter) to context thrift 2016-09-20 08:57:06,151 INFO  [main] http. HttpServer: Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib. StaticUserWebFilter$StaticUserFilter) to context static 2016-09-20 08:57:06,152 INFO  [main] http. HttpServer: Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib. StaticUserWebFilter$StaticUserFilter) to context logs 2016-09-20 08:57:06,173 INFO  [main] http. HttpServer: Jetty bound to port 9095 2016-09-20 08:57:06,174 INFO  [main] mortbay.log: jetty-6.1.26 2016-09-20 08:57:06,680 INFO  [main] mortbay.log: Started  SelectChannelConnector@0.0.0.0 :9095 2016-09-20 08:57:06,695 INFO  [main] thrift2.ThriftServer: starting HBase ThreadPool Thrift server on 0.0.0.0/0.0.0.0:9090

Default external ports: nine thousand and ninety

6. Testing
About eclipse remote connection to Hbase: https://my.oschina.net/OutOfMemory/blog/746860
The class file generated by compiling the schema file and the language support library provided by Thrift are already available in the development environment

 import java.nio.ByteBuffer; import java.util.ArrayList; import java.util.List; import org.apache.hadoop.hbase.thrift2.generated.TColumnValue; import org.apache.hadoop.hbase.thrift2.generated.THBaseService; import org.apache.hadoop.hbase.thrift2.generated.TPut; import org.apache.thrift.protocol.TBinaryProtocol; import org.apache.thrift.protocol.TProtocol; import org.apache.thrift.transport.TSocket; import org.apache.thrift.transport.TTransport; public class ThriftExample { public static void main(String[] args) throws Exception { TTransport transport = new TSocket("192.168.111.129", 9090, 20000); TProtocol protocol = new TBinaryProtocol(transport, true, true); THBaseService.Client client = new THBaseService.Client(protocol); transport.open(); ByteBuffer table = ByteBuffer.wrap("testTable".getBytes()); TPut put = new TPut(); put.setRow("row1".getBytes()); TColumnValue columnValue = new TColumnValue(); columnValue.setFamily("family1".getBytes()); columnValue.setQualifier("qualifier1".getBytes()); columnValue.setValue("value1".getBytes()); List<TColumnValue> columnValues = new ArrayList<TColumnValue>(); columnValues.add(columnValue); put.setColumnValues(columnValues); client.put(table, put); transport.close(); } }

View through hbase shell:

 hbase(main):003:0> scan 'testtable' ROW                         COLUMN+CELL                                                 rthrift                     column=colfam1:qualifier1, timestamp=1474376284145, value=value1

The above tests are conducted in the java language. The above steps are as follows: Compile the schema file and the language support library provided by Thrift With minor modifications, it also applies to other languages

Personal blog: codingo.xyz

Expand to read the full text
Loading
Click to lead the topic 📣 Post and join the discussion 🔥
Reward
zero comment
ten Collection
zero fabulous
 Back to top
Top