Infobright is a columnar database based on patented technology. It is an open source data warehouse software developed based on MySQL, which can be used as a storage engine of MySQL. SELECT query is no different from ordinary MySQL.
1、 Several advantages:
1. High compression ratio, with an average compression ratio of 10:1. (According to the test, the compression ratio of our 15 million 697M logs is 6:1, and the compressed data size is only 114M)
2. Column storage, even if the amount of data is huge, the query speed is fast. (After testing, a composite query in infobright takes 30 seconds and more than one minute in MySQL)
3. Without index building, the problem of index maintenance and index expansion with data is avoided. Each column of data is compressed and stored in blocks, and each knowledge grid node records the statistical information in the block to replace the index and speed up the search.
4. A single server can efficiently read and write 30T data. Scalability, which means that for the same query, when the data volume is 10T, the time consumed should not be too much slower than when the data volume is 1T, which is basically within an order of magnitude.
Note: In fact, infobright takes a record much slower than MySQL, but it takes 100W records faster than MySQL. Therefore, it is applicable to a large amount of data in logs or summaries.
2、 Infobright restrictions
1. Data update is not supported: community version Infobright can only use "LOAD DATA INFILE" to import data, and does not support INSERT, UPDATE, and DELETE
2. High concurrency is not supported: only 10-18 concurrent queries are supported
Install infobright in rpm package mode
1. Install rpm
Download address: http://www.infobright.org/
two Install inforright
# rpm -ivh infobright-4.0.7-0-x86_64-ice.rpm --prefix=/data/infobright #(Here we can write only one level directory, such as/data, which will create the infobright version-x86_64 directory by default)
Prompt that the installation is complete and execute/ postconfig.sh
3. Prompt to/data/inforight/inforight and execute
# ./ postconfig.sh
four Start, close infobright
# /data/infobright/infobright/bin/mysqld_safe start/stop
5. View process
# ps -ef |grep mysql
The infobright port is 5029
six Enter infobright
#/data/infobright/infobright/bin/mysql -u root -p
7. Application
CREATE TABLE `t0` ( `id` bigint(20) NOT NULL , `name` varchar(20) NOT NULL, `age` int(3) NOT NULL ) ENGINE=BRIGHTHOUSE DEFAULT CHARSET=utf8;
Note: The engine used is
BRIGHTHHOUSE primary key, empty, auto increment None of this engine supports.
8. Use load data infile to import logs
Basic syntax: load data [low_priority] [local] infile 'file_name txt' [replace | ignore] into table tbl_name [fields [terminated by't'] [OPTIONALLY] enclosed by ''] [escaped by'\' ]] [lines terminated by'n'] [ignore number lines] [(col_name, )]
low_priority : MySQL will insert data only when no one else reads the table
Replace and ignore : Controls the duplicate processing of existing unique key records. If you specify replace, the new row will replace the existing row with the same unique key value. If you specify ignore, skip the input of duplicate lines of existing lines with only one key. If you do not specify any option, an error will appear when the duplicate key is found, and the rest of the text file will be ignored.
fields : The keyword specifies the file field segmentation format,
terminated by : What character is used as the separator
enclosed by : Field enclosing character
lines : The delimiter of each record specified is' n 'by default, which is the newline character
A relatively complete load data infile
load data infile '/info.txt' ignore into table t0 character set gbk fields terminated by ',' enclosed by '"' lines terminated by '\n';
info.txt
1,"name1",2 2,"name2",3
reference resources: http://hunan.iteye.com/blog/752606
http://blog.csdn.net/liyanyun/article/details/8858093
