huststore
is a open source high performance distributed database system. It not only provides key-value storage service with extremely high performance, up to 100 thousand QPS, but also supports data structures like hash
, set
, sorted set
, etc. Also, it can store binary data as value from a key-value pair, and thus can be used as an alternative of Redis.
In addtion, huststore
implements a distributed message queue by integrating a special HA
module, features including message Push Stream, and message Publish-SubScribe, these features can be used as replacements of the corresponding features in rabbitmq and gearman.
huststore
has two core components, hustdb
and HA
. hustdb
uses fastdb
, a database engine developed by our own, in the fundamental architecture. HA
is implemented as a nginx
module. It is well-known that nginx
is a industry-proven high quality code base, thus by inheriting it huststore
gains the below advantages:
-
High Throughput
hustdb
uses libevhtp, a open source network library, as the inner network communication system, by incorporating it withfastdb
storage engine,hustdb
achieves a extremely high performance, the benchmark shows thatQPS
hits 100 thousand and even more. -
High Concurrency
Please refer to concurrency report ofnginx
for more details. -
High Availability
huststore
architecture providesReplication
(master-master) andload balance
support. Therefore, the availability ofHA
is guaranteed bymaster-worker
design. When one ofworker
process is down, themaster
will load anotherworkder
process, since multipleworker
s work independently, theHA
is guaranteed to work steadily. The fundamental design architecture ofhuststore
guarantees the high availability, by usingmaster-master
architecture, when one of the storage node fails,HA
module will re-direct the request to another livingmaster
node. Also, when a node failure happens,HA
cluster will automatically re-balance the data distribution, thus avoid single point of failure. In addition,HA
cluster uses a distributed architecture design by incorporating LVS as the director, eachHA
node is separated and work independently. When one of theHA
node is down,LVS
will re-direct the request to other availableHA
node, thus avoidsHA
's failure on single point node. -
Language-free Interface
huststroe
usehttp
as the communication protocol, therefore the client side implementation is not limited in any specific programming language. -
Support Binary Key-Value
- Distributed KV storage = HA(hustdb ha) + DB(hustdb)
- Distributed Message Queue = HA(hustmq ha) + DB(hustdb)
Above includes detailed documents of design, deployments, API
usage and test samples. You can refer quickly to common problems in FAQ
part.
hustdb
doc
db
ha
sync
hustmq
doc
ha
hustdb/ha
provides service for storage engine, could configured with multiple worker
s.
hustmq/ha
provides service for message queue, can only configured with one worker
.
Machine Configuration: 24core, 64gb, 1tb sata(7200rpm)
Stress Test: 100 concurrent, 1000 thousand querys
DB CONF: single instance, thread model, 10 workers
Benchmark Result:
(1)PUT
<1>value: 256B; qps: 95 thousand
<2>value: 1KB; qps: 85 thousand
<3>value: 4KB; qps: 25 thousand
<4>value: 16KB; qps: 7 thousand
<5>value: 64KB; qps: 2 thousand
(2)GET
<1>value: 256B; qps: 100 thousand
<2>value: 1KB; qps: 95 thousand
<3>value: 4KB; qps: 25 thousand
<4>value: 16KB; qps: 7 thousand
<5>value: 64KB; qps: 2 thousand
(3)DEL
<1>value: 256B; qps: 100 thousand
<2>value: 1KB; qps: 100 thousand
<3>value: 4KB; qps: 100 thousand
<4>value: 16KB; qps: 100 thousand
<5>value: 64KB; qps: 100 thousand
Machine Configuration: 24core, 64gb, 1tb sata(7200rpm)
Stress Test Parameters: 100 concurrent, 1000 thousand querys, single queue
DB Configuration: single instance, thread model, 10 workers
Benchmark Result:
(1)PUT
<1>item: 256B; qps: 30 thousand
<2>item: 1KB; qps: 25 thousand
<3>item: 4KB; qps: 20 thousand
<4>item: 16KB; qps: 7 thousand
<5>item: 64KB; qps: 2 thousand
(2)GET
<1>item: 256B; qps: 25 thousand
<2>item: 1KB; qps: 20 thousand
<3>item: 4KB; qps: 18 thousand
<4>item: 16KB; qps: 7 thousand
<5>item: 64KB; qps: 2 thousand
huststore
is licensed under New BSD License, a very flexible license to use.
- XuRuibo(hustxrb, [email protected])
- ChengZhuo(jobs, [email protected])