Weekly update (July 23 ~ July 29, 2016)

Last week, we landed 27 PRs in the TiDB repositories and 34 PRs in the TiKV repositories.

Notable changes to TiDB

• Support cost based query optimization.
• Set the new query optimizer as default to improve the speed of complex queries. Meanwhile, a start-up parameter is provided to switch to the old query optimizer.
• Use Varint to encode the Column Value with integer type and Column ID, which saves storage space significantly.
• Add a Documents repository.
• Supprt the Hex Function.
• Simplify the compling and deployment for better usability.
• Fix several bugs.

New Contributor

Huaiyu Xu

Notable changes to TiKV

• Support building TiKV by linking static RocksDB automatically. You can build TiKV in one machine and copy the binary to other machines to use directly as long as the machines have the same architecture and operation system.
• Supprt getting snapshot asynchronously for higher throughput and better performance, see benchmark.
• Use PipeBuf to receive data to reduce system call and memory allocation.
• Use Varint to encode un-comparable integers to save disk space.
• Use one SendCh to clean up duplicated code.
• Add user documents on how to build and use TiKV.

Notable changes to Placement Driver

• Embed etcd to for easier deployment.
• Add a health check for Store.
• Add user documents on how to build and use PD.
• Clean up the command flags.

Benchmark

Use sysbench to benchmark getting snapshot asynchronously and synchronously in 3-node TiKV.

# Prepare data
sysbench --test=./lua-tests/db/oltp.lua --mysql-host=${host} --mysql-port=${port} \
 --mysql-user=${user} --mysql-password=${password} --oltp-tables-count=1 \
 --oltp-table-size=5120000 --rand-init=on prepare

# Run benchmark
sysbench --test=./lua-tests/db/select.lua --mysql-host=${host} --mysql-port=${port} \
 --mysql-user=${user} --mysql-password=${password} --oltp-tables-count=1 \
 --oltp-table-size=5120000 --num-threads=${threads} --report-interval=60 \
 --max-requests=5120000 --percentile=99 run

|Threads|Async qps|Async avg/.99 latency|Sync qps|Sync avg/.99 latency| |—|—|—|—|—|—| |32|13347|2.⁴⁄₄.61|12345|2.⁵⁹⁄₄.78| |64|14210|4.⁵⁰⁄₇.78|11868|5.³⁹⁄₈.50| 128|14075|9.09/15.22|12324|10.³⁸⁄₁₆.68|

As we can see, the qps is increased by about 15%, and the latency is decreased by about 10%.