Skip to content

Home

Jump to bottom
Baoying Wang edited this page Jan 18, 2018 · 55 revisions

Introduction

It is an OrderBook(Matching Engine).

  • It support LIMIT and MARKET order.
  • Generate execution reports and market data
  • Support Web interface and FIX interface
  • the highlight is performance v1 performance - 5000 order/second a) match:2.5us 95% b) e2e: 20ms 95%

Architecture - v2.0 - all bytes (on plan)

  • receive order bytes from sockets (use NIO - netty with poll/select/etc)
  • direct memory(off-heap) is used for orders(fixed size on each field + version for easy upgrade)
  • order book will link to the direct memory with index
  • update direct memory for order status change, e.g. filled, etc
  • clean the order after order completed(filled/rejected/...)

Pro

  1. almost no GC, since off-heap memory operation
  2. save the time for thread context switch - single thread(or 2 threads) to process the orders.
  3. minimize message marshal(to bytes)/unmarshal(to object).

Cos

  1. hard code since all bytes operation. It can be mitigated to wrap the byte operation in a single class
  2. NIO code is more difficult. It can be mitigated to use netty library
  3. Hard to support FIX. We have to introduce our own protocol to define.

Architecture - v1.1 - add an interface with pure TCP without FIX store/IO overhead(ongoing)

see : https://github.com/baoyingwang/OrderBook/wiki/Track-Performance-Tuning-for-v1.1 It is a wrong direction to reduce the Q inside. Because 1.the goal should be reduce e2e. The e2e is about 14 ms. But the Q will only cause a few us(2050). We should focus on the main part of the time cost.

ME_1.1_architecture_20180118.png

Architecture - v1.0 (released)

ME_1.0_architecture_20180107.png

Why not also use Disruptor on the output Q?

  • is it required so fast for the output Q, since order has already executed?
  • guava async event bus is rather easy to use. The code is clean.

Current Performance - v1.0 - Jan 7, 2018

  • note: rate per second during test 5000 orders per second
  • note: java ArrayBlockingQueue is used as input Q. Another option is LMAX Disruptor, which is also supported.
  • note: it is tested on my ubuntu(a VM of win7 64bit). 2 cpu are assigned to this vm. Disruptor_BusySpinWaitStrategy_bg5000perSec_lt60perMin_duration600sec_20180107_102049_latency_overall.png

For more images and test data see https://github.com/baoyingwang/OrderBook/wiki/Track-Performance-Tuning - section: Jan 7, 2018 - add e2e measurement (release v1.0 is defined since today)

Next

  1. performance tuning (GC, more stable performance, etc)
  2. complete the FIX interface (it should be simple to do that, based on my QFJTutorial code lines) almost done - Dec 21, 2017. It support place order and receive streaming execution report. I am on the FIX test client.
  3. complete place order part of Web Interface
  4. complete Marked Data Engine
  5. move some util to standalone project, e.g. FIX test tool, sys statistics collection etc. These can be shared to many other projects. (BTW - refer jitpack to share by github as repo / https://stackoverflow.com/questions/32654448/using-personal-library-hosted-on-github-as-gradle-dependency)

How to use

after clone, build a fat jar with gradle

gradle shadowJar

Then start it

java -jar the_generated_jar_file

OR specific the queue type(-q BlockingQueue-default), buffer size(-b 65536 by default) and disruptor strategy(-s SleepingWaitStrategy default), and gc
jarfile=BaoyingOrderBookFat-2017-12-24_101139.396-all.jar
java -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -jar ${jarfile}  -q BlockingQueue
java -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -jar ${jarfile}  -q Disruptor -s SleepingWaitStrategy
java -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -jar ${jarfile}  -q Disruptor -s YieldingWaitStrategy
java -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -jar ${jarfile}  -q Disruptor -s BusySpinWaitStrategy

another example to tell more gc parameters
java -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -XX:+PrintGC -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -Xloggc:log/GC.txt -jar the_generated_jar_file

Populate OrderBook

jarfile=BaoyingOrderBookFat-2017-12-24_101139.396-all.jar

java -cp ${jarfile} baoying.orderbook.testtool.FirstQFJClientBatch -clientNum 1 -ratePerMinute 10 -client_prefix BACKGROUD_FIX_prepare -symbol USDJPY -side Bid -qty 50000000 -ordType Limit -px 112 -d 5
java -cp ${jarfile} baoying.orderbook.testtool.FirstQFJClientBatch -clientNum 1 -ratePerMinute 10 -client_prefix BACKGROUD_FIX_prepare -symbol USDJPY -side Bid -qty 50000000 -ordType Limit -px 113 -d 5
java -cp ${jarfile} baoying.orderbook.testtool.FirstQFJClientBatch -clientNum 1 -ratePerMinute 10 -client_prefix BACKGROUD_FIX_prepare -symbol USDJPY -side Bid -qty 50000000 -ordType Limit -px 114 -d 5
java -cp ${jarfile} baoying.orderbook.testtool.FirstQFJClientBatch -clientNum 1 -ratePerMinute 10 -client_prefix BACKGROUD_FIX_prepare -symbol USDJPY -side Bid -qty 50000000 -ordType Limit -px 115 -d 5
java -cp ${jarfile} baoying.orderbook.testtool.FirstQFJClientBatch -clientNum 1 -ratePerMinute 10 -client_prefix BACKGROUD_FIX_prepare -symbol USDJPY -side Bid -qty 50000000 -ordType Limit -px 116 -d 5
 

java -cp ${jarfile} baoying.orderbook.testtool.FirstQFJClientBatch -clientNum 1 -ratePerMinute 10 -client_prefix BACKGROUD_FIX_prepare -symbol USDJPY -side Offer -qty 50000000 -ordType Limit -px 122 -d 5
java -cp ${jarfile} baoying.orderbook.testtool.FirstQFJClientBatch -clientNum 1 -ratePerMinute 10 -client_prefix BACKGROUD_FIX_prepare -symbol USDJPY -side Offer -qty 50000000 -ordType Limit -px 123 -d 5
java -cp ${jarfile} baoying.orderbook.testtool.FirstQFJClientBatch -clientNum 1 -ratePerMinute 10 -client_prefix BACKGROUD_FIX_prepare -symbol USDJPY -side Offer -qty 50000000 -ordType Limit -px 124 -d 5
java -cp ${jarfile} baoying.orderbook.testtool.FirstQFJClientBatch -clientNum 1 -ratePerMinute 10 -client_prefix BACKGROUD_FIX_prepare -symbol USDJPY -side Offer -qty 50000000 -ordType Limit -px 125 -d 5
java -cp ${jarfile} baoying.orderbook.testtool.FirstQFJClientBatch -clientNum 1 -ratePerMinute 10 -client_prefix BACKGROUD_FIX_prepare -symbol USDJPY -side Offer -qty 50000000 -ordType Limit -px 126 -d 5

Send massive orders as background

e.g. -d 600 seconds , 10 clients, 1000 orders per minute to Buy
java -cp ${jarfile} baoying.orderbook.testtool.FirstQFJClientBatch -clientNum 10 -ratePerMinute 1000 -client_prefix BACKGROUD_FIX_bid  -symbol USDJPY -side Bid   -qty 5 -ordType Market -d 600

e.g. -d 600 seconds , 10 clients, 1000 orders per minute to Sell
java -cp ${jarfile} baoying.orderbook.testtool.FirstQFJClientBatch -clientNum 10 -ratePerMinute 1000 -client_prefix BACKGROUD_FIX_offer -symbol USDJPY -side Offer -qty 5 -ordType Market -d 600

Send those deals for latency test

java -cp ${jarfile} baoying.orderbook.testtool.FirstQFJClientBatch -clientNum 1 -ratePerMinute 1 -client_prefix 'LTC$$_FIX'  -symbol USDJPY -side Bid   -qty 5 -ordType Market -d 600
java -cp ${jarfile} baoying.orderbook.testtool.FirstQFJClientBatch -clientNum 1 -ratePerMinute 1 -client_prefix 'LTC$$_FIX'  -symbol USDJPY -side Offer -qty 5 -ordType Market -d 600

Check the order book(empty if you don't place order), and place order(not yet supported) http://localhost:8080/main.html

Check latency test summary (empty if you don't place order) http://localhost:8080/test_summary.html

Load JMeter to send test deal, and check above links see the JMeter file - OrderBook\src\test\resources\jmeter\OrderBook.jmx

Clone this wiki locally