quantpylib.hft

quantpylib.hft is our core module designed for hft-trading purposes, including modelling, analysis and simulation of high-frequency data.

quantpylib.hft.feed is our data feeder for public information and market data such as order book and trade feeds, custom candlesticks.

quantpylib.hft.oms is our order management system for positions, orders tracking and trade execution + management.

quantpylib.hft.lob is our internal limit-order book representation, and is designed to store orderbook states and buffers. There are utility functions to keep accurate representations of the internal state for both book snapshots and incremental updates. A host of other utility functions related to orderbook modelling can be found.

quantpylib.hft.trades is our internal trade buffer. There are utility functions to compute useful information and statistics from the data stored in the buffer.

quantpylib.hft.statistics is a statistics library for hft modelling - in addition it should work seamlessly with the data structures in the quantpylib.hft.lob.LOB and quantpylib.hft.trades.Trades object instances.

Examples

Data Feed

Obtaining data feeds are extremely simple with quantpylib.hft.feed.Feed objects. Simply - create a gateway object with the correct keys and pass them in

import os 
import asyncio

from pprint import pprint
from dotenv import load_dotenv
load_dotenv()

from quantpylib.gateway.master import Gateway 
from quantpylib.hft.feed import Feed

keys = {
    "binance": {'key':'','secret':''},
    "hyperliquid": {'key':'','secret':''},
}

async def printer(data):
    print(data)

async def printer_1(data):
    print('1\n',data.as_dict())

async def printer_2(data):
    print('2\n',data.as_dict())

async def main():
    exchange,ticker = 'hyperliquid', 'BTC'
    # exchange,ticker = 'binance', 'BTCUSDT'

    gateway = Gateway(config_keys=keys)
    await gateway.init_clients()
    feed = Feed(gateway=gateway)
    #code goes here
    pass

if __name__ == "__main__":
    asyncio.run(main())

To use binance, simply uncomment the line above. Proceeding...

To get a order book feed, we can write:

    l2_feed = await feed.add_l2_book_feed(exc=exchange,ticker=ticker,handler=printer_1)

and we get hyperliquid/perp/l2book/BTC_depth20. Notice how we added a handler here - this is optional. We can add arbitrary number of coroutine handlers - when the feed is received, it will be broadcast to all handlers asynchronously. All the schema is standardized - there is no need to write exchange-dependent code.

feed.add_handler_to_feed(l2_feed,printer_2)

We will get (the same message)

1
 {'ts': 1725903430442, 
 'bids': array([[5.65460e+04, 8.75000e-03],
       [5.65430e+04, 9.33710e-01],
       ...
       [5.65220e+04, 1.98847e+00]]), 
 'asks': array([[5.65470e+04, 2.20767e+00],
       [5.65480e+04, 1.67035e+00],
       ...
       [5.65710e+04, 2.64770e-01]])}
2
 {'ts': 1725903430442, 
 'bids': array([[5.65460e+04, 8.75000e-03],
       [5.65430e+04, 9.33710e-01],
       ...
       [5.65220e+04, 1.98847e+00]]), 
 'asks': array([[5.65470e+04, 2.20767e+00],
       [5.65480e+04, 1.67035e+00],
       ...
       [5.65710e+04, 2.64770e-01]])}

This message is constructed from order-book delta updates - it is a local order book state. We can also control the depth of the streams, speed of update and how much data buffer we keep. This data buffer is actually given to us as a quantpylib.hft.lob.LOB object - we can retrieve it using the feed id.

lob = feed.get_feed(l2_feed)
print(l2_feed)
print(lob) #`quantpylib.hft.lob.LOB` object
print(lob.get_mid(), lob.get_vamp(notional=30000)) #54884.5 54883.872068603436
print(lob.get_bids_buffer())

We can get the time-series of bids, most recent mid-price, bba, vamp indicators and what not from this object. It is equally simple to get the trades feed, obtain a quantpylib.hft.trades.Trades object and get the running buffer, statistics and what not:

trades_feed = await feed.add_trades_feed(exc=exchange,ticker=ticker,handler=printer)
trades = feed.get_feed(trades_feed)
await asyncio.sleep(5)
print(trades) #`quantpylib.hft.trades.Trades` object
print(trades.get_imbalance()) #0.8328041302230744

The messages look like this:

(1725904146989, 56497.0, 0.00037, -1)
(1725904147170, 56498.0, 0.01171, 1)
(1725904147773, 56498.0, 0.02284, 1)
(1725904148396, 56497.0, 0.03539, -1)
...

It is simple to interpret: time, price, size, dir. When dir = 1, it means a taker buy initiated trade. We can also get custom candlestick bars from the streamed raw tick data.

After add_trades_feed has been called, we can call add_sampling_bars_feed and subscribe to candlestick bars constructed internally from the tick stream. Different bars are provided, including classical time-based bars as well as information bars that have sampling frequency on other variables such as tick imbalance and entropy:

import quantpylib.hft.bars as bars
information_bars = await feed.add_sampling_bars_feed(
    exc=exchange,
    ticker=ticker,
    handler=printer,
    bar_cls=bars.TickBars,
    n_ticks=10
)

and the handler prints

[1747541147009.0, 103352.0, 103353.0, 103352.0, 103352.0, 0.13585, 10, 103352.85542878174, 1747541176079.0]
[1747541177073.0, 103352.0, 103353.0, 103352.0, 103352.0, 0.12148, 10, 103352.07935462627, 1747541206012.0]
[1747541207030.0, 103352.0, 103353.0, 103352.0, 103352.0, 0.41859, 10, 103352.00229341358, 1747541221429.0]
[1747541246062.0, 103352.0, 103353.0, 103352.0, 103352.0, 0.67041, 10, 103352.25309884998, 1747541258870.0]
[1747541261102.0, 103352.0, 103355.0, 103352.0, 103355.0, 0.13013, 10, 103353.21978021978, 1747541268063.0]
[1747541328296.0, 103356.0, 103356.0, 103355.0, 103355.0, 3.42183, 10, 103355.99768544902, 1747541357001.0]

These bars contain the information time_start,open,high,low,close,volume,n_ticks,vwap,time_end.

Get all the running feed ids here:

print(feed.get_feed_ids())

['hyperliquid/perp/l2book/BTC_depth20', 'hyperliquid/perp/trades/BTC', 'hyperliquid/perp/TickBars/BTC']

Data Archival

In the previous section we have seen how easy it is to run tick data streams, add handlers and use LOB and Trade buffers to compute live statistics efficiently for high-frequency strategies. We would also like to have a data archival feature that allows us to store tick-data in a seamless manner to disk.

Like, one-liner seamless. Let me demonstrate. We will do the same imports and setup:

import os 
import asyncio

from pprint import pprint
from dotenv import load_dotenv
load_dotenv()

from quantpylib.hft.feed import Feed
from quantpylib.gateway.master import Gateway 

keys = {
    "binance": {'key':'','secret':''},
    "hyperliquid": {'key':'','secret':''},
}

exchanges = ['binance','hyperliquid']

async def main():
    pass #<<<

if __name__ == "__main__":
    asyncio.run(main())

except this time we instantiate the feed with a flag archiver=True and list of tickers we want to archive:

async def main():
    l2_feeds = {
        "binance": ['BTCUSDT','ETHUSDT'],
        "hyperliquid": ['BTC','SOL']
    }
    trade_feeds = {
        "binance": ['BTCUSDT','ETHUSDT'],
        "hyperliquid": ['BTC','SOL']
    }

    gateway = Gateway(config_keys=keys)
    await gateway.init_clients()
    feed = Feed(gateway=gateway,archiver=True) #enable archiver

then we add the magic line:

    await feed.run_archive_scheduler()

and the rest is as per normal (we skip the optional handler) and let the data stream forever. The run_archive_scheduler can be configured to flush data to disk and free up RAM quicker in large scale data archival with splits argument - by default splits=1.

    for exchange in exchanges:
        l2_feed_ids = await asyncio.gather(*[
            feed.add_l2_book_feed(
                exc=exchange,
                ticker=ticker,
                depth=20,
            ) for ticker in l2_feeds[exchange]
        ])
        print(l2_feed_ids)
        '''
        ['binance/perp/l2book/BTCUSDT_depth20', 'binance/perp/l2book/ETHUSDT_depth20']
        ['hyperliquid/perp/l2book/BTC_depth20', 'hyperliquid/perp/l2book/SOL_depth20']
        '''
        trade_feed_ids = await asyncio.gather(*[
            feed.add_trades_feed(
                exc=exchange,
                ticker=ticker,
            ) for ticker in trade_feeds[exchange]
        ])
        print(trade_feed_ids)
        '''
        ['binance/perp/trades/BTCUSDT', 'binance/perp/trades/ETHUSDT']
        ['hyperliquid/perp/trades/BTC', 'hyperliquid/perp/trades/SOL']
        '''
    await asyncio.sleep(1e59)
    await feed.cleanup()
    await gateway.cleanup_clients()

And you will see that at the turn of every hour, the data for that hourly period is saved like this:

./archives/{exchange}/{feed_cls}/{feed_type}/{yy}/{mm}/{ticker}{_metadata}_{ddhh}.parquet

so from the archives/ folder:

./binance/perp/trades/2024/12/BTCUSDT_3014.parquet
./binance/perp/trades/2024/12/ETHUSDT_3014.parquet
./binance/perp/l2book/2024/12/BTCUSDT_depth20_3014.parquet
./binance/perp/l2book/2024/12/ETHUSDT_depth20_3014.parquet
...

./hyperliquid/perp/trades/2024/12/BTC_3014.parquet
./hyperliquid/perp/trades/2024/12/SOL_3014.parquet
./hyperliquid/perp/l2book/2024/12/BTC_depth20_3014.parquet
./hyperliquid/perp/l2book/2024/12/SOL_depth20_3014.parquet

The archives can be loaded like this which loads sequential chronological, archived data from Dec-30th to Dec-31st of 2024:

lob = Feed.load_lob_archives(
    exc='binance',
    ticker='BTCUSDT',
    depth=20,
    start='2024-12-30:00',
    end='2024-12-31:23',
)

with format:

[
    {'ts': int,'b':np.ndarray,'a':np.ndarray}, {...}, ...
]

and similarly:

trades = Feed.load_trade_archives(
    exc='binance',
    ticker='BTCUSDT',
    start='2024-12-30:00',
    end='2024-12-31:23',
)

to give

[
    [ts,price,size,dir],
    [ts,price,size,dir],
    ...
]

We can also decide to run our own scheduler, if we don't want to run the hourly-archiver. With archiver=True and no scheduler -

feed = Feed(gateway=gateway,archiver=True)
# await feed.run_archive_scheduler()

we simply need to call

feed.flush_archives(path='./data_archives/',prefix='',postfix='')

at any point in our code and the existing buffer is emptied into a parquet file with location:

{path}/{exchange}/{feed_cls}/{feed_type}/{prefix}{ticker}{_metadata}{postfix}.parquet

For example the above call would save existing buffer to

.data_archives/hyperliquid/perp/trades/SOL.parquet
.data_archives/hyperliquid/perp/trades/BTC.parquet
.data_archives/hyperliquid/perp/l2book/BTC_depth20.parquet
.data_archives/hyperliquid/perp/l2book/SOL_depth20.parquet

#same for binance

and this can be loaded:

lobs = Feed.load_lob_archive(
    exc='binance',
    ticker='BTCUSDT',
    depth=20,
    path='./data_archives/',
    # raw=True
)

or as raw parquet with columns (ts,bp,bs,ap,as) timestamp, bid price, bid size, ask price, ask size. Same goes with trades

trades = Feed.load_trade_archive(
    exc='binance',
    ticker='BTCUSDT',
    path='./data_archives/',
    # raw=True
)

and raw parquet:

                ts    price   size  dir
0    1735569691383  92514.9  0.517    1
1    1735569691480  92514.8  0.051   -1
2    1735569691539  92514.9  0.051    1
...

OMS

It is easy to create a manager class - it is similar to thequantpylib.hft.feed.Feed objects. Simply - create a gateway object with the correct keys and pass them in.

We will demonstrate with examples:

import os 
import asyncio 

from pprint import pprint
from dotenv import load_dotenv 
load_dotenv()

from quantpylib.hft.oms import OMS
from quantpylib.gateway.master import Gateway
import quantpylib.standards.markets as markets

config_keys = {
    'binance': {
        'key': '1234',
        'secret': '1234',
    },
    'hyperliquid': {
        'key': '1234',
        'secret': '1234',
    }
}
gateway = Gateway(config_keys)

async def main():
    await gateway.init_clients()
    oms = OMS(gateway)
    await oms.init()
    #code goes here... 
    ###
    await oms.cleanup()

if __name__ == "__main__":
    asyncio.run(main())

For all of our socket-based message handlers, we will use a generic printer to showcase results:

async def printer(data):
    if isinstance(data,dict) or isinstance(data,list):
        pprint(data)
    else:
        try: pprint(data.as_dict())
        except: pprint(data.as_list())

Now, let us take a look at the functionalities. We can get trading specifications for the contracts on initiated exchanges:

pprint(oms.contract_specs(exc='hyperliquid', ticker='BTC'))

we get

{'base_asset': 'BTC',
'min_notional': Decimal('10.0'),
'price_precision': 1,
'quantity_precision': 5,
'quote_asset': 'USDT'}

Some useful statistics and utilities:

pprint(oms.lagged_price(exc='hyperliquid', ticker='ETH')) #Decimal('2368.25')
pprint(oms.lot_precision(exc='hyperliquid', ticker='BTC')) #5
pprint(oms.price_precision(exc='hyperliquid', ticker='DOGE')) #6
pprint(oms.rounded_lots(exc='hyperliquid', ticker='BTC',amount=0.0023032)) #0.0023
pprint(oms.rounded_price(exc='hyperliquid', ticker='BTC',price=62000.1234)) #62000.1
pprint(oms.min_notional(exc='hyperliquid', ticker='BTC')) #Decimal('10.0')
pprint(oms.rand_cloid(exc='binance')) #'b486130e1b35986abc803bb79d2e675d'
pprint(oms.common_lot_precision(ex1='hyperliquid',ex2='binance',ticker1='BTC',ticker2='BTCUSDT')) #3
pprint(oms.common_price_precision(ex1='hyperliquid',ex2='binance',ticker1='BTC',ticker2='BTCUSDT')) #1
pprint(oms.common_min_notional(ex1='hyperliquid',ex2='binance',ticker1='BTC',ticker2='BTCUSDT')) #Decimal('100')

We can get account informations for the included exchanges:

pprint(await oms.get_balance(exc='hyperliquid'))
pprint(await oms.get_all_balances())

We would like to get some positions data. Note that when oms.init() is called, all orders and positions are automatically mirrored using underlying exchange socket subscriptions. We can make connection-less request by retrieving local state:

pprint(oms.get_position(exc='hyperliquid', ticker='SOL')) #Decimal('1.0') (no requests made)

or we can force the OMS to make a HTTP request:

pprint(await oms.positions_get(exc='hyperliquid')) #HTTP requests made
pprint(await oms.positions_get_all())

which gives us outputs

{'SOL': {'amount': Decimal('1.0'),
         'entry': Decimal('134.81'),
         'ticker': 'SOL',
         'unrealized_pnl': -0.3,
         'value': Decimal('134.51')}}

and

{'binance': {'QUANTUSDT': {'amount': Decimal('826'),
                         'entry': Decimal('0.1250522412206'),
                         'ticker': 'QUANTUSDT',
                         'unrealized_pnl': 1.03231002,
                         'value': Decimal('104.32546026')}},
 'hyperliquid': {'SOL': {'amount': Decimal('1.0'),
                         'entry': Decimal('134.81'),
                         'ticker': 'SOL',
                         'unrealized_pnl': -0.3,
                         'value': Decimal('134.51')}}}

respectively. Only one of each schema level is shown - obviously if more positions were held, more entries would be seen.

If we want to get the live positions object that tracks all positions, or register a handler on position change, we may do so. In particular, we can register handler on_update which passes the entire positions page (and/or) on_delta which passes the change in positions. Furthermore, the return value is the Positions object which is 'alive', so to speak, and keeps up to date with filled orders.

live_positions = await oms.positions_mirror(exc='hyperliquid',on_update=printer,on_delta=printer)
print(live_positions)#<quantpylib.standards.portfolio.Positions object at 0x12a98c8f0>

We may do the same with orders:

pprint(await oms.orders_get(exc='hyperliquid')) #HTTP
pprint(await oms.orders_get_all())

for output:

{1234: {
        'amount': Decimal('1.0'),
        'cloid': '',
        'filled_sz': Decimal('0.0'),
        'oid': '1234',
        'ord_status': 'NEW',
        'price': Decimal('100.0'),
        'ticker': 'SOL',
        'timestamp': 1726113126684
        }
}
{'hyperliquid': {1234: {'amount': Decimal('1.0'),
                        'cloid': '',
                        'filled_sz': Decimal('0.0'),
                        'oid': '1234',
                        'ord_status': 'NEW',
                        'price': Decimal('100.0'),
                        'ticker': 'SOL',
                        'timestamp': 1726113126684}}}
'binance': .... {}

Or...register handlers for orders page snapshot (and/or) just the changes. This also returns us a live Orders object:

live_orders = await oms.orders_mirror(exc='hyperliquid',on_update=printer,on_delta=printer)
print(live_orders)#<quantpylib.standards.portfolio.Orders object at 0x13252f050>

Now that we have registered some handlers for orders, and what not - let us see what the messages look like. We can make a limit order through the OMS - the parameters are the same as in Gateway usage:

cloid = oms.rand_cloid(exc='hyperliquid')
await oms.limit_order(exc='hyperliquid',ticker='SOL',amount=1,price=129.56,cloid=cloid)
await oms.limit_order(exc='binance',ticker='SOLUSDT',price_match=markets.PRICE_MATCH_QUEUE_1,amount=1)

Recall that our handlers are registered for hyperliquid, so let's see what gets printed: The on_delta handler receives two messages:

{'amount': Decimal('1'),
 'cloid': '0x272e45bf06706c3259f41079a1d48d2a',
 'exc': 'hyperliquid',
 'filled_sz': Decimal('0'),
 'last_fill_sz': Decimal('0'),
 'oid': None,
 'ord_status': 'PENDING',
 'ord_type': None,
 'price': Decimal('129.56'),
 'price_match': None,
 'reduce_only': None,
 'sl': None,
 'ticker': 'SOL',
 'tif': None,
 'timestamp': 1726154778980,
 'tp': None}
{'amount': Decimal('1.0'),
 'cloid': '0x272e45bf06706c3259f41079a1d48d2a',
 'exc': 'hyperliquid',
 'filled_sz': Decimal('0.0'),
 'last_fill_sz': Decimal('0.0'),
 'oid': '1234',
 'ord_status': 'NEW',
 'ord_type': None,
 'price': Decimal('129.56'),
 'price_match': None,
 'reduce_only': None,
 'sl': None,
 'ticker': 'SOL',
 'tif': None,
 'timestamp': 1726154781307,
 'tp': None}

The first is order-creation: a request has been sent to the exchange, but not yet acknowledged. Only the local trading agent is aware, but the order is possibly unsuccessful; hence PENDING status. This is followed by a NEW order which means the order was acknowledged successful by the exchange. The on_update handler sends this order, along with all the other open orders - which we print as a list:

[{'amount': Decimal('1.0'),
  ...
  'timestamp': 1726123317580,
  'tp': None},
 {'amount': Decimal('1.0'),
  'cloid': '0x272e45bf06706c3259f41079a1d48d2a',
  'exc': 'hyperliquid',
  'filled_sz': Decimal('0.0'),
  'last_fill_sz': Decimal('0.0'),
  'oid': '1234',
  'ord_status': 'NEW',
  'ord_type': None,
  'price': Decimal('129.56'),
  'price_match': None,
  'reduce_only': None,
  'sl': None,
  'ticker': 'SOL',
  'tif': None,
  'timestamp': 1726154781307,
  'tp': None}]

Next we submit an order cancel:

await oms.cancel_order(exc='hyperliquid',ticker='SOL',cloid=cloid) #or use oid

and this is acknowledged on_delta:

{'amount': Decimal('1.0'),
 'cloid': '0x272e45bf06706c3259f41079a1d48d2a',
 'exc': 'hyperliquid',
 'filled_sz': Decimal('0.0'),
 'last_fill_sz': Decimal('0.0'),
 'oid': '1234',
 'ord_status': 'CANCELLED',
 'ord_type': None,
 'price': Decimal('129.56'),
 'price_match': None,
 'reduce_only': None,
 'sl': None,
 'ticker': 'SOL',
 'tif': None,
 'timestamp': 1726154781307,
 'tp': None}

and the on_update prints the new list (not shown), this time without the cancelled order - since it is not on the orders page anymore (it is no longer open).

We can of course, do a market-order:

await oms.market_order(exc='hyperliquid',ticker='SOL',amount=-1)

Which also gives a on_delta, PENDING message:

{'amount': Decimal('-1'),
 'cloid': '0xa70c993f525b2e8106ffd60fd19af35e',
 'exc': 'hyperliquid',
 'filled_sz': Decimal('0'),
 'last_fill_sz': Decimal('0'),
 'oid': None,
 'ord_status': 'PENDING',
 'ord_type': None,
 'price': None,
 'price_match': None,
 'reduce_only': None,
 'sl': None,
 'ticker': 'SOL',
 'tif': None,
 'timestamp': 1726154778980,
 'tp': None}

This is filled immediately in two blocks, and the positions's on_delta message is called with each fill

{'amount': Decimal('0.19'),
 'delta': Decimal('-0.81'),
 'entry': Decimal('134.81'),
 'ticker': 'SOL'}
{'amount': Decimal('0.00'),
 'delta': Decimal('-0.19'),
 'entry': Decimal('134.79'),
 'ticker': 'SOL'}

where amount is the new signed position held after delta is filled - at the end of this market order our SOL position is closed fully, so our positions's on_update receives the positions page (no open positions):

{}

on the other hand the PENDING order created is acknowledged by exchange to NEW and then immediately FILLED on creation with on_delta triggers:

{'amount': Decimal('-1.0'),
 'cloid': '0xa70c993f525b2e8106ffd60fd19af35e',
 'exc': 'hyperliquid',
 'filled_sz': Decimal('0.0'),
 'last_fill_sz': Decimal('0.0'),
 'oid': 'yomama',
 'ord_status': 'NEW',
 'ord_type': None,
 'price': Decimal('127.69'), #hyperliquid's market order is an aggressive limit order
 'price_match': None,
 'reduce_only': None,
 'sl': None,
 'ticker': 'SOL',
 'tif': None,
 'timestamp': 1726154797325,
 'tp': None}
{'amount': Decimal('-1.0'),
 'cloid': '0xa70c993f525b2e8106ffd60fd19af35e',
 'exc': 'hyperliquid',
 'filled_sz': Decimal('1.0'),
 'last_fill_sz': Decimal('1.0'),
 'oid': 'yomama',
 'ord_status': 'FILLED',
 'ord_type': None,
 'price': Decimal('127.69'),
 'price_match': None,
 'reduce_only': None,
 'sl': None,
 'ticker': 'SOL',
 'tif': None,
 'timestamp': 1726154797325,
 'tp': None}

and on_update triggers:

[{'amount': Decimal('1.0'),
  ...
  'timestamp': 1726123317580,
  'tp': None},]

We will demonstrate a complex order supported by the OMS - let's call it hedge_order. It is quite often that we want one order to trigger another in a multi-leg trade. For instance, a triangular arbitrage, cross-exchange market making, funding arbitrage and l/s arbitrage all often use similar fixtures. A hedge order allows us to submit a maker-order, and the matching taker order is triggered with size matching that of the filled amount on the maker-leg. When lot size rounding doesn't allow for complete hedging, the remaining balance is stored and flushed with the next best available order. Let's see how we can make use of this. To get information from both exchanges, we wil add the binance handlers:

await oms.positions_mirror(exc='binance',on_update=printer,on_delta=printer)
await oms.orders_mirror(exc='binance',on_update=printer,on_delta=printer)
await asyncio.sleep(5)

and then make a hedge order

await oms.hedge_order(
    maker_order = {
        "exc": "binance",
        "ticker": "SOLUSDT",
        "amount": -3,
        "price_match": markets.PRICE_MATCH_QUEUE_5
    },
    hedge_order = {
        "exc": "hyperliquid",
        "ticker": "SOL",
    }
)

Note that an amount is not specified for the hedge-order - since we are listening for the filled sizes on binance. First - a pending order is created on binance, then acknowledged

>> orders delta:
{'amount': Decimal('-3'),
 'cloid': 'c484811d8ce145004eeb26c917013c29',
 'exc': 'binance',
 'filled_sz': Decimal('0'),
 'last_fill_sz': Decimal('0'),
 'oid': None,
 'ord_status': 'PENDING',
 'ord_type': None,
 'price': None,
 'price_match': 'QUEUE_5',
 'reduce_only': None,
 'sl': None,
 'ticker': 'SOLUSDT',
 'tif': None,
 'timestamp': 1726199235551,
 'tp': None}
{'amount': Decimal('-3'),
 'cloid': 'c484811d8ce145004eeb26c917013c29',
 'exc': 'binance',
 'filled_sz': Decimal('0'),
 'last_fill_sz': Decimal('0'),
 'oid': '68254554715',
 'ord_status': 'NEW',
 'ord_type': 'LIMIT',
 'price': Decimal('134.5460'),
 'price_match': 'QUEUE_5',
 'reduce_only': False,
 'sl': None,
 'ticker': 'SOLUSDT',
 'tif': 'GTC',
 'timestamp': 1726199241966,
 'tp': None}
>> orders snapshot:
[{'amount': Decimal('-3'),
  'cloid': 'c484811d8ce145004eeb26c917013c29',
  'exc': 'binance',
  'filled_sz': Decimal('0'),
  'last_fill_sz': Decimal('0'),
  'oid': '68254554715',
  'ord_status': 'NEW',
  'ord_type': 'LIMIT',
  'price': Decimal('134.5460'),
  'price_match': 'QUEUE_5',
  'reduce_only': False,
  'sl': None,
  'ticker': 'SOLUSDT',
  'tif': 'GTC',
  'timestamp': 1726199241966,
  'tp': None}]

It is later filled:

>> positions delta:
{'amount': Decimal('-3'),
 'delta': Decimal('-3'),
 'entry': Decimal('134.546'),
 'ticker': 'SOLUSDT'}
>> positions snapshot:
{'SOLUSDT': {'amount': Decimal('-3'),
            'entry': Decimal('134.546'),
            'ticker': 'SOLUSDT'}}

Which also shows up in the orders:

{'amount': Decimal('-3'),
 'cloid': 'c484811d8ce145004eeb26c917013c29',
 'exc': 'binance',
 'filled_sz': Decimal('3'),
 'last_fill_sz': Decimal('3'),
 'oid': '68254554715',
 'ord_status': 'FILLED',
 'ord_type': 'LIMIT',
 'price': Decimal('134.5460'),
 'price_match': 'QUEUE_5',
 'reduce_only': False,
 'sl': None,
 'ticker': 'SOLUSDT',
 'tif': 'GTC',
 'timestamp': 1726199256915,
 'tp': None}

This triggers a taker order on hyperliquid - which goes from PENDING to NEW to FILLED

{'amount': Decimal('3.0'),
 'cloid': '0xb09f965b3b58613bce2b12f15a94ad47',
 'exc': 'hyperliquid',
 'filled_sz': Decimal('0'),
 'last_fill_sz': Decimal('0'),
 'oid': None,
 'ord_status': 'PENDING',
 'ord_type': None,
 'price': None,
 'price_match': None,
 'reduce_only': None,
 'sl': None,
 'ticker': 'SOL',
 'tif': None,
 'timestamp': 1726199235551,
 'tp': None}
 {'amount': Decimal('3.0'),
 'cloid': '0xb09f965b3b58613bce2b12f15a94ad47',
 'exc': 'hyperliquid',
 'filled_sz': Decimal('0.0'),
 'last_fill_sz': Decimal('0.0'),
 'oid': '37771235449',
 'ord_status': 'NEW',
 'ord_type': None,
 'price': Decimal('141.36'),
 'price_match': None,
 'reduce_only': None,
 'sl': None,
 'ticker': 'SOL',
 'tif': None,
 'timestamp': 1726199257706,
 'tp': None}
{'amount': Decimal('3.0'),
 'cloid': '0xb09f965b3b58613bce2b12f15a94ad47',
 'exc': 'hyperliquid',
 'filled_sz': Decimal('3.0'),
 'last_fill_sz': Decimal('3.0'),
 'oid': '37771235449',
 'ord_status': 'FILLED',
 'ord_type': None,
 'price': Decimal('141.36'),
 'price_match': None,
 'reduce_only': None,
 'sl': None,
 'ticker': 'SOL',
 'tif': None,
 'timestamp': 1726199257706,
 'tp': None}

We also get the positions delta and snapshots on hyperliquid:

{'amount': Decimal('3.0'),
 'delta': Decimal('3.0'),
 'entry': Decimal('134.65'),
 'ticker': 'SOL'}

{'SOL': {'amount': Decimal('3.0'), 'entry': Decimal('134.65'), 'ticker': 'SOL'}}

Note that the orders are sent even if it is a partial-fill with matching size - we don't have to wait for the entire maker order to be complete.

Market-Making

Please refer to the market-making tutorial using the Feed and OMS object to write cross-exchange, cross environment (live, backtest and modelling) code for high-frequency data.

Limit-Order Book

You may want to use the orderbook with your own data stream. Here is an example of how you can do it (speudo-code)

from quantpylib.hft.lob import LOB 
ob = LOB(depth=depth,buffer_size=buffer_size)   

async def on_stream(msg):
    ob.update(msg['ts'],msg['b'],msg['a'],is_snapshot=True / False)

A limit order book object is only interesting with data already in it. Although we can create a LOB object directly, for demonstration - we are going to obtain it from a market data stream. Fortunately, our quantpylib.gateway.executor library has a l2_book_mirror method that returns a live orderbook object.

We will get data from hyperliquid through the gateway object:

import asyncio

from quantpylib.gateway.master import Gateway

gateway = Gateway(config_keys={"hyperliquid":{}})

async def ob_handler(ob):
    print(ob.get_mid(), ob.get_spread())
    return

async def main():

    await gateway.init_clients()
    ob_model = await gateway.executor.l2_book_mirror(
        ticker="SOL",
        depth=20,
        buffer_size=200,
        as_dict=False,
        on_update=ob_handler,
        exc='hyperliquid'
    )

    await asyncio.sleep(60)

if __name__ == "__main__":
    asyncio.run(main())

This maps to the l2_book_mirror method in quantpylib.wrappers.hyperliquid wrapper, and as_dict=False specifices we want a quantpylib.hft.lob.LOB object.

We can now use the utility functions of the orderbook class. While we are sleeping, the ob_handler prints out...

182.04500000000002 0.010000000000019327
182.04500000000002 0.010000000000019327
182.04500000000002 0.010000000000019327
182.05 0.020000000000010232
182.075 0.010000000000019327
182.075 0.010000000000019327
182.075 0.010000000000019327
182.075 0.010000000000019327

and so on. We do not have to call the LOB.update method here, since the mirror method takes care of the state of the orderbook on data stream. Let's take a look at some of the utility functions after sleeping:

    print(ob_model.get_mid(),ob_model.get_vamp(notional=3000))
    print(ob_model.get_vol(exp=True))
    print(ob_model.buffer_len())

and we get

182.085 182.06296258333333
0.018219811123247037
104

We get different figures for mid-price, fair-price estimator using quote volume imbalance, volatility and length of running buffer. Of course, these methods have additional parameters, such as the sample size used to compute volatility - we refer to documentation.

Trades

Like the LOB object, Trades is a trade buffer stream. Using the data streamed in, we may get useful information such as trade imbalance.

Filling in the trade buffer is extremely easy, let's get the hyperliquid BTC trade stream:

import asyncio

from quantpylib.hft.trades import Trades
from quantpylib.wrappers.hyperliquid import Hyperliquid

async def main():
    ticker = "BTC"

    hpl = Hyperliquid()
    await hpl.init_client()
    trades = Trades(buffer_size=1_000_000)
    async def trade_handler(trade):
        #trade is (time_ms, price, size, dir)
        #>>> (1722177301662, 67749.0, 0.0003, -1)
        trades.append(trade=trade) 

    await hpl.trades_subscribe(ticker=ticker,handler=trade_handler)
    await asyncio.sleep(1000)

That's pretty much it - from here the buffer is being populated, and we can compute figures such as trade imbalance and so on.

Statistics

This is our statistical library for hft modelling. It is designed to work seamlessly with the data structures from our internal state representations, such as the orderbook LOB and trades Trades, but will work just as well with external data.

For instance, we may be interested in computing orderbook liquidity - to do this we fit an exponential decay model for the hit and -lifted amounts against the distance to mid price. This figure is directly related to the Poisson intensity that is often taken as a model for trade arrivals.

Let's stream both the l2-book data and the trades occuring:

import asyncio
import numpy as np
import matplotlib.pyplot as plt

from quantpylib.hft.lob import LOB
from quantpylib.hft.trades import Trades
from quantpylib.hft.stats import intensity

from quantpylib.wrappers.hyperliquid import Hyperliquid

async def main():
    ticker = "BTC"

    hpl = Hyperliquid()
    await hpl.init_client()

    ob = await hpl.l2_book_mirror(
        ticker=ticker,
        depth=20,
        buffer_size=1_000_000,
        as_dict=False
    )
    trades = Trades(buffer_size=1_000_000)
    async def trade_handler(trade):
        trades.append(trade=trade)

    await hpl.trades_subscribe(ticker=ticker,handler=trade_handler)
    await asyncio.sleep(60 * 20)

    #code goes here...

if __name__ == "__main__":
    asyncio.run(main())

After twenty minutes, we get some two thousand data points and four hundred trades. Let's fit the exponential function using quantpylib.hft.stats.intensity, and make some plots:

    print(ob.buffer_len()) #2051
    print(trades.buffer_len()) #408

    params = intensity(
        lob_timestamps=ob.get_ts_buffer(),
        lob_mids=ob.get_mids_buffer(),
        trades=trades.get_buffer(),
    )   

    alpha = params.get("alpha")
    kappa = params.get("kappa")
    levels = params.get("levels")
    agg_amounts = params.get("amounts")
    fitted_values = alpha * np.exp(-kappa * levels)

    # Plot the actual and fitted
    plt.figure(figsize=(10, 6))
    plt.plot(levels, agg_amounts, 'o', label='Aggregated Amounts', color='red')
    plt.plot(levels, fitted_values, '-', label=f'Fitted Model: $A(d) = {alpha:.2f} e^{{-{kappa:.2f} d}}$', color='blue')
    plt.xlabel('Distance from Mid-Price (d)')
    plt.ylabel('Aggregated Trade Amount (A(d))')
    plt.title('Exponential Decay Model Fit to Aggregated Trade Amounts')
    plt.legend()
    plt.grid(True)
    plt.show()

Obviously, the sample size is rather small, but we will carry on for the sake of demonstration. Here is the decay function:

alt text

Values such as kappa are often used as measures of orderbook liquidity. Higher values of kappa indicate strong decay, hence greater trading near the mid-price. Lower values indicate weak decay - market order sizes often wipe out a few levels in the orderbook and have strong price impact. See Hummingbot implementation of computing trade intensity. In stoikov-avellaneda market making formula, kappa appears as term in optimal spread; see Hummingbot avellaneda_market_making.pyx:

self._optimal_spread = self.gamma * vol * time_left_fraction
self._optimal_spread += 2 * Decimal(1 + self.gamma / self._kappa).ln() / self.gamma

Optimal spread has an additive factor of log(1 + c/kappa), where smaller values of kappa encourages wider maker orders (although I am pretty sure this should be dollar-normalized first).

quantpylib.hft

Examples

Data Feed

Data Archival

OMS

Market-Making

Limit-Order Book

Trades

Statistics

hft.lob

hft.trades

hft.features

hft.stats