Skip to content

quantpylib.datapoller

quantpylib.datapoller is a quant module used for multi-asset, multi-source/vendor/exchange financial data retrieval.

The datapollers are categorized into their data-classes, with current support for:

- crypto
- currencies 
- equities
- exchange
- metadata

The datasources supported are:

- binance
- eodhd
- oanda
- hyperliquid
- yfinance

In general, a trader may have requirements for different asset-class/categories of financial data, subscriptions to multiple data sources, with each data source supporting a subset of the asset-class universe. Each data source has its own endpoints, parameters, rate-restrictions and authentication flow - the datapoller abstracts these concerns to provide a seamless interface for data retrieval while attempting to maximize throughput.

Each datapoller is derived from the base quantpylib.datapoller.base.BasePoller class and implements a set of methods, and routs the requests to the specified data source and endpoints. The supported data sources are written as wrappers on the quantpylib.wrappers module, and their supported endpoints share the same function name as in the datapollers. The different datapollers are accessible via the master data polling class, given in quantpylib.datapoller.master.Datapoller object instance. For instance, if we would like to get trade bars using the quantpylib.datapoller.currencies.Currencies.get_trade_bars, we can do so by creating a obj = DataPoller(...) instance, and calling obj.currencies.get_trade_bars(...,src="oanda"). The function call is only valid if it is supported by both the datapoller and the datasource, which we can check by their matching function signatures. In general each datasource has its own parameters to the vendor-specific endpoint, as well as different specifications (such as start time of OHLCV historical request in YYYY-MM-DD vs unix-timestamp), so the library provides a unified standard that convert between these specifications. In addition, the datapoller take in flexible arguments to support the different configurations for each external API - it would be alot easier to understand through the given examples.

Note that each of the datasource wrappers are also designed to be available for use as standalone Python-SDKs. Supported datapollers and datasources are added and updated frequently.

To understand the valid function signatures of each datapoller, we need to understand the function’s decorator-type, which is indicated @poller or @ts_poller in the documentation, and is defined in quantpylib.datapoller.utils module for polling or time-series polling respectively. The default arguments are supplied through these decorators, which also provide parameter-standardization and parameter-guessing for vendor/source-unique specifications. Source-specific arguments are documented in its corresponding data-wrapper.

We now walkthrough how to construct a valid datapolling request and construct valid parameters to our datapollers for the various datasources. Note that the various endpoints and their matching function name-signatures can be easily looked up in the search-bar (top right): alt text

We need more data sources, more data pollers and more data endpoints to support. If there is a particular functionality you would like documented or implemented, please feel free to reach out to me @ hangukquant@gmail.com or submit a Github issue. Cheers.

Walkthrough

Supposed we are interested in getting some OHLC(V) data for equities. From our quantpylib.wrappers library, we see that the datasources supported with equities data is eodhd and yfinance. Suppose we have no API access to eodhd, so we will make a request through the yfinance library. To see what endpoints are available in the equities datapoller, we can just look at the documentation page - the sidebar has a summary, which we see has a get_trade_bars method. The arguments are not specified, but it is tagged @ts_poller, which means the arguments to supply are given by decorators, documented here. The augmented arguments are ticker, start, end, periods granularity, granularity_multiplier and src. We shall provide ticker, start, periods and src to make our request, and let the rest settle to default arguments.

A dictionary containing configuration keys for different data sources we have access to is formatted as "source" : { arg1 : var1 , arg2 : var2 }, where each arg is the parameter to the wrapper object instance's constructor in quantpylib.wrappers. An additional key-value for "alias" may be specified, which is used for reference to the src parameter in the datapoller methods.

We already have the keys in our .env file in the current working directory, and we will make the following imports:

import os
import asyncio
from dotenv import load_dotenv
load_dotenv()
from datetime import datetime
from quantpylib.datapoller.master import DataPoller

config_keys = {
    "yfinance": {"alias":"yfinance"},
    "eodhd": {"eod_key":os.getenv("EOD_KEY")},
    "binance": {},
    "oanda": {
        "alias":"oanda",
        "account_id":os.getenv("OANDA_ACC"),
        "secret": os.getenv("OANDA_KEY"),
        "env":"practice",
    },
    "hyperliquid": {"alias":"hyp","key":os.getenv("HYP_PUBLIC"),"secret":os.getenv("HYP_KEY")},
}


async def main():
    start,end=datetime(2023,9,30),datetime.now()
    datapoller = DataPoller(config_keys=config_keys)
    print(datapoller.src_pollers)

    await datapoller.init_clients()

    #...code goes between here...

    await datapoller.cleanup_clients()
    print("cleanup")

if __name__ == "__main__":
    asyncio.run(main())
We only need secret keys for private endpoints, and only eodhd, oanda keys are required for the demonstration here. The rest allow retrieval of data through their public endpoints. We can of course omit it all together, if we are not interested in having access to its data archives. We just instantiated a master datapoller with our keys. We used the alias hyp for hyperliquid in our datapoller initiation. We may access the equities get_trade_bars method and specify src="yfinance" alias, like this:
df = await datapoller.equities.get_trade_bars(
    ticker="AAPL",
    start=datetime(2000,1,1),
    periods=365,
    src="yfinance"
)
print(df)
This gives us a year-worth of data for the year 2000~2001:
                               open      high       low     close     volume
datetime                                                                    
2000-01-03 00:00:00-05:00  0.792742  0.850379  0.768648  0.846127  535796800
2000-01-04 00:00:00-05:00  0.818254  0.836206  0.764869  0.774790  512377600
2000-01-05 00:00:00-05:00  0.784238  0.835733  0.778569  0.786128  778321600
2000-01-06 00:00:00-05:00  0.802191  0.808805  0.718098  0.718098  767972800
2000-01-07 00:00:00-05:00  0.729436  0.763452  0.721878  0.752113  460734400
...                             ...       ...       ...       ...        ...
2000-12-22 00:00:00-05:00  0.213539  0.226768  0.213539  0.226768  318052000
2000-12-26 00:00:00-05:00  0.224878  0.226768  0.215429  0.222044  216815200
2000-12-27 00:00:00-05:00  0.216846  0.223933  0.214484  0.223933  325466400
2000-12-28 00:00:00-05:00  0.217319  0.225823  0.216374  0.223933  305177600
2000-12-29 00:00:00-05:00  0.222044  0.226768  0.219209  0.224878  630336000
Now if we were to specify end=datetime(2000,1,1) instead of start=datetime(2000,1,1), then we will have data from 1999~2000. Or we can just specify the start,end directly, which means we can exclude the periods parameter, like this:
df = await datapoller.equities.get_trade_bars(
    ticker="AAPL",
    start=datetime(2000,1,1),
    end=datetime(2020,1,1),
    src="yfinance"
)
print(df)
to get
                                open       high        low      close     volume
datetime                                                                        
2000-01-03 00:00:00-05:00   0.792742   0.850379   0.768648   0.846127  535796800
2000-01-04 00:00:00-05:00   0.818254   0.836206   0.764869   0.774790  512377600
2000-01-05 00:00:00-05:00   0.784238   0.835734   0.778569   0.786128  778321600
2000-01-06 00:00:00-05:00   0.802191   0.808805   0.718097   0.718097  767972800
2000-01-07 00:00:00-05:00   0.729436   0.763452   0.721878   0.752113  460734400
...                              ...        ...        ...        ...        ...
2019-12-24 00:00:00-05:00  69.250147  69.298799  68.819601  69.147980   48478800
2019-12-26 00:00:00-05:00  69.281768  70.536927  69.252580  70.519897   93121200
2019-12-27 00:00:00-05:00  70.814237  71.507495  70.084495  70.493149  146266000
2019-12-30 00:00:00-05:00  70.410455  71.196148  69.379088  70.911545  144114400
2019-12-31 00:00:00-05:00  70.524783  71.436962  70.425051  71.429665  100805600
Under the hood, we are mapping the parameters to the quantpylib.wrappers.yfinance.YFinance.get_trade_bars method. You may use this as a standalone SDK, but the parameters are alot more rigid. With the master datapoller, we can provide any 2 of the 3 of start, end, and periods. But yfinance does not support too many endpoints, and is also not stable, albeit free. Suppose we have an API key for eodhd, we don't need to change much code:
df = await datapoller.equities.get_trade_bars(
    ticker="AAPL",
    start=datetime(2000,1,1),
    end=datetime(2020,1,1),
    src="eodhd"
)
print(df)
                open      high       low     close  adjusted_close     volume
datetime                                                                     
2000-01-03  104.8768  112.5040  101.6848  111.9328          0.8461  535796800
2000-01-04  108.2480  110.6224  101.1920  102.5024          0.7748  512377600
2000-01-05  103.7456  110.5664  102.9952  104.0032          0.7862  778321600
2000-01-06  106.1200  107.0048   94.9984   94.9984          0.7181  767972800
2000-01-07   96.4992  101.0016   95.5024   99.5008          0.7521  460734400
...              ...       ...       ...       ...             ...        ...
2024-03-28  171.7500  172.2300  170.5100  171.4800        171.4800   65672700
2024-04-01  171.1900  171.2500  169.4800  170.0300        170.0300   46240500
2024-04-02  169.0800  169.3400  168.2300  168.8400        168.8400   49329500
2024-04-03  168.7900  170.6800  168.5800  169.6500        169.6500   47602100
2024-04-04  170.2900  171.9200  168.8200  168.8200        168.8200   53289969
Actually we don't even need src="eodhd", the equities datapoller has that as its default, and the @ts_poller decorator is capable of guessing some parameters. This of course routs the request to quantpylib.wrappers.eodhd.Eodhd.get_trade_bars. Different from the yfinance wrapper's get_trade_bars endpoint, there is an additional parameter exchange="US", that is specific to the eodhd REST API. They have parameters to identify which asset with the same ticker the client is referring to, but this nomenclature is unique to this particular vendor. This parameter is actually also guessed by the @ts_poller decorator, but if we know precisely what we want, we should pass it in as arguments, as we may guess wrongly - here is an example of AAPL on the US and Mexican exchanges:
df = await datapoller.equities.get_trade_bars(
    ticker="AAPL",
    start=datetime(2000,1,1),
    end=datetime(2020,1,1),
    src="eodhd",
    exchange="US"
)
print(df)
df = await datapoller.equities.get_trade_bars(
    ticker="AAPL",
    start=datetime(2000,1,1),
    end=datetime(2020,1,1),
    src="eodhd",
    exchange="MX"
)
print(df)
giving
                open      high       low     close  adjusted_close     volume
datetime                                                                     
2000-01-03  104.8768  112.5040  101.6848  111.9328          0.8461  535796800
2000-01-04  108.2480  110.6224  101.1920  102.5024          0.7748  512377600
2000-01-05  103.7456  110.5664  102.9952  104.0032          0.7862  778321600
2000-01-06  106.1200  107.0048   94.9984   94.9984          0.7181  767972800
2000-01-07   96.4992  101.0016   95.5024   99.5008          0.7521  460734400
...              ...       ...       ...       ...             ...        ...
2024-03-28  171.7500  172.2300  170.5100  171.4800        171.4800   65672700
2024-04-01  171.1900  171.2500  169.4800  170.0300        170.0300   46240500
2024-04-02  169.0800  169.3400  168.2300  168.8400        168.8400   49329500
2024-04-03  168.7900  170.6800  168.5800  169.6500        169.6500   47602100
2024-04-04  170.2900  171.9200  168.8200  168.8200        168.8200   53289969

[6102 rows x 6 columns]
                 open       high        low      close  adjusted_close   volume
datetime                                                                       
2004-12-20   704.4184   704.4184   694.5484   697.9980         11.0377   722401
2004-12-21   702.4780   704.4996   702.4780   703.8080         11.1296  1804322
2004-12-27   705.1688   705.1688   705.1688   705.1688         11.1511   140000
2005-01-05   738.7380   738.7380   738.7380   738.7380         11.6819    14560
2005-03-07   471.2496   471.2496   471.2496   471.2496         14.9041   266000
...               ...        ...        ...        ...             ...      ...
2024-03-27  2846.0000  2874.9900  2842.0000  2874.4299       2874.4299     1941
2024-04-01  2873.0000  2873.0000  2819.3500  2827.2100       2827.2100    16860
2024-04-02  2808.0000  2813.6900  2792.5000  2799.7800       2799.7800    23005
2024-04-03  2798.0000  2829.9900  2798.0000  2810.0000       2810.0000     7258
2024-04-04  2810.0000  2839.9900  2800.0000  2807.0300       2807.0300     3685
The best is no-guessing, where you specify the source-specific parameters explicitly. For example, this would raise Exception:
df = await datapoller.equities.get_trade_bars(
    ticker="BTC-USD",
    start=datetime(2000,1,1),
    end=datetime(2020,1,1),
    src="eodhd"
)
but if you change the request using crypto poller
df = await datapoller.crypto.get_trade_bars(
    ticker="BTC-USD",
    start=datetime(2000,1,1),
    end=datetime(2020,1,1),
    src="eodhd"
)
print(df)
we get
                    open          high           low         close  adjusted_close       volume
datetime                                                                                       
2010-07-17      0.049510      0.049510      0.049510      0.049510        0.049510            0
2010-07-18      0.049510      0.049510      0.049510      0.049510        0.049510            0
2010-07-19      0.085840      0.085840      0.085840      0.085840        0.085840            0
2010-07-20      0.080800      0.080800      0.080800      0.080800        0.080800            0
2010-07-21      0.074740      0.074740      0.074740      0.074740        0.074740            0
...                  ...           ...           ...           ...             ...          ...
2024-03-31  69647.779030  71377.779498  69624.868677  71333.647926    71333.647926  20050941373
2024-04-01  71333.484717  71342.091454  68110.696020  69702.146113    69702.146113  34873527352
2024-04-02  69705.024322  69708.381258  64586.594304  65446.974233    65446.974233  50705240709
2024-04-03  65446.671764  66914.322564  64559.899948  65980.808650    65980.808650  34488018367
2024-04-04  65975.696667  69291.254806  65113.796534  68508.841844    68508.841844  34439527442
Although the @ts_poller adds a layer of complexity, it's purpose is to both be extremely flexible in the arguments the methods takes in, to generalize to arbitrary data-sources, while still providing the simplest possible unified-interface at the user-level by routing to the correct wrapper's endpoint. In fact, if we knew how they were routed, some 'wrong' calls actually work:
df = await datapoller.equities.get_trade_bars(
    ticker="BTC-USD",
    start=datetime(2000,1,1),
    end=datetime(2020,1,1),
    src="eodhd",
    exchange="CC"
)
this works because even though we are using the equities datapoller, we specified the CC exchange which maps to the crypto asset class in the eodhd vendor. Of course, we do not recommend such workarounds. Note that additional arguments to otherwise valid requests are simply ignored - this request is valid:
df = await datapoller.crypto.get_trade_bars(
    ticker="BTC-USD",
    start=datetime(2000,1,1),
    end=datetime(2020,1,1),
    src="eodhd",
    gibberish="gibberish"
)
Let's make a request to a @poller method, such as the quantpylib.datapoller.equities.get_ticker_fundamentals. The @poller only asks for the src parameter, while get_ticker_fundamentals asks for a ticker parameter, so we can do
df=datapoller.equities.get_ticker_fundamentals(ticker="AAPL",src="eodhd")
that routs to the eodhd wrapper. Everything else is the same.

Safe-Throttling

As far as possible, when the specified endpoint specifies rate-limits, rapid submission of requests are sent-through to the data-vendor as quickly as possible, while respecting the rate-limits. This is done by using our custom semaphore-like logic, defined in the quantpylib.throttler.rate_semaphore. This is specific to each data-vendor, and each instance of the wrapper around our data-source has its own resource-pool. This design is to maximise throughput of user-requests, and is supported in both asynchronous and synchronous methods. All requests submitted to a specific datasource with throttle-support will be paced, with support for multi-threading or coroutines. For instance, suppose we do

import threading
import requests
try:
    def raw():
        response=requests.get(f"https://eodhd.com/api/fundamentals/AAPL.US?api_token={os.getenv('EOD_KEY')}&fmt=json")
        if response.status_code == 429: raise Exception("TooManyTooFast")
        else: print(response.status_code)
    req=lambda:datapoller.equities.get_ticker_fundamentals(ticker="AAPL")
    threads=[threading.Thread(target=raw,args=()) for i in range(500)]
    # req=lambda:datapoller.equities.get_ticker_fundamentals(ticker="AAPL")
    # threads=[threading.Thread(target=req,args=()) for i in range(500)]
    for thread in threads: thread.start()
    for thread in threads: thread.join()
except Exception as err:
    print(err)
The raw, multi-threaded request will eventually get rejected by the server with a 429 TooManyRequests code, while if we try:
# req=lambda:datapoller.equities.get_ticker_fundamentals(ticker="AAPL")
# threads=[threading.Thread(target=raw,args=()) for i in range(500)]
req=lambda:datapoller.equities.get_ticker_fundamentals(ticker="AAPL")
threads=[threading.Thread(target=req,args=()) for i in range(500)]
the application will gracefully pace the execution of multiple threads and not crash.

Examples

A number of examples are given. This would be the setup:

import os
import asyncio
from dotenv import load_dotenv
load_dotenv()
from datetime import datetime
from quantpylib.datapoller.master import DataPoller

config_keys = {
    "yfinance": {"alias":"yfinance"},
    "eodhd": {"eod_key":os.getenv("EOD_KEY")},
    "binance": {},
    "oanda": {
        "alias":"oanda",
        "account_id":os.getenv("OANDA_ACC"),
        "secret": os.getenv("OANDA_KEY"),
        "env":"practice",
    },
    "hyperliquid": {"alias":"hyp","key":os.getenv("HYP_PUBLIC"),"secret":os.getenv("HYP_KEY")},
}

async def main():
    start,end=datetime(2023,9,30),datetime.now()
    datapoller = DataPoller(config_keys=config_keys)
    print(datapoller.src_pollers)

    await datapoller.init_clients()

    #...code goes between here...

    await datapoller.cleanup_clients()
    print("cleanup")

if __name__ == "__main__":
    asyncio.run(main())

Get OHLCV

Let's get OHLCV for crypto, currencies and equities. We get AAPL, EUR_USD, BTCUSDT OHLCV data from the equities, currencies and crypto datapoller using source eodhd, oanda and binance respectively. The same contract may trade under different tickers, as we see in hyperliquid - we need to match to the correct symbol.

df1=await datapoller.equities.get_trade_bars(ticker="AAPL",periods=500,end=end,granularity="d",src="eodhd")
df2=await datapoller.currencies.get_trade_bars(ticker="EUR_USD",start=start,end=end,granularity="h",src="oanda")
df3=await datapoller.crypto.get_trade_bars(ticker="BTCUSDT",start=start,end=end,granularity="d",src="binance")
df4=await datapoller.crypto.get_trade_bars(ticker="BTC",start=start,end=end,granularity="d",src="hyp")
print(df1,df2,df3,df4)
We can go for other data-granularities and multipliers:
df=await datapoller.currencies.get_trade_bars(ticker="EUR_USD",start=start,end=end,granularity="h",granularity_multiplier=4,src="oanda")
print(df)
and we get 4-hour candles:
                              open     high      low    close  volume
datetime                                                             
2020-01-01 22:00:00+00:00  1.12124  1.12246  1.12124  1.12209     673
2020-01-02 02:00:00+00:00  1.12206  1.12247  1.12010  1.12044     700
2020-01-02 06:00:00+00:00  1.12044  1.12140  1.12013  1.12032    2392
2020-01-02 10:00:00+00:00  1.12034  1.12039  1.11830  1.11966    2610
2020-01-02 14:00:00+00:00  1.11962  1.12030  1.11636  1.11700    4887
...                            ...      ...      ...      ...     ...
2024-04-04 17:00:00+00:00  1.08587  1.08620  1.08319  1.08374   13217
2024-04-04 21:00:00+00:00  1.08401  1.08441  1.08350  1.08416    9891
2024-04-05 01:00:00+00:00  1.08416  1.08431  1.08234  1.08264    7608
2024-04-05 05:00:00+00:00  1.08263  1.08464  1.08226  1.08376    9345
2024-04-05 09:00:00+00:00  1.08374  `1.08422  1.08350  1.08393    1436
The datapoller routs to the oanda wrapper, and because Oanda places a limit of candles per request, given your polling-period and granularity desired, there may be multiple requests to the Oanda REST API server to stitch together the non-overlapping periods. This is the advantage of our datapoller and wrapper interface - the user does not have to concern themselves with the source-specific limitations.

Get Tick Historical

Some endpoints support historical data. eodhd has support for tick data on the US-listed common-stocks.

ticks=datapoller.equities.get_trade_ticks(ticker="META",start=datetime(2023,4,3,0,0,0), end=datetime(2023,4,3,12,0,0))
print(ticks)
We get:
     ex mkt   price      seq  shares    sl sub_mkt             ts
0     Q   K  211.00    77001       5  @ TI          1680508800013
1     Q   K  211.42    77002       1  @ TI          1680508800013
2     Q   K  210.99    77124      10  @ TI          1680508800017
3     Q   K  211.42    77125       7  @ TI          1680508800017
4     Q   K  211.42    77127      30  @ TI          1680508800017
...  ..  ..     ...      ...     ...   ...     ...            ...
2071  Q   Q  210.00  3635409      59  @FTI          1680523180411
2072  Q   Q  210.00  3635410      41  @FTI          1680523180411
2073  Q   K  210.00  3636017     800  @FT           1680523181713
2074  Q   Q  210.00  3636650      20  @FTI          1680523184200
2075  Q   K  210.00  3637810      27  @FTI          1680523185877

[2076 rows x 8 columns]

Get Fundamentals

Some assets and datasources support fundamental data. We could get fundamental data for Goldman Sachs and Ethereum like this:

f1=datapoller.equities.get_ticker_fundamentals(ticker="GS",src="eodhd")
f2=datapoller.crypto.get_ticker_fundamentals(ticker="ETH-USD",src="eodhd")

Get Exchange Universe

We might also be interested in getting the listed set of tickers for a data vendor or exchange. In the case of eodhd, a data vendor, they support multiple exchanges. We can type in the search-bar (top right) for the endpoint get_tickers_in_exchange, and we see binance, eodhd and oanda support this endpoint through the exchange datapoller. We can see that the eodhd endpoint takes in an exchange parameter, so we can ask for their US-listed stocks as follows:

tickers = datapoller.exchange.get_tickers_in_exchange(exchange="US",src="eodhd")
On the other hand Binance is already an exchange, so looking at their endpoint, we see that no additional argument is required, so we can do the route by (same for oanda)
tickers = datapoller.exchange.get_tickers_in_exchange(src="binance")
tickers = datapoller.exchange.get_tickers_in_exchange(src="oanda")
Note that the return type from these endpoints are mostly not-standardized, and no have fixed schema, except for the get_trade_bars and common methods. This is a point for improvement that we are hoping contributors can come in, as we need the manpower!

Queries

Some data vendors provide a functionality to input some string or alternatively formatted specification into their search engine. This is provided in the quantpylib.datapoller.metadata.query_engine method:

query = datapoller.metadata.query_engine(query="AAPL")
print(query)
gives us
      Code Exchange                                               Name          Type  ... Currency          ISIN previousClose  previousCloseDate
0     AAPL       US                                          Apple Inc  Common Stock  ...      USD  US0378331005      168.8200         2024-04-04
1     AAPL       BA                                      Apple Inc DRC  Common Stock  ...      ARS  US0378331005     8925.5000         2024-04-04
2     AAPL       MX                                          Apple Inc  Common Stock  ...      MXN  US0378331005     2807.0300         2024-04-04
3     AAPL      NEO                                      Apple Inc CDR  Common Stock  ...      CAD  CA03785Y1007       25.0400         2024-04-04
4     AAPL       SN                                          Apple Inc  Common Stock  ...      USD  US0378331005      170.5200         2024-04-04
5   AAPL34       SA                                          Apple Inc  Common Stock  ...      BRL  BRAAPLBDR004       42.8200         2024-04-04
6     AAPU       US  Direxion Shares ETF Trust - Direxion Daily AAP...           ETF  ...      USD  US25461A8743       21.9100         2024-04-04
7     AAPD       US  Direxion Shares ETF Trust - Direxion Daily AAP...           ETF  ...      USD  US25461A3041       23.0400         2024-04-04
8     APLY       US           YieldMax AAPL Option Income Strategy ETF           ETF  ...      USD  US88634T8577       16.5600         2024-04-04
9     3SAP       PA                                   Granite -3x AAPL           ETF  ...      EUR  XS2193970030       26.9500         2024-04-04
10    APLY      NEO              APPLE (AAPL) Yield Shares Purpose ETF           ETF  ...      CAD          None       22.7000         2024-04-04
11    3LAP       PA                                   Granite +3x AAPL           ETF  ...      EUR  XS2193969883       18.4260         2024-04-04
12    SALE      LSE  Leverage Shares 3x Short Apple (AAPL) ETP Secu...           ETF  ...      EUR  XS2472334742        2.9949         2024-04-04
13    3SAA    XETRA               Leverage Shares -3x Short Apple AAPL           ETF  ...      EUR  XS2472334742        2.9892         2024-04-04
14    AAPY       US       Kurv Yield Premium Strategy Apple (AAPL) ETF           ETF  ...      USD          None       23.3839         2024-04-04

Streaming Data

Datapoller allows for market data streaming using socket connections when made available by the endpoints of the data source. We can see that both eodhd and oanda support subscription to L1-Book/BBO data, as does the equities,currencies data class. We can make request:

async def handler(msg):
    print(msg)

await datapoller.currencies.l1_book_subscribe(ticker="EURUSD",handler=handler,src="eodhd")
await datapoller.currencies.l1_book_subscribe(ticker="EUR_USD",handler=handler,src="oanda")
await datapoller.equities.l1_book_subscribe(ticker="AAPL",handler=handler,src="eodhd")
Some data sources such as binance and hyperliquid allow for L2-Book data subscriptions. We let it stream for 20 seconds.
await datapoller.crypto.l2_book_subscribe(ticker="BTCUSDT",handler=handler,src="binance")
await datapoller.crypto.l2_book_subscribe(ticker="BTC",handler=handler,src="hyp")
await asyncio.sleep(20)
As the messages come through the subscription channels, our handler prints them out to console:
...
{'ts': 1716111203621, 'b': [[67035.0, 2.44344], [67034.0, 0.1216], ...], 'a': [[67036.0, 0.00303], ...]}
...
Of course, you can stream the same assets from different data sources, different assets from the same data source, and so on simultaneously...

Once we are done, we can close the streams:

await asyncio.gather(
    *[
        datapoller.currencies.l1_book_unsubscribe(ticker="EURUSD",src="eodhd"),
        datapoller.currencies.l1_book_unsubscribe(ticker="EUR_USD",src="oanda"),
        datapoller.equities.l1_book_unsubscribe(ticker="AAPL",src="eodhd"),
        datapoller.crypto.l2_book_unsubscribe(ticker="BTCUSDT",src="binance")
    ]
)
print("closed all but one stream")