Snap Data - How hard can it be?
So, you're likely a Bloomberg customer (isn't everyone?), perhaps you have AIM, TOMS, The Bloomberg Professional Services (BPS) Terminal, maybe Data License, maybe Server API, maybe B-PIPE. There's quite a lot of functionality and data available to you. But understanding what's technically possible coupled with what's possible within the bounds of licensing, can be quite complex. One of the more frequent questions we get asked is around the snap'ing of data., and you might imagine.. how difficult could that possibly be. Well, it's a bit of a Dr Who conundrum, a temporal challenge, in terms of the space-time continuum.
Let me explain, and to make this exciting, let's pretend we have holdings of about 400,000 tickers. Simple task, at 4pm I want the current price for my holdings as it's NAV Time. I'm not interested in ticking prices, just the price. Sounds easy, but as the new Doctor... how tolerant are you of where you are in the space-time continuum. One example of that is; if the first ticker had its price snapped at 4pm, but the 400,000th ticker had its price snapped at 4:17pm, are you happy ? Undoubtedly for some, yes they are happy, for others, not so happy. They want pricing at 4pm, no sooner, no later.
You can see why I chose 400,000 vs 40, in that with smaller numbers the temporal anomaly is minor, although probably not good if you just teleported to the Planet of the Daleks.
There's basically two areas of concern; the timing anomaly, and the right to use (license). let's ask ourselves some questions.
When it's snapped will we be widely distributing the raw data? Need to think about whether we need the liberal enterprise licensing of Data License vs the stricter entitlement based controls around B-PIPE.
When the snap is scheduled, how timely does that scheduled time have to be? With Data License you can schedule the time of the execution, however, there are two (PROGRAMNAME) options; GetData which is more a best endeavours but can handle a variety of tickers and fields, and, GetSnap which is more tuned to pricing fields (eg, OPEN PRICE, HIGH PRICE, LOW PRICE, BID PRICE, MID PRICE, ASK PRICE, BID YIELD, ...) and a little more punctual on the price acquisition. However, you need to contend with the fact that an Enterprise wide license means embargoed (delayed) data only. If you want the Snap to absolutely begin at the allotted time, then it's likely you need to consider B-PIPE, the BLP API, and a custom application.
If the snap runs on time, how problematic is it that the results may cover pricing beyond the scheduled time? Again, it's about predictability, and if that's what you need then it's likely you need a custom application with B-PIPE and the BLP API.
Is the snap time know beforehand ? In some cases, the event or stimulus occurs after the snap time. For example, it might be that a request comes in at 5pm asking for the prices as of 4pm, so that means either you prep the universe first thing in the morning, on the expectation that snapped prices will be required, or, you could access the B-PIPE historical tick data service.
There's many more questions, but the set you have to ask yourself is;
Do you know what the ticker universe is beforehand
Do you know prior to the event when it needs to run
How quickly do you need the results ?
How temporally accurate to do the results have to be?
Will the raw data be (re-)distributed?
But I'm here to support Dr Who... and B-PIPE. We love accuracy, but it sometimes comes at the cost of a Tardis. The problem is not an easy one and as always it's use-case driven. However, for a taste of how difficult this can be, take a look at Patent 10657137 ("Systems and methods for aggregating, filtering, and presenting streaming data") filed by JP Morgan.
and now for something technical...
B-PIPE supports a mode called "Snapshot Template", which in effect, requests that the B-PIPE appliance holds a cache on your behalf. You prime it with the universe of tickers, and you can come back later in the day and interrogate said cache. You then just have to contend with how long it will take you to read all 400,000 ticker values out.
For those wanting to avail yourself of tick history, you could ask your friendly Bloomberg agent for access to the intraday tick history microservice (//blp/intradaytick). This might be a good solution if you were doing a retrospective snap, but not suitable for at-the-time. They also have additional services to cover off end of day closing prices (//blp/historicaldata) and time series market bars (OHLC) (//blp/intradaybar). These are all for Enterprise use. You may recognise some of the names as requests on the //blp/refdata, but those are for Terminal (display) users.
I'm in favour of using a time series database, something like AWS TimeStream, InfluxDB, or, ArcticDB. ArcticDB is interesting in that it was open sourced by Man Group, is Python friendly using dataframes, and, will be integrated into Bloomberg's BQuant platform, enabling quantitative analysts and data scientists in finance to quickly build, test, deploy, and share models for alpha generation, risk and trading. As well as the press release, you'll also be interested to note there's an article about Time Travel with ArcticDB.
Our task with Dr Who is complete.
If you find this interesting and want to discuss more options, please feel free to message us here or at Sherpa Consultancy LTD. If you've got some other ideas on solving this, we'd love to hear about it. We're here to help you get the most out of your Bloomberg investment.
