[WIP][DO NOT MERGE] Historical data cache #254
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Current Usage: ``` from iexfinance.stocks.cache import * prepare_cache(CacheMetadata(cache_path="store.h5", cache_type=CacheType.HDF_STORE)) get_historical_data(...) get_historical_data(...) get_historical_data(...) ``` Current limitations: - Only optimizes message count when querying newer data for the historical prices endpoint. I looked at how the `requests-cache` module handles a global variables/sessions, but it can not apply in the case of iexfinance (without refactoring) because `requests-cache` makes use of `requests`'s global session, so a global variable is used here instead. TODOs: - Discuss the approach. - Add documentation on how to use. - Document the use-cases/limitations of the historical cache. Extra changes: - Added typing to get_historical_data. This was only done because it was the original point of entry in my work and I can reverse this.
Owner
|
Initial thoughts. Going to take a more in-depth look early this week. Looks good! What are your thoughts about allowing the user to pass the cache as an argument to e.g. cache = prepare_cache(CacheMetadata(cache_path="store.h5", cache_type=CacheType.HDF_STORE))
get_historical_data("AAPL", start=start, end=end, cache=cache)and then some logic in if cache is not None:
return HistoricalReaderCache(
symbols, start=start, end=end, close_only=close_only, cache=cache **kwargs
).fetch()
else:
return HistoricalReader(
symbols, start=start, end=end, close_only=close_only, **kwargs
).fetch()
Trivial:
|
addisonlynch
requested changes
Jan 4, 2021
|
|
||
| def _execute_iex_query(self, url): | ||
| if len(self.symbols) > 1: | ||
| raise InternalError("Not supported yet") |
Owner
There was a problem hiding this comment.
This logic can go in the constructor after super
| result = result.loc[:, ["close", "volume"]] | ||
| return result | ||
|
|
||
| def _get_historical_data(self, symbol, start, end): |
Owner
There was a problem hiding this comment.
Can you use fetch in the parent class instead of this method?
| symbol, start=start, end=end, close_only=self.close_only, **self.kwargs | ||
| ).fetch() | ||
|
|
||
| def _get_historical_data_cached(self, symbol): |
Owner
There was a problem hiding this comment.
This is fetch for the subclass.
| cache_type: CacheType = CacheType.NO_CACHE | ||
|
|
||
| def prepare_cache(cache: Union[CacheMetadata, pd.HDFStore]): | ||
| global _IEXFINANCE_CACHE_ |
Owner
There was a problem hiding this comment.
See comment about global variables
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Current Usage:
Current limitations:
for the historical prices endpoint.
I looked at how the
requests-cachemodule handles a globalvariables/sessions, but it can not apply in the case
of iexfinance (without refactoring) because
requests-cachemakes use of
requests's global session, so a globalvariable is used here instead.
TODOs:
Extra changes:
Added typing to get_historical_data. This was only
done because it was the original point of entry in my
work and I can reverse this.
closes #xxxx
tests added / passed
passes
black iexfinancepasses
git diff upstream/master -u -- "*.py" | flake8 --diffadded entry to docs/source/whatsnew/vLATEST.txt