openbb-store 0.1.1

An OBBject extension for object storage and data interchange.

OBBject Store Extension

openbb-store is an OBBject extension for storing and retrieving OBBjects, Data, DataFrames, ExcelFiles, dictionaries, lists, and strings.

Each entry is stored as a compressed pickle, with SHA1 signature, using the LZMA module with the "xz" algorithm set to maximum compression.

The purpose of this module is to efficiently persist binary objects in-memory, and/or to-disk, in a format that is transportable over a network. An example use-case is a server-side cache for redistributing common data resources among various API endpoints and callbacks. Another is to persist frequently-accessed items like, lists and maps of ticker symbols or, static assets in a format suitable for programmatic utility.

Installation

PyPI

Install from PyPI with:

pip install openbb-store

From Source

After cloning the repo, navigate into the project's folder, create a new Python environment, then install with:

pip install -e .

Usage

Every output from the OpenBB Platform Python interface - when obb.user.preferences.output_type = "OBBject" - will have the store attribute.

Importing

Importing is only necessary when used as a standalone class, and it needs to be initialized. The init will accept a filename or path that can be used to load a previously exported object into memory.

from openbb_store.store import Store

store = Store()

Note: Functionality specific to operating as an OpenBB Platform extension - such as defaults - are not available in this mode.

Store Class

Within the OpenBB Platform, the extension acts as Global with methods to add, retrieve, and save groups of data objects to memory or file in a transportable and compressed format.

When used as standalone, the user_data_directory property (preference) should be set to the desired read/write directory upon initialization. Alternatively, specify the complete path to the file when using the IO methods' filename parameter.

Supported Data Types

The following is a list of supported data objects:

• OBBject
• Data (generic OpenBB Data class)
• DataFrame
• List
• Dictionary
• String
• ExcelFile (Pandas instance)

The contents of any object being added must be serializable.

Add Data

from openbb import obb  # If `openbb` is not installed, install it with `pip install openbb`

data = obb.equity.price.historical("NVDA", provider="yfinance", start_date="2023-01-01", end_date="2023-12-31")
data.store.add_store(data=data, name="nvda2023")

A confirmation will display unless the "verbose" property is set to False.

"Data store 'nvda2023' added successfully."

Additional data can be added to the collection, and then exported as a single package.

data = obb.equity.fundamental.metrics("NVDA", provider="yfinance")
data.store.add_store(data = data.to_df().set_index("symbol").T, name="nvdaMetrics", description="Key Valuation Metrics for NVDA.")

"Data store 'nvdaMetrics' added successfully."

Directory Of Objects

An inventory of stored objects is displayed with the 'directory' property.

data.store.directory

{'nvda2023': {'description': None,
  'data_class': 'OBBject',
  'schema_preview': "{'length': 250, 'fields_set': ['open', 'high', 'low', 'close', 'volume', 'split_..."},
 'nvdaMetrics': {'description': 'Key Valuation Metrics for NVDA.',
  'data_class': 'DataFrame',
  'schema_preview': "{'length': 34, 'width': 1, 'columns': Index(['NVDA'], dtype='object', name='symb..."}}

Schemas

Metadata related to the schema are stored independent of the actual data store. Schemas are retrieved with the get_schema method, using the assigned 'name' as the key.

Example DataFrame schema:

data.store.get_schema("nvdaMetrics")

{'length': 34,
 'width': 1,
 'columns': Index(['NVDA'], dtype='object', name='symbol'),
 'index': Index(['market_cap', 'pe_ratio', 'forward_pe', 'peg_ratio', 'peg_ratio_ttm',
        'enterprise_to_ebitda', 'earnings_growth', 'earnings_growth_quarterly',
        'revenue_per_share', 'revenue_growth', 'enterprise_to_revenue',
        'quick_ratio', 'current_ratio', 'debt_to_equity', 'gross_margin',
        'operating_margin', 'ebitda_margin', 'profit_margin',
        'return_on_assets', 'return_on_equity', 'dividend_yield',
        'dividend_yield_5y_avg', 'payout_ratio', 'book_value', 'price_to_book',
        'enterprise_value', 'overall_risk', 'audit_risk', 'board_risk',
        'compensation_risk', 'shareholder_rights_risk', 'beta',
        'price_return_1y', 'currency'],
       dtype='object'),
 'types_map': symbol
 NVDA    object
 dtype: object}

Example Pydantic model schema:

data.store.get_schema("nvda2023")

{'length': 250,
 'fields_set': ['open',
  'high',
  'low',
  'close',
  'volume',
  'split_ratio',
  'dividend'],
 'data_model': {'additionalProperties': True,
  'description': 'Yahoo Finance Equity Historical Price Data.',
  'properties': {'date': {'anyOf': [{'format': 'date', 'type': 'string'},
     {'format': 'date-time', 'type': 'string'}],
    'description': 'The date of the data.',
    'title': 'Date'},
   'open': {'description': 'The open price.',
    'title': 'Open',
    'type': 'number'},
   'high': {'description': 'The high price.',
    'title': 'High',
    'type': 'number'},
   'low': {'description': 'The low price.', 'title': 'Low', 'type': 'number'},
   'close': {'description': 'The close price.',
    'title': 'Close',
    'type': 'number'},
   'volume': {'anyOf': [{'type': 'number'},
     {'type': 'integer'},
     {'type': 'null'}],
    'default': None,
    'description': 'The trading volume.',
    'title': 'Volume'},
   'vwap': {'anyOf': [{'type': 'number'}, {'type': 'null'}],
    'default': None,
    'description': 'Volume Weighted Average Price over the period.',
    'title': 'Vwap'},
   'split_ratio': {'anyOf': [{'type': 'number'}, {'type': 'null'}],
    'default': None,
    'description': 'Ratio of the equity split, if a split occurred.',
    'title': 'Split Ratio'},
   'dividend': {'anyOf': [{'type': 'number'}, {'type': 'null'}],
    'default': None,
    'description': 'Dividend amount (split-adjusted), if a dividend was paid.',
    'title': 'Dividend'}},
  'required': ['date', 'open', 'high', 'low', 'close'],
  'title': 'YFinanceEquityHistoricalData',
  'type': 'object'},
 'created_at': '2024-06-18 13:08:44.778360',
 'uid': '06671e94-d271-7d4f-8000-43094acbb703'}

Restore Data

Restore data from the Store extension by using the get_store method. Each archive and pickled object are validated against a signature before opening.

data.store.get_store("nvdaMetrics")

	NVDA
market_cap	2933787983872.0
pe_ratio	56.15023
forward_pe	29.825434
peg_ratio	0.76
peg_ratio_ttm	1.2189
enterprise_to_ebitda	45.047
earnings_growth	1.68
earnings_growth_quarterly	1.682
revenue_per_share	3.91
revenue_growth	1.224
enterprise_to_revenue	28.619
quick_ratio	3.503
current_ratio	4.269
debt_to_equity	17.221
gross_margin	0.75975996
operating_margin	0.62057
ebitda_margin	0.6353
profit_margin	0.55041003
return_on_assets	0.55258
return_on_equity	1.23767
dividend_yield	0.0004
dividend_yield_5y_avg	0.001
payout_ratio	0.010299999
book_value	2.368
price_to_book	50.506756
enterprise_value	2756181229568
overall_risk	8.0
audit_risk	7.0
board_risk	10.0
compensation_risk	4.0
shareholder_rights_risk	6.0
beta	1.673
price_return_1y	1.7639759
currency	USD

When the stored object is an instance of OBBject, the element to retrieve can be isolated with the element parameter. By default, it is "dataframe". When set as "OBBject", the object is restored in its original form.

data.store.get_store("nvda2023", element="OBBject")

OBBject

id: 06671e94-d271-7d4f-8000-43094acbb703
results: [{'date': datetime.date(2023, 1, 3), 'open': 14.85099983215332, 'high': 14...
provider: yfinance
warnings: None
chart: None
extra: {'metadata': {'arguments': {'provider_choices': {'provider': 'yfinance'}, 's...

Exporting/Importing

Any item(s) loaded into the extension can be exported to file as a ".xz" archive. A list of "names" isolates specific objects for writing to disk. Without supplying names, all entries are exported.

data.store.save_store_to_file(filename="nvda")

Importing works the same way, and a list of "names" can also be included to load only the desired elements.

data.store.load_store_from_file(filename="nvda")

The default path can be overridden by including the complete path, beginning with "/", in the filename. Do not include the file extension with the name.

Loading Excel Files

The ability to handle Excel files requires installing the "excel" extras.

pip install openbb-store["excel"]

Once installed, files can be loaded from a local path or a URL by calling the load_from_excel method.

The ExcelFile object will be returned, in addition to being loaded into memory.

data.store.load_from_excel(
  file="https://www.spglobal.com/spdji/en/documents/additional-material/sp-500-eps-est.xlsx",
  name="sp500_eps",
  description="S&P 500 EPS estimates."
)

Data store 'sp500_eps' added successfully.

<pandas.io.excel._base.ExcelFile at 0x1262f62d0>

ExcelFile objects will have a schema consiting of a list of sheet names.

data.store.get_schema("sp500_eps")

{'sheet_names': ['ESTIMATES&PEs',
  'SECTOR EPS',
  'QUARTERLY DATA',
  'SALES',
  'BEATS AND SHARES',
  'FORWARD SCHEDULE']}

Individual sheets can be recalled via the get_store method, adding the sheet_name parameter.

data.store.get_store("sp500_eps", sheet_name="SALES", skiprows=4).iloc[:12, :4]

QUARTERLY SALES PER SHARE	% CHG Q2/Q1	% CHG Q2/Q2	2024-06-28
ENERGY	0.0675681	0.0872589	134.198
MATERIALS	0.0699618	-0.0240995	65.3953
INDUSTRIALS	0.0479355	0.0151455	114.577
CONSUMER DISCRETIONARY	0.0900365	0.0115968	162.86
CONSUMER STAPLES	0.00634981	0.0179107	139.538
HEALTH CARE	0.0416614	0.0899313	254.513
FINANCIALS	0.0223595	0.0718522	79.645
INFORMATION TECHNOLOGY	0.0259091	0.125514	120.964
COMMUNICATION SERVICES	0.0309041	0.0937237	20.3298
UTILITIES	-0.0882947	0.033352	33.5973
REAL ESTATE*	0.0143508	0.00782597	9.93927
S&P 500	0.0355043	0.0575002	488.705

Setting Default Stores

When used as an OBBject extension, a default state can be configured as a starting point for every OpenBB Platform Python session.

Adding and removing defaults works the same way as adding and removing objects.

Add To Defaults

data.store.add_to_defaults("sp500_eps")

'sp500_eps added to defaults.'

Remove From Defaults

data.store.remove_from_defaults("sp500_eps")

'sp500_eps removed from defaults.'