pyetfdb_scraper: Free ETF data at your fingertips
pyetfdb_scraper is a Python library for extracting ETF data directly from ETFDB, a website providing one of the largest ETF Databases containing ETFs from a vast range of asset classes, industries, issuers, and investment styles.
Quick Start
Install with pip as a package pip. See the pip package here https://pypi.org/project/pyetfdb-scraper/.
pip install pyetfdb-scraper
from pyetfdb_scraper import etf
Example Usage
from pyetfdb_scraper.etf import ETF,load_etfs
# returns list of available ETFs.
etfs = load_etfs()
>>> ['SPY', 'IVV', 'VTI', 'VOO', 'QQQ', 'VEA', 'IEFA', ...]
Retrieve info about a single ticker
# Load ETF
ivv = ETF('IVV')
print(ivv.info)
Results
>>> {
"vitals": {
"etf_name": "iShares Core S&P 500 ETF",
"issuer": "BlackRock, Inc.",
"issuer_link": "/issuer/blackrock-inc/",
"brand": "iShares",
"brand_link": "/issuer/ishares/",
"structure": "ETF",
"structure_link": "",
"expense_ratio": "0.03%",
"hompage_link": "http://us.ishares.com/product_info/fund/overview/IVV.htm?qt=IVV",
"inception": "May 15, 2000",
"index_tracked": "S&P 500 Index",
"index_tracked_link": "/index/sp-500-index/",
},
"dbtheme": {
"category": "Large Cap Growth Equities",
"category_link": "",
"asset_class": "Equity",
"asset_class_link": "/etfs/asset-class/equity/",
"asset_class_size": "Large-Cap",
"asset_class_size_link": "/etfs/size/large-cap/",
"asset_class_style": "Blend",
"asset_class_style_link": "/etfs/style/blend/",
"general_region": "North America",
"general_region_link": "/etfs/region/north-america/",
"specific_region": "U.S.",
"specific_region_link": "/etfs/country/us/",
},
"fact_set": {
"segment": ["Equity: U.S. - Large Cap"],
"category": ["Size and Style"],
"focus": ["Large Cap"],
"niche": ["Broad-based"],
"strategy": ["Vanilla"],
"weighting_scheme": ["Market Cap"],
},
"analyst_report": "Another alternative is VOO, which is slightly cheaper and is eligible for commission free trading within Vanguard accounts. Beyond the S&P 500, RSP may be another alternative worth a closer look; that ETF, which is a bit more expensive, holds all stocks in the S&P 500 but gives an equivalent weighting to each. As such, it might be attractive to investors looking to steer clear of the potential inefficiencies in market cap weighting methodologies.",
"trade_data": {
"open": "",
"volume": "",
"day_low": "",
"day_high": "",
"52_week_low": "$376.34",
"52_week_high": "$491.10",
"aum": "$416,620.0 M",
"shares": "854.5 M",
},
"historical_trade_data": {
"1_month_avg_volume": "5,672,082",
"3_month_avg_volume": "5,182,910",
},
"alternative_etfs": [
{
"type": "Cheapest",
"ticker": "SFY",
"expense_ratio": "0.00%",
"assets": "$622.0 M",
"avg_daily_volume": "175,030",
"ytd_return": "1.89%",
},
{
"type": "Largest (AUM)",
"ticker": "SPY",
"expense_ratio": "0.09%",
"assets": "$485.9 B",
"avg_daily_volume": "79 M",
"ytd_return": "2.68%",
},
{
"type": "Most Liquid (Volume)",
"ticker": "SPY",
"expense_ratio": "0.09%",
"assets": "$485.9 B",
"avg_daily_volume": "79 M",
"ytd_return": "2.68%",
},
{
"type": "Top YTD Performer",
"ticker": "WUGI",
"expense_ratio": "0.75%",
"assets": "$25.1 M",
"avg_daily_volume": "2,590",
"ytd_return": "8.00%",
},
],
"other_alternative_etfs": [
{
"type": "Cheapest",
"ticker": "BKLC",
"expense_ratio": "0.00%",
"assets": "$2.1 B",
"avg_daily_volume": "78,895",
"ytd_return": "2.60%",
},
{
"type": "Largest (AUM)",
"ticker": "SPY",
"expense_ratio": "0.09%",
"assets": "$485.9 B",
"avg_daily_volume": "79 M",
"ytd_return": "2.68%",
},
{
"type": "Most Liquid (Volume)",
"ticker": "SPY",
"expense_ratio": "0.09%",
"assets": "$485.9 B",
"avg_daily_volume": "79 M",
"ytd_return": "2.68%",
},
{
"type": "Top YTD Performer",
"ticker": "AMOM",
"expense_ratio": "0.75%",
"assets": "$16.0 M",
"avg_daily_volume": "4,800",
"ytd_return": "7.15%",
},
],
}
print(ivv.to_dict())
Results
>>> {
"info": {
"vitals": {
"issuer": "BlackRock, Inc.",
"issuer_link": "/issuer/blackrock-inc/",
"brand": "iShares",
"brand_link": "/issuer/ishares/",
"structure": "ETF",
"structure_link": "",
"expense_ratio": "0.03%",
"hompage_link": "http://us.ishares.com/product_info/fund/overview/IVV.htm?qt=IVV",
"inception": "May 15, 2000",
"index_tracked": "S&P 500 Index",
"index_tracked_link": "/index/sp-500-index/",
"etf_name": "iShares Core S&P 500 ETF",
},
"dbtheme": {
"category": "Large Cap Growth Equities",
"category_link": "",
"asset_class": "Equity",
"asset_class_link": "/etfs/asset-class/equity/",
"asset_class_size": "Large-Cap",
"asset_class_size_link": "/etfs/size/large-cap/",
"asset_class_style": "Blend",
"asset_class_style_link": "/etfs/style/blend/",
"general_region": "North America",
"general_region_link": "/etfs/region/north-america/",
"specific_region": "U.S.",
"specific_region_link": "/etfs/country/us/",
},
"fact_set": {
"segment": ["Equity: U.S. - Large Cap"],
"category": ["Size and Style"],
"focus": ["Large Cap"],
"niche": ["Broad-based"],
"strategy": ["Vanilla"],
"weighting_scheme": ["Market Cap"],
},
"analyst_report": "Another alternative is VOO, which is slightly cheaper and is eligible for commission free trading within Vanguard accounts. Beyond the S&P 500, RSP may be another alternative worth a closer look; that ETF, which is a bit more expensive, holds all stocks in the S&P 500 but gives an equivalent weighting to each. As such, it might be attractive to investors looking to steer clear of the potential inefficiencies in market cap weighting methodologies.",
"trade_data": {
"open": "",
"volume": "",
"day_low": "",
"day_high": "",
"52_week_low": "$376.34",
"52_week_high": "$491.10",
"aum": "$416,620.0 M",
"shares": "854.5 M",
},
"historical_trade_data": {
"1_month_avg_volume": "5,672,082",
"3_month_avg_volume": "5,182,910",
},
"alternative_etfs": [
{
"type": "Cheapest",
"ticker": "SFY",
"expense_ratio": "0.00%",
"assets": "$622.0 M",
"avg_daily_volume": "175,030",
"ytd_return": "1.89%",
},
{
"type": "Largest (AUM)",
"ticker": "SPY",
"expense_ratio": "0.09%",
"assets": "$485.9 B",
"avg_daily_volume": "79 M",
"ytd_return": "2.68%",
},
{
"type": "Most Liquid (Volume)",
"ticker": "SPY",
"expense_ratio": "0.09%",
"assets": "$485.9 B",
"avg_daily_volume": "79 M",
"ytd_return": "2.68%",
},
{
"type": "Top YTD Performer",
"ticker": "WUGI",
"expense_ratio": "0.75%",
"assets": "$25.1 M",
"avg_daily_volume": "2,590",
"ytd_return": "8.00%",
},
],
"other_alternative_etfs": [
{
"type": "Cheapest",
"ticker": "BKLC",
"expense_ratio": "0.00%",
"assets": "$2.1 B",
"avg_daily_volume": "78,895",
"ytd_return": "2.60%",
},
{
"type": "Largest (AUM)",
"ticker": "SPY",
"expense_ratio": "0.09%",
"assets": "$485.9 B",
"avg_daily_volume": "79 M",
"ytd_return": "2.68%",
},
{
"type": "Most Liquid (Volume)",
"ticker": "SPY",
"expense_ratio": "0.09%",
"assets": "$485.9 B",
"avg_daily_volume": "79 M",
"ytd_return": "2.68%",
},
{
"type": "Top YTD Performer",
"ticker": "AMOM",
"expense_ratio": "0.75%",
"assets": "$16.0 M",
"avg_daily_volume": "4,800",
"ytd_return": "7.15%",
},
],
},
"expense": {
"tax_analysis": {
"max_short_term_capital_gains_rate": ["39.60%"],
"max_long_term_capital_gains_rate": ["20.00%"],
"tax_on_distributions": ["Qualified dividends"],
"distributes_k1": ["No"],
},
"expense_ratio_analysis": [
{"ivv": "0.03%"},
{"etf_database_category_average": "0.37%"},
{"factset_segment_average": "0.59%"},
],
},
"holdings": {
"top_holdings": [
{
"symbol": "AAPL",
"holding": "Apple Inc.",
"share": "7.18%",
"url": "https://etfdb.com/stock/AAPL/",
},
{
"symbol": "MSFT",
"holding": "Microsoft Corporation",
"share": "6.50%",
"url": "https://etfdb.com/stock/MSFT/",
},
{
"symbol": "AMZN",
"holding": "Amazon.com, Inc.",
"share": "3.32%",
"url": "https://etfdb.com/stock/AMZN/",
},
{
"symbol": "NVDA",
"holding": "NVIDIA Corporation",
"share": "2.95%",
"url": "https://etfdb.com/stock/NVDA/",
},
{
"symbol": "GOOGL",
"holding": "Alphabet Inc. Class A",
"share": "2.03%",
"url": "https://etfdb.com/stock/GOOGL/",
},
{
"symbol": "META",
"holding": "Meta Platforms Inc. Class A",
"share": "1.83%",
"url": "https://etfdb.com/stock/META/",
},
{
"symbol": "TSLA",
"holding": "Tesla, Inc.",
"share": "1.82%",
"url": "https://etfdb.com/stock/TSLA/",
},
{
"symbol": "GOOG",
"holding": "Alphabet Inc. Class C",
"share": "1.75%",
"url": "https://etfdb.com/stock/GOOG/",
},
{
"symbol": "BRK.B",
"holding": "Berkshire Hathaway Inc. Class B",
"share": "1.66%",
"url": "https://etfdb.com/stock/BRK.B/",
},
{
"symbol": "UNH",
"holding": "UnitedHealth Group Incorporated",
"share": "1.25%",
"url": "https://etfdb.com/stock/UNH/",
},
{
"symbol": "JPM",
"holding": "JPMorgan Chase & Co.",
"share": "1.22%",
"url": "https://etfdb.com/stock/JPM/",
},
{
"symbol": "JNJ",
"holding": "Johnson & Johnson",
"share": "1.17%",
"url": "https://etfdb.com/stock/JNJ/",
},
{
"symbol": "XOM",
"holding": "Exxon Mobil Corporation",
"share": "1.16%",
"url": "https://etfdb.com/stock/XOM/",
},
{
"symbol": "V",
"holding": "Visa Inc. Class A",
"share": "1.03%",
"url": "https://etfdb.com/stock/V/",
},
{
"symbol": "AVGO",
"holding": "Broadcom Inc.",
"share": "0.98%",
"url": "https://etfdb.com/stock/AVGO/",
},
],
"holding_comparison": [
{
"number_of_holdings": "1000",
"etf_database_category_average": "418",
"factset_segment_average": "173",
},
{
"pct_of_assets_in_top_10": "41.95%",
"etf_database_category_average": "43.40%",
"factset_segment_average": "60.51%",
},
{
"pct_of_assets_in_top_15": "51.15%",
"etf_database_category_average": "52.16%",
"factset_segment_average": "65.06%",
},
{
"pct_of_assets_in_top_50": "83.62%",
"etf_database_category_average": "81.26%",
"factset_segment_average": "81.64%",
},
],
"size_comparison": [
{
"large_(>12.9b)": "98.31%",
"etf_database_category_average": "86.63%",
"factset_segment_average": "46.30%",
},
{
"mid_(>2.7b)": "1.49%",
"etf_database_category_average": "5.88%",
"factset_segment_average": "3.26%",
},
{
"small_(>600m)": "0.00%",
"etf_database_category_average": "0.58%",
"factset_segment_average": "0.09%",
},
{
"micro_(<600m)": "0.00%",
"etf_database_category_average": "0.12%",
"factset_segment_average": "0.01%",
},
],
},
"holdings_analysis": [
{"North, Central and South America": 99.87, "Other": 0.2},
{
"United States": 96.79,
"Ireland": 1.61,
"United Kingdom": 0.66,
"Switzerland": 0.43,
"Other": 0.2,
"Netherlands": 0.14,
"Canada": 0.13,
"Bermuda": 0.11,
},
{
"Technology Services": 21.08,
"Electronic Technology": 18.45,
"Finance": 12.34,
"Health Technology": 9.51,
"Retail Trade": 7.75,
"Consumer Non-Durables": 4.48,
"Producer Manufacturing": 3.61,
"Consumer Services": 3.4,
"Energy Minerals": 3.11,
"Commercial Services": 2.92,
"Utilities": 2.25,
"Health Services": 2.21,
"Consumer Durables": 1.91,
"Process Industries": 1.78,
"Transportation": 1.76,
"Communications": 0.93,
"Distribution Services": 0.92,
"Industrial Services": 0.92,
"Non-Energy Minerals": 0.54,
"CASH": 0.2,
},
{"Large": 98.31, "Mid": 1.49, "Small": 0, "Micro": 0},
{},
{"Share/Common/Ordinary": 99.87, "CASH": 0.2},
{
"Technology Services": 21.08,
"Electronic Technology": 18.45,
"Finance": 12.34,
"Health Technology": 9.51,
"Retail Trade": 7.75,
"Consumer Non-Durables": 4.48,
"Producer Manufacturing": 3.61,
"Consumer Services": 3.4,
"Energy Minerals": 3.11,
"Commercial Services": 2.92,
"Utilities": 2.25,
"Health Services": 2.21,
"Consumer Durables": 1.91,
"Process Industries": 1.78,
"Transportation": 1.76,
"Communications": 0.93,
"Distribution Services": 0.92,
"Industrial Services": 0.92,
"Non-Energy Minerals": 0.54,
"CASH": 0.2,
},
],
"performance": [
{
"1_month_return": "3.05%",
"etf_database_category_average": "2.89%",
"factset_segment_average": None,
},
{
"3_month_return": "15.68%",
"etf_database_category_average": "16.58%",
"factset_segment_average": None,
},
{
"ytd_return": "2.66%",
"etf_database_category_average": "2.55%",
"factset_segment_average": None,
},
{
"1_year_return": "23.86%",
"etf_database_category_average": "24.93%",
"factset_segment_average": None,
},
{
"3_year_return": "10.10%",
"etf_database_category_average": "5.02%",
"factset_segment_average": None,
},
{
"5_year_return": "15.07%",
"etf_database_category_average": "8.85%",
"factset_segment_average": None,
},
],
"dividends": [
{
"dividend": "$ 1.93",
"etf_database_category_average": "$ 0.35",
"factset_segment_average": "$ 0.22",
},
{
"dividend_date": "2023-12-20",
"etf_database_category_average": "N/A",
"factset_segment_average": "N/A",
},
{
"annual_dividend_rate": "$ 6.90",
"etf_database_category_average": "$ 0.92",
"factset_segment_average": "$ 0.64",
},
{
"annual_dividend_yield": "1.41%",
"etf_database_category_average": "1.11%",
"factset_segment_average": "1.42%",
},
],
"technicals": {
"indicators": {
"20_day_ma": "$478.81",
"60_day_ma": "$461.74",
"macd_15_period": "10.67",
"macd_100_period": "40.06",
"williams_%_range_10_day": "4.05",
"williams_%_range_20_day": "3.43",
"rsi_10_day": "77",
"rsi_20_day": "71",
"rsi_30_day": "68",
"ultimate_oscillator": "65",
"lower_bollinger_(10_day)": "$471.45",
"upper_bollinger_(10_day)": "$492.45",
"lower_bollinger_(20_day)": "$467.48",
"upper_bollinger_(20_day)": "$489.54",
"lower_bollinger_(30_day)": "$465.05",
"upper_bollinger_(30_day)": "$488.05",
"support_level_1": "n/a",
"support_level_2": "$486.67",
"resistance_level_1": "n/a",
"resistance_level_2": "$492.45",
"stochastic_oscillator_%d_(1_day)": "63.66",
"stochastic_oscillator_%d_(5_day)": "86.50",
"stochastic_oscillator_%k_(1_day)": "63.23",
"stochastic_oscillator_%k_(5_day)": "80.88",
"tracking_difference_median_(%)": "-0.03",
"tracking_difference_max_upside_(%)": "-0.02",
"tracking_difference_max_downside_(%)": "-0.06",
"median_premium_discount_(%)": "0.01",
"maximum_premium_discount_(%)": "0.07",
"average_spread_(%)": "2.01",
"average_spread_($)": "2.01",
},
"volatility": {
"5_day_volatility": ["193.65%"],
"20_day_volatility": ["8.87%"],
"50_day_volatility": ["9.26%"],
"200_day_volatility": ["11.69%"],
"beta": ["1.0"],
"standard_deviation": ["25.88%"],
},
},
"realtime_rankings": [
{
"metric": "Liquidity",
"metric_realtime_rating": "A",
"a+_metric_rated_etf": "SPY",
},
{
"metric": "Expenses",
"metric_realtime_rating": "A",
"a+_metric_rated_etf": "BKLC",
},
{
"metric": "Performance",
"metric_realtime_rating": "B",
"a+_metric_rated_etf": "ESPO",
},
{
"metric": "Volatility",
"metric_realtime_rating": "B+",
"a+_metric_rated_etf": "NUSI",
},
{
"metric": "Dividend",
"metric_realtime_rating": "A-",
"a+_metric_rated_etf": "QYLD",
},
{
"metric": "Concentration",
"metric_realtime_rating": "A-",
"a+_metric_rated_etf": "VT",
},
],
}
Help Needed!
I am working full-time, and as such don't have much time to constantly push commits or updates. I will appreciate if some help can be provided, such as:
- Unit tests for the current code
- ETF Category has yet to be updated
Contributing
All contributions, bug reports, bug fixes, documentation improvements, enhancements, and ideas are welcome.
If you are simply looking to start working with the codebase, navigate to the GitHub "issues" tab and start looking through interesting issues. You can also triage issues which may include reproducing bug reports, or asking for vital information such as version numbers or reproduction instructions. To read more about contributing, you can refer to CONTRIBUTING
License
GPLv3
Disclaimer
This package is built with some reference to the existing pyetf package.
1 maintainer
Metadata
1 maintainer
Release files for pyetfdb-scraper 0.4
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| pyetfdb_scraper-0.4.tar.gz | 126.9 kB | Without provenance | Details for pyetfdb_scraper-0.4.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| pyetfdb_scraper-0.4-py3-none-any.whl | Python 3 | none | any | Details for pyetfdb_scraper-0.4-py3-none-any.whl |
Total release size: 256.2 kB
Release files / pyetfdb_scraper-0.4.tar.gz
| Download URL | pyetfdb_scraper-0.4.tar.gz |
|---|---|
| Size | 126.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
a3cc8db91af63d394766df6629d25a9dc2cacbd582690a28db8bb0bffbddd490
|
|
BLAKE2b-256 checksum How to use checksums |
f968c52cfb3511763a8bcf505fbfe27380df9996047e676e4a9da6e7dd61653a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.13
|
Release files / pyetfdb_scraper-0.4-py3-none-any.whl
| Download URL | pyetfdb_scraper-0.4-py3-none-any.whl |
|---|---|
| Size | 129.3 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
bb49586cf90463328c3ce69887a30521fcb364e0a1ac3bd82b06532d6b01e172
|
|
BLAKE2b-256 checksum How to use checksums |
4bbd84570953e5049ea0f38c04a62e5e5484020660c7f1c1d758d316424d113e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.13
|
Security for pyetfdb-scraper 0.4
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
