tradingview-scraper 0.3.0

Tradingview scraper tool

TradingView Scraper

This is a Python library for scraping ideas and indicators from TradingView.com. The goal is to develop this package to scrape anything on TradingView.com with real-time responses.
Thanks to contributors!

To-Do List

Completed Tasks

• Export as a CSV file
• Export as JSON

Pending Tasks

• Scrape Symbol Subpages:
  • Ideas
  • Indicators
  • Overview
  • News
  • Minds
  • Technical
  • Market
  • Get data using TradingView WebSocket
  • Additional suggestions welcome!

To be aware of the latest changes, go to the end of this page.

Features

• Idea page Scraping

  • Title
  • Paragraph
  • Preview Image
  • Author
  • Comments Count
  • Boosts Count
  • Publication Datetime
  • Is Updated
  • Idea Strategy

• News page Scraping

  • Breadcrumbs
  • Title
  • Published datetime
  • Related symbols
  • Body
  • Tags

• Webpage Scraping Options

  • Scrape All Pages
  • Scrape a Specific Range of Pages

• Indicator Extraction

  • Extract values for indicators like RSI, Stoch.K, etc.
  • Full list of indicators

• Export Formats

  • CSV
  • JSON

• Output Format

  • Returns data in JSON format

Here’s a revised version of the Installation section that enhances clarity and encourages user engagement:

Installation

To get started with the TradingView Scraper library, follow these simple steps:

1. Open your terminal: Launch your preferred command line interface.

2. Install the package: Run the following command to install the TradingView Scraper:

   pip install tradingview-scraper
   

3. Upgrade if necessary: If you already have the library installed and want to upgrade to the latest version, use:

   pip install --upgrade tradingview-scraper
   

Here’s a revised version of the Examples section, focusing on clarity, ease of understanding, and providing essential information about default values:

Examples

1. Fast Running (Default Parameters)

To quickly scrape ideas using default settings, use the following code:

from symbols.ideas import Ideas

# Initialize the Ideas scraper with default parameters
ideas_scraper = Ideas()  # Default: export_result=False, export_type='json'
ideas = ideas_scraper.scrape()  # Default symbol: 'BTCUSD'
print("Ideas:", ideas)

Default Parameters:

• export_result: False (no file will be saved)
• export_type: 'json' (output format)

2. Getting Ideas for a Specific Symbol, Export Type, and Pages

To scrape ideas for a specific symbol and export them as a CSV file, you can specify the parameters:

from symbols.ideas import Ideas

# Initialize the Ideas scraper with custom parameters
ideas_scraper = Ideas(
  export_result=True,  # Set to True to save the results
  export_type='csv'    # Specify the export type (json or csv)
)

# Scrape ideas for the ETHUSD symbol, from page 1 to page 2
ideas = ideas_scraper.scrape(
  symbol="ETHUSD",
  startPage=1,
  endPage=2,
  sort="popular"  #  Could be 'popupar' or 'recent'
)
print("Ideas:", ideas)

Output Format: The output will always be a list of Python dictionaries, structured as follows:

[
  {
      "title": "Bitcoin -65% crash, ETH -83%, DOGE -89%, SHIBA -90%",
      "paragraph": "Name your altcoin in the comment section, and I will do a technical analysis for you!\n\nThe crypto market looks ...",
      "preview_image": "https://s3.tradingview.com/6/6VQphWH6_mid.png",
      "author": "Xanrox",
      "comments_count": "295",
      "boosts_count": "678",
      "publication_datetime": "2024-08-18T05:55:19.000Z",
      "is_updated": "True",
      "idea_strategy": "Short"
  }
]

• When export_result=True, the default export_type is 'json', and a JSON file will be saved in the /export directory. To save as CSV, set export_type='csv'.

3. Getting Indicators Status

To scrape the status of specific indicators, use the following code:

from symbols.indicators import Indicators

# Initialize the Indicators scraper with export options
indicators_scraper = Indicators(export_result=True, export_type='json')

# Scrape indicators for the BTCUSD symbol from the BINANCE exchange
indicators = indicators_scraper.scrape(
    exchange="BINANCE",
    symbol="BTCUSD",
    indicators=["RSI", "Stoch.K"]
)
print("Indicators:", indicators)

Output:

{"RSI": "46.34926112", "Stoch.K": "40.40173723"}

4. Getting All Indicators

If you want to retrieve all available indicators for a symbol, set allIndicators=True:

# Scrape all indicators for the BTCUSD symbol
indicators_scraper = Indicators(export_result=True, export_type='json')
indicators = indicators_scraper.scrape(
    symbol="BTCUSD",
    allIndicators=True
)
print("All Indicators:", indicators)

5. Getting News Headlines/Content

# Create an instance of the NewsScraper with export options
news_scraper = NewsScraper(export_result=True, export_type='json')

# Retrieve news headlines from a specific provider
news_headlines = news_scraper.scrape_headlines(
    provider='newsbtc',  # Specify the news provider
    # symbol='BTCUSD',      # Uncomment and specify if needed
    # exchange='BINANCE', # Uncomment and specify if needed
    sort='latest'
)

# Retrieve detailed news content for a specific story
news_content = news_scraper.scrape_news_content(
    story_path=news_headlines[0]['story_path']  # Specify the story path from scraped headlines
)

• To Retrieve News by Providers:
  • Specify a provider.
  • Ensure that both symbol and exchange are left empty.
• Retrieve news by symbol:
  • Leave the provider empty.
  • Specify both symbol and exchange.

Output (news headline):

[
  {
    "breadcrumbs": "News > U.Today > Bitcoin ETFs Record Enormous Outflows",
    "title": "Bitcoin ETFs Record Enormous Outflows",
    "published_datetime": "Wed, 04 Sep 2024 07:55:38 GMT",
    "related_symbols": [
      {
        "name": "BTCUSDT",
        "logo": "https://s3-symbol-logo.tradingview.com/crypto/XTVCUSDT.svg"
      }
    ],
    "body": ["""<List of text page content>"""],
    "tags": ["Crypto", "U.Today"]}
]

Output (news content):

[
  {
    "id": "tag:reuters.com,2024:newsml_L1N3KM09S:0",
    "title": "Goldman Sachs sees biggest boost to US economy from Harris win",
    "provider": "reuters",
    "sourceLogoId": "reuters",
    "published": 1725443676,
    "source": "Reuters",
    "urgency": 2,
    "permission": "preview",
    "relatedSymbols": [
      {
        "symbol": "BITMEX:XBTETH.P",
        "currency-logoid": "country/US",
        "base-currency-logoid": "crypto/XTVCBTC"
      },
      {
        "symbol": "ICEUS:DXY",
        "logoid": "indices/u-s-dollar-index"
      }
    ],
    "storyPath": "/news/reuters.com,2024:newsml_L1N3KM09S:0-goldman-sachs-sees-biggest-boost-to-us-economy-from-harris-win/"
  }
]

Changes:

• Release 0.3.0:
  Add news scraper
• Release 0.2.9:
  Refactor for new TradingView structure
• Release 0.1.0:
  The name of ClassA changed to Ideas

License:

[MIT]