fds.sdk.StandardDatafeed 0.21.7

Standard Datafeed client library for Python

Standard Datafeed client library for Python

The Standard DataFeed (SDF) Download API provides an alternative method for users to request and retrieve SDF packages (schemas & bundles). This service is not a direct replacement and does not have 100% feature parity with the Loader Application. This API provides an alternative for users who are unable to utilize the Loader application due to following reasons:

• Inability to install 3rd party executables due to Corporate Security policies
• Inability to utilize the Loader application due to limitations or restrictions with the environment used to consume Standard Datafeed
• Clients who are utilizing existing delivery method like FTP, who may want to use a more secured & modern solution

This API allows users to retrieve

• SDF packages(excluding Quant Factor Library) they have subscriptions for, going back to August 31, 2021,
• QFL - Quant Factor Library (Factor Family & Factor Groups) packages they have subscriptions for, going back to January 01, 1995.

Additional parameters are available to filter requests to get the exact files users are looking for.

QFL data is delivered through Content API & Bulk Data API (SDF API)

• Content API : Provides direct access to FactSet-hosted QFL data. Suitable for interactive, ad hoc QFL requests. Constraints on large extracts. Costs are based on consumption, i.e. more calls can result in more costs.
• Bulk Data API : Provides access to download locations of zip files for client download. Suitable for production processes within a client environment. Cost is based on the use case and fixed unless scope changes (same as other SDFs).

This Python package is automatically generated by the OpenAPI Generator project:

• API version: 1.0.0
• Package version: 0.21.7
• Build package: org.openapitools.codegen.languages.PythonClientCodegen

Requirements

• Python >= 3.7

Installation

Poetry

poetry add fds.sdk.utils fds.sdk.StandardDatafeed

pip

pip install fds.sdk.utils fds.sdk.StandardDatafeed

Usage

1. Generate authentication credentials.
2. Setup Python environment.

   1. Install and activate python 3.7+. If you're using pyenv:

      pyenv install 3.9.7
      pyenv shell 3.9.7
      

   2. (optional) Install poetry.

3. Install dependencies.
4. Run the following:

from fds.sdk.utils.authentication import ConfidentialClient

import fds.sdk.StandardDatafeed
from fds.sdk.StandardDatafeed.api import sdf_and_qfl_content_library_api
from fds.sdk.StandardDatafeed.models import *
from dateutil.parser import parse as dateutil_parser
from pprint import pprint

# See configuration.py for a list of all supported configuration parameters.

# Examples for each supported authentication method are below,
# choose one that satisfies your use case.

# (Preferred) OAuth 2.0: FactSetOAuth2
# See https://github.com/FactSet/enterprise-sdk#oauth-20
# for information on how to create the app-config.json file
# See https://github.com/FactSet/enterprise-sdk-utils-python#authentication
# for more information on using the ConfidentialClient class
configuration = fds.sdk.StandardDatafeed.Configuration(
    fds_oauth_client=ConfidentialClient('/path/to/app-config.json')
)

# Basic authentication: FactSetApiKey
# See https://github.com/FactSet/enterprise-sdk#api-key
# for information how to create an API key
# configuration = fds.sdk.StandardDatafeed.Configuration(
#     username='USERNAME-SERIAL',
#     password='API-KEY'
# )

# Enter a context with an instance of the API client
with fds.sdk.StandardDatafeed.ApiClient(configuration) as api_client:
    # Create an instance of the API class
    api_instance = sdf_and_qfl_content_library_api.SDFAndQFLContentLibraryApi(api_client)
    # NOTE: The parameter variable defined below is just an example and may potentially contain non valid values. So please replace this with valid values.
    schema = "schema_example" # str | schema name</p> </p> **Example: acta_v1, fgp_v1, yn_v1**  QFL:- To query QFL data, please use value:   **qfl_v1**  (optional)
    # NOTE: The parameter variable defined below is just an example and may potentially contain non valid values. So please replace this with valid values.
    bundle = "bundle_example" # str | bundle name</p> If this parameter is not passed in the request, all the bundles a client is subscribed to are returned.  QFL:- - For QFL its the factor group name - If this parameter is not passed in the request, all the the factor families & factor groups subscribed to by the user are returned. - Supports comma separated list of multiple factor groups (optional)
    # NOTE: The parameter variable defined below is just an example and may potentially contain non valid values. So please replace this with valid values.
    type = "full" # str | file type = Full or Delta</p> Full files are weekly files generated every weekend (Saturday). **The date range should include weekend dates if requesting for \"Full\" files**  QFL:- In case of QFL we have - Historical files (1995 to previous year)- Always \"Full\" . A \"Full\" file is generated for each year - For YTD (Current year)- \"Full\" files are weekly files generated every weekend (Saturday). The date range should include weekend dates if requesting for \"Full\" files (optional)
    # NOTE: The parameter variable defined below is just an example and may potentially contain non valid values. So please replace this with valid values.
    start_date = "startDate_example" # str | The earliest date of the feed file the API should fetch for based on the fileTimestamp.</p>   - If schema parameter is not specified in the request, then data requested should be in batches of 10days or less**(Date range should be less than or equal to 10)** - If the schema parameter is specified in the request, date range can be greater than 10 days - If startDate and endDate parameters are not specified in the request, this endpoint returns past 10 days data by default. - startDate can be absolute- YYYY-MM-DD or relative:-  -1 for yesterday  QFL:- In case of QFL - Historical files available from 1995-01-01 to 2021-12-31. YTD files are current year files. (optional)
    # NOTE: The parameter variable defined below is just an example and may potentially contain non valid values. So please replace this with valid values.
    end_date = "endDate_example" # str | The latest date of the feed file the API should fetch for based on the fileTimestamp  If startDate and endDate parameters are not specified in the request, this endpoint returns past 10 days data by default.  endDate can be in absolute- YYYY-MM-DD or relative date:- 0 for today (optional)
    # NOTE: The parameter variable defined below is just an example and may potentially contain non valid values. So please replace this with valid values.
    pagination_limit = 20 # int | Specifies the number of results to return per page. Default is 20 & Maximum is 500 results per page (optional) if omitted the server will use the default value of 20
    # NOTE: The parameter variable defined below is just an example and may potentially contain non valid values. So please replace this with valid values.
    pagination_offset = 0 # int | Specifies the starting point for pagination. This parameter is used to identify the beginning of next set of results (optional) if omitted the server will use the default value of 0
    # NOTE: The parameter variable defined below is just an example and may potentially contain non valid values. So please replace this with valid values.
    sort = "startDate" # str | Enables to get the data in ascending or descending order based on startTime. Results are in reverse chronological order if this parameter is not used (optional) if omitted the server will use the default value of "startDate"

    try:
        # example passing only required values which don't have defaults set
        # and optional values
        api_response = api_instance.get_v1_list_files(schema=schema, bundle=bundle, type=type, start_date=start_date, end_date=end_date, pagination_limit=pagination_limit, pagination_offset=pagination_offset, sort=sort)

        pprint(api_response)
    except fds.sdk.StandardDatafeed.ApiException as e:
        print("Exception when calling SDFAndQFLContentLibraryApi->get_v1_list_files: %s\n" % e)

    # # Get response, http status code and response headers
    # try:
    #     api_response, http_status_code, response_headers = api_instance.get_v1_list_files_with_http_info(schema=schema, bundle=bundle, type=type, start_date=start_date, end_date=end_date, pagination_limit=pagination_limit, pagination_offset=pagination_offset, sort=sort)


    #     pprint(api_response)
    #     pprint(http_status_code)
    #     pprint(response_headers)
    # except fds.sdk.StandardDatafeed.ApiException as e:
    #     print("Exception when calling SDFAndQFLContentLibraryApi->get_v1_list_files: %s\n" % e)

    # # Get response asynchronous
    # try:
    #     async_result = api_instance.get_v1_list_files_async(schema=schema, bundle=bundle, type=type, start_date=start_date, end_date=end_date, pagination_limit=pagination_limit, pagination_offset=pagination_offset, sort=sort)
    #     api_response = async_result.get()


    #     pprint(api_response)
    # except fds.sdk.StandardDatafeed.ApiException as e:
    #     print("Exception when calling SDFAndQFLContentLibraryApi->get_v1_list_files: %s\n" % e)

    # # Get response, http status code and response headers asynchronous
    # try:
    #     async_result = api_instance.get_v1_list_files_with_http_info_async(schema=schema, bundle=bundle, type=type, start_date=start_date, end_date=end_date, pagination_limit=pagination_limit, pagination_offset=pagination_offset, sort=sort)
    #     api_response, http_status_code, response_headers = async_result.get()


    #     pprint(api_response)
    #     pprint(http_status_code)
    #     pprint(response_headers)
    # except fds.sdk.StandardDatafeed.ApiException as e:
    #     print("Exception when calling SDFAndQFLContentLibraryApi->get_v1_list_files: %s\n" % e)

Using Pandas

To convert an API response to a Pandas DataFrame, it is necessary to transform it first to a dictionary.

import pandas as pd

response_dict = api_response.to_dict()['data']

simple_json_response = pd.DataFrame(response_dict)
nested_json_response = pd.json_normalize(response_dict)

Debugging

The SDK uses the standard library logging module.

Setting debug to True on an instance of the Configuration class sets the log-level of related packages to DEBUG and enables additional logging in Pythons HTTP Client.

Note: This prints out sensitive information (e.g. the full request and response). Use with care.

import logging
import fds.sdk.StandardDatafeed

logging.basicConfig(level=logging.DEBUG)

configuration = fds.sdk.StandardDatafeed.Configuration(...)
configuration.debug = True

Configure a Proxy

You can pass proxy settings to the Configuration class:

• proxy: The URL of the proxy to use.
• proxy_headers: a dictionary to pass additional headers to the proxy (e.g. Proxy-Authorization).

import fds.sdk.StandardDatafeed

configuration = fds.sdk.StandardDatafeed.Configuration(
    # ...
    proxy="http://secret:password@localhost:5050",
    proxy_headers={
        "Custom-Proxy-Header": "Custom-Proxy-Header-Value"
    }
)

Custom SSL Certificate

TLS/SSL certificate verification can be configured with the following Configuration parameters:

• ssl_ca_cert: a path to the certificate to use for verification in PEM format.
• verify_ssl: setting this to False disables the verification of certificates. Disabling the verification is not recommended, but it might be useful during local development or testing.

import fds.sdk.StandardDatafeed

configuration = fds.sdk.StandardDatafeed.Configuration(
    # ...
    ssl_ca_cert='/path/to/ca.pem'
)

Documentation for API Endpoints

All URIs are relative to https://api.factset.com/bulk-documents/sdf/v1

Class	Method	HTTP request	Description
SDFAndQFLContentLibraryApi	get_v1_list_files	GET /list-files
SchemaApi	get_v1_list_schemas	GET /list-schemas	schemas

Documentation For Models

• Dataresponseitems
• DataresponseitemsRelatedBundles
• ListFiles200Response
• ListFiles200ResponseMeta
• ListFiles400Response
• ListSchema400Response
• ListSchemas
• Meta
• Pagination
• Partial
• SchemaDetails
• SchemaList

Documentation For Authorization

FactSetApiKey

• Type: HTTP basic authentication

FactSetOAuth2

• Type: OAuth
• Flow: application
• Authorization URL:
• Scopes: N/A

Notes for Large OpenAPI documents

If the OpenAPI document is large, imports in fds.sdk.StandardDatafeed.apis and fds.sdk.StandardDatafeed.models may fail with a RecursionError indicating the maximum recursion limit has been exceeded. In that case, there are a couple of solutions:

Solution 1: Use specific imports for apis and models like:

• from fds.sdk.StandardDatafeed.api.default_api import DefaultApi
• from fds.sdk.StandardDatafeed.model.pet import Pet

Solution 2: Before importing the package, adjust the maximum recursion limit as shown below:

import sys
sys.setrecursionlimit(1500)
import fds.sdk.StandardDatafeed
from fds.sdk.StandardDatafeed.apis import *
from fds.sdk.StandardDatafeed.models import *

Contributing

Please refer to the contributing guide.

Copyright

Copyright 2022 FactSet Research Systems Inc

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.