Skip to main content

Python package for extracting files from IFS to CDF. Copyright 2020 Cognite AS

Project description

Cognite logo

CDF IFS file extractor

build Code style: black

In progress


The cdf-ifs-file-extractor allows continuous uploading, changing and deletion of files stored in a local directory and connect those files to the corresponding assets with some metadata. Running on a continuous schedule the extractor moves old mapping_files into an Archive and uploads new files if it is fed with a new mapping file. Feedback of unsuccessful upload attempts is given via the two files unknown_files.csv and unknown_assets.csv.

Structure of the Mapping file

The extractor expects the mapping file to have a specific format. This requires the following columns:

  • DOC_NO
  • LAST_UPDATED: to be checked if this column is really included
  • OPERATION: allows one of the three values N, C, D to upload a new file or change or delete an existing one

Requirements local directory

The local directory needs to contain

  • the desired files to operate on
  • a folder with the name Archive
  • the mapping file


The parameters for the function are

  • project:
  • api_key_name: name of the environment variable storing the api key
  • base_url
  • client_name
  • client_timeout
  • log_path
  • log_level
  • path: path of the local directory
  • mapping_file: file name

Feedback of the extractor

For non developing users the extractor provides feedback to the user for unsuccessful file operation attempts in two cases

  • The asset_id specified in the mapping file does not exist: these ids are stored together with the corresponding file name in unknown_assets.csv
  • The specified file does not exist: unknown_files.csv

Both files will be created in the end of a run of the extractor and archived as soon as the extractor is running the next time.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for cdf-ifs-file-extractor, version 0.1.2
Filename, size File type Python version Upload date Hashes
Filename, size cdf_ifs_file_extractor-0.1.2-py3-none-any.whl (16.1 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size cdf-ifs-file-extractor-0.1.2.tar.gz (8.5 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page