Skip to main content

Move data inbetween different Amazon flatfile formats to the correct locations.

Project description

Transfer flatfile format

Overview

Purpose:

The underlying problem, which the project tackles is that Amazon provides different flatfile formats. Either because the user chooses different elements or because amazon applies changes to an existing format. In both cases, it is impossible to simply copy and paste data in between formats, when the column headers don't match.

Strategy:

Find a sub-set of rows, that match a certain condition (don't contain any values, besides the provided SKU), pull the missing data from an external source supplied through the command line option (-o/--original). Map a fallback value for the SKU from the google sheet by searching for one inside of a plentymarkets export (this is a very specific option usable for our system). Write the data to the google-sheet in form of smaller chunks (to avoid problems occuring with uploading >10000 values at once to the API).

Installation:

  • python3 -m pip install transfer_flatfile_format --user --upgrade

  • Place the credentials file from: Google sheets tutorial into the data folder (see section: 'Usage'). (~/.transfer_flatfile_format/.credentials.json)

  • Enter the google-sheets document ID into the config.ini file. (see section 'Usage')

Usage:

There are four options:

  • --orginal / -o:
    • File location of the flatfile format, which is used as source for the values
  • --exclude / -e:
    • Comma-separated list of column names (3rd row of a flatfile), to exclude from writing to the google sheet (use case: some columns from the source contain outdated values)
  • --column / -c:
    • A column name (3rd row of a flatfile), to exclusivly transfer from the source to the google sheet
  • --adjust / -a (only in combination with --column):
    • Use the python expression defined within the config under section: [Adjust] option: 'command' to modify a value from the source flatfile before writing it to the google-sheet.
    • Example: command=(X)*2 will multiply the numbers from the column specified with --column before writing it to the gsheet.
    • These expressions are not "smart", so judge on your own if your data can be modified by a single expression.

Additionally, there is the config.ini file within:

  • ~/.transfer_flatfile_format/config.ini (on Linux)
  • C:\Users{USER}.transfer_flatfile_format (on Windows)

Which is used to specify the ID of the google sheet and optionally a data source for alternative SKUs. The alternative SKU can be used if your system maintains more than one SKU for one entity. That way you can match a product with one of two possible terms.

Example:

config.ini

[General]
google_sheet_id=1PB_XrUqy6qk......
[Match table]
with_matchtable={y|n}
sku_export={Link to csv file or location in file system}
main_sku={column_name of the column where the main SKU is located}
alt_sku={column_name of the column where the alternative SKU is located}
[Adjust]
command=(X)+5//4
Example 1: Upload all values from the source file to the google sheet, when the google sheet has an SKU but no values in 'brand_name' or 'item_name':

python3 -m transfer_flatfile_format -o /home/path/to/source_file.csv

Example 2: Upload all values from the source file at column 'example_column' to the google sheet:

python3 -m transfer_flatfile_format -o /home/path/to/source_file.csv -c example_column

Example 3: Do the same as with 'Example 1' but do not update the columns 'example_col1' & 'example_col2':

python3 -m transfer_flatfile_format -o /home/path/to/source_file.csv -e example_col1,example_col2

Example 4: Get values from a column containing integers and add 3 to them:

python3 -m transfer_flatfile_format -o /home/path/to/source_file.csv -c numeric_column -a

Config:

[General]
google_sheet_id=1PB_XrUqy6qk......
[Adjust]
command=(X)+3

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

transfer_flatfile_format-0.1.0.tar.gz (22.4 kB view details)

Uploaded Source

Built Distribution

transfer_flatfile_format-0.1.0-py3-none-any.whl (23.1 kB view details)

Uploaded Python 3

File details

Details for the file transfer_flatfile_format-0.1.0.tar.gz.

File metadata

  • Download URL: transfer_flatfile_format-0.1.0.tar.gz
  • Upload date:
  • Size: 22.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.0.10 CPython/3.7.3 Linux/5.8.0-rc6

File hashes

Hashes for transfer_flatfile_format-0.1.0.tar.gz
Algorithm Hash digest
SHA256 b0da07d260e7c42a9ffc714f2d9baf1b22eb52b49717ca60455a4f0f1e3a02bc
MD5 931e68dff13a4b1b924eb2fabe455d28
BLAKE2b-256 00bfb8d2338beeec60a1a736efa7eceb6e72117e7900604bfd6b23ee5374fd67

See more details on using hashes here.

File details

Details for the file transfer_flatfile_format-0.1.0-py3-none-any.whl.

File metadata

File hashes

Hashes for transfer_flatfile_format-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 9908dc608d1ab54d477f407d857771c78da05343d3c91e9a1d192fd0d2c07819
MD5 df511ded67aec0e25398e22d8f6a2592
BLAKE2b-256 83f7d2cf51c5318795deb84ef3a9ae435d2b8f80cce67fe1be032d37f601294d

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page