This is a pre-production deployment of Warehouse, however changes made here WILL affect the production instance of PyPI.
Latest Version Dependencies status unknown Test status unknown Test coverage unknown
Project Description

Manifest Ingest

Author: Tim Santor tsantor@xstudios.agency

Overview

Download a project manifest and its media files via SFTP or HTTP. This tool was developed for internal use on all X Studios interactive installations which need to have the ability to run “offline” when there is no active network connection.

Installation

pip install manifest-ingest

Or if you want to install a development version:

pip install git+https://bitbucket.org/xstudios/manifest-ingest.git@tag

Usage

Create a Config file

Create a config file at ~/manifest-ingest.cfg:

[default]
; API endpoint which returns JSON
api_url = http://127.0.0.1:8000/api/v1/coffeecollege/presentation/
; Auth token needed to access the API
api_token = cca017b59773a966a4ce598817bf3ffaf05b5ffd
; Regex of key names to search for eg - (filename|file|image|thumbnail)
keys = (filename|file|image|thumbnail)
; Local directory to place downloaded files
local_dir = /Users/tsantor/smucker/manifest/
; Base URL
base_url = http://127.0.0.1:8000/
; Media directory
media_dir = media
; Where to place the log file for the application
log_file = /Users/tsantor/smucker/logs/manifest-ingest.log

[sftp]
server = 192.168.1.176
service = smucker_sftp
username = USERNAME
; Remote directory on server to prepend to file URLs in manifest
remote_dir = /Users/tsantor/Projects/Smuckers/Innovation-Center-Admin/

[post_download]
; Command to execute when complete
command = /usr/local/bin/powerpoint-launch /Users/tsantor/smucker/manifest/

NOTE: The config file can be located anywhere that is readable by
the user.

SFTP

The SFTP version of Manifest Ingest is the preferred method as it is not only more secure than the other two methods, but also supports preservation of file modified times. This means that future executions of manifest-sftp will only download files which are newer than the files it already has locally.

manifest-sftp --config=~/manifest-ingest.cfg

HTTP

The HTTP version of Manifest Ingest, by nature of using HTTP, is not secure and does not support the preservation of file modified times. This means that future executions of manifest-http will download all files again.

manifest-http --config=~/manifest-ingest.cfg

Run at Startup (Mac)

Create a plist file as /Library/LaunchAgents/com.user.loginscript.plist

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
   <key>Label</key>
   <string>com.user.loginscript</string>
   <key>ProgramArguments</key>
    <array>
        <string>/usr/local/bin/manifest-sftp</string>
        <string>--config</string>
        <string>/Users/USERNAME/smucker/config/manifest-ingest.cfg</string>
    </array>
   <key>RunAtLoad</key>
   <true/>
</dict>
</plist>

Then run:

launchctl load /Library/LaunchAgents/com.user.loginscript.plist

Run at Startup (PC)

Import this settings file RunManifest.xml:

<?xml version="1.0" encoding="UTF-16"?>
<Task version="1.2" xmlns="http://schemas.microsoft.com/windows/2004/02/mit/task">
  <RegistrationInfo>
    <Date>2015-10-22T13:31:01.3773173</Date>
    <Author>Tsolu118226\TechnomediaSolutions</Author>
    <Description>Launch manifest-ingest after reboot.</Description>
  </RegistrationInfo>
  <Triggers>
    <BootTrigger>
      <Enabled>true</Enabled>
      <Delay>PT1M</Delay>
    </BootTrigger>
  </Triggers>
  <Principals>
    <Principal id="Author">
      <UserId>Tsolu118226\TechnomediaSolutions</UserId>
      <LogonType>InteractiveToken</LogonType>
      <RunLevel>LeastPrivilege</RunLevel>
    </Principal>
  </Principals>
  <Settings>
    <MultipleInstancesPolicy>IgnoreNew</MultipleInstancesPolicy>
    <DisallowStartIfOnBatteries>true</DisallowStartIfOnBatteries>
    <StopIfGoingOnBatteries>true</StopIfGoingOnBatteries>
    <AllowHardTerminate>true</AllowHardTerminate>
    <StartWhenAvailable>false</StartWhenAvailable>
    <RunOnlyIfNetworkAvailable>false</RunOnlyIfNetworkAvailable>
    <IdleSettings>
      <StopOnIdleEnd>true</StopOnIdleEnd>
      <RestartOnIdle>false</RestartOnIdle>
    </IdleSettings>
    <AllowStartOnDemand>true</AllowStartOnDemand>
    <Enabled>true</Enabled>
    <Hidden>false</Hidden>
    <RunOnlyIfIdle>false</RunOnlyIfIdle>
    <WakeToRun>false</WakeToRun>
    <ExecutionTimeLimit>P3D</ExecutionTimeLimit>
    <Priority>7</Priority>
  </Settings>
  <Actions Context="Author">
    <Exec>
      <Command>manifest-sftp</Command>
      <Arguments>--config=C:\Users\TechnomediaSolutions\smucker\config\manifest-ingest.cfg</Arguments>
    </Exec>
  </Actions>
</Task>

NOTE: Remember to change the ``Author`` and ``UserId`` nodes.

Version History

  • 0.0.1 - Initial release
  • 0.0.2 - Added saving and backup of manifest locally
  • 0.0.3 - Graceful error handling for when a remote file doesn’t exist and we attempt to download it
  • 0.0.4 - Added config option to launch command line app when download finished
  • 0.0.5 - Added alternate media URL strip method
  • 0.0.6 - Major refactor to make more configurable
  • 0.0.7 - Added single package entry point (all script entry points us same config) and Removed manifest-quickstart
  • 0.0.8 - Fixed bug where we expected a api_token in the config (not always needed)
  • 0.0.9 - Fixed bug with config setup importing from wrong package
  • 0.1.0 - Fixed bug with run_command call typo
  • 0.1.1 - Fixed url path removal from JSON and correct os paths
  • 0.1.2 - Fixed remote path bug on Windows
  • 0.1.3 - Added elapsed time to log
  • 0.2.0 - Key names are now a regex which is more flexible & powerful
  • 0.2.1 - Fixed bug where local filename would be striped of the word “media”
  • 0.2.2 - Additional logging
  • 0.2.3 - Removed some logging clutter
  • 0.2.4 - If manifest is empty (eg - {}), then we abort and do not launch post_download commands
  • 0.2.5 - Use Keychain (OSX) or Credential Manager (Windows) to get SFTP password
  • 0.2.6 - When executing manifest-sftp over SSH, Keychain/Credential Manager do not provide the password so we added the ability to manually provide a password via the -p or --password argument on the command line (use with caution)
  • 0.2.7 - Due to complications with running via command line, we removed the Keychain/Credential Manager support. You may now pass passwords via Base64 to at least thwart over the shoulder attacks. Also changed the way we execute the post download command to be non-blocking.

Issues

If you experience any issues, please create an issue on Bitbucket.

Release History

Release History

0.2.7

This version

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.2.6

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.2.5

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.2.4

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.2.3

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.2.1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.2.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.3

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.2

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.0.9

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.0.8

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.0.7

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.0.6

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.0.5

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.0.4

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

Download Files

Download Files

TODO: Brief introduction on what you do with files - including link to relevant help section.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
manifest_ingest-0.2.7-py2.7.egg (12.0 kB) Copy SHA256 Checksum SHA256 2.7 Egg May 16, 2016
manifest_ingest-0.2.7-py2.py3-none-any.whl (16.4 kB) Copy SHA256 Checksum SHA256 2.7 Wheel May 16, 2016
manifest-ingest-0.2.7.tar.gz (10.0 kB) Copy SHA256 Checksum SHA256 Source May 16, 2016

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS HPE HPE Development Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting