MediaCloud API Client Library
Project description
This is the source code of the python client for the MediaCloud API v2.
Usage
First sign up for an API key. Then
pip install mediacloud
Examples
To get the first 2000 the stories associated with a query and dump the output to json:
import mediacloud, json, datetime
mc = mediacloud.api.MediaCloud('MY_API_KEY')
fetch_size = 1000
stories = []
last_processed_stories_id = 0
while len(stories) < 2000:
fetched_stories = mc.storyList('( obama AND policy ) OR ( whitehouse AND policy)',
solr_filter=[ mc.publish_date_query( datetime.date(2013,1,1), datetime.date(2015,1,1)),
'media_sets_id:1'],
last_processed_stories_id=last_processed_stories_id, rows= fetch_size)
stories.extend( fetched_stories)
if len( fetched_stories) < fetch_size:
break
last_processed_stories_id = stories[-1]['processed_stories_id']
print json.dumps(stories)
Find out how many sentences in the US mainstream media that mentioned “Zimbabwe” and “president” in 2013:
import mediacloud, datetime
mc = mediacloud.api.MediaCloud('MY_API_KEY')
res = mc.sentenceCount('( zimbabwe AND president)', solr_filter=[mc.publish_date_query( datetime.date( 2013, 1, 1), datetime.date( 2014, 1, 1) ), 'media_sets_id:1' ])
print res['count'] # prints the number of sentences found
Alternatively, this query could be specified as follows
import mediacloud
mc = mediacloud.api.MediaCloud('MY_API_KEY')
mc.sentenceCount('( zimbabwe AND president)', '+publish_date:[2013-01-01T00:00:00Z TO 2014-01-01T00:00:00Z} AND +media_sets_id:1')
print res['count']
Find the most commonly used words in sentences from the US mainstream media that mentioned “Zimbabwe” and “president” in 2013:
import mediacloud, datetime
mc = mediacloud.api.MediaCloud('MY_API_KEY')
words = mc.wordCount('( zimbabwe AND president)', solr_filter=[mc.publish_date_query( datetime.date( 2013, 1, 1), datetime.date( 2014, 1, 1) ), 'media_sets_id:1' ] )
print words[0] #prints the most common word
To find out all the details about one particular story by id:
import mediacloud
mc = mediacloud.api.MediaCloud('MY_API_KEY')
story = mc.story(169440976)
print story['url'] # prints the url the story came from
To save the first 100 stories from one day to a database:
import mediacloud, datetime
mc = mediacloud.api.MediaCloud('MY_API_KEY')
db = mediacloud.storage.MongoStoryDatabase('one_day')
stories = mc.storyList(mc.publish_date_query( datetime.date (2014, 01, 01), datetime.date(2014,01,02) ), last_processed_stories_id=0,rows=100)
[db.addStory(s) for story in stories]
print db.storyCount()
Take a look at the apitest.py and storagetest.py for more detailed examples.
Development
If you are interested in adding code to this module, first clone the GitHub repository.
Testing
First run all the tests. Copy mc-client.config.template to mc-client.config and edit it. Then run python tests.py. Notice you get a mediacloud-api.log that tells you about each query it runs.
Distribution
Run python test.py to make sure all the test pass
Update the version number in mediacloud/__init__.py
Make a brief note in the version history section in the README file about the changes
Run python setup.py sdist to test out a version locally
Then run python setup.py sdist upload -r pypitest to release a test version to PyPI’s test server
Run pip install -i https://testpypi.python.org/pypi mediacloud somewhere and then use it with Python to make sure the test release works.
When you’re ready to push to pypi run python setup.py sdist upload -r pypi
Run pip install mediacloud somewhere and then try it to make sure it worked.
Version History
v2.26.0: add storyCount endpoint, cleanup some failing test cases
v2.25.0: add mediaHealth endpoint, support ap_stories_id flag in storiesList, fix controversy_dump_time_slices endpoint, remove mediaSet and Dashboard endpoints
v2.24.1: fixes tab/spaces bug
v2.24.0: adds new params to the mediaList query (searching by controversy, solr query, tags_id, etc)
v2.23.0: adds solr date generation helpers
v2.22.2: fixes the PyPI readme
v2.22.1: moves sentenceList to the admin client, preps for PyPI release
v2.22.0: adds the option to enable all_fields at the API client level (ie. for all requests)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.