This is a python API that allows you to obtain the discussion that occurs in the comments of a YouTube video as a tree structure.
Project description
YouTube Discussion Tree API
This is a python API that allows you to obtain the discussion that occurs in the comments of a YouTube video as a tree structure. It also controls the quota usage that consumes your implementation over YouTube Data API through this library, and allows you to represent and serialize the discussion tree.
Install
It is recommended to install it through pip:
pip install youtube_discussion_tree_api
Also, if you want to use it from source you will have to install manually the dependencies:
pip install -r requeriments.txt
API
Now we're going to dive into the features that the API provide and how to use them.
Generating a tree
This is the main feature of the API. The easiest way of generating a tree is:
from youtube_discussion_tree_api import YoutubeDiscussionTreeApi
api = YoutubeDiscussionTreeApi("<put your gcs api and services api key>")
tree = api.generateTree("put the videoId")
First, you create a YoutubeDiscussionTreeApi object with the API KEY of your GCP project. This object is thought as the template object that holds all the interactions with the YouTube Data API.
Then, we call the method generateTree, that give us as a response a YoutubeDiscussionTree object that holds the tree structure.
Each node of the tree is a Node object:
class Node:
id: int
author_name: str
author_id: int
text: str
like_count: int
parent_id: int
published_at : str
The root node represents the video. In the text field of the root Node, a transcription of the whole video is provided. The API can only generate trees of the videos that have manual or auto-generated English subtitles.
The method generateTree has two optional parameters that are:
- sumarization -> bool: This parameter is for applying a summarization to the transcription of the video.
- conflict_solving_algorithm -> function: This parameter is a function that does the resolution of conflicts that we face when generating the tree. Will be explained later.
We can get the list of all the nodes of our tree by doing:
tree.get_nodes()
Serializing a tree
We can serialize our tree into a xml file:
tree.serialize("outuput_file.xml")
This will construct a xml file with the following structure:
<entailment-corpus num_edges="162" num_nodes="163">
<argument-list>
...
<argument-list>
<argument-pairs>
...
<argument-pairs>
</entailment-corpus>
In argument-list component there will be all the comment nodes with some information of the Node objects. In the argument-pairs component, there will be all the edges that communicate the nodes described in the previous component.
A node will be represented as:
<arg author="Flash Man" author_id="UCeFi97LktRRtpCvi_vqEmfg" id="Ugh8N1Ch9gCr-HgCoAEC" likeCount="1145">
My dad is an expert dragon slayer. "eeer but I don't see any dragons around... " You're welcome.
</arg>
And a pair as:
<pair id="1">
<t id="Ugh8N1Ch9gCr-HgCoAEC"/>
<h id="LnX3B9oaKzw"/>
</pair>
Where h is the destination and t the origin
Representing a tree
If you want to see a representation of the tree, you can do:
tree.show()
And this will output a tree like:
LnX3B9oaKzw
├── Ugg06_f0qAVH6HgCoAEC
├── Ugg662Arr0neQXgCoAEC
│ ├── Ugg662Arr0neQXgCoAEC.8LvCxcl0tY18LvI0OyzxUW
│ ├── Ugg662Arr0neQXgCoAEC.8LvCxcl0tY18LvIRl9v4kU
│ ├── Ugg662Arr0neQXgCoAEC.8LvCxcl0tY18LvIlAkRi3e
.
.
.
Conflict Solving Algorithm
If you read the "Generating a tree" part, you may have seen that in the generateTree function there is an optional argument for a conflict solving algorithm. Well, lets explain what we define as a conflict.
We say that there is a conflict when a user wants to reply to another user's comment, but this second one has made more than one contribution previously to the comment thread. This is because YouTube doesn't store the reference to the comment of the comment thread that we want to reply. It automatically sets the parent id of the reply to the top level comment of the comment thread, instead of the id of the comment that we are replying to.
In order to solve that, in this library there is an implementation of an algorithm that solve this conflicts automatically. The algorithm it's an implementation of the tf-idf algorithm that selects a candidate from a set of candidates for the reply that we don't know to which of this candidate belongs.
In order to make this library as flexible as possible, it's open to accept a function with an implementation of a conflict solving algorithm made by you through the optional argument of generateTree.
This function will receive as parameter the reply Node and a set of Node candidates.
Search videos
Through the YoutubeDiscussionTreeAPI object, you can also request for a set of videos that are found matching a given query:
from youtube_discussion_tree_api import YoutubeDiscussionTreeApi
api = YoutubeDiscussionTreeApi("<put your gcs api and services api key>")
videos = api.search_videos("Functional Programming")
This will return a set with 5 Video objects:
class Video:
id: int
title : str
description: str
channel_name: str
channel_id: str
published_at : str
You can expand or diminish the number of results in the result set by passing another optional parameter:
from youtube_discussion_tree_api import YoutubeDiscussionTreeApi
api = YoutubeDiscussionTreeApi("<put your gcs api and services api key>")
videos = api.search_videos("Functional Programming", 50)
Quota
Additionally, this library maintains the number of quota that your implementation is consuming against the Google YouTube API service. This is because the free service is limited to a quota of 10000 daily. Each tree generation consumes 2 quota, and each video search consumes 100 quota.
A data object is serialized in a pickle file. You will see that it will appear in the path in which your script that uses the library is. It automatically resets its value day after day.
In order to get quota information, you can do:
from youtube_discussion_tree_api import YoutubeDiscussionTreeApi
api = YoutubeDiscussionTreeApi("<put your gcs api and services api key>")
api.quota_info()
This will give you a dict like:
{
"limit" : 10000
"spent" : <num-expended-quota>
}
Author Note
Hey, Quim here! :man_technologist:
So, this is it for the YouTube Discussion Tree API, hope you enjoy the package! If you have any comment(xD) on the implementation, or you want to share some features that can be added to de library, hit me up! Any kind of feedback will be pleasantly accepted! :smile:
Quim10^-12
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for youtube_discussion_tree_api-1.0.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | d323385c2cd1cb0b05ba90c0fc36ca9ad6c6c1fce58c958f65ab369e91982a05 |
|
MD5 | cae8e58d5b5e9816bc8db732c3320c18 |
|
BLAKE2b-256 | a0accf43fadb8073bd5288cd190ab57ec6239b4a77486795997c1490614d673f |
Hashes for youtube_discussion_tree_api-1.0.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9d157981ea8c6f267f173b328b583a19366083ecd1a7ed6bf97f264969cd29ad |
|
MD5 | 6661d491842725b4bb278010ee885b0f |
|
BLAKE2b-256 | 28bfbf8155ca088fdd3578471260d054f250ad95438aa010ca7cd4e78415e282 |