sites Module

YouTubeChatDownloader

class chat_downloader.sites.YouTubeChatDownloader(**kwargs)

Bases: chat_downloader.sites.common.BaseChatDownloader

__init__(**kwargs)

Initialise a session with various parameters

Raises

CookieError – if unable to read or parse the cookie file

generate_urls(**kwargs)

This method should be implemented in a subclass and should return a generator which yields URLs for testing.

Raises

NotImplementedError – if not implemented and called from a subclass

get_chat_by_channel_id(channel_id, params)
get_chat_by_clip_id(clip_id, params)
get_chat_by_custom_username(custom_username, params)
get_chat_by_user_id(user_id, params)

Such as NASAtelevision in https://www.youtube.com/user/NASAtelevision

Parameters

user_id ([type]) –

get_chat_by_video_id(video_id, params)

Get chat messages for a YouTube video, given its ID.

Parameters

video_id (str) – YouTube video ID

Returns

Chat object for the corresponding YouTube video

Return type

Chat

get_playlist_items(playlist_url, params=None)
get_user_videos(channel_id=None, user_id=None, custom_username=None, video_status='all', params=None)

Retrieve all videos listed on the user’s channel

If more than one of channel_id, user_id and custom_username are specifed, the first one specified will be returned.

Parameters
  • channel_id (str, optional) – The user’s channel ID, defaults to None. (e.g., https://www.youtube.com/channel/<channel_id>)

  • user_id (str, optional) – The user’s ID, defaults to None (e.g., https://www.youtube.com/user/<user_id>)

  • custom_username (str, optional) – [description], defaults to None (e.g., https://www.youtube.com/c/<custom_username>)

  • video_status (str, optional) – Determines which videos will be retrieved, defaults to ‘all’. Must be one of ‘all’, ‘live’, ‘upcoming’ or ‘past’.

  • params (dict, optional) – Additional program parameters, defaults to None

Raises
  • ValueError – If no user is specified or an invalid video_status is specified

  • UserNotFound – If the user cannot be found

  • NoVideos – If the channel has no videos

Yield

The next video

Return type

dict

get_video_data(video_id, params=None)

TwitchChatDownloader

class chat_downloader.sites.TwitchChatDownloader(**kwargs)

Bases: chat_downloader.sites.common.BaseChatDownloader

__init__(**kwargs)

Initialise a session with various parameters

Raises

CookieError – if unable to read or parse the cookie file

generate_urls(livestream_limit, vod_limit, clip_limit, **kwargs)

This method should be implemented in a subclass and should return a generator which yields URLs for testing.

Raises

NotImplementedError – if not implemented and called from a subclass

get_chat_by_clip_id(clip_id, params)
get_chat_by_stream_id(stream_id, params)
get_chat_by_vod_id(vod_id, params)
get_top_livestreams(limit=30)
get_user_clips(username, limit=100, filter_by='LAST_WEEK')
get_user_videos(username, limit=None, video_type=None, sort='TIME')

FacebookChatDownloader

BaseChatDownloader

class chat_downloader.sites.BaseChatDownloader(**kwargs)

Bases: object

Base class for chat downloaders. Each supported site should have its own chat downloader. Subclasses should redefine the _VALID_URLS dictionary which creates a mapping between functions and their matching regular expressions. Optionally, subclasses should also redefine _NAME, _SITE_DEFAULT_PARAMS and _TESTS fields.

__init__(**kwargs)

Initialise a session with various parameters

Raises

CookieError – if unable to read or parse the cookie file

static check_for_invalid_types(messages_types_to_add, allowed_message_types)

Used to check for invalid message types

Parameters
  • messages_types_to_add (list) – List of message types to add

  • allowed_message_types (list) – List of allowed message type

Raises

InvalidParameter – if invalid types are specified

clear_cookies()

Clear the session’s cookies.

close()

Close the session. Once this has been called, no more requests can be made.

generate_urls(**kwargs)

This method should be implemented in a subclass and should return a generator which yields URLs for testing.

Raises

NotImplementedError – if not implemented and called from a subclass

Return the value for key if key is in the cookie dictionary, else default.

Parameters
  • name (str) – The key of the cookie

  • default (object, optional) – Return this value if the specified cookie cannot be found, defaults to None

Returns

The cookie value, or default

Return type

Union[str, object, None]

static get_mapped_keys(remapping)
get_session_headers(key)
get_site_value(value)

Get the site’s default value for a certain parameter

Parameters

value (Union[SiteDefault, object]) – The value

Returns

The site’s default value

Return type

object

classmethod matches(url)

Used to check if a url matches any of the regular expressions specified in the classes _VALID_URLS dictionary.

Returns

If a match is found, the function name and match object is returned, otherwise None.

Return type

(str, re.Match)

static retry(attempt_number, max_attempts=1, error=None, retry_timeout=None, text=None, interruptible_retry=True, **kwargs)

Retry to occur after an error occurs

Parameters
  • attempt_number (int) – The current attempt number

  • max_attempts (int, optional) – The maximum number of attempts allowed

  • error (Exception, optional) – The error which was raised, defaults to None

  • retry_timeout (float, optional) – The number of seconds to sleep after failing, defaults to None (i.e. use exponential back-off)

  • text (object, optional) – Items to display on retry, defaults to None

Raises

RetriesExceeded – if the maximum number of retries has been exceeded

update_session_headers(new_headers)