o {#iE@s@dZddlZddlZddlZddlmZddlmZddlm Z m Z m Z m Z m Z ddlmZmZmZmZddlmZddlmZmZmZdd lmZdd lmZdd lmZmZee Z!d e"fd dZ#ddZ$ddZ%d e"de&fddZ'd e"de"e"ffddZ(de"de"fddZ)de"de"fddZ*de"de"fddZ+de"de"de"fd d!Z,de"d"e"de"fd#d$Z-d%ede"fd&d'Z.d(e"de"fd)d*Z/d+e"de e"e e"ffd,d+Z0d(e"de fd-d.Z1d(e"de fd/d0Z2d(e"de"fd1d2Z3d3e d4e d5e"ddfd6d7Z4d8e ddfd9d:Z5d e"de"fd;d<Z6d e"de"fd=d>Z7de efd?d@Z8dS)AzBThis module contains all non-cipher related data extraction logic.N) OrderedDict)datetime)AnyDictListOptionalTuple)parse_qsquote urlencodeurlparse)Cipher)HTMLParseErrorLiveStreamErrorRegexMatchError regex_search)YouTubeMetadata)parse_for_objectparse_for_all_objects watch_htmlcCs4z td|dd}Wn tyYdSwt|dS)zExtract publish date :param str watch_html: The html contents of the watch page. :rtype: str :returns: Publish date of the video. z;(?<=itemprop=\"datePublished\" content=\")\d{4}-\d{2}-\d{2}rgroupNz%Y-%m-%d)rrrstrptime)rresultrU/var/www/GraceOrthoHospital/venv_grace/lib/python3.10/site-packages/pytube/extract.py publish_dates  rcCs"dg}|D] }||vrdSqdS)zCheck if live stream recording is available. :param str watch_html: The html contents of the watch page. :rtype: bool :returns: Whether or not the content is private. z,This live stream recording is not available.FTr)runavailable_stringsstringrrrrecording_available&s r cCs$gd}|D] }||vrdSqdS)zCheck if content is private. :param str watch_html: The html contents of the watch page. :rtype: bool :returns: Whether or not the content is private. )zFThis is a private video. Please sign in to verify that you may see it.z"simpleText":"Private video"zThis video is private.TFr)rprivate_stringsrrrr is_private8s  r"returncCs*z td|ddWdStyYdSw)zCheck if content is age restricted. :param str watch_html: The html contents of the watch page. :rtype: bool :returns: Whether or not the content is age restricted. zog:restrictions:agerrFT)rr)rrrris_age_restrictedLs   r$cCsdt|}|di}d|vrdSd|vr-d|vr!|d|dgfSd|vr-|d|dfSddgfS)aReturn the playability status and status explanation of a video. For example, a video may have a status of LOGIN_REQUIRED, and an explanation of "This is a private video. Please sign in to verify that you may see it." This explanation is what gets incorporated into the media player overlay. :param str watch_html: The html contents of the watch page. :rtype: bool :returns: Playability status and reason of the video. playabilityStatusliveStreamability) LIVE_STREAMzVideo is a live stream.statusreasonmessagesN)initial_player_responseget)rplayer_response status_dictrrrplayability_status\s  r/urlcCstd|ddS)arExtract the ``video_id`` from a YouTube url. This function supports the following patterns: - :samp:`https://youtube.com/watch?v={video_id}` - :samp:`https://youtube.com/embed/{video_id}` - :samp:`https://youtu.be/{video_id}` :param str url: A YouTube url containing a video id. :rtype: str :returns: YouTube video id. z(?:v=|\/)([0-9A-Za-z_-]{11}).*rr)r0rrrvideo_idvsr2cCstj|}t|jddS)aoExtract the ``playlist_id`` from a YouTube url. This function supports the following patterns: - :samp:`https://youtube.com/playlist?list={playlist_id}` - :samp:`https://youtube.com/watch?v={video_id}&list={playlist_id}` :param str url: A YouTube url containing a playlist id. :rtype: str :returns: YouTube playlist id. listr)urllibparser r query)r0parsedrrr playlist_ids r8cCsjgd}|D](}t|}||}|r.td||d}|d}d|d|Sqtddd) aExtract the ``channel_name`` or ``channel_id`` from a YouTube url. This function supports the following patterns: - :samp:`https://youtube.com/c/{channel_name}/*` - :samp:`https://youtube.com/channel/{channel_id}/* - :samp:`https://youtube.com/u/{channel_name}/*` - :samp:`https://youtube.com/user/{channel_id}/* :param str url: A YouTube url containing a channel name. :rtype: str :returns: YouTube channel name. )z(?:\/(c)\/([%\d\w_\-]+)(\/.*)?)z%(?:\/(channel)\/([%\w\d_\-]+)(\/.*)?)z(?:\/(u)\/([%\d\w_\-]+)(\/.*)?)z"(?:\/(user)\/([%\w\d_\-]+)(\/.*)?)"finished regex search, matched: %sr1/ channel_namepatternscallerpatternrecompilesearchloggerdebugrr)r0r=r@regexfunction_match uri_styleuri_identifierrrrr<s     r< watch_urlcCs*td|fddt|fddddg}t|S)aConstruct the video_info url. :param str video_id: A YouTube video identifier. :param str watch_url: A YouTube watch url. :rtype: str :returns: :samp:`https://youtube.com/get_video_info` with necessary GET parameters. r2)psdefaulteurl)hlen_UShtml51cTVHTML5cverz 7.20201028)rr _video_info_url)r2rKparamsrrrvideo_info_urls   r[ embed_htmlcCs\z td|dd}Wn tyd}Ynwd|}td|fd|fd|fd d d g}t|S) a<Construct the video_info url. :param str video_id: A YouTube video identifier. :param str embed_html: The html contents of the embed page (for age restricted videos). :rtype: str :returns: :samp:`https://youtube.com/get_video_info` with necessary GET parameters. z"sts"\s*:\s*(\d+)r1rz!https://youtube.googleapis.com/v/r2rNstsrQrTrW)rrrrY)r2r\r^rNrZrrrvideo_info_url_age_restricteds     r_rZcCs dt|S)Nz'https://www.youtube.com/get_video_info?)r )rZrrrrYs rYhtmlc Cs@z t|dd}Wd|Sttfyt|}Yd|Sw)zGet the base JavaScript url. Construct the base JavaScript url, which contains the decipher "transforms". :param str html: The html contents of the watch page. assetsjszhttps://youtube.com)get_ytplayer_configKeyErrorrget_ytplayer_js)r`base_jsrrrjs_urls  rgmime_type_codeccCsLd}t|}||}|std|d|\}}|dd|dDfS)aParse the type data. Breaks up the data in the ``type`` key of the manifest, which contains the mime type and codecs serialized together, and splits them into separate elements. **Example**: mime_type_codec('audio/webm; codecs="opus"') -> ('audio/webm', ['opus']) :param str mime_type_codec: String containing mime type and codecs. :rtype: tuple :returns: The mime type and a list of codecs. z,(\w+\/\w+)\;\scodecs=\"([a-zA-Z-0-9.,\s]*)\"rhr>cSsg|]}|qSr)strip).0rUrrr $z#mime_type_codec..,)rBrCrDrgroupssplit)rhr@rGresults mime_typecodecsrrrrh s    cCsRdg}|D]}t|}||}|r"td||d}|Sqtddd)zGet the YouTube player base JavaScript path. :param str html The html contents of the watch page. :rtype: str :returns: Path to YouTube's base.js file. z'(/s/player/[\w\d]+/[\w\d_/.]+/base\.js)r9r1rejs_url_patternsr>rA)r`rsr@rGrH yt_player_jsrrrre's     rec Cstdddg}|D])}zt||WSty4}ztd|t|WYd}~q d}~wwdg}|D]}zt||WStyMYq:wtddd ) aGet the YouTube player configuration data from the watch html. Extract the ``ytplayer_config``, which is json data embedded within the watch html and serves as the primary source of obtaining the stream manifest data. :param str html: The html contents of the watch page. :rtype: str :returns: Substring of the html containing the encoded manifest data. zfinding initial function namezytplayer\.config\s*=\s*ytInitialPlayerResponse\s*=\s*zPattern failed: Nz,yt\.setConfig\(.*['\"]PLAYER_CONFIG['\"]:\s*rcz#config_patterns, setconfig_patternsr>)rErFrrr)r`config_patternsr@esetconfig_patternsrrrrc@s0     rcc Csfi}ddg}|D]}zt||}|D]}||qWqty$Yqwt|dkr-|Stddd)a;Get the entirety of the ytcfg object. This is built over multiple pieces, so we have to find all matches and combine the dicts together. :param str html: The html contents of the watch page. :rtype: str :returns: Substring of the html containing the encoded manifest data. z ytcfg\s=\sz ytcfg\.set\(r get_ytcfgytcfg_pattenrsr>)rupdaterlenr)r`ytcfgytcfg_patternsr@ found_objectsobjrrrryns$     rystream_manifestvid_inforbc Cs2t|d}t|D]\}}z|d}Wnty+|did}|r)tdYnwd|vssz#apply_signature..sig ratebypassnz://?N)r enumeraterdr,rrErF get_signaturer r r6itemskeysr3 calculate_nschemenetlocpathr ) rrrbcipheristreamr0 live_streamr parsed_url query_params initial_nnew_nrrrapply_signaturesB        "r stream_datacCsd|vrdSg}d|vr||dd|vr"||d|D])}d|vrDd|vrDt|d}|dd|d<|dd|d<|dd k|d <q$td |S) a-Apply various in-place transforms to YouTube's media stream data. Creates a ``list`` of dictionaries by string splitting on commas, then taking each list item, parsing it as a query string, converting it to a ``dict`` and unquoting the value. :param dict stream_data: Dictionary containing query string encoded values. **Example**: >>> d = {'foo': 'bar=1&var=test,em=5&t=url%20encoded'} >>> apply_descrambler(d, 'foo') >>> print(d) {'foo': [{'bar': '1', 'var': 'test'}, {'em': '5', 't': 'url encoded'}]} r0NformatsadaptiveFormatssignatureCipherrrtypeFORMAT_STREAM_TYPE_OTFis_otfzapplying descrambler)rextendr r,rErF)rrdata cipher_urlrrrapply_descramblers     rc C@ddg}|D]}zt||WStyYqwtddd)zExtract the ytInitialData json from the watch_html page. This mostly contains metadata necessary for rendering the page on-load, such as video information, copyright notices, etc. @param watch_html: Html of the watch page @return: z'window\[['\"]ytInitialData['\"]]\s*=\s*zytInitialData\s*=\s* initial_datainitial_data_patternr>rrrrr=r@rrrrs   rc Cr)aExtract the ytInitialPlayerResponse json from the watch_html page. This mostly contains metadata necessary for rendering the page on-load, such as video information, copyright notices, etc. @param watch_html: Html of the watch page @return: z1window\[['\"]ytInitialPlayerResponse['\"]]\s*=\s*rur+initial_player_response_patternr>rrrrrr+s  r+c Cstz|dddddddddd}Wnttfy'tgYSwtd d |}d d |D}t|S) u<Get the informational metadata for the video. e.g.: [ { 'Song': '강남스타일(Gangnam Style)', 'Artist': 'PSY', 'Album': 'PSY SIX RULES Pt.1', 'Licensed to YouTube by': 'YG Entertainment Inc. [...]' } ] :rtype: YouTubeMetadata contentstwoColumnWatchNextResultsrpr1videoSecondaryInfoRenderermetadataRowContainermetadataRowContainerRendererrowscSs d|vS)NmetadataRowRenderer)r)xrrr;s zmetadata..cSsg|]}|dqS)rr)rjrrrrrkArlzmetadata..)rd IndexErrorrfilter)r metadata_rowsrrrmetadata!s6  r)9__doc__logging urllib.parser4rB collectionsrrtypingrrrrrr r r r pytube.cipherr pytube.exceptionsrrrpytube.helpersrpytube.metadatar pytube.parserrr getLogger__name__rEstrrr r"boolr$r/r2r8r<r[r_rYrgrhrercryrrrr+rrrrrsF      $ ."9)